How to set Key frame correctly when saving AVI with AVStream

NTDEV
1

Hello,
I am developing a PCI capture driver, and I use “avshws” as sample to develp the driver.
I can capture AVI with H.264 codec by AMCAP, and the avi file can play well.
My problem is as following:

In my video packets, there is only one key frame in every 90 frames. However, the frames in the captured AVI file are all labeled as key frame.

I have set following frame information before the data is sent by capture pin:

PKSSTREAM_HEADER pHeader = pStreamPointer->StreamHeader;
…
llClockTime = m_Clock->GetCorrelatedTime(&llClockTime);
pHeader->PresentationTime.Time = llClockTime;
pHeader->Duration = -liTimeout.QuadPart;
pHeader->PresentationTime.Numerator = pHeader->PresentationTime.Denominator = 1;
pHeader->OptionsFlags = KSSTREAM_HEADER_OPTIONSF_TIMEVALID |
KSSTREAM_HEADER_OPTIONSF_DURATIONVALID;
if(pHeader->Size >= sizeof(KSSTREAM_HEADER) + sizeof(KS_FRAME_INFO)){
PKS_FRAME_INFO FrameInfo = reinterpret_cast <pks_frame_info>(pHeader + 1);
FrameInfo->ExtendedHeaderSize = sizeof (KS_FRAME_INFO);
FrameInfo->PictureNumber = m_nFrameNum;
FrameInfo->DropCount = 0;
switch(nFrameType)
{
case 0:
FrameInfo->dwFrameFlags = KS_VIDEO_FLAG_I_FRAME;
break;
case 3:
FrameInfo->dwFrameFlags = KS_VIDEO_FLAG_P_FRAME;
}
}

But it seems no effect for the AVI file. All the frames in AVI file are still labeled as key frame.

The capture pin video data range is as below:

KS_DATARANGE_VIDEO
FormatH264_Capture = {
//
// KSDATARANGE
//
{
sizeof (KS_DATARANGE_VIDEO), // FormatSize
0, // Flags
DMAX_X * DMAX_Y * 2, // SampleSize
0, // Reserved
STATICGUIDOF (KSDATAFORMAT_TYPE_VIDEO), // aka. MEDIATYPE_Video

0x34363248, 0x0000, 0x0010, 0x80, 0x00,
0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71, //aka. MEDIASUBTYPE_YUY2,
STATICGUIDOF (KSDATAFORMAT_SPECIFIER_VIDEOINFO) // FORMAT_VideoInfo
},
FALSE, // BOOL, bFixedSizeSamples (all samples same size?)
FALSE, // BOOL, bTemporalCompression (all I frames?)
0, // Reserved (was StreamDescriptionFlags)
0, // Reserved (was MemoryAllocationFlags
// (KS_VIDEO_ALLOC_*))

//
// _KS_VIDEO_STREAM_CONFIG_CAPS
//
{
STATICGUIDOF( KSDATAFORMAT_SPECIFIER_VIDEOINFO ), // GUID
KS_AnalogVideo_None, // AnalogVideoStandard
DMAX_X, DMAX_Y, // InputSize, (the inherent size of the incoming signal
D_X,D_Y, // MinCroppingSize, smallest rcSrc cropping rect allowed
DMAX_X, DMAX_Y, // MaxCroppingSize, largest rcSrc cropping rect allowed
8, // CropGranularityX, granularity of cropping size
1, // CropGranularityY
8, // CropAlignX, alignment of cropping rect
1, // CropAlignY;
D_X, D_Y, // MinOutputSize, smallest bitmap stream can produce
DMAX_X, DMAX_Y, // MaxOutputSize, largest bitmap stream can produce
8, // OutputGranularityX, granularity of output bitmap size
1, // OutputGranularityY;
0, // StretchTapsX (0 no stretch, 1 pix dup, 2 interp…)
0, // StretchTapsY
0, // ShrinkTapsX
0, // ShrinkTapsY
1428571, // MinFrameInterval, 100 nS units >>7 frames per second
640000000, // MaxFrameInterval, 100 nS units
8 * 2 * 30 * D_X * D_Y, // MinBitsPerSecond;
8 * 2 * 30 * DMAX_X * DMAX_Y, // MaxBitsPerSecond;
},

//
// KS_VIDEOINFOHEADER (default format)
//
{
0, 0, D_X, D_Y, // RECT rcSource;
0, 0, 0, 0, // RECT rcTarget;
DMAX_X * DMAX_Y * 2 * 30, // DWORD dwBitRate;
0L, // DWORD dwBitErrorRate;
1428571, // REFERENCE_TIME AvgTimePerFrame;
sizeof (KS_BITMAPINFOHEADER), // DWORD biSize;
DMAX_X, // LONG biWidth;
DMAX_Y, // LONG biHeight;
1, // WORD biPlanes;
16, // WORD biBitCount;
FOURCC_H264, // DWORD biCompression;
DMAX_X * DMAX_Y * 2, // DWORD biSizeImage;
0, // LONG biXPelsPerMeter;
0, // LONG biYPelsPerMeter;
0, // DWORD biClrUsed;
0 // DWORD biClrImportant;
}

Can anybody tell me hwo to set the correct key frame information to AVI file?
Thank you!
Best Regards,
Gordon</pks_frame_info>

2

xxxxx@sunplusct.com wrote:

Hello,
I am developing a PCI capture driver, and I use “avshws” as sample to develp the driver.
I can capture AVI with H.264 codec by AMCAP, and the avi file can play well.
My problem is as following:

In my video packets, there is only one key frame in every 90 frames. However, the frames in the captured AVI file are all labeled as key frame.

I have set following frame information before the data is sent by capture pin:

PKSSTREAM_HEADER pHeader = pStreamPointer->StreamHeader;
…
llClockTime = m_Clock->GetCorrelatedTime(&llClockTime);
pHeader->PresentationTime.Time = llClockTime;
pHeader->Duration = -liTimeout.QuadPart;
pHeader->PresentationTime.Numerator = pHeader->PresentationTime.Denominator = 1;
pHeader->OptionsFlags = KSSTREAM_HEADER_OPTIONSF_TIMEVALID |
KSSTREAM_HEADER_OPTIONSF_DURATIONVALID;

You aren’t setting ANY key frames here. I would add:
if( nFrameType == 0 )
pHeader->OptionsFlags |= KSSTREAM_HEADER_OPTIONSF_SPLICEPOINT;

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

3

Thanks for your response.

I add following code:
if( nFrameType == 0 )
pHeader->OptionsFlags |= KSSTREAM_HEADER_OPTIONSF_SPLICEPOINT;
But it seems no use for my AVI file. All frames in AVI are still key frames.
What do you suggest for the problem?
By the way, I have other questions:

  1. To use H264 compressed codec, should I set “bTemporalCompression”( in KS_DATARANGE_VIDEO ) to be TRUE or FALSE?
  2. If I set FrameInfo->dwFrameFlags = KS_VIDEO_FLAG_I_FRAME or KS_VIDEO_FLAG_P_FRAME, how can I use the information in my application? Or what would be changed if I set FrameInfo->dwFrameFlags?

Thanks for your help!
Gordon

4

xxxxx@sunplusct.com wrote:

I add following code:
if( nFrameType == 0 )
pHeader->OptionsFlags |= KSSTREAM_HEADER_OPTIONSF_SPLICEPOINT;
But it seems no use for my AVI file. All frames in AVI are still key frames.
What do you suggest for the problem?
By the way, I have other questions:

  1. To use H264 compressed codec, should I set “bTemporalCompression”( in KS_DATARANGE_VIDEO ) to be TRUE or FALSE?

The documentation on this is pretty clear: TRUE. If you set it to
FALSE, that means all frames are key frames.

  1. If I set FrameInfo->dwFrameFlags = KS_VIDEO_FLAG_I_FRAME or KS_VIDEO_FLAG_P_FRAME, how can I use the information in my application? Or what would be changed if I set FrameInfo->dwFrameFlags?

I don’t know how or if this is exposed to user-mode. Typically, your
frames are going to be fed into a decoder, and the decoder will
determine from the bitstream which fields are I and which are P.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

5

I have changed “bTemporalCompression” to be TRUE, and I connect capture pin to “dump” filter.
I traced in avshws driver, it really shows “bTemporalCompression” is true. But when I trace into dump filter, in “CheckMediaType” function, CMediaType * pMT->bTemporalCompression is still 0. I think it’s the reason that all frames in AVI are all marked as key frame.
Do you know how to modify avshws driver and let “bTemporalCompression” to be TRUE in transform filter?

By the way, I trace a ASF Mux filter, in “ReceiveVideo” funtion, there are following codes:
if (S_OK == pSample->IsSyncPoint())
flags |= WM_SF_CLEANPOINT;
mIWriterAdvanced->WriteStreamSample(cVideoStreamNum, rtStart,
DWORD(rtStart / 10000), duration, flags, pWmSample);
And MSDN says:
WM_SF_CLEANPOINT Indicates the sample is a key frame. Set this flag if and only if the compressed input sample is a key frame.

So, if I can set sync point, the current frame will be marked as a key frame.
Do you know how to set sync point in avshws driver?

My data range defines as following:
KS_DATARANGE_VIDEO
FormatH264_Capture = {
//
// KSDATARANGE
//
{
sizeof (KS_DATARANGE_VIDEO), // FormatSize
0, // Flags
DMAX_X * DMAX_Y * 2, // SampleSize
0, // Reserved
STATICGUIDOF (KSDATAFORMAT_TYPE_VIDEO), // aka. MEDIATYPE_Video

0x34363248, 0x0000, 0x0010, 0x80, 0x00,
0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71, //aka. MEDIASUBTYPE_YUY2,
STATICGUIDOF (KSDATAFORMAT_SPECIFIER_VIDEOINFO) // FORMAT_VideoInfo
},
FALSE, // BOOL, bFixedSizeSamples (all samples same size?)
TRUE, // BOOL, bTemporalCompression (all I frames?)
0, // Reserved (was StreamDescriptionFlags)
0, // Reserved (was MemoryAllocationFlags
// (KS_VIDEO_ALLOC_*))

//
// _KS_VIDEO_STREAM_CONFIG_CAPS
//
{
STATICGUIDOF( KSDATAFORMAT_SPECIFIER_VIDEOINFO ), // GUID
KS_AnalogVideo_None, // AnalogVideoStandard
DMAX_X, DMAX_Y, // InputSize, (the inherent size of the incoming signal
D_X,D_Y, // MinCroppingSize, smallest rcSrc cropping rect
allowed
DMAX_X, DMAX_Y, // MaxCroppingSize, largest rcSrc cropping rect allowed
8, // CropGranularityX, granularity of cropping size
1, // CropGranularityY
8, // CropAlignX, alignment of cropping rect
1, // CropAlignY;
D_X, D_Y, // MinOutputSize, smallest bitmap stream can produce
DMAX_X, DMAX_Y, // MaxOutputSize, largest bitmap stream can produce
8, // OutputGranularityX, granularity of output bitmap size
1, // OutputGranularityY;
0, // StretchTapsX (0 no stretch, 1 pix dup, 2 interp…)
0, // StretchTapsY
0, // ShrinkTapsX
0, // ShrinkTapsY
1428571, // MinFrameInterval, 100 nS units >>7 frames per second
640000000, // MaxFrameInterval, 100 nS units
8 * 2 * 30 * D_X * D_Y, // MinBitsPerSecond;
8 * 2 * 30 * DMAX_X * DMAX_Y, // MaxBitsPerSecond;
},

//
// KS_VIDEOINFOHEADER (default format)
//
{
0, 0, D_X, D_Y, // RECT rcSource;
0, 0, 0, 0, // RECT rcTarget;
DMAX_X * DMAX_Y * 2 * 30, // DWORD dwBitRate;
0L, // DWORD dwBitErrorRate;
1428571, // REFERENCE_TIME AvgTimePerFrame;

sizeof (KS_BITMAPINFOHEADER), // DWORD biSize;
DMAX_X, // LONG biWidth;
DMAX_Y, // LONG biHeight;
1, // WORD biPlanes;
16, // WORD biBitCount;
FOURCC_H264, // DWORD biCompression;
DMAX_X * DMAX_Y * 2, // DWORD biSizeImage;
0, // LONG biXPelsPerMeter;
0, // LONG biYPelsPerMeter;
0, // DWORD biClrUsed;
0 // DWORD biClrImportant;
}

Thanks!
Gordon

6

xxxxx@sunplusct.com wrote:

I have changed “bTemporalCompression” to be TRUE, and I connect capture pin to “dump” filter.
I traced in avshws driver, it really shows “bTemporalCompression” is true. But when I trace into dump filter, in “CheckMediaType” function, CMediaType * pMT->bTemporalCompression is still 0. I think it’s the reason that all frames in AVI are all marked as key frame.

CheckMediaType is called many, many times as a connection is
negotiated. Are you sure you traced through the final one?

Do you know how to modify avshws driver and let “bTemporalCompression” to be TRUE in transform filter?

bTemporalCompression in KS_DATARANGE_VIDEO is the way.

By the way, I trace a ASF Mux filter, in “ReceiveVideo” funtion, there are following codes:
if (S_OK == pSample->IsSyncPoint())
flags |= WM_SF_CLEANPOINT;
mIWriterAdvanced->WriteStreamSample(cVideoStreamNum, rtStart,
DWORD(rtStart / 10000), duration, flags, pWmSample);
And MSDN says:
WM_SF_CLEANPOINT Indicates the sample is a key frame. Set this flag if and only if the compressed input sample is a key frame.

So, if I can set sync point, the current frame will be marked as a key frame.
Do you know how to set sync point in avshws driver?

I already told you that, in my very first reply in this thread.

My data range defines as following:
KS_DATARANGE_VIDEO
FormatH264_Capture = {
//
// KSDATARANGE
//
{
sizeof (KS_DATARANGE_VIDEO), // FormatSize
0, // Flags
DMAX_X * DMAX_Y * 2, // SampleSize
0, // Reserved
STATICGUIDOF (KSDATAFORMAT_TYPE_VIDEO), // aka. MEDIATYPE_Video

0x34363248, 0x0000, 0x0010, 0x80, 0x00,
0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71, //aka. MEDIASUBTYPE_YUY2,
STATICGUIDOF (KSDATAFORMAT_SPECIFIER_VIDEOINFO) // FORMAT_VideoInfo
},

Are you actually producing individual H.264 video frames, or are you
producing a stream that has to be demuxed?

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

7

>>CheckMediaType is called many, many times as a connection is

>negotiated. Are you sure you traced through the final one?
Yes, I am sure I traced through the final one.

>Are you actually producing individual H.264 video frames, or are you
>producing a stream that has to be demuxed?
Yes, I actually producing individual H.264 video frame.
Do you think what’s wrong in my KS_DATARANGE_VIDEO?

By the way, I found “bFixedSizeSamples” is also unable to update to the transform filter. That is, even I modify bFixedSizeSamples to be FALSE in avshws, the value will be TRUE in transform filter.
Should I do something in “CCapturePin::IntersectHandlerIntersectHandler” or anywhere to make “bTemporalCompression” updated in user mode filter?

8

xxxxx@sunplusct.com wrote:

>> Are you actually producing individual H.264 video frames, or are you
>> producing a stream that has to be demuxed?
>>
Yes, I actually producing individual H.264 video frame.
Do you think what’s wrong in my KS_DATARANGE_VIDEO?

No. If you were producing a transport stream, for example, transport
streams don’t have the concept of frames, and would be
KSDATAFORMAT_TYPE_STREAM instead of KSDATAFORMAT_TYPE_VIDEO, but that
doesn’t seem to be your issue. You really have individual, compressed
video frames.

By the way, I found “bFixedSizeSamples” is also unable to update to the transform filter. That is, even I modify bFixedSizeSamples to be FALSE in avshws, the value will be TRUE in transform filter.
Should I do something in “CCapturePin::IntersectHandlerIntersectHandler” or anywhere to make “bTemporalCompression” updated in user mode filter?

As long as you are roughly what the sample does (combining
CallerDataRange and DescriptorDataRange and copying the merge result to
Data), that should be all it takes. You might double check that the
flags you are returning in IntersectHandler still has those bits set.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

9

>>As long as you are roughly what the sample does (combining

>CallerDataRange and DescriptorDataRange and copying the merge result to
>Data), that should be all it takes.
In “IntersectHandlerIntersectHandler” of original avshws, I found it doesn’t copy bTemporalCompression and bFixedSizeSamples into “Data”, should I make extra copy to put the tow flags into “Data”?

In addition, I have tried avshws, avssamp and testcap samples and connect their output pin to “Dump” filter. The value of bTemporalCompression in three samples are all TRUE, but when I trace into dump filter, bTemporalCompression are all FALSE. Do you think is it reasonable?

I have another question, the following is the KS_DATARANGE_VIDEO in testcap:
static KS_DATARANGE_VIDEO StreamFormatRGB24Bpp_Capture =
{
// KSDATARANGE
{
sizeof (KS_DATARANGE_VIDEO), // FormatSize
0, // Flags
D_X * D_Y * 3, // SampleSize
0, // Reserved

STATIC_KSDATAFORMAT_TYPE_VIDEO, // aka. MEDIATYPE_Video
0xe436eb7d, 0x524f, 0x11ce, 0x9f, 0x53, 0x00, 0x20, 0xaf, 0x0b,
0xa7, 0x70, //MEDIASUBTYPE_RGB24,
STATIC_KSDATAFORMAT_SPECIFIER_VIDEOINFO // aka. FORMAT_VideoInfo
},

TRUE, // BOOL, bFixedSizeSamples (all samples same size?)
TRUE, // BOOL, bTemporalCompression (all I frames?)
0, // Reserved (was StreamDescriptionFlags)
0, // Reserved (was MemoryAllocationFlags
(KS_VIDEO_ALLOC_*))

It’s subtype is RGB24, and is non compressed format. Do you know why its bTemporalCompression are set to be true? (avshws has the same setting.)

Thank you!
Gordon