AVStream Video Capture Filter 1080i 10 bit samples

I am working on a project to develope a 1080i video capture card supporting
8 bit and 10 bit luma / chroma samples. This driver is for hardware
demostration / evaluation
purposes and not a production driver.

I have modified the AVSHWS sample driver to support Interlaced video -
basically convert the IntersectHandler, CaptureVideoInfoHeader, and
DispatchSetFormat routines
to use KS_DATARANGE_VIDEO2 and KS_DATAFORMAT_VIDEOINFOHEADER2.
With 8 bit samples I was successful in displaying the color bar pattern, in
1080 interlaced format.

I modified the data range for the FourCC code of v210 which supports 10 bit
samples. I have installed
the BlackMagic codec for 10 bit 4:2:2 which is the filter I am trying to
connect to.
The KS_DATARANGE_VIDEO2 parameters are provide below should there still be
problems with it.

The FourCC description for v210 can be found at:
http://developer.apple.com/quicktime/icefloe/dispatch019.html#v210

My problem is that after eleven calls to my IntersectHandler that return
STATUS_SUCCESS
there is never a call to my DispatchSetFormat routine. (Each was preceeded
by the STATUS_BUFFER_OVERFLOW
return status) Graphedit then displays the following message after the
Render Pin command.
No combination of intermediate filters could be found to make the
connection. Return Code: 0x80040217

I have used KsStudio to build a graph, after two attemps the capture output
pin connects to the Video Capture Sink filter.
When I check the Istanciated Objects window the capture filter data range
seems to be incorrect parameter for biCompression
parameter which is now 808530550 (0x0p) (yes there is an extra nibble) was
0x30, 0x31, 0x32, 0x76 (012v) when returned in
the IntersectHandler. The 8 bit sample version of the driver displayed
FOURCC_YVU2 mmioFourCC(“Y”,“U”,“Y”,“2”)

Also the KsStudio Check topology option displays the following message:
Pin has 0 internal connections. This is an orphaned pin and will be ignored
by most graph builders.
This same message was displayed for the 8 bit sample size, so it may not be
significant.

My next step is to compile the DirectX SDK baseclasses and see if I can
trace into the callers of my IntersectHandler, as suggested at:
http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2276461&SiteID=1

Any suggestions to a better approach would be appriecated.

Below is the KS_DATARANGE_VIDEO2 structure that is used by the
IntersectHandler, I have
added the 8 bit sample values as comments that were used to display the
1080i color bars.
I only compile single data range at any time.

#define DMAX_X 1920
#define DMAX_Y 540
#define D_X 48
#define D_Y 16

const
KS_DATARANGE_VIDEO2
Formatv210_Capture = {

//
// KSDATARANGE_VIDEO2
//
{
sizeof (KS_DATARANGE_VIDEO2), // FormatSize
0, // Flags

/* 8 bit DMAX_X * DMAX_Y * 2, // SampleSize end
8 bit */
(DMAX_X / 6) * 16 * DMAX_Y, // SampleSize 6 luma/chroma
pairs packed in 16 bytes

0, // Reserved
STATICGUIDOF (KSDATAFORMAT_TYPE_VIDEO), // aka. MEDIATYPE_Video

/* 8 bit FOURCC_YUY2, 0x0000, 0x0010, 0x80, 0x00,
0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71, //aka. MEDIASUBTYPE_YUY2,
end 8 bit */
FOURCC_v210, 0x0000, 0x0010, 0x80, 0x00,
0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71, //aka. MEDIASUBTYPE_v210,

STATICGUIDOF (KSDATAFORMAT_SPECIFIER_VIDEOINFO2) // aka.
FORMAT_VideoInfo
},

TRUE, // BOOL, bFixedSizeSamples (all samples same size?)
TRUE, // BOOL, bTemporalCompression (all I frames?)
0, // Reserved (was StreamDescriptionFlags)
0, // Reserved (was MemoryAllocationFlags
// (KS_VIDEO_ALLOC_*))

//
// _KS_VIDEO_STREAM_CONFIG_CAPS
//
{
STATICGUIDOF( KSDATAFORMAT_SPECIFIER_VIDEOINFO2 ), // GUID
KS_AnalogVideo_NTSC_M_J | KS_AnalogVideo_NTSC_M, //
AnalogVideoStandard AnalogVideoStandard
DMAX_X, DMAX_Y, // InputSize, (the inherent size of the incoming
signal
// with every digitized pixel
unique)
D_X,D_Y, // MinCroppingSize, smallest rcSrc
cropping rect allowed
DMAX_X, DMAX_Y, // MaxCroppingSize, largest rcSrc cropping rect
allowed
1, // CropGranularityX, granularity of cropping size
1, // CropGranularityY
1, // CropAlignX, alignment of cropping rect
1, // CropAlignY;
D_X, D_Y, // MinOutputSize, smallest bitmap stream
can produce
DMAX_X, DMAX_Y, // MaxOutputSize, largest bitmap stream can produce
2, // OutputGranularityX, granularity of output bitmap
size
1, // OutputGranularityY;
0, // StretchTapsX (0 no stretch, 1 pix dup, 2
interp…)
0, // StretchTapsY
2, // ShrinkTapsX
2, // ShrinkTapsY
333667, // MinFrameInterval, 100 nS units 29.97 fps
100000000, // MaxFrameInterval, 100 nS units
10 * 2 * 30 * D_X * D_Y, // MinBitsPerSecond;
10 * 2 * 30 * DMAX_X * DMAX_Y, // MaxBitsPerSecond;
},

//
// KS_VIDEOINFOHEADER2 (default format)
//
{
0, 0, 0, 0, // RECT rcSource;
0, 0, 0, 0, // RECT rcTarget;
DMAX_X * DMAX_Y * 2 * 30 * 10 * 2, // DWORD dwBitRate;
0L, // DWORD dwBitErrorRate;
333667, // REFERENCE_TIME
AvgTimePerFrame; // 29.97 fps
(KS_INTERLACE_IsInterlaced | KS_INTERLACE_1FieldPerSample), //
DWORD dwInterlaceFlags;
0, // DWORD dwCopyProtectionFlags;
16, // DWORD dwPicAspectRatioX;
9, // DWORD dwPicAspectRatioY;
0, // DWORD dwReserved1;
0, // DWORD dwReserved2;
{
sizeof (KS_BITMAPINFOHEADER), // DWORD biSize;
DMAX_X, // LONG biWidth;
DMAX_Y, // LONG biHeight;
1, // WORD biPlanes;

/* 8 bit 16, // WORD biBitCount;
end 8 bit */
20, // WORD biBitCount;

/* 8 bit FOURCC_YUV2, // DWORD biCompression;
end 8 bit */
FOURCC_v210, // DWORD biCompression;

/* 8 bit DMAX_X * DMAX_Y * 2, // SampleSize end
8 bit */
(DMAX_X / 6) * 16 * DMAX_Y, // DWORD biSizeImage;
0, // LONG biXPelsPerMeter;
0, // LONG biYPelsPerMeter;
0, // DWORD biClrUsed;
0 // DWORD biClrImportant;
}
}
};

On Mon, Dec 31, 2007 at 12:26:07PM -0500, Phil Sikora wrote:

I am working on a project to develope a 1080i video capture card supporting
8 bit and 10 bit luma / chroma samples. This driver is for hardware
demostration / evaluation purposes and not a production driver.

I have modified the AVSHWS sample driver to support Interlaced video -
basically convert the IntersectHandler, CaptureVideoInfoHeader, and
DispatchSetFormat routines

Why would you deliver the format as interlaced? In my experience, most
applications don’t really care whether the original source was interlaced
or not. They want frames, not fields.

I modified the data range for the FourCC code of v210 which supports 10 bit
samples. I have installed the BlackMagic codec for 10 bit 4:2:2 which is
the filter I am trying to connect to. The KS_DATARANGE_VIDEO2 parameters
are provide below should there still be problems with it.

My problem is that after eleven calls to my IntersectHandler that return
STATUS_SUCCESS
there is never a call to my DispatchSetFormat routine. (Each was preceeded
by the STATUS_BUFFER_OVERFLOW
return status) Graphedit then displays the following message after the
Render Pin command.
No combination of intermediate filters could be found to make the
connection. Return Code: 0x80040217

How are you testing this? Have you tried adding the BlackMagic codec
to the graph before rendering the stream, to see if it will make the
connection? The graph builder will only try one intermediate codec,
so if you produce v210 and your renderer wants YUY2, as long as the codec
will do that conversion, everything should work. Is the codec doing the
interleaving?

I have used KsStudio to build a graph, after two attemps the capture output
pin connects to the Video Capture Sink filter.
seems to be incorrect parameter for biCompression
parameter which is now 808530550 (0x0p) (yes there is an extra nibble) was
0x30, 0x31, 0x32, 0x76 (012v) when returned in the IntersectHandler.

There isn’t an extra nibble. 808530550 is a decimal number. Convert
it to hex, and you get 30313276, which is the fourcc code “v210”.

Your datarange below is shipping fields (1920x540), but you are declaring
the maximum frame rate as 29.97fps. If you are shipping fields, you should
set the minimum frame time to 166833.

Tim Roberts, xxxxx@probo.com
Providenza & Boeklheide, Inc.

wrote in message news:xxxxx@ntdev…
> On Mon, Dec 31, 2007 at 12:26:07PM -0500, Phil Sikora wrote:
> Why would you deliver the format as interlaced? In my experience, most
> applications don’t really care whether the original source was interlaced
> or not. They want frames, not fields.

The hardware will eventually deliver 1920x540 fields, so I just want to
simulate this format as close as possible. One of my objectives is to
save this un-compressed video stream to a RAID array.

> How are you testing this?
I am using Graphedit and the directshow filters for the most part. I have
to objectives to meet with this simulated video format. To display the
video with a Render and to save the video to a AVI file. I have tried
KsStudio
with limited success, and have tried AMCap but this seems to want a preview
pin.
I have WinDbg installed to check the data structures and debug the
code.

> Have you tried adding the BlackMagic codec
> to the graph before rendering the stream, to see if it will make the
> connection?
Yes, I have the BlackMagic codec in inserted in Graphedit also have tried
to manually connect the AVI Decompressor filter but can’t start with the
codec to connect the pins.

The graph builder will only try one intermediate codec,
> so if you produce v210 and your renderer wants YUY2, as long as the codec
> will do that conversion, everything should work. Is the codec doing the
> interleaving?

I did some testing with not setting the KS_INTERLACE_IsInterlaced |
KS_INTERLACE_1FieldPerSample flags and it seems that the Render is doing
the deinterlacing, just checked this again, the height is less but not
exactly half.
The width was also less, may be I was to quick to accept the interlace
format
as working.

This was with the first instance of the Video Render (quartz.dll) Merit
00800001
I only found the KSSTREAM_HEADER to have parameters that are set per
field (PresentationTime and Duration) didn’t find a flag to mark a even or
odd
field.
I haven’t been able to build the DUMP filter yet to use it for debugging.

> There isn’t an extra nibble. 808530550 is a decimal number. Convert
> it to hex, and you get 30313276, which is the fourcc code “v210”.

Missed that one, thanks!

> Your datarange below is shipping fields (1920x540), but you are declaring
> the maximum frame rate as 29.97fps. If you are shipping fields, you
> should
> set the minimum frame time to 166833.
I have tried this value as well but decided to leave it specifing the frame
rate.
Just tried it again, still won’t connect to the Video Render, however it
connects
to the Null Render.

Phil

> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boeklheide, Inc.
>

Discovered that FILE_LONG_ALIGNMENT parameter in the KSALLOCATOR_FRAMING_EX
structure can also prevent the capture pin from connecting to a filter.

I was working on trying to reduce the number of map registers required when
using the Null filter compared to the video render filter.
When I switched back to 8 bit samples, I couldn’t build a working graph
anymore.

Phil

Just to be clear, I still am un-able to connect the capture pin in 1080i 10
bit mode to the either the video render or 10 bit 4:2:2 codec.

Continue to hunt for other reasons why the IntersectHanlder can be called
with formats that seem to match and yet not connect.

Phil
“Phil Sikora” wrote in message news:xxxxx@ntdev…
> Discovered that FILE_LONG_ALIGNMENT parameter in the
> KSALLOCATOR_FRAMING_EX structure can also prevent the capture pin from
> connecting to a filter.
>
> I was working on trying to reduce the number of map registers required
> when using the Null filter compared to the video render filter.
> When I switched back to 8 bit samples, I couldn’t build a working graph
> anymore.
>
> Phil
>
>

Phil Sikora wrote:

wrote in message news:xxxxx@ntdev…
>
>> On Mon, Dec 31, 2007 at 12:26:07PM -0500, Phil Sikora wrote:
>> Why would you deliver the format as interlaced? In my experience, most
>> applications don’t really care whether the original source was interlaced
>> or not. They want frames, not fields.
>>
>
> The hardware will eventually deliver 1920x540 fields, so I just want to
> simulate this format as close as possible.

That doesn’t mean your driver has to deliver fields. In the last two
interlaced camera drivers I’ve done, I did the interleaving in the
driver and delivered frames.

>> How are you testing this?
>>
> I am using Graphedit and the directshow filters for the most part. I have
> to objectives to meet with this simulated video format. To display the
> video with a Render and to save the video to a AVI file. I have tried
> KsStudio
> with limited success, and have tried AMCap but this seems to want a preview
> pin.
>

AMCap looks for a preview pin first, but if not found it falls back to a
capture pin. Again, however, it’s going to want only one intermediate
filter to convert your capture pin’s format to something the renderer
will accept.

When you connect the decoder to your capture pin, what formats does the
decoder’s output pin advertise?

> I only found the KSSTREAM_HEADER to have parameters that are set per
> field (PresentationTime and Duration) didn’t find a flag to mark a even or
> odd field.
>

Right. That’s why I found it better to interleave in the driver.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

“Tim Roberts” wrote in message news:xxxxx@ntdev…
> Phil Sikora wrote:
>> wrote in message news:xxxxx@ntdev…
>>
>>> On Mon, Dec 31, 2007 at 12:26:07PM -0500, Phil Sikora wrote:
>>> Why would you deliver the format as interlaced? In my experience, most
>>> applications don’t really care whether the original source was
>>> interlaced
>>> or not. They want frames, not fields.
>>>
>>
>> The hardware will eventually deliver 1920x540 fields, so I just want to
>> simulate this format as close as possible.
>
> That doesn’t mean your driver has to deliver fields. In the last two
> interlaced camera drivers I’ve done, I did the interleaving in the
> driver and delivered frames.

I just checked with the video processing group and de-interlacing can be
quite complex if there is motion. I guess in some cases de-interlacer can
drop a field and just line double a field it decides to keep. We would
prefer to keep the interlaced format if at all possible.

>
> When you connect the decoder to your capture pin, what formats does the
> decoder’s output pin advertise?
I get back to you on this one later tonight. Thanks for yor feedback and
pointing out the interleaving option.

> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>

>“Tim Roberts” wrote in message news:xxxxx@ntdev…
>
> When you connect the decoder to your capture pin, what formats does the
> decoder’s output pin advertise?

This is what I get when I try connecting the capture pin to the BlackMagic
10 bit 4:2:2 filter.

Only the following parameters in the KSDATAFOMAT structure are different,
FormatSize, SampleSize, MajorFormat and Alignment.

callerDataRange 0x890bf5f8 struct tagKS_DATARANGE_VIDEO2 * struct
tagKS_DATARANGE_VIDEO2 *
DataRange union KSDATAFORMAT union KSDATAFORMAT
FormatSize 0 unsigned long
Flags 0 unsigned long
SampleSize 0 unsigned long
Reserved 0 unsigned long
MajorFormat struct _GUID {00000000-0000-0000-8000-00aa00389b71} struct
_GUID
SubFormat struct _GUID {30313276-0000-0010-8000-00aa00389b71} struct _GUID
Specifier struct _GUID {f72a76a0-eb0a-11d0-ace4-0000c0cc16ba} struct _GUID
Alignment 0 int64

descriptorDataRange 0xf77d1a50 struct tagKS_DATARANGE_VIDEO2 * struct
tagKS_DATARANGE_VIDEO2 *
DataRange union KSDATAFORMAT union KSDATAFORMAT
FormatSize 0x140 unsigned long
Flags 0 unsigned long
SampleSize 0x2a3000 unsigned long
Reserved 0 unsigned long
MajorFormat struct _GUID {73646976-0000-0010-8000-00aa00389b71} struct
_GUID
SubFormat struct _GUID {30313276-0000-0010-8000-00aa00389b71} struct _GUID
Specifier struct _GUID {f72a76a0-eb0a-11d0-ace4-0000c0cc16ba} struct _GUID
Alignment 320 int64

The callerDataRange also tries these MajorFormats, some several times.

struct _GUID {00000001-0000-0000-8000-00aa00389b71}
struct _GUID {a92e9aa5-0000-0000-8000-00aa00389b71}

It seems that the callerDataRange is a copy of my KS_DATARANGE_VIDEO2
parameters minus the above differences.

The IntersectHandler returns the KS_DATAFORMAT_VIDEOINFOHEADER2 structure.

Also tried setting the rcSource in KS_VIDEOHEADER2 from 0,0,0,0 to
0,0,1920,540 but no connection.

Phil

> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>

Phil Sikora wrote:

This is what I get when I try connecting the capture pin to the BlackMagic
10 bit 4:2:2 filter.

Only the following parameters in the KSDATAFOMAT structure are different,
FormatSize, SampleSize, MajorFormat and Alignment.

descriptorDataRange 0xf77d1a50 struct tagKS_DATARANGE_VIDEO2 * struct
tagKS_DATARANGE_VIDEO2 *
DataRange union KSDATAFORMAT union KSDATAFORMAT
FormatSize 0x140 unsigned long
Flags 0 unsigned long
SampleSize 0x2a3000 unsigned long
Reserved 0 unsigned long
MajorFormat struct _GUID {73646976-0000-0010-8000-00aa00389b71} struct
_GUID
SubFormat struct _GUID {30313276-0000-0010-8000-00aa00389b71} struct _GUID
Specifier struct _GUID {f72a76a0-eb0a-11d0-ace4-0000c0cc16ba} struct _GUID
Alignment 320 int64

Debugging intersection problems is one of the most frustrating parts of
an AVStream driver, because the debug information doesn’t tell you WHY
it is being rejected.

Are you setting an analog standard? I suggest you simply hardcode
KS_AnalogVideo_None. As an experiment, you might try KS_DATARANGE_VIDEO
instead of KS_DATARANGE_VIDEO2, just to see if it makes a difference.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

“Tim Roberts” wrote in message news:xxxxx@ntdev…
> Debugging intersection problems is one of the most frustrating parts of
> an AVStream driver, because the debug information doesn’t tell you WHY
> it is being rejected.

I totally agree, I have been on this for four weeks part time, and the best
I was able to get working is the 8 bit interlaced tables in the orginal
post.
>
> Are you setting an analog standard? I suggest you simply hardcode
> KS_AnalogVideo_None. As an experiment, you might try KS_DATARANGE_VIDEO
> instead of KS_DATARANGE_VIDEO2, just to see if it makes a difference.
>
I started with KS_DATARANGE_VIDEO so that I feel I have covered that option,
just tryed KS_Analog_Video_None still no connection.

I really appreciated the time you spent on this and your experience with
these capture filters, that second set of eyes help decide to leave this
aside for now.

I will use the 1080i 8 bit format for now, the video formatter will just
need to be programmed to dither 10 bit to 8 bit samples for now. I need to
move forward with writing the video stream to disk.

Phil

> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>

>> I only found the KSSTREAM_HEADER to have parameters that are set per

> field (PresentationTime and Duration) didn’t find a flag to mark a even
> or
> odd field.

I was looking at the DirectShow docs for AVI filters and noticed that they
recommend that the dwInterlaceFlags be set as follows:
(KS_INTERLACE_IsInterlaced |
KS_INTERLACE_1FieldPerSample |
KS_INTERLACE_DisplayModeBobOnly), // DWORD dwInterlaceFlags;

This kind of convinced me that there must be a struct to set the even /odd
field flags. Turns out that AVSSAMP sample has a KS_FRAME_INFO struct that
is appended to the KSSTREAM_HEADER struct, which as a KS_VIDEO_FLAG_FIELD1
/2
flags for dwFrameFlags parameter. This will certainly help the downstream
filters to determine which field they are working with.

KSSTREAM_HEADER_OPTIONSF_DATADISCONTINUITY flag and DropCount looks like
another combination of parameters to help signal a status like lost frames
in a capture filter. The PictureNumber parameter is also updated.

However my main problem of connecting filters and determining what failed to
allow the connection persists.
The callerDataRange seems to be just a copy on my filter’s DataRange (minus
the differences mention else where in this post), not very helpful struct to
determine what filter is trying to connect.

While my 1080i 8 bit video can be connect to a Video Render filter, if fails
to connect to the AVI Mux filer which will connect to the FileWriter filter
to save the stream. The other significant difference when using the
FileWriter (non-interlaced video) is the high number of map registers
required.
I tried setting the AllocatorFaming parameter to NULL but still no
connection was allowed. The struggle continues…

Phil

Narrowed the connection problem to changing FOURCC_YUY2 so significant.
Capture pin would connect to Video Render but when Play was pressed “Graph
can’t play” One or more arguments are invalid.

Next I changed the 20 WORD biBitCount; (KS_BITMAPINFOHEADER) to 16 and the
graph was able to play.

Not a solution, but narrows the connection issue, to the v210 GUID.

Phil