AVStream network capture card questions

NTDEV
1

We are developing a PCIe network stream capture card that outputs raw video frames. The number of streams to capture should be configurable from 1 up to 8.

We were thinking of developing an AVStream driver based on the avshws sample: a pin-centric filter with one output capture pin (number of instances of this pin configurable) which should have some custom properties to show up in graphedit:

  • stream settings: multicast IP address + port
  • start/stop capture

Questions:

1/ Is AVStream still the right approach for new developments ? We have taken the course “WDF drivers” and are more or less familiar with this framework. The reason for AVStream would be to avoid having to develop a “custom” IOCTL interface that is not compatible with any user mode frameworks like DirectX… If we can present the capture card as a standard KS filter, this should save us some work in theory… ?

2/ Is there an example of a network based capture pin ?
I am aware of this article about custom pin propteries: http://www.wd-3.com/archive/KsProxyPlugin.htm.
But maybe there is already a standard solution for network based streams ?

Thank you in advance for any information.

  • Bernard Willaert
    Software development engineer
    Barco Healthcare division
    Belgium
2

xxxxx@hotmail.com wrote:

We are developing a PCIe network stream capture card that outputs raw video frames. The number of streams to capture should be configurable from 1 up to 8.

When you say “output”, what do you mean? Do you mean you are sucking
streams from the network and sending frames into memory?

Will 8 uncompressed streams fit in your bandwidth?

We were thinking of developing an AVStream driver based on the avshws sample: a pin-centric filter with one output capture pin (number of instances of this pin configurable) which should have some custom properties to show up in graphedit:

  • stream settings: multicast IP address + port
  • start/stop capture

I’m a bit confused as to your architecture. Does your hardware appear
to the system as a network card? Or does it appear as a completely
custom device? If it’s a custom device, are you writing the whole
network stack yourself? That’s a heck of a job.

Questions:

1/ Is AVStream still the right approach for new developments ? We have taken the course “WDF drivers” and are more or less familiar with this framework. The reason for AVStream would be to avoid having to develop a “custom” IOCTL interface that is not compatible with any user mode frameworks like DirectX… If we can present the capture card as a standard KS filter, this should save us some work in theory… ?

Yes, AVStream is still the right approach for video drivers. You won’t
use KMDF, however. The AVStream framework already handles dispatching
and supplies many of the abstractions that KMDF supplies. (Technically,
you can use KMDF in “miniport” mode, but I’m not sure there’s a good
reason to do so. Doron will disagree with me.)

2/ Is there an example of a network based capture pin ?

It’s kind of an odd question, because you wouldn’t usually build a
network-based video capture card. You’d use a stock network adapter,
with user-mode DirectShow filters to read and decode your frames. What
does the hardware being to the party? Are you doing hardware
decompression? Your “raw video frame” comment makes me thing you don’t.

I am aware of this article about custom pin propteries: http://www.wd-3.com/archive/KsProxyPlugin.htm.
But maybe there is already a standard solution for network based streams ?

Applications like VideoLAN and ffmpeg already have solutions for
handling network-based streams, when those streams are coming in over a
normal network adapter. There are standard protocols for doing that
(like RTP and RTSP).

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

3

Hi Tim,
Thank You for Your quick response. When I searched the forum, it seems that You are The AVStream guru.
First, let me clarify a couple of things, because this isn’t anything like a “normal” NIC we are developing.

>>When you say “output”, what do you mean? Do you mean you are sucking streams from the network and sending frames into memory?
Yes. We are capturing raw video streams from the network - uncompressed. Images from all sorts of medical modalities are streamed on the network through our dedicated encoders (e.g. DVI or DP in -> raw network video out).

>>Will 8 uncompressed streams fit in your bandwidth?
Yes. The network itself is 4x 10G optical and the PCIe interface is gen3.

>>I’m a bit confused as to your architecture. Does your hardware appear to the system as a network card? Or does it appear as a completely custom device? If it’s a custom device, are you writing the whole network stack yourself? That’s a heck of a job.

The hardware appears on the PCI bus as a custom device. We feed it a WDF bus driver that spawns several child drivers: 4 x NDIS driver and 1 x AVStream driver. Each individual driver will then map its own portion of the hardware resources. The NDIS driver supports the ‘legacy’ part of the 4 network interfaces: to Windows they appear in the device manager as 4 extra regular network interfaces.
The video streams network handling however is done completely in hardware. That is why we need to be able to set a stream’s multicast address and port into the hardware to enable it to split off the video stream from the normal legacy network packets. Handling of individual frames is also done in hardware - as DMA bus master. We only need to set up the DMA SG list to dump the incoming frames to memory.

So, my questions are reduced to a single one:

How can we add 2 extra properties to a capture pin that enable us to set a multicast IP address and a port number ? Are there any examples of this ?

Thank You,

  • Bernard
4

xxxxx@hotmail.com wrote:

So, my questions are reduced to a single one:

How can we add 2 extra properties to a capture pin that enable us to set a multicast IP address and a port number ? Are there any examples of this ?

I was going to point you to the AVStream examples, but I see that
neither of them actually defines any properties. If you have the 7600
WDK, you can look at the audio\gfxswap.xp\driver sample. Despite it’s
name, that’s an AVStream filter driver, and it defines a raft of properties.

Here’s the basic idea. You need to generate a new GUID for your
property set. (You can have more than one.) Within that property set,
you define a set of property IDs. These are just integers; I usually
use an enum for that.

You define the properties in each set in a KSPROPERTY_TABLE, using:

DEFINE_KSPROPERTY_TABLE(MyFilter_MyCustomPropertySet)
{
DEFINE_KSPROPERTY_ITEM
(
KSPROPERTY_MYSET_POWERLEVEL,
…name of Get handler, if property can be read…,
sizeof(KSPROPERTY),
sizeof(ULONG), // <– size of the information your
property will exchange
…name of Set handler, if property can be written…,
NULL, 0, NULL, NULL,
sizeof(ULONG)
),
… repeated for other ids …
};

Because so much of that is common, you can define a macro to make it
less wordy.

Then, you gather all of your property sets into one property set table
that associates the GUIDs:

DEFINE_KSPROPERTY_SET_TABLE(MyFilterProperties)
{
DEFINE_KSPROPERTY_SET
(
&KSPROPSETID_MyPropertySet, // <– your GUID
SIZEOF_ARRAY(MyFilter_MyCustomPropertySet),
MyFilter_MyCustomPropertySet,
0, NULL
),
… repeated for other property sets …
}

Then, you put a link to that table in your automation table:

DEFINE_KSAUTOMATION_TABLE(MyFilterAutomationTable)
{
DEFINE_KSAUTOMATION_PROPERTIES(MyFilterProperties),
DEFINE_KSAUTOMATION_METHODS_NULL,
DEFINE_KSAUTOMATION_EVENTS_NULL
};

Then, you put a link to the automation table in your
KSFILTER_DESCRIPTOR. AVStream and ksproxy does the rest. DirectShow
apps that instantiate your filter can fetch an IKsControl interface and
make KsProperty requests to talk to the property. Or, if you want, you
can use the WD-3 ksproxy article you mentioned earlier to create a
friendlier COM wrapper around the IKsControl interface.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

5

Tim Roberts wrote:

Then, you put a link to the automation table in your
KSFILTER_DESCRIPTOR.

… or, in your case, in the KSPIN_DESCRIPTOR…

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

6

Thanks, Tim !
This is definitely very valuable information to AVStream driver newbies like us.
We have the 7600 DDK but we were not aware of the audio example as AVStream driver.
I am sure new questions will pop up during further development…
Thanks again for Your time and effort !

  • Bernard