[AVStream] Extending avshws for audio capture

NTDEV
1

I’m trying to come up with an AVStream minidriver for audio / video
capture. I’m using AVSHWS as my reference and trying to add audio capture
functionality.

Towards this, I have added an additional pin and pin info to filter
descriptor -

const

KSPIN_DESCRIPTOR_EX

CaptureFilterPinDescriptors [CAPTURE_FILTER_PIN_COUNT] = {

//

// Video Capture Pin

//

{

&CapturePinDispatch,

NULL,

{

0, // Interfaces (NULL, 0 ==
default)

NULL,

0, // Mediums (NULL, 0 == default)

NULL,

SIZEOF_ARRAY(CapturePinDataRanges),// Range Count

CapturePinDataRanges, // Ranges

KSPIN_DATAFLOW_OUT, // Dataflow

KSPIN_COMMUNICATION_BOTH, // Communication

&PIN_CATEGORY_CAPTURE, // Category

&g_PINNAME_VIDEO_CAPTURE, // Name

0 // Reserved

},

#ifdef X86

KSPIN_FLAG_GENERATE_MAPPINGS | // Pin Flags

#endif

KSPIN_FLAG_PROCESS_IN_RUN_STATE_ONLY,

1, // Instances Possible

1, // Instances Necessary

&CapturePinAllocatorFraming, // Allocator Framing

reinterpret_cast

(CCapturePin::IntersectHandler)

},

//

// Audio Capture Pin

//

{

&AudioCapturePinDispatch,

NULL,

{

0, // Interfaces (NULL, 0 == default)

NULL,

0, // Mediums (NULL, 0 == default)

NULL,

SIZEOF_ARRAY(AudioCapturePinDataRanges),// Range Count

AudioCapturePinDataRanges, // Ranges

KSPIN_DATAFLOW_OUT, // Dataflow

KSPIN_COMMUNICATION_BOTH, // Communication

&KSCATEGORY_AUDIO, // Category

&g_PINNAME_AUDIO_CAPTURE, // Name

0 // Reserved

},

KSPIN_FLAG_DO_NOT_INITIATE_PROCESSING | // Flags

KSPIN_FLAG_PROCESS_IN_RUN_STATE_ONLY |

KSPIN_FLAG_FIXED_FORMAT,

1, // Instances Possible

0, // Instances Necessary

&AudCapturePinAllocatorFraming, // Allocator Framing

reinterpret_cast

(CAudioCapturePin::AudioIntersectHandler)

}

};

As a first step, I’m trying to see if this audio pin can be connected to
any standard render pin. But I get the following error in graphEdit -

Return code 0x80040274 (No Video port hardware is available or hardware is
not responding). However, the video capture pin is working fine

I checked what is happening through debugView. Looks like a whole lot of
IntersectHandler callbacks are being made. There are no errors caught
either.

Here’s a snippet of my intersectHandler implementation -

…

const GUID AudioInfoSpecifier =
{ STATICGUIDOF(KSDATAFORMAT_SPECIFIER_WAVEFORMATEX) };

NT_ASSERT(Filter);
NT_ASSERT(Irp);
NT_ASSERT(PinInstance);
NT_ASSERT(CallerDataRange);
NT_ASSERT(DescriptorDataRange);
NT_ASSERT(DataSize);

ULONG DataFormatSize;

//
// Specifier FORMAT_VideoInfo for VIDEOINFOHEADER
//
if (IsEqualGUID(CallerDataRange->Specifier, AudioInfoSpecifier) &&
CallerDataRange->FormatSize >= sizeof(KSDATARANGE_AUDIO)) {

PKSDATARANGE_AUDIO callerDataRange =
reinterpret_cast <pksdatarange_audio> (CallerDataRange);

PKSDATARANGE_AUDIO descriptorDataRange =
reinterpret_cast <pksdatarange_audio> (DescriptorDataRange);

//
// If the passed buffer size is 0, it indicates that this is a size
// only query. Return the size of the intersecting data format and
// pass back STATUS_BUFFER_OVERFLOW.
//
if (BufferSize == 0) {

*DataSize = sizeof(KSDATAFORMAT_WAVEFORMATEX);

return STATUS_BUFFER_OVERFLOW;

}

//
// Check that the other fields match
//
if ((callerDataRange->MaximumBitsPerSample !=
descriptorDataRange->MaximumBitsPerSample) ||
(callerDataRange->MaximumChannels !=
descriptorDataRange->MaximumChannels) ||
(callerDataRange->MaximumSampleFrequency !=
descriptorDataRange->MaximumSampleFrequency) ||
(callerDataRange->MinimumBitsPerSample !=
descriptorDataRange->MinimumBitsPerSample) ||
(callerDataRange->MinimumSampleFrequency !=
descriptorDataRange->MinimumSampleFrequency))
{
return STATUS_NO_MATCH;
}

PKSDATAFORMAT_WAVEFORMATEX WaveFormat =
reinterpret_cast <pksdataformat_waveformatex> (Data);

DataFormatSize =
sizeof(KSDATAFORMAT_WAVEFORMATEX);

RtlCopyMemory(
&WaveFormat->DataFormat,
&descriptorDataRange->DataRange,
sizeof(KSDATAFORMAT)
);

//
// Verify that the provided structure is large enough to
// accept the result.
//
if (BufferSize < DataFormatSize)
{
return STATUS_BUFFER_TOO_SMALL;
}

WaveFormat->WaveFormatEx.wFormatTag = WAVE_FORMAT_PCM;
WaveFormat->WaveFormatEx.nChannels =
(WORD)descriptorDataRange->MaximumChannels;
WaveFormat->WaveFormatEx.nSamplesPerSec =
descriptorDataRange->MaximumSampleFrequency;
WaveFormat->WaveFormatEx.wBitsPerSample =
(WORD)descriptorDataRange->MaximumBitsPerSample;
WaveFormat->WaveFormatEx.nBlockAlign =
(WaveFormat->WaveFormatEx.wBitsPerSample / 8) *
WaveFormat->WaveFormatEx.nChannels;
WaveFormat->WaveFormatEx.nAvgBytesPerSec =
WaveFormat->WaveFormatEx.nSamplesPerSec *
WaveFormat->WaveFormatEx.nBlockAlign;
WaveFormat->WaveFormatEx.cbSize = 0;
WaveFormat->DataFormat.SampleSize =
WaveFormat->WaveFormatEx.nBlockAlign;

//
// Copy over the KSDATAFORMAT, followed by the actual VideoInfoHeader
//
*DataSize = DataFormatSize;

//
// REVIEW - Perform other validation such as cropping and scaling checks
//

return STATUS_SUCCESS;

} // End of AudioInfoSpecifier
…

Please help me out here !!</pksdataformat_waveformatex></pksdatarange_audio></pksdatarange_audio>

2

[update]

I just cleaned up my code a little and tried on a different Windows 7 system. Now when I try to “render pin” in GraphEdit, I’m getting error code 0x80040256

By definition, the error seems to be -
VFW_E_NO_AUDIO_HARDWARE
0x80040256
Cannot play back the audio stream: no audio hardware is available, or the hardware is not supported.

… and after some internet search, I happened to come across this -
http://www.freelists.org/post/wdmaudiodev/Few-Questions-on-AVStream

I’m trying to simulate an audio source similar to the video source simulation. I do not understand why video simulation works but audio doesn’t without an actual soundcard (if that IS the root cause).

Also, I’m only getting IntersectHandler callbacks, but no SetFormat or SetState. This behavior is different from the post I referred to earlier.

Any help towards this is appreciated. Thanks in advance

3

xxxxx@gmail.com wrote:

I just cleaned up my code a little and tried on a different Windows 7 system. Now when I try to “render pin” in GraphEdit, I’m getting error code 0x80040256

By definition, the error seems to be -
VFW_E_NO_AUDIO_HARDWARE
0x80040256
Cannot play back the audio stream: no audio hardware is available, or the hardware is not supported.
…
I’m trying to simulate an audio source similar to the video source simulation. I do not understand why video simulation works but audio doesn’t without an actual soundcard (if that IS the root cause).

It’s not that hard to understand. You asked it to render. You do have
a device that is capable of video rendering (that is, your graphics
card), but you do not have a device that capable of audio rendering.
Thus, it cannot possibly render the stream.

Now, there are certainly other sink filters that you can use, like the
oscilloscope filter or the null renderer. You just can’t use the
default rendering unless you have a sound card.

Also, I’m only getting IntersectHandler callbacks, but no SetFormat or SetState. This behavior is different from the post I referred to earlier.

If it never finished the graph, then it can’t set the format or change
the state.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

4

Manu R Iyengar wrote:

I’m trying to come up with an AVStream minidriver for audio / video
capture. I’m using AVSHWS as my reference and trying to add audio
capture functionality.

Towards this, I have added an additional pin and pin info to filter
descriptor -

…

As a first step, I’m trying to see if this audio pin can be connected
to any standard render pin. But I get the following error in graphEdit -

Did you borrow code from the avssamp sample? It includes both a video
pin and an audio pin, the only difference being that it is
filter-centric instead of pin-centric. That only affects the process
callback; the format negotiation part is identical.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

5

Thanks for the reply, Tim

It is as you said -

You just can’t use the default rendering unless you have a sound card.

I did try this with an actual audio render device connected and the pin connection succeeded

Did you borrow code from the avssamp sample? It includes both a video
pin and an audio pin, the only difference being that it is
filter-centric instead of pin-centric. That only affects the process
callback; the format negotiation part is identical.

I was just trying to see if audio pin settings are fine. I did borrow some code from avssamp to achieve this. I have yet to add the actual audio simulation (through file read).

Thanks again for your time

Manu

6

Another question…

I’m working on a PCI based hardware which supports video and audio both. My (preferred) design approach is to have a single AVStream driver which takes care of both audio and video capture. But what I see is that with typical user applications like VLC, I can only do video capture with this method.

Now I can only think of the following possible approaches -

  1. Split the driver into 3 parts - a PCI communication driver, a virtual video driver and a virtual audio driver (the virtual drivers should talk to PCI communication driver for data capture)
  2. make AVStream driver export both audio and video interfaces that VLC (and other typical media applications) can understand

    I’m a newbie to windows and need some help in figuring out the best possible alternative here (even if its not listed above). Can you please help me on this ?
7

[update on A/V capture question]
I have done some study (mostly other related threads) and this is what I have come up with -

I can write one AVStream driver with two filters, one for audio and another for video. The AVStream driver will handle PCI communication also. With this approach, I can handle both audio and video capture and support applications like VLC.
(This sounds kind of trivial, I know. But I’m posting this for closure)

If there are other method/s to handle this (A/V capture from PCI device), please let me know.

8

xxxxx@gmail.com wrote:

[update on A/V capture question]
I have done some study (mostly other related threads) and this is what I have come up with -

I can write one AVStream driver with two filters, one for audio and another for video. The AVStream driver will handle PCI communication also. With this approach, I can handle both audio and video capture and support applications like VLC.
(This sounds kind of trivial, I know. But I’m posting this for closure)

If there are other method/s to handle this (A/V capture from PCI device), please let me know.

It depends. Are the audio and video independent of each other? Do you
want to be able to use the microphone and the camera in separate
applications? If so, then two filters is the right way. If you have a
USB web camera with one video interface and one audio interface, it will
appear as two unrelated filters.

But that’s certainly not the only way. A television device, for
example, where the video and audio streams are handled together, usually
has one filter with an audio pin and a video pin. That’s what the
“avssamp” sample demonstrates.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.