Windows 10 anniversary update and TestCap/AVStream drivers

Windows 10 anniversary update is still being rolled out to users.
I already updated and found that my TestCap based virtual video drivers does not work anymore with multiple apps simultaneously.
The same applies to AVStream based driver.

From what I observe, Windows does not call driver functions anymore in the client app (Skype, flash, etc) thread.
Instead, driver has only one ‘svchost.exe’ client now.
So it looks like operating system now controls which app is communicating to webcams.

Like for any real hardware webcam, only one application can connect to virtual camera at the moment.
When the second application is trying to connect - it displays “Camera is busy”, “Unable to connect”, etc (exact message text is up to the client app).

Does anybody know what kind of breaking change is that? Does it sound like a bug and thus worth contacting Microsoft?
Or is it just a dawn of virtual webcam apps?

Anton

Just found what what’s hidden under svchost - it’s new Windows Camera Frame Server.
Here’s the screenshot - https://www.dropbox.com/s/9umu891qsomq5nf/225521481_9485545356209439258.jpg?dl=0

Funny, but descriptions is as follows:
“Enables multiple clients to access video frames from camera devices.”
Though in reality it doesn’t provide access to multiple clients neither for real webcams nor for the virtual ones.

There are more changes via this.

MediaTypes get changed by this Frame Server.

So that

AM_MEDIA_TYPE VIDEO_CONFIG_CAPS
[0] RGB32 640x480x30fps Min=10, Max=30 fps,
MinOutputSize=320x240,MaxOutputSize=640x480

get_framerates( index=0,dim=640x480 ) returns 10, 15, 30
get_framerates( index=0,dim=320x240 ) returns 20, 30, 60

now becomes
AM_MEDIA_TYPE VIDEO_CONFIG_CAPS
[0] RGB32 640x480x30fps 30 fps, 640x480 Min == Max
[1] RGB32 640x480x15fps 15 fps, 640x480 Min == Max
[2] RGB32 640x480x10fps 10 fps, 640x480 Min == Max

get_framerates( index=0, dim=640x480 ) returns 30
get_framerates( index=0, dim=320x240 ) returns 30

get_framerates( index=1, dim=640x480 ) returns 15
get_framerates( index=1, dim=320x240 ) returns 15

This breakes GraphEdit framerate selector.

A AnalogVideoInput pin on one of my devices vanishes …
A AUDIO_OUT_PIN on one of my devices cannot deliver PCM samples anymore …

This update breaks many things for my AVStream based drivers.

On 08.08.2016 09:37, xxxxx@gmail.com wrote:

Just found what what’s hidden under svchost - it’s new Windows Camera Frame Server.
Here’s the screenshot - https://www.dropbox.com/s/9umu891qsomq5nf/225521481_9485545356209439258.jpg?dl=0

Funny, but descriptions is as follows:
“Enables multiple clients to access video frames from camera devices.”
Though in reality it doesn’t provide access to multiple clients neither for real webcams nor for the virtual ones.


NTDEV is sponsored by OSR

Visit the list online at: http:
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at http:
></http:></http:></http:>

As explained by Mike M in this thread on MSDN Forums https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/9d6a8704-764f-46df-a41c-8e9d84f7f0f3/mjpg-encoded-media-type-is-not-available-for-usbuvc-webcameras-after-windows-10-version-1607-os?forum=mediafoundationdevelopment

With the Anniversary Update to Windows 10, it is now possible for multiple applications to access the camera in ways that weren?t possible before. It was important for us to enable concurrent camera access, so Windows Hello, Microsoft Hololens and other products and features could reliably assume that the camera would be available at any given time, regardless of what other applications may be accessing it.

I however was unable to run two simultaneous video capture sessions. Both DirectShow and Media Foundation API video capture sessions however are indeed now routed through this new helper service.

So I read that thread on the MSDN forum.

It seems like the issue may be the camera virtualization layer is preferring uncompressed over compressed formats when negotiating with the physical camera, as no clients will be requesting compressed formats, because the camera virtualization layer seems to only support uncompressed formats. This will be a disaster for high resolution video over USB2.

A potential workaround for physical camera drivers would be to NOT expose any uncompressed formats, even if the hardware supports them, forcing the camera virtualization layer to negotiate a compressed format from the physical camera, which then does software decompression before cloning the frames for each app. This would keep the data rate over USB2 reasonable. I have no idea if WHQL tests require exposing some uncompressed formats, as a guaranteed format the camera and app can agree on.

Jan

On 8/14/16, 11:33 AM, “xxxxx@lists.osr.com on behalf of xxxxx@roman.adenix.net” wrote:

As explained by Mike M in this thread on MSDN Forums https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/9d6a8704-764f-46df-a41c-8e9d84f7f0f3/mjpg-encoded-media-type-is-not-available-for-usbuvc-webcameras-after-windows-10-version-1607-os?forum=mediafoundationdevelopment

> With the Anniversary Update to Windows 10, it is now possible for multiple applications to access the camera in ways that weren?t possible before. It was important for us to enable concurrent camera access, so Windows Hello, Microsoft Hololens and other products and features could reliably assume that the camera would be available at any given time, regardless of what other applications may be accessing it.

I however was unable to run two simultaneous video capture sessions. Both DirectShow and Media Foundation API video capture sessions however are indeed now routed through this new helper service.


NTDEV is sponsored by OSR

Visit the list online at: http:

MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at http:

To unsubscribe, visit the List Server section of OSR Online at http:</http:></http:></http:>

My impression is that:

  • raw video formats are passed through by video sharing service
  • MJPG is decoded into NV12 for Media Foundation clients only (as opposed to DirectShow clients)
  • H264 available on second pin of UVC 1.5 camera… dropped? No details on whether virtualization layer would prefer H264 to MJPG

It seems like the issue may be the camera virtualization layer is preferring uncompressed over compressed formats when negotiating with the physical camera, as no clients will be requesting compressed formats, because the camera virtualization layer seems to only support uncompressed formats. This will be a disaster for high resolution video over USB2.

Sorry, I should have completed my thought about

  • MJPG is decoded into NV12 for Media Foundation clients only (as opposed to DirectShow clients)

Media Foundation clients are reported resolution/rates which correspond to format that exceed USB 2.0 bandwidth, so it looks like a client which selects NV12 format can expect that virtualization layer requests compressed video from device and applies decoding. There is also no information on what decoding is effectively used: Microsoft says “decodes in optimal way” so I suppose they might pick hardware accelerated decoder when it’s available - I would assume they have this implemented.

Jan Bottorff wrote:

It seems like the issue may be the camera virtualization layer is preferring uncompressed over compressed formats when negotiating with the physical camera, as no clients will be requesting compressed formats, because the camera virtualization layer seems to only support uncompressed formats. This will be a disaster for high resolution video over USB2.

It’s likely a disaster in more ways than one. This is not the first
time that some enthusiastic engineer at Microsoft has concluded that
“every camera in the world obviously must work exactly like the four
samples I have in my office”, and every time that happens, they’re wrong.

There are going to be a lot of people who are upset to learn that there
is a Windows component intercepting and reading every camera stream that
gets produced. I (and my clients) don’t give a crap about Windows Hello
or Windows HoloLens, and it’s totally unacceptable that they should
break a vast installed base to optimize for their applications. If
there’s not an opt-out strategy already, I’m guessing one will be hacked
in.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.