Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Sept/Oct 2019 Issue of The NT Insider available


Download PDF here: http://insider.osr.com/2019/ntinsider_2019_01.pdf

It’s a particularly BIG issue, too: 40 pages of technical goodness, ranging from WDF to Minifilters. Check it out.
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

Windows 10 anniversary update and TestCap/AVStream drivers

OSR_Community_UserOSR_Community_User Member Posts: 110,217
Windows 10 anniversary update is still being rolled out to users.
I already updated and found that my TestCap based virtual video drivers does not work anymore with multiple apps simultaneously.
The same applies to AVStream based driver.

From what I observe, Windows does not call driver functions anymore in the client app (Skype, flash, etc) thread.
Instead, driver has only one 'svchost.exe' client now.
So it looks like operating system now controls which app is communicating to webcams.

Like for any real hardware webcam, only one application can connect to virtual camera at the moment.
When the second application is trying to connect - it displays "Camera is busy", "Unable to connect", etc (exact message text is up to the client app).

Does anybody know what kind of breaking change is that? Does it sound like a bug and thus worth contacting Microsoft?
Or is it just a dawn of virtual webcam apps?

Anton

Comments

  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Just found what what's hidden under svchost - it's new Windows Camera Frame Server.
    Here's the screenshot - https://www.dropbox.com/s/9umu891qsomq5nf/225521481_9485545356209439258.jpg?dl=0

    Funny, but descriptions is as follows:
    "Enables multiple clients to access video frames from camera devices."
    Though in reality it doesn't provide access to multiple clients neither for real webcams nor for the virtual ones.
  • There are more changes via this.

    MediaTypes get changed by this Frame Server.

    So that

    AM_MEDIA_TYPE VIDEO_CONFIG_CAPS
    [0] RGB32 640x480x30fps Min=10, Max=30 fps,
    MinOutputSize=320x240,MaxOutputSize=640x480

    get_framerates( index=0,dim=640x480 ) returns 10, 15, 30
    get_framerates( index=0,dim=320x240 ) returns 20, 30, 60

    now becomes
    AM_MEDIA_TYPE VIDEO_CONFIG_CAPS
    [0] RGB32 640x480x30fps 30 fps, 640x480 Min == Max
    [1] RGB32 640x480x15fps 15 fps, 640x480 Min == Max
    [2] RGB32 640x480x10fps 10 fps, 640x480 Min == Max

    get_framerates( index=0, dim=640x480 ) returns 30
    get_framerates( index=0, dim=320x240 ) returns 30

    get_framerates( index=1, dim=640x480 ) returns 15
    get_framerates( index=1, dim=320x240 ) returns 15

    This breakes GraphEdit framerate selector.

    A AnalogVideoInput pin on one of my devices vanishes ...
    A AUDIO_OUT_PIN on one of my devices cannot deliver PCM samples anymore ...

    This update breaks many things for my AVStream based drivers.

    On 08.08.2016 09:37, xxxxx@gmail.com wrote:
    > Just found what what's hidden under svchost - it's new Windows Camera Frame Server.
    > Here's the screenshot - https://www.dropbox.com/s/9umu891qsomq5nf/225521481_9485545356209439258.jpg?dl=0
    >
    > Funny, but descriptions is as follows:
    > "Enables multiple clients to access video frames from camera devices."
    > Though in reality it doesn't provide access to multiple clients neither for real webcams nor for the virtual ones.
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: <http://www.osronline.com/showlists.cfm?list=ntdev>;
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    > Details at <http://www.osr.com/seminars>;
    >
    > To unsubscribe, visit the List Server section of OSR Online at <http://www.osronline.com/page.cfm?name=ListServer>;
    >
  • Roman_RyltsovRoman_Ryltsov Member Posts: 3
    As explained by Mike M in this thread on MSDN Forums https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/9d6a8704-764f-46df-a41c-8e9d84f7f0f3/mjpg-encoded-media-type-is-not-available-for-usbuvc-webcameras-after-windows-10-version-1607-os?forum=mediafoundationdevelopment

    > With the Anniversary Update to Windows 10, it is now possible for multiple applications to access the camera in ways that weren?t possible before. It was important for us to enable concurrent camera access, so Windows Hello, Microsoft Hololens and other products and features could reliably assume that the camera would be available at any given time, regardless of what other applications may be accessing it.

    I however was unable to run two simultaneous video capture sessions. Both DirectShow and Media Foundation API video capture sessions however are indeed now routed through this new helper service.
  • Jan_BottorffJan_Bottorff Member - All Emails Posts: 471
    So I read that thread on the MSDN forum.

    It seems like the issue may be the camera virtualization layer is preferring uncompressed over compressed formats when negotiating with the physical camera, as no clients will be requesting compressed formats, because the camera virtualization layer seems to only support uncompressed formats. This will be a disaster for high resolution video over USB2.

    A potential workaround for physical camera drivers would be to NOT expose any uncompressed formats, even if the hardware supports them, forcing the camera virtualization layer to negotiate a compressed format from the physical camera, which then does software decompression before cloning the frames for each app. This would keep the data rate over USB2 reasonable. I have no idea if WHQL tests require exposing some uncompressed formats, as a guaranteed format the camera and app can agree on.

    Jan

    On 8/14/16, 11:33 AM, "xxxxx@lists.osr.com on behalf of xxxxx@roman.adenix.net" <xxxxx@lists.osr.com on behalf of xxxxx@roman.adenix.net> wrote:

    As explained by Mike M in this thread on MSDN Forums https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/9d6a8704-764f-46df-a41c-8e9d84f7f0f3/mjpg-encoded-media-type-is-not-available-for-usbuvc-webcameras-after-windows-10-version-1607-os?forum=mediafoundationdevelopment

    > With the Anniversary Update to Windows 10, it is now possible for multiple applications to access the camera in ways that weren?t possible before. It was important for us to enable concurrent camera access, so Windows Hello, Microsoft Hololens and other products and features could reliably assume that the camera would be available at any given time, regardless of what other applications may be accessing it.

    I however was unable to run two simultaneous video capture sessions. Both DirectShow and Media Foundation API video capture sessions however are indeed now routed through this new helper service.

    ---
    NTDEV is sponsored by OSR

    Visit the list online at: <http://www.osronline.com/showlists.cfm?list=ntdev>;

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at <http://www.osr.com/seminars>;

    To unsubscribe, visit the List Server section of OSR Online at <http://www.osronline.com/page.cfm?name=ListServer>;
  • Roman_RyltsovRoman_Ryltsov Member Posts: 3
    My impression is that:

    - raw video formats are passed through by video sharing service
    - MJPG is decoded into NV12 for Media Foundation clients only (as opposed to DirectShow clients)
    - H264 available on second pin of UVC 1.5 camera... dropped? No details on whether virtualization layer would prefer H264 to MJPG

    > It seems like the issue may be the camera virtualization layer is preferring uncompressed over compressed formats when negotiating with the physical camera, as no clients will be requesting compressed formats, because the camera virtualization layer seems to only support uncompressed formats. This will be a disaster for high resolution video over USB2.
  • Roman_RyltsovRoman_Ryltsov Member Posts: 3
    Sorry, I should have completed my thought about

    > - MJPG is decoded into NV12 for Media Foundation clients only (as opposed to DirectShow clients)

    Media Foundation clients are reported resolution/rates which correspond to format that exceed USB 2.0 bandwidth, so it looks like a client which selects NV12 format can expect that virtualization layer requests compressed video from device and applies decoding. There is also no information on what decoding is effectively used: Microsoft says "decodes in optimal way" so I suppose they might pick hardware accelerated decoder when it's available - I would assume they have this implemented.
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,103
    Jan Bottorff wrote:
    > It seems like the issue may be the camera virtualization layer is preferring uncompressed over compressed formats when negotiating with the physical camera, as no clients will be requesting compressed formats, because the camera virtualization layer seems to only support uncompressed formats. This will be a disaster for high resolution video over USB2.

    It's likely a disaster in more ways than one. This is not the first
    time that some enthusiastic engineer at Microsoft has concluded that
    "every camera in the world obviously must work exactly like the four
    samples I have in my office", and every time that happens, they're wrong.

    There are going to be a lot of people who are upset to learn that there
    is a Windows component intercepting and reading every camera stream that
    gets produced. I (and my clients) don't give a crap about Windows Hello
    or Windows HoloLens, and it's totally unacceptable that they should
    break a vast installed base to optimize for their applications. If
    there's not an opt-out strategy already, I'm guessing one will be hacked
    in.

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
Writing WDF Drivers 21 Oct 2019 OSR Seminar Space & ONLINE
Internals & Software Drivers 18 Nov 2019 Dulles, VA
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 27 Apr 2020 OSR Seminar Space & ONLINE