Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTDEV

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


Before Posting...

Please check out the Community Guidelines in the Announcements and Administration Category.

Driver or filter to add RAW8 support to UVC?

KenEEKenEE Member Posts: 3

Hi,
We have created a custom UVC camera board with a Sony IMX586 48MP sensor module. The main chip is the Cypress CX3 and the solution is "working" - that is we get video, stills, and can control the focus etc. through the standard UVC methods on Windows 10.
The issue is that we did not include any image processing on the board to convert the sensor's RAW8 format to YUV or MJPEG - something supported directly by UVC. So we pack the RAW8 data as YUV and then our custom app can convert it for proper display.
So we need our camera to work as any UVC webcam would. Before we respin our board and add an ISP or FPGA I wanted to explore writing a custom driver or filter to convert the RAW8 to YUV on the fly and automatically so that the camera will work in any windows app.
Can you driver experts comment on the best approach and if its even a good idea? or even possible?

Comments

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,970

    UVC will supports any arbitrary fourcc you can think of. UYVY and NV12 are in the "uncompressed" formats, but if you use the "frame-based" format, you can embed any fourcc that your computer can handle.

    By "RAW8", do you mean 8-bit monochrome data? That's usually described as "Y8" or "Y800". There are many commonly available codecs that can handle that format.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • KenEEKenEE Member Posts: 3

    Hi Tim,

    Thanks for the response! Are you saying fourcc may already have a codec or filter to perform the conversion? My first time to hear of this so will learn what I can. Is it free?
    By RAW8 I mean 1 byte per pixel in Bayer arrangement. This format is listed on fourcc.com as BA81 or possibly BYR1.
    Not sure how to make use of fourcc... (?)

    To be clear - what do you mean by "any fourcc that your computer can handle" ? Where do I find and/or augment the list that my computer can handle?

    Thanks,
    KenEE

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,970

    what do you mean by "any fourcc that your computer can handle" ?

    Most computers have accrued a rather wide variety of codecs through the installation of various video products. It used to be we all were keenly aware of what video we could handle and what video we couldn't. We commonly used to refer people to the "FFDShow Codec Pack" or various other codec packs. OpenCV supports a large set of codecs.

    My real point was that many people read the "Video Payload -- Uncompressed" document in the UVC spec and conclude that YUY2 and NV12 are the only possible uncompressed formats. That's not so. The "Video Payload -- Frame Based" document describes ANY rectangular arrangement of pixels. The format descriptor is slightly different, so be careful. You just include the GUID of your format, and any DirectShow fourcc can be encoded as XXXXXXXX-0000-0010-8000-00AA00389B71. I have a couple of clients with 8-bit monochrome cameras; we advertise Y800 and most computers can decode it.

    As for BYR1, this codec claims to support it: https://www.norpix.com/products/codecs/directshowcodec.php . I know nothing about that; my friend Google offered it to me.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • KenEEKenEE Member Posts: 3

    Hi Tim, I am in contact with Norpix to see if their codec will work for us. I will report back any results when available.
    But reading Microsoft's documentation makes me think one might/should be able to get DirectX/Show/3D to do the conversion using fourcc/D3DFORMAT definition. But I don't see any clear examples yet. Seems BayerToRGB/YUV would be super common since many/most Camera sensors output only RAW bayer encoded data. (at least all of the Sony, OnSemi, and OmniVision sensors we have)

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 13,970

    But reading Microsoft's documentation makes me think one might/should be able to get DirectX/Show/3D to do the conversion
    using fourcc/D3DFORMAT definition.

    Well, DirectShow doesn't actually do any conversions. It just provides the mechanism and sets up the framework into which arbitrary converters can be loaded. The built-in codecs can convert between the most common YUV formats (YUY2, UYVY, YV12, I420/IYUV, a few others) and the common RGB formats (8, 15, 16, 24, 32). A DirectShow transform filter is not very hard to write; even if you can't find one, you could write your own.

    Seems BayerToRGB/YUV would be super common since many/most Camera sensors output only RAW bayer encoded data

    But because Windows does not support it natively, every camera vendor I've worked with has had their own FPGA/CPLD to convert the Bayer codes to YUV or RGB, or to do MJPG compression.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Developing Minifilters 24 May 2021 Live, Online
Writing WDF Drivers 14 June 2021 Live, Online
Internals & Software Drivers 27 September 2021 Live, Online
Kernel Debugging 15 November 2021 Live, Online