Modify KMDOD sample for USB display

workingmailing@163.com wrote:

As you known, Windows have native USB VIDEO CLASS driver.

Now USB-IF release the USB AV class spec, especially for USB display.
Displaylink is also the member of this spec contributor.

Can you, the experts, give a calculation of the time, money, on the USB AV class driver development?

Or any company, software design house who could supplied this USB AV class driver service?

Supporting USB AV class is not the hard part. It is largely a
combination of USB Video Class and USB Audio Class. The hard part is
the WDDM part. Frankly, I doubt we’ll ever see that from Microsoft.

It’s worth pointing out that Windows 8 now includes the “Desktop
Duplication API”, in which user-mode applications now have a supported
mechanism for getting a surface that contains an image of the entire
desktop. It’s possible that a lot of the magic that has been done in
the past in this area can now be done through this API.

http://msdn.microsoft.com/en-us/library/windows/desktop/hh404487.aspx


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

If you look at typical developer productivity, something like 5,000-10,000 lines of debugged tested system level code per developer man-year is about right. I know there are periods when my productivity is way higher than that, but I have informally kept track of how many lines of code I and the teams I’m on produced per year and have come to the conclusion those numbers are realistic. Even though a developer may crank our 1000 lines of new code one day, he may also then spend the next 3 days testing it, and a day in meetings, and a few days reading newsgroup messages. Don’t forget there are the bugs you spend 2 weeks on, and don’t find a fix for, so you get zero lines of code done for that 2 weeks. The average vs short burst productivity is often vastly different.

There are a number of papers on the Internet that look at developer productivity. I remember one analysis about a Cisco switch OS, and an analysis about Linux kernel development seems like good comparable projects for driver development.

I see the paper at http://www.sjm06.com/SJM%20ISSN1452-4864/7_1_2012_May_1_170/7_1_65-75.pdf has a bunch of references to development productivity.

Estimating 5-10 man-years for a production quality complex kernel driver that’s 50K lines of code is not unrealistic. If you translate this to dollars, one rule of thumb is $250K per developer man-year (salary/overhead/equipment/office space), so that could be a $2+ million project.

It’s possible in the OPs case, management had not planned on driver development costing $2 million (5 developers for 2 years), yet that’s what it may take to be competitive.

The interesting things about code development productivity is it tends to stay the same number of lines of code across different languages, which is why you get more functionally in a certain time in say C# than C. There are metrics about how “dense” different languages are, like C++ I believe is around twice the functional density as C.

For a while I’ve made rough estimates on project size by estimating how many lines of code I thought it would be, and then assuming 7K-10K lines per developer man-year (use an environment adjustment factor). The tricky part is finding some valid comparable project to estimate the lines of code, although you can often find things like open source or other sample projects that you decide are “similar” complexity, and you can easily measure their lines of code. Estimating also gets a lot harder when you have some existing code. I know of one project where it was like a man-year just to get the existing header files to compile. I sometimes make an estimate of new code plus modifying existing code, applying some adjustment factor (like 0.5) to the existing code. You can work with the existing code for a while and measure your progress to tune the adjustment factor. Adjustment factors can also be > 1.0, when it takes longer to modify existing code than it would take to write it from scratch. If you have to make existing code work on a new platform, without breaking it on another platform, adjustment factors > 1.0 happen.

Jan

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Wednesday, June 4, 2014 7:56 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Modify KMDOD sample for USB display

Marcel Ruedinger wrote:

  1. WDDM Hook driver as described above.
  2. WDDM virtual graphics adapter driver. Same
    effort: ~10 man years.

You said your driver was 50,000 lines of code. Why would that take ten man years to develop?

The KM DOD works with the in-box (win8+) usermode “warp” driver. You only
need your own usermode driver for win7. Oh, and the sample can be readily
down-leveled to win7 support.

Mark Roddy

On Tue, Jun 3, 2014 at 9:08 AM, wrote:

> Thank you, it seems that you are work for DisplayLink.
>
> BTW, I have another question, cause WDDM framework need user mode and
> kernel mode driver.
> Here the KMDOD sample is a KMD, should I also need to develop a user mode
> driver, or can use the existing user mode driver in system, but if it is
> the later case, which user mode driver it could be used?
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

@Chris

Code metrics and their relevance to driver development actually are one of our favorite topics for late night engineering discussions here. This subject has already accelerated the consumption of countless mugs of Bavarian beer and glasses of French red wine (even a few Californian Pinot Noirs recently)…
Yet, the result is only theoretic - no proven answer!
I guess that’s the nature of code metrics anyway…
This applies even more to the simplest one of all:
“Lines of code per day”.

Just two tiny thoughts about this question (keeping things short and readable):

Paragraph One - Borderline range of the simple metrics:

  1. Window drivers average “Lines of code per day”:
    Our simplest rule of thumb for development of average simplest WDM or miniport drivers is 10.000 lines of code and 6 weeks core development time (average after almost two decades of Windows driver development experience).
    => Window drivers average “Lines of code per day”: ~300
    (very high value, probably because most of the code is copied from samples).

  2. Critical high quality embedded systems code:
    From our defense, aviation, medical and automotive embedded customers developing critical code (typically non-Windows embedded), we heard numbers ranging as low as
    => “Lines of code per day”: ~4

Obviously, our WDDM code is much closer to critical highest quality embedded systems code than to average driver code. This is boiling down to ~20 “Lines of code per day”. I can confirm that every single one of these lines is covered with blood sweat and tears.

Paragraph Two - WDDM specific:

After the first one and a half years, we only had a stable Windows 7 Non Aero implementation with sufficient performance. What takes so long? Example: Faking video memory seems simple at first. When supporting multiple monitors, it needs more contiguous memory than regular PCs have. If video memory is “CPU visible”, then DirectX Kernel allocates it as contiguous memory which is completely useless due to wrong caching attributes (write combined). For multiple monitors, it allocates too much of it so it will not get any (not even upon early boot). Actually a bizarre situation: Useless and unavailable memory prevents DirectX kernel from operating and you cannot prevent DirectX kernel from allocating and probing it. If you make the memory “Non-CPU visible”, then the same problem has a different name: “Swizzling Range”. Dead end for every regular driver developer. Massive loss of time for anyone who is able to create a workaround…

…I promised to keep it short…

…finally after 10 man years of development, the performance reaches 26-28FPS on 1080p video playback (Windows 7 AERO, Windows 8 and 8.1). Happy customers use it worldwide BUT unfortunately, this is the absolute limit. There is no way to ever get it any faster than that! Need a completely different solution for 1080p at 60FPS: Full WDDM Filter Hook Driver

If you are interested in more details about our favorite late night engineering discussion topics “Code Metrics” or “WDDM Pitfalls”, feel invited, but don’t forget to bring a solid supply of French red wine (1997 Chateau Cheval Blanc is my favorite one).

@Jan. Interesting contribution! You would be invited without need to bring any substances :wink:

Marcel Ruedinger
datronicsoft

The KM DOD works with the in-box (win8+) usermode “warp” driver. You only
need your own usermode driver for win7. Oh, and the sample can be readily
down-leveled to win7 support.

Hi, Mark

Here your “down-leveled” is change the KMDOD sample to full graphic mode on Win7, casue Win7 WDDM1.1 could not support KMDOD?
So if we change the KMDOD to full graphic mode driver, in Win7 we can also use the wrap usermode too?

Thank you.

Let me add only a few more questions…

Down leveled? Still a DOD or a full WDDM?

Calling DxgkInitializeDisplayOnlyDriver?

  • Never heard that this could be possible on Win7.
  • If yes, would it coexist with WDDM 1.0 and WDDM 1.1
    main graphics adapter drivers?

Or calling DxgkInitialize?

  • How did you fake Video Memory (see my example above)?
  • Or did you “just have” the video memory because installed
    on top of some virtual machine graphics adapter hardware?

How could such a “down leveled” driver support Windows 7 Non-Aero scenarios?
Windows 7 Non-Aero scenarios don’t rely on D3D nor WDDM User Mode Drivers
(thus WARP does not help). DOD sample does not contain any code for Windows 7
Non-Aero scenarios.

How did you implement the WDDM User Mode driver code?
Reverse translating DX10 DDI to DX10 API redirecting to Warp?
If yes, then can you also redirect to D3D9?
Can you also support video playback by DXVA which needs DX9 User Mode Driver?
Does it also work in scenarios with WDDM 1.0 DX9ex main graphics adapter?

These are only the minimum questions necessary to understand what this “down level” post is talking about. Actually, I have about 100 more such questions. Just not asking because I don’t want to appear rude or ask for too much.

Marcel Ruedinger
datronicsoft

No you have to have your own usermode driver, and that is a huge pile of
work. There is no way I know of to get warp running on win7.

Mark Roddy

On Wed, Jun 4, 2014 at 10:00 PM, wrote:

> The KM DOD works with the in-box (win8+) usermode “warp” driver. You only
> need your own usermode driver for win7. Oh, and the sample can be readily
> down-leveled to win7 support.
>
> Hi, Mark
>
> Here your “down-leveled” is change the KMDOD sample to full graphic mode
> on Win7, casue Win7 WDDM1.1 could not support KMDOD?
> So if we change the KMDOD to full graphic mode driver, in Win7 we can also
> use the wrap usermode too?
>
> Thank you.
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

The KMDOD sample can be readily modified to be a full WDDM 1.1 kernel mode
driver, but you have to have your own usermode display driver and obviously
no you cannot call DxgkInitializeDisplayOnlyDriver. Our usermode driver
does DX9 and fully supports aero and non-aero modes. From our driver’s
perspective “aero” is using flips and double buffering, “non-aero” is
single buffered. In both cases we have our own mechanism for getting
rectangles to the actual graphics adapter, this is a VM environment, we
just tell the user mode driver where to render and it tells us when to flip.

I have no idea about cohabitation with other wddm drivers as that isn’t an
issue for us.

I didn’t implement the user mode driver, I implemented our win8 KMDOD,
enhanced from the sample to be a lot more functional and to support our
hyper-visor, and did the initial work to move that back to win7 with a
non-warp user mode driver.

If you want to look at how other people are doing similar things, vbox has
an open source win7 WDDM driver, however their usermode driver is
translating DXn to OGL.

Mark Roddy

On Wed, Jun 4, 2014 at 11:43 PM, wrote:

> Let me add only a few more questions…
>
> Down leveled? Still a DOD or a full WDDM?
>
> Calling DxgkInitializeDisplayOnlyDriver?
> - Never heard that this could be possible on Win7.
> - If yes, would it coexist with WDDM 1.0 and WDDM 1.1
> main graphics adapter drivers?
>
> Or calling DxgkInitialize?
> - How did you fake Video Memory (see my example above)?
> - Or did you “just have” the video memory because installed
> on top of some virtual machine graphics adapter hardware?
>
> How could such a “down leveled” driver support Windows 7 Non-Aero
> scenarios?
> Windows 7 Non-Aero scenarios don’t rely on D3D nor WDDM User Mode Drivers
> (thus WARP does not help). DOD sample does not contain any code for
> Windows 7
> Non-Aero scenarios.
>
> How did you implement the WDDM User Mode driver code?
> Reverse translating DX10 DDI to DX10 API redirecting to Warp?
> If yes, then can you also redirect to D3D9?
> Can you also support video playback by DXVA which needs DX9 User Mode
> Driver?
> Does it also work in scenarios with WDDM 1.0 DX9ex main graphics adapter?
>
> These are only the minimum questions necessary to understand what this
> “down level” post is talking about. Actually, I have about 100 more such
> questions. Just not asking because I don’t want to appear rude or ask for
> too much.
>
> Marcel Ruedinger
> datronicsoft
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Thanks for the comprehensive info and solid base for discussion.
So “down-leveling” actually means “writing a full WDDM 1.1 DX9 driver
reusing kernel mode code (mostly VidPn) from DOD sample”.

While obviously feasible in this virtual machine environment, the scenario discussed in this thread (arbitrary consumer PC environment with connected USB display gadget) is very different. There it takes massive resources (again: 10 man years), if possible at all. Let us have a quick look again:

VM environment has resources which regular consumer PCs don’t have:

  1. VM has Video memory, regular PCs don’t have (as discussed above).
  2. Your VM environment seems to have a software DX9 renderer, regular PCs don’t have. One available DX9 software renderer is swiftshader. It was very expensive when we checked a longer time ago. Does anybody here know any other reasonable and affordable D3D9 software renderer?
  3. Etc.etc.etc.

Regular consumer PCs have degrees of freedom which VM environment does not have:

Below, only one example issue about cohabitation with other WDDM drivers.
In dual adapter environments (e.g. Laptops with Intel/Nvidia dual adapter combinations), NVidia has an own WDDM hook driver nvkflt.sys and nvumdshim.dll. This driver is not well behaved. Upon presence of other WDDM drivers, it always chooses one out of three options:

  1. It crashes Nvidia Control Panel.
  2. It crashes Desktop Window Manager.
  3. It crashes the whole operating system with Blue Screen of Death.
    So even if you manage to write your own compliant WDDM driver (which is already very unlikely), others will still kill you. How can you survive such violence from the tallest and strongest guys? Can you start to imagine how 10 man years can accumulate very quickly?

PS: Nice hint about DircetX to OpenGL translation in vbox driver. Will have a close look again. Haven’t looked much at this driver since we last discussed it here in October 2010.

Marcel Ruedinger
datronicsoft

One more question:
Cause USB Video class driver is actually a AVSTREAM miniport driver, it is specific for USB webcam.
Which create the graph by filters, pins and their dispatch and automation table by parser the USB device descriptors(topology)

Here USB AV class specific for USB display, is very similar to USB VIDEO class, by report the usb descriptors to host side driver, host will create the graph by parse them.

My question is, if a USB display driver which is USB AV class compliant, should it is a combination of WDDM miniport driver and AVSTREAM miniclass driver?

I have no time to analyze this in detail, nor can I make a solid statement.
However, this statement sounds roughly reasonable and very close to correct.

As discussed above in this long thread, I would not call it “WDDM miniport driver”. I would rather call it “Some kind of WDDM driver”. It could either be a WDDM filter hook driver OR a WDDM virtual adapter driver.

Marcel Ruedinger
datronicsoft

WARP is abbreviation for?
And what .sys or .dll file or files includes by WARP?

Windows Advanced Rasterization Platform

Sorry.

But I still find in this page, that Win7 have WARP in it, why can not use it as user mode driver?

What is WARP?
WARP is a high speed, fully conformant software rasterizer. It is a component of the DirectX graphics technology that was introduced by the Direct3D 11 runtime. The Direct3D 11 runtime is installed on Windows 7, Windows Server 2008 R2, and Windows Vista with the [KB971644] update. Windows 8, Windows Server 2012, and Windows RT include the Direct3D 11.1 runtime, which has an updated version of WARP.

Yes, I was already wondering, why WARP should not work on Win7.
Actually, I am also wondering about a few more statements above.
E.g. how could a WDDM User Mode Driver support Non-Aero?
This is implemented in Kernel Mode only…

But I don’t want to flood this newsgroup with too many WDDM details.
Can only focus on the most important topics here…

Marcel Ruedinger
datronicsoft

workingmailing@163.com wrote:

Here USB AV class specific for USB display, is very similar to USB VIDEO class, by report the usb descriptors to host side driver, host will create the graph by parse them.

My question is, if a USB display driver which is USB AV class compliant, should it is a combination of WDDM miniport driver and AVSTREAM miniclass driver?

In practical terms, it should be a WDDM driver.

Theoretically, you could make an AVStream driver out of it. In that
case, you could choose it as a renderer in a DirectShow graph. However,
no one expects to do that. People expect to render to a monitor.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

I supports it by getting out of the way and letting the kernel device
operate in single buffer rendering mode.

Mark Roddy

On Fri, Jun 6, 2014 at 11:35 AM, wrote:

> Yes, I was already wondering, why WARP should not work on Win7.
> Actually, I am also wondering about a few more statements above.
> E.g. how could a WDDM User Mode Driver support Non-Aero?
> This is implemented in Kernel Mode only…
>
> But I don’t want to flood this newsgroup with too many WDDM details.
> Can only focus on the most important topics here…
>
> Marcel Ruedinger
> datronicsoft
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

It does not need to get out of the way. It is never even asked nor involved in the first place…
Non-Aero is only dxgkrnl.sys talking to the WDDM kernel mode driver.
Possibly interesting fact:
Win7 Non-Aero even works with a completely dysfunctional WDDM User Mode Driver.

Marcel Ruedinger
datronicsoft

> Code metrics and their relevance to driver development

All of this stuff is about writing the version of 0.9 of some product from scratch by a single person, or a very small team of old friends who are also serious professionals.

When you have version, say, 5.0, you cannot measure this way. In this picture, developers work in “issue closing” mode, and the issue’s cost is not typing the code lines: it is the research on the old code (written by a friend or by yourself 3 years ago) and decision making based on this research.

And, with a large team, you have huge administrative overhead of heavyweight processes/tools and interpersonal communication (meetings, etc).


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

This gets it straight to the point. Actually we can take this even further:
On the way from version 1.0 to version 5.0, I am often trying
to enforce what I call “Continuous Improvement Process”.
While adding new features, also correct code inefficiencies spotted on the way
and try to increase code reuse and coverage…

Very often, this results in a negative value for the metrics “Lines of code per day”
over long periods of time while functionality and stability are increasing.
Of course, there is always a J-Curve effect first, also requiring additional testing…

Why did I learn about the J-Curve effect only in economics?
It applies much more even to computer science…

The above of course only applies to long term development projects with quality aware customers
(yes - this endangered species does still exist :wink:

PS: Probably time to close the thread before it turns into
a “Late night engineering discussion”.
Maybe we can leave it with a final reply count of 42
until the Second Coming of Zarquon.

Marcel Ruedinger
datronicsoft

> On the way from version 1.0 to version 5.0, I am often trying

to enforce what I call “Continuous Improvement Process”.

…and you cannot apply SLOC metric to it, unless you want to get extremely low numbers which show nothing.

The above of course only applies to long term development projects with quality aware customers
(yes - this endangered species does still exist :wink:

Yes, surely.

For “sign acceptance document, get the money and forget” other stuff is usable, like the Agile stuff (invented for this kinds of work like websites and business/accounting automation for small/medium businesses, as also to please the investors on the early stages of the startup).


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com