Benchmarking WinUSB, libusbk and Cypress cyusb driver

Just FYI on the performance benchmarking using three different generic USB drivers.

HW: Cypress CYUSB3KIT-003 EZ-USB® FX3™ SuperSpeed Explorer Kit
https://www.cypress.com/documentation/development-kitsboards/cyusb3kit-003-ez-usb-fx3-superspeed-explorer-kit

FW:
https://github.com/mcuee/libusbk/tree/master/BmFW/CYPRESS_FX3

Host SW to compare libusbk vs WinUSB:
https://github.com/mcuee/libusbk/tree/master/libusbK/src/kBench

Cypress data: from AN86947
https://www.cypress.com/file/125281/download

From the data, libusbk and WinUSB are pretty close. But WinUSB is a
bit picky for the host when it comes to SuperSpeed isoc transfer compared
to libusbk, based on my testing. Cypress driver may still have an edge
in this aspect.

Info about libusbk project: open source USB driver and library. The
library supports WinUSB, libusbk kernel driver and libusb-win32
kernel driver.

Best regards,
Xiaofan

Benchmark results of libusbk vs WinUSB are here. They are more or less the same.
https://github.com/mcuee/libusbk/tree/master/BmFW/CYPRESS_FX3

I was investigating why cyusb3.sys is faster than libusbk and WinUSB for bulk transfer (IN or OUT), and one of the theory from Travis Robinson (the developer of libusbk) is that libusbk/WinUSB are based on KMDF but Cypress cyusb3.sys is most likely based on WDM. To check that theory, I use libusb-win32 device driver (libusb0.sys) which is also based on WDM, indeed I got the same result as cyusb3.sys.

Is the overhead significant in this case?

Benchmark data: superspeed bulk IN/OUT transfer with the bulksrcsink example from Cypress (1024B packet size, burst length 16, theoretical max speed up to 454,300 KB/sec as per https://www.cypress.com/file/125281/download ).

Real world speed using the same Windows 10 computer and same Cypress BulkSrcSink FW.
437,800 KB/sec for cyusb3.sys using Cypress Cystream application
437,700 KB/sec for libusb0.sys using libusbk kBench application
386,600 KB/sec for libusbk.sys using libusbk kBench application
385,300 KB/sec for WinUSB using libusbk kBench application

Is the KMDF overhead over WDM so significant that this is the root cause of the slower speed?

Thank you for this whole thread, Mr. @Xiaofan_Chen…. But especially for this WDF vs WDM comparison.

The results are extremely interesting, and not what I would have expected. There’s nothing inherent in KMDF that would make it significantly slower.

It IS true that there are a lot of design and implementation choices in KMDF that (in some cases) allow a dev to trade-off ease of implementation for speed. This is particularly true with USB…. There are many “convenience” functions that will take care of things for you and make life simple (right now to allocating a new Request, formatting it, sending it, and waiting for it to return). They’re easy, and fun to use… but not necessarily efficient.

In WDM there are few such temptations, right? There’s IRP and URBs and IoCallDriver and it’s up to you to plumb everything together.

I suspect the perf diff is a matter of design choices, and not a matter of inherent required overhead, that you’re seeing in these drivers.

That’s my guess, anyways…

Peter

The above is my test results using a 9th Core i7 Dell Laptop with 16GB RAM.

With a faster desktop PC with 64GB RAM, Travis got 426,088 KB/sec with libusbk and similar results with WinUSB. libusb-win32 is still a bit faster at 440,504 KB/sec.

So it seems to me whatever the design choices are, the gap can be bridged with a faster PC.

On the other hand, it was difficult to maintain the code base of libusb-win32 to add better isochronous transfer support, and that is why Travis chose to use KMDF to create libusbk kernel driver since WinUSB did not support isochronous transfer at that time for Windows XP and Windows7. Now that WinUSB isochronous transfer has been working nicely in Windows 8.1 and Windows 10, we have shifted the focus to WinUSB support.

We have just released libusbk 3.1.0.0 which adds support of isochronous support for WinUSB in libusbk API.
https://github.com/mcuee/libusbk/releases

KMDF is definitely the right driver model to use. When performance is a significant goal, one must take care to keep that in mind when choosing features to use through the implementation. This is true regardless of the driver model, of course… but it’s doubly so in the case of KMDF in general and KMDF USB where there are so many tempting features that can cost significant overhead.

While it certainly adds some, in general KMDF doesn’t add what I consider a LOT of overhead, over and above a similarly featured WDM driver. Much of what KMDF does are the same things you’d need to do “on your own” in your WDM driver. There’s not a lot of gratuitous nonsense in the KMDF model or Framework implementation… though the balance is generally in favor of robustness, ease of use, and debug-ability over raw speed.

Peter

1 Like

Peter is definitely correct on this. I don’t have the experience with USB, but developed a driver for a firm supply hardware to Microsoft Research a number of years ago. The goal was one million messages per second over a custom communication device. With the first stock KMDF driver we were under 150K/sec but by tuning the critical paths we hit the 1M/sec goals. KMDF takes care of a lot of the basics for you which I have found in multiple drivers where performance was critical the basics are 80% to 90% of the code. KMDF does allow the flexibility for to create what you need for critical code paths.

Thanks. Yes I totally agree that KMDF is the way to go.

And I just checked, Cypress cyusb3 driver is also based on KMDF, unlike the old cyusb.sys.
http://www.cypress.com/file/316186/download (cyusb3.sys is based on WDF)
https://www.cypress.com/comment/297641 (older cyusb.sys is based on WDM).