USB HID question ...

NTDEV
2

Hmm… so much confusion here…
@Xiaofan_Chen could you take a look, please?

Bi-directional communication with a device does not require anything OTG.
The choice of HID or “plain” winusb is yours. Both allow bi-directional communication from software POV.
With HID you have specific API (reports, requests, descriptors etc.; usually small messages, usually one EP besides of the EP0).
With plain winusb you can define any number of EPs including interrupt EPs, and move bulk data with higher efficiency than HID.
No need for pending requests (a-la inverse calls), maybe you confuse this with selective suspend (D2) procedure.

Regards,
– pa

3

A USB device cannot send anything on its own. That is a universal truth. Now, it’s common to have a HID device use an interrupt endpoint (for an input collection, you can choose whether to use EP0 or the interrupt endpoint), but the device cannot send anything to you unless you have queued up a read request. That read request might wait for 4 hours until they send something, but the request still has to come from you.

So, if you really need to get notified of stuff asynchronously, then perhaps you should use an interrupt endpoint. But if you can get by by polling once in a while, and you have that path working, then it’s not clear you gain very much.

1 Like
4

Tim Roberts is the real USB expert here and I think he already answered OP’s question.

We at libusb project recommend HIDAPI for cross platfrom HID devices (USB, Bluetooth, I2C, SPI, etc). It will make communication with HID device easier.
https://github.com/libusb/hidapi

As for Windows drivers for USB function controllers (kind of Windows counterparts for Linux USB Gadget Driver), I think it is not so popular and normal x86/x64 PCs do not support function controller.
https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/developing-windows-drivers-for-usb-function-controllers

1 Like
5

@Tim_Roberts said:
A USB device cannot send anything on its own. That is a universal truth. Now, it’s common to have a HID device use an interrupt endpoint (for an input collection, you can choose whether to use EP0 or the interrupt endpoint), but the device cannot send anything to you unless you have queued up a read request. That read request might wait for 4 hours until they send something, but the request still has to come from you.

So, if you really need to get notified of stuff asynchronously, then perhaps you should use an interrupt endpoint. But if you can get by by polling once in a while, and you have that path working, then it’s not clear you gain very much.

That’s how I had understood things … there are hosts and devices, and hosts talk to devices but not vice versa.

I’ll explain to the firmware fellow that it’s simply not possible in the current ecosystem for me to expose a HID interrupt endpoint on the host side that he can make a report request to and the host will respond back with a HID response when there’s something of note. We can do a bidirectional control EP exchange (what I do now), I can push more information to him in the HID report request but devices are devices, hosts are hosts …

Thx all!

6

Still confusion :frowning:
The host (PC) does not expose endpoints to device, it is the other way. HID devices often expose one IN interrupt endpoint to indicate their data (aka input reports). The host of course has to poll the device endpoints, as Mr. Roberts has reminded.
But the host controller does it in hardware, invisible to the host software.
The host can send a message to device anytime, via EP0, as OUT report or vendor specific request. EP0 is bidirectional.

IMHO the main decision is between HID style API and “raw” winusb, it depends on how much data the device needs to move, how often (i.e. are bulk endpoints needed? Power management needed?)
The HID library wraps some details of communication but has some more overhead (creation of HID device entities on top on USB functions, reading and parsing report descriptors).

Your firmware developer should be able to explain how it works to you, and suggest how to debug and test the communication.
The most important communication occurs among humans and it should work well ))
– pa

7

@Pavel_A said:
The host (PC) does not expose endpoints to device, it is the other way. HID devices often expose one IN interrupt endpoint to indicate their data (aka input reports). The host of course has to poll the device endpoints, as Mr. Roberts has reminded.

Correct, and that’s how I had understood things … but things change so much in the ecosystem that what once was A is now B (for instance, apparently with Thunderbolt3 you can have hot swappable PCIe bus devices [and there’s a CVE tool called ThunderSpy that takes advantage of that] … it used to be you needed a specialized BladeServer to do that and so a PCIe hot swap event was a black swan for folks, now apparently anyone with a $250 StarLink 3T chassis and a Thunderbolt3 MB can hot swap PCIe cards during runtime … That’s great for my Xilinx PCIe FPGA development so I don’t need a disconnect board anymore and don’t have to reboot when I’m developing but also means I have to test my bus drivers against a potential PCIe rebalance event when a TB3 connect/ disconnect happens … )

But the host controller does it in hardware, invisible to the host software.
The host can send a message to device anytime, via EP0, as OUT report or vendor specific request. EP0 is bidirectional.

Correct also, and that’s a design pattern I like to use for USB traffic … the business logic is in the various EP’s, the host/ firmware control logic is in the EP0 communications … EP0 is designed for that, is required by spec and is always there …

IMHO the main decision is between HID style API and “raw” winusb, it depends on how much data the device needs to move, how often (i.e. are bulk endpoints needed? Power management needed?)

There’s actually more to it than that … quite a few of my clients are small medical device companies or groups inside of larger corporations that want to make a POC to show to higher ups for funding … in both cases they don’t have the time and resources to get certified for a signature, and putting the machine in test mode isn’t an option so what I will try to do if at all possible is leverage the WinUSB functionality (with WCID descriptors) to talk to the firmware and lib-usb to handle things in usermode; it’s clean, doesn’t require a driver and works. If I can’t get that to work then I’ll write a UMDF v2 driver to talk to the firmware, it’s only for high security USB applications that I will have to write a KMDF USB driver and they know what they are signing up for already … but again, I will really try to handle stuff using WinUSB and lib-usb.

For a HID device it’s actually a bit annoying, normally you send the report request on the other side of the INT endpoint but in the MS USB HID driver MS will use the other side of the INT endpoint for a report request but will also use the EP0 EP in a WriteFile() command to send a report request to the INT endpoint, which goofs up my bidirectional EP0 control pattern … unless I can use WinUSB as the OS driver. That forces me to use WCID to make a composite device for WInUSB to recognize the device, and I think the firmware fellow got annoyed by that and just wanted to only make a pure HID device in firmware, not monitor EP0 (the base firmware hides the EP0 control/ setup stuff) and wanted a bidirectional HID device as an overloaded C&C interface [which I also really avoid, because I don’t like to write sporks] … which as I mentioned I had never heard of before, but before I opened my mouth and proved that I didn’t know what I was talking about I wanted to ask some of the erudite experts here :smile: )

The HID library wraps some details of communication but has some more overhead (creation of HID device entities on top on USB functions, reading and parsing report descriptors).

The lib-usb library is definitely good but you have to be careful about what lower driver is being loaded by MS for the device; that’s one really nice thing about WCID, I can specify exactly what driver I want the OS to load …

Your firmware developer should be able to explain how it works to you, and suggest how to debug and test the communication.
The most important communication occurs among humans and it should work well ))

Yep, and the inter-human communication is usually the one that has the most errors … :smiley: Thanks for your comments and thoughts!

8

For pure HID devices, MS Windows HID APIs have some limitations, but it should be able to do the job without adding a WinUSB interface. I can understand the FW developer’s unhappiness to deal with both HID and WinUSB.

You can use the Feature IN/OUT Report to do what you do with EP0 IN/OUT Control transfer. The Output report is done using control transfer (EP0 OUT) as well. Only the Input report is a bit special, it can be done with EP0 IN control transfer, or with Interrupt transfer with another Input Endpoint.

HID Spec:
https://www.usb.org/document-library/device-class-definition-hid-111

HIDAPI: this will probably simplify the communications with HID device
https://github.com/libusb/hidapi

Before WCID (MS OS Descriptors for USB Devices) becomes popular, HID device is commonly used (you can say mis-used) because of the Windows built-in driver.
https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/microsoft-defined-usb-descriptors

1 Like
9

@Xiaofan_Chen said:
For pure HID devices, MS Windows HID APIs have some limitations, but it should be able to do the job without adding a WinUSB interface. I can understand the FW developer’s unhappiness to deal with both HID and WinUSB.

You can use the Feature IN/OUT Report to do what you do with EP0 IN/OUT Control transfer. The Output report is done using control transfer (EP0 OUT) as well. Only the Input report is a bit special, it can be done with EP0 IN control transfer, or with Interrupt transfer with another Input Endpoint.

You’re quite correct, the USB HID Feature report is bidirectional and ideal for just this kind of exchange between a device (like a joystick) and the host [https://www.tracesystemsinc.com/USB_Tutorials_web/USB/B1_USB_Classes/Books/A3_A_Closer_Look_at_HID_Class/slide10.htm] and for simple command/ responses it would be a good fit, there are two issues I would face with this …

… first, I use EP0 for more complex traffic like firmware updates (there’s a thread earlier about this very topic, actually) with a home grown version of the DFU spec (few vendors implement it today and it doesn’t allow for signed binaries, which makes it useless) and for retrieving device logs (which is a requirement for medical devices, even for HID ones) … this requires a device with enough bandwidth to push 2-3MB firmware files and 1MB logs. The HID report request is 64b, as is the return report, and these have a maximum speed of 1ms per request; in the “real world” with report structure overhead and such it’s 48b usable and about 1.2ms round trip. That means I can transfer at most 39K per second, which means the 3MB firmware file would take about a minute and a half to transfer and the log file about 45sec to transfer, and …

… the second issue is that the HID device is only going to enumerate, well, when it’s a working HID device. If it’s in an unconfigured state (which it would be unprogrammed from the factory or if the device has gone into “safe” mode) then it’s not going to enumerate as a HID device, it will only show as something with a known VID/ PID presenting a WCID descriptor that WinUSB and handle and able to respond to EP0. Once the driver has retrieved the logs and determined why it’s in “safe” mode, or is ready to program the device the driver will use EP0 to program the firmware, the device renumerates and now it will show up as a HID (or Audio, or Video, or MSD, or whatever is the flavor of the month) device …

Great suggestion though, using Feature requests is definitely a good way to exchange info with a HID device!

Before WCID (MS OS Descriptors for USB Devices) becomes popular, HID device is commonly used (you can say mis-used) because of the Windows built-in driver.
https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/microsoft-defined-usb-descriptors

I’m really a big fan of WCID descriptors and the Extended Interface GUID, now if only more firmware vendors get inline with them and maybe then MS can make DFU into something actually usable! :smile:

10

I use EP0 for more complex traffic like firmware updates … to push 2-3MB firmware files and 1MB logs.
Oh no… This is precisely what Xiaofan_Chen called a misuse.
Hope patients won’t die waiting while the device logs are moved over the EP0. Consider upgrading your firmware guy.

11

@Pavel_A said:

I use EP0 for more complex traffic like firmware updates … to push 2-3MB firmware files and 1MB logs.
Oh no… This is precisely what Xiaofan_Chen called a misuse.
Hope patients won’t die waiting while the device logs are moved over the EP0. Consider upgrading your firmware guy.

@Xiaofan_Chen was talking about a misuse of the HID device, specifically overloading the report request and description return and the Feature capability …

Endpoint 0, or the control endpoint, exists completely independent of what device is ultimately described in the descriptor [https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/usb-endpoints-and-their-pipes] and has nothing to do with HID, or CDC, or MSD, or Video, or Audio, or Composite or whatever … it is entirely designed specifically for bidirectional data traffic. It runs at the same speed as any other endpoint, if you’re using the WinUSB/ lib-usb framework can transfer 4K per packet as rapidly as the protocol will allow, operates at the same priority as INT traffic and if you’re going custom you can send up to 64K per transaction group … 2-3MB firmware on a USB 2 high speed link completes in about a second, a 1MB log in about 400ms and on USB3 will be done in under 50ms …

… pretty sure there’s no risk of patient death there … :smiley:

12

Indeed it is perfectly fine to use Control Transfer. DFU (Device FW Update Class) uses Control Transfer (control pipe).
https://www.usb.org/sites/default/files/DFU_1.1.pdf

1 Like
13

I agree entirely. Poor default control endpoint zero…. He’s typically badly underused, in favor of adding other EPS, or all manner of more elaborate schemes (additional Configs, for example). Using it for your own DFU is a very good idea.

Peter

14

Yes, as long as you recognize the limitations. The control endpoint is low priority on the scheduling list, and because every control endpoint transaction is bidirectional (the only pipe type to do so), the throughput is awful.

15

Thinking about this more… ISTR that Control transfers also do not do next-period retry… so they can fail. Do I remember that right? Kind of a downside…

Peter

16

@Tim_Roberts , @“Peter_Viscarola_(OSR)” … all tools have limitations, the trick is to recognize what you’re using the tool for and what it’s limitations are.

A framing nailer is great for driving nails, but it requires an air compressor to run and will split thinner or smaller pieces of wood … so you don’t use it if you’re going to be moving around a lot or putting up crown molding. A fly rod is great for catching jumping fish like trout, but doesn’t have a lot of strength or can handle a lot of line so you won’t use it for fishing for marlin or for ice fishing …

The control endpoint [https://www.beyondlogic.org/usbnutshell/usb4.shtml], as @“Peter_Viscarola_(OSR)” said, is the unappreciated sibling of the USB pipe complex and as @Tim_Roberts said has bandwidth limitations (not sure about the priority, I’ve never seen it delayed but when it’s in use point to point there isn’t anything else on once it hits a hub all USB priorities get goofed up to some extent but I’ll take his word for it, low priority)

If I want high throughput, if I want low latency, if I want unidirectional streaming then there are better solutions to use than a control endpoint. What the control endpoint excels at, however, are [https://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/usb-control-transfer]:

  • It’s always there … I don’t have to worry that the firmware got some interface right, that some firmware even implemented something, by definition there will always, without question, be an EP0 there to talk to. If for some reason it’s not there then the PHY layer will ensure that I never know about it and the user has essentially plugged in a cable …
  • It’s bidirectional, with built in send/ response logic; I can always receive data that the firmware is posting, and I can always canoodle the firmware engineer to start responding to the EP0 endpoint rather than just letting it go the the default IP … nothing much to set up, just a callback
  • It’s got the bandwidth I need … packet sizes are 64b for USB2, 512b with USB3, typically I’m more limited by the FIFO depth on the firmware than any protocol size constraints so I’ll use 64K transfer blocks and everyone is happy

I’m still working on getting my home brew DFU implementation migrated over to follow the DFU 1.1 spec so that someday if one of the chip vendors I work with (Cypress/Infineon, MicroChip, nRF or STM) also implements DFU 1.1 I’ll be a happy camper, until then I simply have a block of firmware source for each chip that I support that I give to the firmware fellow and say “compile and link this” … that has all of my EP0 C&C and DFU business logic in there so it’s usually an easy sell to get into the device

This particular firmware developer though was absolutely convinced that a) a host could have a HID endpoint (device to host) and b) it would be easier to use HID Feature requests to shuffle data back and forth rather than my custom EP0 code. The second point was easy to address once I pointed out how often I could send the Feature request under the HID spec and how much data it could actually carry each time versus what we need, the first needed me to confirm with the experts here that it just wasn’t possible in the ecosystem that the host has to live in …

He’s now begrudgingly adding my sources to the project and progress continues on the task at hand … :slight_smile:

17

@“Peter_Viscarola_(OSR)” Yes, control transfers retry. Isochronous pipes are the only ones that don’t.

18

He’s now begrudgingly adding my sources
My sympathies (as i’m now a 50% firmware guy myself)

STM in their documentation says their DFU already is ver. 1.1 (Aug. 2004) Not sure about other vendors.
If something is buggy there, please let them know on their forum.

Regards,
–pa

19

@craig_howard said:

… the second issue is that the HID device is only going to enumerate, well, when it’s a working HID device. If it’s in an unconfigured state (which it would be unprogrammed from the factory or if the device has gone into “safe” mode) then it’s not going to enumerate as a HID device, it will only show as something with a known VID/ PID presenting a WCID descriptor that WinUSB and handle and able to respond to EP0. Once the driver has retrieved the logs and determined why it’s in “safe” mode, or is ready to program the device the driver will use EP0 to program the firmware, the device renumerates and now it will show up as a HID (or Audio, or Video, or MSD, or whatever is the flavor of the month) device …

Days back when we had it done similarly, but one more step to add complexity.
When USB client device was powered up it enumerated as one VID PID subsystem device hw ids. Then device waited a sec (to allow USB host to talk to it if needed and thus change some setting or to update bootloader) and went into boot state and advertised a different PID subsystem for same VID. If nobody talked to it for a short interval again (now allow USB host to get some diagnostics of device or to update firmware) it changed PID subsystem second time to represent running state of device.
I had a custom driver for all these based on WinUSB, no HID no lib-usb were needed. It worked but looked annoying to potential customer as plugging in such device caused multiple re-enumerations and beeps if audio output was enabled on desktop PC

20

Mistakes of youth :slight_smile:

21

Pushing back a bit on the “mistakes of youth” thought, there actually is a real world case (again from medical devices, you would be surprised at what hoops you must pass through for FDA approvals) for this kind of PID bouncing (although definitely not in the way that @rusakov2 describes, no crazy whack a mole PID bouncing!)

With medical devices, and also for devices that have potential for causing human harm like robotic control mechanisms (which need to be approved by OSHA, another big briar patch) you need to be able to guarantee three clearly distinct operating modes: pre-program, normal operating and “safe” mode.

Pre-program means “this thing has no idea what it’s supposed to be doing or doesn’t have the right info for the task at hand” so the only thing that is allowed to be able to communicate with it is a programmer

Normal operating means “this thing knows what it is supposed to do, and is operating within accepted parameters, and is checking those parameters” so it is in a state where it can actually “do” that important something (monitor IV pump status, control the robotic can sorting, launch the missiles, etc.)

“Safe” mode means “something went wrong with this thing” so the only thing that is allowed is to be able to communicate with a diagnostics application. Note that this is not the same as pre-programmed; you have to be able to diagnose the fault first … the pump sensor failed, the puffer video camera failed, the hydrazine pump for the missiles failed

The only real way to accomplish these items are by having the device firmware change PID’s, one PID for each state, which will ensure that only the appropriate client can access the device … that’s three device drivers in days past (as @rusakov2 mentioned) and either three firmware images within the device or in the case of programmable device firmware three firmware loads.

These days I can accomplish this by having a single driver contain the firmware for each mode, and monitor for each PID; one driver, three operating modes. When the device is in the pre-program state the driver can push the firmware for the programmer application based on the PID, when the device resets and uses the normal mode PID the driver again loads up the firmware for the operating client, when the device goes into safe mode again same thing, load up the safe mode firmware … that’s one device driver, it just contains three firmware images

Why contain the images and not just read from files? Security, you need to be able to verify the firmware can never change. Even if you were to sign the firmware there is still the danger of an “old” buggy (signed) firmware file being copied over the “new” one or the firmware simply being deleted so by encapsulating the firmware into the driver that checkbox is addressed …