Should I write a driver to disregard the exposed polling interval of a non-HID vendor device ?

Hmmmm… probably not possible. The “polling rate” reflects the max latency the device is willing to tolerate, and using the the Host Controller Driver (HCD) builds his schedule. This is a private transaction between the HCD and the device.

The enumerator for USB is the USB Hub Driver. So, it MAY be that you could interpose yourself between the HCD and the hub driver during device discovery and change the contents of the endpoint descriptor before it’s returned. It’s complicated, because it’s the HCD himself that you want to fool… not the client driver for the device.

Somebody here may have tried something similar… let’s see if they comment.

Peter

On Apr 7, 2019, at 1:59 PM, Arte wrote:
>
> The surprising bit is that it exposes a Interval of 8, which in the context of Full Speed devices, means it asks for an 8ms polling interval.
> …
> This is surprising because on the other end of this conversion device, the device polls periodically every ~1,4ms, and the data sent to the computer only provides the (post-conversion) data of the last of these ~1,4ms period polls. So I strongly suspect that new data is available in the device-side USB controller every ~1,4ms, and that the 8ms interval exposed in the descriptor is outright a mistake.

Is this a vendor you’re friendly with? They ought to be able to provide one with a different descriptor that sets the interval to 2, unless there is some specific reason they have chosen 8ms. Is there any useful information in those intermediate reads?

> While I am not certain of it, I would very much like to check and obtain all this data. If this is not possible, the next best thing I’m looking for is to be able to choose precisely the USB frame I poll that device at. So, an 8ms interval some times, a 10ms some other time…

That much is not possible. The host controller driver does all of the scheduling based on the descriptor, and it is nothing if not regular.

> Because this doesn’t appear to be possible with WinUSB.

This is not possible with USB, period.

> WinUSB respects the polling interval requested by the device and will block calls and the emission of the request to at least 8ms after the previous request.

No, none of that is under the control of WinUSB. The application just makes a call to WinUsb_ReadPipe. WinUSB submits an IRP. As long as the host controller driver has an empty IRP waiting, it will send an IN token to the device during its timing interval. The timing is all handled by the host controller driver, which schedules every frame in advance.

> And not only does it respect it, but it appears from my tests, both through libusb and directly using WinUSB functions, that WinUSB schedules an interval timer to check, every 8ms, if there’s a new request for data from this device, and proceeds to fire a request if there is.

No. USB is a completely host-driven bus. As long as there is an empty IRP waiting, the host controller driver will schedule a transfer. It’s up to the device to decide whether it has anything to send when it gets that token. If it does, it sends. If not, it send a NAK, and won’t get another chance until the next interval. If there is no waiting IRP, then the host controller won’t send a token, and you miss your slot.

> The consequence of this is that accurately timed WinUSB function calls will be postponed to the next “8ms-period poll”. As an example, if I attempt to read a pipe every 10ms very accurately, the time difference between two calls will be 75% of the time 8ms, and 25% of the time 16ms, in a cycle of [8ms - 8ms - 8ms - 16ms].

That’s a side effect of the host controller scheduling. With an interval of 8, your driver will be assigned to one particular frame in every 8, forever.

> My current beliefs are that this is due to WinUSB’s implementation and not the USB host scheduler,

You are incorrect. The timing is strictly under the control of the host controller driver.

> …if I were to write my own driver, I would probably be able to control the polling rate regardless of the exposed interval, so, up to 1000Hz, and that if I can’t, I would at least be able to schedule transfers on the frames I want as long as they’re at least 8 frames apart.

Sorry, no. The only way to change the scheduling behavior is to change the descriptor.

> So my questions are the following:
>
> * Is it possible ?

No. There are SOME descriptor problems you can fix by using a filter driver to intercept and rewrite the descriptors, but in this case, the descriptor is being read and used by the host controller driver, which is in direct communication with the hardware. There is no slot into which you can insert a filter.

> * Are my assumptions about the role of the driver in the transfer scheduling process correct ? If so, does it mean that writing my own driver will work ?

No, sorry. You must alter the descriptors.

> * Is there any kind of middle-man deception possible where I’d convince the OS that the exposed interval is not actually 8, but 1, without involving an external device ?

No, sorry.

> When @Maxim_S._Shatskih says in the first answer to https://community.osr.com/discussion/190849/usb-polling-rate that “For interrupt pipe, the polling rate it is hardcoded in the config descriptor to be BusPollingRate/( 1 << N ), where N is some small number (1 and up).” - does that mean it’s impossible ? This is the message that had me seek confirmation from experts before spending weeks or more attempting to do the impossible.

Max is correct. What you are attempting is impossible.

Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

Added more support where none is needed: Mr. Roberts is, of course, correct. Once being Informed of the arrival of the device, the HCD talks directly to device. No way of “getting in there” and changing the data.

Do you want to do this a limited number of device units for in-house use only? If so, it MAY be possible to re-program the endpoint descriptor on the hardware. Most are stored in EEPROM and can be changed… this is the stuff of which USB hacking/exploits are made.

Peter

Thanks a lot for your answers, you’ve saved me weeks of pointless struggle, and taught me much.

I have another device that could do the work, and is HID. Using it isn’t as clean as using the device initially at hand, but it still sounds like a possibility to me so far. Actually, the devices I am working with are controller adapters. I hardly care about the latency/responsiveness, I care about the fact that feeding data at 125Hz into a state-machine-like engine that updates at 60/120Hz creates discreet but very big problems, and I’m trying to fix these globally. The HID adapter polls every ~1,4ms as well but converts the data to a HID Controller format. I’ve yet to determine what the benefits of using a dedicated format are compared to using HID conversion, but that’s not the subject here.

This device also has an 8ms interval declared, however I am confused about whether HID obeys to the same rules. Considering this is a driver-level change and you’ve made it very clear intervals were enforced at HCD level, it shouldn’t solve the problem.
So what does HidUsbF do ? In https://community.osr.com/discussion/190849/usb-polling-rate , Tim, are you talking about HidUsbF ? Does that mean Cay Bremer’s answer is incorrect ? He mentions that it “allows users to customize the polling interval of USB mice and other HIDs” and “consists primarily of a lower-level filter driver to HidUsb”.

As for re-programming the endpoint descriptor on the hardware directly, it would be a partial victory, as the initial goal was to fix these problems for everyone, but a victory nonetheless. I had actually considered it and began looking elsewhere before the complexity of the task (and not having the tools), hoping to find a software-only way. I will resume looking this way if HID conversion doesn’t work out - and it doesn’t look like it’s going to. Looking at the device’s board (the original device), the only microcontroller in sight is a ST 32bits (is inscribed: 32LP151 C6NTO GH21E VG CHN 521). I don’t know if the security bit is set. But before asking any question on that subject, I’ll look more into how vendor-provided chip reprogramming works.

Arte wrote:

I’ve yet to determine what the benefits of using a dedicated format are compared to using HID conversion, but that’s not the subject here.

The HID format can be access from user-mode without writing a driver
package, but it requires you to shoehorn your data into HID-acceptable
packets.  With a custom interface using WinUSB, it is POSSIBLE to go
without a driver package, if you add Microsoft OS-specific descriptors
to give your device a GUID, but you gain more freedom in how the device
communicates.

This device also has an 8ms interval declared, however I am confused about whether HID obeys to the same rules. Considering this is a driver-level change and you’ve made it very clear intervals were enforced at HCD level, it shouldn’t solve the problem.

The device class has zero impact at the wire level.  All it does is
determine which driver to load.  An interrupt endpoint in a HID
interface is bound by the same rules as an interrupt endpoint in a
vendor-defined class.

So what does HidUsbF do ? In https://community.osr.com/discussion/190849/usb-polling-rate , Tim, are you talking about HidUsbF ? Does that mean Cay Bremer’s answer is incorrect ? He mentions that it “allows users to customize the polling interval of USB mice and other HIDs” and “consists primarily of a lower-level filter driver to HidUsb”.

My answer explains that.  They don’t change the polling interval
(because they can’t).  The standard HID driver does reads with a large
buffer, so it can accumulate several packets before returning.  That
reduces overhead, but increases latency.  The hack mentioned in that
thread chops that larger buffer into a set of smaller reads.  The
individual reads still go out on the wire at the descriptor-specified rate.

With all due respect to these who answered…
The endpoint descriptors are read by the host USB stack, and then the HCD is programmed by the host stack (software).
The host controller does not interpret the descriptors by itself.
So yes, if you can intercept the EP descriptor and change the interval, it can help verify your theory that the device can provide more data if polled faster.

But it isn’t quick and easy for beginner. So it depends on your motivation.

There’s at least one opensource project related to libusb which provides a hub filter driver; it allows to modify descriptors read from the device, on their way up. Similar to what VirtualBox and VMware do to hide USB devices from the host and redirect them into VMs.
This means to deal with a kernel driver.

Next option : Hack the OS USB drivers at some layer to force desired polling interval.
The OS won’t be Windows, of course. But you have the sources.
It can be faster than previous option, if you are familiar with that OS.

Next option. You’ve mentioned the ST microcontroller - it is likely STM32L151. Yes, if the “security bit” isn’t set (TL;DR), you can try to find and patch the USB descriptors in its firmware.
Again, this is not a task for beginner.

Regards,
– pa

On Apr 8, 2019, at 4:40 PM, Pavel_A wrote:
>
> With all due respect to these who answered…
>
> The endpoint descriptors are read by the host USB stack, and then the HCD is programmed by the host stack (software).
>
> The host controller does not interpret the descriptors by itself.

“HCD” means “host controller driver”, not “host controller”. I’m very careful to distinguish the two.

Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

Does that mean Cay Bremer’s answer is incorrect ?
His answer is correct, except the “of USB mice and other HIDs”. hidusbf always and setup.exe now can handle any device with Interrupt Endpoints. Read about the /all switch of setup.exe in readme

considering that the device isn’t HID. (so, no HIDUSBF stuff)
See above :wink:

Mr. Roberts, in https://www.winvistatips.com/threads/modify-usb-device-descriptor-by-filter-driver.820278/ you answered to

Since device descriptor (firmed in chip) can’t be changed, I want to modify
it in a lower filter driver before host controller driver gets my devcie
descriptor. Can this lower filter driver achieve this goal?
with:
No. The host controller driver is the bottom-most driver in the USB stack.
It performs transfers through DMA. It doesn’t send any requests down for
you to filter, because there IS no driver below it.

and said in https://community.osr.com/discussion/190849/usb-polling-rate that :

[…]there was a custom driver for Windows XP
that increased the “USB polling rate,” which magically made the games
more responsive. Maybe you even used it. That driver doesn’t work on
Vista and beyond, so you want to write one. Is that right?
What that driver did was replace the standard USB HID driver with one
that submitted smaller buffers. It didn’t poll any faster (it can’t do
so – that’s determined by the hardware), but by submitting smaller
buffers more often, it was able to get results back with lower latency.

But Cay Bremer claims, supported by SweetLow, that

It consists primarily of a lower-level filter driver to HidUsb that
listens for IOCTL_INTERNAL_USB_SUBMIT_URB requests, and when an URB of
URB_FUNCTION_SELECT_CONFIGURATION arrives, sets the member bInterval of
the first endpoint descriptor of the first interface descriptor of the
configuration descriptor to a registry-supplied value. […]
Since Windows does not allow a polling interval below 8 ms for low-speed
USB devices, the driver furthermore attempts to remove this restriction by
patching the responsible driver usbport.sys in memory.

If I cross this information with Pavel_A’s

The endpoint descriptors are read by the host USB stack, and then the HCD is programmed by the host stack (software).
The host controller does not interpret the descriptors by itself.
So yes, if you can intercept the EP descriptor and change the interval, it can help verify your theory that the device can provide more data if polled faster.

Are you referring to HidUsbF when mentionning the driver that slices buffers ? If not, am I understand from your responses that:

  • The HCD schedules transfer directly, with one change per interval, interval that’s defined by the endpoint descriptor initially received by the HCD.

" the descriptor is being read and used by the host controller driver, which is in direct communication with the hardware. There is no slot into which you can insert a filter."

  • There is no way to insert a filter driver “before” the HCD, as in, that alters communication between the device and the HCD.
  • However, the HCD is initially configured by the software, in particular, by the driver. By writing a lower-level filter driver that intervenes before “the” driver (HidUsb in the legacy HidUsbF case, but not only anymore), you can mess with the Endpoint descriptor and get the HCD to receive altered data at initialization - which is the only shot at modifying its behaviour. Which is what HidUsbF does.

But then, why do we need a filter driver located before HidUsb instead of an alternative version of HidUsb directly ? Does it have any more rights ? Task difficulty apart, is there any reason a HidUsb clone that does the job of modifying the data it’s going to configure the HCD with itself wouldn’t work ?

As for the buffer slicing driver, I don’t get what “What that driver did was replace the standard USB HID driver with one that submitted smaller buffers.” means. What does submitting too large a buffer does ? Does you internally (at driver level) wait for enough reads to fill the buffer ? So for example, a 1000Hz-interval mice would read 8 times and then play it out by applying the result of the 8 reads in order over the next 8ms, and that driver would force the (naively expected ?) behaviour of the mice polling once a millisecond and the result being applied a.s.a.p ?

@SweetLow, if you indeed use a lower level filter driver and not buffer slicing tricks, that means the mouses you overclock above 1000Hz aren’t only mouses whose endpoint descriptor intervals were strictly below 8 to begin with, right ? (8 being 1000Hz for HighSpeed)

“HCD” means “host controller driver”, not “host controller”. I’m very careful to distinguish the two.
Mr. Roberts, I stand corrected, you’re right as typically you are.
What I tried to say, the OP has number of quick [and dirty] options to resolve his doubt - that the device can send data at higher rate.
These require either skills, or other way that we’re not allowed to discuss here :wink:
The host controller or OS should not be a limiting factor for this task. It can be replaced.

– pa

if you indeed use a lower level filter driver and not buffer slicing tricks, that means the mouses you overclock above 1000Hz aren’t only mouses whose endpoint descriptor intervals were strictly below 8 to begin with, right ? (8 being 1000Hz for HighSpeed)

[Unlike ordinary] overclocking over 1000Hz is some kind of hack. I use filter just to ask the lowest possible polling rates but instead of setting the lowest rates i change HCD code to set the highest rates when asked lowest rates. And no, i don’t fool HCD to “think” the device is High or Super speed, i just poll Full speed device with higher than standard rate.

I don’t understand most of this thread… and I’ve written a USB host controller driver, hub driver, and drivers for several USB client devices. Perhaps I am missing something very clever.

I’m not sure if people I this thread (I’m including you Mr. SweetLow) are talking in vague nonsensical terms to be clever, or if their command of English is just insufficient to allow them to describe what their meaning recisely and clearly.

But, regardless: there’s some pretty stupid shit in this thread.

Fact: The Host Controller Driver is responsible for building the schedule by which it polls devices.

Fact: It builds this schedule based on the content of device’s endpoint descriptors.

Fact: You can’t change that Host Controller Drivers schedule without either (a) changing the endpoint descriptors presented by the device to the host controller driver, or (b) changing the host controller driver itself to build the schedule differently.

Get a USB hardware analyzer and watch the device initializatuin sequence on the bus. It’s not rocket science.

That’s the way I understand USB works. If somebody would like to educate me on some other way(s) to change the actual polling rate on the bus, I’ll be most happy to hear it and I’ll thank them for their efforts.

Lacking that, let us not randomly spew nonsense and confuse the OP further, OK?

Peter

What does submitting too large a buffer does ?
From a software perspective, interrupt and bulk pipes are absolutely identical. You can see that they even use the same URB request code. In both cases, single “transfer” can span multiple packets.

So, let’s say my endpoint has a max packet size of 8 bytes (as mice often do), and I submit an URB to read 64 bytes. The “transfer” is not complete until either the buffer is full or the device returns a short packet. As long as the device keeps returning 8-byte packets, the request will remain pending until the devices sends 8 packets and the buffer fills.

The “optimization” discussed in those old messages was to intercept that 64-byte read and submit it as 8 separate 8-byte reads. The device doesn’t send any faster, but the driver would see the results faster.

Mr. @SweetLow is arguing that the host controller driver configures itself based on the configuration descriptor passed down in the URB_SELECT_CONFIGURATION request, and not on the configuration descriptor received from the hardware. I have not heard this claim before, and I am skeptical. If true, that does represent a way to work around odd device descriptor issues.

(a) changing the endpoint descriptors presented by the device to the host controller driver, or (b) changing the host controller driver itself to build the schedule differently.
That is exact what my driver do and intended for. Both (a) and (b) parts. Once again - Cay Bremer description is correct, my driver can do what Arte asked in the first post - make 2 or 1ms polling on device with 8ms bInterval in Interrupt Endpoint Descriptor. May be he asked something else - but i understand his question exactly this way.

That’s the way I understand USB works.
You understand right. But i don’t understand why you think that some driver can not do (a) and (b).

Lacking that, let us not randomly spew nonsense and confuse the OP further, OK?
Looks like OP lost interest in topic. But we can discuss his question here or in any other place.

@Tim_Roberts

If true, that does represent a way to work around odd device descriptor issues.
It’s very easy to test (take usbview.exe - and compare descriptor and pipe bInterval values). And was checked by hardware even:
https://www.overclock.net/forum/375-mice/1589644-usb-mouse-hard-overclocking-2000-hz-3.html#post24849487

some driver can not do (a) and (b)

We are clearly not understanding each other, and that’s why I’m confused. I assume there’s a language problem involved, so I apologize for only being being able to speak English. I mean that.

In order to do (b) you need the source code for the host controller driver. I suspect you do not have source code for the Windows USB Host Controller Driver. This is part of my confusion.

Peter

In order to do (b) you need the source code for the host controller driver.

What he said earlier in the thread is that they hot-patched usbport.sys on the fly. Does that make your skin crawl?

What he said earlier in the thread is that they hot-patched usbport.sys on the fly

You know, I must be getting stupid in my old age… but I can’t find that anywhere in this thread. I actually looked back, again, right now. I don’t see it. At least not in any way that I can understand it.

AH! OK… I see it’s in the other thread and sort-of quoted here.

Does that make your skin crawl?

It makes me positively nauseous.

I would re-do the descriptors on the device before I did that. Bring on the reflow station!

P

I certainly haven’t lost interest and won’t any time soon, I just like testing and researching based on the answers given before reacting, and I get little time to do that during work days.

So, addressing one point in particular, the too large sized buffers : I attempted to send to my device (the WinUsb one) buffer sizes twice too large than the actual transfer. Let’s say that’s the MaximumPacketSize is 50, the actual buffer size sent from the device is 50, and I’m sending a buffer of size 100 (or more generally, 50*n)

Must I conclude from the fact that pipe reads take twice as long, so, 16ms (or more generally, n times as long; 8*n ms) that internally, the device only feeds his USB device controller data every 8ms, in which case my dreams are gone (and the level of bad design in this device is astonishing), or did WinUsb broke down my pipe read attempt because I was violating the maximum packet size ?

I’m afraid the former is more likely, but eh, I still believe.

I just like testing and researching based on the answers given before reacting

Bravo. +5 points for Mr. Arte. Pretty soon, you’re well on your way to graduating to Senior Forum Member!

Must I conclude from the fact that pipe reads take twice as long…

You must conclude that the polling interval you expected was, in fact, the polling interval.

You know, you could save yourself a lot of time and guessing if you had a hardware USB Bus Analyzer. I’ve used several, including this one, and while it’s not my favorite, it’s very good… and extremely reasonable in price (just US$400). In the interest of full disclosure, I have no association with or interest in the manufacturer of this device.

Peter