Should I write a driver to disregard the exposed polling interval of a non-HID vendor device ?

I certainly haven’t lost interest and won’t any time soon, I just like testing and researching based on the answers given before reacting, and I get little time to do that during work days.

So, addressing one point in particular, the too large sized buffers : I attempted to send to my device (the WinUsb one) buffer sizes twice too large than the actual transfer. Let’s say that’s the MaximumPacketSize is 50, the actual buffer size sent from the device is 50, and I’m sending a buffer of size 100 (or more generally, 50*n)

Must I conclude from the fact that pipe reads take twice as long, so, 16ms (or more generally, n times as long; 8*n ms) that internally, the device only feeds his USB device controller data every 8ms, in which case my dreams are gone (and the level of bad design in this device is astonishing), or did WinUsb broke down my pipe read attempt because I was violating the maximum packet size ?

I’m afraid the former is more likely, but eh, I still believe.

I just like testing and researching based on the answers given before reacting

Bravo. +5 points for Mr. Arte. Pretty soon, you’re well on your way to graduating to Senior Forum Member!

Must I conclude from the fact that pipe reads take twice as long…

You must conclude that the polling interval you expected was, in fact, the polling interval.

You know, you could save yourself a lot of time and guessing if you had a hardware USB Bus Analyzer. I’ve used several, including this one, and while it’s not my favorite, it’s very good… and extremely reasonable in price (just US$400). In the interest of full disclosure, I have no association with or interest in the manufacturer of this device.

Peter

The internal workings of the device I expected, since it’s an adapter and polls the controllers every 1.4ms ; was that every 1.4ms, it would convert and then push the obtained data to the device’s USB controller, and that data would be available to pull if the OS sent a request. Why else would it poll controllers every 1,4ms ? And then, for some reason, it tells the OS… that it only wants to be asked its state every 8ms.

This is why I’d be surprised to conclude that the USB controller only receives data every 8ms, internally. Can it be so badly made that it’s another buffer than the USB controller that’s filled after the data translation (assuming these 2 parts run asynchronously - which sounds logical to me as the controller polling occurs periodically very precisely, regardless of USB polls - it’s not that farfetched, you wouldn’t want to modify the buffer the USB controller is reading), but that buffer is only copied to the buffer the USB controller reads every 8ms ?

Also, I haven’t looked into physical USB analyzers. I’ll look into what it offers more than software ones (I’ve been using Microsoft Message Analyzer). To be honest the price scares me right away - that’s a big sum for me, I’m only beginning my career.

PS : I was reacting to SweetLow’s “Looks like OP lost interest in topic.”
PS2 : Misjudged the usage of usernames on these boards, had I known I would have used my real name. Not editable.

J. Bernard

Must I conclude from the fact that pipe reads take twice as long, so, 16ms (or more generally, n times as long; 8*n ms) that internally, the device only feeds his USB device controller data every 8ms

Exactly because of this, you don’t want to move large amounts of data over interrupt endpoints. Exactly because of this, they are named “interrupt”.
The purpose of interrupt endpoint is to pass a short message, like “I am ready to send /receive more data”.
Massive data pumping goes over bulk endpoints. As Mr. Roberts noted, interrupt and bulk pipes are absolutely identical -
except that bulk pipes do not have polling interval. The host controller moves the bulk data whenever the bus is free after the scheduled traffic.
HID devices use interrupt pipes for their main data, it’s understandable why.
Reports of normal HID devices should fit in the max. packet size ( < 8 bytes low speed, < 64 bytes full speed, such as your STM32).
Every single typical report is a short packet, and it ends the transfer.

So you’re asking, is this device badly designed? Maybe yes, maybe not.
All we know from you is that it is not HID, and the direction is OUT.
May be it indeed cannot consume data at high speed. You don’t know until you do the experiment (and use the USB analyzer!).
Are you the only customer of this vendor? Can you reach them for help or explanation?

– pa

The internal workings of the device I expected, since it’s an adapter and polls the controllers every 1.4ms ; was that every 1.4ms, it would convert and then push the obtained data to the device’s USB controller, and that data would be available to pull if the OS sent a request. Why else would it poll controllers every 1,4ms ? And then, for some reason, it tells the OS… that it only wants to be asked its state every 8ms.

This is why I’d be surprised to conclude that the USB controller only receives data every 8ms, internally. Can it be so badly made that it’s another buffer than the USB controller that’s filled after the data translation (assuming these 2 parts run asynchronously - which sounds logical to me as the controller polling occurs periodically very precisely, regardless of USB polls - it’s not that farfetched, you wouldn’t want to modify the buffer the USB controller is reading), but that buffer is only copied to the buffer the USB controller reads every 8ms ?

Also, I haven’t looked into physical USB analyzers. I’ll look into what it offers more than software ones (I’ve been using Microsoft Message Analyzer). To be honest the price scares me right away - that’s a big sum for me, I’m only beginning my career.

PS : I was reacting to SweetLow’s “Looks like OP lost interest in topic.”
PS2 : Misjudged the usage of usernames on these boards, had I known, I would have used my real name. Not editable.
J. Bernard

@Peter_Viscarola_(OSR)

It makes me positively nauseous.
Don’t forget - primary target of hidusbf is OVERclocking. And you are right, this is not open source, so I have to do it.

@Arte

I was reacting to SweetLow’s “Looks like OP lost interest in topic.”
In this case, i just ask for confirmation if i correctly understood your initial request:
“make 2 or 1ms polling on device with 8ms bInterval in Interrupt Endpoint Descriptor”

Mr. Bernard, formerly Mr, Arte… your username has been changed at your request. Thank you for using a real name, I appreciate it.

On Apr 11, 2019, at 2:38 PM, Arte wrote:
>
> This is why I’d be surprised to conclude that the USB controller only receives data every 8ms, internally. Can it be so badly made that it’s another buffer than the USB controller that’s filled after the data translation … but that buffer is only copied to the buffer the USB controller reads every 8ms ?

I think you may still be confused about how USB works.

Let’s say you have an endpoint descriptor with a max packet size of 50 bytes and an interval of 8ms. Let’s say you submit a read of 400 bytes. So, the host controller driver is layout the schedule for time 0ms. Since you have a request waiting, it will schedule time for your device. When the schedule is executed, the host controller will send one IN token. That tells your device it is allowed to transfer. It will send a 50-byte packet in return. Because that matches your device’s max packet size, the overall transfer is still not complete, so it remains pending.

Nothing else will happen for the next 7ms. When the host controller driver is laying out the schedule for the frame at 8ms, it will once again put your device on the schedule, and when executed, once again the controller will send exactly one IN token, and in response your device will send another 50-byte packet.

A USB device never gets to decide when to send. It has to wait until it get an invitation (i.e., an IN token), and an interrupt endpoint only gets an IN token ONCE during each interval.

Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

Note: The email was trying to reply to an invalid Comment (293418).

> The purpose of interrupt endpoint is to pass a short message, like "I am

ready to send /receive more data".

I see it a bit differently. It simulates real interrupt i.e. delivery of an asynchronous event (which definition depends on device or protocol designer). The main purpose is to save bandwidth and that’s what is polling interval for. Polling bulk endpoints is continuous, interrupt is polled much less frequently.

Are you the only customer of this vendor? Can you reach them for help or
explanation?

Actually, it should be the first. Ask the vendor why the device is designed this way and if it is a mistake, fix it on the device side. There can be a good reason for it. Or maybe not and the designer just thought 8 ms ought to be enough for anybody :wink:

Michal

Note: The email was trying to reply to an invalid Comment (293419).

but that buffer is only copied to the buffer the USB controller reads every 8ms

Well, it sort of makes sense. The device polls at some “very short” time period… for whatever reason, you know… Nyquist frequency or something… and accumulates some samples and sends those samples at a lower frequency to the host. I don’t think that’s necessarily a bad plan. It’s one of the advantages to having hardware on the “front end”…

Peter

The main purpose is to save bandwidth and that’s what is polling interval for

Not really. I suspect you’re looking at this backwards :wink:

The purpose of the polling interval is to provide an upper bound on latency. Bulk endpoints have no such upper bound, and have do not make bandwidth reservations. Interrupt endpoints do. When you build an Interrupt endpoint, you are assured that your endpoint will be samples at least as frequently as the polling interval. You can be sampled more frequently.

So, the polling interval isn’t a “you can ignore me for this long” thing… it’s really a “you MUST ask me for my data at least this frequently” thing. It was designed to prevent things like mouse skipping (where the pointer jumps instead of moves smoothly) and ensuring keyboard characters echo on key press.

Peter

Note: The email was trying to reply to an invalid Comment (293419).

It was my comment. Sorry for its disappearance, it was not deliberate. I’ve edited it, and the Post Quality Analyzer deemed it not worthy :wink:

– pa

This is why I’d be surprised to conclude that the USB controller only receives data every 8ms, internally.

I want to re-emphasize the point I made in the “reply to invalid message” above.

A USB controller never receives data unprompted. The USB controller is in control. In this case, it ASKS for data every 8ms, and the device responds by sending a packet when it is politely asked to do so. In between invitations, it must be quiet.

Must I conclude from the fact that pipe reads take twice as long, so, 16ms (or more generally, n times as long; 8*n ms) that internally, the device only feeds his USB device controller data every 8ms

Exactly because of this, you don’t want to move large amounts of data over interrupt endpoints. Exactly because of this, they are named “interrupt”.
The purpose of interrupt endpoint is to pass a short message, like “I am ready to send /receive more data”.
Massive data pumping goes over bulk endpoints. As Mr. Roberts noted, interrupt and bulk pipes are absolutely identical -
except that bulk pipes do not have polling interval. The host controller moves the bulk data whenever the bus is free after the scheduled traffic.
HID devices use interrupt pipes for their main data, it’s understandable why.
Reports of normal HID devices should fit in the max. packet size ( < 8 bytes low speed, < 64 bytes full speed, such as your STM32).
Every single typical report is a short packet, and it ends the transfer.

So you’re asking, is this device badly designed? Maybe yes, maybe not.
All we know from you is that it is not HID, and the direction is OUT.
May be it indeed cannot consume data at high speed. You don’t know until you do the experiment (and use the USB analyzer!).
Are you the only customer of this vendor? Can you reach them for help or explanation?

– pa

A USB controller never receives data unprompted.

Nit: Isn’t there a USB 3 protocol situation where this is not strictly true? Something to do with endpoints waking up after being asleep or some such?

Peter

Thank you Mr. Viscarola for restoring my post.
I tried to edit it to add that the original question is in fact very unclear.
It says that the interrupt endpoint is OUT. Which means from host to device. And the OP asks about moving data from the device to host.
What is correct? Using OUT endpoints of interrupt type is unusual.

– pa

On Apr 13, 2019, at 3:22 PM, Pavel_A wrote:
>
> It says that the interrupt endpoint is OUT. Which means from host to device. And the OP asks about moving data from the device to host.
>
> What is correct? Using OUT endpoints of interrupt type is unusual.

I assume he simply misspoke. It’s a very common mistake to make, because the data seems to be going “out” of the device. For those following along at home, the IN and OUT directions in USB are always designated from the host’s point of view.

Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

I misspoke indeed, sorry about that. The device actually has 2 endpoints, an interrupt OUT and an interrupt IN, but only one interests me, and it’s the IN (device->host), not the OUT (host->device).

@Tim_Roberts, I was also unclear by saying “USB controller”. In this case, I meant the USB device controller.

To clarify the architecture of the problem, although I’m not sure how much specifics belong here, the device is a controller adapter (used by numerous people that won’t be able to mess with hardware - and contacting the vendor isn’t an option), which means that there is (in my mind):

  • The controllers (that provide data upon receiving a message using the communication protocol established with the adapter)
  • The main process in the adapter that manages communication with the controllers. (Polls controllers every ~1,4ms)
  • The USB device controller (Exposes a 8ms bInterval, not HID. I am not knowledgeable about how it’s implemented. I’ve seen in another device two chips coexist - one dedicated to USB and one to the main task, but this one appears to only have the STM32L151 chip.)
  • The USB HC + HCD
  • The Windows driver
  • Whatever application is using the data. Said application’s engine is most likely similar to a 60Hz state machine that uses one snapshot of controller data per state increment.

The problem I’m trying to fix in the end is that feeding a 60Hz state machine with an data polled at 62,5Hz (every 8ms) makes things go very wrong. (Think of the “time distortion” that happens when 3 polls, spaced by 8ms, are fed in a single 16,67ms period of the state machine, instead of the usual 2? If we had 1000Hz controller data, we would get time distortions of 16ms or 17ms into 16,67ms - much less harmful than 24ms into 16,67ms. The consequences of this are another subject entirely, though, and I doubt they belong here. In short, it’s terrible on the user end.)

So the problem is the following : is it possible to access the data of the controllers that the adapter chip is pulling every ~1,4ms ?

And if I understood this correctly, interrupt endpoints on the device side are necessarily implemented using a buffer that the device fills and that the USB device controller reads right away when queried.

To fix the problems with software only, there is one precondition, and then we have to poll it faster than allowed.

The precondition is that the buffer the USB device controller is updated at the request of the device’s main process, every ~1,4ms, instead of being updated at the request of the USB device controller, at the same frequency as the exposed bInterval, so, every 8ms. If the latter, all is lost. I don’t know which case we’re in yet.
And if the former, that means that USB device controller actually could provide more data than what it tells the host and there only remain to disregard the exposed polling interval - so to speak, overclock.

I digressed earlier, wanting to check if by some alignments of the stars, sending a longer buffer than the returned data would cause the chip to accumulate every 1,4ms data in the buffer and return it every 8ms, but there are numerous reasons this probably wasn’t going to work, and actually I had understood nothing. As per Mr. Roberts explanation, if I got it right, data certainly doesn’t accumulate in blocks one after another in a buffer too large for the device to internally handle it (how did I ever think this was going to work ?) but is split at the HC/HCD level into requests fit for the device, one after another, and the Windows driver will get all the answers back at once once they’re all complete.

I hope this clarified my various unclear statements from earlier. I’ll now try HidUsbF.

In this case, i just ask for confirmation if i correctly understood your initial request:
“make 2 or 1ms polling on device with 8ms bInterval in Interrupt Endpoint Descriptor”

And @SweetLow , I hope this clarifies my request !

> The problem I’m trying to fix in the end is that feeding a 60Hz state machine with an data polled at 62,5Hz (every 8ms) makes things go very wrong.

8ms is 125Hz. That is above the Nyquist frequency for a 60Hz signal. And you certainly have enough timing data to know when those packets were transferred. But if the data is really samples at a near-millisecond rate, then why is the difference important? You have to reconstruct the timing anyway, right?

So the problem is the following : is it possible to access the data of the controllers that the adapter chip is pulling every ~1,4ms ?

In a word, no.

And if I understood this correctly, interrupt endpoints on the device side are necessarily implemented using a buffer that the device fills and that the USB device controller reads right away when queried.

Right.

The precondition is that the buffer the USB device controller is updated at the request of the device’s main process, every ~1,4ms, instead of being updated at the request of the USB device controller, at the same frequency as the exposed bInterval, so, every 8ms. If the latter, all is lost. I don’t know which case we’re in yet.

I don’t know how you would ever determine that without seeing the device firmware from the manufacturer.

Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

It worked !

SweetLow’s driver successfully overclocked the polling of the non-HID device to 1000Hz. From the fact that I can see stick values moving from a millisecond from another repeatedly, internally, the USB device controller’s buffer is updated every ~1,4ms. The best part in this is that, as expected, I get NAKs when there is no new data instead of a copy of the previous state, which means I will be able to reconstruct on the software end when the polling happened inside the device, and nullify the damage of the 1000Hz/60Hz frequency misalignement by paying only 1ms of extra input lag.

Also, yes, I meant 125Hz, my mistake. And there is no Nyquist frequency problem here - I’m not trying to reconstruct a signal. Milliseconds are actually important in my scenario. Picture someone trying to input a button on an engine (state machine) frame, then another button exactly 5 engine frames later (so, 5 time units of the state machine later). And let’s say a frame lasts 20ms. In an utopian world, if he presses the second button 98ms after the first, the polling mechanism is such that it will have a 10% chance of being processed as a 4 engine frames difference, and a 90% chance of being processed as a 5 engine frames difference. But now an adapter steps in between, and it polls every - let’s say - 15 ms. It doesn’t matter which timing you had when you pressed the two buttons, what the PC sees is either a 90ms or 105ms difference. Best case, 105ms, your inputs only have a 75% chance of being processed as 5 engine frames apart. This is where the millisecond precision requirement comes from, and how the adapter would ruin it.

Thanks a lot to everyone who helped ! Thanks to you, I learnt a lot about USB, saved weeks+ of pointless work and got things to work as well as in my dreams.