It appears that WinUSB doesn’t automagically break messages up into endpoint max packet sizes and append a zero byte packet to terminate mesage, when there isn’t a short packet left over.
I have an application with a 16 byte bulk in and a 16 byte bulk out endpoint. If I send a 16 byte message from the host, and trap on 0 byte packet in the device, the trap never triggers. If I explicitly send a 0 byte packet after the 16 byte packet, the trap on the device triggers.
This is a fundamental function of a USB driver, and it seems unlikely to me that I would be the first to stumble on such a fundamental bug. So what is more likely is that I’ve set the pipe policies incorrectly. However, the only policy that might affect this “IGNORE_SHORT_PACKETS” doesn’t mention anything like this?
Someday when I have time for a beer, it might be worth discussing the availability of a pipe policy that breaks the fundamental comm protocol of USB. But, not today, I just need to get it working.
What am I doing wrong?
You wrote:
It appears that WinUSB doesn’t automagically break messages up into
endpoint max packet sizes and append a zero byte packet to terminate
mesage, when there isn’t a short packet left over.
The host controller driver chops transfers into packet-sized transfers. However, no one does the ZLP automatically.
I have an application with a 16 byte bulk in and a 16 byte bulk out
endpoint. If I send a 16 byte message from the host, and trap on 0 byte
packet in the device, the trap never triggers. If I explicitly send a 0
byte packet after the 16 byte packet, the trap on the device triggers.
Yep, that’s what you have to do.
This is a fundamental function of a USB driver, and it seems unlikely
to me that I would be the first to stumble on such a fundamental bug.
When you use WinUSB, you essentially **ARE** the driver. WinUSB is just a mechanism that lets you write drivers in user-mode. If your device needs ZLPs, then it is up to you (as the driver) to generate them. It’s just that simple.
Someday when I have time for a beer, it might be worth discussing the
availability of a pipe policy that breaks the fundamental comm protocol
of USB.
Come, now. ZLPs are not “the fundamental comm protocol of USB”. It’s just one way to implement variable-sized packets. If WinUSB enforced this, how could you do a 32MB transfer? You have to chop that into smaller pieces yourself, but WinUSB doesn’t have any way to know which transfer is the last one.
What am I doing wrong?
I’d say “incorrect assumption.”
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
> -----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Sunday, May 09, 2010 5:36 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] WinUSB and transaction terminationWhen you use WinUSB, you essentially **ARE** the driver.
WinUSB is just a mechanism that lets you write drivers in
user-mode. If your device needs ZLPs, then it is up to you
(as the driver) to generate them. It’s just that simple.
It may not be the case. WinUsb can be used purely as a driver and called
directly by application. For simple devices it works with no problem.
Actually, it may be the simplest way how to start USB development. Write
an INF, install WinUsb for the device and then experiment at application
level.
Come, now. ZLPs are not “the fundamental comm protocol of
USB”. It’s just one way to implement variable-sized packets.
If WinUSB enforced this, how could you do a 32MB transfer?
You have to chop that into smaller pieces yourself, but
WinUSB doesn’t have any way to know which transfer is the last one.
App/UMDF can send whole 32 MB transfer to WinUsb at once. I can imagine
pipe policy which’d add ZLP on the end of every transfer. Why not if a
(pretty dumb) device needs it? However, AFAIK WinUsb doesn’t support it.
Yet?
Best regards,
Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]
Michal Vodicka wrote:
> Tim wrote:
> When you use WinUSB, you essentially **ARE** the driver.
> WinUSB is just a mechanism that lets you write drivers in
> user-mode. If your device needs ZLPs, then it is up to you
> (as the driver) to generate them. It’s just that simple.
>It may not be the case. WinUsb can be used purely as a driver and called
directly by application. For simple devices it works with no problem.
Actually, it may be the simplest way how to start USB development. Write
an INF, install WinUsb for the device and then experiment at application
level.
True. And in that case, the application **IS** the driver. You can’t
escape the need to understand the hardware and how to manipulate it,
just because the URBs are created in a user-mode app instead of in a
kernel driver. You’re still talking to hardware. WinUSB’s behavior
here matches the behavior of the USB hub driver. The WinUSB client
application is being asked to do the same things that a USB client
driver would be asked to do, and the responsibilities are very much the
same. To me, that’s a Good Thing.
App/UMDF can send whole 32 MB transfer to WinUsb at once.
Are you sure? There must be a limit. I just don’t happen to know what
it is.
I can imagine pipe policy which’d add ZLP on the end of every transfer. Why not if a
(pretty dumb) device needs it?
Because you can already do that yourself. No changes are needed. It’s
not automatic, but so what? If you want sugar coating, write a driver,
or a DLL wrapper.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
> -----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Monday, May 10, 2010 10:01 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] WinUSB and transaction terminationTrue. And in that case, the application **IS** the driver. You can’t
escape the need to understand the hardware and how to manipulate it,
just because the URBs are created in a user-mode app instead of in a
kernel driver. You’re still talking to hardware. WinUSB’s behavior
here matches the behavior of the USB hub driver. The WinUSB client
application is being asked to do the same things that a USB client
driver would be asked to do, and the responsibilities are
very much the
same. To me, that’s a Good Thing.
WinUsb does something unless you enable raw I/O mode. App just need to
know comunication protocol between host and device. I don’t see such an
app as a driver but well, that’s terminology thing.
Are you sure? There must be a limit. I just don’t happen to
know what
it is.
Maybe there is a limit but I also don’t know it. Limit for buffered I/O
transfer size?
Because you can already do that yourself. No changes are
needed. It’s
not automatic, but so what? If you want sugar coating, write
a driver,
or a DLL wrapper.
How do you send a ZLP when standard i.e. not raw I/O is used? I’m not
saying it is impossible, just don’t know it now. Sure, if it is
possible, doing it at driver level isn’t necessary but why don’t add it
and make user more happy?
Best regards,
Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]
There is a pipe policy to turn this on. Look for SHORT_PACKET_TERMINATE in the docs http://msdn.microsoft.com/en-us/library/ff540304(VS.85).aspx
The default has this policy turned off, as many protocols don’t rely on short packet termination for OUT endpoints.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@earthlink.net
Sent: Saturday, May 08, 2010 11:35 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] WinUSB and transaction termination
It appears that WinUSB doesn’t automagically break messages up into endpoint max packet sizes and append a zero byte packet to terminate mesage, when there isn’t a short packet left over.
I have an application with a 16 byte bulk in and a 16 byte bulk out endpoint. If I send a 16 byte message from the host, and trap on 0 byte packet in the device, the trap never triggers. If I explicitly send a 0 byte packet after the 16 byte packet, the trap on the device triggers.
This is a fundamental function of a USB driver, and it seems unlikely to me that I would be the first to stumble on such a fundamental bug. So what is more likely is that I’ve set the pipe policies incorrectly. However, the only policy that might affect this “IGNORE_SHORT_PACKETS” doesn’t mention anything like this?
Someday when I have time for a beer, it might be worth discussing the availability of a pipe policy that breaks the fundamental comm protocol of USB. But, not today, I just need to get it working.
What am I doing wrong?
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Randy:
Once again you are correct, if you set the IGNORE_SHORT_PACKETS to true for OUT pipes, a ZLP is appended, when needed.
Sadly, the documentation sucks spuds through a straw. The MSDN & WDK dox don’t document the function of this policy on OUT pipes, only IN pipes. I certainly hope that there are no alligators in them there tall weeds.
You may wonder why this is important to me. The answer is that my device processor is an 8 bit microcontroller with very limited RAM resources. Consequent to this I cannot build a buffer bigger than my biggest message, so I process the message in 16 byte or less chunks. So, no ZLPs is a source of much heartache. And I definitely don’t agree with the ZLP being the province of the host app. This is part of the USB spec, and hence part of the protocol or even physical layer. To have the app keep track of it means the app needs to know the buffer size (not easy to extract, since I have to call 42 SetupDi functions to do so, and don’t even get me started on Marshaling). Extracting this, and keeping track of it adds *no* value to the app.
Anywho, problema solved. Now, can someone tell me why I’m managing the return buffer memory for the Winusb_GetDescriptor call???
If the docs suck, send feedback on the docs on the doc page using the link at the top right of the page (if you don’t have it, you need to go to preferences change yoru view to classic).
Now, can someone tell me why I’m managing the return buffer memory for the Winusb_GetDescriptor call???
Huh? The docs say you allocate it
Buffer [out]
A caller-allocated buffer that receives the requested descriptor
d
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@earthlink.net
Sent: Wednesday, May 12, 2010 3:14 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] WinUSB and transaction termination
Randy:
Once again you are correct, if you set the IGNORE_SHORT_PACKETS to true for OUT pipes, a ZLP is appended, when needed.
Sadly, the documentation sucks spuds through a straw. The MSDN & WDK dox don’t document the function of this policy on OUT pipes, only IN pipes. I certainly hope that there are no alligators in them there tall weeds.
You may wonder why this is important to me. The answer is that my device processor is an 8 bit microcontroller with very limited RAM resources. Consequent to this I cannot build a buffer bigger than my biggest message, so I process the message in 16 byte or less chunks. So, no ZLPs is a source of much heartache. And I definitely don’t agree with the ZLP being the province of the host app. This is part of the USB spec, and hence part of the protocol or even physical layer. To have the app keep track of it means the app needs to know the buffer size (not easy to extract, since I have to call 42 SetupDi functions to do so, and don’t even get me started on Marshaling). Extracting this, and keeping track of it adds *no* value to the app.
Anywho, problema solved. Now, can someone tell me why I’m managing the return buffer memory for the Winusb_GetDescriptor call???
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
xxxxx@earthlink.net wrote:
To have the app keep track of it means the app needs to know the buffer size (not easy to extract, since I have to call 42 SetupDi functions to do so, and don’t even get me started on Marshaling).
What do you mean by “buffer size” here? The size of the buffers that
your device deals with is strictly a private matter between you and your
device. That’s not present in any standard descriptor, nor is it
present in any SetupDi result.
The endpoint max packet size, on the other hand, is easily available
from WinUSB, and is also something you need to be aware of when you are
driving a device.
Perhaps we are all just becoming spoiled. If you had started this
project in Windows 2000 and had been forced to write a kernel-mode USB
driver with error checking and PnP/power support, you would appreciate
how much drudgery WinUSB has eliminated.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Nope I’m referring to ‘bufferptr = Marshal.AllocHGlobal(Marshal.SizeOf(mydevicedescriptor))’, and etc. I’ve become Malloc, sigh.
It requires 8 statements to pass a buffer pointer, and get the data back. This I have to do everytime, it doesn’t add value, and is easy to screwup. So, it meets all the criteria of a pain in the posterior.
Doron:
I did give feedback on the dox from the MSDN site, but I doubt it will do any good. MS needs to have two columns for each of the policies. One for IN pipes and one for OUT pipes. They also need to have the signatures documented for each of the calls, *in each of the languages they support*! I usually have better luck finding documented signatures at pinvoke.net (though not for winusb, most unfortunate).
Clearly, MS people have access to much better dox, or Redmond would be a giant asylum by now. Consequently, crappy dox at MSDN or in the WDK are not a problem for MS…
So, I imagine spuds will be going transonic for the foreseeable future, 
Tim:
I greatly appreciate Winusb and what it could do for me. But when I look around I don’t see that many people using it. Could it be that it is so poorly documented that people are sticking with the ACM model, and just treating the USB connection as a new and improved serial I/O port?
If MS wants developers to move to Winusb (or even C#) they better get religion about documentation.
For example, suppose that I have a Winusb device that I don’t want to power down, say a signal generator. I have to power down if there is no buss activity for > 3ms, this is in the spec, and required. So, I need to set the power policy for my device to be send SOF tokens at least once every 3ms. I cannot for the life of me figure out how you set this with winusb_setpowerpolicy. Yet, this is a basic requirement that must be supported. The lack of dox make it appear impossible?
You wrote:
Clearly, MS people have access to much better dox, or Redmond would be
a giant asylum by now. Consequently, crappy dox at MSDN or in the
WDK are not a problem for MS…
You are incorrect. MS people have access to MS people. If there is an question that is not answered in the docs, they can call someone up to get it resolved.
However, much of what you’re going through is just the learning curve that everyone goes through. It’s frustrating, but there’s really no good way to eliminate that, short of having someone more experienced sit next to you and explain it. Also, you’re poking at a narrowly focused area; they can’t cover everything in the docs.
I greatly appreciate Winusb and what it could do for me. But when I
look around I don’t see that many people using it. Could it be that it
is so poorly documented that people are sticking with the ACM model,
and just treating the USB connection as a new and improved serial I/O
port?
No, I don’t think so. A big part of it is that WinUSB is relatively new, and product development cycles are rather long. WinUSB is extensively used in development labs for experimental projects, but when a product gets released, the marketing department wants a real driver. Plus, there are still a number of device classes where WinUSB is just not workable.
Also remember that USB UMDF drivers use WinUSB, and there are more of those appearing each day.
If MS wants developers to move to Winusb (or even C#) they better get
religion about documentation.
I don’t think MS particularly cares whether you use WinUSB. It’s just another option – another tool in the toolbox.
For example, suppose that I have a Winusb device that I don’t want to
power down, say a signal generator. I have to power down if there is no
buss activity for > 3ms, this is in the spec, and required.
Whose spec?
So, I need
to set the power policy for my device to be send SOF tokens at least once
every 3ms. I cannot for the life of me figure out how you set this with
winusb_setpowerpolicy. Yet, this is a basic requirement that must be
supported. The lack of dox make it appear impossible?
USB host controllers don’t have that level of control. Either you support suspend, or you don’t. If your device doesn’t support suspend, you’ll get an SOF in every frame forever, unless the whole machine changes power state. That’s your power policy.
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
> -----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Thursday, May 13, 2010 9:04 PM
To: Windows System Software Devs Interest List
Subject: Re: RE:[ntdev] WinUSB and transaction terminationWinUSB is extensively used in development labs for
experimental projects, but when a product gets released, the
marketing department wants a real driver.
I don’t quite understand. WinUsb + INF for given device is real driver
and (at least our) marketing department wouldn’t care if it works
sufficiently. I presume the need for own driver is only when there has
to be added functionality.
Also remember that USB UMDF drivers use WinUSB, and there are
more of those appearing each day.
Yes. I believe it will be major use.
I don’t think MS particularly cares whether you use WinUSB.
It’s just another option – another tool in the toolbox.
Well, it seems MS prefers and pushes UMDF drivers when possible and in
turn WinUsb.
>For example, suppose that I have a Winusb device that I don’t want to
>power down, say a signal generator. I have to power down if
there is no
>buss activity for > 3ms, this is in the spec, and required.Whose spec?
I presume he means USB spec, idle state and makes wrong conclusion.
USB host controllers don’t have that level of control.
Either you support suspend, or you don’t. If your device
doesn’t support suspend, you’ll get an SOF in every frame
forever, unless the whole machine changes power state.
That’s your power policy.
Exactly. Device should be suspended only when idle and it is on power
policy owner (i.e. WinUsb client) to know when it is idle.
Best regards,
Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]
Michael:
No, I don’t make the wrong conclusion. The USB1.1-3.0 spec states that if there is no buss activity for more than 3ms, the device *must* enter suspend. Consultation with the USB IF testing group, reveals there are no exceptions (other than not getting the USB logo).
The original spec. had 500?A of suspend current per ‘unit’ load, up to 2.5mA for non-composite device. The ECN of May 2008, made 2.5mA the only suspend current, even for USB 3.0.
As you might of guessed I don’t write code for a living (well my current living anywho), I design analog ICs-battery chargers, DC/DC and even ac/DC switching regulators. I’m very well acquainted with the USB specifications and this is a requirement.
Returning to my ‘hobby’, the USB controllers handle SOF generation in hardware, and this can be turned on or off, otherwise you could not do selective suspend. While Tim says that the driver doesn’t have this level of control, that is obviously incorrect. A kernel mode driver has complete control of the hardware. What may be true, at least for Winusb, is that the driver is not exposing that fact to the user through a power policy setting. And that would be unfortunate, since winusb is targeted at custom devices, custom devices are the most likely to need these kinds of features.
Tim already explained it. Unless port enters suspended state, HC sends SOFs every ms so your device won’t need to enter suspend. It is the default state unless your driver says otherwise. Your driver is power policy owner and it has complete control over device state. If you don’t want to support suspend, you don’t need to. If your customers accept increased power consuption, of course.
With WinUsb you just set power policy and that’s all. You can change it according to current circumstances, for example if app has opened handle, if your device is set to some state etc.
I’m not sure what you mean with:
A kernel mode driver has complete
control of the hardware. What may be true, at least for
Winusb, is that the driver is not exposing that fact to the
user through a power policy setting.
It does. Both using Power Management tab in the Device Manager and programatically. As I said, your UMDF driver or an app using WinUsb directly has full control over suspending device. Which doesn’t mean anything else than turning off bus activity for more than 3 ms.
I’d recommend to study docs and ask questions. It seems you make too quick and incorrect conclustion based on partial knowledge.
Best regards,
Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@earthlink.net
Sent: Thursday, May 13, 2010 11:57 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] WinUSB and transaction terminationMichael:
No, I don’t make the wrong conclusion. The USB1.1-3.0 spec
states that if there is no buss activity for more than 3ms,
the device *must* enter suspend. Consultation with the USB
IF testing group, reveals there are no exceptions (other than
not getting the USB logo).
The original spec. had 500?A of suspend current per ‘unit’
load, up to 2.5mA for non-composite device. The ECN of May
2008, made 2.5mA the only suspend current, even for USB 3.0.
As you might of guessed I don’t write code for a living (well
my current living anywho), I design analog ICs-battery
chargers, DC/DC and even ac/DC switching regulators. I’m
very well acquainted with the USB specifications and this is
a requirement.
Returning to my ‘hobby’, the USB controllers handle SOF
generation in hardware, and this can be turned on or off,
otherwise you could not do selective suspend. While Tim says
that the driver doesn’t have this level of control, that is
obviously incorrect. A kernel mode driver has complete
control of the hardware. What may be true, at least for
Winusb, is that the driver is not exposing that fact to the
user through a power policy setting. And that would be
unfortunate, since winusb is targeted at custom devices,
custom devices are the most likely to need these kinds of features.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online
at http://www.osronline.com/page.cfm?name=ListServer
Actually, this policy only talks about OUTs, as it talks about “writes”. It is completely silent on Ins.
ZLP for OUT pipes has definitely become the domain of the higher level device protocol (definitely WAY higher than the PHY (heck all packets are higher than the PHY)). Many device protocols are built in a way that the device already knows how many bytes to expect, and an extra ZLP would foul up the works. On the other hand, some device protocols require that OUTs be terminated with a short packet. This is why this is the responsibility of the software that knows the protocol implemented in the device (which is above the core USB stack). Winusb doesn’t know this either, so in this case, it is the “application’s” responsibility to know the needs. Since this is a common enough need, Winusb helps the application out by doing it on behalf of the application.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@earthlink.net
Sent: Wednesday, May 12, 2010 3:14 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] WinUSB and transaction termination
Randy:
Once again you are correct, if you set the IGNORE_SHORT_PACKETS to true for OUT pipes, a ZLP is appended, when needed.
Sadly, the documentation sucks spuds through a straw. The MSDN & WDK dox don’t document the function of this policy on OUT pipes, only IN pipes. I certainly hope that there are no alligators in them there tall weeds.
You may wonder why this is important to me. The answer is that my device processor is an 8 bit microcontroller with very limited RAM resources. Consequent to this I cannot build a buffer bigger than my biggest message, so I process the message in 16 byte or less chunks. So, no ZLPs is a source of much heartache. And I definitely don’t agree with the ZLP being the province of the host app. This is part of the USB spec, and hence part of the protocol or even physical layer. To have the app keep track of it means the app needs to know the buffer size (not easy to extract, since I have to call 42 SetupDi functions to do so, and don’t even get me started on Marshaling). Extracting this, and keeping track of it adds *no* value to the app.
Anywho, problema solved. Now, can someone tell me why I’m managing the return buffer memory for the Winusb_GetDescriptor call???
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer