is it the bug of wdk and wdm(continue)

hello ,my dear friend,thank you for you answer in “is it the bug of wdk and wdm”,and now I post a new thread to summarize the subject which discussed some day ago.

At first,I introduce the background of my driver. this is a Camera driver,and I Communication with the hardware through communication protocol,First i build a vendor request and Send a setup packet,and listen the Endpoint ,then read the dada.The hardware receive my request and prepare send dada.the hareware buffer is 2K,and every time fill 512 bytes in it ,and it fill the buffer whether the data read or not. if the read speed is slower,then the dada will be cover with,at this time,it will be receive not enough dada.
and the camera driver must judge whether the dada is enough or not ,when it not enough ,I will send a request to the hardware ,and the hardware send the dada again.

I change the wdk’s usbsamp for my Camera driver,and use bulk transfer.
my application pragram set the buffer equal to the driver’s MAX_TRANSFER_SIZE, it is run well usually, but when the request timeout,I can not receive any datas,the transfer size is zero.

And now ,there are some problem with it ?

(1) I am modify the wdk’s usbsamp driver for my camera device ,and the
application buffer is 2014208 bytes. I change usbsamp’s MAX_TRANSFER_SIZE to
2014208,and usually run well,but some time IO request timeout ,and at this time
,it can not receive any datas.

(2)if the device send dada size less than the MAX_TRANSFER_SIZE of driver,at this time ,is it the driver can not receive any dada?

(3) I change MAX_TRANSFER_SIZE to 512,or 2048, It timeout every time ,but it can return some data which less than 2014208. I am confused why MAX_TRANSFER_SIZE influence the results? and when MAX_TRANSFER_SIZE is smaller,it every time timeout ,is it stand for call Complete routine need much time. so I modify MAX_TRANSFER_SIZE to 2014208,at this time ,only call complete routine one time. but when it timeout ,will receive no dada.

(4)Chris Aseltine say" I suspect you’ve sent down a request for 2MB
down to USB, and the device has responded with some fraction of that which is an
exact multiple of the endpoint’s maximum packet size. Since the device did not
send a short packet, the transfer never completed."
yes, the hardware not send a short packet or zero packet,but now the camera usually run well ,and sometime can not receive any data? why?

(5)and thanks for Tim Roberts,you say “Well, here is a question for the KMDF folks. If I submit a USB requestand set a timeout in the WDF_REQUEST_SEND_OPTIONS, will I be able torecover whatever partial buffer had been transferred by the time of the timeout? I’ve never had occasion to test that.” ,I think it is a good topic to discuss?

(6)Tim Roberts say "Interesting. There must be a solution for this, because there are
situations in USB that cannot be reasonably handled in any other way. Consider a device that sends between 100k and 200k bytes at random(potentially long) intervals. You can’t afford to send a bazillion 512-byte requests because of the overhead. You need to send a larger request, but then you need a way to bring back the partial buffer after
a timeout. " it is a good topic to discuss ,because my device run well in linux ,and my hardware team say it is not the device problem because of this ,and say it driver problem or windows system problem.

(7)thank you Peter Viscarola,and you say "OR, he COULD just hang a bunch of 512 byte packets out there… since his device isn’t transferring short blocks, those packets would be completely filled by a transfer, right?? Isn’t that what we’re talking about? ",to you answer,i will say ,though it not send a short parket or zero parket ,but in my opinion,it will receive some dada,because It is already send some dada before receive short parket or zero parket .

(8) and Hagen Patzke says
"Well, here Chris is backed by the USB 2.0 spec, section 5.8.3:

A bulk transfer is complete when the endpoint does one of the
following: ? Has transferred exactly the amount of data expected ?
Transfers a packet with a payload size less than wMaxPacketSize or
transfers a zero-length packet
" thank you Hagen Patzke ,I have a question ,if the device not send a short packet or zero packet,and when it lose some dada ,is it the application program can not receive any dada? thank you ?

(9)and some people suggest me to receive 512 bytes every time ,but I test it ,every time timeout ,and Tim Roberts says " For a 2M transfer, you’retalking about 4,000 IRPs, and they will be completed at a rate of 120 per millisecond. For one frame, you could do it, but if you’re streaming, it can’t be done. You can’t sustain it" and I say before ,I reduce the max_transfer_size to 512 bytes,and also timeout,because so many complete route.

(10)thank you Peter Viscarola ,you say “TOTALLY right. Shit flows down hill, and it’s us driver devs who are perennially at the bottom of that hill looking up. OS problem? We get to work around it. Bug in the silicon? We get to compensate for it. Firmware deficiency? Fix it up. And when these things don’t work, it’s ALWAYS our fault.”
I agree with you standpoint ,and my hardware say my device run well in linux ,so it you problem.

thank you ,every one,and welcome every body discuss this problem with me ?

thank you very much indeed.

stone
2010.09.06

>send dada.the hareware buffer is 2K,and every time fill 512 bytes in it ,and it fill the buffer whether the

data read or not. if the read speed is slower,then the dada will be cover with,at this time,it will be

Sorry, but flow control on a bulk pipe is done automatically on a low level (NAK packets), lower then any software.

No need to repro it in the driver.


Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

thank you Maxim S. Shatskih, thank you for you answer,you say bulk pipe control is lower than software.In you opinion ,it can not lose any dada,but why i can not receive any dada when it timeout ,thank you

zhonghong200@163.com wrote:

hello ,my dear friend,thank you for you answer in “is it the bug of wdk and wdm”,and now I post a new thread to summarize the subject which discussed some day ago.

At first,I introduce the background of my driver. this is a Camera driver,and I Communication with the hardware through communication protocol,First i build a vendor request and Send a setup packet,and listen the Endpoint ,then read the dada.

PLEASE – it’s “data”. The word is “data”. We’ve told you this two or
three times already. Stop using the nonsense word “dada” if you wish to
be taken seriously.

The hardware receive my request and prepare send dada.the hareware buffer is 2K,and every time fill 512 bytes in it ,and it fill the buffer whether the data read or not. if the read speed is slower,then the dada will be cover with,at this time,it will be receive not enough dada.
and the camera driver must judge whether the dada is enough or not ,when it not enough ,I will send a request to the hardware ,and the hardware send the dada again.

Yes, I think we are all pretty clear on your hardware design now. Your
hardware design has a bug. Your hardware needs to be modified to send a
short packet when the transfer is complete. That’s how USB bulk
requests are supposed to work. Alternatively, you could modify the
hardware to use an isochronous pipe instead of a bulk pipe. That, also,
would solve your problem, and is BY FAR the most traditional solution
for cameras, exactly because of this issue.

The fact that your current design can be made to work on Linux is an
accident of Linux. The USB spec does not require it. Windows doesn’t
allow it, and Windows will not be modified to allow it. Those are the
facts.

yes, the hardware not send a short packet or zero packet,but now the camera usually run well ,and sometime can not receive any data? why?

There are many flawed designed that “usually run well.” The problems
happen when the system comes under stress. That’s the case here.

(5)and thanks for Tim Roberts,you say “Well, here is a question for the KMDF folks. If I submit a USB requestand set a timeout in the WDF_REQUEST_SEND_OPTIONS, will I be able to recover whatever partial buffer had been transferred by the time of the timeout? I’ve never had occasion to test that.” ,I think it is a good topic to discuss?

It WAS discussed. The answer is “no”. The end.

it is a good topic to discuss ,because my device run well in linux ,and my hardware team say it is not the device problem because of this ,and say it driver problem or windows system problem.

I’m sorry. Your hardware team is wrong. The hardware design is
flawed. It relies on behavior that is not provided by the USB
specification. In order to be compliant, you either need to change to
an isochronous pipe, or send a short packet when the frame is done.
That’s what the spec demands.

" thank you Hagen Patzke ,I have a question ,if the device not send a
short packet or zero packet,and when it lose some dada ,is it the
application program can not receive any dada? thank you ?

If a large transfer is terminated before it is complete, the behavior of
the operating system is not specified by the spec. Linux happens to
return the partial data. Windows does not. BOTH approaches are
perfectly valid. Your hardware team has made a huge mistake by
designing a piece of hardware that relies on unspecified behavior.

Here is the executive summary in a nutshell. If your company wishes to
have its device used in Windows, and not simply because a money-losing
garage floor experiment, then you MUST change your hardware. There is
no alternative.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

I think Marcel Duchamp would probably agree with you:

http://emptyeasel.com/2007/10/04/the-dada-art-movement-or-anti-art-movement-if-you-prefer/

Peter
OSR

thank you ,Tim Roberts,“data not dada,” I understand now

and i have another question,if the hardware send a short parket after a frame,but when the transfer process is timeout ,is it the application program can receive the short parket ?

On the other word,at this time ,how to judge the transfer end or not?

thank you Tim Roberts ,you are enthusiastic and helpful.

thank you ,every one who give me suggestion.

I’m not sure I understand the question: When the hardware sends a short packet, the current read for the endpoint is terminated by the host controller driver. IF there was already a timeout, that packet is completed back to the requestor with NO DATA returned.

Peter
OSR

thank you ,Peter,I mean even if the hareware can send a short parket,but when the transfer process timeout ,may be at this time,the short parket may not send,so the transfer not terminated, at this time,is it return data or not?

zhonghong200@163.com wrote:

thank you ,Peter,I mean even if the hareware can send a short parket,but when the transfer process timeout ,may be at this time,the short parket may not send,so the transfer not terminated, at this time,is it return data or not?

It’s not that complicated. When the hardware sends a short packet, the
transfer completes. The request is sent back to the driver. If you
cancel the request before the transfer completes (meaning before the
hardware sends a short packet), then you won’t get the partial information.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

By the way, when a transfer URB is cancelled, it should accurately report the number of bytes that had been successfully transferred before the cancellation in its TransferBufferLength field.

If on completion the URB indicates that no bytes were transferred, then it is likely that no bytes were transferred.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of zhonghong200@163.com
Sent: Tuesday, September 07, 2010 8:14 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] is it the bug of wdk and wdm(continue)

thank you ,Peter,I mean even if the hareware can send a short parket,but when the transfer process timeout ,may be at this time,the short parket may not send,so the transfer not terminated, at this time,is it return data or not?


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Randy Aull wrote:

By the way, when a transfer URB is cancelled, it should accurately report the number of bytes that had been successfully transferred before the cancellation in its TransferBufferLength field.

If on completion the URB indicates that no bytes were transferred, then it is likely that no bytes were transferred.

The original poster is using WinUSB. Perhaps it is WinUSB that is not
returning the partial count when an URB is canceled.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

thank you ,every body.

I mean even if the hardware can send the short parket,it can not solve the problem which transfer timeout can not receive data,because when it timeout ,the short parket may not send.
is it right?

Look, you’re making this way too hard. When the short packet is sent, the timeout no longer matters. The transfer ends right there. What you would do is set the timeout to a very long time – an emergency timeout only – and plan that it would never happen.

Sent from my iPhone

On Sep 8, 2010, at 6:05 PM, “zhonghong200@163.com” wrote:

> thank you ,every body.
>
> I mean even if the hardware can send the short parket,it can not solve the problem which transfer timeout can not receive data,because when it timeout ,the short parket may not send.
> is it right?
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

thank you Tim Roberts.

if the hardware not change,even if the hardware can send short parket,it can not solve the problem.
If the hardware change it to synchronize transfer,is it can solve this problem? I am not sure.

stone

I thought he claimed he had built a driver based on usbsamp.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, September 08, 2010 4:48 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] is it the bug of wdk and wdm(continue)

Randy Aull wrote:

By the way, when a transfer URB is cancelled, it should accurately report the number of bytes that had been successfully transferred before the cancellation in its TransferBufferLength field.

If on completion the URB indicates that no bytes were transferred, then it is likely that no bytes were transferred.

The original poster is using WinUSB. Perhaps it is WinUSB that is not returning the partial count when an URB is canceled.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

yes i based usbsamp,Randy Aull,is it have bug in usbsamp,thank you

zhonghong200@163.com wrote:

yes i based usbsamp,Randy Aull,is it have bug in usbsamp

Yes, this was my mistake. He’s not using the WinUSB abstraction, he’s
using the KMDF abstraction. That’s why he can talk about a “timeout”,
which is something that has no single parallel in WDM.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.