I have a USB 2.0 device (base on cypress chip) that send a stream of data to the PC using 2 endpoints.
The stream rate through one endpoint is 2208 MB per second and through the other endpoint is 17664 MB per second (total 19872 MB per second).
I’m using a bulk transfer mode and I’m sending asynchronous 20 IRP (URB) to one endpoing each one has a buffer of 16 KB and another 20 IRP (URB) to the other endpoing each one has a buffer of 128 KB.
The thread that send these buffer runs in priority 30.
When the buffers are full with data a comkpletion routine is called. I insert these buffers to a queue.
In a processing thread that run in priority of 29 I do some checking on the data, copy the data to other buffers, a bigger one (0.5 MB and 4 MB according to the endpoint), and when these buffers are full I insert these buffers to another queue. I have another thread that take out these buffers (0.5 MB and 4 MB) and save them on the hard disk. At the begining I’m allocating (creating) a one big file and unfragmented to hold the data.
I test the driver in my lab on 6 computers and I found that on 3 computers with Pentium 1.6GHz and 500MB ram it works fine and on another 3 computers and much more robust (2.4, 2.6 and 3.0 GHz) sometimes I’m losing data from the endpoint with the higher data rate (17664 MB per second).
If I’m not save the data to the disk then I’m not losing any data from the endpoing.
My device has 3KB of buffers for the endpoing with the higher data rate and 1KB for the second endpoint.
It looks like that sometimes there is latency on that endpoint or another interrupt involves and take a CPU.
Any idea what makes the problem?
OS.
Try sending more IRPs then 20.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: osmlist
To: Windows System Software Devs Interest List
Sent: Sunday, June 27, 2004 9:15 PM
Subject: [ntdev] losing data through USB 2.0
I have a USB 2.0 device (base on cypress chip) that send a stream of data to the PC using 2 endpoints.
The stream rate through one endpoint is 2208 MB per second and through the other endpoint is 17664 MB per second (total 19872 MB per second).
I’m using a bulk transfer mode and I’m sending asynchronous 20 IRP (URB) to one endpoing each one has a buffer of 16 KB and another 20 IRP (URB) to the other endpoing each one has a buffer of 128 KB.
The thread that send these buffer runs in priority 30.
When the buffers are full with data a comkpletion routine is called. I insert these buffers to a queue.
In a processing thread that run in priority of 29 I do some checking on the data, copy the data to other buffers, a bigger one (0.5 MB and 4 MB according to the endpoint), and when these buffers are full I insert these buffers to another queue. I have another thread that take out these buffers (0.5 MB and 4 MB) and save them on the hard disk. At the begining I’m allocating (creating) a one big file and unfragmented to hold the data.
I test the driver in my lab on 6 computers and I found that on 3 computers with Pentium 1.6GHz and 500MB ram it works fine and on another 3 computers and much more robust (2.4, 2.6 and 3.0 GHz) sometimes I’m losing data from the endpoint with the higher data rate (17664 MB per second).
If I’m not save the data to the disk then I’m not losing any data from the endpoing.
My device has 3KB of buffers for the endpoing with the higher data rate and 1KB for the second endpoint.
It looks like that sometimes there is latency on that endpoint or another interrupt involves and take a CPU.
Any idea what makes the problem?
OS.
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
- I tried 40 and it failed either.
- I’m using a counter (interlock counter) that I increase by one every time that I send the URB and I decrease by one in the completion routine. When I recognize the losing data I’m checking my counter and it says that most of the buffers are pending (waiting for data from the USB).
- I tried also to allocate a larger buffers (10 of 512KB and 10 of 64KB) and also got the same results.
OS.
----- Original Message -----
From: Maxim S. Shatskih
To: Windows System Software Devs Interest List
Sent: Sunday, June 27, 2004 6:31 PM
Subject: Re: [ntdev] losing data through USB 2.0
Try sending more IRPs then 20.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: osmlist
To: Windows System Software Devs Interest List
Sent: Sunday, June 27, 2004 9:15 PM
Subject: [ntdev] losing data through USB 2.0
I have a USB 2.0 device (base on cypress chip) that send a stream of data to the PC using 2 endpoints.
The stream rate through one endpoint is 2208 MB per second and through the other endpoint is 17664 MB per second (total 19872 MB per second).
I’m using a bulk transfer mode and I’m sending asynchronous 20 IRP (URB) to one endpoing each one has a buffer of 16 KB and another 20 IRP (URB) to the other endpoing each one has a buffer of 128 KB.
The thread that send these buffer runs in priority 30.
When the buffers are full with data a comkpletion routine is called. I insert these buffers to a queue.
In a processing thread that run in priority of 29 I do some checking on the data, copy the data to other buffers, a bigger one (0.5 MB and 4 MB according to the endpoint), and when these buffers are full I insert these buffers to another queue. I have another thread that take out these buffers (0.5 MB and 4 MB) and save them on the hard disk. At the begining I’m allocating (creating) a one big file and unfragmented to hold the data.
I test the driver in my lab on 6 computers and I found that on 3 computers with Pentium 1.6GHz and 500MB ram it works fine and on another 3 computers and much more robust (2.4, 2.6 and 3.0 GHz) sometimes I’m losing data from the endpoint with the higher data rate (17664 MB per second).
If I’m not save the data to the disk then I’m not losing any data from the endpoing.
My device has 3KB of buffers for the endpoing with the higher data rate and 1KB for the second endpoint.
It looks like that sometimes there is latency on that endpoint or another interrupt involves and take a CPU.
Any idea what makes the problem?
OS.
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@hotmail.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
1 What OS is this on?
2 Are you resending the URB in the context of the previous transfers’ completion routine?
3 If you are writing to disk, that means you are thunking to passive level or just completing a user mode irp. Which one are you doing?
4 It sounds like you have at least one of your own threads b/c you mention 2 priorities. One for sending and one for processing?
5 when configuring your endpoints, did you set the MaximumTransferSize to some value other then the default?
I have found that resending from the completion routine will always be faster then letting another thread reinitiate the transfer. I also think if you are just completing user mode irps, you don’t have to muck with a driver thread just to write out to disk (if that is what you are doing).
d
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of osmlist
Sent: Sunday, June 27, 2004 10:49 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] losing data through USB 2.0
- I tried 40 and it failed either.
- I’m using a counter (interlock counter) that I increase by one every time that I send the URB and I decrease by one in the completion routine. When I recognize the losing data I’m checking my counter and?it says that most of the buffers are pending (waiting for data from the USB).
- I tried also to allocate a larger buffers (10 of 512KB and 10 of 64KB) and also got the same results.
?
OS.
----- Original Message -----
From: Maxim S. Shatskih
To: Windows System Software Devs Interest List
Sent: Sunday, June 27, 2004 6:31 PM
Subject: Re: [ntdev] losing data through USB 2.0
??? Try sending more IRPs then 20.
?
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: osmlist
To: Windows System Software Devs Interest List
Sent: Sunday, June 27, 2004 9:15 PM
Subject: [ntdev] losing data through USB 2.0
I have a USB 2.0 device (base on cypress chip) that send a stream of data to the PC using 2 endpoints.
The stream rate through one endpoint is 2208 MB per second and through the other endpoint is 17664 MB per second (total 19872 MB per second).
I’m using a bulk transfer mode and I’m sending asynchronous 20 IRP (URB) to one endpoing each one has a buffer of 16 KB and another 20 IRP (URB) to the other endpoing each one has a buffer of 128 KB.
The thread that send these buffer runs in priority 30.
When the buffers are full with data a comkpletion routine is called. I insert these buffers to a queue.
In a processing thread that run in priority of 29 I do some checking on the data, copy the data to other buffers, a bigger one (0.5 MB and 4 MB according to the endpoint), and when these buffers are full I insert these buffers to another queue. I have another thread that take out these buffers (0.5 MB and 4 MB) and save them on the hard disk. At the begining I’m?allocating (creating)?a one big file and unfragmented to hold the data.
I test the driver in my lab on 6 computers and I found that on 3 computers with Pentium 1.6GHz and 500MB ram it works fine and on another 3 computers and much more robust (2.4, 2.6 and 3.0 GHz) sometimes I’m losing data from the endpoint with the higher data rate (17664 MB per second).
If I’m not save the data to the disk then I’m not losing any data from the endpoing.
My device has 3KB of buffers for the endpoing with the higher data rate and 1KB for the second endpoint.
It looks like that sometimes there is latency on that endpoint or another interrupt involves and take a CPU.
?
Any idea what makes the problem?
?
OS.
?
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@hotmail.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Just a few questions for clarification. What kind of device is this? Are
you actually pumping 2,208 Mega BYTES (i.e. 2 GIGA bytes) per second across
USB which is on the order of 12 mega BITS per second or 100+ Mb/sec(?) for
USB2)?? Perhaps that is 22.08 or even 2.208 MB/second? Those would be much
more reasonable to attempt.
You’re doing a lot of buffer copies in tihs approach. Is there any way to
do this better with pointer manipulation? Perhaps try writing the data
directly to the correct slot in the large buffer initially and then queue
the large buffer to be written to disk. Also, if for some reason bus
mastering is not enabled on your disk, performance there will suffer
tremendously.
I hope that helps.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of osmlist
Sent: Sunday, June 27, 2004 12:16 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] losing data through USB 2.0
I have a USB 2.0 device (base on cypress chip) that send a stream of data
to the PC using 2 endpoints.
The stream rate through one endpoint is 2208 MB per second and through the
other endpoint is 17664 MB per second (total 19872 MB per second).
I’m using a bulk transfer mode and I’m sending asynchronous 20 IRP (URB)
to one endpoing each one has a buffer of 16 KB and another 20 IRP (URB) to
the other endpoing each one has a buffer of 128 KB.
The thread that send these buffer runs in priority 30.
When the buffers are full with data a comkpletion routine is called. I
insert these buffers to a queue.
In a processing thread that run in priority of 29 I do some checking on
the data, copy the data to other buffers, a bigger one (0.5 MB and 4 MB
according to the endpoint), and when these buffers are full I insert these
buffers to another queue. I have another thread that take out these buffers
(0.5 MB and 4 MB) and save them on the hard disk. At the begining I’m
allocating (creating) a one big file and unfragmented to hold the data.
I test the driver in my lab on 6 computers and I found that on 3 computers
with Pentium 1.6GHz and 500MB ram it works fine and on another 3 computers
and much more robust (2.4, 2.6 and 3.0 GHz) sometimes I’m losing data from
the endpoint with the higher data rate (17664 MB per second).
If I’m not save the data to the disk then I’m not losing any data from the
endpoing.
My device has 3KB of buffers for the endpoing with the higher data rate
and 1KB for the second endpoint.
It looks like that sometimes there is latency on that endpoint or another
interrupt involves and take a CPU.
Any idea what makes the problem?
OS.
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@pdq.net
To unsubscribe send a blank email to xxxxx@lists.osr.com
Sorry, you right, I made a mistake. The numbers (the rate) is in KB, in
other words, the total rate from both endpoint is 19872 KB per second or
about 19.9 MB.
BTW The device is sitting on an ADSL line and sample it and transfer the
data to the PC using USB 2.0.
The way that we recognize data lose is: the device inserts every pre
configure number of bytes (for example 1000 bytes) a counter and when the
buffer arrive to the PC we first check that the counter is in the right
place and then we copy JUST the relevant data (without the counter) to
another buffer, a bigger one, that later on will send to the disk
(filesystem), this processing is not consume to much CPU (less the 1%).
OS.
From: “Gregory G. Dyess”
>Reply-To: “Windows System Software Devs Interest List”
>
>To: “Windows System Software Devs Interest List”
>Subject: RE: [ntdev] losing data through USB 2.0
>Date: Mon, 28 Jun 2004 08:14:04 -0500
>
>Just a few questions for clarification. What kind of device is this? Are
>you actually pumping 2,208 Mega BYTES (i.e. 2 GIGA bytes) per second across
>USB which is on the order of 12 mega BITS per second or 100+ Mb/sec(?) for
>USB2)?? Perhaps that is 22.08 or even 2.208 MB/second? Those would be
>much
>more reasonable to attempt.
>
>You’re doing a lot of buffer copies in tihs approach. Is there any way to
>do this better with pointer manipulation? Perhaps try writing the data
>directly to the correct slot in the large buffer initially and then queue
>the large buffer to be written to disk. Also, if for some reason bus
>mastering is not enabled on your disk, performance there will suffer
>tremendously.
>
>I hope that helps.
> -----Original Message-----
> From: xxxxx@lists.osr.com
>[mailto:xxxxx@lists.osr.com]On Behalf Of osmlist
> Sent: Sunday, June 27, 2004 12:16 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] losing data through USB 2.0
>
>
> I have a USB 2.0 device (base on cypress chip) that send a stream of
>data
>to the PC using 2 endpoints.
> The stream rate through one endpoint is 2208 MB per second and through
>the
>other endpoint is 17664 MB per second (total 19872 MB per second).
> I’m using a bulk transfer mode and I’m sending asynchronous 20 IRP (URB)
>to one endpoing each one has a buffer of 16 KB and another 20 IRP (URB) to
>the other endpoing each one has a buffer of 128 KB.
> The thread that send these buffer runs in priority 30.
> When the buffers are full with data a comkpletion routine is called. I
>insert these buffers to a queue.
> In a processing thread that run in priority of 29 I do some checking on
>the data, copy the data to other buffers, a bigger one (0.5 MB and 4 MB
>according to the endpoint), and when these buffers are full I insert these
>buffers to another queue. I have another thread that take out these buffers
>(0.5 MB and 4 MB) and save them on the hard disk. At the begining I’m
>allocating (creating) a one big file and unfragmented to hold the data.
> I test the driver in my lab on 6 computers and I found that on 3
>computers
>with Pentium 1.6GHz and 500MB ram it works fine and on another 3 computers
>and much more robust (2.4, 2.6 and 3.0 GHz) sometimes I’m losing data from
>the endpoint with the higher data rate (17664 MB per second).
> If I’m not save the data to the disk then I’m not losing any data from
>the
>endpoing.
> My device has 3KB of buffers for the endpoing with the higher data rate
>and 1KB for the second endpoint.
> It looks like that sometimes there is latency on that endpoint or
>another
>interrupt involves and take a CPU.
>
> Any idea what makes the problem?
>
> OS.
>
> —
> Questions? First check the Kernel Driver FAQ at
>http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@pdq.net
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>—
>Questions? First check the Kernel Driver FAQ at
>http://www.osronline.com/article.cfm?id=256
>
>You are currently subscribed to ntdev as: xxxxx@hotmail.com
>To unsubscribe send a blank email to xxxxx@lists.osr.com
_________________________________________________________________
MSN Movies - Trailers, showtimes, DVD’s, and the latest news from Hollywood!
http://movies.msn.click-url.com/go/onm00200509ave/direct/01/