Hi all,
I am currently making a utility for testing a usb hard disk drive and have been using IOCTL_SCSI_PASS_THROUGH_DIRECT in DeviceIOControl for sending CDB commands to the disk. I have been successful in using this method but I am encountering a 128kB limitation when using commands with data transfer. GetLastError just returns 87 whenever I try to increase the transfer size above 128kB.
Anyone here experienced a similar case before? I had been googling it for a while now but haven’t got any clear answer for this one. I also use Bustrace and I have been seeing some applications sending SRBs and wondering how they send this type of IRP.
I am quite new in using windows i/o since I used ASPI before for sending CDB to scsi drives. Any pointers and suggestions are greatly appreciated.
Many adapters are in effect limited to 128K transfers. Query the STORAGE_ADAPTER_DESCRIPTOR, minding the MaximumTransferLength and MaximumPhysicalPages fields.
I see, but I noticed that some programs can issue higher than that by using
SRBs? How do they do that?
On Fri, Jul 22, 2011 at 4:42 PM, wrote:
> Many adapters are in effect limited to 128K transfers. Query the
> STORAGE_ADAPTER_DESCRIPTOR, minding the MaximumTransferLength and
> MaximumPhysicalPages fields.
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>been successful in using this method but I am encountering a 128kB limitation when using commands with data
transfer. GetLastError just returns 87 whenever I try to increase the transfer size above 128kB.
This is normal. The storage adapters do have a max IO size limit.
I think some of the “get adapter info” IOCTLs like IOCTL_STORAGE_QUERY_PROPERTY will return this limitation.
I think the usual (S)ATA controller is limited to 64K.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
Split your SCSI command to several smaller ones.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
“Honiewelle” wrote in message news:xxxxx@ntdev…
I see, but I noticed that some programs can issue higher than that by using SRBs? How do they do that?
On Fri, Jul 22, 2011 at 4:42 PM, wrote:
Many adapters are in effect limited to 128K transfers. Query the STORAGE_ADAPTER_DESCRIPTOR, minding the MaximumTransferLength and MaximumPhysicalPages fields.
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
> I noticed that some programs can issue higher
than that by using SRBs? How do they do that?
There are a variety of tedious techniques that can be performed to raise the limit such as contiguous memory allocation.
PASSTHROUGH commands are not split automatically into the max transfer sized pieces.
But when application are simply doing file IO, their large requests are split appropriately. So don’t worry. If you can’t do over 128k, that’s OK.
Thank you so much guys for clearing that up.
I have one more question though, the program I was doing enables the user to
test multiple devices at the same time and I am worried that my high
resolution timing (i am using QueryPerformance…) might be off if there are
too many threads are running at the same time. So I am wondering if there
are other methods/techniques that will do the timing more accurate.
Honiewelle wrote:
I have one more question though, the program I was doing enables the
user to test multiple devices at the same time and I am worried that
my high resolution timing (i am using QueryPerformance…) might be
off if there are too many threads are running at the same time. So I
am wondering if there are other methods/techniques that will do the
timing more accurate.
What is it that you want to measure? If “too many threads” is a
real-life condition, then that’s exactly what you should be capturing.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
I am measuring how fast the DeviceIOControl returns so that I can get the
time how fast a certain disk command completes.
On Sat, Jul 23, 2011 at 9:26 AM, Tim Roberts wrote:
> Honiewelle wrote:
> >
> > I have one more question though, the program I was doing enables the
> > user to test multiple devices at the same time and I am worried that
> > my high resolution timing (i am using QueryPerformance…) might be
> > off if there are too many threads are running at the same time. So I
> > am wondering if there are other methods/techniques that will do the
> > timing more accurate.
>
> What is it that you want to measure? If “too many threads” is a
> real-life condition, then that’s exactly what you should be capturing.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
DeviceIoControl will return an immediate true/false, which most likely is not what you want. You want to know when the IO completes, not when DeviceIoControl returns.
Gary G. Little
C 952-454-4629
H 952-223-1349
On Jul 22, 2011, at 20:31, Honiewelle wrote:
> I am measuring how fast the DeviceIOControl returns so that I can get the time how fast a certain disk command completes.
>
> On Sat, Jul 23, 2011 at 9:26 AM, Tim Roberts wrote:
> Honiewelle wrote:
> >
> > I have one more question though, the program I was doing enables the
> > user to test multiple devices at the same time and I am worried that
> > my high resolution timing (i am using QueryPerformance…) might be
> > off if there are too many threads are running at the same time. So I
> > am wondering if there are other methods/techniques that will do the
> > timing more accurate.
>
> What is it that you want to measure? If “too many threads” is a
> real-life condition, then that’s exactly what you should be capturing.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
That’s likely not my case since I am not using DeviceIOControl in
asynchronous mode. But in case if what you said was true, can you please
recommend
a technique or a workaround to get the completion time of a certain the ATA
IO command?
On Sun, Jul 24, 2011 at 4:59 AM, Gary G. Little wrote:
> DeviceIoControl will return an immediate true/false, which most likely is
> not what you want. You want to know when the IO completes, not when
> DeviceIoControl returns.
>
> Gary G. Little
> C 952-454-4629
> H 952-223-1349
>
>
> On Jul 22, 2011, at 20:31, Honiewelle wrote:
>
> I am measuring how fast the DeviceIOControl returns so that I can get the
> time how fast a certain disk command completes.
>
> On Sat, Jul 23, 2011 at 9:26 AM, Tim Roberts <
> xxxxx@probo.com> wrote:
>
>> Honiewelle wrote:
>> >
>> > I have one more question though, the program I was doing enables the
>> > user to test multiple devices at the same time and I am worried that
>> > my high resolution timing (i am using QueryPerformance…) might be
>> > off if there are too many threads are running at the same time. So I
>> > am wondering if there are other methods/techniques that will do the
>> > timing more accurate.
>>
>> What is it that you want to measure? If “too many threads” is a
>> real-life condition, then that’s exactly what you should be capturing.
>>
>> –
>> Tim Roberts, xxxxx@probo.com
>> Providenza & Boekelheide, Inc.
>>
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http:http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http:
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http:
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
></http:></http:></http:>
Sigh, don’t worry about Gary’s post. You on the other hand already understand the principles of operation. Gary has what appears to be an uncontrollable urge to participate in storage related topics, littering them with questions, incorrect information, and off topic junk. I am sure Gary will have a cow for me pointing this out as he also has a very short fuse. But he is welcome to keep an on-topic, technical discussion and explain to us all how a scsi pass through DeviceIoControl returns this “immediate” true/false before the device finishes processing it in the device specified manner (rofl).
Whether or not you are doing asynchronous IO to your given media does not mean that async IO isn’t being done by another thread somewhere in the system, and that your request has possibly been “queued” up with dozens of other requests running rampant through the system. Hence, the true/false return from DeviceIoControl, ReadFile, or WriteFile is not an adequate indication that your IO has completed. All you can swear to after checking GetLastError() is that the parameters of your IO request have passed muster and been accepted by the system. Your IO may or may not even be started when you get the true/false return from your IO request. You cannot KNOW, with a certainty, that your IO has completed until you have checked GetLastError(), GetOverlappedResult(), or waited on the event and checked the status.
By getting a benchmark time before DeviceIoControl() and then checking the time duration AFTER the return from DeviceIoControl(), all you have done is check how long it takes the system to accept your request. I don’t thank that is what you want to do.
> You cannot KNOW, with a certainty, that your IO has
completed until you have checked GetLastError()
Interesting. So can you explain how the spti sample in the wdk manages to work, because it doesn’t do that? Since the spti code is intended for developers, reconciling this issue would be of interest to a lot of people.
> Interesting. So can you explain how the spti sample in the wdk manages to work
SPTI is always synchronous.
More so, if the file is opened without FILE_FLAG_OVERLAPPED, all IO is also synchronous.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
> -----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-467203-
xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Tuesday, July 26, 2011 10:51 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Sending Disk Commands
> Interesting. So can you explain how the spti sample in the wdk manages
to work
SPTI is always synchronous.
That’s what we thought. We had reused some code from an async API with
SPTI, so the handle was opened FILE_FLAG_OVERLAPPED, and the OVERLAPPED
structure had an event in it. It worked synchronously until a Window XP
hotfix or SP changed it. I think it was XP SP3, but I wouldn’t testify to
that in court. We changed the code to not assume that it was synchronous
any more. 
More so, if the file is opened without FILE_FLAG_OVERLAPPED, all IO is
also synchronous.
We found this to be true, even when the above happened.
Phil
Philip D. Barila
Yeah, pass through is synchronous, and synchronous IO requests do not return until IO is complete .
Thanks Sven for so graciously pointing out my error.
Gary G. Little
----- Original Message -----
From: “Maxim S. Shatskih”
To: “Windows System Software Devs Interest List”
Sent: Tuesday, July 26, 2011 11:50:47 PM
Subject: Re:[ntdev] Sending Disk Commands
> Interesting. So can you explain how the spti sample in the wdk manages to work
SPTI is always synchronous.
More so, if the file is opened without FILE_FLAG_OVERLAPPED, all IO is also synchronous.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>> More so, if the file is opened without FILE_FLAG_OVERLAPPED, all IO is
> also synchronous.
We found this to be true, even when the above happened.
This is documented.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
So what do you guys recommend in getting a more accurate i/o timing in a
multi-threaded app? Other than getting the time before and after
deviceiocontrol?
On Wednesday, July 27, 2011, Maxim S. Shatskih
wrote:
>>> More so, if the file is opened without FILE_FLAG_OVERLAPPED, all IO is
>>> also synchronous.
>>
>> We found this to be true, even when the above happened.
>
> This is documented.
>
> –
> Maxim S. Shatskih
> Windows DDK MVP
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>