about scatter/gather storport miniport driver

NTDEV
1

greetings,

I need to build a storport miniport driver for a non-scsi(memory) device. What is strange to me now is that the port driver always sends me scatter/gather list with some rather small memory blocks (for read/write scsi commands). In most cases, the length of a physical address range in a scatter/gather list element is 4K bytes. As a large continious physical memory range can highly improve performance of my driver, is there any way to told the port driver allocate large memory for me?

thanks!

2

I’m not sure I understand your question, but if you’re asking:

“I’m seeing a lot of 4K reads and writes… Is there any way to tell the system to send me larger read/write requests?”

then the answer is no, sorry. Welcome to the Wonderful World of Windows.

OTOH, you CAN get rather LARGE requests as well… so it’s not like you’ll ONLY get 4K reads/writes. This is especially true on Vista.

Hope that helps,

Peter
OSR

3

Not really. The port driver isn’t “allocating” this memory for you. It’s taking the buffers provided by the client application and getting the physical addresses for that buffer. Physical memory is generally very fragmented and there is little chance that you’ll get a physically contiguous request.

You could ask for several pages of uncached extension and use that for your read/write operations then copy back to the client buffers. This is much more expensive in terms of CPU but if your device is very bad with fragmented buffers you might get better overall performance with this technique. Just keep in mind that programs rarely do disk I/O for the fun of it - they tend to want some CPU left over for processing.

-p

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Patrick
Sent: Thursday, April 10, 2008 7:57 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] about scatter/gather storport miniport driver

greetings,

I need to build a storport miniport driver for a non-scsi(memory) device. What is strange to me now is that the port driver always sends me scatter/gather list with some rather small memory blocks (for read/write scsi commands). In most cases, the length of a physical address range in a scatter/gather list element is 4K bytes. As a large continious physical memory range can highly improve performance of my driver, is there any way to told the port driver allocate large memory for me?

thanks!

[http://proimg.163.com/channel01/4251_10050_0409.jpg]

???Ϊ???˼???http:
?ð??˻ᡢ?ݳ???Ʊhttp:

— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</http:></http:>

4

Hmmmm… If I understand what you’re saying, you’re suggesting he could delay some of his writes (for example), in the hopes that the following writes will be contiguous??

Aren’t there some real problems with this? First, it’s hard to know how long you delay the write in the hope of getting another, contiguous, operation. Do you wait to get the NEXT write and agglomerate it if it’s contiguous? Do you wait for TWO more writes? What if those writes take a long time to arrive?

Also, in this scheme aren’t you risking data loss in the event of system failure? You’ve delayed the write, the system crashes… the file system thinks the data is committed to disk, you’re screwed. No??

Am I on the right track here, or has the sudden warm weather here in New England caused me to totally lose my mind and miss your point entirely?

Peter
OSR

5

You read it wrong.

I’m suggesting something more like what NDIS drivers do to consolidate highly fragmented packets before sending. Map the I/O, copy the data into common buffer and then issue one physically contiguous write (or map the I/O, issue one physically contiguous read then copy the data from the common buffer). It introduces resource contention around the shared common buffer, but if fragmented operations are really very slow on this device then it may overall be faster.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Thursday, April 10, 2008 11:50 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] about scatter/gather storport miniport driver

Hmmmm… If I understand what you’re saying, you’re suggesting he could delay some of his writes (for example), in the hopes that the following writes will be contiguous??

Aren’t there some real problems with this? First, it’s hard to know how long you delay the write in the hope of getting another, contiguous, operation. Do you wait to get the NEXT write and agglomerate it if it’s contiguous? Do you wait for TWO more writes? What if those writes take a long time to arrive?

Also, in this scheme aren’t you risking data loss in the event of system failure? You’ve delayed the write, the system crashes… the file system thinks the data is committed to disk, you’re screwed. No??

Am I on the right track here, or has the sudden warm weather here in New England caused me to totally lose my mind and miss your point entirely?

Peter
OSR

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

6

That is what I thought. The performance penalty for demanding the system
va’s for every read/write SRB plus the copy operation cost is likely to
overwhelm the benefit of consolidation.

On Thu, Apr 10, 2008 at 5:21 PM, Peter Wieland <
xxxxx@windows.microsoft.com> wrote:

You read it wrong.

I’m suggesting something more like what NDIS drivers do to consolidate
highly fragmented packets before sending. Map the I/O, copy the data into
common buffer and then issue one physically contiguous write (or map the
I/O, issue one physically contiguous read then copy the data from the common
buffer). It introduces resource contention around the shared common buffer,
but if fragmented operations are really very slow on this device then it may
overall be faster.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:
xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Thursday, April 10, 2008 11:50 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] about scatter/gather storport miniport driver

Hmmmm… If I understand what you’re saying, you’re suggesting he could
delay some of his writes (for example), in the hopes that the following
writes will be contiguous??

Aren’t there some real problems with this? First, it’s hard to know how
long you delay the write in the hope of getting another, contiguous,
operation. Do you wait to get the NEXT write and agglomerate it if it’s
contiguous? Do you wait for TWO more writes? What if those writes take a
long time to arrive?

Also, in this scheme aren’t you risking data loss in the event of system
failure? You’ve delayed the write, the system crashes… the file system
thinks the data is committed to disk, you’re screwed. No??

Am I on the right track here, or has the sudden warm weather here in New
England caused me to totally lose my mind and miss your point entirely?

Peter
OSR

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

–
Mark Roddy

7

Hi Peter,

of course your approach will work.In fact, I built two version of my miniport port driver, one of which uses scsi-miniport schematic and the other one uses storport-miniport schematic. What confused me now is that scsi-port driver can send up to 28K bytes contiguous physical memory to my scsi-miniport driver (and scsi-port driver seldom sends small memory like 4K or 8K bytes when read/write large files), but storport driver always send small memory blocks to my storport-miniport driver. So I’m wondering if I make some mistakes in choosing parameters for the miniport driver. Is it just by design?

thanks!

2008-04-11??“Peter Wieland” xxxxx@windows.microsoft.com:

You read it wrong.

I’m suggesting something more like what NDIS drivers do to consolidate highly fragmented packets before sending. Map the I/O, copy the data into common buffer and then issue one physically contiguous write (or map the I/O, issue one physically contiguous read then copy the data from the common buffer). It introduces resource contention around the shared common buffer, but if fragmented operations are really very slow on this device then it may overall be faster.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Thursday, April 10, 2008 11:50 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] about scatter/gather storport miniport driver

Hmmmm… If I understand what you’re saying, you’re suggesting he could delay some of his writes (for example), in the hopes that the following writes will be contiguous??

Aren’t there some real problems with this? First, it’s hard to know how long you delay the write in the hope of getting another, contiguous, operation. Do you wait to get the NEXT write and agglomerate it if it’s contiguous? Do you wait for TWO more writes? What if those writes take a long time to arrive?

Also, in this scheme aren’t you risking data loss in the event of system failure? You’ve delayed the write, the system crashes… the file system thinks the data is committed to disk, you’re screwed. No??

Am I on the right track here, or has the sudden warm weather here in New England caused me to totally lose my mind and miss your point entirely?

Peter
OSR

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer