Is there a good StorPort driver reference that ties together scatter/gather file operations?

Hi, I would be very grateful if anyone has a good reference for how to use the scatter/gather interface from the MSFT StorPort driver. I am in the process of porting some application code away from a custom driver to a COTS driver. I am able to talk to my end device using SCSI pass through commands and a handle to the device driver, but I have some requirements to be able to reads/writes using scatter/gather. The device driver says it supports Scatter/Gather, but I think I need to use the StorPort driver to perform these operations. I think I should be using the ā€œReadFileScatterā€ and ā€œWriteFileGatherā€ functions, but it would be very helpful to me if anyone knew of a good reference that ties all of this together in one place. Thanks very much in advance!

You probably want this question to be in NTDEV (device driver type questions), not NTFSD (file system and minifilter type questions).

Iā€™ll move it for you.

Peter

1 Like

Iā€™m a bit confused by your question.

Scatter/Gather support has to do with the ability of the device to do ā€œDMA Chainingā€ ā€“ that is, processing I/O operations using a LIST of base-address/length pairs, instead of requiring each base-address and length be submitted to the device as a separate transfer.

The addresses in the base-address/length pairs that weā€™re talking about here can be thought of as ā€œphysical addressesā€ (that arenā€™tā€¦ in fact, they are device bus logical addresses and subject to MMIO relocationā€¦ but whatever) .

ReadFileScatter and WriteFileGather are user-mode APIs that allow you to specify a list of user-mode (virtual address) buffers to be used as the source or target of an I/O operation. This might trigger the use of scatter/gather I/O in a given driverā€¦ or it might not. Support for ReadFileScatter and WriteFileGather does not require a device driver that supports scatter/gather.

These two things are similar concepts (there is a list of elements to read/write as opposed to a single base address and length)ā€¦ but they are not linked. Theyā€™re also in different address spaces, and represent different fundamental activities.

Does that helpā€¦ at all?

Peter

1 Like

In this question, does COTS mean Commercial Off The Shelf?

If so, then I assume that the code that you are working on will be a user mode application and not a driver?

The ReadFileScatter and WriteFileGather UM APIs were essentially custom made for MS SQL server and the IO pattern it needs to maintain its buffer pool. They are designed to reduce the number of UM / KM transitions and increase the amount of data that each operation can process when the application keeps file data in small memory buffers ā€˜scatteredā€™ throughout the address space that should be read / written together. If your application does that, then use them. If not, then donā€™t.

Using ReadFileScatter vs ReadFile has no bearing on how the driver will communicate with the hardware. The only thing that changes is the efficiency of getting from UM to KM and tracking the completion if your memory layout meets specific criteria. Unless your application operates at extreme performance levels, the difference is unmeasurable on modern hardware even when it is beneficial to use them

1 Like

Thank you, Peter! Yes to COTS and also application level. I am looking at the ReadFileScatter and WriteFileGather UM-APIs. I think this is the right fit for what I want to do. I appreciate your time and feedback. Thank you!

Just to be clear

The WriteFile UM API accepts a single buffer consisting of continuous virtual address space. If the size of that buffer exceeds 1 page, it is very likely that the physical memory that backs those virtual addresses will be spread around the physical address space. Part of the ordinary job of drivers is to describe those discontinuous physical addresses to devices in a way that the device can understand. The device always has to work with bus local addresses. If it can perform DMA only one ā€˜blockā€™ at a time, then that is how it must be done, but devices that can consume data from multiple discontinuous physical addresses together can ā€˜gatherā€™ it as part of a single DMA.

The WriteFileGather UM API accepts an array of buffers in virtual address space. Each of these buffers is at least 1 page in size and are always even multiples of the page size. These buffers represent logically continuous data that is not continuous in UM virtual address space and may be even more fragmented in physical memory. In simple terms, the job that the driver has to do with this set of buffers is no different than the job it has to do with the ordinary WriteFile buffer.

Obviously I am skipping many details, but the point is that you cannot control driver behaviour by choosing one API over the other. Both APIs need scatter / gather DMA if sufficiently large IRPs are sent and the hardware will support it. The scatter / gather APIs are useful only when there is a specific layout of memory in UM virtual address space and you want to get it to or from KM faster. And on modern hardware, the ā€˜fasterā€™ is very slightly faster if at all unless there are very specific conditions

Thank you MBond2. This is most helpful. I think I might be able to get by with ReadFile and WriteFile. After doing some digging around my code base I think I have a better understanding of whatā€™s happening and why. I really appreciate your time here. Thank you!