Virtual Storport Driver Thrashing

Samster · September 3, 2011, 11:44am

Hi

I have a Virtual Storport driver over USB on Windows 7 32-bit. When I copy a file from the disk that my driver exposes, the file is written properly to the target location (on another local drive not controlled by my driver). However, after that, the driver starts receiving an endless stream of calls to write the file (to the SOURCE drive exposed by my Virtual Storport driver).

When I perform the equivalent operations using USBSTOR.sys the file is written and the mysterious writes do not happen. I need this same behavior in my driver.

If I read a 1-byte file the problem does not happen. But, if I read 2-Megabyte file, then the problem happens.

In my driver, after I read a file from the my device’s USB disk and write it to my local c: drive, is see the following:

SysInternals Process Monitor shows and unending series of:

FASTIO_ACQUIRE_FOR_MOD_WRITE
WriteFile Offset: 0 Length 8,192, I?O FLAGS: Non-cached
FASTIO_RELEASE_FOR_MOD_WRITE EndingOffset: 9,851,829,641,261,907,056

… an endless stream of the the above with different lengths and offsets. Obviously that Ending Offset is troubling as I only have a 30 GB disk (incidentally that EndingOffset is 88B8BADC 8B147C70 in hex, suspiciously like two virtual memory addresses)
I see this usually for the file I just read, but sometimes also for other files that exist on that disk.

One one of those calls, I placed a breakpoint in my driver to see who is issuing the write request. This is the stack:

0: kd> k 100
ChildEBP RetAddr
[… my usbdrv.sys code to handle the write]
a2d3f8a8 9ca17634 usbdrv!HwStartIo+0xc8
a2d3f8b8 9ca191d0 storport!RaCallMiniportStartIo+0x1e
a2d3f8e0 9ca1944e storport!RaidAdapterPostScatterGatherExecute+0x156
a2d3f8f0 9ca26ed5 storport!RaidAdapterExecuteXrb+0x2d
a2d3f910 9ca1e3c9 storport!RaUnitStartIo+0xbf
a2d3f950 9ca249bc storport!RaidStartIoPacket+0x10c
a2d3f970 9ca27610 storport!RaidUnitSubmitRequest+0x4e
a2d3f990 9ca1d4f0 storport!RaUnitScsiIrp+0x114
a2d3f9ac 88e1ea51 storport!RaDriverScsiIrp+0x60
WARNING: Stack unwind information not available. Following frames may be wrong.
a2d3f9dc 88e1e40d bustrce9+0x4a51
a2d3f9f8 832514bc bustrce9+0x440d
a2d3fa10 893885a4 nt!IofCallDriver+0x63
a2d3fa20 89387fe8 CLASSPNP!SubmitTransferPacket+0x103
a2d3fa60 89388303 CLASSPNP!ServiceTransferRequest+0x225
a2d3fa88 893883bf CLASSPNP!ClassReadWrite+0x172
a2d3fa9c 832514bc CLASSPNP!ClassGlobalDispatch+0x20
a2d3fab4 88f39230 nt!IofCallDriver+0x63
a2d3fac8 88f392e5 partmgr!PmReadWrite+0x112
a2d3fadc 832514bc partmgr!PmGlobalDispatch+0x1d
a2d3faf4 88f4a9ae nt!IofCallDriver+0x63
a2d3fb10 832514bc volmgr!VmReadWrite+0x1a8
a2d3fb28 89342475 nt!IofCallDriver+0x63
a2d3fb34 89342548 fvevol!FveRequestPassThrough+0x31
a2d3fb50 89342759 fvevol!FveReadWrite+0x4e
a2d3fb80 893427a9 fvevol!FveFilterRundownReadWrite+0x197
a2d3fb90 832514bc fvevol!FveFilterRundownWrite+0x33
a2d3fba8 8940c76e nt!IofCallDriver+0x63
a2d3fc88 8940c8a5 rdyboost!SmdProcessReadWrite+0xa14
a2d3fca8 832514bc rdyboost!SmdDispatchReadWrite+0xcb
a2d3fcc0 895bbfd9 nt!IofCallDriver+0x63
a2d3fce8 895bc2fd volsnap!VolsnapWriteFilter+0x265
a2d3fcf8 832514bc volsnap!VolSnapWrite+0x21
a2d3fd10 8901d94c nt!IofCallDriver+0x63
a2d3fd1c 8328527e Ntfs!NtfsStorageDriverCallout+0x14
a2d3fd1c 83285375 nt!KiSwapKernelStackAndExit+0x15a
8b147828 832a523d nt!KiSwitchKernelStackAndCallout+0x31
8b14789c 8901c952 nt!KeExpandKernelStackAndCalloutEx+0x29d
8b1478c8 8901d5d6 Ntfs!NtfsCallStorageDriver+0x2d
8b14790c 8901c0b9 Ntfs!NtfsMultipleAsync+0x4d
8b147a0c 8901b0c6 Ntfs!NtfsNonCachedIo+0x413
8b147b24 8901c878 Ntfs!NtfsCommonWrite+0x1eed
8b147b9c 832514bc Ntfs!NtfsFsdWrite+0x2e1
8b147bb4 88dbd20c nt!IofCallDriver+0x63
8b147bd8 88dbd3cb fltmgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x2aa
8b147c10 832514bc fltmgr!FltpDispatch+0xc5
8b147c28 8324e811 nt!IofCallDriver+0x63
8b147c3c 832991a1 nt!IoAsynchronousPageWrite+0x1c1
8b147cec 83299364 nt!MiGatherMappedPages+0xa62
8b147d50 83423a55 nt!MiMappedPageWriter+0x13d
8b147d90 832d5239 nt!PspSystemThreadStartup+0x9e
00000000 00000000 nt!KiThreadStartup+0x19

And this is the SRB that is sent to the driver:

0: kd> dt /b _SCSI_REQUEST_BLOCK 87052d1c
CLASSPNP!_SCSI_REQUEST_BLOCK
+0x000 Length : 0x40
+0x002 Function : 0 ‘’
+0x003 SrbStatus : 0 ‘’
+0x004 ScsiStatus : 0 ‘’
+0x005 PathId : 0 ‘’
+0x006 TargetId : 0 ‘’
+0x007 Lun : 0 ‘’
+0x008 QueueTag : 0xfc ‘’
+0x009 QueueAction : 0x20 ’ ’
+0x00a CdbLength : 0xa ‘’
+0x00b SenseInfoBufferLength : 0x14 ‘’
+0x00c SrbFlags : 0x40200382
+0x010 DataTransferLength : 0x200000
+0x014 TimeOutValue : 0x3c
+0x018 DataBuffer : 0xa3600000
+0x01c SenseInfoBuffer : 0x866f0230
+0x020 NextSrb : (null)
+0x024 OriginalRequest : 0x866f00b0
+0x028 SrbExtension : 0x866f0330
+0x02c InternalStatus : 0x5ebb90
+0x02c QueueSortKey : 0x5ebb90
+0x02c LinkTimeoutValue : 0x5ebb90
+0x030 Cdb : “*”
[00] 0x2a ‘*’
[01] 0 ‘’
[02] 0 ‘’
[03] 0x5e ‘^’
[04] 0xbb ‘’
[05] 0x90 ‘’
[06] 0 ‘’
[07] 0x10 ‘’
[08] 0 ‘’
[09] 0 ‘’
[10] 0 ‘’
[11] 0 ‘’
[12] 0 ‘’
[13] 0 ‘’
[14] 0 ‘’
[15] 0 ‘’

Might someone be able help me understand how to fix this?

Igor_Sharovar · September 3, 2011, 5:24pm

The stack shows that system writes data to page file, which is likely allocated in your driver. Check the system property to see the location of the page file.

Igor Sharovar

Samster · September 4, 2011, 2:16am

I am not sure if you mean if I am working for MCCI, but if so, the answer is
no.

Subject: RE: About virtual storport miniport driver
From: workingmailing@163.com
Date: Sat, 3 Sep 2011 14:38:22 -0400 (EDT)
X-Message-Number: 8

thank you.

Are you a member of MCCI?

Samster · September 4, 2011, 10:19am

Hi, Igor,

Yes, I also suspected that there is some kind of paging operations going on.
The page file should be on the c: drive of my boot drive not on the external
drive I just attached.

When I unhide all system files, I see $RECYCLE.BIN and “System Volume
Information” as well as two files I wrote on the disk earlier.

When I look at the Virtual Memory settings, I see that “None” is listed as
in the Paging File Size column for my external drive.

I agree that there is some kind of memory management thrashing going on but
I don’t see any reason for it. Before I read from a file on the disk, there
is no activity except for the activity generated by opening the disk in
explorer.

You said to check a “System Property” below. What property do you mean
exactly? Also, I am not sure what you mean that the “page file is allocated
in my driver”. Do you mean that I am allocating memory that is paged? All
memory I allocate is from NonPagedPool.

Alex_Grig · September 4, 2011, 11:52am

The delayed writes to a cached file are handled by MM, and look about the same as a dirty page(s) flush.

I think System monitor shows EndingOffset incorrectly.

How do you complete these requests? Do you send whole 2MB URBs to USB? Do you check maximum transfer size of the USB device? Do you ever return DataTransferLength different from the requested length in the SRB?

Samster · September 4, 2011, 5:27pm

Hi,

>> How do you complete these requests?
I receive them as I would any other request. The driver submits them to
card and they appear to be processed by the device successfully, so I return
STATUS_SRB_SUCCESS after setting the DataTransferLength as I would with any
other requests.

>> Do you send whole 2MB URBs to USB?
I send the whole buffer to the underlying USB stack. From what I have seen
everything is processed correctly.

>> Do you check maximum transfer size of the USB device? I did not check
this specifically…I did do a couple experiments to reduce sizes but they
seemed to make no difference. I have been trying to avoid the submission of
the writes to the device. I will look into this more, however.

>> Do you ever return DataTransferLength different from the requested
length in the SRB?
I would only do this in the case when USB indicates that there was a Short
Transfer, but I don’t think this is happening.

Two more things:

When I make my device READ-ONLY by intercepting the
MODE_PAGE_DEVICE_CAPABILITIES and setting MODE_DSP_WRITE_PROTECT in
DeviceSpecificPararameters of MODE_PARAMETER_HEADER, those writes to not
happen (but that I cannot write to the device). In this case, the writes are
never submitted to my driver.
I noticed that the writes start happening while I am reading the
file…They don’t start happening after I finish reading.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@broadcom.com
Sent: Sunday, September 04, 2011 8:51 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Virtual Storport Driver Thrashing

The delayed writes to a cached file are handled by MM, and look about the
same as a dirty page(s) flush.

I think System monitor shows EndingOffset incorrectly.

How do you complete these requests? Do you send whole 2MB URBs to USB? Do
you check maximum transfer size of the USB device? Do you ever return
DataTransferLength different from the requested length in the SRB?

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Alex_Grig · September 4, 2011, 8:23pm

Are volume snapshots (“previous versions”) enabled on the volume?

Samster · September 4, 2011, 10:38pm

Hi,

I have corrected the issue of massive spurious writes by page mapping
manager when reading from external device using Virtual Storport driver.

The fix was:

Use:
status = StorPortGetOriginalMdl( DeviceExtension,
Srb,
( PVOID* )Mdl );

Instead of:

status = StorPortGetSystemAddress( DeviceExtension,
Srb,
&Srb->DataBuffer );

in order to get the address from the SRB READ/WRITE buffers.

I would be interested knowing why using StorPortGetSystemAddress() causes
such havoc. I have probably read everything about Virtual Storport that
exists and it was not clear to me that I should use one function over
another. It seems like StorPortGetSystemAddress allocates some resources to
do its work, but there is no way, it seems, to deallocate them?
(StorPortGetOriginalMdl probably just reuses an existing MDL?..not sure.)

Thank you the people who answered my queries.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@broadcom.com
Sent: Sunday, September 04, 2011 5:22 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Virtual Storport Driver Thrashing

Are volume snapshots (“previous versions”) enabled on the volume?

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Peter_Wieland · September 5, 2011, 10:42am

When reading data do you write to those buffers in your driver? Or do you just hand them down to the USB stack directly?

How were you reading the file? Was it memory mapped, or just normal ReadFile calls?

Did you happen to have driver verified turned on?

-p

On Sep 4, 2011, at 7:37 PM, “Sam Tertzakian” wrote:

> Hi,
>
> I have corrected the issue of massive spurious writes by page mapping
> manager when reading from external device using Virtual Storport driver.
>
> The fix was:
>
> Use:
> status = StorPortGetOriginalMdl( DeviceExtension,
> Srb,
> ( PVOID* )Mdl );
>
> Instead of:
>
> status = StorPortGetSystemAddress( DeviceExtension,
> Srb,
> &Srb->DataBuffer );
>
> in order to get the address from the SRB READ/WRITE buffers.
>
> I would be interested knowing why using StorPortGetSystemAddress() causes
> such havoc. I have probably read everything about Virtual Storport that
> exists and it was not clear to me that I should use one function over
> another. It seems like StorPortGetSystemAddress allocates some resources to
> do its work, but there is no way, it seems, to deallocate them?
> (StorPortGetOriginalMdl probably just reuses an existing MDL?..not sure.)
>
> Thank you the people who answered my queries.
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@broadcom.com
> Sent: Sunday, September 04, 2011 5:22 PM
> To: Windows System Software Devs Interest List
> Subject: RE:[ntdev] Virtual Storport Driver Thrashing
>
> Are volume snapshots (“previous versions”) enabled on the volume?
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>

Samster · September 5, 2011, 6:41pm

>>>>When reading data do you write to those buffers in your driver? Or do
you just hand them down to the USB stack directly?

I was using StorPortGetSystemAddress() and then using the VirtualAddress
field to pass down the buffer to the USB driver. And it worked in that the
data was written and read correctly. But it had that thrashing problem. Also
I noticed that memory usage increased significantly while I was
reading/writing the file. (In the working version, I get the MDL using
StorPortGetOriginalMdl and I pass the MDL directly instead of using the
Virtual Address field.)

>>>How were you reading the file? Was it memory mapped, or just normal
ReadFile calls?

I was using Explorer (drag and drop) to copy a file from the USB drive to
the local SATA drive.

>>>Did you happen to have driver verified turned on?

Yes, it is always on.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Peter Wieland
Sent: Monday, September 05, 2011 7:42 AM
To: Windows System Software Devs Interest List
Cc: Windows System Software Devs Interest List
Subject: Re: [ntdev] Virtual Storport Driver Thrashing

When reading data do you write to those buffers in your driver? Or do you
just hand them down to the USB stack directly?

How were you reading the file? Was it memory mapped, or just normal
ReadFile calls?

Did you happen to have driver verified turned on?

-p

On Sep 4, 2011, at 7:37 PM, “Sam Tertzakian” wrote:

> Hi,
>
> I have corrected the issue of massive spurious writes by page mapping
> manager when reading from external device using Virtual Storport driver.
>
> The fix was:
>
> Use:
> status = StorPortGetOriginalMdl( DeviceExtension,
> Srb,
> ( PVOID* )Mdl );
>
> Instead of:
>
> status = StorPortGetSystemAddress( DeviceExtension,
> Srb,
> &Srb->DataBuffer );
>
> in order to get the address from the SRB READ/WRITE buffers.
>
> I would be interested knowing why using StorPortGetSystemAddress() causes
> such havoc. I have probably read everything about Virtual Storport that
> exists and it was not clear to me that I should use one function over
> another. It seems like StorPortGetSystemAddress allocates some resources
to
> do its work, but there is no way, it seems, to deallocate them?
> (StorPortGetOriginalMdl probably just reuses an existing MDL?..not sure.)
>
> Thank you the people who answered my queries.
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@broadcom.com
> Sent: Sunday, September 04, 2011 5:22 PM
> To: Windows System Software Devs Interest List
> Subject: RE:[ntdev] Virtual Storport Driver Thrashing
>
> Are volume snapshots (“previous versions”) enabled on the volume?
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

OSR_Community_User · September 5, 2011, 8:26pm

This is an intriguing discovery. While it sounds like developers who work hard enough can find a work around to this if they are forced to, this sounds like something Microsoft should fix or at least document.

I never understood the decision that Srb->DataBuffer can never be trusted for a virtual storport miniport. Such miniports must have a valid address every time and for comparison sake scsiport miniports were able to set MapBuffers to get a valid DataBuffer automatically since the beginning of time. So certainly a few steps backwards with the new architecture: first not provdiing a valid DataBuffer, and now an apparent penalty for looking it up using the most straight forward method.

Alex_Grig · September 5, 2011, 10:29pm

“I never understood the decision that Srb->DataBuffer can never be trusted for a virtual storport miniport.”

Unfortunately, the shortcuts that storport makes for a virtual miniport are NOT documented well (if at all), and some of them are just misguided. As it happens often (not only with storport), the IHV developers are left to wander in the dark. While occasionally hitting the corners, very painfully.

Samster · September 6, 2011, 6:10am

Hi,

I modified the OSR Sample to use StorPortGetSystemAddress() instead of using
the StoPortGetOriginalMdl(). And the problem (writes related to memory
caching) I was seeing in my driver did NOT happen.

So, it is possible there is some other variable that is causing this
behavior. It is not necessarily a MSFT bug or lack of documentation (I never
said so…I just reported my findings).

The change to OSR Sample was not difficult and I verified the changed code
paths using breakpoints to make sure. I will try a bit later to see if I can
understand the exact set of circumstances that causes the behavior I saw.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Monday, September 05, 2011 5:25 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Virtual Storport Driver Thrashing

This is an intriguing discovery. While it sounds like developers who work
hard enough can find a work around to this if they are forced to, this
sounds like something Microsoft should fix or at least document.

I never understood the decision that Srb->DataBuffer can never be trusted
for a virtual storport miniport. Such miniports must have a valid address
every time and for comparison sake scsiport miniports were able to set
MapBuffers to get a valid DataBuffer automatically since the beginning of
time. So certainly a few steps backwards with the new architecture: first
not provdiing a valid DataBuffer, and now an apparent penalty for looking it
up using the most straight forward method.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Scott_Noone_OSR · September 6, 2011, 10:22am

The I/O here is coming from the mapped page writer, which means that the
pages backing a memory mapped image are being marked dirty somehow (note
that the I/O is paging I/O, but not paging *file* I/O). This usually happens
for one of two reasons:

Someone writes to a virtual address mapping these buffers
Someone is building an MDL to these pages, locking it
MmProbeAndLockPages, and specifying IoWriteAccess or IoModifyAccess, which
will mark the pages as dirty on unlock (even if they were never written
to!).

Based on your workaround, I would say that #2 is a very likely candidate. If
you submit the incoming MDL to the USB stack they use that, it’s properly
built for read access so everything works OK. If you just send the buffer,
someone (erroneously) builds an MDL for write access even though it’s being
read from and you end up re-writing the source file.

You could probably confirm this by stepping through the target USB driver
and seeing how they build the MDL for the buffer that you sent them.

-scott

–
Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc.
http://www.osronline.com

“Sam Tertzakian” wrote in message news:xxxxx@ntdev…

Hi,

I modified the OSR Sample to use StorPortGetSystemAddress() instead of using
the StoPortGetOriginalMdl(). And the problem (writes related to memory
caching) I was seeing in my driver did NOT happen.

So, it is possible there is some other variable that is causing this
behavior. It is not necessarily a MSFT bug or lack of documentation (I never
said so…I just reported my findings).

The change to OSR Sample was not difficult and I verified the changed code
paths using breakpoints to make sure. I will try a bit later to see if I can
understand the exact set of circumstances that causes the behavior I saw.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Monday, September 05, 2011 5:25 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Virtual Storport Driver Thrashing

This is an intriguing discovery. While it sounds like developers who work
hard enough can find a work around to this if they are forced to, this
sounds like something Microsoft should fix or at least document.

I never understood the decision that Srb->DataBuffer can never be trusted
for a virtual storport miniport. Such miniports must have a valid address
every time and for comparison sake scsiport miniports were able to set
MapBuffers to get a valid DataBuffer automatically since the beginning of
time. So certainly a few steps backwards with the new architecture: first
not provdiing a valid DataBuffer, and now an apparent penalty for looking it
up using the most straight forward method.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Alex_Grig · September 6, 2011, 2:15pm

How do you build your URBs? Do you set correct USBD_TRANSFER_DIRECTION_IN and USBD_TRANSFER_DIRECTION_OUT flag, depending on the transfer type? If you set USBD_TRANSFER_DIRECTION_IN for an OUT (Write) request, the pages may get marked as dirty.

Samster · September 6, 2011, 9:13pm

Hi, Scott,

I have been investigating option #2 below. So, far I have not seen read
buffers mapped as write, but I am not finished with my investigation.

Once I have finished, I will let you know.

Thank you for your time.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Scott Noone
Sent: Tuesday, September 06, 2011 7:23 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Virtual Storport Driver Thrashing

The I/O here is coming from the mapped page writer, which means that the
pages backing a memory mapped image are being marked dirty somehow (note
that the I/O is paging I/O, but not paging *file* I/O). This usually happens

for one of two reasons:

Someone writes to a virtual address mapping these buffers
Someone is building an MDL to these pages, locking it
MmProbeAndLockPages, and specifying IoWriteAccess or IoModifyAccess, which
will mark the pages as dirty on unlock (even if they were never written
to!).

Based on your workaround, I would say that #2 is a very likely candidate. If

you submit the incoming MDL to the USB stack they use that, it’s properly
built for read access so everything works OK. If you just send the buffer,
someone (erroneously) builds an MDL for write access even though it’s being
read from and you end up re-writing the source file.

You could probably confirm this by stepping through the target USB driver
and seeing how they build the MDL for the buffer that you sent them.

-scott

–
Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc.
http://www.osronline.com

“Sam Tertzakian” wrote in message news:xxxxx@ntdev…

Hi,

I modified the OSR Sample to use StorPortGetSystemAddress() instead of using
the StoPortGetOriginalMdl(). And the problem (writes related to memory
caching) I was seeing in my driver did NOT happen.

So, it is possible there is some other variable that is causing this
behavior. It is not necessarily a MSFT bug or lack of documentation (I never
said so…I just reported my findings).

The change to OSR Sample was not difficult and I verified the changed code
paths using breakpoints to make sure. I will try a bit later to see if I can
understand the exact set of circumstances that causes the behavior I saw.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Monday, September 05, 2011 5:25 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Virtual Storport Driver Thrashing

This is an intriguing discovery. While it sounds like developers who work
hard enough can find a work around to this if they are forced to, this
sounds like something Microsoft should fix or at least document.

I never understood the decision that Srb->DataBuffer can never be trusted
for a virtual storport miniport. Such miniports must have a valid address
every time and for comparison sake scsiport miniports were able to set
MapBuffers to get a valid DataBuffer automatically since the beginning of
time. So certainly a few steps backwards with the new architecture: first
not provdiing a valid DataBuffer, and now an apparent penalty for looking it
up using the most straight forward method.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Pavel_Lebedinsky · September 7, 2011, 1:56am

You can find the exact place where a page get marked dirty by setting a write breakpoint on the PFN’s Modified bit:

http://www.osronline.com/ShowThread.cfm?link=159431

This will likely be inside MmUnlockPages as Scott mentioned. If you also set a breakpoint on the ReferenceCount field you can see all lock/unlock operations.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Sam Tertzakian
Sent: Tuesday, September 06, 2011 6:13 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Virtual Storport Driver Thrashing

Hi, Scott,

I have been investigating option #2 below. So, far I have not seen read buffers mapped as write, but I am not finished with my investigation.

Once I have finished, I will let you know.

Thank you for your time.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Scott Noone
Sent: Tuesday, September 06, 2011 7:23 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Virtual Storport Driver Thrashing

The I/O here is coming from the mapped page writer, which means that the pages backing a memory mapped image are being marked dirty somehow (note that the I/O is paging I/O, but not paging *file* I/O). This usually happens

for one of two reasons:

Someone writes to a virtual address mapping these buffers
Someone is building an MDL to these pages, locking it MmProbeAndLockPages, and specifying IoWriteAccess or IoModifyAccess, which will mark the pages as dirty on unlock (even if they were never written to!).

Based on your workaround, I would say that #2 is a very likely candidate. If

you submit the incoming MDL to the USB stack they use that, it’s properly built for read access so everything works OK. If you just send the buffer, someone (erroneously) builds an MDL for write access even though it’s being read from and you end up re-writing the source file.

You could probably confirm this by stepping through the target USB driver and seeing how they build the MDL for the buffer that you sent them.

-scott

–
Scott Noone
Consulting Associate and Chief System Problem Analyst OSR Open Systems Resources, Inc.
http://www.osronline.com

“Sam Tertzakian” wrote in message news:xxxxx@ntdev…

Hi,

I modified the OSR Sample to use StorPortGetSystemAddress() instead of using the StoPortGetOriginalMdl(). And the problem (writes related to memory
caching) I was seeing in my driver did NOT happen.

So, it is possible there is some other variable that is causing this behavior. It is not necessarily a MSFT bug or lack of documentation (I never said so…I just reported my findings).

The change to OSR Sample was not difficult and I verified the changed code paths using breakpoints to make sure. I will try a bit later to see if I can understand the exact set of circumstances that causes the behavior I saw.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Monday, September 05, 2011 5:25 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Virtual Storport Driver Thrashing

This is an intriguing discovery. While it sounds like developers who work hard enough can find a work around to this if they are forced to, this sounds like something Microsoft should fix or at least document.

I never understood the decision that Srb->DataBuffer can never be trusted for a virtual storport miniport. Such miniports must have a valid address every time and for comparison sake scsiport miniports were able to set MapBuffers to get a valid DataBuffer automatically since the beginning of time. So certainly a few steps backwards with the new architecture: first not provdiing a valid DataBuffer, and now an apparent penalty for looking it up using the most straight forward method.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Maxim_S_Shatskih · September 7, 2011, 4:17am

> I have a Virtual Storport driver over USB on Windows 7 32-bit.

If you talk to USB stack using pointer, then the USB stack will build its own MDL, and probably this MDL will be built with IoWrite/ModifyAccess, depending on how USB stack works. Probably it depends on USB transfer direction flags.

And, if you talk to USB using MDLs, then the USB stack will not build any MDLs except probably partial ones off yours, and in this case it will not dirty the pages.

This explains all your experience.

Note that, since MDLs are the primary means of data transport for both storport and USB, I would never ever map them explicitly to system space, since probably this is 100% redundant and the whole stack will not do such a mapping during the whole processing, unless you will do it manually.

So, get the MDL from the SRB and talk to USB using this MDL, this is the best way.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

Samster · September 7, 2011, 7:48am

Hi, Maxim,

Thank you for your notes.

I agree with you about the fact that the Mdl appears to be built
incorrectly. Certainly, it would explain everything. I spent some time to
look into it yesterday per Steve’s post. Although I was not able to do a
full investigation, what I did showed that the Mdl was being built with
IoReadAccess.

Later today, I will try to complete my investigation and come up with some
hard data one way or the other.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Wednesday, September 07, 2011 1:17 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Virtual Storport Driver Thrashing

I have a Virtual Storport driver over USB on Windows 7 32-bit.

If you talk to USB stack using pointer, then the USB stack will build its
own MDL, and probably this MDL will be built with IoWrite/ModifyAccess,
depending on how USB stack works. Probably it depends on USB transfer
direction flags.

And, if you talk to USB using MDLs, then the USB stack will not build any
MDLs except probably partial ones off yours, and in this case it will not
dirty the pages.

This explains all your experience.

Note that, since MDLs are the primary means of data transport for both
storport and USB, I would never ever map them explicitly to system space,
since probably this is 100% redundant and the whole stack will not do such a
mapping during the whole processing, unless you will do it manually.

So, get the MDL from the SRB and talk to USB using this MDL, this is the
best way.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer