No paging-I/O writes for certain networked files?

Well, I thought it was a cardinal rule that for any read/write performed
on a file, a filesystem filter will ALWAYS see at least one paging I/O
or non-cached I/O request for that data. Not so, apparently. I was
playing around with files on a network share and wondered why I was
seeing hundreds of ordinary writes when I copied a file to the share,
but no paging-I/O writes. This is bad since I’m writing an encryption
filter driver, and so my filter MUST always handle a given read/write
once and ONLY once. Not zero times, not twice, but once. To accomplish I
handling paging/non-cached I/O requests only, as suggested in previous
postings to ntfsd.

Eventually I figured out this was because network redirectors like to
set an internal flag called SRVOPEN_FLAG_DONTUSE_WRITE_CACHEING when a
file is opened for write-only, which causes the redirector to send all
writes across the network as soon as it gets them, bypass the NT cache.
This means any layered filter will see the ordinary write request, but
never a corresponding paging-I/O request. To get around this, my filter
now has to forcibly turn every write-only network file open into a
read/write open.

The reason why I’m ranting in public is that seems that I can never know
a-priori whether I will see a read or write request as both paging and
non-paging I/O, or one or the other, for a given filesystem. Instead, I
must special case my code for each filesystem and pray that I’ve covered
every scenario that can result in my not handling a read/write or
handling it twice. The only alternative I can come up with is to force
ALL reads/writes to a filtered file to be non-cached, with the
corresponding performance penalties. Is there an elegant way out of this
mess?

  • Nicholas Ryan

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

But all you really want to filter are the non-cached requests, correct?
Hence, if you look for the single case of non-cached I/O you have
accomplished your task since paging I/O is non-cached and any component
which performs ‘direct-to-media’;i.e. non-cached, I/O, you will also catch.

Pete
Peter Scott
xxxxx@KernelDrivers.com
http://www.KernelDrivers.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Nicholas Ryan
Sent: Sunday, December 30, 2001 10:49 PM
To: File Systems Developers
Subject: [ntfsd] No paging-I/O writes for certain networked files?

Well, I thought it was a cardinal rule that for any read/write performed
on a file, a filesystem filter will ALWAYS see at least one paging I/O or
non-cached I/O request for that data. Not so, apparently. I was playing
around with files on a network share and wondered why I was seeing hundreds
of ordinary writes when I copied a file to the share, but no paging-I/O
writes. This is bad since I’m writing an encryption filter driver, and so my
filter MUST always handle a given read/write once and ONLY once. Not zero
times, not twice, but once. To accomplish I handling paging/non-cached I/O
requests only, as suggested in previous postings to ntfsd.

Eventually I figured out this was because network redirectors like to set
an internal flag called SRVOPEN_FLAG_DONTUSE_WRITE_CACHEING when a file is
opened for write-only, which causes the redirector to send all writes across
the network as soon as it gets them, bypass the NT cache. This means any
layered filter will see the ordinary write request, but never a
corresponding paging-I/O request. To get around this, my filter now has to
forcibly turn every write-only network file open into a read/write open.

The reason why I’m ranting in public is that seems that I can never know
a-priori whether I will see a read or write request as both paging and
non-paging I/O, or one or the other, for a given filesystem. Instead, I must
special case my code for each filesystem and pray that I’ve covered every
scenario that can result in my not handling a read/write or handling it
twice. The only alternative I can come up with is to force ALL reads/writes
to a filtered file to be non-cached, with the corresponding performance
penalties. Is there an elegant way out of this mess?

  • Nicholas Ryan

You are currently subscribed to ntfsd as: xxxxx@KernelDrivers.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

My problem is my filter needs to be able to encrypt files on a network
share while running on the remote computer (not the host). On the
computer accessing the share remotely, the write is only seen once as a
non-paging I/O write, since the network redirector bypasses the NT cache
for write-only files and sends the writes directly across the network. I
think I will solve this by transforming all writes I see made on a
network filesystem into IRP_NOCACHE writes, thus guaranteeing I will see
the write once and only once.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Pete Scott
Sent: Monday, December 31, 2001 6:53 AM
To: File Systems Developers
Subject: [ntfsd] RE: No paging-I/O writes for certain networked files?

But all you really want to filter are the non-cached requests, correct?
Hence, if you look for the single case of non-cached I/O you have
accomplished your task since paging I/O is non-cached and any component
which performs ‘direct-to-media’;i.e. non-cached, I/O, you will also
catch.

Pete
Peter Scott
xxxxx@KernelDrivers.com
http://www.KernelDrivers.com http:</http:>
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Nicholas Ryan
Sent: Sunday, December 30, 2001 10:49 PM
To: File Systems Developers
Subject: [ntfsd] No paging-I/O writes for certain networked files?
Well, I thought it was a cardinal rule that for any read/write performed
on a file, a filesystem filter will ALWAYS see at least one paging I/O
or non-cached I/O request for that data. Not so, apparently. I was
playing around with files on a network share and wondered why I was
seeing hundreds of ordinary writes when I copied a file to the share,
but no paging-I/O writes. This is bad since I’m writing an encryption
filter driver, and so my filter MUST always handle a given read/write
once and ONLY once. Not zero times, not twice, but once. To accomplish I
handling paging/non-cached I/O requests only, as suggested in previous
postings to ntfsd.

Eventually I figured out this was because network redirectors like to
set an internal flag called SRVOPEN_FLAG_DONTUSE_WRITE_CACHEING when a
file is opened for write-only, which causes the redirector to send all
writes across the network as soon as it gets them, bypass the NT cache.
This means any layered filter will see the ordinary write request, but
never a corresponding paging-I/O request. To get around this, my filter
now has to forcibly turn every write-only network file open into a
read/write open.

The reason why I’m ranting in public is that seems that I can never know
a-priori whether I will see a read or write request as both paging and
non-paging I/O, or one or the other, for a given filesystem. Instead, I
must special case my code for each filesystem and pray that I’ve covered
every scenario that can result in my not handling a read/write or
handling it twice. The only alternative I can come up with is to force
ALL reads/writes to a filtered file to be non-cached, with the
corresponding performance penalties. Is there an elegant way out of this
mess?

  • Nicholas Ryan

You are currently subscribed to ntfsd as: xxxxx@KernelDrivers.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@secretseal.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com