minifilters and cached I/O data

Hi,

I have written a minifilter driver to monitor all the IRPs printing out
the first n bytes of each IRP’s associated data to a log. Everything
seems to be fine, except with memory mapped I/O. I have read previous
posts about notepad and the like, and I am still stumped.

What I am seeing are a bunch of IRPs that are associated with the Cache
Manager and some NTFS reads/writes for this file (correlated with the
FsContext of the file object), but I believe the minifilter does not see
the data when the Cache Manager performs any of the reads or writes to
the disk for a memory mapped file. I believe I am missing the reads
because of earlier IRPs I see on the directory of the memory mapped file
in question (e.g. IRP_MJ_DIRECTORY_CONTROL). However, I am unsure as to
why I am missing the writes.

I can see all the I/O I think I should be seeing except this memory
mapped I/O. I verified this by running XP in a virtual machine on Linux
and logging data being written in Linux. I can see that the data that
is memory mapped in Windows is being written in the Linux trace data,
but the minifilter in XP always misses it.

Can a minifilter see the data from the cache manager? Would I instead
need to implement a disk driver like diskperf to see this data?

Any help would be much appreciated. Thanks,
Nate

Then there is a bug in your mini-filter. All I/O, including paging I/O,
goes through your mini-filter, unless you tell filter manager to do
otherwise.

Can you confirm that you are NOT setting the “skip paging I/O” option in
your mini-filter (e.g., FLTFL_OPERATION_REGISTRATION_SKIP_PAGING_IO)?
That’s the only obvious reason I could think of that you wouldn’t see
these I/O operations.

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Nathanael Paul
Sent: Friday, July 28, 2006 1:21 AM
To: ntfsd redirect
Subject: [ntfsd] minifilters and cached I/O data

Hi,

I have written a minifilter driver to monitor all the IRPs printing out
the first n bytes of each IRP’s associated data to a log. Everything
seems to be fine, except with memory mapped I/O. I have read previous
posts about notepad and the like, and I am still stumped.

What I am seeing are a bunch of IRPs that are associated with the Cache
Manager and some NTFS reads/writes for this file (correlated with the
FsContext of the file object), but I believe the minifilter does not see

the data when the Cache Manager performs any of the reads or writes to
the disk for a memory mapped file. I believe I am missing the reads
because of earlier IRPs I see on the directory of the memory mapped file

in question (e.g. IRP_MJ_DIRECTORY_CONTROL). However, I am unsure as to

why I am missing the writes.

I can see all the I/O I think I should be seeing except this memory
mapped I/O. I verified this by running XP in a virtual machine on Linux

and logging data being written in Linux. I can see that the data that
is memory mapped in Windows is being written in the Linux trace data,
but the minifilter in XP always misses it.

Can a minifilter see the data from the cache manager? Would I instead
need to implement a disk driver like diskperf to see this data?

Any help would be much appreciated. Thanks,
Nate


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Ok,

This sounds good. I had read in the Nagar book that the flags set on
the IRP should be NOCACHE and PAGING_IO, but I was unsure if the
minifilter (modified Minispy) would catch it. I checked for the
FLTFL_OPERATION_REGISTRATION_SKIP_PAGING_IO, and I didn’t see it.

Thank you for the help. I am going to keep trying to debug the
minifilter now that I know it’s a bug there.
Nate

Tony Mason wrote:

Then there is a bug in your mini-filter. All I/O, including paging I/O,
goes through your mini-filter, unless you tell filter manager to do
otherwise.

Can you confirm that you are NOT setting the “skip paging I/O” option in
your mini-filter (e.g., FLTFL_OPERATION_REGISTRATION_SKIP_PAGING_IO)?
That’s the only obvious reason I could think of that you wouldn’t see
these I/O operations.

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Nathanael Paul
Sent: Friday, July 28, 2006 1:21 AM
To: ntfsd redirect
Subject: [ntfsd] minifilters and cached I/O data

Hi,

I have written a minifilter driver to monitor all the IRPs printing out
the first n bytes of each IRP’s associated data to a log. Everything
seems to be fine, except with memory mapped I/O. I have read previous
posts about notepad and the like, and I am still stumped.

What I am seeing are a bunch of IRPs that are associated with the Cache
Manager and some NTFS reads/writes for this file (correlated with the
FsContext of the file object), but I believe the minifilter does not see

the data when the Cache Manager performs any of the reads or writes to
the disk for a memory mapped file. I believe I am missing the reads
because of earlier IRPs I see on the directory of the memory mapped file

in question (e.g. IRP_MJ_DIRECTORY_CONTROL). However, I am unsure as to

why I am missing the writes.

I can see all the I/O I think I should be seeing except this memory
mapped I/O. I verified this by running XP in a virtual machine on Linux

and logging data being written in Linux. I can see that the data that
is memory mapped in Windows is being written in the Linux trace data,
but the minifilter in XP always misses it.

Can a minifilter see the data from the cache manager? Would I instead
need to implement a disk driver like diskperf to see this data?

Any help would be much appreciated. Thanks,
Nate


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

How do you print the data to the log? I had a bug where reads were not caught
because I used FltDoCompletionProcessingWhenSafe in paging I/O path, and of course,
the log would never show those reads. (I keep telling myself RTFM;-)

Nathanael Paul wrote:

Ok,

This sounds good. I had read in the Nagar book that the flags set on
the IRP should be NOCACHE and PAGING_IO, but I was unsure if the
minifilter (modified Minispy) would catch it. I checked for the
FLTFL_OPERATION_REGISTRATION_SKIP_PAGING_IO, and I didn’t see it.

Thank you for the help. I am going to keep trying to debug the
minifilter now that I know it’s a bug there.


Kind regards, Dejan M.
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.

Ah,

I’m using a userspace process that receives the logs of IRPs and then
performs direct I/O to a log. I did have a bug with that some time
earlier (not writing on an aligned 512 bytes), but I believe that is ok
now. I’ll keep debugging.

Thank you for the suggestion,
Nate

Dejan Maksimovic wrote:

How do you print the data to the log? I had a bug where reads were not caught
because I used FltDoCompletionProcessingWhenSafe in paging I/O path, and of course,
the log would never show those reads. (I keep telling myself RTFM;-)

Nathanael Paul wrote:

> Ok,
>
> This sounds good. I had read in the Nagar book that the flags set on
> the IRP should be NOCACHE and PAGING_IO, but I was unsure if the
> minifilter (modified Minispy) would catch it. I checked for the
> FLTFL_OPERATION_REGISTRATION_SKIP_PAGING_IO, and I didn’t see it.
>
> Thank you for the help. I am going to keep trying to debug the
> minifilter now that I know it’s a bug there.


Kind regards, Dejan M.
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.

Thanks to everyone for the help. The problem was that I was not
allocating enough memory to track all the IRPs. When a call to memory
allocation failed, I was not logging the issued IRP causing me to miss
the data I thought I should be seeing.

Nate

Nathanael Paul wrote:

Ah,

I’m using a userspace process that receives the logs of IRPs and then
performs direct I/O to a log. I did have a bug with that some time
earlier (not writing on an aligned 512 bytes), but I believe that is ok
now. I’ll keep debugging.

Thank you for the suggestion,
Nate

Dejan Maksimovic wrote:
> How do you print the data to the log? I had a bug where reads were
> not caught
> because I used FltDoCompletionProcessingWhenSafe in paging I/O path,
> and of course,
> the log would never show those reads. (I keep telling myself RTFM;-)
>
> Nathanael Paul wrote:
>
>> Ok,
>>
>> This sounds good. I had read in the Nagar book that the flags set on
>> the IRP should be NOCACHE and PAGING_IO, but I was unsure if the
>> minifilter (modified Minispy) would catch it. I checked for the
>> FLTFL_OPERATION_REGISTRATION_SKIP_PAGING_IO, and I didn’t see it.
>>
>> Thank you for the help. I am going to keep trying to debug the
>> minifilter now that I know it’s a bug there.
>
> –
> Kind regards, Dejan M.
> http://www.alfasp.com E-mail: xxxxx@alfasp.com
> Alfa Transparent File Encryptor - Transparent file encryption services.
> Alfa File Protector - File protection and hiding library for Win32
> developers.
> Alfa File Monitor - File monitoring library for Win32 developers.
>
>
>