I think this has been discussed several times here,
so your problem is probably exactly the same like
the other posters had.
Let me guess.
Your driver is not loaded at boot time. The file you are
trying to open has been loaded into system cache far sooner
than your driver can catch the I/O request.
Is that true ?
L.
I’m checking Irp->Flags.
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gunko Andrey
Sent: Wednesday, February 21, 2007 10:41 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Missing READ IOs in filter driver
One stupid question: where are you watching those flags? The right place is
Irp->Flags, the pointer to irp stack not the current stack location.
Andrey Gunko
soft Xpansion Ukraine Ltd.
Programmer
Powered by eKnow-how
Artjoma St. 118B … 83048 Donetsk … Tel/Fax: +38 062 3818874 …
Internet: [http: www.soft-xpansion.com]
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gregory Dardyk
Sent: Wednesday, February 21, 2007 10:18 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Missing READ IOs in filter driver
Andrey,
Thanks for the tip. That’s exactly what my driver does. The question now is
whether I missed something and it is possible to read data without paging
IOs or I got everything right and there is just a bug in the code.
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gunko Andrey
Sent: Wednesday, February 21, 2007 9:54 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Missing READ IOs in filter driver
NTFS uses stream file object to handle paging I/O. For this FO
there are no IRP_MJ_CREATE request. This FO is created with
IoCreateStreamFileObject/IoCreateStreamFileObjectLite. For original FO and
stream FO uses the same fscontext. So you must save fscontext for the
original FO in IRP_MJ_CREATE to be able to determine paging I/O for that
file with stream file object.
Andrey Gunko
soft Xpansion Ukraine Ltd.
Programmer
Powered by eKnow-how
Artjoma St. 118B … 83048 Donetsk … Tel/Fax: +38 062 3818874 …
Internet: [http: www.soft-xpansion.com]
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gregory Dardyk
Sent: Wednesday, February 21, 2007 9:37 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Missing READ IOs in filter driver
David,
Thanks for your reply. The three bits you are talking about must be
IRP_PAGING_IO, IRP_NOCACHE and IRP_SYNCHRONOUS_PAGING_IO that I’ve listed in
the original post. These are exactly the requests that I am filtering in my
driver. The problem is that I never get a READ with any of these flags,
neither during CreateFile() nor during ReadFile(). I cannot think of a way
to load the data from the storage device to the cache without sending a READ
request with one of the three flags.
Thanks,
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of David Craig
Sent: Wednesday, February 21, 2007 9:12 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Missing READ IOs in filter driver
Non-cached is not necessarily the same as paging IO. Non-cached is
requested to not be cached, while paging IO is the IO that will read/write
the data to the media from the cache. I guess I will try to break it out,
but this is complex and easily confused.
1. CreateFile() creates context info in the file system driver. It might
also initiate reads via the cache as the OS desires, but who cares?
2. ReadFile() goes to the FSD which then sends it to the CacheManager.
When the CacheManager has the data it returns.
3. IRP_MJ_READ goes to the FSD from the CacheManager and then down the
storage stack where the data is located. This is where you decrypt unless
the file was opened with no buffering. Some FSDs can in some cases never
honor the no buffering request. NTFS and compressed data is one case.
In #2 above, the ReadFile() can arrive as a FastIo or an IRP_MJ_READ. If a
file has been opened before and the FSD and CacheManager knows it, then the
CacheManager can provide the data without any more storage stack IOs. If it
was decrypted before by a file system filter driver (FSFD) it will remain
decrypted in the cache.
Why do you care if the file is cached or not? You only care if the request
is going to the storage. There are flags in the IRPs that indicate they
will go there. I think there are three bits, but I forget since I don’t do
encryption any more.
“Gregory Dardyk” wrote in message news:xxxxx@ntfsd…
Thanks, Tony.
I went through the archives of this list and noticed that the usual
recommendation for encryption drivers was to filter non-cached IOs only. So
I assumed that all the data is at some time read/written from/to the disk
using non-cached IOs. Now I’m trying to understand what are the cases when
this assumption is wrong. From Rajeev Nagar’s book I understood that the
file system drivers usually first try to read the data from the cache,
causing a page fault if the data is not there. Now it appears that sometimes
a shortcut is taken. Is there a way to recognize a case when the file system
will skip the cache? My first guess is that I can check in the FILE_OBJECT
whether the cache is initialized for the file. Is this correct?
Thanks a lot,
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Tuesday, February 20, 2007 8:35 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Missing READ IOs in filter driver
Absolutely. The read is cached from the application, the data is already in
the cache and thus you do not see any non-cached operations because they
aren’t necessary.
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com</http:></http:>
Gregory,
As Ladislav pointed out, you are most likely missing the paging requests
because you are not loaded when they are occurring. There is no ‘short cut’
to get the data into cache; paging requests are how it is handled on the
read side.
The reason why the file may be getting pulled into cache prior to your
driver loading has several answers but the most common is prefetch. I would
ensure your driver is a boot loaded driver. If it is, then you are missing
the paging requests probably because of your context tracking problems.
By the way, in the original post you used the word ‘AND’ when describing the
bits. I assume this is simply a mis-type and that you are actually checking
for the ‘OR’ of these bits? Specifically (IRP_PAGING_IO |
IRP_SYNCHRONOUS_PAGING_IO); as Tony indicated at a recent plugfest, the
paging bits WITHOUT the IRP_NOCACHE bit set would be an interesting, but
never encountered, case.
Pete
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
(303)546-0300
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gregory Dardyk
Sent: Wednesday, February 21, 2007 2:18 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Missing READ IOs in filter driver
Andrey,
Thanks for the tip. That’s exactly what my driver does. The question now is
whether I missed something and it is possible to read data without paging
IOs or I got everything right and there is just a bug in the code.
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gunko Andrey
Sent: Wednesday, February 21, 2007 9:54 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Missing READ IOs in filter driver
NTFS uses stream file object to handle paging I/O. For this FO
there are no IRP_MJ_CREATE request. This FO is created with
IoCreateStreamFileObject/IoCreateStreamFileObjectLite. For original FO and
stream FO uses the same fscontext. So you must save fscontext for the
original FO in IRP_MJ_CREATE to be able to determine paging I/O for that
file with stream file object.
Andrey Gunko
soft Xpansion Ukraine Ltd.
Programmer
Powered by eKnow-how
Artjoma St. 118B … 83048 Donetsk … Tel/Fax: +38 062 3818874 …
Internet: [http: www.soft-xpansion.com]
_____
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gregory Dardyk
Sent: Wednesday, February 21, 2007 9:37 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Missing READ IOs in filter driver
David,
Thanks for your reply. The three bits you are talking about must be
IRP_PAGING_IO, IRP_NOCACHE and IRP_SYNCHRONOUS_PAGING_IO that I’ve listed in
the original post. These are exactly the requests that I am filtering in my
driver. The problem is that I never get a READ with any of these flags,
neither during CreateFile() nor during ReadFile(). I cannot think of a way
to load the data from the storage device to the cache without sending a READ
request with one of the three flags.
Thanks,
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of David Craig
Sent: Wednesday, February 21, 2007 9:12 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Missing READ IOs in filter driver
Non-cached is not necessarily the same as paging IO. Non-cached is
requested to not be cached, while paging IO is the IO that will read/write
the data to the media from the cache. I guess I will try to break it out,
but this is complex and easily confused.
1. CreateFile() creates context info in the file system driver. It might
also initiate reads via the cache as the OS desires, but who cares?
2. ReadFile() goes to the FSD which then sends it to the CacheManager.
When the CacheManager has the data it returns.
3. IRP_MJ_READ goes to the FSD from the CacheManager and then down the
storage stack where the data is located. This is where you decrypt unless
the file was opened with no buffering. Some FSDs can in some cases never
honor the no buffering request. NTFS and compressed data is one case.
In #2 above, the ReadFile() can arrive as a FastIo or an IRP_MJ_READ. If a
file has been opened before and the FSD and CacheManager knows it, then the
CacheManager can provide the data without any more storage stack IOs. If it
was decrypted before by a file system filter driver (FSFD) it will remain
decrypted in the cache.
Why do you care if the file is cached or not? You only care if the request
is going to the storage. There are flags in the IRPs that indicate they
will go there. I think there are three bits, but I forget since I don’t do
encryption any more.
“Gregory Dardyk” wrote in message news:xxxxx@ntfsd…
Thanks, Tony.
I went through the archives of this list and noticed that the usual
recommendation for encryption drivers was to filter non-cached IOs only. So
I assumed that all the data is at some time read/written from/to the disk
using non-cached IOs. Now I’m trying to understand what are the cases when
this assumption is wrong. From Rajeev Nagar’s book I understood that the
file system drivers usually first try to read the data from the cache,
causing a page fault if the data is not there. Now it appears that sometimes
a shortcut is taken. Is there a way to recognize a case when the file system
will skip the cache? My first guess is that I can check in the FILE_OBJECT
whether the cache is initialized for the file. Is this correct?
Thanks a lot,
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Tuesday, February 20, 2007 8:35 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Missing READ IOs in filter driver
Absolutely. The read is cached from the application, the data is already in
the cache and thus you do not see any non-cached operations because they
aren’t necessary.
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com</http:>
No, the driver is loaded at boot time.
I think I found the problem. As David mentioned, the FS can read some data
during the create. Since I don’t have the FsContext for a file until the
CREATE request is completed by the FS driver, I cannot associate the READs
with the file. For small files the whole file is loaded into the cache
during create, that’s why I never see PAGING_IO requests. For large files my
driver functions correctly. Is there a classic way to solve this problem?
Thanks,
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, February 21, 2007 10:56 AM
To: Windows File Systems Devs Interest List
Subject: Re[2]: [ntfsd] Missing READ IOs in filter driver
I think this has been discussed several times here,
so your problem is probably exactly the same like
the other posters had.
Let me guess.
Your driver is not loaded at boot time. The file you are
trying to open has been loaded into system cache far sooner
than your driver can catch the I/O request.
Is that true ?
L.
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Gregory,
The underlying filesystem itself will not pull data into cache before the
IRP_MJ_CREATE completes. I think that David was referring to the filesystem
initiating caching on the file by setting up the SOPs or by calling
CcInitializeCM() in some cases; though generally cache initialization is
delayed until the first IO on a file.
Other filesystem filter drivers may cause read requests to occur during
IRP_MJ_CREATE handling, such as an AV product performing a scan on a file
through cached IO, but this is known to break things particularly with stack
based fileobjects and some other filter higher in the stack cancels the
open. Also, if what you are describing, where a filesystem reads data into
the cache before it would complete the IRP_MJ_CREATE, then just about every
encryption and compression driver on the market would be broken; and I can
attest, since I have implemented many of both types, that they are not.
What other products are installed on the system?
Pete
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
(303)546-0300
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gregory Dardyk
Sent: Wednesday, February 21, 2007 8:05 AM
To: Windows File Systems Devs Interest List
Subject: RE: Re[2]: [ntfsd] Missing READ IOs in filter driver
No, the driver is loaded at boot time.
I think I found the problem. As David mentioned, the FS can read some data
during the create. Since I don’t have the FsContext for a file until the
CREATE request is completed by the FS driver, I cannot associate the READs
with the file. For small files the whole file is loaded into the cache
during create, that’s why I never see PAGING_IO requests. For large files my
driver functions correctly. Is there a classic way to solve this problem?
Thanks,
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, February 21, 2007 10:56 AM
To: Windows File Systems Devs Interest List
Subject: Re[2]: [ntfsd] Missing READ IOs in filter driver
I think this has been discussed several times here,
so your problem is probably exactly the same like
the other posters had.
Let me guess.
Your driver is not loaded at boot time. The file you are
trying to open has been loaded into system cache far sooner
than your driver can catch the I/O request.
Is that true ?
L.
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Gregory,
Is this on NTFS? If so, the file data will be stored in the MFT if it is small enough to fit. Thus, caching of the file data is done against the MFT, not against the file itself. It isn’t until the file cannot fit insize the MFT record (and the size of the MFT record is either 1K or 4K depending upon when the NTFS file system was first written out to the disk.)
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Gregory Dardyk
Sent: Wednesday, February 21, 2007 6:05 AM
To: ntfsd redirect
Subject: RE: Re[2]: [ntfsd] Missing READ IOs in filter driver
No, the driver is loaded at boot time.
I think I found the problem. As David mentioned, the FS can read some data during the create. Since I don’t have the FsContext for a file until the CREATE request is completed by the FS driver, I cannot associate the READs with the file. For small files the whole file is loaded into the cache during create, that’s why I never see PAGING_IO requests. For large files my driver functions correctly. Is there a classic way to solve this problem?
Thanks,
Gregory
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, February 21, 2007 10:56 AM
To: Windows File Systems Devs Interest List
Subject: Re[2]: [ntfsd] Missing READ IOs in filter driver
I think this has been discussed several times here,?
so your problem is probably exactly the same like
the other posters had.
Let me guess.
Your driver is not loaded at boot time. The file you are
trying to open has been loaded into system cache far sooner
than your driver can catch the I/O request.
Is that true ?
L.
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Bravo. I’ve been waiting for someone to mention that, Oh yeah, NTFS stores
a certain amount of file data with the metadata. So when you read the
metadata (file size, creation time, etc.) you get some of the file data for
free. (A very nice optimization over its roots in the VMS file system.)
See Helen Custer’s “Inside the Windows NT File System”.
Ken
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Wednesday, February 21, 2007 7:20 PM
To: Windows File Systems Devs Interest List
Subject: RE: Re[2]: [ntfsd] Missing READ IOs in filter driver
Gregory,
Is this on NTFS? If so, the file data will be stored in the MFT if it is
small enough to fit. Thus, caching of the file data is done against the
MFT, not against the file itself. It isn’t until the file cannot fit insize
the MFT record (and the size of the MFT record is either 1K or 4K depending
upon when the NTFS file system was first written out to the disk.)
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com]
On Behalf Of Gregory Dardyk
Sent: Wednesday, February 21, 2007 6:05 AM
To: ntfsd redirect
Subject: RE: Re[2]: [ntfsd] Missing READ IOs in filter driver
No, the driver is loaded at boot time.
I think I found the problem. As David mentioned, the FS can read some data
during the create. Since I don’t have the FsContext for a file until the
CREATE request is completed by the FS driver, I cannot associate the READs
with the file. For small files the whole file is loaded into the cache
during create, that’s why I never see PAGING_IO requests. For large files my
driver functions correctly. Is there a classic way to solve this problem?
Thanks,
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, February 21, 2007 10:56 AM
To: Windows File Systems Devs Interest List
Subject: Re[2]: [ntfsd] Missing READ IOs in filter driver
I think this has been discussed several times here,?
so your problem is probably exactly the same like
the other posters had.
Let me guess.
Your driver is not loaded at boot time. The file you are
trying to open has been loaded into system cache far sooner
than your driver can catch the I/O request.
Is that true ?
L.
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Tony,
This is on NTFS. The data that is stored in the MFT was one of my first
guesses, so until this moment I’ve only tested with files larger than 10K.
But I am still interested to know what is the right way to handle this
issue. Is it possible to create an encryption filter driver that encrypts
small files without encrypting the MFT?
Thanks,
Gregory
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Thursday, February 22, 2007 2:20 AM
To: Windows File Systems Devs Interest List
Subject: RE: Re[2]: [ntfsd] Missing READ IOs in filter driver
Gregory,
Is this on NTFS? If so, the file data will be stored in the MFT if it is
small enough to fit. Thus, caching of the file data is done against the
MFT, not against the file itself. It isn’t until the file cannot fit insize
the MFT record (and the size of the MFT record is either 1K or 4K depending
upon when the NTFS file system was first written out to the disk.)
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com]
On Behalf Of Gregory Dardyk
Sent: Wednesday, February 21, 2007 6:05 AM
To: ntfsd redirect
Subject: RE: Re[2]: [ntfsd] Missing READ IOs in filter driver
No, the driver is loaded at boot time.
I think I found the problem. As David mentioned, the FS can read some data
during the create. Since I don’t have the FsContext for a file until the
CREATE request is completed by the FS driver, I cannot associate the READs
with the file. For small files the whole file is loaded into the cache
during create, that’s why I never see PAGING_IO requests. For large files my
driver functions correctly. Is there a classic way to solve this problem?
Thanks,
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, February 21, 2007 10:56 AM
To: Windows File Systems Devs Interest List
Subject: Re[2]: [ntfsd] Missing READ IOs in filter driver
I think this has been discussed several times here,?
so your problem is probably exactly the same like
the other posters had.
Let me guess.
Your driver is not loaded at boot time. The file you are
trying to open has been loaded into system cache far sooner
than your driver can catch the I/O request.
Is that true ?
L.
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Gregory,
Welcome to the wonderful world of encryption filters. As we continue to
reaffirm every day, it’s fiendishly complex, in no small amount due to
Byzantine behavior in the underlying file systems (ergo, they lie to
you.)
The general approach I’ve suggested over the years is to make the file
larger than the MFT; from what I can tell, NTFS doesn’t move
non-resident attributes back into the MFT if they “fit again”. So, make
it bigger and then adjust its size back.
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Peter,
Thanks for the explanation. The driver works correctly now and it seems that
I shot myself in the foot - some part of my code was fooling my own context
tracking mechanism. J
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Peter Scott
Sent: Wednesday, February 21, 2007 4:29 PM
To: Windows File Systems Devs Interest List
Subject: RE: Re[2]: [ntfsd] Missing READ IOs in filter driver
Gregory,
The underlying filesystem itself will not pull data into cache before the
IRP_MJ_CREATE completes. I think that David was referring to the filesystem
initiating caching on the file by setting up the SOPs or by calling
CcInitializeCM() in some cases; though generally cache initialization is
delayed until the first IO on a file.
Other filesystem filter drivers may cause read requests to occur during
IRP_MJ_CREATE handling, such as an AV product performing a scan on a file
through cached IO, but this is known to break things particularly with stack
based fileobjects and some other filter higher in the stack cancels the
open. Also, if what you are describing, where a filesystem reads data into
the cache before it would complete the IRP_MJ_CREATE, then just about every
encryption and compression driver on the market would be broken; and I can
attest, since I have implemented many of both types, that they are not.
What other products are installed on the system?
Pete
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
(303)546-0300
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gregory Dardyk
Sent: Wednesday, February 21, 2007 8:05 AM
To: Windows File Systems Devs Interest List
Subject: RE: Re[2]: [ntfsd] Missing READ IOs in filter driver
No, the driver is loaded at boot time.
I think I found the problem. As David mentioned, the FS can read some data
during the create. Since I don’t have the FsContext for a file until the
CREATE request is completed by the FS driver, I cannot associate the READs
with the file. For small files the whole file is loaded into the cache
during create, that’s why I never see PAGING_IO requests. For large files my
driver functions correctly. Is there a classic way to solve this problem?
Thanks,
Gregory
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, February 21, 2007 10:56 AM
To: Windows File Systems Devs Interest List
Subject: Re[2]: [ntfsd] Missing READ IOs in filter driver
I think this has been discussed several times here,
so your problem is probably exactly the same like
the other posters had.
Let me guess.
Your driver is not loaded at boot time. The file you are
trying to open has been loaded into system cache far sooner
than your driver can catch the I/O request.
Is that true ?
L.
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com