Hello people,
I had posted the same question on microsoft’s discussion site, but was told to try posting a query at NTFSD. I have already read quite a few discussions on the concept of Paging I/O and TopLevelIrp.
I am developing a FS minifilter in which i need to read the contents of the file on receiving IRP_MJ_WRITE.
I am able to perform the read operation in all cases except when TopLevelIrp != NULL.
After consulting some people, i got one method to overcome this problem. And that was to read from the disk directly. That is, get the specific cluster and read it.
Is this the only solution to this problem?
Is there any other solution that anyone would like to propose?
Can i build a Paging I/O read and then send it? Will that work?
Regards,
Tushar A.
What do you need to read during paging write?
xxxxx@gmail.com wrote:
Hello people,
I had posted the same question on microsoft’s discussion site, but was told to try posting a query at NTFSD. I have already read quite a few discussions on the concept of Paging I/O and TopLevelIrp.
I am developing a FS minifilter in which i need to read the contents of the file on receiving IRP_MJ_WRITE.
I am able to perform the read operation in all cases except when TopLevelIrp != NULL.
After consulting some people, i got one method to overcome this problem. And that was to read from the disk directly. That is, get the specific cluster and read it.
Is this the only solution to this problem?
Is there any other solution that anyone would like to propose?
Can i build a Paging I/O read and then send it? Will that work?
Regards,
Tushar A.
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: xxxxx@alfasp.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.
Thanks for responding Sir,
Actually I am implementing a FS minifilter in which i need to read the contents of the file when i receive an IRP_MJ_WRITE for it. I am not considering Cached I/O (including Fast I/Os). I am only focussing on Non Cached I/O, since these Writes actually change the on-disk data.
So, my aim is to take a backup of the original data present in the file when a Non-cached Write request comes.
I want to take a backup of only the data that is going to be modified and not the entire file…
So i thought of reading the contents of the file in my pre-callback for Write.
But i want to find out if there is a way to read a file on receiving Write IRP for it?
Is there any solution that you can propose of?
If you need to read the same range, you can reuse the write IRP to read the data, then reissue the IRP to write the data.
xxxxx@gmail.com wrote:
Thanks for responding Sir,
Actually I am implementing a FS minifilter in which i need to read the contents of the file when i receive an IRP_MJ_WRITE for it. I am not considering Cached I/O (including Fast I/Os). I am only focussing on Non Cached I/O, since these Writes actually change the on-disk data.
So, my aim is to take a backup of the original data present in the file when a Non-cached Write request comes.
I want to take a backup of only the data that is going to be modified and not the entire file…
So i thought of reading the contents of the file in my pre-callback for Write.
But i want to find out if there is a way to read a file on receiving Write IRP for it?
Is there any solution that you can propose of?
–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.
Thank you Sir,
But i am developing a minifilter.
In a minifilter there is no concept of IRP. We get only Callbackdata.
Should i allocate a new callbackdata and then use FltPerformSynchronousIo?
No, you would call FltReadFile, passing your own instance. I never tried
whether FltReadFile fails in paging I/O path, or whether it allows the caller
to proceed. If it doesn’t, you have no way of doing the required read in the
mini-filter.
xxxxx@gmail.com wrote:
Thank you Sir,
But i am developing a minifilter.
In a minifilter there is no concept of IRP. We get only Callbackdata.
Should i allocate a new callbackdata and then use FltPerformSynchronousIo?
–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.
I have been able to call FltCreate, FltReadFile, and FltWriteFile without any issues during a paging IO. And I have done it when TopLevelIrp == NULL and != NULL. It will do the call. That said, I wouldn’t bet my life on it simply because even when TopLevelIrp is null, that only indicates that there is no information about a lock available, not that one actually exists or not.
In either case, be ready to deal with deadlocks when trying to accomplish those calls in the paging path.
–Royal
A few obvious caveats here are:
- Don’t do this for directories (NTFS sends reads/writes on Directory entries as well, be ready to filter these out)
- Don’t do this for metadata files
- Don’t do this for the paging file (doh :P)
And don’t try handling a file system that does not use the cache manager fully (Lanman for example).
Dejan.
xxxxx@obwando.com wrote:
I have been able to call FltCreate, FltReadFile, and FltWriteFile without any issues during a paging IO. And I have done it when TopLevelIrp == NULL and != NULL. It will do the call. That said, I wouldn’t bet my life on it simply because even when TopLevelIrp is null, that only indicates that there is no information about a lock available, not that one actually exists or not.
In either case, be ready to deal with deadlocks when trying to accomplish those calls in the paging path.
–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.
Thank you Dejan and Royal.
Royal, i saw the documentation of FltReadFile. It says “Callers of FltReadFile must be running at IRQL PASSIVE_LEVEL.”. And the paging Write comes at APC_LEVEL. Then how can i issue FltReadFile.
Dejan, from your description, it seems that it is impossible to read from the file in a paging write path, in a minifilter.
Do you recommend the development of a full legacy filter driver rather than a minifilter?
The minifilter model won’t help with this problem. It will make it much
more difficult to write the filter since so many of the difficult problems
are the filter manager’s problem.
The problem is that doing saving of data is best done during the normal
write and not the paging write in the file system area. Another place where
it can be done is in the storage stack. You cannot read the file data when
a paging write has been issued unless you do your own IRP synchronously
since every part of the file system believes the data has been changed.
What you get with FltReadFile is what is going to be written and not the on
storage data. It is the same as closing the barn door after the horses have
left.
–
David J. Craig
Engineer, Sr. Staff Software Systems
Broadcom Corporation
wrote in message news:xxxxx@ntfsd…
> Thank you Dejan and Royal.
>
> Royal, i saw the documentation of FltReadFile. It says “Callers of
> FltReadFile must be running at IRQL PASSIVE_LEVEL.”. And the paging Write
> comes at APC_LEVEL. Then how can i issue FltReadFile.
>
> Dejan, from your description, it seems that it is impossible to read from
> the file in a paging write path, in a minifilter.
> Do you recommend the development of a full legacy filter driver rather
> than a minifilter?
>
>
Thank you David,
You said,
“You cannot read the file data when
a paging write has been issued unless you do your own IRP synchronously
since every part of the file system believes the data has been changed.”
There is an option to read the data synchronously by allocating a callbackdata and issuing FltPerformSynchronousIo.
Wouldn’t that give data on the disk?
Why would it? The file system knows the data is in the cache and will give
it to you from there. I would study the file system model in Windows. Look
at how all the pieces relate to each other, especially the file system
driver, cache manager, memory manager, disk class driver, and storage port
driver. Read Nagar’s book. You will not succeed or even come close until
you understand how the pieces are related.
Maybe you should consider hiring one of the consultants. OSR does file
system work. Don Burn is another choice. I am currently in the world of
NDIS miniports and am not accepting any consulting jobs.
–
David J. Craig
Engineer, Sr. Staff Software Systems
Broadcom Corporation
wrote in message news:xxxxx@ntfsd…
> Thank you David,
> You said,
> “You cannot read the file data when
> a paging write has been issued unless you do your own IRP synchronously
> since every part of the file system believes the data has been changed.”
>
> There is an option to read the data synchronously by allocating a
> callbackdata and issuing FltPerformSynchronousIo.
> Wouldn’t that give data on the disk?
>
>
>
it to you from there.>
Does this mean that even if we issue a Non-cached ( PAGING I/O ) READ IRP, the data can actually be fetched from the cache instead of the disk?
Set the paging I/O flag when you call FltReadFile.
This is important because in that case the file system will not try to flush the Cc data before it satisfies the read (if you issue a non-cached read only, the file system will make sure the
cache is flushed, which would cause a paging write, for the SAME region to be issued again, thus calling your write handler; you’d issue the read again and cause an endless loop, well… a stack
overflow will occur at some point).
If the paging I/O flag is set, the FS assumes the Cc does not have that particular region, and will simply read it from the disk.
Dejan.
xxxxx@gmail.com wrote:
Thank you Dejan and Royal.
Royal, i saw the documentation of FltReadFile. It says “Callers of FltReadFile must be running at IRQL PASSIVE_LEVEL.”. And the paging Write comes at APC_LEVEL. Then how can i issue FltReadFile.
Dejan, from your description, it seems that it is impossible to read from the file in a paging write path, in a minifilter.
Do you recommend the development of a full legacy filter driver rather than a minifilter?
–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.
Disclaimer: Of course, what I suggested (setting Paging I/O flag for your
own I/O) is not kosher, and regardless of me never having issues with it, noone
will guarantee it will work! Noone will guarantee it will work at all in future
Windows versions.
–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.
Thank You Dejan Sir,
Can you please explain me one more thing?
Is it legal to call FltReadFile at APC_LEVEL?
Because you are suggesting to call FltReadFile. But according to the documentation, “Callers of FltReadFile must be running at IRQL PASSIVE_LEVEL.”
That’s why I say I don’t guarantee anything.
I do know most would never suggest/approve of it, but at least by looking at fastfat locks, I don’t see how reading the same block that will be
written right after using a paging flag would deadlock.
Maybe NTFS is different… it sure is for compressed files, but for others I have not observed different Cc behavior.
xxxxx@gmail.com wrote:
Can you please explain me one more thing?
Is it legal to call FltReadFile at APC_LEVEL?
Because you are suggesting to call FltReadFile. But according to the documentation, “Callers of FltReadFile must be running at IRQL PASSIVE_LEVEL.”
–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.
Thanks once again Sir.
Is it not better to use FltAllocateCallbackdata ( can be called at APC_LEVEL ) and FltPerformSynchronousIo ( can be called at APC_LEVEL ) rather than calling FltReadFile ( which can be called at PASSIVE_LEVEL )?
As such from the discussion i am getting a feeling that though reading is possible, but its behaviour in various environments is sort of unpredictable. So, wouldn’t it be advisable to use functions that are documented enough to be called at APC_LEVEL rather than the one that the document says can be called at PASSIVE_LEVEL?
No. You have a big chance of screwing the FLT_CALLBACK_DATA up. Using FltReadFile is better.
xxxxx@gmail.com wrote:
Thanks once again Sir.
Is it not better to use FltAllocateCallbackdata ( can be called at APC_LEVEL ) and FltPerformSynchronousIo ( can be called at APC_LEVEL ) rather than calling FltReadFile ( which can be called at PASSIVE_LEVEL )?
As such from the discussion i am getting a feeling that though reading is possible, but its behaviour in various environments is sort of unpredictable. So, wouldn’t it be advisable to use functions that are documented enough to be called at APC_LEVEL rather than the one that the document says can be called at PASSIVE_LEVEL?
–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.
Thanks a lot Sir for clarifying my doubts.
In the same thread, Mr. David said that “Why would it? The file system knows the data is in the cache and will give it to you from there.”…
Does it mean that the FSD can actually use the cache instead of reading the data from the disk?
If it really happens, can you please explain me the reason?