Hi all,
My goal is to figure out the most efficient way of getting file-id for logging.
A simple idea use FltQueryInformationFile() for FILE_INTERNAL_INFORMATION in the IRP_MJ_WRITE handler itself. But the documentation (https://msdn.microsoft.com/en-us/library/windows/hardware/ff543439(v=vs.85).aspx ) says that “Before calling this routine, call IoGetTopLevelIrp. If IoGetTopLevelIrp returns a non-NULL value, do not call the routine as this can cause a system deadlock.” IRP_MJ_WRITE can be called in paging IO context (interrupt context). So if its is paging IO I sshould to do something different.
I’ve already read topic http://www.osronline.com/showThread.cfm?link=113995 and from it I understood that theoretically I can use FltPerformAsynchronousIo.
I am new in this issue. Please, could somebody advise me a good solution for my task or explain how I can use FltPerformAsynchronousIo for my goal here.
Thank you very much!
> Please, could somebody advise me a good solution for my task or explain
how I can use FltPerformAsynchronousIo for my goal here.
Rather than pounding on the FSD cache the value. You know it will never
change until the file is deleted (modulo the fact that it might change for
some FSDs but you are obviously OK with that), so cache it.
Grab it in post create, slam it into a FileContext and then interrogate
that.
/Rod
Thank you, Rod. That’s a good idea.
But are you sure that I will have no any problems with storing file-id in fileContext? After all, the context can be overwritten by someone. Or not?
It’s YOUR filter’s context, so if “someone” overwrites it other than you it’s a memory corruption and it will cause problems independent of your caching the value or not.
Tony
OSR
Do this in CREATE, not in WRITE
wrote in message news:xxxxx@ntfsd…
> Hi all,
>
> My goal is to figure out the most efficient way of getting file-id for logging.
> A simple idea use FltQueryInformationFile() for FILE_INTERNAL_INFORMATION in the IRP_MJ_WRITE handler itself. But the documentation (https://msdn.microsoft.com/en-us/library/windows/hardware/ff543439(v=vs.85).aspx ) says that “Before calling this routine, call IoGetTopLevelIrp. If IoGetTopLevelIrp returns a non-NULL value, do not call the routine as this can cause a system deadlock.” IRP_MJ_WRITE can be called in paging IO context (interrupt context). So if its is paging IO I sshould to do something different.
> I’ve already read topic http://www.osronline.com/showThread.cfm?link=113995 and from it I understood that theoretically I can use FltPerformAsynchronousIo.
> I am new in this issue. Please, could somebody advise me a good solution for my task or explain how I can use FltPerformAsynchronousIo for my goal here.
>
> Thank you very much!
>
>context can be overwritten by someone.
No object can be overwritten until it is alive, this is obvious.
–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com
Such a solution does not suit me. May be a situation where a driver has to be loaded after some files were opened. Then when trying to write to one of these files, I can not get it file-id this way. Any another idea’s?
> Such a solution does not suit me. May be a situation where a driver has to be loaded
Cannot be such a situation. Make your driver boot-start.
Also, doing QI on each write will absolutely kill the performance.
–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com
Boot-start is not a solution because I need to start it after snapshot
> Boot-start is not a solution because I need to start it after snapshot
I don’t see the linkage. If you boot start then you can still see the
snapshots. Remember that you’ll you’ll get called to attach before a new
snapshot sees the first create.
Even If you don’t boot start then you’ll still see all new snapshots as they
are created, but you’ll see existing snapshots and they may have open
handles.
Such a solution does not suit me. May be a situation where a driver has to
be loaded after some files were opened. Then when trying to write to one
of these files,
I can not get it file-id this way. Any another idea’s?
What exactly is your requirement. How much dither do you allow? If a write
happens immediately before you attach then you won’t log it, therefore you
cannot care for logging one which attaches immediately after. So how big is
your window?
Modulo all that I’d still do what we recommended earlier.
If there was a reason to not boot start I’d then treat the exception case
exceptionally. For paging IO I’d probably pin the file object and post (to
get out from under the paging IO). But bear in mind that many of the file
objects you see will have no open handles on them so in that case you’ll
need to send the request as a Paging Query Information and I’d not be sure
that the FID is documented as being available to a Paging Query Info.
R
I need be able to upload to the server machine without stopping its operation and start log all entries in the file system since the one snapshot until the next snapshot.
Sounds like the making of a pretty standard COW real time backup
system/disaster recovery/system reset solution
I still don’t see any reason to not boot attach.
I’d also recommend (in case you haven’t got that far) not not worry about
paging IO - just log the create section change and schedule a full restore.
And if you are not worrying about paging then you can do your QueryFileInfo
in the exceptional case inline.
> Boot-start is not a solution because I need to start it after snapshot
Why?
BTW - VolSnap writes to snapshot storage by itself, and neither FSD nor the file-level filters see this.
–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com
But still you can apply the same logic even if the operation isn’t a CREATE
(as in, if you find a file without your FileContext, then you should be
able to query the fileId and then set it in a file context).
Thanks,
Alex.
On Tue, Apr 7, 2015 at 4:50 AM, wrote:
> I need be able to upload to the server machine without stopping its
> operation and start log all entries in the file system since the one
> snapshot until the next snapshot.
>
> —
> NTFSD is sponsored by OSR
>
> OSR is hiring!! Info at http://www.osr.com/careers
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
Thank you for your help.
I tried to use FltPerformAsynchronousIo in Post-Write and call FltQueryInformationFile in callback. It works. Is there some phantom problems?
> Is there some phantom problems?
Assuming that you are not looking at paging IO, are happy with the fileID
restrictions, and are happy with panning the performance ; I’d say not.
> Assuming that you are not looking at paging IO
Why not? If understood right from documentation it can be used for paging IO.
"The caller of FltPerformAsynchronousIo can be running at IRQL <= APC_LEVEL if the IRP_PAGING_IO flag is set in the IrpFlags member of the FLT_IO_PARAMETER_BLOCK structure for the operation. (This flag is only valid for IRP_MJ_READ, IRP_MJ_WRITE, IRP_MJ_QUERY_INFORMATION, and IRP_MJ_SET_INFORMATION operations.) Otherwise, the caller must be running at IRQL PASSIVE_LEVEL. "
I apologize if I ask or say some nonsense. I’m new and not yet very well-versed in this area.
> > Assuming that you are not looking at paging IO
Why not? If understood right from documentation it can be used for paging
IO.
I agree. Its just theory and practice and a fair amount of paranoia.
I’m always super nervous about straying off the beaten track when I’m down
the paging IO path. NTFS might suddenly decide to say STATUS_FILE_LOCKING
or some such. And only for a specific version and under certain
circumstances. So if you rely on it you’ll probably be fine, but unless
you have access to the NTFS (or whatever else) sources you’ll never know.
Nor will you be able to diagnose it or make a guess at fixing it.
Or it might all be OK.
It could me that this is regularly called in normal operation (just like
getting the file name is) but if it isn’t then there is a chance that it
will have rotted, or will rot.
Another approach (which is generally safer in my opinion) is to generate a
GUID whenever you encounter a file without your context (as in, if you’re a
Post-Write or something) and then queue a request to a thread (take a
reference on the FILE_OBJECT and pass it to the thread) to issue a
FltQueryInformationFile and replace the GUID with the proper FileID. I
think the key thing here is that since you need this for logging then it
doesn’t matter that you get the FileID right away (as in, in the context of
THAT operation -> so no need to issue the FltQueryInformationFile in
postWrite) and instead what matters is that you eventually associate the
operation with the right FileID before writing the log to file or however
you plan to log things…
Thanks,
Alex.
On Wed, Apr 8, 2015 at 8:04 AM, Rod Widdowson
wrote:
> > Assuming that you are not looking at paging IO
>>
>> Why not? If understood right from documentation it can be used for paging
>> IO.
>>
>
> I agree. Its just theory and practice and a fair amount of paranoia.
>
> I’m always super nervous about straying off the beaten track when I’m
> down the paging IO path. NTFS might suddenly decide to say
> STATUS_FILE_LOCKING or some such. And only for a specific version and under
> certain circumstances. So if you rely on it you’ll probably be fine, but
> unless you have access to the NTFS (or whatever else) sources you’ll never
> know. Nor will you be able to diagnose it or make a guess at fixing it.
>
> Or it might all be OK.
>
> It could me that this is regularly called in normal operation (just like
> getting the file name is) but if it isn’t then there is a chance that it
> will have rotted, or will rot.
>
>
>
> —
> NTFSD is sponsored by OSR
>
> OSR is hiring!! Info at http://www.osr.com/careers
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>