Hi, I need to track the number of times certain files are accessed.
I’m using the passThrough sample from wdk. I registered callback
for read/write. Now in the pre-op, I need to compare the file name
associated with the io against my prebuild list of files.
Q1. May I use iopb->TargetFileObject->FileName directly in pre-read/write
callback?
Q2. How can I compare two unicode strings in pre-read/write callbacks?
if I understand correctly they can be invoked at > PASSIVE_LEVEL and
RtlCompareUnicodeString can only be called at PASSIVE_LEVEL (as per msdn).
If I’m totally off the track, then is using FltGetFileNameInformation in
pre-create to obtain file name, do the compare and save the result in the
stream context to be used in pre-read/write the correct way of doing this?
Help is appreciated. Thank you.
about name query, you can refer to the minispy sample…
If I’m totally off the track, then is using FltGetFileNameInformation in
pre-create to obtain file name, do the compare and save the result in
the
stream context to be used in pre-read/write the correct way of doing
this?
depends on the query method. refer to
FLT_FILE_NAME_QUERY_ALWAYS_ALLOW_CACHE_LOOKUP
and FLT_FILE_NAME_QUERY_DEFAULT. You will need to know when it is safe to
query the name.
On Thu, Mar 24, 2011 at 12:08 PM, wrote:
> Hi, I need to track the number of times certain files are accessed.
> I’m using the passThrough sample from wdk. I registered callback
> for read/write. Now in the pre-op, I need to compare the file name
> associated with the io against my prebuild list of files.
> Q1. May I use iopb->TargetFileObject->FileName directly in
> pre-read/write
> callback?
> Q2. How can I compare two unicode strings in pre-read/write callbacks?
> if I understand correctly they can be invoked at > PASSIVE_LEVEL and
> RtlCompareUnicodeString can only be called at PASSIVE_LEVEL (as per
> msdn).
>
> If I’m totally off the track, then is using FltGetFileNameInformation in
> pre-create to obtain file name, do the compare and save the result in
> the
> stream context to be used in pre-read/write the correct way of doing
> this?
>
> Help is appreciated. Thank you.
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
–
- amitr0
Q1: No, the FILE_OBJECT->FileName is not guaranteed to be meaningful for any
operation other than IRP_MJ_CREATE. Not only are the contents not guaranteed
to be valid, but they can also be out-of-date. When a file is renamed there
is no mechanism to go and visit all FILE_OBJECTs opened against it and
update their FILE_OBJECT->FileName.
Q2. This limitation only really applies if you plan to compare the strings
case insensitive. In that case a table is needed for converting lowercase to
uppercase characters (or the other way around), and that table is from paged
pool. If you compare case insensitive then it’s pretty much a memcmp.
Whether you should use case sensitive or case insensitive comparison depends
on the way the OS and the file system are set up.
I would suggest using FltGetFileNameInformation in postCreate
(FltGetFileNameInformation in preCreate should be avoided if possible and
based on your description it looks possible). When choosing the context
type, keep in mind that some files might have multiple names (hardlinks) and
so they might actually be in the list of files you are tracking, but being
opened under a different name. Also, I’m not sure whether this is a concern
but files might move in and out of your list of names through renames and
you might need to track those as well…
There is this blog post about name usage in file system filters,
http://fsfilters.blogspot.com/2010/02/names-and-file-systems-filters.html.
Maybe it’ll help you chose your design.
Thanks,
Alex.
Thanks Amit and Alex.
Is renaming the only issue with using FILE_OBJECT->FileName? The files I’m tracking are gauranteed to not be renamed.
Also other than MJ_READ and MJ_WRITE, are there any other operations that may generate disk io? e.g, MJ_SET_INFORMATION etc. ? Or MJ_SET_INFORMATION will in-turn trigger a MJ_WRITE?
Alex, also thank you for your very informative blog.
No, that’s not the only issue, it was just an example. FILE_OBJECT->FileName
is not defined except in the IRP_MJ_CREATE path. Therefore, you can’t safely
use it except in the create path.
It’s hard to say, but many operations (potentially all operations?) may end
up generating disk IO… For example, IRP_MJ_SET_INFORMATION might issue IO
to the directory entry for the file and so on…
Thanks,
Alex.
>>No, that’s not the only issue, it was just an example. FILE_OBJECT->FileName
is not defined except in the IRP_MJ_CREATE path. Therefore, you can’t safely
use it except in the create path.
got it. Thanks.
> It’s hard to say, but many operations (potentially all operations?) may end
up generating disk IO… For example, IRP_MJ_SET_INFORMATION might issue IO
to the directory entry for the file and so on…
ok. But will the io generated by these come back to my minifilter in pre-READ/WRITE callbacks? say e.g, will MJ_SET_INFORMATION on a file may generate write to $Mft and then my pre-write callback will be invoked? Essentially I need to know what pre/post-op I should be looking at to track activities that my potentially generate io.
One more question:
My minifilter is StartType=0, Class = “ActivityMonitor”, Altitude= “370030” . Is there a possibily that some of the file/directory handles have been opened before by driver loads (e.g, c:\Windows etc) and I won’t be able to see any MJ_CREATE for them? Thank you.
>But will the io generated by these come back to my minifilter in
pre-READ/WRITE callbacks?
[AB] No they wont.
say e.g, will MJ_SET_INFORMATION on a file may generate write to $Mft and
then my pre-write callback will be invoked? Essentially I need to know what
pre/post-op I should be looking at to track activities that my potentially
generate io
[AB] I don’t knwo what you are trying to make, but if you want to capture
such IO like MFT updates etc, you gotta be at a much lower level, like
volume or disk.
My minifilter is StartType=0, Class = “ActivityMonitor”, Altitude= “370030”
. Is there a possibily that some of the file/directory handles have been
opened before by driver loads (e.g, c:\Windows etc) and I won’t be able to
see any MJ_CREATE for them? Thank you.
[AB] yes it is possible. in fcat there are ways to bypass your filter even
after it is loaded. there are IO manager APIs to write directly to the FS or
to any device in the stack below your filter. Or one might simply write raw
sectors inside a file.
Amitr0
On Fri, Mar 25, 2011 at 1:24 AM, wrote:
> >>No, that’s not the only issue, it was just an example.
> FILE_OBJECT->FileName
> is not defined except in the IRP_MJ_CREATE path. Therefore, you can’t
> safely
> use it except in the create path.
>
> got it. Thanks.
>
> >> It’s hard to say, but many operations (potentially all operations?) may
> end
> up generating disk IO… For example, IRP_MJ_SET_INFORMATION might issue IO
> to the directory entry for the file and so on…
>
> ok. But will the io generated by these come back to my minifilter in
> pre-READ/WRITE callbacks? say e.g, will MJ_SET_INFORMATION on a file may
> generate write to $Mft and then my pre-write callback will be invoked?
> Essentially I need to know what pre/post-op I should be looking at to track
> activities that my potentially generate io.
>
> One more question:
> My minifilter is StartType=0, Class = “ActivityMonitor”, Altitude= “370030”
> . Is there a possibily that some of the file/directory handles have been
> opened before by driver loads (e.g, c:\Windows etc) and I won’t be able to
> see any MJ_CREATE for them? Thank you.
>
>
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
–
- amitr0
I’ve been thinking and I can’t come up with an IRP that I can guarantee
won’t generate disk level IO under any circumstance. Which is another way of
saying you probably need to filter all operations.
It depends. There is file system and disk IO that happens during system
start. However, your minifilter will be there as the boot volume is mounted
(after SERVICE_BOOT_START drivers are started) and you’ll see that (the
volume being mounted) and all subsequent operations. However, you will not
see IO initiated by minifilters below yours.
What are you trying to do again ?
Thanks,
Alex.
Alex:
okay. so I need to filter all the operations. but I’m suprised to know that a file read/write (even to $Mft) triggerred as a result of a non-read/write op will not invoke my minifilter’s callback. Currently in my minifilter I do see access to $Mft file when I access/create a new folder/file.
If I understand correctly here in your blog post " http://fsfilters.blogspot.com/2010/02/issuing-io-in-minifilters-part-1.html ", all the filters (except the calling) will get called for a createfile issued by any minifilter. So this is not true for other operations?
I’m trying to filter all the operations that may potentially generate any io for given files (that includes $Mft also). I understand that some io may be served from cache but that’s ok as long as my minifilter is able to see them.
Amit:
[AB] No they wont.
this is what I don’t understand. if they are not, then filesystem minifilter is not truly a filesystem minifilter.
[AB] I don’t knwo what you are trying to make, but if you want to capture
such IO like MFT updates etc, you gotta be at a much lower level, like
volume or disk.
At a lower level how do I map a sector to a file name?
[AB] yes it is possible. in fcat there are ways to bypass your filter even
after it is loaded. there are IO manager APIs to write directly to the FS or
to any device in the stack below your filter. Or one might simply write raw
sectors inside a file.
Again if somebody can bypass my filesystem filter, then it is not truly a filesystem filter.
Sorry but this is true of all filter drivers, there are ways to bypass
them. Some of the anti-virus products were pretty bad about this, I
don’t know if they have changed.
You should if at all possible design your filter so it can survive being
bypasses. By survive I don’t mean it has to do the right thing, but it
should not do harm.
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
“xxxxx@gmail.com” wrote in message
news:xxxxx@ntfsd:
> Alex:
>
> okay. so I need to filter all the operations. but I’m suprised to know that a file read/write (even to $Mft) triggerred as a result of a non-read/write op will not invoke my minifilter’s callback. Currently in my minifilter I do see access to $Mft file when I access/create a new folder/file.
>
> If I understand correctly here in your blog post " http://fsfilters.blogspot.com/2010/02/issuing-io-in-minifilters-part-1.html ", all the filters (except the calling) will get called for a createfile issued by any minifilter. So this is not true for other operations?
>
> I’m trying to filter all the operations that may potentially generate any io for given files (that includes $Mft also). I understand that some io may be served from cache but that’s ok as long as my minifilter is able to see them.
>
> Amit:
> [AB] No they wont.
> this is what I don’t understand. if they are not, then filesystem minifilter is not truly a filesystem minifilter.
>
> [AB] I don’t knwo what you are trying to make, but if you want to capture
> such IO like MFT updates etc, you gotta be at a much lower level, like
> volume or disk.
>
> At a lower level how do I map a sector to a file name?
>
> [AB] yes it is possible. in fcat there are ways to bypass your filter even
> after it is loaded. there are IO manager APIs to write directly to the FS or
> to any device in the stack below your filter. Or one might simply write raw
> sectors inside a file.
>
> Again if somebody can bypass my filesystem filter, then it is not truly a filesystem filter.
>> If I understand correctly here in your blog post "
http://fsfilters.blogspot.com/2010/02/issuing-io-in-minifilters-part-1.html ",
all the filters (except the calling) will get called for a createfile issued by
any minifilter. So this is not true for other operations?
after reading the part 2, I think I mistook this part from your blog. io issued by a minifilter goes to the filters below it.
but then what I’m not able to understand is why my filter is able to intercept read/writes to $Mft which resulted due to me doing a create/read/write of a “new folder/file”. shouldn’t my minifilter be seeing only the folder being accessed and not the $Mft access?
Not all the filters get called, only the filters below the minifilter
issuing the create. The same is true for other operations. However, the way
NT is designed, someone can open a device directly and send IO to it,
bypassing any filters attached to it. It doesn’t usually happen and it’s
considered pretty bad design when a filter does that, but in theory it can
happen. File system filters usually accept this potential limitation.
As I’ve said before, in other posts on this list, I don’t think it’s
possible to trace each disk IO back to a specific file. So I don’t think
you can map sector to file name in the general case.
Thanks,
Alex.
Yes, the IO goes to filters below it.
The layers aren’t as nicely defined as you (and I) would like. There is
recursion in the IO stack. The reason you see the MFT updates is because the
file system uses cached IO for the MFT and flushes go to the top of the IO
stack.
Search this list (or my blog) for stream file objects, maybe that’ll explain
some things.
Thanks,
Alex.
ok. Thanks again for all valuable information and being patient with my questions.
>As I’ve said before, in other posts on this list, I don’t think
it’s possible to trace each disk IO back to a specific file. So I don’t
think you can map sector to file name in the general case.
Well, what if the OP wants to do it for a ‘very specific set of files’, say
exclusively for his meta data. Then, since the number of such ‘files’ would
be very very few (one or two), then it might be possible to write an FS
parser in a volume filter driver, fetch the extents of those ‘special files’
and then monitor the activity on them…
On Fri, Mar 25, 2011 at 4:47 AM, wrote:
> ok. Thanks again for all valuable information and being patient with my
> questions.
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
–
- amitr0
That could work for file contents. But my understanding was that the OP was
concerned about file system metadata related to those files as well (all
disk IO related to the files on his list). That includes parsing the NTFS
journal, transaction log, USN journal and such. Even so, there will be file
system metadata updates for more than one file (e.g. one page flush might
containg $Mft changes for two files, one that the OP cares about and one
not.) so there is not a one-to-one mapping between disk IO and files.
Also, in my opinion, writing a volume level FS parser that operates
concurrently with the FS (in other words, that works while the file system
is mounted on the volume) is a very tricky proposition and I think it
requires understanding not only the on disk structures but also the runtime
behavior of the file system (which is much more likely to change between OS
versions). I’m not going to say it’s impossible, but I think it would be
unfeasible under any circumstances. If you need this sort of thing, I
believe that the effort to write a file system that is actually designed to
do that is significantly lower.
Thanks,
Alex.