Do we need to support AcquireFileForNtCreateSection in a file system driver? What about AcquireForModWrite and AcquireForCcFlush?

NTFSD
1

We have a file system filter driver, but we have set the following callbacks for the fastio to NULL:

AcquireFileForNtCreateSection
ReleaseFileForNtCreateSection
AcquireForModWrite
ReleaseForModWrite
AcquireForCcFlush
ReleaseForCcFlush

This implementation is similar to the following post:

My question is, is this a correct approach? If not, how should we implement these callbacks?

I assume that NTFS file system drivers do not support this, but is there any file system that supports this, thus us not implementing these causing a performance hit?

And if we do implement them, wouldn't that cause issue with file systems that do not support these after we attach to them? What is the correct approach?

I'm asking this because a customer is experiencing some slow compile time on some IDEs when our filter driver is installed, but our IRP callbacks are very short and don't do anything to cause a performance hit, so I thought maybe this is happening because we haven't implemented these callbacks? Our fastio implementation is similar to the above project that I linked.

2

You’re way down the wrong path on multiple fronts…If you’re writing a file system filter driver you need to write a Filter Manager based minifilter. That article about how to write a “legacy” file system filter has been wrong since about the Win2K SP4/XP SP2 timeframe.

So, unless you’re targeting platforms older than that, here’s really no point in trying to debug anything related to it. The whole thing needs to be entirely thrown away and rewritten.

3

I would love to work on a minifilter instead, but I am just a simple developer given a task to find the issue with a legacy filesystem filter driver. I don't have the authority to tell them let's completely drop this project and rewrite it from ground up as a minifilter..

Also, this filter driver has been working without any issue for years, even on latest windows 11 builds, but this particular client says this driver is causing performance hit when compiling some apps, and procmon doesn't show any sign of failure in any file event. And normal IRP callbacks do not do anything that can cause any performance hit, they are just doing very basic stuff such as finding the file name of the file object, so the only thing I can think of right now is fastIo callbacks.

But back to the question, what is the correct implementation of FastIo callbacks, if the above article is not correct? do I need to actually implement those missing callbacks or...?

4

I have no idea if that article is right or not, once I looked and saw it was a legacy filter I lost interest.

I worked on a lot of legacy file systems filters back in the day but it’s been more than 10 years since I wrote my last one. There’s a LOT of complexity in getting every possible thing right, hence the reason why Filter Manager was created…

I can say for the general question of “do I need to support Fast I/O in a legacy filter?” the answer is YES. You need to implement all lf them and pass them all through, otherwise Fast I/O is disabled for the stack. But, before you get too excited, whether or not that will do anything to address the problem that you’re seeing is another matter. Debugging a performance issue in a legacy file system filter is something even I wouldn’t want to be dealing with in 2024…

I realize none of that helps, so my practical advice would be:

  1. Strip your code down to literally doing nothing and see if the perf problem still exists. That will help narrow down whether it’s something you’re doing or something you’re not doing
  2. Track down the legacy file system samples from an old WDK and see if the issue reproduces with those. I think the Vista WDKs were the last ones to have these samples…
  3. Set IoBlockLegacyFsFilters and see the problem magically disappear :magic_wand: The fact that this driver hasn’t caused problems for this long is either lucky or unlucky, depending on how you view technical debt…
1 Like
5

On Windows XP and later, file system filter drivers shouldn't use any of the following fast I/O callback routines because they're obsolete.

  • AcquireForCcFlush
  • AcquireFileForNtCreateSection
  • AcquireForModWrite
  • ReleaseForCcFlush
  • ReleaseFileForNtCreateSection
  • ReleaseForModWrite

Filter Device Object Attached to a Volume - Windows drivers | Microsoft Learn

1 Like
6

I think I found the issue.

We are using ObQueryNameString on the FILE_OBJECT in the IRP_MJ_CREATE callback to get its name, but I think this API causes some performance hit when the file doesn't exists (?).

If the FILE_OBJECT has a RelatedFileObject, we pass that to ObQueryNameString, otherwise we pass the FILE_OBJECT itself.

My question is, is this a good approach? Can this be at fault for the performance issue? If so, what is the better approach to find the file name of the FILE_OBJECT in our IRP_MJ_CREATE callback?

7

Ah, yes, my bad...These specific Fast I/O entry points are in the subcategory that always go to the base FS Device Object and not the top of the stack. So, unlike the others, you don't have to deal with passing them through. And the need to hook them (if necessary) was replaced by a set of FsRtl callbacks...Good times.

8

Querying names is complicated and certainly can be expensive...This was another driving motivator for FltMgr, which is in a position to hide the complexity (for better or worse) and cache the results across multiple filters.

You haven't specified so I'm assuming that you're in your DIspatch Entry Point for IRP_MJ_CREATE (i.e. PreCreate) and not in a completion routine (i.e. PostCreate). This means you're dealing with an unopened FIle Object (FsContext == NULL) and here's a gross simplification of you query names for those:

  • If there's no RelatedFileObject, this is an absolute open and you get the name from FileObject->FileName
  • If there is a RelatedFileObject, this is a relative open and you need to reconstruct the absolute path.
    You first need to query the name of the RelatedFileObject by either directly sending an IRP_MJ_QUERY_INFORMATION/FileNameInformation or indirectly by calling ObQueryNameString. You then append FileObject->FileName to the result.

This of course introduces overhead even in cases where the open won't succeed (e.g. the path doesn't exist). You also need to take special care for, at a minimum, volume opens, FILE_OPEN_BY_FILE_ID, and SL_OPEN_TARGET_DIRECTORY.

Things are different in PostCreate because the open has already happened (FsContext != NULL) so you can just query the FileObject itself to get the absolute path. You also get to ignore unsuccessful opens and not add additional overhead in the negative case...However, whether or not you can do all of your processing in PostCreate depends very much on your design and what you're trying to accomplish. You can't just wholesale move your PreCreate processing to PostCreate and expect everything to work properly...

In either case you really need to cache the results in order to make any of this reasonable. That then leads to the problem of needing to invalidate those caches (e.g. on rename) and it's all a big mess really.

I'd be very wary of making any changes to this code. Hopefully this driver has an excellent test infrastructure and passes the HLKs, otherwise it's going to be very easy to regress the code in subtle ways. This is all very arcane stuff at this point.

9

Things are different in PostCreate because the open has already happened

Sadly this needs to be done in the precreate, because the purpose of this callback in the first place is to block certain file opens on protected files.

And creating an entire cache for this is not possible as it introduces way too much complexity as you already said, such as in cases of renames, etc.

So your suggestion is moving to a minifilter? Are you sure that moving to minifitler will improve performance in this case? If so, why? Does FltMgr implement a cache for the file names in the FILE_OBJECTs or...?

Is there any other way to fix this problem in the filter driver, instead of creating an entire cache for file objects? Maybe using another API for getting the file name or..?

10

Ultimately, yes. Filter Manager was introduced as a Service Pack Update to Win2K SP4 and XP SP2 to make it easier to write file system filter drivers. There have been no legacy samples provided by Microsoft since Vista. You're probably only one of a handful of people still working on one of these. Minifilters also have additional benefits, such as the ability to dynamically unload and better interoperability with the OS and other filters.

And, as I pointed out previously, there's already a Registry value that can be set to disable your driver from loading. That's a pretty clear warning that this type of driver could be entirely deprecated in a future release.

I don't have your design or your code so, no, absolutely not sure it's going to solve your performance problems. Filter Manager is itself a legacy file system filter, so it doesn't wholesale replace the way file system filtering works it just abstracts and rationalizes it for you. You can poorly design/implement a minifilter and have just as many problems as you do with a legacy filter. However, you'll have a much easier time getting help with minifilter questions and you'll have a lot less code to deal with.

You can read more about what Filter Manager is (and is not) here:

Understanding Minifilters: Why and How File System Filter Drivers Evolved – OSR

For opened File Objects FltMgr caches the name in a Stream Context. For unopened File Objects FltMgr caches the name along with a context associated with the IRP_MJ_CREATE IRP. And this cache is shared across minifilters, so once one minifilter queries the name the other minifilters get the benefit of the cache. This is yet another motivation for FltMgr as previously every filter had to have their own cache.

No, there's no magical way to get the name...If it's an absolute open (RelatedFIleObject == NULL) then you just copy the name out of the File Object and you're done. If it's a relative open, you need to get the name for the RelatedFileObject and the only way to do that is to ask the file system. This is typically where you would use caching via a Stream Context so that you wouldn't have to keep asking. (Note that Open By ID is a screw case and requires that you actually perform an open yourself so that you can query the resulting name of the File Object...Again FltMgr abstracts this for you and will cache the result.)

Aside from caching, the only obvious optimization is to only do this processing for operations that request data access or destructive create dispositions. If you're trying to "protect files" then there's no point in adding all this overhead to attribute queries. From there I'd say any additional effort is wasted and better spent working on porting this to a minifilter.

1 Like
11

I worked on a mini-filter that also had a performance problem related to a java build system incessantly attempting to re-open a lot of files that didn't exist. We ended up building a cache of 'doesn't exist' filenames to limit the damage. It was impressively effective but a total PITA as one had to be very careful about correctly invalidating cache entries.

2 Likes
12

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.