Get Normalized name when FltGetFileNameInformation fails

I have a minifilter based on the Microsoft sample, it works mostly fine, but the problem is that I need normalized names only, and FltGetFileNameInformation call with FLT_FILE_NAME_NORMALIZED fails occasionally.
I don’t filter paging IO.

The obvious solution seems to be, use FltDoCompletionProcessingWhenSafe or FltQueueDeferredIoWorkItem. But I don’t like it because delaying IO completion will defrade performance (besides, these functions are only allowed for IRP operations).

What am I supposed to do? Opened names are useless for my task. Is there some approach for solving this problem?

The normalised name is orthogonal to the opened name. Normalisation is about removing short names as such like, so you might get away with just not specifying any bits.

What I usually do is ask for the normalized name and if that fails ask for non normalized since my application can handle short names. In your case if you have to have the normalized name you could chose to post only when the normalisation fails.

What scenarios does it fail in exactly? Can you handle such cases in
post-create?

I have a minifilter based on the Microsoft sample, it works mostly fine, but
the problem is that I need normalized names only, and
FltGetFileNameInformation call with FLT_FILE_NAME_NORMALIZED fails
occasionally.

The obvious solution seems to be, use FltDoCompletionProcessingWhenSafe or
FltQueueDeferredIoWorkItem. But I don’t like it because delaying IO
completion will defrade performance (besides, these functions are only
allowed for IRP operations).

What am I supposed to do? Opened names are useless for my task. Is there
some approach for solving this problem?

@rod_widdowson said:
The normalised name is orthogonal to the opened name. Normalisation is about removing short names as such like, so you might get away with just not specifying any bits.

Normalization also resolves reparse points, which is also important for me.

“Not specifying any bits” you mean, pass NameOptions = 0? I didn’t know you can do it, but even if zero is a valid parameter, I’m not sure it will not fail just as a normalized request.

if you have to have the normalized name you could chose to post only when the normalisation fails.

I’m sorry, by “post” you mean post a work item? If yes, I thought about that, but as I said, I’m afraid it may slow down some important low-level IO, and besides, it’s not allowed for FASTIO (according to msdn).

@Dejan_Maksimovic said:
What scenarios does it fail in exactly? Can you handle such cases in post-create?

I call FltGetFileNameInformation for every modifying IO (not just create, but also WRITE, SET_INFORMATION, CLEANUP after DeleteOnClose, etc), so I’m not sure about exact scenarios. I assume it can happen in many cases.

In case of CREATE, FltGetFileNameInformation is called in post-create only. For other IO, pre-operation only.
I assume, if FltGetFileName has failed in pre-, it will also fail in post. Is that correct?

I actually have a workaround, but it’s cumbersome and still not 100% reliable.

Whenever normalization fails, ask for opened name and pass that name to a worker thread. In the worker thread, open the file (via the obtained opened name) and safely ask filesystem for the normalized name. The problem is that the file may be already deleted or renamed.

I also filter delete, rename, createhardlink so in theory I can handle deletions, but what if normalization fails in pre-delete? Then I’m screwed and the real name of the file is lost forever.

That’s not the right or good way to get names. Get it in Post-Create
(if possible, as doing it in pre-create is slower), add it to your
stream/handle/file context, and use that context in all other I/O
paths.

FGFNI is guarnateed to fail in several casse, when the I/O is not
pre/post-create. There are also rare cases where it can fail in
pre-create, but will succeed in post-create.

By post, I meant post-opeartion callback, not posting to a queue.

@Dejan_Maksimovic said:
That’s not the right or good way to get names. Get it in Post-Create
(if possible, as doing it in pre-create is slower), add it to your
stream/handle/file context, and use that context in all other I/O
paths.

Thanks for the idea.
But what about renames? I’ve read somewhere that FGFNI is the most reliable way to get names because the system handles all cases such as rename and ensures you always get the actual name.

Additionally, what if the file was opened before the driver starts? I still have to call GetName for operations other than CREATE at least once. (the driver is running on servers, so we can’t afford to reboot every time it restarts). Granted, it’s a rare case and the probability of failure is much lower.

FGFNI is guarnateed to fail in several casse, when the I/O is not
pre/post-create. There are also rare cases where it can fail in
pre-create, but will succeed in post-create.

I don’t need names in pre-create so that’s not an issue.

By post, I meant post-opeartion callback, not posting to a queue.

I understand that, that response wasn’t to you.

Call FltGetDESTINATIONFileName for a rename destination file name.

You cannot be sure you will always get a file name for files opened
before your driver was loaded, unfortunately. But those cases
(including cases where FGFNI might fail) are actually rare enough
that, combined with keeping a file name so you don’t query on every
I/O but keep it in your context, a post to a work queue will be of no
impact to performance.

Thank you, I was afraid I’ll have to do that, but probably you’re right, it’s the only way.

Is it guaranteed that post create is absolutely 100% always called at PASSIVE_LEVEL ?

Yep, PostCreate is guaranteed to also be in the same process context
as the pre-create. That’s not the case for other operations.

Note that stream file objects will often be created, without you
seeing any IRP_MJ_CREATE. This is, unfortunately, "normal.
IIRC, you would have a stream/file context associated with such a
file, but not a handle context (I mean you would provided you
associated a stream/file context with the file in IRP_MJ_CREATE for
the same file before).
I.e.:

  • IRP_MJ_CREATE arrives for a file
  • you create and associate a stream, file and handle object with it
    At this point, it is possible that the FS will create stream file
    objects (not an file for the file streams!), for which you see no
    IRP_MJ_CREATE
    In read/write/close, you check for your contexts, but only stream/file
    would be present, not handle context.

Thank you, I was afraid I’ll have to do that, but probably you’re right,
it’s the only way.

Is it guaranteed that post create is absolutely 100% always called at
PASSIVE_LEVEL ?

I was thinking about this approach (get name in Create and save it in the context), and I’m still wondering if it’s the most efficient way. The reason for my doubts is the comments of Alex Carp (ex-Microsofot, the author of http://fsfilters.blogspot.com) that I read before.

Basically, he’s saying that calling FltGetFileNameInformation every time is better because it handles all possible cases of name invalidation and it internally does the same as manually storing the name in stream handle context. So I’m still confused, should I really do it, or it can make things worse?

Besides, I would still have to defer completion of some operations in cases when the name wasn’t obtained yet. Wouldn’t it be the same as delaying completion with FltDoCompletionProcessingWhenSafe and then call FltGetFileNameInformation in the delayed post-op? (in theory, I’d have to do that just once, since the name is cached).

https://community.osr.com/discussion/comment/214149/#Comment_214149

Knowing when the name has become invalid is not that easy. You listed postCreate and postRename but in my oppinion that’s an oversimplication. What about overwriting renames? Hardlink creation ? Overwriting hardlink creation? Directory rename (which invalidates all the names in all the files under that directory) ? What about setting a short name on a file or directory which might invalidate some names (opened names) but not others (normalized names)? FltMgr’s name caching is implemented to take these cases into account, but keeping a reference to a name obtained at some point in the past doesn’t guarantee that the name is still accurate. Basically, calling FltGetFileNameInformation() every time is the right way to do it because if FltMgr has seen a condition that required invalidation of the name then you’ll get a new name. Also, the name is cached so it won’t cost you much more in terms of performance than keeping your own reference.

https://community.osr.com/discussion/comment/163696/#Comment_163696

Also, storing the name of the file in a stream context does not help in this case. It only works if you don’t care if it’s accurate at all or if you only want to reflect the name of the file when it was created. However, if you want to report the name of the file as best you can at the time operations happen then you have the issues you’ve already found and IMO you are better off using filter manager’s APIs all the time, because the name cache in filter manager is nothing but a stream handle context for the name, with a lot of optimizations.
So now we’ve come full circle :). FLT_FILE_NAME_QUERY_ALWAYS_ALLOW_CACHE_LOOKUP means ALWAYS look in the cache first, and if the name isn’t there then try to get it from the file system, which is exactly what you would end up implementing if you tried to have your own caching scheme.

Curiously, in one of his comments he links to his own blog post, where he says the exact opposite:

http://fsfilters.blogspot.com/2010/02/names-and-file-systems-filters.html

It is a bad idea to check if the file is interesting by querying and parsing the file name every single time the filters needs to know this. A better design is to cache the information about the file somewhere and then update it only when it changes. Since we are talking about name based policy here, the only place where it can change is in the rename path. Stream contexts are particularly suited for this task and what filters normally do is attach a stream context if the file is interesting. Then, when they need to decide whether the file is interesting or not they can simply get the stream context and if one is present then it is interesting.
The stream context is initialized at create time and is potentially changed at rename. Both these operations happen at PASSIVE_LEVEL. Some filters prefer to query the name when the operation they care about happens, but this approach usually generates more problems that it solves.

Wait, there’s another problem. Actually, the same problem.

Regardless of using or not using stream context handle to cache names, the original problem still persists. I still sometimes need to get a normalized name not in Create at least once, when I didn’t see Create and FGFNI fails. How do I do it? I though about posting a work item or using FltDoCompletionProcessingWhenSafe, but they have the same issues:

FltQueueDeferredIoWorkItem - can return STATUS_FLT_NOT_SAFE_TO_POST_OPERATION.
FltQueueGenericWorkItem - doesn’t perform the safety checks, but another comment says “even if you use FltQueueGenericWorkItem for pending IO you should
implement same parameter checking as FltQueueDeferredIoWorkItem perform.”

Return FLT_PREOP_SYNCHRONIZE - “Minifilter drivers must never return FLT_PREOP_SYNCHRONIZE for asynchronous read or write operations. Doing so can … cause deadlocks”

FltDoCompletionProcessingWhenSafe - only checks for IRQL <= APC_LEVEL and doesn’t check TopLevelIrp, so it’s possible that calling FGFNI in it can still fail.

What do I do?

Wait, there’s another problem. Actually, the same problem.

Regardless of using or not using stream handle context for caching names, the same problem persists. I still need to get normalized name not in Create at least once, when I didn’t see Create and FGFNI fails. I thought about using work items or DoCompletionWhenSafe, but they have the same issues:

FltQueueDeferredIoWorkItem - can return STATUS_FLT_NOT_SAFE_TO_POST_OPERATION, just like FGFNI when it’s not safe.
FltQueueGenericWorkItem - doesn’t perform the safety checks, but “even if you use FltQueueGenericWorkItem for pending IO you should
implement same parameter checking as FltQueueDeferredIoWorkItem perform”
.
FLT_PREOP_SYNCHRONIZE - “Minifilter drivers must never return FLT_PREOP_SYNCHRONIZE for asynchronous read or write operations. Doing so can … cause deadlocks”.
FltDoCompletionProcessingWhenSafe - only checks for IRQL, but doesn’t check TopLevelIrp, so calling FGFNI in it probably can still fail.

What do I do?

Maybe FLT_PREOP_SYNCHRONIZE is the way? I’m only worried about possible deadlocks in case of asynchronous writes.
Maybe it’s not an issue since I don’t filter paging operations? MSDN says “FLT_PREOP_SYNCHRONIZE … can cause deadlocks if, for example, the modified page writer thread is blocked”.
Can modified page writer thread be blocked during non-paging IO?

It womt fail in pre or post create if the name can be retrieved at all.
So it does remove the need to post the call.

Invalidation xan occur in post create and rename IIRC?
Im which case you just replace yoir context.

If FltMgr were sure to keep a cache entry, you could query it all the time,
by specifying ONLY ALLOW CACHE LOOKUP during nonCreate events.

Note: The email was trying to reply to an invalid Comment (292997).

@Dejan_Maksimovic said:
It womt fail in pre or post create if the name can be retrieved at all.
So it does remove the need to post the call.

Yes, but I’m still trying to figure out what to do if I need to get name outside of CREATE? As I said before, some files are opened before my driver starts.

@Dejan_Maksimovic said:
If FltMgr were sure to keep a cache entry, you could query it all the time,
by specifying ONLY ALLOW CACHE LOOKUP during nonCreate events.

My idea was that in cases when FGFNI fails during nonCreate, just delay the IO and call FGFNI later when it’s safe. So that the name cache gets updated, and FGFNI won’t fail next time.

And I will have to implement something like that anyway even if I will use stream context (because, as I said, some creates happen before my driver starts).

Here’s the problem as I see it: FltGetFileNameInformation() can fail if IoGetTopLevelIrp() != 0 (among other reasons). In that case, I can’t use work items because FltQueueDeferredIoWorkItem() will also fail, and FltQueueGenericWorkItem() is not a good idea for the same reason. FltDoCompletionProcessingWhenSafe() is useless, because it doesn’t check for IoGetTopLevelIrp().

MSDN says: “If your minifilter driver cannot handle such failures, you should consider using FLT_PREOP_SYNCHRONIZE instead of pending the I/O operation”.

So I’m left with FLT_PREOP_SYNCHRONIZE, but the question is will it work? Synchronized postop is executed at IRQL <= APC_LEVEL, but MSDN says nothing about IoGetTopLevelIrp().

I found an MS presentation, that says “Calling FltGetFileNameInformation will work if the operation is synchronized”, which is good, but it’s not clear if synchronization helps when IoGetTopLevelIrp() != 0.

Another clue from Alex Carp’s blog:

http://fsfilters.blogspot.com/2010/12/more-thoughts-on-fltdocompletionprocess.html

To avoid deadlocks, minifilters should not perform synchronous requests from a postOp callback and should instead either:
queue the operation and return FLT_POSTOP_MORE_PROCESSING_REQUIRED from the postOp callback or
return FLT_PREOP_SYNCHRONIZE from the preOp

So I hope FLT_PREOP_SYNCHRONIZE is the way to get FltGetFileNameInformation() succeed, but I’m not really sure. Any thoughts?

You must not synchronize READ/WRITE requests.

Try a test version which DbgPrints exact locations, cases and error
codes returned when FGFNI fails.

I doubt FltDoCompletionProcessingWhenSafe would not post to a queue
where FGFNI call is safe in cases where the call itself fails.

Mind you, I have seen FGFNI fail often enough, that there is no good
answer to the problem, other than being a boot driver. In which case,
you can propagate the failures (if you keep the name in your context!)