I’ve seen a lot of posts in the archives about the finicky nature of FltGetFileNameInformation() but can’t seem to figure out my exact problem. I’m calling FltGetFileNameInformation with FLT_FILE_NAME_NORMALIZED in my pre-create path. In the process of testing I found that if the create comes in with a short name (i.e. TargetFileObject->FileName = mytestdir1\LENGTH~1.TXT) then my call for a normalized file name returns something like \Devicestuff\Volumestuff\mytestdir1\LENGTH~1.TXT. The whole path is normalized except the actual file name.
Since I have the sort of filter that wants to copy this file under some conditions, I need the proper long name for the file. Any thoughts on what the problem is? Or how to get the long file name?
Does the file exist ? I would guess that if it doesn’t filter manager can’t normalize the final component’s name.
Another thing to try is to call FltGetTunneledName in post-create to see if returns something else.
Regards,
Alex.
This posting is provided “AS IS” with no warranties, and confers no rights.
You’ve read about the tunnel cache?
“Matthew Norgren” wrote in message
news:xxxxx@ntfsd…
I’ve seen a lot of posts in the archives about the finicky nature of
FltGetFileNameInformation() but can’t seem to figure out my exact problem.
I’m calling FltGetFileNameInformation with FLT_FILE_NAME_NORMALIZED in my
pre-create path. In the process of testing I found that if the create comes
in with a short name (i.e. TargetFileObject->FileName =
mytestdir1\LENGTH~1.TXT) then my call for a normalized file name returns
something like \Devicestuff\Volumestuff\mytestdir1\LENGTH~1.TXT. The whole
path is normalized except the actual file name.
Since I have the sort of filter that wants to copy this file under some
conditions, I need the proper long name for the file. Any thoughts on what
the problem is? Or how to get the long file name?
Yes thank you, I was not paying attention to what volumes were involved. The file does not exist on the volume specified in my callback data when I try to get the file name information. Since the call did not fail outright and I knew the file to exist on other volumes I didn’t suspect that to be the problem.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alexandru Carp
Sent: Wednesday, November 05, 2008 4:07 PM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] trouble getting normalized file name
Does the file exist ? I would guess that if it doesn’t filter manager can’t normalize the final component’s name.
Another thing to try is to call FltGetTunneledName in post-create to see if returns something else.
Regards,
Alex.
This posting is provided “AS IS” with no warranties, and confers no rights.
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
No virus found in this incoming message.
Checked by AVG - http://www.avg.com
Version: 8.0.175 / Virus Database: 270.8.6/1765 - Release Date: 11/4/2008 8:26 AM
>Since I have the sort of filter that wants to copy this file under some conditions, I need the proper long
name for the file.
- enumerate all files in the directory
- query FileAlternateNameInformation for each
- if it matches to the short name - then query FileNameInformation
- repeat for each next “word” in the pathname after each slash.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
Isn’t that a race condition with respect to file renaming or creation or deletion?
-----Original Message-----
From: Maxim S. Shatskih
Sent: Thursday, November 06, 2008 15:11
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] trouble getting normalized file name
>Since I have the sort of filter that wants to copy this file under some conditions, I need the proper long
>name for the file.
- enumerate all files in the directory
- query FileAlternateNameInformation for each
- if it matches to the short name - then query FileNameInformation
- repeat for each next “word” in the pathname after each slash.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
—
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
It is.
Can you suggest a better solution?
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
“Skywing” wrote in message news:xxxxx@ntfsd…
Isn’t that a race condition with respect to file renaming or creation or deletion?
- S
-----Original Message-----
From: Maxim S. Shatskih
Sent: Thursday, November 06, 2008 15:11
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] trouble getting normalized file name
>Since I have the sort of filter that wants to copy this file under some conditions, I need the proper long
>name for the file.
- enumerate all files in the directory
- query FileAlternateNameInformation for each
- if it matches to the short name - then query FileNameInformation
- repeat for each next “word” in the pathname after each slash.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
—
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
I can. Use FltGetFileNameInformation. 
Regards,
Alex.
This posting is provided “AS IS” with no warranties, and confers no rights.
Yes, of course, you’re right about that - but didn’t you know, file system
filters have to live with all sorts of race conditions
“Skywing” wrote in message news:xxxxx@ntfsd…
Isn’t that a race condition with respect to file renaming or creation or
deletion?
- S
-----Original Message-----
From: Maxim S. Shatskih
Sent: Thursday, November 06, 2008 15:11
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] trouble getting normalized file name
>Since I have the sort of filter that wants to copy this file under some
>conditions, I need the proper long
>name for the file.
- enumerate all files in the directory
- query FileAlternateNameInformation for each
- if it matches to the short name - then query FileNameInformation
- repeat for each next “word” in the pathname after each slash.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
—
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Hi Alex
Fascinating post, challenging response
a. Filter manager team has solved all of the race conditions (please tell us
how).
b. Filter manager uses super secret undocumented api (please tell us more!
also does it work for third part file systems?)
c. something completely different (please tell us)
Thanks,
Lyndon
“Alexandru Carp” wrote in message
news:xxxxx@ntfsd…
I can. Use FltGetFileNameInformation.
Regards,
Alex.
This posting is provided “AS IS” with no warranties, and confers no rights.
Heh…
You know, the more I think about file systems the more I realize that they’re very much like quantum physics (Windows file systems at least). Questions that are simple to ask are sometimes impossible to answer. I think it’s because we got so used to lying (as Tony says, as file system developer we’re all liars) that the model that is exposed to the user is too different from what is actually going on (very similar to Newton’s law vs. relativity and quantum mechanics). So at this point I’m thinking that it’s impossible to solve all race conditions anyway. And some questions like “How can I know when a file has been deleted?” or “What is the name of this file NOW?” might seem to make sense, but in fact they don’t. Sure, the corner cases are very rare but we want 100% certainty.
On the other hand, I’ve been told that it is possible that a broken cup will un-break itself (though with a very low probability; haven’t seen that happen yet; maybe I should break more cups). So if in nature you can’t be 100% sure that if you initiate breaking a cup that it will actually be broken…
Anyhow, I still recommend using FltGetFileNameInformation. There aren’t any undocumented APIs at work and I’m sure many of the veterans on this list have implementations that are at least as reliable as ours, but for people that don’t have one I still think it’s the better choice. Not looking to start a flame war here or anything, but I hate it when I read quotes like " I’ve seen a lot of posts in the archives about the finicky nature of FltGetFileNameInformation() but can’t seem to figure out my exact problem. ". I still believe that the perceived “finickyness” of this API is due in big part to the way the system works and all the things that must be taken into account (as well as to the people on this list complaining about it).
And I would advise (again) that anyone that starts writing their own function to achieve this should make sure that they understand the exact limitation of FltGetFileNameInformation that they are trying to work around or they might realize at some point (most likely after a lot of code has been written) that the limitation is in the system and not in the function. There was a nice quote somewhere which I think applies to this quite well: “Printf is not broken”…
There was another thread on this a couple of months back and I won’t repeat what was said there. Just don’t dismiss FltGetFileNameInformation() too quickly, it’s probably the easiest way out.
Regards,
Alex.
This posting is provided “AS IS” with no warranties, and confers no rights.
Hey Alex,
I’ve been meaning to say thanks, by the way, for getting the
documenation updated about *WHY* FltGetFileNameInformation may return
STATUS_FLT_INVALID_NAME_REQUEST. I seem to recall it was fairly
uninformative about what could cause it to fail (“sometimes it just
doesn’t work, and you’re shit out of luck!”). It’s awfully hard to make
a decision to use something that you know doesn’t work all the time but
you don’t know when it won’t work.
Now that it’s at least documented it’s a lot easier to know when it’s
okay to use it and when you shouldn’t. Take solace, we do sometimes
notice the little things 
Please pass on my thanks to the doc folks as appropriate.
Speaking of little things, emails seem to be coming faster recently,
thanks OSR. At least for me, somebody else was complaining about it
being slower earlier today. You can’t please everybody.
~Eric
Yes, the listserv has been processing mail much faster in recent days. Used to regularly get 10-15 minute delays, but SNoone seems to have repaired it.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Eric Diven
Sent: Thursday, November 06, 2008 7:58 PM
To: Windows File Systems Devs Interest List
Subject: RE: Re:[ntfsd] Re:Re:trouble getting normalized file name
Hey Alex,
I’ve been meaning to say thanks, by the way, for getting the
documenation updated about *WHY* FltGetFileNameInformation may return
STATUS_FLT_INVALID_NAME_REQUEST. I seem to recall it was fairly
uninformative about what could cause it to fail (“sometimes it just
doesn’t work, and you’re shit out of luck!”). It’s awfully hard to make
a decision to use something that you know doesn’t work all the time but
you don’t know when it won’t work.
Now that it’s at least documented it’s a lot easier to know when it’s
okay to use it and when you shouldn’t. Take solace, we do sometimes
notice the little things
Please pass on my thanks to the doc folks as appropriate.
Speaking of little things, emails seem to be coming faster recently,
thanks OSR. At least for me, somebody else was complaining about it
being slower earlier today. You can’t please everybody.
~Eric
NTFSD is sponsored by OSR
For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Very good point Alex. The problem with asynchronous systems are that
operations are inherently not ordered. There are people that exploit
this (I learned the term “Heisenbug” from some folks that learned a long
time ago that repeating the same operations on a different system yields
different outcomes…)
Naming is a total swamp in Windows. It isn’t going to get any better.
Building a system to handle it from scratch isn’t going to be easier
than understanding the limitations of Filter Manager’s implementation
and it’s going to take a lot of time to get there. When you do arrive,
you’ll find you have similar complex limitations.
Tony
OSR
I would take this one step further. With a stack model where locks cannot be
held above the lowest layer, it is actually impossible to know what the
order of operations are.
What we discovered in name changer was that without serializing renames, we
could not determine weather or not the file was in or out of the mapping. As
it turns out this can be generalized to say, “It is impossible to keep
accurate metadata in a context which mirrors the filesystem’s state.” This
is because the system is asynchronous, and there is no indication as to
which order the operations were completed in.
It seems like that is the dirty little secret about minifilters. People
typically work in create, which we know is raceless. When dealing with other
operations the trick is how do be stateless, because managing state is such
a pain in the butt.
As for these guys trying to know when a file is deleted: good luck. Keeping
the metadata in your context accurate is, in my opinion, impossible.
–Andrew
On 11/6/08 5:45 PM, in article xxxxx@ntfsd, “Alexandru Carp”
wrote:
> You know, the more I think about file systems the more I realize that they’re
> very much like quantum physics (Windows file systems at least). Questions that
> are simple to ask are sometimes impossible to answer. I think it’s because we
> got so used to lying (as Tony says, as file system developer we’re all liars)
> that the model that is exposed to the user is too different from what is
> actually going on (very similar to Newton’s law vs. relativity and quantum
> mechanics). So at this point I’m thinking that it’s impossible to solve all
> race conditions anyway. And some questions like “How can I know when a file
> has been deleted?” or “What is the name of this file NOW?” might seem to make
> sense, but in fact they don’t. Sure, the corner cases are very rare but we
> want 100% certainty.
–Andrew Thomson
–Microsoft
Wow. I’ve been beating my drum to the “you can’t do that” song solo for
so long, it’s strange to suddenly hear others picking up the song as
well.
One thing you noted is of particular note Andrew: you can’t take out
locks above the file system driver. People do try it and always seem to
get burned by it (read: “they deadlock.”)
Heck, getting locking right at the bottom layer is tough. One of the
amazing things to see in our own file systems kit are the sheer number
of race conditions that exist *within the FSD layer*.
One of my favorites is when we create a brand new file - we handle (and
have actually seen) cases in which between the time we get the file
visible on the disk and the time we return to the caller that the file
has gone away. There is so much work to do in a create operation that
we have to worry about all the race conditions (about four years back,
we did a big chunk of work to parallelize the create operations as
well.) When you consider how this plays with hard links (for example,)
destructive rename, etc, you realize how hard it is to get this right
*when you can safely lock things.*
So, you find the file, you start to set up the data structures and you
find that someone else has swooped in and deleted the file while you’re
trying to open it. Serializing this fixes the problem, but becomes a
choke point on performance. Not serializing it means you have these
amazing race conditions you must handle (I’m sick of doing “look up the
object I want, grab the lock I want that depends on the object I just
looked up, look it up again to ensure I still have the right lock, and
proceed…” but it’s amazing how nicely parallel it gets. Anyone need
to process thousands of simultaneous IRP_MJ_CREATE operations? We do
it…")
Thanks for the observation, and thanks for picking up the harmony on
this particular battle cry. Asynchronous systems are amazing, but like
Alex suggested, trying to predict their results is more like describing
quantum mechanics (“probability clouds”) than classical mechanics
(“Newton’s Laws.”)
Tony
OSR
Interesting thread.
My observation is that you can (have to?) bring these things back to
requirements.
Take the thread about “how do I know whether a file has been deleted”. It
may well be (as somneone said), that the question was in support of an
undelete filter. In that case a better question would be “How can I find
out that a file *might* be deleted”. If you can build the rest of your
system to be tolerant to being told that a file is deleted when it hasn’t
been then you just end up with an exercise in optimization. So long as you
only ever get false positives (and no false negatives) you will be fine.
In this case what is the requirement? Do you just need something friendly
looking to throw at the user? Well it doesn’t really matter if there is a
race condition. If however you need to get a moniker to allow you to reopen
exactly the same file then you may well be SOL. In that case there are
other better technques like File ID or relative opens.
In some cases there really is no right answer (in my example above RDR
spoils the day by being broken for both relative opens and for querying the
FID). So you then need to work out what the failure cases would look like
and work out whether they matter in the face of your requirements. If they
do, you then have to take a judgement on which is the least bad solution in
the face of your requirements. We do this a lot - CRC, signing and
encyrption all involve the possibility that the “cup will reform itself”,
but for the implementors it is considered to be the least bad solution in
the face of the requirements.
Rod
Well. Alex, I’ve been using FltGetFileNameInformation for quite a while. I’m
still very happy to dump my old file name retrieval and normalization code
“Alexandru Carp” wrote in message
news:xxxxx@ntfsd…
Heh…
You know, the more I think about file systems the more I realize that
they’re very much like quantum physics (Windows file systems at least).
Questions that are simple to ask are sometimes impossible to answer. I think
it’s because we got so used to lying (as Tony says, as file system developer
we’re all liars) that the model that is exposed to the user is too different
from what is actually going on (very similar to Newton’s law vs. relativity
and quantum mechanics). So at this point I’m thinking that it’s impossible
to solve all race conditions anyway. And some questions like “How can I know
when a file has been deleted?” or “What is the name of this file NOW?” might
seem to make sense, but in fact they don’t. Sure, the corner cases are very
rare but we want 100% certainty.
On the other hand, I’ve been told that it is possible that a broken cup
will un-break itself (though with a very low probability; haven’t seen that
happen yet; maybe I should break more cups). So if in nature you can’t be
100% sure that if you initiate breaking a cup that it will actually be
broken…
Anyhow, I still recommend using FltGetFileNameInformation. There aren’t any
undocumented APIs at work and I’m sure many of the veterans on this list
have implementations that are at least as reliable as ours, but for people
that don’t have one I still think it’s the better choice. Not looking to
start a flame war here or anything, but I hate it when I read quotes like "
I’ve seen a lot of posts in the archives about the finicky nature of
FltGetFileNameInformation() but can’t seem to figure out my exact problem.
". I still believe that the perceived “finickyness” of this API is due in
big part to the way the system works and all the things that must be taken
into account (as well as to the people on this list complaining about it).
And I would advise (again) that anyone that starts writing their own
function to achieve this should make sure that they understand the exact
limitation of FltGetFileNameInformation that they are trying to work around
or they might realize at some point (most likely after a lot of code has
been written) that the limitation is in the system and not in the function.
There was a nice quote somewhere which I think applies to this quite well:
“Printf is not broken”…
There was another thread on this a couple of months back and I won’t repeat
what was said there. Just don’t dismiss FltGetFileNameInformation() too
quickly, it’s probably the easiest way out.
Regards,
Alex.
This posting is provided “AS IS” with no warranties, and confers no rights.