Yet another FltGetFileNameInformation problem ;)

OSR_Community_User · September 13, 2007, 8:36am

(I’m making a list of these already, and will post bug descriptions,
affected cases and possible workarounds as soon as time permits)
Sometimes you cannot retrieve the normalized file name during
post-create, as ALL the name query APIs will return
STATUS_FLT_INVALID_NAME_REQUEST. Even FltGetFileNameInformationUnsafe.
That lead me to think the file is not actually opened, but
Data->IoStatus.Status is STATUS_SUCCESS.
On XP, this is reproducible by renaming a file on a network share.
C01C0005 is returned by all APIs for target rename open.
I don’t have time to figure why the error is returned, as I’m
looking for a workaround.

If anyone is considering whether to go FltMgr or legacy route for a
new project, considering the amount of issues I have seen with the basic
file name query APIs only (not to mention several others issues), I
would strongly encourage you not to go FltMgr route!

–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.

OSR_Community_User · September 18, 2007, 2:51pm

I don’t know if you are implementing these on a local file system or a remote file system, but our experience has been that names on remote file systems are, at best, problematic to retrieve.

Fortunately, the same techniques that we’ve been using over the years can be fitted into the mini-filter model when necessary.

Tony

Tony Mason
Consulting Partner
OSR Open systems Resources, Inc.
http://www.osr.com

OSR_Community_User · September 18, 2007, 3:50pm

It’s a local file name. All these “can’t do it here and you shouldn’t do it there” things make the API fairly… not useless, but an almost useless API. If we need to work around them all
the time, we could have kept the legacy model simply

xxxxx@osr.com wrote:

I don’t know if you are implementing these on a local file system or a remote file system, but our experience has been that names on remote file systems are, at best, problematic to retrieve.

Fortunately, the same techniques that we’ve been using over the years can be fitted into the mini-filter model when necessary.

–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.

OSR_Community_User · September 18, 2007, 8:48pm

The naming functionality is a ‘library’ API set of fltmgr, not the core i/o path. Choosing to write a legacy filter for this reason, is essentially saying that since this is hard, I want to inherit all the other hard things about writing filters and making them interop with each other, as well. Getting full file names is one of the trickiest and also performance intensive operations one can attempt, and also one of the riskiest (apart from stuff such as blocking in paging i/o path), given that there is no native file system support for caching, and keeping file names up to date as files get renamed.

There are valid reasons why filter manager does not return a name sometimes: for instance FltGetFileNameInformationUnsafe() will refuse to return a name on a cleaned-up file object, because the file system natively does not guarantee that the name is valid (i.e a parent could have been renamed, since the semantics is that the parent is only guaranteed to be not renamed while there is a valid handle open). This API is really a helper API for non-file system filters to use, since they are already safe from recursion issues.

Having said that - if you can revive the specific instances in which the API failed and you feel it should not, and send it it would be valuable - as there will either be a good reason or there is a genuine bug, in which case I will forward it appropriately to see if we can resolve it. I know you probably did that over several threads, but if you can re-compile that and send it out it’d be great.

Thanks, Ravi

From: xxxxx@lists.osr.com [xxxxx@lists.osr.com] On Behalf Of Dejan Maksimovic [xxxxx@alfasp.com]
Sent: Tuesday, September 18, 2007 12:49 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Yet another FltGetFileNameInformation problem

It’s a local file name. All these “can’t do it here and you shouldn’t do it there” things make the API fairly… not useless, but an almost useless API. If we need to work around them all
the time, we could have kept the legacy model simply

xxxxx@osr.com wrote:

I don’t know if you are implementing these on a local file system or a remote file system, but our experience has been that names on remote file systems are, at best, problematic to retrieve.

Fortunately, the same techniques that we’ve been using over the years can be fitted into the mini-filter model when necessary.

–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.

NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · September 18, 2007, 10:28pm

Hi,

I would agree with that, however it’s these “library” APIs that make using FltMgr the preferred choice. I never had a stack overflow happen with a legacy filter, never had to investigate MS API calls during a crash dump.
FltMgr is a great wrapper, the overall “modularity” is much better, but that’s not enough… the library should be there, fully working so common stuff is not something we need to crash-track
That said, all the cases I have had problems with are either pre-create or post-create, and all completely legal as per documentation. I will compile a list of cases, and e-mail them to you ASAP.

Ravi Pudipeddi wrote:

The naming functionality is a ‘library’ API set of fltmgr, not the core i/o path. Choosing to write a legacy filter for this reason, is essentially saying that since this is hard, I want to inherit all the other hard things about writing filters and making them interop with each other, as well. Getting full file names is one of the trickiest and also performance intensive operations one can attempt, and also one of the riskiest (apart from stuff such as blocking in paging i/o path), given that there is no native file system support for caching, and keeping file names up to date as files get renamed.

There are valid reasons why filter manager does not return a name sometimes: for instance FltGetFileNameInformationUnsafe() will refuse to return a name on a cleaned-up file object, because the file system natively does not guarantee that the name is valid (i.e a parent could have been renamed, since the semantics is that the parent is only guaranteed to be not renamed while there is a valid handle open). This API is really a helper API for non-file system filters to use, since they are already safe from recursion issues.

Having said that - if you can revive the specific instances in which the API failed and you feel it should not, and send it it would be valuable - as there will either be a good reason or there is a genuine bug, in which case I will forward it appropriately to see if we can resolve it. I know you probably did that over several threads, but if you can re-compile that and send it out it’d be great.

Thanks, Ravi

–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.

Tracy_Camp · September 19, 2007, 12:56pm

The issue here is that fltmgr is essentially an abstraction layer over
some fairly complex stuff. Like all abstraction layers it will tend to
leak complexity through (i.e. ‘bugs’, ‘caveats’ etc.) and will probably
not be capable of solving all of the problems that could be solved by the
more complex layer it is abstracting (a very concerning factor in the
persistent rumours that “legacy” filters will be banned in the future).
The hard problems remain hard. Once you stray away from file systems that
are called “NTFS.sys” your results are likely to vary.

The trouble with abstraction layers is that since the abstraction looks
simpler, the parts that will leak complexity through are not necissarily
obvious since the dizzying array of implementation detail that a filter
driver would need to implement are not visible for display. I think it is
entirely fair at this point to say that FltMgrGetFileNameInformation()
leaks complexity and will probably always leak complexity.

We will always need ntfsd, there will always be business of consulting
firms that sell more purpose built abstraction toolkits etc.

t.

On Wed, 19 Sep 2007, Dejan Maksimovic wrote:

Hi,

I would agree with that, however it’s these “library” APIs that
make using FltMgr the preferred choice. I never had a stack overflow
happen with a legacy filter, never had to investigate MS API calls
during a crash dump.
FltMgr is a great wrapper, the overall “modularity” is much
better, but that’s not enough… the library should be there, fully
working so common stuff is not something we need to crash-track
That said, all the cases I have had problems with are either
pre-create or post-create, and all completely legal as per
documentation. I will compile a list of cases, and e-mail them to you
ASAP.

Ravi Pudipeddi wrote:

> The naming functionality is a ‘library’ API set of fltmgr, not the core
> i/o path. Choosing to write a legacy filter for this reason, is
> essentially saying that since this is hard, I want to inherit all the
> other hard things about writing filters and making them interop with
> each other, as well. Getting full file names is one of the trickiest
> and also performance intensive operations one can attempt, and also one
> of the riskiest (apart from stuff such as blocking in paging i/o path),
> given that there is no native file system support for caching, and
> keeping file names up to date as files get renamed.
>
> There are valid reasons why filter manager does not return a name
> sometimes: for instance FltGetFileNameInformationUnsafe() will refuse
> to return a name on a cleaned-up file object, because the file system
> natively does not guarantee that the name is valid (i.e a parent could
> have been renamed, since the semantics is that the parent is only
> guaranteed to be not renamed while there is a valid handle open). This
> API is really a helper API for non-file system filters to use, since
> they are already safe from recursion issues.
>
> Having said that - if you can revive the specific instances in which
> the API failed and you feel it should not, and send it it would be
> valuable - as there will either be a good reason or there is a genuine
> bug, in which case I will forward it appropriately to see if we can
> resolve it. I know you probably did that over several threads, but if
> you can re-compile that and send it out it’d be great.
>
> Thanks, Ravi

–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.

NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@openmars.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · September 19, 2007, 8:48pm

> The issue here is that fltmgr is essentially an abstraction layer over some

fairly complex stuff. Like all abstraction layers it will tend to leak
complexity through (i.e. ‘bugs’, ‘caveats’ etc.) and will probably not be
capable of solving all of the problems that could be solved by the more
complex layer it is abstracting (a very concerning factor in the
persistent rumours that “legacy” filters will be banned in the future).

I’ll agree to the the leaking complexity, however, the cases I’ve
mentioned are far from complex.
Sticking the particular API in question, I understand it cannot always
work in read/write/SetInfo/etc. paths. However, I would expect it to always
work in pre/post-create for either Opened or Normalized file name (not both
but at least one). That is how the complex legacy world worked.
The feeling I have is that I have switched one problem for another
I never had big issues with the path-query paths (once we ironed out different
network redirectors we had no issues). But debugging a mini-filter is much
easier.

–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.

OSR_Community_User · September 26, 2007, 2:31pm

Dejan,

I am interesting in understanding the pain points you are seeing with this API. I believe some of the problems you are seeing with this API you would also see with a legacy filter. For example your complaint about not getting a name after a rename of a remote file exists in legacy filters as well.

Just to be perfectly clear, there are plans in a future release to not support legacy filters. It is important for all vendors to move forward as appropriate with a minifilter implementation. If you have problems please work with Microsoft to address those issues because we want this environment to be as robust and reliable as possible

Neal Christiansen

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Dejan Maksimovic
Sent: Wednesday, September 19, 2007 5:48 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Yet another FltGetFileNameInformation problem

The issue here is that fltmgr is essentially an abstraction layer over some
fairly complex stuff. Like all abstraction layers it will tend to leak
complexity through (i.e. ‘bugs’, ‘caveats’ etc.) and will probably not be
capable of solving all of the problems that could be solved by the more
complex layer it is abstracting (a very concerning factor in the
persistent rumours that “legacy” filters will be banned in the future).

I’ll agree to the the leaking complexity, however, the cases I’ve
mentioned are far from complex.
Sticking the particular API in question, I understand it cannot always
work in read/write/SetInfo/etc. paths. However, I would expect it to always
work in pre/post-create for either Opened or Normalized file name (not both
but at least one). That is how the complex legacy world worked.
The feeling I have is that I have switched one problem for another
I never had big issues with the path-query paths (once we ironed out different
network redirectors we had no issues). But debugging a mini-filter is much
easier.

–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.

NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · September 27, 2007, 3:47am

Neal,

You wrote:

Just to be perfectly clear, there are plans in a future release to not >support legacy filters.

So do you have plans to replace/remove the current layered drivers architecture ?

Inaki.

-----Mensaje original-----
De: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] En nombre de Neal Christiansen
Enviado el: mi?rcoles, 26 de septiembre de 2007 20:31
Para: Windows File Systems Devs Interest List
Asunto: RE: [ntfsd] Yet another FltGetFileNameInformation problem

Dejan,

I am interesting in understanding the pain points you are seeing with this API. I believe some of the problems you are seeing with this API you would also see with a legacy filter. For example your complaint about not getting a name after a rename of a remote file exists in legacy filters as well.

Just to be perfectly clear, there are plans in a future release to not support legacy filters. It is important for all vendors to move forward as appropriate with a minifilter implementation. If you have problems please work with Microsoft to address those issues because we want this environment to be as robust and reliable as possible

Neal Christiansen

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Dejan Maksimovic
Sent: Wednesday, September 19, 2007 5:48 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Yet another FltGetFileNameInformation problem

The issue here is that fltmgr is essentially an abstraction layer over some
fairly complex stuff. Like all abstraction layers it will tend to leak
complexity through (i.e. ‘bugs’, ‘caveats’ etc.) and will probably not be
capable of solving all of the problems that could be solved by the more
complex layer it is abstracting (a very concerning factor in the
persistent rumours that “legacy” filters will be banned in the future).

I’ll agree to the the leaking complexity, however, the cases I’ve
mentioned are far from complex.
Sticking the particular API in question, I understand it cannot always
work in read/write/SetInfo/etc. paths. However, I would expect it to always
work in pre/post-create for either Opened or Normalized file name (not both
but at least one). That is how the complex legacy world worked.
The feeling I have is that I have switched one problem for another
I never had big issues with the path-query paths (once we ironed out different
network redirectors we had no issues). But debugging a mini-filter is much
easier.

–
Kind regards, Dejan
http://www.alfasp.com
File system audit, security and encryption kits.

NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · October 3, 2007, 6:53pm

Here’s another: FltGetDestinationFileName fails with STATUS_INVALID_LOCK_SEQUENCE on most NAS devices (actually on all NAS devices we’ve tested, and also all that our customers tested, but it might not fail on some).
Nice of us to keep all the legacy code so we can use it in a mini-filter and rewrite the FltMgr API (sarcasm)

–
Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
http://www.alfasp.com
File system audit, security and encryption kits.

OSR_Community_User · October 31, 2007, 8:30am

After some time, I think it’s fair I’d update the list on some of the issues I’ve mentioned:

Calling the API during pre-create on an Inline TPM Module (Secure drive) causes a bugcheck. This happens only if the mini-filter is loaded before the TPM drive is mounted. No other code, just FltGetFileNameInformation and FltReleaseFileNameInforamtion (on success) in pre/post-create. We haven’t found a workaround for this yet

My fault for trusting the customer on this issue. After they’ve sent us a system to repro, I noticed right away the FltMgr.sys is NOT updated on it, and any recent update fixes the issue. The customer was always correct in providing the right information and doing the right tests, noone at my company thought of checking this (I see why there’s a checklist now…)

The API will not retrieve Normalized name for failed post-create. This seems right, however… if parts of the path can be normalized it would be prudent to allow normalized names in post-create

No change. I still think lots of users would want this.

Querying Normalized file name during post-create causes a hang with Altiris Virtualization software.

Still the same.

During some (successful) post-create calls, the API cannot retrieve both Normalized or Opened file name (returning STATUS_FLT_INVALID_NAME_REQUEST). This seems to happen if the SL_OPEN_TARGET_DIRECTORY bit is set. FltGetFileNameInformationUnsafe does not work either in this case.

No change.

–
Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
http://www.alfasp.com
File system audit, security and encryption kits.