Directory filtering - related to XP

NTFSD
1

Regarding the possible problem I mentioned some time ago. The
following might happen:

  • Many forget to handle STATUS_PENDING. This won’t make problems on
    NT/2K as it rarely happens. It would crash XP as soon as the logon
    screen comes up.
  • If files are found by the FSD, but you filter all of them out,
    recall the driver - DON’T return zero in the information field.
  • Don’t use Irp->UserBuffer in the completion routine - it is run in
    arbitrary context, and the pointer is likely to be invalid
  • Don’t return STATUS_PENDING for IRP_MJ_DIRECTORY_CONTROL and do
    processing in the worker thread. Same reason as last one.
  • Since you need to handle STATUS_PENDING, you must have a
    completion routine and wait for it in the dispatch.
  • Remember to update the Irp->IoStatus.Status filed accordingly!
    It’s likely that FSD returns STATUS_NO_MORE_FILES. In a special case it
    is required to set the status to STATUS_NO_SUCH_FILE (only if the call
    had SL_RESTART flag, and all files are filtered out!)
  • Don’t return STATUS_NO_SUCH_FILE, if more files might be
    available! This is the most common mistake.

I’ll be happy to answer any specific questions about this, as I have
spent quite some time on directory filtering, and it takes about 40% of
my driver code:-)
Also, feel free to add additional tips and comment above ones.

This seems lame for old timers, but I was a victim of STATUS_PENDING
in early version of my driver, and I have found that many people make
the above errors.
(BTW, I made this post because I had many people contacting me via
e-mail about directory filtering)

–
Kind regards, Dejan M. CEO Alfa Co. www.alfasp.com
E-mail: xxxxx@alfasp.com
ICQ#: 56570367
Alfa File Monitor - File monitoring system for Win32 developers.
Alfa File Protector - File protection and hiding system for Win32
developers.

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

2

I’m looking this obsevation and don’t underestand a few.

Deja/////////////////////
Regarding the possible problem I mentioned some time ago. The
following might happen:

  • Many forget to handle STATUS_PENDING. This won’t make problems on
    NT/2K as it rarely happens. It would crash XP as soon as the logon
    screen comes up.
    END Deja///////////////////////////

You mean, that if is STATUS_PENDING, mark a irp with status pending?

Deja/////////////////////

  • If files are found by the FSD, but you filter all of them out,
    recall the driver - DON’T return zero in the information field.
    END Deja///////////////////////////

Look I Initialize an Event, set a Completion routine, and them
Iocalldriver(…), then I wait for Event, and set at completion routine
this event, then at dispatch make my filtering OUT ALL FILES OF BUFFER,
in this case I return STATUS_NO_SUCH_FILE and return zero in the
information field. But you say that don’t do this, just recall a
driver… How can I make This?

Deja/////////////////////

  • Don’t use Irp->UserBuffer in the completion routine - it is run in
    arbitrary context, and the pointer is likely to be invalid
    END Deja///////////////////////////

That’s OK

Deja/////////////////////

  • Don’t return STATUS_PENDING for IRP_MJ_DIRECTORY_CONTROL and do
    processing in the worker thread. Same reason as last one.
    END Deja///////////////////////////

That’s OK too

Deja/////////////////////

  • Since you need to handle STATUS_PENDING, you must have a
    completion routine and wait for it in the dispatch.
    END Deja///////////////////////////

That’s OK too

Deja/////////////////////

  • Remember to update the Irp->IoStatus.Status filed accordingly!
    It’s likely that FSD returns STATUS_NO_MORE_FILES. In a special case it
    is required to set the status to STATUS_NO_SUCH_FILE (only if the call
    had SL_RESTART flag, and all files are filtered out!)
    END Deja///////////////////////////

Why just when
only if the call had SL_RESTART flag, and all files are filtered out!,
if I clean all buffer
It mean that don’t exist more files… Or not? If invoke a DIR command
with a single entry and I need filter this entry what you return?

Deja/////////////////////

  • Don’t return STATUS_NO_SUCH_FILE, if more files might be
    available! This is the most common mistake.
    END Deja///////////////////////////
    That’s OK too

Thaks to Deja and all the people that help me

Moises

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

3

> Deja/////////////////////

Regarding the possible problem I mentioned some time ago. The
following might happen:

  • Many forget to handle STATUS_PENDING. This won’t make problems on
    NT/2K as it rarely happens. It would crash XP as soon as the logon
    screen comes up.
    END Deja///////////////////////////

You mean, that if is STATUS_PENDING, mark a irp with status pending?

No, I mean, if the lower driver returns STATUS_PENDING, you must
wait
for the completion routine to set your initialized event - otherwise,
you
get the wrong status code from IoCallDriver.

Look I Initialize an Event, set a Completion routine, and them
Iocalldriver(…), then I wait for Event, and set at completion routine
this event, then at dispatch make my filtering OUT ALL FILES OF BUFFER,
in this case I return STATUS_NO_SUCH_FILE and return zero in the
information field. But you say that don’t do this, just recall a
driver…

Imagine this scenario:

  • You filter all the files in this call, but there are more entries
    available in the directory
  • By returning STATUS_NO_SUCH_FILE you tell the caller that no such
    file
    is available, but yet it MIGHT be available if more entries are
    available
    (those you still haven’t filtered out).
    Example:
    Say, C:\Test has A.txt, b.txt and C.txt. Say the caller sets the
    buffer
    size to sizeof(FILE_BOTH_DIRECTORY_INFORMATION) + sizeof(WCHAR) * 10.
    This
    is enough to hold only a.txt, and NOT b.txt and c.txt
    Say you should only filter a.txt, but let b.txt and c.txt
    Now, if I type dir *.txt:
  • Your filter gets control, and starts filtering. It sees only
    a.txt, as
    it’s the only entry that would fit into the buffer. Now, you filter this
    entry out, and return STATUS_NO_SUCH_FILE
  • The caller gets STATUS_NO_SUCH_FILE, and he stops calling for new
    entries - PROBLEM! The caller would need to call for new entries as
    b.txt
    and c.txt are available!!!

How can I make This? //// Dejan: recalling the driver

Just reuse the original IRP - and don’t change any fields in it!

Why just when only if the call had SL_RESTART flag, and all files are
filtered out!, if I clean all buffer It mean that don’t exist more
files… Or not?

Of course not - this is totally wrong. This isn’t something that
requires extra driver knowledge, only pure logic! Not all the files are
returned in a SINGLE call - if you list System32 directory, it would
take
50+ calls!

If invoke a DIR command with a single entry and I need filter this entry
what you return?

See previous comment. Use logics.

Deja/////////////////////

  • Don’t return STATUS_NO_SUCH_FILE, if more files might be
    available! This is the most common mistake.
    END Deja///////////////////////////
    That’s OK too

Well, you don’t seem to be using it above:-)

–
Kind regards, Dejan M. CEO Alfa Co. www.alfasp.com
E-mail: xxxxx@alfasp.com
ICQ#: 56570367
Alfa File Monitor - File monitoring system for Win32 developers.
Alfa File Protector - File protection and hiding system for Win32
developers.

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

4

> How can I make This? //// Dejan: recalling the driver

Just reuse the original IRP - and don’t change any fields in it!

Paul:

I think it isn’t a good idea. There are some situations where you
must do that or your scanning of the directory will never end :-((

Yes, I mean when the original Irp has SL_RESTART_SCAN or
if it has SL_INDEX_SPECIFIED. Then you must clear those flags
before you reissue the Irp.

Why is this correct ?
Simply because:

  1. When the SL_RESTART_SCAN is specified and after the first
    query, the CCB will contain the proper information about the next
    entry, so the next query will continue where the previous one has ended.
    Without beginning from the starting point - of course. And so on…
  2. The SL_INDEX_SPECIFIED is anlogous - it is nothing more than
    generalized restart scan with the case you have the way to specify
    the starting point exactly (with the only exception, even if you set
    the Id to zero you have no way to catch the first entry).

So those flags are the way to start the search at the preferred entry
and every next query should start exactly where the previous one has ened -
which is ensured by CCB (maintained by the FSD).

Is there anybody who don’t think so ?

Paul

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

5

> Yes, I mean when the original Irp has SL_RESTART_SCAN or

if it has SL_INDEX_SPECIFIED. Then you must clear those flags
before you reissue the Irp.

No, these are cleared automatically.

–
Kind regards, Dejan M. CEO Alfa Co. www.alfasp.com
E-mail: xxxxx@alfasp.com
ICQ#: 56570367
Alfa File Monitor - File monitoring system for Win32 developers.
Alfa File Protector - File protection and hiding system for Win32 developers.

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

6

Really ?
So then please tell me what (or who) does this task.

Thanks
Paul

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Dejan Maksimovic
Sent: Thursday, October 25, 2001 11:57 AM
To: File Systems Developers
Subject: [ntfsd] RE: Directory filtering - related to XP

Yes, I mean when the original Irp has SL_RESTART_SCAN or
if it has SL_INDEX_SPECIFIED. Then you must clear those flags
before you reissue the Irp.

No, these are cleared automatically.

–
Kind regards, Dejan M. CEO Alfa Co. www.alfasp.com
E-mail: xxxxx@alfasp.com
ICQ#: 56570367
Alfa File Monitor - File monitoring system for Win32 developers.
Alfa File Protector - File protection and hiding system for Win32
developers.

You are currently subscribed to ntfsd as: xxxxx@compelson.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

7

I haven’t a clue:-) I just know it works:-) My driver does nothing to
the IRPs nor their context, and still it filters correctly.
I DO recopy the stack locations, though.
I think this is done by the FSD - it should set the index so that
next time the caller knows where to restart.

Regards, Dejan.

Pavel Hrdina wrote:

Really ?
So then please tell me what (or who) does this task.

Thanks
Paul

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Dejan Maksimovic
Sent: Thursday, October 25, 2001 11:57 AM
To: File Systems Developers
Subject: [ntfsd] RE: Directory filtering - related to XP

> Yes, I mean when the original Irp has SL_RESTART_SCAN or
> if it has SL_INDEX_SPECIFIED. Then you must clear those flags
> before you reissue the Irp.

No, these are cleared automatically.

–
Kind regards, Dejan M. CEO Alfa Co. www.alfasp.com
E-mail: xxxxx@alfasp.com
ICQ#: 56570367
Alfa File Monitor - File monitoring system for Win32 developers.
Alfa File Protector - File protection and hiding system for Win32
developers.

You are currently subscribed to ntfsd as: xxxxx@compelson.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@alfasp.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

–
Kind regards, Dejan M. CEO Alfa Co. www.alfasp.com
E-mail: xxxxx@alfasp.com
ICQ#: 56570367
Alfa File Monitor - File monitoring system for Win32 developers.
Alfa File Protector - File protection and hiding system for Win32
developers.

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

8

Hmmmm…
I have to say that your arguments haven’t convinced me.

You have to copy your IrpSp to the next one every time before you
pass the Irp down. And if you don’t do the change so who do it instead ?
I don’t believe the FSD can do it because it doesn’t care about such
situation - if someone call it twice or more times with the same parameters
indicating the exact beginning of scan it should do exactly the same in
every call. It can’t guess that this can be the previous request rescheduled
because it was completely filtered out. Don’t you think so ?
Finally look into fastfat sources and you can see that it doesn’t do any
guess… and I consider it as a ‘NT FSD template’.

So, does anybody of you know the answer ?
(in the case Dejan doesn’t mystify us all :-))))

Paul

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Dejan Maksimovic
Sent: Friday, October 26, 2001 1:14 AM
To: File Systems Developers
Subject: [ntfsd] RE: Directory filtering - related to XP

I haven’t a clue:-) I just know it works:-) My driver does nothing to
the IRPs nor their context, and still it filters correctly.
I DO recopy the stack locations, though.
I think this is done by the FSD - it should set the index so that
next time the caller knows where to restart.

Regards, Dejan.

Pavel Hrdina wrote:

Really ?
So then please tell me what (or who) does this task.

Thanks
Paul

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Dejan Maksimovic
Sent: Thursday, October 25, 2001 11:57 AM
To: File Systems Developers
Subject: [ntfsd] RE: Directory filtering - related to XP

> Yes, I mean when the original Irp has SL_RESTART_SCAN or
> if it has SL_INDEX_SPECIFIED. Then you must clear those flags
> before you reissue the Irp.

No, these are cleared automatically.

–
Kind regards, Dejan M. CEO Alfa Co. www.alfasp.com
E-mail: xxxxx@alfasp.com
ICQ#: 56570367
Alfa File Monitor - File monitoring system for Win32 developers.
Alfa File Protector - File protection and hiding system for Win32
developers.

You are currently subscribed to ntfsd as: xxxxx@compelson.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: xxxxx@alfasp.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

–
Kind regards, Dejan M. CEO Alfa Co. www.alfasp.com
E-mail: xxxxx@alfasp.com
ICQ#: 56570367
Alfa File Monitor - File monitoring system for Win32 developers.
Alfa File Protector - File protection and hiding system for Win32
developers.

You are currently subscribed to ntfsd as: xxxxx@compelson.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com