I’m trying to solve a problem that I hope someone can
give me some advice on.
I am writing a filter driver that needs to combine the
contents of two directories and make them appear as if
they are all in one directory.
What is the best way to do this?
I can think of a couple of ways:
-
Use ZwQueryDirectoryFile in my dispatch routine to
fill the buffer and complete the IRP myself. Of
course, I’d have to handle all of the re-entrancy
issues…
-
Redirect the IRP by changing the FILE_OBJECT so
that the request goes where I want it to go, and then
removing duplicate file names in the completion
routine.
Thanks for any comments/suggestions,
Randy Cook
Lucid Systems Inc.
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com
If you consider all files in dir #1 to be indices 1 to x, and all files
in dir #2 to be indices x+1 to y, then it becomes simpler. Intercept
IRP_MN_QUERY_DIRECTORY, and pay attention to the FileIndex and the SL_*
flags. Depending on the range of file indices you need to return for a
given call, you may need to request information from dir #1, dir #2, or
both. To request this information I’d roll my own IRP each time in a
helper routine and send it directly to the lower device instead of using
ZwQueryDirectoryFile. This avoids the re-entrancy problem.
In order to do this, you need file objects for both dir #1 and dir #2
handy, no matter which directory the requestor has opened himself.
Handle this by using ZwCreateFile to grab two private handles for BOTH
directories the first time you intercept an IRP_MJ_CREATE from the
requestor for either one (do this in a blocking worker thread if
necessary if you’re short on stack space). Store away the handles&the
associated file object pointers in a global area for use by your
IRP_MN_QUERY_DIRECTORY hook. How long you want to keep these handles
open is up to you (try caching them for 30 seconds).
Handling duplicate file names is a pain in the ass here, since the
requestor may not (will not) ask for the ENTIRE contents of the
directory in one go. He’ll send down multiple requests, pulling up
subsequent index ranges each time. Keeping track of whether you already
returned a certain filename or not for a given directory query is
problematic.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Randy Cook
Sent: Friday, October 11, 2002 5:53 PM
To: File Systems Developers
Subject: [ntfsd] How to combine contents of two directories?
I’m trying to solve a problem that I hope someone can
give me some advice on.
I am writing a filter driver that needs to combine the
contents of two directories and make them appear as if
they are all in one directory.
What is the best way to do this?
I can think of a couple of ways:
-
Use ZwQueryDirectoryFile in my dispatch routine to
fill the buffer and complete the IRP myself. Of
course, I’d have to handle all of the re-entrancy
issues…
-
Redirect the IRP by changing the FILE_OBJECT so
that the request goes where I want it to go, and then
removing duplicate file names in the completion
routine.
Thanks for any comments/suggestions,
Randy Cook
Lucid Systems Inc.
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com
You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to %%email.unsub%%
Thanks a lot for the response. This is sounding
really good to me. Just a few clarifications below:
— Nicholas Ryan wrote:
> In order to do this, you need file objects for both
> dir #1 and dir #2
> handy, no matter which directory the requestor has
> opened himself.
FILE_OBJECTS are valid until I close them and are
valid across different threads (unlike handles),
right?
> Handle this by using ZwCreateFile to grab two
> private handles for BOTH
> directories the first time you intercept an
> IRP_MJ_CREATE from the
> requestor for either one (do this in a blocking
> worker thread if
> necessary if you’re short on stack space). Store
> away the handles&the
> associated file object pointers in a global area for
> use by your
> IRP_MN_QUERY_DIRECTORY hook.
I guess I need to keep the handles so I can close
them? Any other reason?
Is there a danger that when I try to close the handles
I may be running on a different thread and therefore
the handles would be invalid? If so, I guess I could
queue all of these Zw* operations to a worker thread
(who sticks around while my driver is loaded) and the
handles would always be valid for that thread.
> How long you want to
> keep these handles
> open is up to you (try caching them for 30 seconds).
I should also close them when I get a IRP_MJ_CLEANUP
on the original FILE_OBJECT, right?
Thanks again,
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf
> Of Randy Cook
> > Sent: Friday, October 11, 2002 5:53 PM
> > To: File Systems Developers
> > Subject: [ntfsd] How to combine contents of two
> directories?
> >
> >
> > I’m trying to solve a problem that I hope someone
> can
> > give me some advice on.
> >
> > I am writing a filter driver that needs to combine
> the
> > contents of two directories and make them appear
> as if
> > they are all in one directory.
> >
> > What is the best way to do this?
> >
> > I can think of a couple of ways:
> > 1. Use ZwQueryDirectoryFile in my dispatch
> routine to
> > fill the buffer and complete the IRP myself. Of
> > course, I’d have to handle all of the re-entrancy
> > issues…
> >
> > 2. Redirect the IRP by changing the FILE_OBJECT
> so
> > that the request goes where I want it to go, and
> then
> > removing duplicate file names in the completion
> > routine.
> >
> > Thanks for any comments/suggestions,
> >
> > Randy Cook
> > Lucid Systems Inc.
> >
> >
> > Do you Yahoo!?
> > Faith Hill - Exclusive Performances, Videos & More
>
> http://faith.yahoo.com
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@nryan.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@yahoo.com
> To unsubscribe send a blank email to
%%email.unsub%%
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com
> in dir #2 to be indices x+1 to y, then it becomes simpler. Intercept
IRP_MN_QUERY_DIRECTORY, and pay attention to the FileIndex
Afaik FileIndex is not necessarily used, or do I confuse something?
Maintaining
the list manually will also ease those issues with incomplete directory
listings
or finding out which entries already were passed to the caller, like
Nicholas
described below.
----- Original Message -----
From: “Nicholas Ryan”
To: “File Systems Developers”
Sent: Saturday, October 12, 2002 3:43 AM
Subject: [ntfsd] RE: How to combine contents of two directories?
> If you consider all files in dir #1 to be indices 1 to x, and all files
> in dir #2 to be indices x+1 to y, then it becomes simpler. Intercept
> IRP_MN_QUERY_DIRECTORY, and pay attention to the FileIndex and the SL_*
> flags. Depending on the range of file indices you need to return for a
> given call, you may need to request information from dir #1, dir #2, or
> both. To request this information I’d roll my own IRP each time in a
> helper routine and send it directly to the lower device instead of using
> ZwQueryDirectoryFile. This avoids the re-entrancy problem.
>
> In order to do this, you need file objects for both dir #1 and dir #2
> handy, no matter which directory the requestor has opened himself.
> Handle this by using ZwCreateFile to grab two private handles for BOTH
> directories the first time you intercept an IRP_MJ_CREATE from the
> requestor for either one (do this in a blocking worker thread if
> necessary if you’re short on stack space). Store away the handles&the
> associated file object pointers in a global area for use by your
> IRP_MN_QUERY_DIRECTORY hook. How long you want to keep these handles
> open is up to you (try caching them for 30 seconds).
>
> Handling duplicate file names is a pain in the ass here, since the
> requestor may not (will not) ask for the ENTIRE contents of the
> directory in one go. He’ll send down multiple requests, pulling up
> subsequent index ranges each time. Keeping track of whether you already
> returned a certain filename or not for a given directory query is
> problematic.
>
> - Nicholas Ryan
>
>
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Randy Cook
> > Sent: Friday, October 11, 2002 5:53 PM
> > To: File Systems Developers
> > Subject: [ntfsd] How to combine contents of two directories?
> >
> >
> > I’m trying to solve a problem that I hope someone can
> > give me some advice on.
> >
> > I am writing a filter driver that needs to combine the
> > contents of two directories and make them appear as if
> > they are all in one directory.
> >
> > What is the best way to do this?
> >
> > I can think of a couple of ways:
> > 1. Use ZwQueryDirectoryFile in my dispatch routine to
> > fill the buffer and complete the IRP myself. Of
> > course, I’d have to handle all of the re-entrancy
> > issues…
> >
> > 2. Redirect the IRP by changing the FILE_OBJECT so
> > that the request goes where I want it to go, and then
> > removing duplicate file names in the completion
> > routine.
> >
> > Thanks for any comments/suggestions,
> >
> > Randy Cook
> > Lucid Systems Inc.
> >
> > __________________________________________________
> > Do you Yahoo!?
> > Faith Hill - Exclusive Performances, Videos & More
> http://faith.yahoo.com
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@nryan.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@linkwave.org
> To unsubscribe send a blank email to %%email.unsub%%
> > In order to do this, you need file objects for both
> dir #1 and dir #2
> handy, no matter which directory the requestor has
> opened himself.
FILE_OBJECTS are valid until I close them and are
valid across different threads (unlike handles),
right?
Yes.
> Handle this by using ZwCreateFile to grab two
> private handles for BOTH
> directories the first time you intercept an
> IRP_MJ_CREATE from the
> requestor for either one (do this in a blocking
> worker thread if
> necessary if you’re short on stack space). Store
> away the handles&the
> associated file object pointers in a global area for
> use by your
> IRP_MN_QUERY_DIRECTORY hook.
I guess I need to keep the handles so I can close
them? Any other reason?
Is there a danger that when I try to close the handles
I may be running on a different thread and therefore
the handles would be invalid? If so, I guess I could
queue all of these Zw* operations to a worker thread
(who sticks around while my driver is loaded) and the
handles would always be valid for that thread.
Actually, when you call ZwCreateFile, you should specifiy the
OBJ_KERNEL_HANDLE flag in InitializeObjectAttributes. This ensures that
the resulting handle is usable across all threads, and only usable from
kenel-mode (i.e. can’t be hijacked by user-mode code).
> How long you want to
> keep these handles
> open is up to you (try caching them for 30 seconds).
I should also close them when I get a IRP_MJ_CLEANUP
on the original FILE_OBJECT, right?
You don’t have to. The two handles you open are completely independent
of any handles anybody else may open for the same directories. You can
keep them open until the cows come home.
It’s used if the caller sets the SL_INDEX_SPECIFIED flag, if he wants to
start the query at a specific file index.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tobias
Sent: Saturday, October 12, 2002 12:47 AM
To: File Systems Developers
Subject: [ntfsd] RE: How to combine contents of two directories?
> in dir #2 to be indices x+1 to y, then it becomes simpler.
Intercept
> IRP_MN_QUERY_DIRECTORY, and pay attention to the FileIndex
Afaik FileIndex is not necessarily used, or do I confuse
something? Maintaining the list manually will also ease those
issues with incomplete directory listings or finding out
which entries already were passed to the caller, like
Nicholas described below.
----- Original Message -----
From: “Nicholas Ryan”
> To: “File Systems Developers”
> Sent: Saturday, October 12, 2002 3:43 AM
> Subject: [ntfsd] RE: How to combine contents of two directories?
>
>
> > If you consider all files in dir #1 to be indices 1 to x, and all
> > files in dir #2 to be indices x+1 to y, then it becomes simpler.
> > Intercept IRP_MN_QUERY_DIRECTORY, and pay attention to the
> FileIndex
> > and the SL_* flags. Depending on the range of file indices
> you need to
> > return for a given call, you may need to request
> information from dir
> > #1, dir #2, or both. To request this information I’d roll
> my own IRP
> > each time in a helper routine and send it directly to the
> lower device
> > instead of using ZwQueryDirectoryFile. This avoids the re-entrancy
> > problem.
> >
> > In order to do this, you need file objects for both dir #1
> and dir #2
> > handy, no matter which directory the requestor has opened himself.
> > Handle this by using ZwCreateFile to grab two private
> handles for BOTH
> > directories the first time you intercept an IRP_MJ_CREATE from the
> > requestor for either one (do this in a blocking worker thread if
> > necessary if you’re short on stack space). Store away the
> handles&the
> > associated file object pointers in a global area for use by your
> > IRP_MN_QUERY_DIRECTORY hook. How long you want to keep
> these handles
> > open is up to you (try caching them for 30 seconds).
> >
> > Handling duplicate file names is a pain in the ass here, since the
> > requestor may not (will not) ask for the ENTIRE contents of the
> > directory in one go. He’ll send down multiple requests, pulling up
> > subsequent index ranges each time. Keeping track of whether you
> > already returned a certain filename or not for a given
> directory query
> > is problematic.
> >
> > - Nicholas Ryan
> >
> >
> >
> > > -----Original Message-----
> > > From: xxxxx@lists.osr.com
> > > [mailto:xxxxx@lists.osr.com] On Behalf Of Randy Cook
> > > Sent: Friday, October 11, 2002 5:53 PM
> > > To: File Systems Developers
> > > Subject: [ntfsd] How to combine contents of two directories?
> > >
> > >
> > > I’m trying to solve a problem that I hope someone can
> > > give me some advice on.
> > >
> > > I am writing a filter driver that needs to combine the
> contents of
> > > two directories and make them appear as if they are all in one
> > > directory.
> > >
> > > What is the best way to do this?
> > >
> > > I can think of a couple of ways:
> > > 1. Use ZwQueryDirectoryFile in my dispatch routine to
> > > fill the buffer and complete the IRP myself. Of
> > > course, I’d have to handle all of the re-entrancy
> > > issues…
> > >
> > > 2. Redirect the IRP by changing the FILE_OBJECT so
> > > that the request goes where I want it to go, and then removing
> > > duplicate file names in the completion routine.
> > >
> > > Thanks for any comments/suggestions,
> > >
> > > Randy Cook
> > > Lucid Systems Inc.
> > >
> > > __________________________________________________
> > > Do you Yahoo!?
> > > Faith Hill - Exclusive Performances, Videos & More
> > http://faith.yahoo.com
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@linkwave.org To
> > unsubscribe send a blank email to %%email.unsub%%
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@nryan.com
> To unsubscribe send a blank email to %%email.unsub%%
>
> From: Nicholas Ryan [mailto:xxxxx@nryan.com]
You don’t have to. The two handles you open are completely independent
of any handles anybody else may open for the same directories. You can
keep them open until the cows come home.
Yes, you can keep the directory handles open as long as you like, if you don’t mind explaining behavior users don’t expect. For example, if an application tries to remove a directory, and you keep the directory handle open, the directory remains but becomes inaccessible until you close that handle.
I would think one way to support deletions transparently would be to
open the original handles with FILE_SHARE_DELETE share access, then
close the cached handles when a cleanup comes down with
FileObject->DeletePending set to TRUE.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Fuller, Rob
Sent: Monday, October 14, 2002 6:40 AM
To: File Systems Developers
Subject: [ntfsd] RE: How to combine contents of two directories?
> From: Nicholas Ryan [mailto:xxxxx@nryan.com]
>
> You don’t have to. The two handles you open are completely
independent
> of any handles anybody else may open for the same
directories. You can
> keep them open until the cows come home.
Yes, you can keep the directory handles open as long as you
like, if you don’t mind explaining behavior users don’t
expect. For example, if an application tries to remove a
directory, and you keep the directory handle open, the
directory remains but becomes inaccessible until you close
that handle.
You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to %%email.unsub%%