Greetings,
Can a monitoring lower filter driver perform context tracking of a reparse
point (not opened with “Reparse flag”) yet allow the real driver on higher
altitude handle the reparse point it owns?
Does the filter manager care that 2 different filter drivers built stream
contexts for the file in question? The filter driver at lower altitude
simply tracks the IO where the one at higher altitude would handle the
redirection by reissuing the IO etc.
Thanks in advance.
Kamran Tavakoli wrote:
Greetings,
Can a monitoring lower filter driver perform context tracking of a
reparse point (not opened with “Reparse flag”) yet allow the real driver
on higher altitude handle the reparse point it owns?Does the filter manager care that 2 different filter drivers built
stream contexts for the file in question? The filter driver at lower
altitude simply tracks the IO where the one at higher altitude would
handle the redirection by reissuing the IO etc.
Not quite sure I follow the question but let me try to answer. If
Filter1 is sitting at a higher altitude than Filter2 and Filter1 is
implementing the reparse point for some component in a path. In this
case, Filter2 would never see the ‘open’ of the reparse point since
Filter 1 would handle it and return STATUS_REPARSE before Filter 2 sees
the request. It would, of course, see the open for the target of the
reparse but by that time Filter2 has no idea that the open is a result
of a reparse.
Remember reparse processing is only relevant in the create handler, in
‘normal’ cases.
Pete
–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295
Peters response is not correct. Filter1 does not return STATUS_REPARSE (at least not in the conventional use of reparse points). The first open of the file is seen by both drivers and would go all the way to the file system. Neither driver knows the file has a reparse point yet (unless they open the file and query). The file system returns STATUS_REPARSE and both drivers would see this in the post-create path. The lower driver can not add a stream context at this point though because the file is not yet open. The driver that handles the RP needs to send it back down (FltReissueSynchronousIo) with the open reparse flag and then the file gets opened. So the trick in your case is for your lower level driver to recognize when the file really does get opened so it can add the context you need.
The filter manager does not care if multiple drivers add stream contexts. It is a list of contexts and it know which filter owns which context.
xxxxx@yahoo.com wrote:
Peters response is not correct. Filter1 does not return STATUS_REPARSE (at least not in the conventional use of reparse points). The first open of the file is seen by both drivers and would go all the way to the file system. Neither driver knows the file has a reparse point yet (unless they open the file and query). The file system returns STATUS_REPARSE and both drivers would see this in the post-create path. The lower driver can not add a stream context at this point though because the file is not yet open. The driver that handles the RP needs to send it back down (FltReissueSynchronousIo) with the open reparse flag and then the file gets opened. So the trick in your case is for your lower level driver to recognize when the file really does get opened so it can add the context you need.
Well, I suppose it depends on your understanding, interpretation and
point of view. The OP said that Filter1 ‘implements’ the reparse point.
I took this as meaning it determines the node in the path IS a reparse
point and returns STATUS_REPARSE, like I said in my response. In this
case, you are incorrect in saying that Filter2 would ever see the
initial open which gets the STATUS_REPARSE returned on it.
Please clarify what you mean when you say such things … I don’t take
it personally but I did indicate in my response, clearly, that his
questions was a little vague. As well, I think I made it quite clear in
my email that the reparse point is NOT implemented in the file system
but handled by the filter, which is perfectly legal.
Pete
–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295
You are correct that it depends on the interpretation of the question, and I certainly did not mean to make it personal. BTW, the question never used the term implement. It used the term “handle” which to me means the driver that does whatever processing the reparse point requires.
I cannot think of any reason that a driver writer would use reparse points and not let the file system handle the implementation (i.e. detecting the reparse point and returning STATUS_REPARSE). It may be legal but I do not see where it makes sense to do it. I can see someone writing a driver like simrep that simulates reparse points and returns STATUS_REPARSE, but a driver that uses real reparse points would not return STATUS_REPARSE anyway even if it took over the task of implementing them. What driver above it would see the STATUS_REPARSE and know what to do with it (assuming it was not a standard Microsoft RP)?
xxxxx@yahoo.com wrote:
You are correct that it depends on the interpretation of the question, and I certainly did not mean to make it personal. BTW, the question never used the term implement. It used the term “handle” which to me means the driver that does whatever processing the reparse point requires.
I cannot think of any reason that a driver writer would use reparse points and not let the file system handle the implementation (i.e. detecting the reparse point and returning STATUS_REPARSE). It may be legal but I do not see where it makes sense to do it. I can see someone writing a driver like simrep that simulates reparse points and returns STATUS_REPARSE, but a driver that uses real reparse points would not return STATUS_REPARSE anyway even if it took over the task of implementing them. What driver above it would see the STATUS_REPARSE and know what to do with it (assuming it was not a standard Microsoft RP)?
There are a large number of implementations where a filter would modify
the name in the file object and return STATUS_REPARSE without sending it
down the stack. While there are large schools on both sides of this type
of implementation, it is perfectly legal and there are highly available
products in the field which us this type of design. You can do this for
encryption, compression, HSM, AV, nearly any sort of implementation can
use reparse processing at a filter level which reparses to some other
location or to a specific named device object.
If you are a filter driver and receive STATUS_REPARSE on completion,
there is not a lot you can do with the request regardless of who
implements the reparse point. Basically, in my opinion, is was a mistake
to make STATUS_REPARSE a success code, it should be treated as a failure
code and the filter not handle it on completion. When the IO Manager
gets the request, it will ‘reparse’ the name in the file object and pass
the IRP_MJ_CREATE back down the stack. Hence if you are a filter you
would then receive the new IRP_MJ_CREATE accordingly and be able to
handle it without trying to decide what to do with the reparse information.
So back to the OP’s question … if a filter receives STATUS_REPARSE on
completion it should, in nearly all cases, handle it as a failure code.
The file object is not opened, you can not create a stream context on
the returned file object, etc.
Pete
–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295
Modifying the name in the fileobject and returning STATUS_REPARSE is not the same as putting a reparse point on the file. Not all uses of reparse points modify the path. An HSM driver, for example, does not change the path (at least not any of the HSM drivers I wrote did) but tracks IO to the file so that it can redirect reads to where it stored the data (or recall the data before letting the read through).
I think the disconnect here is the fact that a driver can return STATUS_REPARSE when there it no actual reparse point involved (by reparse point I mean data attached to a file or directory using FSCTL_SET_REPARSE_POINT).
Any driver that actually sets a custom reparse point must handle it when they see STATUS_REPARSE and the callback data points to their reparse tag. No other code will know what to do with it.
xxxxx@yahoo.com wrote:
Modifying the name in the fileobject and returning STATUS_REPARSE is not the same as putting a reparse point on the file. Not all uses of reparse points modify the path. An HSM driver, for example, does not change the path (at least not any of the HSM drivers I wrote did) but tracks IO to the file so that it can redirect reads to where it stored the data (or recall the data before letting the read through).
I think the disconnect here is the fact that a driver can return STATUS_REPARSE when there it no actual reparse point involved (by reparse point I mean data attached to a file or directory using FSCTL_SET_REPARSE_POINT).
Any driver that actually sets a custom reparse point must handle it when they see STATUS_REPARSE and the callback data points to their reparse tag. No other code will know what to do with it.
I have simply given the generic usage of a reparse point. You are
focusing on a specific implementation of a reparse point. I think the OP
originally asked more of a general question but at this point I am
certain they are thoroughly confused.
Pete
–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295
Now you have me confused. I do not know of any implementation of reparse points that work the way you are saying they do (i.e. the driver detects the RP in pre-create and returns STATUS_REPARSE). If by the term generic usage you mean mount points or symbolic links that is not how they work. The create goes all the way to the file system which detects the rp and returns STATUS_REPARSE and then the io manager replaces the path and send the create back down. If you mean the recommended way to design a driver that uses reparse points the same thing happens except the 3rd party driver does whatever is required to handle the RP rather than the io manager.
xxxxx@yahoo.com wrote:
Now you have me confused. I do not know of any implementation of reparse points that work the way you are saying they do (i.e. the driver detects the RP in pre-create and returns STATUS_REPARSE). If by the term generic usage you mean mount points or symbolic links that is not how they work. The create goes all the way to the file system which detects the rp and returns STATUS_REPARSE and then the io manager replaces the path and send the create back down. If you mean the recommended way to design a driver that uses reparse points the same thing happens except the 3rd party driver does whatever is required to handle the RP rather than the io manager.
In all cases you mention, and the one I will mention, the end result is
either 1) The name being updated in the file object and reparsed by the
io manager, 2) Some intermediate component recognizes the reparse point
and redirects the open to some other file. While this ‘updating’ or
‘redirecting’ is based on information stored within the reparse
information blob, it still accomplishes the same thing.
Now, in general, any filter can grab an open in pre-create, update the
file name in the file object, set the Irp->IoStatus.Information =
IO_REPARSE and return STATUS_REPARSE. In this case, no filter below the
filter implementing this nor the file system will see this access. This
type of implementation is common whenever you implement, say, your own
file system and you want accesses to some given sub-directory to be
redirected to your file system. In this case the name you update in the
file object would be something like
\device\your-file-system-name\foo.txt. This sort of design is leveraged
in encryption, compression, HSM, or anything where you implement a full
set of functionality within some stand alone driver.
The above mechanism is the base concept for the implementation of a
reparse point. While it may not reparse to some stand alone driver, it
can just as easily redirect to some other file or directory location in
the same manner.
NOTE: Like I said previously, there are two schools of thought on this
design. Some feel it introduces unwanted effects in the file system such
as breaking targeted opens but that is a completely different discussion
thread.
Pete
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295
Thanks to both of you for taking the time to respond. And I apologize for
not making it very clear to begin with. We are talking about regular reparse
handling that is used in HSM or SIS type filter drivers.
Lets use your terminology for upper and lower filter drivers in question:
i.e. Filter1 at higher altitude actually handles the reparse point by doing
redirection. Filter2 at lower altitude wants to monitor the files that are
being handled by Filter2. Please note that Filter1 wants to monitor the
tagged file not the target of the tagged file (i.e. it wants to monitor the
reparse point file and how it is being used by the system or programs).
During completion processing (PostCreate) the lower filter sees
STATUS_REPARSE as well as the upper filter, but naturally only the owner of
the Tag will handle the reparse point by reissuing the IO (i.e. Filter1).
So if Filter2 wants to track the tagged file (not the target of the tagged
file), where and when would it attach a stream context to it?
I guess in summary: How can 2 filter drivers attach a stream context to a
single “reparse point file” safely.
Thanks.
Kamran Tavakoli wrote:
Thanks to both of you for taking the time to respond. And I apologize
for not making it very clear to begin with. We are talking about regular
reparse handling that is used in HSM or SIS type filter drivers.Lets use your terminology for upper and lower filter drivers in
question: i.e. Filter1 at higher altitude actually handles the reparse
point by doing redirection. Filter2 at lower altitude wants to monitor
the files that are being handled by Filter2. Please note that Filter1
wants to monitor the tagged file not the target of the tagged file (i.e.
it wants to monitor the reparse point file and how it is being used by
the system or programs).During completion processing (PostCreate) the lower filter sees
STATUS_REPARSE as well as the upper filter, but naturally only the owner
of the Tag will handle the reparse point by reissuing the IO (i.e. Filter1).So if Filter2 wants to track the tagged file (not the target of the
tagged file), where and when would it attach a stream context to it?I guess in summary: How can 2 filter drivers attach a stream context to
a single “reparse point file” safely.
Unless the tagged file is opened, and you wouldn’t see STATUS_REPARSE in
this case, then you can’t. STATUS_REPARSE should be treated as a failure
code by all except those who know what to do with the information. But
for the file which the STATUS_REPARSE is being returned, it is NOT
opened and thus you can not associate a stream context to it.
Pete
Thanks.
— NTFSD is sponsored by OSR For our schedule of debugging and file
system seminars (including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars To unsubscribe, visit the List Server
section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295
I guess I only could query the FRN when I see the STATUS_REPARSE in the
Lower filter, and let the upper filter handle the tag and reissue the IO,
then if I see a Create coming down with “FILE_FLAG_OPEN_REPARSE_POINT” to
open the FRN in question and that the IO was issued by Filter1, then I could
attach a stream context on successful completion of the second PostCreate…
Too much work for little reward.
Thanks for your help. (Pete & Rick)
Unless the tagged file is opened, and you wouldn’t see STATUS_REPARSE in
this case, then you can’t. STATUS_REPARSE should be treated as a failure
code by all except those who know what to do with the information. But for
the file which the STATUS_REPARSE is being returned, it is NOT opened and
thus you can not associate a stream context to it.Pete
Thanks.
>
> — NTFSD is sponsored by OSR For our schedule of debugging and file
> system seminars (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars To unsubscribe, visit the List Server section
> of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
First a quick response to Pete:
I agree that any driver can change the path and return STATUS_REPARSE. You can even call this a reparse point if you want to, but to me a reparse point is what gets set by FSCTL_SET_REPARSE_POINT and it may redirect the create to a different path but it does not have to. If you go back to the Remote Storage component that was in Windows 2000 it did not redirect the create. It simply sent the create back down (like FltReissueSynchronousIO would do) and then set a context on the resulting file object. When it saw a read for a file object with this context it would fetch the data and write it back to the file and let the read go through.
Kamran;
If you are redirecting the create to a different path/file then filter2 will see only one access to the file with the RP and since it returns STATUS_REPARSE the file is not yet opened so you cannot attach a context and you do not really need to because there will not be any further operations on this file. If you are NOT redirecting to a different path/file then you will see a second create (from filter1 and with FILE_OPEN_REPARSE_POINT) and this create will be successful assuming no other errors occur.
If you are redirecting the create based on something other than the fact that there is a reparse tag on the file then filter1 would do that and filter2 would not see it. This is the model the simrep example uses and it uses a simple path check to decide what to redirect.
Reparse points aside, 2 (or more) different drivers can each attach stream contexts to a file object because the FsContext has in it a linked list of contexts (assuming the file system supports contexts) and it knows which filter owns each of them based on the FLT_INSTANCE passed to FltSetStreamContext.
xxxxx@yahoo.com wrote:
First a quick response to Pete:
I agree that any driver can change the path and return STATUS_REPARSE. You can even call this a reparse point if you want to, but to me a reparse point is what gets set by FSCTL_SET_REPARSE_POINT and it may redirect the create to a different path but it does not have to. If you go back to the Remote Storage component that was in Windows 2000 it did not redirect the create. It simply sent the create back down (like FltReissueSynchronousIO would do) and then set a context on the resulting file object. When it saw a read for a file object with this context it would fetch the data and write it back to the file and let the read go through.
Semantics I suppose. I have always considered reparse points set through
FSCTL_SET_REPARSE_POINT as a subclass of the general concept of reparse
point processing.
Pete
–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295