Sara,
The original discussion dealt with detecting recursion within a file system
filter driver for IRP_MJ_CREATE handling. Since a filter that is recursing
back into the storage stack has not yet reached the file system layer, the
NTFS/FAT mechanism wouldn’t be useful in detecting same.
Even in a case where you were using the TopLevelIrp field, it does not work
correctly in the case where there are TWO threads involved in processing the
operation. This might occur, for instance, for an HSM filter where the file
must be migrated back from tape (or optical, or whatever) to disk. An HSM
filter would block the IRP_MJ_CREATE and then dispatch the operation to a
service. The service (using a separate thread, typically) would then
migrate the file back. It could do so by opening the file (but this will be
passed by the filter since it comes from the “well known” thread of this
separate service) and pouring the contents back into the file.
This scheme works fine. The filter detects the “recursion” of the second
thread and passes it correctly.
Now, imagine combining this with a second filter that uses the mutex
approach. In this case, the first thread acquires the mutex and passes the
call to the HSM filter. The HSM filter then dispatches to its service. The
service thread then attempts to open the file. When the filter attempts to
acquire the mutex, the operation will block because the mutex is owned by
the other thread.
That was the scenario with which I was concerned.
With respect to using the top level IRP field to detect recursion, if you
are constructing an FSD, you own that field and you can use it in any way
you wish (note that the network redirector uses it to point to a context
data structure, not an IRP.) If you are constructing a filter driver (or a
file system layered on top of another file system) you should not use that
field (you can if you are willing to save/restore that state information
around calls to/from the FSD, in which case it is typically just as easy to
maintain a nice CPU-cached hash table of thread local storage information.)
I suppose one could use the mutex model where you made the mutex the thread
local storage. That only just occured to me because using a mutex in such a
case would be quite heavyweight - after all, once you have thread local
storage, all you need to maintain is a counter (increment when you enter the
filter, decrement when you leave the filter.)
Does this make sense now?
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: Sara Abraham [mailto:xxxxx@veritas.com]
Sent: Wednesday, July 12, 2000 5:55 PM
To: File Systems Developers
Subject: [ntfsd] RE: preventing recursive loop in create dispatch hand ler
Tony,
I must be missing something. NTFS and FAT rely on the TopLevelIrp
mechanism. If I have a filter driver on top of NTFS/FAT that decides to post
a write request, for example, to a different thread, without updating the
new thread’s TopLevelIrp information (really copying the information from
the original thread), then NTFS/FAT are not going to work properly. So, I
think that whatever a filter driver decides to do, it cannot do anything
that will corrupt the TopLevelIrp info. To detect recursion in a filter
driver, we should be able to use the TopLevelIrp mechanism that FastFat is
using.
sara
-----Original Message-----
From: Tony Mason
To: File Systems Developers
Date: Wednesday, July 12, 2000 1:05 PM
Subject: [ntfsd] RE: preventing recursive loop in create dispatch hand ler
>Sara,
>
>This works until you load a filter that does NOT access the file in the
same
>thread context. I’ve seen implementations which do this (for example, they
>open the file in a service to examine it, or to obtain other information
>about the file.) Those implementations work fine (because they track that
>it is a “special thread” accessing the file and hence do not block such
>access) and your implementation will work fine.
>
>When you put the two together (such as with layered filter drivers) they
>will not function properly. Thus, it isn’t that this solution doesn’t work
>by itself, it is that this solution does not work when combined with a
>filter that works differently. I’m not proposing a hypothetical case here,
>this is a problem I have actually seen more than once (interaction between
>multiple filters using differing techniques to detect thread recursion.)
>
>Another potential solution I’ve been considering is that for each device
>object you create that filters an existing FSD volume, you create a SECOND
>device object that does NOT filter anything. Then, when you need to issue
a
>ZwCreateFile, you issue it against your second device object
>(“\Device\MyFilterAlternativeDevice0063A” or whatever unique name you
decide
>to use.) Then, when you receive the IRP against the second device object,
>you send it down to the original FSD. That bypasses the requirement to
>build an IRP_MJ_CREATE, detects the recursion (since nobody else is calling
>your device object) and avoids the difficulty of having other filter
drivers
>involved (since it is unlikely that any other filter driver would attatch
to
>your random named device object.)
>
>I hope this clarifies things (or perhaps it muddies the water even more.)
>This whole issue of detecting recursion is an ugly one, and I’ve seen at
>least a half-dozen approaches, each of which has some special drawback. No
>doubt, I’ll find a drawback in the scheme I just suggested one of these
days
>as well. I like to think of it as “job security.”
>
>Regards,
>
>Tony Mason
>Consulting Partner
>OSR Open Systems Resources, Inc.
>http://www.osr.com
>
>
>-----Original Message-----
>From: Sara Abraham [mailto:xxxxx@veritas.com]
>Sent: Wednesday, July 12, 2000 3:54 PM
>To: File Systems Developers
>Subject: [ntfsd] RE: preventing recursive loop in create dispatch hand ler
>
>
>Tony,
>
> I also cannot understand the problem that you see with Marc’s solution.
>Marc’s solution is equivalent to a simpler and nicer solution of using top
>level IRPs (IoGetTopLevelIrp()/IoSetTopLevelIrp) which will detect
recursion
>within a thread. We assume that when CreateDisptachHandler() is invoked a
>second time (on behalf of it’s own ZwCreateFile()), it will happen in the
>same thread context as the first CreateDisptachHandler(). Isn’t this always
>a true assumption ? We don’t worry about other threads accessing the same
>file.
>
>Sara
>
>-----Original Message-----
>From: Tony Mason
>To: File Systems Developers
>Date: Wednesday, July 12, 2000 12:04 PM
>Subject: [ntfsd] RE: preventing recursive loop in create dispatch hand ler
>
>
>>Marc,
>>
>>The point is that the original IRP_MJ_CREATE could be blocked while a
>>DIFFERENT thread accesses the (same) file. That doesn’t violate any rules
>>and is an implementation that I’ve seen numerous times.
>>
>>Of course, it is possible to process an IRP_MJ_CREATE in arbitrary thread
>>context, but it isn’t common, and it can be complex (and FSD dependent) to
>>get it right. But the problems typically involve security context
>>(credentials) and not memory (since everything has been captured when the
>>IRP was constructed.) So this wasn’t the case that concerned me very
much.
>>
>>As anyone who has attended the PlugFests at Microsoft can attest,
>>filter-to-filter interactions are a serious problem. IRP_MJ_CREATE
>handling
>>has been a source of a fair number of those problems.
>>
>>Regards,
>>
>>Tony
>>
>>Tony Mason
>>Consulting Partner
>>OSR Open Systems Resources, Inc.
>>http://www.osr.com http:
>>
>>-----Original Message-----
>>From: Marc Sherman [mailto:xxxxx@bionetrix.com]
>>Sent: Wednesday, July 12, 2000 2:52 PM
>>To: File Systems Developers
>>Subject: [ntfsd] RE: preventing recursive loop in create dispatch hand ler
>>
>>
>>
>>Tony,
>>
>>I thought fsd entry points must be called in the context of the
>requesting
>>thread. This implies that fs filters should not cause any change in thread
>>context. In this case, it should be the same thread that originally called
>>ZWCreateFile.
>>
>>Marc
>>
>>> -----Original Message-----
>>> From: Tony Mason [mailto:xxxxx@osr.com mailto:xxxxx]
>>> Sent: Wednesday, July 12, 2000 2:36 PM
>>> To: File Systems Developers
>>> Subject: [ntfsd] RE: preventing recursive loop in create
>>> dispatch hand ler
>>>
>>>
>>> Marc,
>>>
>>> The general problem with such approaches is that they “work”
>>> until you begin
>>> to deal with interactions involving other filter drivers,
>>> when any change in
>>> thread context will cause you to deadlock (because the mutex is not
>>> available, the second thread blocks. The first thread waits
>>> for the second
>>> thread to finish.) This is a problem that can occur when
>>> someone else’s
>>> filter is using handles (and thus switches to a known thread
>>> context) or
>>> uses captive threads (in a user service, or even system threads.)
>>>
>>> Regards,
>>>
>>> Tony Mason
>>> Consulting Partner
>>> OSR Open Systems Resources, Inc.
>>> http://www.osr.com http: < http://www.osr.com
>>http: >
>>>
>>>
>>> -----Original Message-----
>>> From: Marc Sherman [mailto:xxxxx@bionetrix.com
>>mailto:xxxxx]
>>> Sent: Wednesday, July 12, 2000 2:32 PM
>>> To: File Systems Developers
>>> Subject: [ntfsd] RE: preventing recursive loop in create
>>> dispatch hand ler
>>>
>>>
>>> Before calling ZWCreateFile, aquire a mutex, then check to
>>> see if you’ve
>>> aquired it recursively (mutex.Header.SignalState < 0). If
>>> this is true, your
>>> hadling the irp that resulted from your ZWCreateFile. Release
>>> the mutex and
>>> return. Remember to release your mutex after your ZWClose as
>>> well. This
>>> works for one of our filters.
>>>
>>> good luck,
>>> Marc
>>>
>>> -----Original Message-----
>>> From: Smith, Joel [mailto:xxxxx@ntpsoftware.com
>>mailto:xxxxx]
>>> Sent: Wednesday, July 12, 2000 2:21 PM
>>> To: File Systems Developers
>>> Subject: [ntfsd] RE: preventing recursive loop in create
>>> dispatch hand ler
>>>
>>>
>>>
>>> Thanks,
>>> In my case, though, I need to open the file before
>>> the file system
>>> because it is important that I know the file’s size (if the
>>> file exists),
>>> and, of course, a create might change the file’s size (overwrite,
>>> superceded, etc). I suppose I could backup to the parent
>>> directory and do a
>>> directory query for the file in question to determine its
>>> size. Then again,
>>> I wonder if FsRtlGetFileSize is smart enough to work with a
>>> FILE_OBJECT that
>>> has not been opened yet? Anyway, I’d still be interested in
>>> answers to my
>>> original question because just recognizing a recursive open
>>> may be simpler
>>> than solving the problem another way (like I mention above).
>>>
>>> Thanks,
>>> Joel
>>>
>>> -----Original Message-----
>>> From: Pavel Hrdina [mailto:xxxxx@sodatsw.cz mailto:xxxxx
>>> < mailto:xxxxx@sodatsw.cz mailto:xxxxx >]
>>> Sent: Wednesday, July 12, 2000 2:09 PM
>>> To: File Systems Developers
>>> Subject: [ntfsd] RE: preventing recursive loop in create
>>> dispatch handler
>>>
>>>
>>> The best solution is to let the FSD to process the desired open first
>>> and then you can read from the file. You do not need to close
>>> it because
>>> you’re using the same file object as the successful requestor
>>> of this create
>>>
>>> request.
>>>
>>> Paul
>>>
>>> > -----Pùvodní zpráva-----
>>> > Od: Smith, Joel [SMTP:xxxxx@ntpsoftware.com]
>>> > Odesláno: 12. èervence 2000 20:00
>>> > Komu: File Systems Developers
>>> > Pøedmìt: [ntfsd] preventing recursive loop in create
>>> dispatch handler
>>>
>>> >
>>> > Hello,
>>> > I know this has been asked before, and I apologize
>>> for asking
>>> > again, but…
>>> > Can anyone suggest a good way to prevent a
>>> recursive loop when
>>> > opening the destination file for a create in the create
>>> dispatch routine.
>>> >
>>> > for example:
>>> >
>>> > CreateDisptachHandler(device, irp)
>>> >
>
>>> > -ZwCreateFile(target of create) - this will cause a
>>> > recursive loop, obviously
>>> >
>>> > -read some settings from file if it exists
>>> >
>>> > -ZwClose(target of create)
>>> > }
>>> >
>>> > I believe there is a ‘well known’ solution to this
>>> problem, but I
>>> > don’t know it.
>>> >
>>> > Thanks,
>>> > Joel
>>> >
>>>
>>> —
>>> You are currently subscribed to ntfsd as: xxxxx@ntpsoftware.com
>>> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>>>
>>>
>>> —
>>> You are currently subscribed to ntfsd as: xxxxx@bionetrix.com
>>> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>>>
>>
>>
>>—
>>You are currently subscribed to ntfsd as: xxxxx@veritas.com
>>To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>>
>
>
>—
>You are currently subscribed to ntfsd as: xxxxx@osr.com
>To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
>—
>You are currently subscribed to ntfsd as: xxxxx@veritas.com
>To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
—
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)</mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></http:></http:></mailto:xxxxx></http:>