While I was reading about the differences between creating a complete new FS stack and implementing an isolation filter I encountered the possibility of bypassing the filter by other drivers that send file objects directly to the underlying FS.
I thought about creating a stub-FS that only loads and unloads, and absorbs bypassed-FOs by returning error status codes when different IRPs are called.
Then the minifilter reparse the interesting files to the stub-FS (and volume) and use the filter itself to do the actual work (and complete all operations in the filter level).
Apart from the fact that I?m already creating a FS using IRPs (so why not do the rest), are there any long-term disadvantages to that approach?
The motivation for using a minifilter is mostly the more convenient API (unless there are other motivations). Another thing that I?ll gain from the approach is that it will allow me to proceed directly with the filter implementation, and I could add the stub-FS in the future if needed.
There are kernel components that do not handle STATUS_REPARSE directly.
I’ve always preferred the shadow file object model (which we first used - and coined the phrase “shadow file object” - back in the mid 1990s) as it is cleaner. But some of the folks in Redmond decided that filters get in the way and started passing file objects around the filter. NTFS and FAT are not tolerant of receiving the wrong file object, unfortunately.
Crashes in “XxxDecodeFileObject” are typically indicative of this problem. We’ve seen this arise in a number of different OS internal components over the years. Even filter manager skips over filters in cases (again, resolved in more recent OS versions, which is great if you can live in a plastic bubble and imagine that nobody cares about Windows XP or Server 2003 or Vista…)
That is why we ended up wandering down the parallel file system stack, with the thinking being if the association is internal, they can’t really pass the wrong file object to NTFS, can they? Turns out some developers are tricky and still manage to do so. And of course the parallel file system stack has its own complications.
Filter Manager also creates complications when experiencing a STATUS_REPARSE. IoCreateFileSpecifyDeviceObjectHint does not provide a graceful mechanism for communicating back information when a reparse point is found (you get a distinguished error), though this issue has been addressed in Windows 8 (via an ECP as I recall). Since FltCreateFile and its brethren are based on the I/O Manager function, they inherited the same problem. I’m sure you’re thinking “but nobody runs Windows versions prior to Windows 8, so this isn’t much of an issue anymore”. But of course there’s still people who aren’t aware of this issue.
The one good thing about components that don’t handle STATUS_REPARSE is you can break them without a filter. Just create a cross-volume symbolic link somewhere in the path. BOOM… I’ve pointed this out numerous times when people complain about seeing a STATUS_REPARSE (“hey, it’s a latent bug in your own code that you’re seeing because we take advantage of an existing feature/semantic.” They still grumble and complain about it and say not nice things about you when you aren’t looking.)
If you can stick with just a filter, your life will be more pleasant.
Though building filters is still harder than building file systems in many cases…
By creating an independent stack, a certain filter will have to reparse create operations into that stack anyway (as far as I understand). isn?t it the case with SDOs usually?
That is in fact the point: the only way to “get to” that alternate stack is to use STATUS_REPARSE.
I’m not sure what an SDO is, but I’ll guess that you means Shadow File Object.
Shadow File Objects were originally created so we could do this in a filter. Our first filter permitted sharing a single NTFS volume between multiple clients on a fiber channel network. We did this by permitting only a single node to write to the volume at a time. To refresh state, we’d dismount the NTFS volume and then remount it.
So a shadow file object would be what we sent to NTFS. The filter would get the I/O manager’s file object. We would call IoCreateStreamFileObject (or variant) and pass that file object down to NTFS. We’d store a pointer to that in the data structure we hung off FileObject->FsContext. We also kept those file object in a volume-level list. This permitted us to “disconnect” all those file objects on demand - we’d just close them. Then when a call came from the application, we’d reconnect the shadow file object (basically, we re-opened the file) to the underlying NTFS file system volume.
This worked quite well except for a leak in NTFS when you dismounted a volume. After 24 hours of leaking memory in this fashion a typical workstation would need to be rebooted. While the NTFS team did eventually fix that bug, there wasn’t much we could do to get around it. Fortunately, they also supported HFS (the old Mac file system) and the same trick worked just fine there. I still fondly remember plugging in MAC formatted volumes and having them show up properly in Disk Administrator (via a disk level filter that mapped the Mac partition table format to the Windows partition table format and vice versa). The fun bit then was to write an NTFS file system onto one of the spare Mac partitions and give it back for them to read.
We had to abandon that trick when Microsoft started bypassing filters. LOTS of components did (DFS and SRV). Someone even changed some of the FsRtl routines to bypass them as well. The fix that the filter manager team put into place eventually are the filter callbacks which in turn form the basis of the context association logic that filter manager exports to you today.
I’ve been assured by a highly credible source that no components bypass filters any longer, at least as of Windows 7. We do know of at least one filter manager case in XP that bypasses. Thus, inline filter models for doing this are once again possible. STATUS_REPARSE is supported architecturally, but creates interop issues with folks who don’t think they should follow all the obscure rules. Not that I’m bitter.
One of these days I’ll get around to finishing the isolation filter articles and provide sample code for it. I was massively disincented when some folks at Microsoft decided I wasn’t contributing enough to the community. That’s not the community’s fault, of course.
I understand. In my initial design I had one FO that points to the underlying file and many shadow files that that reference it in their contexts (for my solution I don?t need to have a pure solution that is completely transparent from the FS point of view and thus having a master FO that was opened with all permissions is good enough).
Currently I see two issues. When creating an underlying FO (unlike my initial design, i’m talking about an actual underlying FO per shadow FO), and initializing the shadow FO structure, should I just copy the Vpb pointer from the underlying FO? Would I need to reference it (or some of its members) in some way?
The second issue is paging-io locking ? I assume that for any of the following ?IRPs? (in the minifilter) I should lock the Shadow FO resources, in addition to letting them go down the stack to the underlying FO (which will result in double locking), is that correct?
There’s not a lot I can add to Tony’s comments beyond “+1”.
I’ll point out that with a parallel stack your interop story is likely to
get really difficult:
People (and this includes MS filters) won’t attach to your stack because
“you are not NTFS” (nothing to do with the characteristics you support)
Unless you are really careful, the filename you return won’t make sense to
many applications and filters and their user experience can be painful.
There are also some issues around making the device object acceptable to
WinInit if you are on the system volume.
On the other hand, “word on the streets” is that there are filters out there
which skip the stacks.
Given the emphasis that the Filter Manager team has put to making things
better for Isolation filter, it is a shame that they didn’t go the extra
mile and make their file system more robust to the wrong file object - an
error status back is a lot more friendly than a BSFOD. And remember that
its always the last installed filter that is blamed, not the buggy one.
As with most things in this space it’s a matter of choosing the least bad
option.
