Satisfying reads/writes from multiple sources (files)

I’m considering how to implement satisfying reads to a file from
multiple sources. FltReadFile/IssueSyncIo from the PostOp is not an
option, nor is FltDoCompletionProcessingWhenSafe, since the read/write
could be a paging one and these would both fault sooner or later.

Is there any proper mechanism for this, or am I forced to get a copy
of the data in read/write PreOp and fullfill it in the PostOp?
Or would even this not work, calling FltReadFile in the PreOp at some point?

I’ve tried to answer this a bunch of times, but I’m not sure where to start as this gets very complicated very quickly (problems like this is why we have wall to wall whiteboards in the office :)). The short answers are:

  1. You can page fault in the paging I/O path, you just can’t page fault in the paging **file ** I/O path

  2. Can you elaborate on the exact scenarios that you want to support? If you can wish away paging I/O then life is much better, but then you can’t support memory mapped files (maybe that’s OK?).

  3. If you do need to deal with paging I/Os, life is easier if you hide/restrict access to the redirected files. For example, if logical file A is made up of physical files B+C, you don’t ever want to let anyone write to B or C without going through the logical view of A (otherwise you need to invalidate A’s cache, which means you need isolation).

Isolation is not an issue, and you got the idea right: A is made up of
contents from B and C, and B and C are NEVER accessed by anyone but
ourselves.
If the read is to be satisfied fully from B or fully from C, it is trivial.

But fairly often the read (not write, thankfully!) needs to be
satisfied one part from B, second from C.

It will never be paging file I/O, we do not intend to touch the paging file.

I do not see page faulting as the issue, but a deadlock. Since this
path is very specific and rare, we use (of course, considering this is
read/write path…) nonpaged code AND nonpaged memory for buffers.

But I did get a compltaint from Driver Verifier about using
FltReadFile in the ReadPostOp. I presume the same could be possible in
ReadPreOp, as APCs can be disabled, and that would trigger the assert.

But I do not see any different way of handling this, and there must be
an option.

I’ve tried to answer this a bunch of times, but I’m not sure where to start
as this gets very complicated very quickly (problems like this is why we
have wall to wall whiteboards in the office ). The short answers are:

* You can page fault in the paging I/O path, you just can’t page fault in
the paging **file ** I/O path

* Can you elaborate on the exact scenarios that you want to support? If you
can wish away paging I/O then life is much better, but then you can’t
support memory mapped files (maybe that’s OK?).

* If you do need to deal with paging I/Os, life is easier if you
hide/restrict access to the redirected files. For example, if logical file A
is made up of physical files B+C, you don’t ever want to let anyone write to
B or C without going through the logical view of A (otherwise you need to
invalidate A’s cache, which means you need isolation).

Hmm, I tried FltReadFile in PreCreate, but it wont worl at all when called for image section creation.

Scott, do you see any issue with this approach:

  • Redirect file _A to A (and set it up with identical size)
  • When data is read from A beyond _A’s size, just let it through
  • When data is read from A below the size of _A, redirect it to original file
    in terms of supporting paging I/O?
    There are also corner cases, e.g. when data is written to _A, below the original size of A, but this is not something seen before the part where I have issues.

Trying to figure if a BSOD I am seeing is related to this part or something else entirely. All buffer checks passed.

I don’t see any problem with that, we’ve certainly done something similar (e.g. in a write protect filter that redirected all modifications to an alternate copy of the file).