Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

Satisfying reads/writes from multiple sources (files)

Dejan_MaksimovicDejan_Maksimovic Member - All Emails Posts: 239
via Email in NTFSD
I'm considering how to implement satisfying reads to a file from
multiple sources. FltReadFile/IssueSyncIo from the PostOp is not an
option, nor is FltDoCompletionProcessingWhenSafe, since the read/write
could be a paging one and these would both fault sooner or later.


Is there any proper mechanism for this, or am I forced to get a copy
of the data in read/write PreOp and fullfill it in the PostOp?
Or would even this not work, calling FltReadFile in the PreOp at some point?

Comments

  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,151

    I've tried to answer this a bunch of times, but I'm not sure where to start as this gets very complicated very quickly (problems like this is why we have wall to wall whiteboards in the office :)). The short answers are:

    1. You can page fault in the paging I/O path, you just can't page fault in the paging **file ** I/O path

    2. Can you elaborate on the exact scenarios that you want to support? If you can wish away paging I/O then life is much better, but then you can't support memory mapped files (maybe that's OK?).

    3. If you do need to deal with paging I/Os, life is easier if you hide/restrict access to the redirected files. For example, if logical file A is made up of physical files B+C, you don't ever want to let anyone write to B or C without going through the logical view of A (otherwise you need to invalidate A's cache, which means you need isolation).

    -scott
    OSR

  • Dejan_MaksimovicDejan_Maksimovic Member - All Emails Posts: 239
    via Email
    Isolation is not an issue, and you got the idea right: A is made up of
    contents from B and C, and B and C are NEVER accessed by anyone but
    ourselves.
    If the read is to be satisfied fully from B or fully from C, it is trivial.

    But fairly often the read (not write, thankfully!) needs to be
    satisfied one part from B, second from C.

    It will never be paging file I/O, we do not intend to touch the paging file.

    I do not see page faulting as the issue, but a deadlock. Since this
    path is very specific and rare, we use (of course, considering this is
    read/write path..) nonpaged code AND nonpaged memory for buffers.

    But I did get a compltaint from Driver Verifier about using
    FltReadFile in the ReadPostOp. I presume the same could be possible in
    ReadPreOp, as APCs can be disabled, and that would trigger the assert.

    But I do not see any different way of handling this, and there must be
    an option.

    > I've tried to answer this a bunch of times, but I'm not sure where to start
    > as this gets very complicated very quickly (problems like this is why we
    > have wall to wall whiteboards in the office ). The short answers are:
    >
    >
    > * You can page fault in the paging I/O path, you just can't page fault in
    > the paging **file ** I/O path
    >
    >
    > * Can you elaborate on the exact scenarios that you want to support? If you
    > can wish away paging I/O then life is much better, but then you can't
    > support memory mapped files (maybe that's OK?).
    >
    >
    > * If you do need to deal with paging I/Os, life is easier if you
    > hide/restrict access to the redirected files. For example, if logical file A
    > is made up of physical files B+C, you don't ever want to let anyone write to
    > B or C without going through the logical view of A (otherwise you need to
    > invalidate A's cache, which means you need isolation).
  • Dejan_MaksimovicDejan_Maksimovic Member - All Emails Posts: 239
    Hmm, I tried FltReadFile in PreCreate, but it wont worl at all when called for image section creation.
  • Dejan_MaksimovicDejan_Maksimovic Member - All Emails Posts: 239
    edited April 24

    Scott, do you see any issue with this approach:

    • Redirect file _A to A (and set it up with identical size)
    • When data is read from A beyond _A's size, just let it through
    • When data is read from A below the size of _A, redirect it to original file
      in terms of supporting paging I/O?
      There are also corner cases, e.g. when data is written to _A, below the original size of A, but this is not something seen before the part where I have issues.

    Trying to figure if a BSOD I am seeing is related to this part or something else entirely. All buffer checks passed.

  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,151

    I don't see any problem with that, we've certainly done something similar (e.g. in a write protect filter that redirected all modifications to an alternate copy of the file).

    -scott
    OSR

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
Developing Minifilters 29 July 2019 OSR Seminar Space
Writing WDF Drivers 23 Sept 2019 OSR Seminar Space
Kernel Debugging 21 Oct 2019 OSR Seminar Space
Internals & Software Drivers 18 Nov 2019 Dulles, VA