Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTFSD
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


Problems when the filter handles the cache callbacks

OSR_Community_UserOSR_Community_User Member Posts: 110,217
Hi,

My filter controls the cache of the files. When a cached write
comes to it, the filter itself calls CcCopyWrite. Noncached
writes are being passed down.

To achieve this, the filter must call CcInitializeCacheMap
for a file object, which I want to filter. This call requires
(among others) a pointer to four callbacks.
The callbacks are called by the Cache manager
and my filter handles them well.

Now the problem.

When I filter files on Fastfat, they sometimes get filled by zeros
after they're written. This is because Fastfat calls CcZeroData
in the case when

1. The write does not come from lazy writer (!!!!)
2. The write starts at offset greater than the file size.

Well, continuing on analyze, I asked why Fastfat thinks that the write
does not come from the lazy writer (which is not true in my case)

I found that FastFat tests the current thread with the value of
LazyWriteThread in the global data. This value is retrieved
in FatAcquireForLazyWrite function.

And now the conclusion - because my filter handles the
AcquireForLazyWrite callback, the Fastfat will never know
the value of LazyWriterThread and will never know that the
current write comes from the lazy writer.
Therefore, the file can be sometimes zeroed at random positions.

Does this problem have some solution ?
Should I somehow call the fastfat's AcquireForLazyWrite
function ? Or should I ensure that lazywriter will always
write to the end of the file ?

(I wonder how NTFS handles this, but unfortunately,
I don't have the sources).

Thanks for any advices

L.

Comments

  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    You shoud not call CcInitializeCacheMap for a file object that belongs to
    FAT. If you want to handle cached write yourself you need to create your own
    file object and use it to initialize cache and pass this file object to
    CcCopyWrite.

    Alexei.

    -----Original Message-----
    From: Ladislav Zezula [mailto:[email protected]]
    Sent: Wednesday, August 04, 2004 4:45 AM
    To: Windows File Systems Devs Interest List
    Subject: [ntfsd] Problems when the filter handles the cache callbacks


    Hi,

    My filter controls the cache of the files. When a cached write
    comes to it, the filter itself calls CcCopyWrite. Noncached
    writes are being passed down.

    To achieve this, the filter must call CcInitializeCacheMap
    for a file object, which I want to filter. This call requires
    (among others) a pointer to four callbacks.
    The callbacks are called by the Cache manager
    and my filter handles them well.

    Now the problem.

    When I filter files on Fastfat, they sometimes get filled by zeros
    after they're written. This is because Fastfat calls CcZeroData
    in the case when

    1. The write does not come from lazy writer (!!!!)
    2. The write starts at offset greater than the file size.

    Well, continuing on analyze, I asked why Fastfat thinks that the write
    does not come from the lazy writer (which is not true in my case)

    I found that FastFat tests the current thread with the value of
    LazyWriteThread in the global data. This value is retrieved
    in FatAcquireForLazyWrite function.

    And now the conclusion - because my filter handles the
    AcquireForLazyWrite callback, the Fastfat will never know
    the value of LazyWriterThread and will never know that the
    current write comes from the lazy writer.
    Therefore, the file can be sometimes zeroed at random positions.

    Does this problem have some solution ?
    Should I somehow call the fastfat's AcquireForLazyWrite
    function ? Or should I ensure that lazywriter will always
    write to the end of the file ?

    (I wonder how NTFS handles this, but unfortunately,
    I don't have the sources).

    Thanks for any advices

    L.


    ---
    Questions? First check the IFS FAQ at
    https://www.osronline.com/article.cfm?id=17

    You are currently subscribed to ntfsd as: [email protected]
    To unsubscribe send a blank email to [email protected]
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > You shoud not call CcInitializeCacheMap for a file object
    > that belongs to FAT. If you want to handle cached write
    > yourself you need to create your own file object and use
    > it to initialize cache and pass this file object to CcCopyWrite.

    Indeed. This is exactly the way I do. There are two file objects,
    the "upper" which belongs to my filter, and the "lower", which
    is send to FAT when non-cached write comes.

    L.
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Both NTFS and FAT use the AcquireForLazyWrite callback (which they
    registered with the cache manager when initializing caching on the file)
    to somehow designate that this thread is the lazy writer. FAT remembers
    the thread id and NTFS sets TopLevelIrp to FSRTL_CACHE_TOP_LEVEL_IRP.
    If a file system is going to treat the lazy writer threads differently
    somehow, I can't think of a different way to do this than somehow
    flagging the thread when the file system gets the AcquireForLazyWrite
    callback.

    There is not a good way for you to call the underlying file system's
    AcquireForLazyWrite callback because you don't know what it is. That
    routine is only stored inside the private cache manager structures, and
    while I guess you could reverse engineer where these function pointers
    are in the shared cache map, I wouldn't recommend that option since the
    structure can change between OS releases (including service packs).

    As I look at the FAT code, I think that one or both of these problems
    are in play:

    1. FAT's idea of ValidDataLength is getting out of sync with your
    filter's idea of ValidDataLength - i.e., it is getting behind where you
    expect it to be. This causes zeroing occurs when it doesn't really need
    to.

    2. FAT is not properly synchronized in this path as your lazy write
    looks like a paging IO that is not the lazy writer, which FAT would
    assume is coming from Mm's mapped page writer. Mm would have always
    called the FAT's FastIoDispatch->AcquireForCcFlush in this case,
    therefore, some synchronization would already be held before this write
    came through.

    I think you have a couple options here. You could convert the lazy
    writes to look like top-level non-cached user writes to the file. This
    will cause FAT to push out ValidDataLength with each lazy write and that
    may keep you from getting FAT's ValidDataLength well behind where you
    are writing. The second option would be to make the lazy writes act
    like mapped writes - you can call through FAT's
    FastIoDispatch->AcquireForCcFlush to acquire the appropriate
    synchronization.

    What will work best depends on how you manage keeping the underlying
    file system's understanding about the file in sync with your file
    object. If you are ensuring that all accesses are coming through you
    and the underlying file system's file object is not getting accessed
    directly, you could rely solely on your filter's synchronization of the
    file object and then translating all this paging IOs to noncached writes
    may be the easiest answer.

    Just one caveat here -- I haven't actually ever tried writing a filter
    doing what you describe, so there may be some other problems here that
    I'm not aware of :).

    Regards,
    Molly Brown
    Microsoft Corporation

    This posting is provided "AS IS" with no warranties and confers no
    rights.

    -----Original Message-----
    From: [email protected]
    [mailto:[email protected]] On Behalf Of Ladislav Zezula
    Sent: Wednesday, August 04, 2004 10:33 PM
    To: Windows File Systems Devs Interest List
    Subject: Re: [ntfsd] Problems when the filter handles the cache
    callbacks

    > You shoud not call CcInitializeCacheMap for a file object
    > that belongs to FAT. If you want to handle cached write
    > yourself you need to create your own file object and use
    > it to initialize cache and pass this file object to CcCopyWrite.

    Indeed. This is exactly the way I do. There are two file objects,
    the "upper" which belongs to my filter, and the "lower", which
    is send to FAT when non-cached write comes.

    L.


    ---
    Questions? First check the IFS FAQ at
    https://www.osronline.com/article.cfm?id=17

    You are currently subscribed to ntfsd as: [email protected]
    To unsubscribe send a blank email to [email protected]
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Hi, Molly,

    Thankyou for the answer.

    > while I guess you could reverse engineer where these function pointers
    > are in the shared cache map,

    No, be sure that I won't go this way. The problem that
    I currently have is enough, I don't need another ones :-))

    > FAT's idea of ValidDataLength is getting out of sync with your
    > filter's idea of ValidDataLength

    Although the filter changed the EndOfFile value (by calling
    IRP_MJ_SET+INFORMATION), the ValidDataLength
    remains unchanged. The ValidDataLength changes later in the moment
    when LazyWriter's write completes.

    Because I cannot change the ValidDataLength by any call
    of IRP_MJ_SET_INFORMATION
    (AFAIK, FileValidDataLengthInformation works on NTFS only)
    I have to rely on Fastfat itself.

    > I think you have a couple options here. You could convert the lazy
    > writes to look like top-level non-cached user writes to the file. This

    Well, but this will not help. The problem occurs when
    Fastfat assumes the write is NOT the LazyWriter. Changing
    LazyWriter's IRP to anything else will not change this.
    If I change it to mapped write, the write will be rejected by FastFat.
    (Paged writes behind the end of file are not allowed, right ?)

    I think the (only) option will be to zero the middle space in the
    filter by sending non-cached write IRPs. Although this solution hurts
    me because of degraded performance.

    > Just one caveat here -- I haven't actually ever tried writing a filter
    > doing what you describe, so there may be some other problems here that
    > I'm not aware of :).

    Heh, don't even try it, unless you will have at least one year
    of free time :-))). Anyway, thanks again for the answer.

    L.
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    I don't think I explained my point about changing these writes to
    non-paging non-cached writes clearly enough.

    I'm assuming that you don't pass through all the cached operations to
    FAT since that would cause the data to be double cached in memory. But,
    normally, it is through these cached interfaces that FAT knows the file
    size and valid data length are getting extended. Therefore, you've got
    to translate paging IO actions from the cached IO to your file into a
    set of actions which would have the same net affect for FAT (causing the
    needed file extensions and valid data length extension).

    It sounds like you are already managing the file length extensions. If
    you send down a non-paging non-cached write, FAT allows that type of
    write to extend valid data length and FAT internally manages all the
    needed synchronization. FAT also allows the mapped page writer to
    extend valid data length, but it expects some synchronization to already
    be held. Right now, my guess is that FAT is interpreting your paging
    write like it would a paging write from the mapped page writer (since it
    doesn't think you are the lazy writer) and therefore making incorrect
    assumptions about what synchronization is already held as you haven't
    preacquire the same synchronization that the mapped page write does.
    This could lead to intermittent corruption as the movement of valid data
    length isn't properly synchronized.

    I'm confused by your statement that your only option is to zero the
    middle range yourself. My understanding from your initial description
    was that the range of the file getting zeroed previously had data
    written to it and that data was overwritten with the incorrect zeros.
    If you can make all the writes extend valid data length when they occur,
    you won't be in the case where you've written out correct data, but
    valid data length didn't get extended therefore the next write incorrect
    zeros out the data between valid data length and your current write.

    Regards,
    Molly Brown
    Microsoft Corporation

    This posting is provided "AS IS" with no warranties and confers no
    rights.

    -----Original Message-----
    From: [email protected]
    [mailto:[email protected]] On Behalf Of Ladislav Zezula
    Sent: Thursday, August 05, 2004 10:53 PM
    To: Windows File Systems Devs Interest List
    Subject: Re: [ntfsd] Problems when the filter handles the cache
    callbacks

    Hi, Molly,

    Thankyou for the answer.

    > while I guess you could reverse engineer where these function pointers
    > are in the shared cache map,

    No, be sure that I won't go this way. The problem that
    I currently have is enough, I don't need another ones :-))

    > FAT's idea of ValidDataLength is getting out of sync with your
    > filter's idea of ValidDataLength

    Although the filter changed the EndOfFile value (by calling
    IRP_MJ_SET+INFORMATION), the ValidDataLength
    remains unchanged. The ValidDataLength changes later in the moment
    when LazyWriter's write completes.

    Because I cannot change the ValidDataLength by any call
    of IRP_MJ_SET_INFORMATION
    (AFAIK, FileValidDataLengthInformation works on NTFS only)
    I have to rely on Fastfat itself.

    > I think you have a couple options here. You could convert the lazy
    > writes to look like top-level non-cached user writes to the file.
    This

    Well, but this will not help. The problem occurs when
    Fastfat assumes the write is NOT the LazyWriter. Changing
    LazyWriter's IRP to anything else will not change this.
    If I change it to mapped write, the write will be rejected by FastFat.
    (Paged writes behind the end of file are not allowed, right ?)

    I think the (only) option will be to zero the middle space in the
    filter by sending non-cached write IRPs. Although this solution hurts
    me because of degraded performance.

    > Just one caveat here -- I haven't actually ever tried writing a filter
    > doing what you describe, so there may be some other problems here that
    > I'm not aware of :).

    Heh, don't even try it, unless you will have at least one year
    of free time :-))). Anyway, thanks again for the answer.

    L.


    ---
    Questions? First check the IFS FAQ at
    https://www.osronline.com/article.cfm?id=17

    You are currently subscribed to ntfsd as: [email protected]
    To unsubscribe send a blank email to [email protected]
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Maybe I haven't explained it clearly too, it's
    quite difficult to do it without a large amounts
    of texts which no one wants to read :-)

    First, answer to all your assumptions is "Yes"
    (not passing cached operations, managing file sizes,
    acquiring locks, ...)

    > I'm confused by your statement that your only option is to zero the
    > middle range yourself. My understanding from your initial description

    Eh, no. Sorry for the confusion. The effect occurs when

    1. Create New File
    2. WriteFile (10 MB of data)
    3. Close file

    The "WriteFile" step generates a series of cached
    IRP_MJ_WRITEs, which are catched by the filter
    and sent to the cache using CcCopyWrite.

    Then another series of IRP_MJ_WRITEs follows, (initiated
    by the LazyWriter). According to your comment in some
    previous list threads, the cache manager writes
    the data sequentionally.

    This is the point where the filter does one thing wrong.
    * It posts some of the lazy writer requests *
    So it can happen that e.g. a write at offset 0x1000,
    length 0x1000 arrives into the FASTFAT before
    the write at 0, length 0x1000. In such case,
    FastFat calls FatZeroData (i.e. CcZeroData),
    which will result zeroing some portion(s) of the
    cached data and thus, zeroing some portion(s)
    file.

    This is not a bug in FASTFAT, but (as usually)
    a bug in the filter, which occurs only on this special
    combination of the events (one of them is that
    the Fastfat's AcquireForLazyWrite is never called).

    The problem is solved now, thank you for your
    answers.

    L.

    ----- Original Message -----
    From: "Molly Brown" <[email protected]>
    To: "Windows File Systems Devs Interest List" <[email protected]>
    Sent: Friday, August 06, 2004 6:27 PM
    Subject: RE: [ntfsd] Problems when the filter handles the cache callbacks


    I don't think I explained my point about changing these writes to
    non-paging non-cached writes clearly enough.

    I'm assuming that you don't pass through all the cached operations to
    FAT since that would cause the data to be double cached in memory. But,
    normally, it is through these cached interfaces that FAT knows the file
    size and valid data length are getting extended. Therefore, you've got
    to translate paging IO actions from the cached IO to your file into a
    set of actions which would have the same net affect for FAT (causing the
    needed file extensions and valid data length extension).

    It sounds like you are already managing the file length extensions. If
    you send down a non-paging non-cached write, FAT allows that type of
    write to extend valid data length and FAT internally manages all the
    needed synchronization. FAT also allows the mapped page writer to
    extend valid data length, but it expects some synchronization to already
    be held. Right now, my guess is that FAT is interpreting your paging
    write like it would a paging write from the mapped page writer (since it
    doesn't think you are the lazy writer) and therefore making incorrect
    assumptions about what synchronization is already held as you haven't
    preacquire the same synchronization that the mapped page write does.
    This could lead to intermittent corruption as the movement of valid data
    length isn't properly synchronized.

    I'm confused by your statement that your only option is to zero the
    middle range yourself. My understanding from your initial description
    was that the range of the file getting zeroed previously had data
    written to it and that data was overwritten with the incorrect zeros.
    If you can make all the writes extend valid data length when they occur,
    you won't be in the case where you've written out correct data, but
    valid data length didn't get extended therefore the next write incorrect
    zeros out the data between valid data length and your current write.

    Regards,
    Molly Brown
    Microsoft Corporation

    This posting is provided "AS IS" with no warranties and confers no
    rights.

    -----Original Message-----
    From: [email protected]
    [mailto:[email protected]] On Behalf Of Ladislav Zezula
    Sent: Thursday, August 05, 2004 10:53 PM
    To: Windows File Systems Devs Interest List
    Subject: Re: [ntfsd] Problems when the filter handles the cache
    callbacks

    Hi, Molly,

    Thankyou for the answer.

    > while I guess you could reverse engineer where these function pointers
    > are in the shared cache map,

    No, be sure that I won't go this way. The problem that
    I currently have is enough, I don't need another ones :-))

    > FAT's idea of ValidDataLength is getting out of sync with your
    > filter's idea of ValidDataLength

    Although the filter changed the EndOfFile value (by calling
    IRP_MJ_SET+INFORMATION), the ValidDataLength
    remains unchanged. The ValidDataLength changes later in the moment
    when LazyWriter's write completes.

    Because I cannot change the ValidDataLength by any call
    of IRP_MJ_SET_INFORMATION
    (AFAIK, FileValidDataLengthInformation works on NTFS only)
    I have to rely on Fastfat itself.

    > I think you have a couple options here. You could convert the lazy
    > writes to look like top-level non-cached user writes to the file.
    This

    Well, but this will not help. The problem occurs when
    Fastfat assumes the write is NOT the LazyWriter. Changing
    LazyWriter's IRP to anything else will not change this.
    If I change it to mapped write, the write will be rejected by FastFat.
    (Paged writes behind the end of file are not allowed, right ?)

    I think the (only) option will be to zero the middle space in the
    filter by sending non-cached write IRPs. Although this solution hurts
    me because of degraded performance.

    > Just one caveat here -- I haven't actually ever tried writing a filter
    > doing what you describe, so there may be some other problems here that
    > I'm not aware of :).

    Heh, don't even try it, unless you will have at least one year
    of free time :-))). Anyway, thanks again for the answer.

    L.


    ---
    Questions? First check the IFS FAQ at
    https://www.osronline.com/article.cfm?id=17

    You are currently subscribed to ntfsd as: [email protected]
    To unsubscribe send a blank email to [email protected]

    ---
    Questions? First check the IFS FAQ at
    https://www.osronline.com/article.cfm?id=17

    You are currently subscribed to ntfsd as: [email protected]
    To unsubscribe send a blank email to [email protected]
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Kernel Debugging 30 Mar 2020 OSR Seminar Space
Developing Minifilters 15 Jun 2020 LIVE ONLINE
Writing WDF Drivers 22 June 2020 LIVE ONLINE
Internals & Software Drivers 28 Sept 2020 Dulles, VA