Problems when the filter handles the cache callbacks

Hi,

My filter controls the cache of the files. When a cached write
comes to it, the filter itself calls CcCopyWrite. Noncached
writes are being passed down.

To achieve this, the filter must call CcInitializeCacheMap
for a file object, which I want to filter. This call requires
(among others) a pointer to four callbacks.
The callbacks are called by the Cache manager
and my filter handles them well.

Now the problem.

When I filter files on Fastfat, they sometimes get filled by zeros
after they’re written. This is because Fastfat calls CcZeroData
in the case when

  1. The write does not come from lazy writer (!!!)
  2. The write starts at offset greater than the file size.

Well, continuing on analyze, I asked why Fastfat thinks that the write
does not come from the lazy writer (which is not true in my case)

I found that FastFat tests the current thread with the value of
LazyWriteThread in the global data. This value is retrieved
in FatAcquireForLazyWrite function.

And now the conclusion - because my filter handles the
AcquireForLazyWrite callback, the Fastfat will never know
the value of LazyWriterThread and will never know that the
current write comes from the lazy writer.
Therefore, the file can be sometimes zeroed at random positions.

Does this problem have some solution ?
Should I somehow call the fastfat’s AcquireForLazyWrite
function ? Or should I ensure that lazywriter will always
write to the end of the file ?

(I wonder how NTFS handles this, but unfortunately,
I don’t have the sources).

Thanks for any advices

L.

You shoud not call CcInitializeCacheMap for a file object that belongs to
FAT. If you want to handle cached write yourself you need to create your own
file object and use it to initialize cache and pass this file object to
CcCopyWrite.

Alexei.

-----Original Message-----
From: Ladislav Zezula [mailto:xxxxx@volny.cz]
Sent: Wednesday, August 04, 2004 4:45 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Problems when the filter handles the cache callbacks

Hi,

My filter controls the cache of the files. When a cached write
comes to it, the filter itself calls CcCopyWrite. Noncached
writes are being passed down.

To achieve this, the filter must call CcInitializeCacheMap
for a file object, which I want to filter. This call requires
(among others) a pointer to four callbacks.
The callbacks are called by the Cache manager
and my filter handles them well.

Now the problem.

When I filter files on Fastfat, they sometimes get filled by zeros
after they’re written. This is because Fastfat calls CcZeroData
in the case when

  1. The write does not come from lazy writer (!!!)
  2. The write starts at offset greater than the file size.

Well, continuing on analyze, I asked why Fastfat thinks that the write
does not come from the lazy writer (which is not true in my case)

I found that FastFat tests the current thread with the value of
LazyWriteThread in the global data. This value is retrieved
in FatAcquireForLazyWrite function.

And now the conclusion - because my filter handles the
AcquireForLazyWrite callback, the Fastfat will never know
the value of LazyWriterThread and will never know that the
current write comes from the lazy writer.
Therefore, the file can be sometimes zeroed at random positions.

Does this problem have some solution ?
Should I somehow call the fastfat’s AcquireForLazyWrite
function ? Or should I ensure that lazywriter will always
write to the end of the file ?

(I wonder how NTFS handles this, but unfortunately,
I don’t have the sources).

Thanks for any advices

L.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@vmware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

> You shoud not call CcInitializeCacheMap for a file object

that belongs to FAT. If you want to handle cached write
yourself you need to create your own file object and use
it to initialize cache and pass this file object to CcCopyWrite.

Indeed. This is exactly the way I do. There are two file objects,
the “upper” which belongs to my filter, and the “lower”, which
is send to FAT when non-cached write comes.

L.

Both NTFS and FAT use the AcquireForLazyWrite callback (which they
registered with the cache manager when initializing caching on the file)
to somehow designate that this thread is the lazy writer. FAT remembers
the thread id and NTFS sets TopLevelIrp to FSRTL_CACHE_TOP_LEVEL_IRP.
If a file system is going to treat the lazy writer threads differently
somehow, I can’t think of a different way to do this than somehow
flagging the thread when the file system gets the AcquireForLazyWrite
callback.

There is not a good way for you to call the underlying file system’s
AcquireForLazyWrite callback because you don’t know what it is. That
routine is only stored inside the private cache manager structures, and
while I guess you could reverse engineer where these function pointers
are in the shared cache map, I wouldn’t recommend that option since the
structure can change between OS releases (including service packs).

As I look at the FAT code, I think that one or both of these problems
are in play:

  1. FAT’s idea of ValidDataLength is getting out of sync with your
    filter’s idea of ValidDataLength - i.e., it is getting behind where you
    expect it to be. This causes zeroing occurs when it doesn’t really need
    to.

  2. FAT is not properly synchronized in this path as your lazy write
    looks like a paging IO that is not the lazy writer, which FAT would
    assume is coming from Mm’s mapped page writer. Mm would have always
    called the FAT’s FastIoDispatch->AcquireForCcFlush in this case,
    therefore, some synchronization would already be held before this write
    came through.

I think you have a couple options here. You could convert the lazy
writes to look like top-level non-cached user writes to the file. This
will cause FAT to push out ValidDataLength with each lazy write and that
may keep you from getting FAT’s ValidDataLength well behind where you
are writing. The second option would be to make the lazy writes act
like mapped writes - you can call through FAT’s
FastIoDispatch->AcquireForCcFlush to acquire the appropriate
synchronization.

What will work best depends on how you manage keeping the underlying
file system’s understanding about the file in sync with your file
object. If you are ensuring that all accesses are coming through you
and the underlying file system’s file object is not getting accessed
directly, you could rely solely on your filter’s synchronization of the
file object and then translating all this paging IOs to noncached writes
may be the easiest answer.

Just one caveat here – I haven’t actually ever tried writing a filter
doing what you describe, so there may be some other problems here that
I’m not aware of :).

Regards,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, August 04, 2004 10:33 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Problems when the filter handles the cache
callbacks

You shoud not call CcInitializeCacheMap for a file object
that belongs to FAT. If you want to handle cached write
yourself you need to create your own file object and use
it to initialize cache and pass this file object to CcCopyWrite.

Indeed. This is exactly the way I do. There are two file objects,
the “upper” which belongs to my filter, and the “lower”, which
is send to FAT when non-cached write comes.

L.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Hi, Molly,

Thankyou for the answer.

while I guess you could reverse engineer where these function pointers
are in the shared cache map,

No, be sure that I won’t go this way. The problem that
I currently have is enough, I don’t need another ones :-))

FAT’s idea of ValidDataLength is getting out of sync with your
filter’s idea of ValidDataLength

Although the filter changed the EndOfFile value (by calling
IRP_MJ_SET+INFORMATION), the ValidDataLength
remains unchanged. The ValidDataLength changes later in the moment
when LazyWriter’s write completes.

Because I cannot change the ValidDataLength by any call
of IRP_MJ_SET_INFORMATION
(AFAIK, FileValidDataLengthInformation works on NTFS only)
I have to rely on Fastfat itself.

I think you have a couple options here. You could convert the lazy
writes to look like top-level non-cached user writes to the file. This

Well, but this will not help. The problem occurs when
Fastfat assumes the write is NOT the LazyWriter. Changing
LazyWriter’s IRP to anything else will not change this.
If I change it to mapped write, the write will be rejected by FastFat.
(Paged writes behind the end of file are not allowed, right ?)

I think the (only) option will be to zero the middle space in the
filter by sending non-cached write IRPs. Although this solution hurts
me because of degraded performance.

Just one caveat here – I haven’t actually ever tried writing a filter
doing what you describe, so there may be some other problems here that
I’m not aware of :).

Heh, don’t even try it, unless you will have at least one year
of free time :-))). Anyway, thanks again for the answer.

L.

I don’t think I explained my point about changing these writes to
non-paging non-cached writes clearly enough.

I’m assuming that you don’t pass through all the cached operations to
FAT since that would cause the data to be double cached in memory. But,
normally, it is through these cached interfaces that FAT knows the file
size and valid data length are getting extended. Therefore, you’ve got
to translate paging IO actions from the cached IO to your file into a
set of actions which would have the same net affect for FAT (causing the
needed file extensions and valid data length extension).

It sounds like you are already managing the file length extensions. If
you send down a non-paging non-cached write, FAT allows that type of
write to extend valid data length and FAT internally manages all the
needed synchronization. FAT also allows the mapped page writer to
extend valid data length, but it expects some synchronization to already
be held. Right now, my guess is that FAT is interpreting your paging
write like it would a paging write from the mapped page writer (since it
doesn’t think you are the lazy writer) and therefore making incorrect
assumptions about what synchronization is already held as you haven’t
preacquire the same synchronization that the mapped page write does.
This could lead to intermittent corruption as the movement of valid data
length isn’t properly synchronized.

I’m confused by your statement that your only option is to zero the
middle range yourself. My understanding from your initial description
was that the range of the file getting zeroed previously had data
written to it and that data was overwritten with the incorrect zeros.
If you can make all the writes extend valid data length when they occur,
you won’t be in the case where you’ve written out correct data, but
valid data length didn’t get extended therefore the next write incorrect
zeros out the data between valid data length and your current write.

Regards,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Thursday, August 05, 2004 10:53 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Problems when the filter handles the cache
callbacks

Hi, Molly,

Thankyou for the answer.

while I guess you could reverse engineer where these function pointers
are in the shared cache map,

No, be sure that I won’t go this way. The problem that
I currently have is enough, I don’t need another ones :-))

FAT’s idea of ValidDataLength is getting out of sync with your
filter’s idea of ValidDataLength

Although the filter changed the EndOfFile value (by calling
IRP_MJ_SET+INFORMATION), the ValidDataLength
remains unchanged. The ValidDataLength changes later in the moment
when LazyWriter’s write completes.

Because I cannot change the ValidDataLength by any call
of IRP_MJ_SET_INFORMATION
(AFAIK, FileValidDataLengthInformation works on NTFS only)
I have to rely on Fastfat itself.

I think you have a couple options here. You could convert the lazy
writes to look like top-level non-cached user writes to the file.
This

Well, but this will not help. The problem occurs when
Fastfat assumes the write is NOT the LazyWriter. Changing
LazyWriter’s IRP to anything else will not change this.
If I change it to mapped write, the write will be rejected by FastFat.
(Paged writes behind the end of file are not allowed, right ?)

I think the (only) option will be to zero the middle space in the
filter by sending non-cached write IRPs. Although this solution hurts
me because of degraded performance.

Just one caveat here – I haven’t actually ever tried writing a filter
doing what you describe, so there may be some other problems here that
I’m not aware of :).

Heh, don’t even try it, unless you will have at least one year
of free time :-))). Anyway, thanks again for the answer.

L.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Maybe I haven’t explained it clearly too, it’s
quite difficult to do it without a large amounts
of texts which no one wants to read :slight_smile:

First, answer to all your assumptions is “Yes”
(not passing cached operations, managing file sizes,
acquiring locks, …)

I’m confused by your statement that your only option is to zero the
middle range yourself. My understanding from your initial description

Eh, no. Sorry for the confusion. The effect occurs when

  1. Create New File
  2. WriteFile (10 MB of data)
  3. Close file

The “WriteFile” step generates a series of cached
IRP_MJ_WRITEs, which are catched by the filter
and sent to the cache using CcCopyWrite.

Then another series of IRP_MJ_WRITEs follows, (initiated
by the LazyWriter). According to your comment in some
previous list threads, the cache manager writes
the data sequentionally.

This is the point where the filter does one thing wrong.
* It posts some of the lazy writer requests *
So it can happen that e.g. a write at offset 0x1000,
length 0x1000 arrives into the FASTFAT before
the write at 0, length 0x1000. In such case,
FastFat calls FatZeroData (i.e. CcZeroData),
which will result zeroing some portion(s) of the
cached data and thus, zeroing some portion(s)
file.

This is not a bug in FASTFAT, but (as usually)
a bug in the filter, which occurs only on this special
combination of the events (one of them is that
the Fastfat’s AcquireForLazyWrite is never called).

The problem is solved now, thank you for your
answers.

L.

----- Original Message -----
From: “Molly Brown”
To: “Windows File Systems Devs Interest List”
Sent: Friday, August 06, 2004 6:27 PM
Subject: RE: [ntfsd] Problems when the filter handles the cache callbacks

I don’t think I explained my point about changing these writes to
non-paging non-cached writes clearly enough.

I’m assuming that you don’t pass through all the cached operations to
FAT since that would cause the data to be double cached in memory. But,
normally, it is through these cached interfaces that FAT knows the file
size and valid data length are getting extended. Therefore, you’ve got
to translate paging IO actions from the cached IO to your file into a
set of actions which would have the same net affect for FAT (causing the
needed file extensions and valid data length extension).

It sounds like you are already managing the file length extensions. If
you send down a non-paging non-cached write, FAT allows that type of
write to extend valid data length and FAT internally manages all the
needed synchronization. FAT also allows the mapped page writer to
extend valid data length, but it expects some synchronization to already
be held. Right now, my guess is that FAT is interpreting your paging
write like it would a paging write from the mapped page writer (since it
doesn’t think you are the lazy writer) and therefore making incorrect
assumptions about what synchronization is already held as you haven’t
preacquire the same synchronization that the mapped page write does.
This could lead to intermittent corruption as the movement of valid data
length isn’t properly synchronized.

I’m confused by your statement that your only option is to zero the
middle range yourself. My understanding from your initial description
was that the range of the file getting zeroed previously had data
written to it and that data was overwritten with the incorrect zeros.
If you can make all the writes extend valid data length when they occur,
you won’t be in the case where you’ve written out correct data, but
valid data length didn’t get extended therefore the next write incorrect
zeros out the data between valid data length and your current write.

Regards,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Thursday, August 05, 2004 10:53 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Problems when the filter handles the cache
callbacks

Hi, Molly,

Thankyou for the answer.

> while I guess you could reverse engineer where these function pointers
> are in the shared cache map,

No, be sure that I won’t go this way. The problem that
I currently have is enough, I don’t need another ones :-))

> FAT’s idea of ValidDataLength is getting out of sync with your
> filter’s idea of ValidDataLength

Although the filter changed the EndOfFile value (by calling
IRP_MJ_SET+INFORMATION), the ValidDataLength
remains unchanged. The ValidDataLength changes later in the moment
when LazyWriter’s write completes.

Because I cannot change the ValidDataLength by any call
of IRP_MJ_SET_INFORMATION
(AFAIK, FileValidDataLengthInformation works on NTFS only)
I have to rely on Fastfat itself.

> I think you have a couple options here. You could convert the lazy
> writes to look like top-level non-cached user writes to the file.
This

Well, but this will not help. The problem occurs when
Fastfat assumes the write is NOT the LazyWriter. Changing
LazyWriter’s IRP to anything else will not change this.
If I change it to mapped write, the write will be rejected by FastFat.
(Paged writes behind the end of file are not allowed, right ?)

I think the (only) option will be to zero the middle space in the
filter by sending non-cached write IRPs. Although this solution hurts
me because of degraded performance.

> Just one caveat here – I haven’t actually ever tried writing a filter
> doing what you describe, so there may be some other problems here that
> I’m not aware of :).

Heh, don’t even try it, unless you will have at least one year
of free time :-))). Anyway, thanks again for the answer.

L.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@volny.cz
To unsubscribe send a blank email to xxxxx@lists.osr.com