Implementing CacheManager

Okay, I’m adding CacheManager to my NT filesystem. I’ve read the docs and
examined the MS IFS examples. But, I’m still missing part of the ‘big
picture.’ (I have a nickel and I’d like to buy a clue…thank you)

Q1: Who is responsible for copying the file data into the cache? When and
why does it get copied? (i.e. Does the CM copy the data to the cache
automatically after each IRP_MJ_READ? Or, should my filesystem memcpy the
data into the cache after each IRP_MJ_READ? I understand the CM init,
uninit, and support routines but I’m obviously missing something slightly
large here!)

Q2: Can anyone recommend a good user-mode application for testing purposes
that hammers on the CacheManager?

thanks in advance,
Fred Wallace.

Q1: Look at the FastFat code. There is no copying of data done by the
FSD, data transfer is done within the CM subsystem… just call the
appropriate routines.

For example, if a cached read comes in on a file and you call the CcRead
routine to satisfy it. If the data is in cache you get it back. If not,
then the system generates a non-cached read Irp which faults in the data
(much more details are under the covers involving VM-CM interactions
here, see Tony’s wonderful delineation on this in his FS class) The FSD
satisfies this non-cached read from the underlying device. In turn, the
CM then finally will return from your CcRead call with the data buffer
populated and the data in cache. Of course this assumes no errors have
occurred, etc.

Q2: Most of the MS products use cached IO. Notepad, in 2K, uses memory
mapped IO to catch even more obscure cases.

Pete

Peter Scott
xxxxx@KernelDrivers.com
http://www.KernelDrivers.com

>-----Original Message-----
>From: xxxxx@lists.osr.com [mailto:bounce-ntfsd-
>xxxxx@lists.osr.com] On Behalf Of xxxxx@charter.net
>Sent: Wednesday, August 14, 2002 10:41 AM
>To: File Systems Developers
>Subject: [ntfsd] Implementing CacheManager
>
>Okay, I’m adding CacheManager to my NT filesystem. I’ve read the docs
and
>examined the MS IFS examples. But, I’m still missing part of the ‘big
>picture.’ (I have a nickel and I’d like to buy a clue…thank you)
>
>Q1: Who is responsible for copying the file data into the cache? When
and
>why does it get copied? (i.e. Does the CM copy the data to the cache
>automatically after each IRP_MJ_READ? Or, should my filesystem memcpy
the
>data into the cache after each IRP_MJ_READ? I understand the CM init,
>uninit, and support routines but I’m obviously missing something
slightly
>large here!)
>
>Q2: Can anyone recommend a good user-mode application for testing
purposes
>that hammers on the CacheManager?
>
>thanks in advance,
>Fred Wallace.
>
>—
>You are currently subscribed to ntfsd as: xxxxx@KernelDrivers.com
>To unsubscribe send a blank email to %%email.unsub%%

> Q1: Who is responsible for copying the file data into the cache?
When and

why does it get copied? (i.e. Does the CM copy the data to the
cache
automatically after each IRP_MJ_READ? Or, should my filesystem
memcpy the
data into the cache after each IRP_MJ_READ? I understand the CM
init,
uninit, and support routines but I’m obviously missing something
slightly
large here!)

The cache pages are loaded with the file’s data using the paging IO
path of the same filesystem.

Q2: Can anyone recommend a good user-mode application for testing
purposes
that hammers on the CacheManager?

Try the IFS Test Kit.

Max

I started writing some code so I could check out what goes on with the
cache manager. Here’s what I found:

  1. On the first read. My IRP_MJ_READ handler gets called and I call
    CcInitializeCacheMap() to setup caching. Yea!

  2. On the second read, my FastIoRead function gets called. I call
    FsRtlCopyRead() which in turn calls my IRP_MJ_READ handler with the
    IRP_PAGING_IO flag set (Good so far). I read the data from the device,
    copy the data into the MDL, and return success to the calling
    FsRtlCopyRead() function. But, here’s the wrinkle, FsRtlCopyRead() then
    returns FALSE to my FastIoRead function.

  3. Since the FastIoRead fails, my IRP_MJ_READ handler gets called again, I
    re-read the data from the device and pass it back (again).

Anyone have an idea why FsRtlCopyRead() would fail (even though its
internal call to my IRP_MJ_READ succeeds)?

Thanks!

Are you correctly handling the paging request? Setting up the
information field, etc. correctly? It seems as though something maybe
failing in this pathway. What is coming back in your IoStatus.Status
field of the FsRtlCopyRead call?

Pete

Peter Scott
xxxxx@KernelDrivers.com
http://www.KernelDrivers.com

>-----Original Message-----
>From: xxxxx@lists.osr.com [mailto:bounce-ntfsd-
>xxxxx@lists.osr.com] On Behalf Of xxxxx@charter.net
>Sent: Wednesday, August 14, 2002 4:00 PM
>To: File Systems Developers
>Subject: [ntfsd] Re: Implementing CacheManager
>
>I started writing some code so I could check out what goes on with the
>cache manager. Here’s what I found:
>
>1. On the first read. My IRP_MJ_READ handler gets called and I call
>CcInitializeCacheMap() to setup caching. Yea!
>
>2. On the second read, my FastIoRead function gets called. I call
>FsRtlCopyRead() which in turn calls my IRP_MJ_READ handler with the
>IRP_PAGING_IO flag set (Good so far). I read the data from the
device,
>copy the data into the MDL, and return success to the calling
>FsRtlCopyRead() function. But, here’s the wrinkle, FsRtlCopyRead()
then
>returns FALSE to my FastIoRead function.
>
>3. Since the FastIoRead fails, my IRP_MJ_READ handler gets called
again, I
>re-read the data from the device and pass it back (again).
>
>
>Anyone have an idea why FsRtlCopyRead() would fail (even though its
>internal call to my IRP_MJ_READ succeeds)?
>
>Thanks!
>
>
>
>—
>You are currently subscribed to ntfsd as: xxxxx@KernelDrivers.com
>To unsubscribe send a blank email to %%email.unsub%%

I handle the IRP_PAGING_IO path the same as the non IRP_PAGING_IO path. I
stepped through both paths and they both succeed. I return
irp->IoStatus.Status = STATUS_SUCCESS and irp->IoStatus.Information =
actualByteCountRead in both cases.

FsRtlCopyRead() returns 0 in both IoStatus.Status and
IoStatus.Information. But, I verified that’s only because the function
never touches the IoStatus fields (they simply remain null).
FsRtlCopyRead() must not like something about the completed IRP_MJ_READ
irp or the fileobject it’s attempting to cache.