A question about memory mapped files and HSM.

NTFSD
1

In my HSM minifilter, I’ve been restoring on
IRP_MJ_QUERY_VOLUME_INFORMATION. This has so far been masking the issue
of dealing with memory mapped files, because I’m restoring the whole
file in one go on the first IRP that hits it.

Notepad, for instance, causes a QVI to happen before an
IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION comes in. I’ve been
restoring on the QVI, which is an IRP operation (and thus pendable).

I started looking into handling the memory mapped file issue more
directly, i.e. restoring on the IRP_MJ_AFSS (only for sync type
CreateSection) instead of the QVI. The problem, of course, is that I
can’t pend the IRP_MJ_AFSS when it comes. That didn’t stop me from
trying just to see what would happen, of course :wink:

The result of that is that when I try to FltCompletePendedPreOperation,
I BSOD (presumably because the operation has ceased to exist once I
returned STATUS_PENDING to FltMgr. This, as I understand it, would have
the effect of failing the operation, because it’s not a success code.)

In any event, that leaves me at square one wondering what a good
approach to this would be. I can think of several, but none that I
like:

  1. Block while the file restores. This could be several minutes. It
    seems like a really bad idea, though I’m having trouble finding
    documentation or anything in the list archives to that effect.

  2. Restore on QVI. This depends on getting a QVI before an AFSS. In
    procmon, I see that this isn’t the case when explorer tries to map in an
    executable. That’s not really a problem, since we’re not going to allow
    releasing of executable files anyway. However, it also looks like it’s
    not necessary for the QVI to happen. Procmon itself causes a lot of
    AFSS/CreateSection operations on files without a QVI.

  3. Disallow memory mapping of files that have been released. Even if I
    figured out how to do this robustly without causing pandemonium, it
    doesn’t seem like such a hot idea either.

Could anybody offer a suggestion about how to approach this problem?

Thanks,

~Eric

2

Maybe it is a good idea to restore on first read?

Blocking the paging write is a bad idea (MPW thread). But blocking a
paging read - the app is blocked in MiWaitForInPageComplete anyway.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

“Eric Diven” wrote in message news:xxxxx@ntfsd…
In my HSM minifilter, I’ve been restoring on
IRP_MJ_QUERY_VOLUME_INFORMATION. This has so far been masking the issue
of dealing with memory mapped files, because I’m restoring the whole
file in one go on the first IRP that hits it.

Notepad, for instance, causes a QVI to happen before an
IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION comes in. I’ve been
restoring on the QVI, which is an IRP operation (and thus pendable).

I started looking into handling the memory mapped file issue more
directly, i.e. restoring on the IRP_MJ_AFSS (only for sync type
CreateSection) instead of the QVI. The problem, of course, is that I
can’t pend the IRP_MJ_AFSS when it comes. That didn’t stop me from
trying just to see what would happen, of course :wink:

The result of that is that when I try to FltCompletePendedPreOperation,
I BSOD (presumably because the operation has ceased to exist once I
returned STATUS_PENDING to FltMgr. This, as I understand it, would have
the effect of failing the operation, because it’s not a success code.)

In any event, that leaves me at square one wondering what a good
approach to this would be. I can think of several, but none that I
like:

1) Block while the file restores. This could be several minutes. It
seems like a really bad idea, though I’m having trouble finding
documentation or anything in the list archives to that effect.

2) Restore on QVI. This depends on getting a QVI before an AFSS. In
procmon, I see that this isn’t the case when explorer tries to map in an
executable. That’s not really a problem, since we’re not going to allow
releasing of executable files anyway. However, it also looks like it’s
not necessary for the QVI to happen. Procmon itself causes a lot of
AFSS/CreateSection operations on files without a QVI.

3) Disallow memory mapping of files that have been released. Even if I
figured out how to do this robustly without causing pandemonium, it
doesn’t seem like such a hot idea either.

Could anybody offer a suggestion about how to approach this problem?

Thanks,

~Eric

3

Thanks, I hadn’t thought of approaching the problem at that layer. I’ll
try that once I’ve got the code in a stable state again. The only
question that doing it this way would raise for me is what happens if an
application maps a file, and then starts writing without ever having
caused the file to come back, which leaves me in the situation of
possibly having a paging write to a file that isn’t really there. I
have no idea if it’s possible for that situation to arise, but it should
be easy enough to test by writing a small test app that memory maps the
file and starts writing without reading and watching with IRP tracker or
procmon.

~Eric

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
Sent: Saturday, January 12, 2008 8:03 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] A question about memory mapped files and HSM.

Maybe it is a good idea to restore on first read?

Blocking the paging write is a bad idea (MPW thread). But blocking
a paging read - the app is blocked in MiWaitForInPageComplete anyway.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

“Eric Diven” wrote in message
news:xxxxx@ntfsd…
In my HSM minifilter, I’ve been restoring on
IRP_MJ_QUERY_VOLUME_INFORMATION. This has so far been masking the issue
of dealing with memory mapped files, because I’m restoring the whole
file in one go on the first IRP that hits it.

Notepad, for instance, causes a QVI to happen before an
IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION comes in. I’ve been
restoring on the QVI, which is an IRP operation (and thus pendable).

I started looking into handling the memory mapped file issue more
directly, i.e. restoring on the IRP_MJ_AFSS (only for sync type
CreateSection) instead of the QVI. The problem, of course, is that I
can’t pend the IRP_MJ_AFSS when it comes. That didn’t stop me from
trying just to see what would happen, of course :wink:

The result of that is that when I try to FltCompletePendedPreOperation,
I BSOD (presumably because the operation has ceased to exist once I
returned STATUS_PENDING to FltMgr. This, as I understand it, would have
the effect of failing the operation, because it’s not a success code.)

In any event, that leaves me at square one wondering what a good
approach to this would be. I can think of several, but none that I
like:

1) Block while the file restores. This could be several minutes. It
seems like a really bad idea, though I’m having trouble finding
documentation or anything in the list archives to that effect.

2) Restore on QVI. This depends on getting a QVI before an AFSS. In
procmon, I see that this isn’t the case when explorer tries to map in an
executable. That’s not really a problem, since we’re not going to allow
releasing of executable files anyway. However, it also looks like it’s
not necessary for the QVI to happen. Procmon itself causes a lot of
AFSS/CreateSection operations on files without a QVI.

3) Disallow memory mapping of files that have been released. Even if I
figured out how to do this robustly without causing pandemonium, it
doesn’t seem like such a hot idea either.

Could anybody offer a suggestion about how to approach this problem?

Thanks,

~Eric

—
NTFSD is sponsored by OSR

For our schedule debugging and file system seminars (including our new
fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@edsiohio.com To
unsubscribe send a blank email to xxxxx@lists.osr.com

4

Hi Eric,

If I am getting your question correct, you are asking whether it is possible
for an application to just map a file and then start writing to it without
actually having the original contents being read into memory?

If yes, then there are two scenarios that I can think of:

  1. If the application issues a write that is of the size of a page,
    then a corresponding paging write is issued. The contents are not read from
    the file via paging read operation. And if you see it logically, this seems
    reasonable; the entire page was going to be overwritten anyways.
  2. If the application issues a write of length that is not a multiple
    of page size then in this case a paging read will happen to bring in the
    contents. Then the page will be written to the length requested by the
    application. And later a paging write will happen to write it to disk.

Regards,
Ayush Gupta

5

The proven thing to do is recall the file on the acquire, when the section is created for read or write, as you already experimented. Recalling on a paging read or write will cause deadlocks. You can’t pend the acquire, so it is blocking, but that does not mean it is not cancellable: you will just have to redesign your recall so that it can wake it up on completion or cancellation. You will also have to respond to user-cancellation on the thread itself in the blocking call.

Ravi

From: xxxxx@lists.osr.com [xxxxx@lists.osr.com] On Behalf Of Eric Diven [xxxxx@edsiohio.com]
Sent: Wednesday, January 16, 2008 8:32 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] A question about memory mapped files and HSM.

Thanks, I hadn’t thought of approaching the problem at that layer. I’ll
try that once I’ve got the code in a stable state again. The only
question that doing it this way would raise for me is what happens if an
application maps a file, and then starts writing without ever having
caused the file to come back, which leaves me in the situation of
possibly having a paging write to a file that isn’t really there. I
have no idea if it’s possible for that situation to arise, but it should
be easy enough to test by writing a small test app that memory maps the
file and starts writing without reading and watching with IRP tracker or
procmon.

~Eric

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
Sent: Saturday, January 12, 2008 8:03 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] A question about memory mapped files and HSM.

Maybe it is a good idea to restore on first read?

Blocking the paging write is a bad idea (MPW thread). But blocking
a paging read - the app is blocked in MiWaitForInPageComplete anyway.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

“Eric Diven” wrote in message
news:xxxxx@ntfsd…
In my HSM minifilter, I’ve been restoring on
IRP_MJ_QUERY_VOLUME_INFORMATION. This has so far been masking the issue
of dealing with memory mapped files, because I’m restoring the whole
file in one go on the first IRP that hits it.

Notepad, for instance, causes a QVI to happen before an
IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION comes in. I’ve been
restoring on the QVI, which is an IRP operation (and thus pendable).

I started looking into handling the memory mapped file issue more
directly, i.e. restoring on the IRP_MJ_AFSS (only for sync type
CreateSection) instead of the QVI. The problem, of course, is that I
can’t pend the IRP_MJ_AFSS when it comes. That didn’t stop me from
trying just to see what would happen, of course :wink:

The result of that is that when I try to FltCompletePendedPreOperation,
I BSOD (presumably because the operation has ceased to exist once I
returned STATUS_PENDING to FltMgr. This, as I understand it, would have
the effect of failing the operation, because it’s not a success code.)

In any event, that leaves me at square one wondering what a good
approach to this would be. I can think of several, but none that I
like:

1) Block while the file restores. This could be several minutes. It
seems like a really bad idea, though I’m having trouble finding
documentation or anything in the list archives to that effect.

2) Restore on QVI. This depends on getting a QVI before an AFSS. In
procmon, I see that this isn’t the case when explorer tries to map in an
executable. That’s not really a problem, since we’re not going to allow
releasing of executable files anyway. However, it also looks like it’s
not necessary for the QVI to happen. Procmon itself causes a lot of
AFSS/CreateSection operations on files without a QVI.

3) Disallow memory mapping of files that have been released. Even if I
figured out how to do this robustly without causing pandemonium, it
doesn’t seem like such a hot idea either.

Could anybody offer a suggestion about how to approach this problem?

Thanks,

~Eric

—
NTFSD is sponsored by OSR

For our schedule debugging and file system seminars (including our new
fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@edsiohio.com To
unsubscribe send a blank email to xxxxx@lists.osr.com

—
NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

6

> 1. If the application issues a write that is of the size of a page,

then a corresponding paging write is issued.

This is true only for WriteFile calls, not true for memcpy() to a mapped file.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

7

Thanks Ravi, if this is the way I should go about this, I have several
questions:

The docs for FltSetCancelCompletion indicate that I can only set a
cancel routine for IRP based I/O. My understanding is that Acquire For
Section Synch is going to be either an FS_FILTER operation or a FASTIO
operation (the fast io path for this is supposedly not preferred, but
when I watch in ProcessMonitor, I see the operation as FASTIO?).

Apart from that detail, a reasonable approach seems like it might be (in
very rough pseudocode, no sync or anything):

cancel_fn ()
{
signal (cancel_event);
}

AFSS_preop ()
{
FltSetCancelCompletion (cancel_fn)
start_restore ()
wait (finished_event or cancel_event);

switch (event) {
case cancel_event:
callback_data->Iopb->Status = STATUS_CANCELED;
return FLT_PREOP_COMPLETE;
case finish_event:
return FLT_PREOP_SUCCESS_[WITH|NO]_CALLBACK;
}
}

A few other questions then:

I can’t find any documentation saying what IRQL the supplied
cancelcompletion routine is called at, other than it’s not called with
the CancelSpinlock held.
The docs say there can only be one cancel routine on a callbackdata at a
time, but don’t indicate whether or not I need to
FltClearCancelCompletion () before sending the IRP downwards when a
restore happens successfully.

~Eric

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ravi Pudipeddi
Sent: Wednesday, January 16, 2008 12:11 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] A question about memory mapped files and HSM.

The proven thing to do is recall the file on the acquire, when the
section is created for read or write, as you already experimented.
Recalling on a paging read or write will cause deadlocks. You can’t pend
the acquire, so it is blocking, but that does not mean it is not
cancellable: you will just have to redesign your recall so that it can
wake it up on completion or cancellation. You will also have to respond
to user-cancellation on the thread itself in the blocking call.

Ravi

8

Hi Maxim,

  1. If the application issues a write that is of the size of a page,
    then a corresponding paging write is issued.

This is true only for WriteFile calls, not true for memcpy() to a mapped
file.

Hmmm… Why is it so?

Regards,
Ayush Gupta

9

> Hmmm… Why is it so?

When memcpy() hits the first page, MM does not know the length of memcpy() and
thus cannot satisfy a fault using the free page.

When ReadFile/CcCopyRead hits the cache page, the read length is known and Cc
can satisfy the fault using the free page without reading it from the file.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com