memory mapped file detection logic

>I have found that if I call FltFlushBuffers() in the pre-processing

callback for IRP_MJ_CLEANUP, all the dirty pages of a file mapping that are
in memory at >that time will be flushed and that this is a synchronous
operation.

What is the advantage of this over waiting for the data to be written out by
the mapped page writer? Also, this does not work if the pattern the
application uses is:

OpenFile()
MapFile()
CloseFile()
**Dirty mapping**

Which is quite common.

I realize that the file mapping can still be modified but, when the
IRP_MJ_CLOSE and ContextCleanup callbacks are invoked for that file object,
is it then >certain that all changes to the file mapping have been written
to the disk?

You can’t be sure that there won’t be any more changes via memory mapping
until the section goes away. For example, nothing prevents me from using two
different file objects to create writable data mappings to the file (i.e.
performing my steps above twice). The first file object will create and back
the data section and the second one will simply reference the existing data
section, so just because you see a close to *a* file object used for memory
mapping doesn’t mean there isn’t still a section out there (*).

Given that, as a filter it’s not really your business to know when the
section is deleted. You don’t have access to the necessary state to be able
to make that determination and the FS/Mm don’t provide you any way to get
that information. However, what you *do* know is when the last reference to
the stream (or file) goes away, by registering your StreamContext (or
FileContext) cleanup callback. When this is called you know that there are
no user or kernel references to the stream/file (including section
references) and therefore there won’t be any outstanding modifications.

Your other choice here is a layered FS, which is a LOT more work but might
provide more flexibility in the implementation. Depending on your
requirements though this might be the equivalent of using a shotgun as a fly
swatter :slight_smile:

-scott

*Note that it’s also possible for a file system to change the file object
that backs the section (see FsRtlChangeBackingFileObject), so, while
tempting, it’s dangerous to just say, “the section will be deleted after the
first file object goes away.” You also have a potential race in determining
what, “first” means, especially if you have filters beneath you (which can
muddy the waters even further here anyway).


Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc.
http://www.osronline.com

Scott,

I think you answered my question fully but, just to be sure, this is what I understood.

If I use FltSetStreamContext() to attach a stream context to a file object in my filter instance and I specified a ContextCleanupCallback for stream contexts in my filter registration data, then when I receive the context cleanup callback for a given file object, I can be certain that all dirty pages associated with all mappings for that file have been written to disk.

Is that what you said?

Thanks for the guidance,
David

>when I receive the context cleanup callback for a given file object, I can

be certain that all dirty pages associated with all mappings for that file
have >been written to disk.

A couple of corrections:

“when I receive the context cleanup callback for a given *stream context*, I
can be certain that all dirty pages associated with all mappings for that
*stream* have been written to disk.”

The idea is that the stream context only goes away when there are no more
file objects to the stream, therefore there can’t be anymore I/O (until
someone opens the stream again, that is).

(Also, to be clear because I have yet to mention it, up until this point
we’ve only been discussing local files.)

-scott


Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc.
http://www.osronline.com

wrote in message news:xxxxx@ntfsd…

Scott,

I think you answered my question fully but, just to be sure, this is what I
understood.

If I use FltSetStreamContext() to attach a stream context to a file object
in my filter instance and I specified a ContextCleanupCallback for stream
contexts in my filter registration data, then when I receive the context
cleanup callback for a given file object, I can be certain that all dirty
pages associated with all mappings for that file have been written to disk.

Is that what you said?

Thanks for the guidance,
David

One thing to keep in mind is that the ContextCleanupCallback for stream
contexts on W2K8 might not occur for hours or even days after the memory
mapped section is closed. On W2K3 I have seen it take about 5 seconds.

Bill Wandel

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Wednesday, November 23, 2011 11:05 AM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] memory mapped file detection logic

Scott,

I think you answered my question fully but, just to be sure, this is what I
understood.

If I use FltSetStreamContext() to attach a stream context to a file object
in my filter instance and I specified a ContextCleanupCallback for stream
contexts in my filter registration data, then when I receive the context
cleanup callback for a given file object, I can be certain that all dirty
pages associated with all mappings for that file have been written to disk.

Is that what you said?

Thanks for the guidance,
David


NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Right, in the case of memory mapped or cached file it’s going to be delayed
until Mm/Cc tear down their structures and deref the backing file object.
Often this won’t happen until there’s memory pressure (why throw away a
perfectly good data structure if you don’t need to?).

-scott


Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc.
http://www.osronline.com

“Bill Wandel” wrote in message news:xxxxx@ntfsd…

One thing to keep in mind is that the ContextCleanupCallback for stream
contexts on W2K8 might not occur for hours or even days after the memory
mapped section is closed. On W2K3 I have seen it take about 5 seconds.

Bill Wandel

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Wednesday, November 23, 2011 11:05 AM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] memory mapped file detection logic

Scott,

I think you answered my question fully but, just to be sure, this is what I
understood.

If I use FltSetStreamContext() to attach a stream context to a file object
in my filter instance and I specified a ContextCleanupCallback for stream
contexts in my filter registration data, then when I receive the context
cleanup callback for a given file object, I can be certain that all dirty
pages associated with all mappings for that file have been written to disk.

Is that what you said?

Thanks for the guidance,
David


NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer