Memory mapped IO consistency problem

NTFSD
1

Hi,
My FSD supports memory mapped IO but not cache manager interactions or
fastIO. So Ive set FileObject->sectionObjectPointers to my
Fcb->SectionObject (duly initialized), and fileObject->fsContext to the
commonFcbHeader.

Im trying to open a text file with Notepad, save some new contents, close,
and reopen the same file. I find that when I reopen the file I get the old
contents back. However I know that the contents of the file stream have
actually been modified after the save. So this seems to be a data
consistency problem with my memory mapped IO implementation.

Specifically, the initial read of the file is a paging IO read (after my
createSection callbacks are invoked). The writes are not paging IO writes.
Finally, when the file is reopened, the FSD doesnt receive a read request
at all, indicating that the VMM satisfies this read itself. Am I missing
some step here? Why is the write not updating the memory where the file is
mapped?

In an anient post Tony says:
“The disadvantage of implementing the VMM interface without the cache
manager
is that you have a consistency “problem” that you must solve (or ignore.
That’s ONE “solution” - just ignore consistency.)”

Is this the problem Im seeing? If it is, could you elaborate a little on
this consistency problem and how to solve it?

Thanks,
Jagannath

2

Jagnnath,

I haven’t looked at trying to implement a “paging I/O only” file system in a
very long time, but from your description it sounds like this might be the
worst possible scenario for you: the application READS the file using memory
mapped I/O and MODIFIES it using write.

This would describe what you are seeing - the next read is satisfied from
memory (since MM sees it there, no need to fetch it again). If this is the
case, you’d need to invalidate the cache (since the write has made whatever
is in memory invalid.)

While this can certainly happen in the “real world” (and that is what I was
mentioning in my earlier discussion) what you are seeing is a single
application that is causing exactly this behavior.

What you must do is purge the cache in this case. Of course, you can’t do
that while it is mapped (purge will fail) so you will have to do it as soon
as you can (e.g., when the file gets closed) or keep retrying it until you
successfully purge the cache.

I find it scary that posts I made are now described as “ancient”. Perhaps
I’ve been doing this too long!

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Jagannath Krishnan [mailto:xxxxx@hotmail.com]
Sent: Monday, March 10, 2003 5:10 PM
To: File Systems Developers
Subject: [ntfsd] Memory mapped IO consistency problem

Hi,
My FSD supports memory mapped IO but not cache manager interactions or
fastIO. So Ive set FileObject->sectionObjectPointers to my
Fcb->SectionObject (duly initialized), and fileObject->fsContext to the
commonFcbHeader.

Im trying to open a text file with Notepad, save some new contents, close,
and reopen the same file. I find that when I reopen the file I get the old
contents back. However I know that the contents of the file stream have
actually been modified after the save. So this seems to be a data
consistency problem with my memory mapped IO implementation.

Specifically, the initial read of the file is a paging IO read (after my
createSection callbacks are invoked). The writes are not paging IO writes.
Finally, when the file is reopened, the FSD doesnt receive a read request
at all, indicating that the VMM satisfies this read itself. Am I missing
some step here? Why is the write not updating the memory where the file is
mapped?

In an anient post Tony says:
“The disadvantage of implementing the VMM interface without the cache
manager
is that you have a consistency “problem” that you must solve (or ignore.
That’s ONE “solution” - just ignore consistency.)”

Is this the problem Im seeing? If it is, could you elaborate a little on
this consistency problem and how to solve it?

Thanks,
Jagannath

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

3

Thanks for your reply, Tony.

As you mentioned purging the ‘cache’ wont work if the file is memory
mapped. In this case since Notepad does map the file into memory, the
contents will not be purged. Can we really afford to keep retrying the
purge? Seems like it could take a very long time before the VMM decides to
unmap the file. Is this the standard practice?

Which component is keeping the file mapping around? Notepad has been
closed, the cache manager has never been invoked, and there are no dirty
pages that the VMM needs to flush (the file is mapped when I first open it
with Notepad to read the file, so its an open-read-close by Notepad).

What Id most like to know is has anyone who has implemented a non-caching,
memory-mapped-IO-only file system come across this kind of a consistency
problem with Notepad or am I doing something wrong? Its not just Notepad,
I saw the symptoms with Word too, but debuggged it only for Notepad.

Thanks,
Jagannath