Hi,
I’m working on a file system driver and have a question about behavior
differences between Windows 2003 and Windows 2008.
I have a test program that maps a file, makes changes to the file, and
does not exit for some time.
If the file system driver issues CcFlushCache() for that file after the
changes are made, but before the program exits running on Windows 2003 I
do get paging I/O writes for the changed pages directly from the thread
issuing the CcFlushCache() via:
fltMgr!FltpDispatch+0x1c2
nt!IoSynchronousPageWrite+0x199
nt!MiFlushSectionInternal+0xdb8
nt!MmFlushSection+0x303
nt!CcFlushCache+0x323
If the test program later calls msync() or FlushViewOfFile(), I get
paging I/O writes for the same pages again via:
fltMgr!FltpDispatch+0x1c2
nt!IoSynchronousPageWrite+0x199
nt!MiFlushSectionInternal+0xdb8
nt!MmFlushVirtualMemory+0x883
nt!NtFlushVirtualMemory+0x116
Running the same program on Windows 2008 the CcFlushCache from the file
system driver does not write back the dirty pages. The pages are
written back by the msync() (via NtFlushVirtualMemory), or by the
program exit, but I can’t figure out how to force the dirty pages to be
written by the file system driver.
It appears that for Windows 2008 R2 the new
CcCoherencyFlushAndPurgeCache(CC_FLUSH_AND_PURGE_NO_PURGE) will provide
the behavior I need, although I haven’t tried it yet.
Is there anything I can do in a file system driver to get mapped dirty
pages written to disk prior to or without any calls to
msync()/FlushViewOfFile() by the application program in Windows 2008?
I’d like to get the same behavior as the 2003 CcFlushCache() with or
without the duplicate writes later.
Thanks for your help!
Anne
> Running the same program on Windows 2008 the CcFlushCache from the file
system driver does not write back the dirty pages.
Just to check - you are sure that the writes are not coming down in the
context of the LazyWriter? Going back to XP and previously FAT has always
taken steps to syncrhonize with it (although I have given up trying to work
out what the comments mean and if I have to I rely on the
performance-destroying CcWaitForCurrentLazyWriterActivity).
It appears that for Windows 2008 R2 the new
CcCoherencyFlushAndPurgeCache(CC_FLUSH_AND_PURGE_NO_PURGE) will provide
the behavior I need, although I haven’t tried it yet.
I’ve not looked at it, but I’d agree that it looks as though that might
work. OTOH I’d want to test pretty heavily to make sure that they had
sorted out the Collided Page Fault deadlock.
Rod
–
Rod Widdowson
Consulting Partner
Steading System Software LLP
+44 1368 850217 +1 508 915 4790
Hi Rod,
Thanks for your response! I’m setting a breakpoint in the filesystem’s IRP_MJ_WRITE handler routine, and it would be fine to get the writes from the LazyWriter or MappedPageWriter. My test program maps the file and makes changes, then sleeps for 10 minutes. During that 10 minutes the filesystem does a CcFlushCache() call for that file and in 2003 I hit the breakpoint and see the paging I/O writes. Running on 2008 I hit no breakpoint until the program exits or calls msync()/FlushViewOfFile() after the 10 minutes.
I have the impression that in 2008 the CcFlushCache() doesn’t see the dirty pages until NtFlushVirtualMemory()->MmFlushVirtualMemory()->MiFlushDirtyBitsToPfn(). However, that same squence of calls happens for the msync() on 2003, and the CcFlushCache() works fine prior to the user-space flush or exit.
So, I was hoping to find out that there’s some other call like the new CcCoherencyFlushAndPurgeCache() that I can use with 2008.
Anne
Anne,
My apologies - I had missed that your app has the file mapped as a section.
I must admit that I’m surprised that CcFlush used to do the right thing.
What you need of course is MmFlush, but its neither documented, nor in the
header files. I have never tried it, but maybe ZwFlushVirtualMemory will
help. But if CcCoherencyFlushAndPurgeCache(CC_FLUSH_AND_PURGE_NO_PURGE)
does it for you why not try that? But like I say be careful of the collided
page fault deadlock.
If you get it working I’d be interested in your findings.
Rod
wrote in message news:xxxxx@ntfsd…
> Hi Rod,
> Thanks for your response! I’m setting a breakpoint in the filesystem’s
> IRP_MJ_WRITE handler routine, and it would be fine to get the writes from
> the LazyWriter or MappedPageWriter. My test program maps the file and
> makes changes, then sleeps for 10 minutes. During that 10 minutes the
> filesystem does a CcFlushCache() call for that file and in 2003 I hit the
> breakpoint and see the paging I/O writes. Running on 2008 I hit no
> breakpoint until the program exits or calls msync()/FlushViewOfFile()
> after the 10 minutes.
>
> I have the impression that in 2008 the CcFlushCache() doesn’t see the
> dirty pages until
> NtFlushVirtualMemory()->MmFlushVirtualMemory()->MiFlushDirtyBitsToPfn().
> However, that same squence of calls happens for the msync() on 2003, and
> the CcFlushCache() works fine prior to the user-space flush or exit.
>
> So, I was hoping to find out that there’s some other call like the new
> CcCoherencyFlushAndPurgeCache() that I can use with 2008.
>
> Anne
>
>
>
Hi Rod,
Thanks again! By MmFlush do you mean MmFlushImageSection()? I have tried that, but it made no difference as my mapped file only has a DataSectionObject and no ImageSectionObject.
I haven’t tried to use ZwFlushVirtualMemory(). I would need the ProcessHandle for the mapper which I might be able to record in the FastIo AcquireFileForNtCreateSection call, but I only get a FileHandle parameter so I don’t know the virtual address and size of the mapped region in the test program’s address space. The CcFlushCache() issued by the filesystem is running in another process executing an ioctl().
Unfortunately, CcCoherencyFlushAndPurgeCache(CC_FLUSH_AND_PURGE_NO_PURGE) is only in 2008 R2 and I’m trying to port to Windows 2008, so it’s not available.
A collided page fault is a fault by another thread while I/O is in progress for an earlier fault? Can you point me to information on the deadlock?
Thanks for all your help!
Anne
> Thanks again! By MmFlush do you mean MmFlushImageSection()? I have tried
that,
but it made no difference as my mapped file only has a DataSectionObject
and no
ImageSectionObject.
No, there is an internal-to-the-kernel function called MmFlush (or perhaps
MmFlushSection) which actually does the work for Cc.
I haven’t tried to use ZwFlushVirtualMemory(). I would need the
ProcessHandle for the mapper which I might be able to record in the FastIo
AcquireFileForNtCreateSection call, but I only get a FileHandle parameter
so I don’t know the virtual address and size of the mapped region in the
test program’s address space. The CcFlushCache() issued by the filesystem
is running in another process executing an ioctl().
Or you could map another section to the file (I assume that this is “just
one” file and this is an occassional activity) in kernel mode and send the
flush to that. Remember - dirtiness is a function of the PFN, not the PTE…
A collided page fault is a fault by another thread while I/O is in
progress for an earlier fault?
Can you point me to information on the deadlock?
The canonical case is
- CcCoherencyFlushAndPurgeCache - says “have to have the file locked
exclusively” so. I assume this means the paging lock.
- Grab the paging lock Ex
- Meantime another thread takes a pagefault on the file. That pagefault
hits your paging read code and you grab the paging lock
- Original thread calls CcFlush
- This happens upon the page in transition. In CcFlush it will then go
into a collided page fault wait until the thread in (3) fulfils the read.
So you deadlock.
So either I have misinterpreted the instructions about the exclusive lock in
(1) or the code in CcCoherencyFlushAndPurgeCache ignores collided page
faults. Or something.
Which is why I suggest that you test it heavily and see if you can work out
from the black box what the behavior is…
Rod