Regarding Memory mapped files

I am writing a sample file system driver, I need to identify when a file is getting mapped and unmapped. When a file is mapped, memory manager creates a section object of which file system is not aware of. But, access to the mapped memory results in a page fault. So, read request comes to a file system driver.
Here, in IRP_MJ_READ path, Can I check for DataSectionObject and ImageSectionObject in SECTION_OBJECT_POINTERS for NONULL values, to conclude file is mapped ?
In the IRP_MJ_CLOSE path, if DataSectionObject or ImageSectionObject are NULL, can I conclude the file has been unmapped ?

I ran notepad but observed that, IRP_MJ_CLOSE for file object which mapped the file never got called, even on shutdown. Does memory manager release its reference on file object only if memory becomes low and need to purge some ? Before releasing the reference, does memory manager NULL out DataSectionObject and ImageSectionObject ?

I’m not certain if this will solve all of your problems, but for file objects created while your filter is running, you can be notified of section object creation by filtering they pseudo-IRP IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION watching for SyncType==SyncTypeCreateSection.

See http://msdn.microsoft.com/en-us/library/ms795401.aspx for details.

Thanks for the reply.

I am writing a sample file system not a file system filter. FsRtlRegisterFileSystemFilterCallbacks can be called from only file system filter. I cant use FsRtlRegisterFileSystemFilterCallbacks .

Any suggestions and explanations will be highly appreciated

xxxxx@gmail.com wrote:

Thanks for the reply.

I am writing a sample file system not a file system filter. FsRtlRegisterFileSystemFilterCallbacks can be called from only file system filter. I cant use FsRtlRegisterFileSystemFilterCallbacks .

Any suggestions and explanations will be highly appreciated

But in your fast IO dispatch table you setup callbacks to be made when
sections are initialized.

Pete


Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

> I am writing a sample file system not a file system filter.

FsRtlRegisterFileSystemFilterCallbacks can be called
from only file system filter. I cant use
FsRtlRegisterFileSystemFilterCallbacks .

You can (check recent FAT sources)

Indeed I’d say that would be well advised to do so. You can get away with
doing it the old way, but you get more information by doing it this way.
ISTR that there are other good reasons to do this…

R

You have AcquireFileForNtCreateSection(sp?) &c in your fastio dispatch - see
“Life in the fast lane” http://www.osronline.com/article.cfm?id=166 and
consult fastat - good luck!

wrote in message news:xxxxx@ntfsd…
>I am writing a sample file system driver, I need to identify when a file is
>getting mapped and unmapped. When a file is mapped, memory manager creates
>a section object of which file system is not aware of. But, access to the
>mapped memory results in a page fault. So, read request comes to a file
>system driver.
> Here, in IRP_MJ_READ path, Can I check for DataSectionObject and
> ImageSectionObject in SECTION_OBJECT_POINTERS for NONULL values, to
> conclude file is mapped ?
> In the IRP_MJ_CLOSE path, if DataSectionObject or ImageSectionObject are
> NULL, can I conclude the file has been unmapped ?
>
> I ran notepad but observed that, IRP_MJ_CLOSE for file object which mapped
> the file never got called, even on shutdown. Does memory manager release
> its reference on file object only if memory becomes low and need to purge
> some ? Before releasing the reference, does memory manager NULL out
> DataSectionObject and ImageSectionObject ?
>

Actually, it may be better to use
FsRtlRegisterFileSystemFilterCallbacks.

Ignore the name - this call CAN (and is) used by file systems, as it
provides greater control than the fast I/O entry point (the fast I/O
entry point is VOID and poorly specified.)

The Memory Manager will only create a single “data section object”
(which is really a control area these days) per section object pointers
structure. Thus, when someone calls NtCreateSection (or one of the
variants of this) to create a data section, they either get back a
pointer to the existing data section or a new one is created.

If a new one is to be created, the memory manager must lock down the
file object for the duration of the creation - otherwise, the state of
the file (notably it’s sizes) might change while the section is itself
being set up (it was the discovery of this race condition/timing windows
that ultimately led to creation of the fast I/O entry point, in fact.)
This is why you observe the fast I/O callback or the filter callback -
they are triggered when the memory manager locks down the file while it
sets up the section.

You can also use MmDoesFileHaveUserWriteableReferences (to determine if
the file is mapped writeable,) CcIsFileCached (to determine if the file
is mapped by the cache manager,) and CcPurgeCacheSection (which fails if
the file is memory mapped as a data section.) But the presence of
values in the SectionObjectPointers structure really isn’t sufficient to
determine if the file is memory mapped by a user.

Note: CcPurgeCacheSection is ONLY safe for the owner of the file object
to use (normally the file system.) It is NEVER safe for a filter to
invoke this function, no matter that it will work “most of the time”.

It would be unusual for the memory manager to never release its
reference on a file, but it is always possible. Normally, it will go
away as a result of memory pressure; I seriously doubt that it is
forcibly discarded during shutdown (dirty pages are written back
however.) But, I must admit I’m often mystified at the shutdown process
in Windows these days (I’ve waited as long as 12 hours for my Win 7 box
to shutdown while it warns me about the dire consequences of powering
off prematurely. I view it as being my own personal manifestation of
the halting problem.)

Good luck.

Tony
OSR

Thanks for replying.

I ran notepad but observed that, IRP_MJ_CLOSE for file object which mapped the
file never got called, even on shutdown. Does memory manager release its
reference on file object only if memory becomes low and need to purge some ?
Before releasing the reference, does memory manager NULL out DataSectionObject
and ImageSectionObject ?

When I tried unmounting the filesystem rather shutting the system down, IRP_MJ_CLOSE got triggered.

_SECTION_OBJECT_POINTERS before calling CcPurgeCacheSection

kd> dt _SECTION_OBJECT_POINTERS 0x822d98d8
nt!_SECTION_OBJECT_POINTERS
+0x000 DataSectionObject : 0x822c0008
+0x004 SharedCacheMap : (null)
+0x008 ImageSectionObject : (null)

kd> kb 100
ChildEBP RetAddr Args to Child
b9f6168c ba889a6d 819f0d18 e130c1b8 e12ad980 Fastfat!FatCommonClose [c:\winddk\6001.18000\src\filesys\fastfat\wnet\close.c @ 836] b9f6171c 809cc57d 819f0c20 834c2e48 82328498 Fastfat!FatFsdClose+0x17d [c:\winddk\6001.18000\src\filesys\fastfat\wnet\close.c @ 177] b9f6174c 80853648 f731fd36 b9f61784 f731fd36 nt!IovCallDriver+0x112
b9f61758 f731fd36 00000000 82328498 80a6ea90 nt!IofCallDriver+0x13
b9f61784 809cc57d 82328498 834c2e48 00000000 fltMgr!FltpDispatch+0x152
b9f617b4 80853648 f731fb43 b9f617e4 f731fb43 nt!IovCallDriver+0x112 b9f617c0 f731fb43 82334e40 834c2e48 8232f550 nt!IofCallDriver+0x13
b9f617e4 f731fd03 b9f61804 82334e40 00000000 fltMgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x20b
b9f6181c 809cc57d 82334e40 834c2e48 834c2e48 fltMgr!FltpDispatch+0x11f b9f6184c 80853648 8090afbf b9f61890 8090afbf nt!IovCallDriver+0x112
b9f61858 8090afbf 80a6ea90 81f0e4c8 81f0e4d8 nt!IofCallDriver+0x13 b9f61890 80904834 00f0e4f0 81f0e4d8 00000000 nt!IopDeleteFile+0x13a
b9f618a8 80828af5 81f0e4f0 00000000 822c0008 nt!ObpRemoveObjectRoutine+0xde
b9f618c8 80835847 822c0040 822c0008 00000000 nt!ObfDereferenceObject+0x67
b9f618f4 808283ce e1982ca8 822c0040 81049f68 nt!MiSegmentDelete+0xba
b9f61914 8082e987 822c0008 822d9800 00000000 nt!MiCheckControlArea+0x198
b9f61954 8082e7fd 822d9800 01000000 822c0040 nt!MmPurgeSection+0x57e
b9f61984 ba8a3374 822d98d8 00000000 00000000 nt!CcPurgeCacheSection+0xd4
b9f619e8 ba8a3156 81f36f90 e130c1b8 00000001 Fastfat!FatForceCacheMiss+0x204 [c:\winddk\6001.18000\src\filesys\fastfat\wnet\filobsup.c @ 507]
b9f61a08 ba87a587 81f36f90 e130c1b8 00000001 Fastfat!FatPurgeReferencedFileObjects+0x86 [c:\winddk\6001.18000\src\filesys\fastfat\wnet\filobsup.c @ 371] b9f61a30 ba8a6ea7 81f36f90 819f0d18 822fb348 Fastfat!FatLockVolumeInternal+0xb7 [c:\winddk\6001.18000\src\filesys\fastfat\wnet\fsctrl.c @ 3066]
b9f61a84 ba8a68f1 81f36f90 82976e48 00000018 Fastfat!FatLockVolume+0x107 [c:\winddk\6001.18000\src\filesys\fastfat\wnet\fsctrl.c @ 2884]
b9f61aa4 ba8a49fb 81f36f90 82976e48 822fb300 Fastfat!FatUserFsCtrl+0x91 [c:\winddk\6001.18000\src\filesys\fastfat\wnet\fsctrl.c @ 2561] b9f61ac0 ba8a4946 81f36f90 82976e48 0371eaa2 Fastfat!FatCommonFileSystemControl+0x3b [c:\winddk\6001.18000\src\filesys\fastfat\wnet\fsctrl.c @ 666] b9f61b10 809cc57d 819f0c20 82976e48 82328498 Fastfat!FatFsdFileSystemControl+0xe6 [c:\winddk\6001.18000\src\filesys\fastfat\wnet\fsctrl.c @ 590] b9f61b40 80853648 f732d693 b9f61b78 f732d693 nt!IovCallDriver+0x112 b9f61b4c f732d693 00000000 80a6ea90 82328498 nt!IofCallDriver+0x13
b9f61b78 809cc57d 82328498 82976e48 00000000 fltMgr!FltpFsControl+0xd7
b9f61ba8 80853648 f731fb43 b9f61bd8 f731fb43 nt!IovCallDriver+0x112
b9f61bb4 f731fb43 82334e40 82976e48 00000000 nt!IofCallDriver+0x13
b9f61bd8 f732d676 b9f61bf8 82334e40 00000000 fltMgr!FltpLegacyProcessingAfterPreCallbacksCompleted+0x20b
b9f61c10 809cc57d 82334e40 82976e48 82976e48 fltMgr!FltpFsControl+0xba b9f61c40 80853648 8090b989 b9f61c60 8090b989 nt!IovCallDriver+0x112 b9f61c4c 8090b989 82976fd8 822fb348 82976e48 nt!IofCallDriver+0x13 b9f61c60 8090eb17 82334e40 82976e48 822fb348 nt!IopSynchronousServiceTail+0x10b
b9f61d00 8092cdcb 000007c8 00000000 00000000 nt!IopXxxControlFile+0x605
b9f61d34 8082337b 000007c8 00000000 00000000 nt!NtFsControlFile+0x2a
b9f61d34 7c82ed54 000007c8 00000000 00000000 nt!KiFastCallEntry+0xf8
WARNING: Frame IP not in any known module. Following frames may be wrong.
0007f6b0 0100199f 000007c8 00090018 00000000 0x7c82ed54
0007f8f4 01001c09 00000030 0007ff4c 00000000 0x100199f 0007f990 0007fa10 7c82f680 7c832b88 ffffffff 0x1001c09
0007f994 7c82f680 7c832b88 ffffffff 7c832b7f 0x7fa10 0007fa10 7c82f680 7c836cd0 ffffffff 7c836b56 0x7c82f680
0007fa78 77e6b798 77e665d0 ffffffff 77e665cb 0x7c82f680 0007ffd0 808207bc 0007ffc8 82424818 ffffffff 0x77e6b798
0007ffd4 0007ffc8 82424818 ffffffff 77e6b798 nt!KiSwapContext+0x25 00000000 00000000 00000000 00000000 00000000 0x7ffc8

_SECTION_OBJECT_POINTERS in FatCommonClose

kd> dt _SECTION_OBJECT_POINTERS 0x822d98d8
nt!_SECTION_OBJECT_POINTERS
+0x000 DataSectionObject : 0x822c0008
+0x004 SharedCacheMap : (null)
+0x008 ImageSectionObject : (null)

I observed that CcPurgeCacheSection causes close to be called to release memory manager reference on the fileobject. It NULL out the DataSectionObject pointer. Can someone help me understanding how CcPurgeCacheSection works ?

One more question, is there any way to decide whether a paging IO is generated by Memory manager or cache manager ? Any guidelines regarding the same ?

Sorry typo in earlier message. All members of _SECTION_OBJECT_POINTERS were NULL in FatCommonClose

_SECTION_OBJECT_POINTERS in FatCommonClose

kd> dt _SECTION_OBJECT_POINTERS 0x822d98d8
nt!_SECTION_OBJECT_POINTERS
+0x000 DataSectionObject : (null)
+0x004 SharedCacheMap : (null)
+0x008 ImageSectionObject : (null)

“You can also use MmDoesFileHaveUserWriteableReferences…”

MmDoesFileHaveUserWriteableReferences? Wow - this seems awfully useful. Are there any MSFT lurkers on the list who can provide additional information on this API?

I’m not an MSFT lurker, but I’ll tell you that this API is not available on Windows XP and Server 03 which significantly limits its usefulness. And that the function name is actually MmDoesFileHaveUserWritableReferences

" And that the function name is actually MmDoesFileHaveUserWritableReferences"

That explains why I couldn’t find any references to it. MSDN documents it here:

http://msdn.microsoft.com/en-us/library/bb742874.aspx

Thanks, Sergei.

Thanks for replying.