FltReadFile from file system filter driver and memory usage...

>I’d be quite surprised if you saw the ImageSectionObject used after you performed I/O to or from a file, since that’s only used for a file being mapped as an executable image (hence the name).

I am sorry, I was wrong. I double confirmed and ImageSectionObject member is not set after doing the read. However, DataSectionObject and SharedCacheMap members are set after calling FltReadFile. Doing non-cached read doesn’t set any of the members.

The DataSectionObject is used to support memory mapping of the file. This would include applications that do memory mapping (e.g., Notepad,) as well as the cache manager. The SharedCacheMap is used (along with the PrivateCacheMap in the File Object) by the Cache Manager to managed it’s mapped views of the given file. So, by not doing cached I/O on a file without a
shared cache map, you discouraged the cache manager from mapping the file.

I agree. However, doing cached reads is causing substantial increase in memory usage and apparently this memory is never returned back e.g. After running X instances of cmd.exe each consuming 20MB. I am not able launch any process on the system. So what other options do I have here? Further, why should doing cached reads result in each cmd.exe consuming 20MB vs. 2MB?

This, in turn, would limit the memory usage of the system; when you do cached I/O the memory will be used to back the cache. By doing non-cached I/O you only use the memory for your own buffer. As for “solving a problem” it would seem that your problem statement is itself fundamentally flawed - the OS uses memory for caching. You seem unhappy when it does so because you are doing cached I/O to the file.

Again I agree. However, I am still failing to understand per process memory shoot-up here.

While not doing cached I/O “fixes” this problem, in many scenarios it will do so at the cost of added I/O, which will have a different performance impact on the system. Tony OSR

Again, I agree. However, again, what other options do I have here? I am just failing to understand where is this extra per process memory going? The VMMap tool reports this extra memory being attributed to page tables. That’s too high a usage for something like page table. The working set, VAD dump doesn’t show anything increase in private bytes of the process.

Thanks.
-Prasad

>That would mean the cache has as much importance as a regular memory allocation, which, from my experience, is not the case (it can grow with more available memory, but if any memory is required, the cache manager will release its own memory to support such allocations if needed).

This is my point. If it was eventually returning back that memory, I was fine. I noticed that this memory is never returned back. e.g. After running X instances of cmd.exe each consuming 20 MB (against 2MB without the driver), I am not able to start any new processes in the system.

So, he might get increased total usage of memory, but not total usage of same-importance memory by using the cache. Also, the XP Task Manager does not show the cache use in the memory used graph. So if that graph shows 500MB are used it’s without the cache. Enabling pool tagging and seeing which tag uses the most memory should help.

I had done that before and maximum memory was attributed towards MmCm tag. We are not using MmAllocateContinguousMemory in our driver.

Thanks.
-Prasad

Hello,

I again looked at the top 11 usages by pooltag and there isn’t much difference with and without our filter driver loaded scenarios.

Without filter driver:

kd> !poolused /t 11 2
Sorting by NonPaged Pool Consumed

Pool Used:
NonPaged Paged
Tag Allocs Used Allocs Used
MmCm 3 672288 0 0 Calls made to MmAllocateContiguousMemory , Binary: nt!mm
AmlH 6 393216 0 0 ACPI AMLI Pooltags
AmlC 34 278528 0 0 ACPI AMLI Pooltags
Devi 324 240936 0 0 Device objects
Thre 328 207296 0 0 Thread objects , Binary: nt!ps
File 1347 206152 0 0 File objects
Ntf0 3 196608 726 21496 general pool allocation , Binary: ntfs.sys
AxMv 1 192512 0 0 UNKNOWN pooltag ‘AxMv’, please update pooltag.txt
HGIX 2 147456 0 0 UNKNOWN pooltag ‘HGIX’, please update pooltag.txt
AcpD 337 120200 0 0 ACPI device data , Binary: acpi.sys
Irp 323 109648 0 0 Io, IRP packets
Pool 3 98304 0 0 Pool tables, etc.
NDpp 27 86496 0 0 packet pool , Binary: ndis.sys
Ntfr 1256 80840 0 0 ERESOURCE , Binary: ntfs.sys
TCPt 24 78880 0 0 TCP/IP network protocol , Binary: TCP
Even 1592 77936 0 0 Event objects
CTGC 2 73728 0 0 UNKNOWN pooltag ‘CTGC’, please update pooltag.txt
TOTAL 15985 4725888 25283 13867000

With filter driver loaded and doing cached reads.

kd> !poolused /t 11 2
Sorting by NonPaged Pool Consumed

Pool Used:
NonPaged Paged
Tag Allocs Used Allocs Used
MmCm 3 672288 0 0 Calls made to MmAllocateContiguousMemory , Binary: nt!mm
AmlH 6 393216 0 0 ACPI AMLI Pooltags
AmlC 31 253952 0 0 ACPI AMLI Pooltags
Devi 339 246080 0 0 Device objects
Ntf0 3 196608 650 19672 general pool allocation , Binary: ntfs.sys
Thre 311 196552 0 0 Thread objects , Binary: nt!ps
AxMv 1 192512 0 0 UNKNOWN pooltag ‘AxMv’, please update pooltag.txt
File 1058 161760 0 0 File objects
HGIX 2 147456 0 0 UNKNOWN pooltag ‘HGIX’, please update pooltag.txt
AcpD 337 120200 0 0 ACPI device data , Binary: acpi.sys
Irp 310 112104 0 0 Io, IRP packets
Pool 3 98304 0 0 Pool tables, etc.
FMsl 438 91104 0 0 STREAM_LIST_CTRL structure , Binary: fltmgr.sys
VFls 420 87360 0 0 UNKNOWN pooltag ‘VFls’, please update pooltag.txt
NDpp 27 86496 0 0 packet pool , Binary: ndis.sys
TCPt 24 78880 0 0 TCP/IP network protocol , Binary: TCP
CTGC 2 73728 0 0 UNKNOWN pooltag ‘CTGC’, please update pooltag.txt
TOTAL 16024 4799104 22488 12669112

Thanks.
-Prasad

Hello,

Let me put in some memory numbers as reported by windbg and they just don’t seem reflect the *real physical memory shootup*. However, it is having *real impact* e.g. Not able to start processes anymore etc.

The system comprises of default install of 32-bit Windows XP virtual machine with 512MB of RAM and no virtual memory. After booting up, I started one instance of cmd.exe and then break into debugger and ran commands to display various pieces of memory information. Can experts on the list see if these numbers provide any pointers?

Here are various numbers without the filter driver loaded:

kd> !vm

*** Virtual Memory Usage ***
Physical Memory: 130940 ( 523760 Kb)

************ NO PAGING FILE *********************

Available Pages: 96744 ( 386976 Kb)
ResAvail Pages: 96658 ( 386632 Kb)
Locked IO Pages: 42 ( 168 Kb)
Free System PTEs: 257400 ( 1029600 Kb)
Free NP PTEs: 28139 ( 112556 Kb)
Free Special NP: 0 ( 0 Kb)
Modified Pages: 1464 ( 5856 Kb)
Modified PF Pages: 1464 ( 5856 Kb)
NonPagedPool Usage: 0 ( 0 Kb)
NonPagedPoolNx Usage: 1185 ( 4740 Kb)
NonPagedPool Max: 32768 ( 131072 Kb)
PagedPool 0 Usage: 2507 ( 10028 Kb)
PagedPool 1 Usage: 508 ( 2032 Kb)
PagedPool 2 Usage: 500 ( 2000 Kb)
PagedPool Usage: 3515 ( 14060 Kb)
PagedPool Maximum: 65536 ( 262144 Kb)
Session Commit: 121 ( 484 Kb)
Shared Commit: 1628 ( 6512 Kb)
Special Pool: 0 ( 0 Kb)
Shared Process: 1915 ( 7660 Kb)
PagedPool Commit: 3515 ( 14060 Kb)
Driver Commit: 1125 ( 4500 Kb)
Committed pages: 24517 ( 98068 Kb)
Commit limit: 122544 ( 490176 Kb)

Total Private: 16052 ( 64208 Kb)
040c svchost.exe 2912 ( 11648 Kb)
07a4 vmtoolsd.exe 1831 ( 7324 Kb)
0638 explorer.exe 1807 ( 7228 Kb)
0270 winlogon.exe 1605 ( 6420 Kb)
06a0 vmtoolsd.exe 1195 ( 4780 Kb)
02b0 lsass.exe 930 ( 3720 Kb)
0590 spoolsv.exe 926 ( 3704 Kb)
035c svchost.exe 677 ( 2708 Kb)
0118 wmiprvse.exe 664 ( 2656 Kb)
02a4 services.exe 491 ( 1964 Kb)
0784 cmd.exe 477 ( 1908 Kb)
03ac svchost.exe 431 ( 1724 Kb)
0258 csrss.exe 421 ( 1684 Kb)
0680 VMwareTray.exe 411 ( 1644 Kb)
0490 svchost.exe 398 ( 1592 Kb)
0444 svchost.exe 293 ( 1172 Kb)
0520 alg.exe 276 ( 1104 Kb)
034c vmacthlp.exe 141 ( 564 Kb)
037c wscntfy.exe 117 ( 468 Kb)
0218 smss.exe 42 ( 168 Kb)
0004 System 7 ( 28 Kb)
kd> !sysptes

System PTE Information
Total System Ptes 264690
SysPtes list of size 1 has 371 free
SysPtes list of size 2 has 87 free
SysPtes list of size 4 has 33 free
SysPtes list of size 8 has 31 free
SysPtes list of size 16 has 50 free

starting PTE: c0788000
ending PTE: c07c6f88

free blocks: 1 total free: 257400 largest free block: 232448

kd> !memusage 8

*** CacheSize too low - increasing to 25 MB

Max cache size is : 26816512 bytes (0x664c KB)
Total memory in cache : 25864 bytes (0x1a KB)
Number of regions cached: 143
337 full reads broken into 340 partial reads
counts: 196 cached/144 uncached, 57.65% cached
bytes : 45965 cached/17856 uncached, 72.02% cached
** Transition PTEs are implicitly decoded
** Prototype PTEs are implicitly decoded
loading PFN database
loading (100% complete)
Compiling memory usage data (99% Complete).
Zeroed: 85361 (341444 kb)
Free: 0 ( 0 kb)
Standby: 11383 ( 45532 kb)
Modified: 1464 ( 5856 kb)
ModifiedNoWrite: 0 ( 0 kb)
Active/Valid: 32749 (130996 kb)
Transition: 0 ( 0 kb)
Bad: 0 ( 0 kb)
Unknown: 0 ( 0 kb)
TOTAL: 130957 (523828 kb)

kd> !process 82175020
PROCESS 82175020 SessionId: 0 Cid: 0784 Peb: 7ffd5000 ParentCid: 0638
DirBase: 046c0260 ObjectTable: e1f944f8 HandleCount: 30.
Image: cmd.exe
VadRoot 82181e00 Vads 52 Clone 0 Private 130. Modified 0. Locked 0.
DeviceMap e1ab1730
Token e2054d48
ElapsedTime 00:00:11.140
UserTime 00:00:00.015
KernelTime 00:00:00.015
QuotaPoolUsage[PagedPool] 58252
QuotaPoolUsage[NonPagedPool] 2080
Working Set Sizes (now,min,max) (580, 50, 345) (2320KB, 200KB, 1380KB)
PeakWorkingSetSize 580
VirtualSize 28 Mb
PeakVirtualSize 35 Mb
PageFaultCount 611
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 477

THREAD 82174020 Cid 0784.0788 Teb: 7ffdf000 Win32Thread: e1ee16a8 WAIT: (WrLpcReply) UserMode Non-Alertable
82174214 Semaphore Limit 0x1
Waiting for reply to LPC MessageId 00006a1f:
Current LPC port e1ef0468
Not impersonating
DeviceMap e1ab1730
Owning Process 0 Image:
Attached Process 82175020 Image: cmd.exe
Wait Start TickCount 2455 Ticks: 589 (0:00:00:09.203)
Context Switch Count 92 LargeStack
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address 0x4ad05046
Start Address 0x7c8106f5
Stack Init f7498000 Current f7497c50 Base f7498000 Limit f7494000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr
f7497c68 80500cd6 nt!KiSwapContext+0x2e (FPO: [Uses EBP] [0,0,4])
f7497c74 804f9d62 nt!KiSwapThread+0x46 (FPO: [0,0,0])
f7497c9c 805986cb nt!KeWaitForSingleObject+0x1c2 (FPO: [Non-Fpo])
f7497d50 8053d638 nt!NtRequestWaitReplyPort+0x63d (FPO: [Non-Fpo])
f7497d50 7c90e4f4 nt!KiFastCallEntry+0xf8 (FPO: [0,0] TrapFrame @ f7497d64)
0013fc6c 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])

kd> !vad 82181e00
VAD level start end commit
8230c170 ( 1) 10 10 1 Private READWRITE
82181e00 ( 0) 20 20 1 Private READWRITE
821d0300 ( 4) 30 3f 5 Private READWRITE
81d87e00 ( 3) 40 13f 256 Private READWRITE
82301fa8 ( 2) 140 142 0 Mapped READONLY Pagefile-backed section
821ccd70 ( 4) 150 24f 10 Private READWRITE
81d8d308 ( 3) 250 25f 6 Private READWRITE
8203fbc8 ( 5) 260 26f 0 Mapped READWRITE Pagefile-backed section
81f6b8c8 ( 4) 270 285 0 Mapped READONLY \WINDOWS\system32\unicode.nls
822bf4a0 ( 6) 290 2d0 0 Mapped READONLY \WINDOWS\system32\locale.nls
822bf410 ( 5) 2e0 320 0 Mapped READONLY \WINDOWS\system32\sortkey.nls
8203fb38 ( 6) 330 335 0 Mapped READONLY \WINDOWS\system32\sorttbls.nls
8203f278 ( 8) 340 407 0 Mapped EXECUTE_READ Pagefile-backed section
82303230 ( 9) 410 410 1 Private READWRITE
81f6e350 (10) 420 420 1 Private READWRITE
82043748 (11) 430 431 0 Mapped READONLY Pagefile-backed section
81d82e30 (13) 440 44f 4 Private READWRITE
820437d8 (12) 450 451 0 Mapped READONLY Pagefile-backed section
82043808 (13) 460 461 0 Mapped READONLY Pagefile-backed section
822f7078 ( 7) 470 47f 8 Private READWRITE
81f65218 ( 8) 480 48f 4 Private READWRITE
81f62828 ( 9) 490 492 0 Mapped READONLY \WINDOWS\system32\ctype.nls
8203cfe8 (10) 4a0 4df 3 Private READWRITE
81f630a8 (11) 4e0 5e2 0 Mapped READONLY Pagefile-backed section
82306290 (12) 5f0 8ef 0 Mapped EXECUTE_READ Pagefile-backed section
81f62b60 (13) 8f0 8f0 1 Private READWRITE
81d8a718 ( 1) 4ad00 4ad60 30 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\cmd.exe
81f62718 ( 6) 5ad70 5ada7 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\uxtheme.dll
822b3de8 ( 5) 5cb70 5cb95 20 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\shimeng.dll
82043778 ( 7) 5d090 5d129 3 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\comctl32.dll
8203f248 ( 6) 6f880 6fa49 9 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\AppPatch\AcGenral.dll
823033c0 ( 8) 769c0 76a73 3 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\userenv.dll
82301a18 ( 7) 76b40 76b6c 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\winmm.dll
82301aa8 ( 9) 77120 771aa 4 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\oleaut32.dll
82043718 (10) 773d0 774d2 2 Mapped Exe EXECUTE_WRITECOPY

\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.5512_x-ww_35d4ce83\comctl32.dll
82301a78 ( 8) 774e0 7761c 8 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\ole32.dll
82301b18 ( 9) 77be0 77bf4 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\msacm32.dll
82301a48 (10) 77c00 77c07 1 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\version.dll
81d82d00 ( 4) 77c10 77c67 7 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\msvcrt.dll
82040330 ( 6) 77dd0 77e6a 5 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\advapi32.dll
8217ac68 ( 7) 77e70 77f01 1 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\rpcrt4.dll
822b0160 ( 5) 77f10 77f58 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\gdi32.dll
82303310 ( 7) 77f60 77fd5 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\shlwapi.dll
822b3150 ( 6) 77fe0 77ff0 1 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\secur32.dll
82301f78 ( 3) 7c800 7c8f5 5 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\kernel32.dll
821e9a28 ( 2) 7c900 7c9ae 5 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\ntdll.dll
823032e0 ( 6) 7c9c0 7d1d6 30 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\shell32.dll
81de3cd0 ( 5) 7e410 7e4a0 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\user32.dll
8203fb98 ( 4) 7f6f0 7f7ef 0 Mapped EXECUTE_READ Pagefile-backed section
81d8a688 ( 3) 7ffb0 7ffd3 0 Mapped READONLY Pagefile-backed section
823043d0 ( 4) 7ffd5 7ffd5 1 Private READWRITE
82306440 ( 5) 7ffdf 7ffdf 1 Private READWRITE

Total VADs: 52 average level: 7 maximum depth: 13

Here are various numbers with filter driver loaded doing FltReadFile for min(128K, filesize) bytes in PostOpCreate for all files. After system booted up, I was able to start only one instance of cmd.exe and then it failed to launch any process due to lack of memory.

kd> !vm

Virtual Memory Usage
Physical Memory: 130940 ( 523760 Kb)

NO PAGING FILE*********

Available Pages: 83625 ( 334500 Kb)
ResAvail Pages: 96579 ( 386316 Kb)
Locked IO Pages: 42 ( 168 Kb)
Free System PTEs: 257670 ( 1030680 Kb)
Free NP PTEs: 28139 ( 112556 Kb)
Free Special NP: 0 ( 0 Kb)
Modified Pages: 3394 ( 13576 Kb)
Modified PF Pages: 3394 ( 13576 Kb)
NonPagedPool Usage: 0 ( 0 Kb)
NonPagedPoolNx Usage: 1219 ( 4876 Kb)
NonPagedPool Max: 32768 ( 131072 Kb)
PagedPool 0 Usage: 2327 ( 9308 Kb)
PagedPool 1 Usage: 484 ( 1936 Kb)
PagedPool 2 Usage: 478 ( 1912 Kb)
PagedPool Usage: 3289 ( 13156 Kb)
PagedPool Maximum: 65536 ( 262144 Kb)
Session Commit: 121 ( 484 Kb)
Shared Commit: 1594 ( 6376 Kb)
Special Pool: 0 ( 0 Kb)
Shared Process: 1764 ( 7056 Kb)
PagedPool Commit: 3289 ( 13156 Kb)
Driver Commit: 1125 ( 4500 Kb)
Committed pages: 119883 ( 479532 Kb)
Commit limit: 122502 ( 490008 Kb)

2254 commit requests have failed

Total Private: 111604 ( 446416 Kb)
0428 svchost.exe 13978 ( 55912 Kb)
010c vmtoolsd.exe 9945 ( 39780 Kb)
0610 explorer.exe 9872 ( 39488 Kb)
027c winlogon.exe 8692 ( 34768 Kb)
01f4 wmiprvse.exe 7681 ( 30724 Kb)
02b4 lsass.exe 7581 ( 30324 Kb)
066c spoolsv.exe 7524 ( 30096 Kb)
0370 svchost.exe 7345 ( 29380 Kb)
06b4 vmtoolsd.exe 7143 ( 28572 Kb)
0510 svchost.exe 6606 ( 26424 Kb)
03c8 svchost.exe 6380 ( 25520 Kb)
06ac VMwareTray.exe 5443 ( 21772 Kb)
04b8 svchost.exe 5422 ( 21688 Kb)
0350 vmacthlp.exe 2408 ( 9632 Kb)
02a8 services.exe 2078 ( 8312 Kb)
0264 csrss.exe 1629 ( 6516 Kb)
00dc cmd.exe 1465 ( 5860 Kb)
021c smss.exe 231 ( 924 Kb)
0004 System 181 ( 724 Kb)
kd> !sysptes

System PTE Information
Total System Ptes 264690
SysPtes list of size 1 has 371 free
SysPtes list of size 2 has 87 free
SysPtes list of size 4 has 53 free
SysPtes list of size 8 has 31 free
SysPtes list of size 16 has 35 free

starting PTE: c0788000
ending PTE: c07c6f88

free blocks: 1 total free: 257670 largest free block: 232448

kd> !memusage 8

CacheSize too low - increasing to 25 MB

Max cache size is : 26816512 bytes (0x664c KB)
Total memory in cache : 21764 bytes (0x16 KB)
Number of regions cached: 116
303 full reads broken into 306 partial reads
counts: 189 cached/117 uncached, 61.76% cached
bytes : 45873 cached/15252 uncached, 75.05% cached
Transition PTEs are implicitly decoded
* Prototype PTEs are implicitly decoded
loading PFN database
loading (100% complete)
Compiling memory usage data (99% Complete).
Zeroed: 82728 (330912 kb)
Free: 0 ( 0 kb)
Standby: 897 ( 3588 kb)
Modified: 3394 ( 13576 kb)
ModifiedNoWrite: 0 ( 0 kb)
Active/Valid: 43938 (175752 kb)
Transition: 0 ( 0 kb)
Bad: 0 ( 0 kb)
Unknown: 0 ( 0 kb)
TOTAL: 130957 (523828 kb)

kd> !process 81e17be0
PROCESS 81e17be0 SessionId: 0 Cid: 00dc Peb: 7ffdf000 ParentCid: 0610
DirBase: 057801a0 ObjectTable: e1b919a8 HandleCount: 17.
Image: cmd.exe
VadRoot 8204f210 Vads 32 Clone 0 Private 60. Modified 34. Locked 0.
DeviceMap e15510b0
Token e1ac9778
ElapsedTime 00:00:19.234
UserTime 00:00:00.015
KernelTime 00:00:00.046
QuotaPoolUsage[PagedPool] 27292
QuotaPoolUsage[NonPagedPool] 1280
Working Set Sizes (now,min,max) (1220, 50, 345) (4880KB, 200KB, 1380KB)
PeakWorkingSetSize 2315
VirtualSize 13 Mb
PeakVirtualSize 21 Mb
PageFaultCount 2655
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 1465

THREAD 8202bbe8 Cid 00dc.00e0 Teb: 7ffde000 Win32Thread: e1b6a338 WAIT: (WrLpcReply) UserMode Non-Alertable
8202bddc Semaphore Limit 0x1
Waiting for reply to LPC MessageId 000066f4:
Current LPC port e1c9a5f0
Not impersonating
DeviceMap e15510b0
Owning Process 0 Image:
Attached Process 81e17be0 Image: cmd.exe
Wait Start TickCount 5450 Ticks: 1220 (0:00:00:19.062)
Context Switch Count 75 LargeStack
UserTime 00:00:00.000
KernelTime 00:00:00.046
Win32 Start Address 0x4ad05046
Start Address 0x7c8106f5
Stack Init f72f1000 Current f72f0c50 Base f72f1000 Limit f72ed000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr
f72f0c68 80500cd6 nt!KiSwapContext+0x2e (FPO: [Uses EBP] [0,0,4])
f72f0c74 804f9d62 nt!KiSwapThread+0x46 (FPO: [0,0,0])
f72f0c9c 805986cb nt!KeWaitForSingleObject+0x1c2 (FPO: [Non-Fpo])
f72f0d50 8053d638 nt!NtRequestWaitReplyPort+0x63d (FPO: [Non-Fpo])
f72f0d50 7c90e4f4 nt!KiFastCallEntry+0xf8 (FPO: [0,0] TrapFrame @ f72f0d64)
0013fc6c 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])

kd> !vad 8204f210
VAD level start end commit
820f0778 ( 1) 10 10 1 Private READWRITE
8204f210 ( 0) 20 20 1 Private READWRITE
82109fe8 ( 4) 30 3f 5 Private READWRITE
8209ae58 ( 3) 40 13f 256 Private READWRITE
820e7880 ( 2) 140 142 0 Mapped READONLY Pagefile-backed section
820f7b68 ( 4) 150 24f 5 Private READWRITE
821b5bf0 ( 3) 250 25f 6 Private READWRITE
81e1be18 ( 5) 260 26f 0 Mapped READWRITE Pagefile-backed section
822d7588 ( 4) 270 285 0 Mapped READONLY \WINDOWS\system32\unicode.nls
821d77e8 ( 6) 290 2d0 0 Mapped READONLY \WINDOWS\system32\locale.nls
820e9dd0 ( 5) 2e0 320 0 Mapped READONLY \WINDOWS\system32\sortkey.nls
82310250 ( 6) 330 335 0 Mapped READONLY \WINDOWS\system32\sorttbls.nls
820dabf8 ( 7) 340 342 0 Mapped READONLY \WINDOWS\system32\ctype.nls
820dac28 ( 8) 350 417 0 Mapped EXECUTE_READ Pagefile-backed section
81e1bda8 ( 9) 420 522 0 Mapped READONLY Pagefile-backed section
82039218 (11) 530 530 1 Private READWRITE
81e1afa8 (10) 540 83f 0 Mapped EXECUTE_READ Pagefile-backed section
8227f120 (11) 840 840 1 Private READWRITE
81e1af88 (12) 850 850 1 Private READWRITE
82318508 ( 1) 4ad00 4ad60 126 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\cmd.exe
820f3a98 ( 4) 77c10 77c67 90 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\msvcrt.dll
81e25bf0 ( 6) 77dd0 77e6a 156 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\advapi32.dll
820d7390 ( 7) 77e70 77f01 145 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\rpcrt4.dll
820f0e68 ( 5) 77f10 77f58 73 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\gdi32.dll
820dbbf8 ( 6) 77fe0 77ff0 16 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\secur32.dll
82248558 ( 3) 7c800 7c8f5 245 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\kernel32.dll
8209cd40 ( 2) 7c900 7c9ae 179 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\ntdll.dll
8230b858 ( 5) 7e410 7e4a0 144 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\user32.dll
82217d60 ( 4) 7f6f0 7f7ef 0 Mapped EXECUTE_READ Pagefile-backed section
81e21298 ( 3) 7ffb0 7ffd3 0 Mapped READONLY Pagefile-backed section
82099b88 ( 5) 7ffde 7ffde 1 Private READWRITE
81e1bdd8 ( 4) 7ffdf 7ffdf 1 Private READWRITE

Total VADs: 32 average level: 6 maximum depth: 12

Thanks.
-Prasad

Enough to explain the increase or just most of memory allocated?

If I understand correctly, the Task Manager memory use shows that almost 500MB are in use (the graph, not including the cached label)? That in turn should not count the MmCm allocations.

>So, he might get increased total usage of memory, but not total usage of same-importance memory by using the cache. Also, the XP Task Manager does not show the cache use in the memory used graph. So if that graph shows 500MB are used it’s without the cache. Enabling pool tagging and seeing which tag uses the most memory should help.

I had done that before and maximum memory was attributed towards MmCm tag. We are not using MmAllocateContinguousMemory in our driver.


Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
http://www.alfasp.com
File system audit, security and encryption kits.

The difference between these two cases is that all executables seem to be pagefile-backed when your filter is enabled:

Before: 82301f78 ( 3) 7c800 7c8f5 5 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\kernel32.dll
After: 82248558 ( 3) 7c800 7c8f5 245 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\kernel32.dll

Notice how in the second case commit has been charged for all 245 pages of the VAD, compared to only 5 pages before.

I suspect that the FltReadFile call causes a data section to be created, then when the memory manager creates an image section for a DLL it thinks there are still active data references to the file and copies all image pages to the pagefile (this is similar to what happens to images linked as /swaprun:net and launched from network).

To check if this theory is right you can run “!vad 82248558 1” and see if the FloppyMedia bit is set in the control area.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@vmware.com
Sent: Friday, September 30, 2011 4:16 AM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] FltReadFile from file system filter driver and memory usage…

Hello,

Let me put in some memory numbers as reported by windbg and they just don’t seem reflect the *real physical memory shootup*. However, it is having *real impact* e.g. Not able to start processes anymore etc.

The system comprises of default install of 32-bit Windows XP virtual machine with 512MB of RAM and no virtual memory. After booting up, I started one instance of cmd.exe and then break into debugger and ran commands to display various pieces of memory information. Can experts on the list see if these numbers provide any pointers?

Here are various numbers without the filter driver loaded:

kd> !vm

*** Virtual Memory Usage ***
Physical Memory: 130940 ( 523760 Kb)

************ NO PAGING FILE *********************

Available Pages: 96744 ( 386976 Kb)
ResAvail Pages: 96658 ( 386632 Kb)
Locked IO Pages: 42 ( 168 Kb)
Free System PTEs: 257400 ( 1029600 Kb)
Free NP PTEs: 28139 ( 112556 Kb)
Free Special NP: 0 ( 0 Kb)
Modified Pages: 1464 ( 5856 Kb)
Modified PF Pages: 1464 ( 5856 Kb)
NonPagedPool Usage: 0 ( 0 Kb)
NonPagedPoolNx Usage: 1185 ( 4740 Kb)
NonPagedPool Max: 32768 ( 131072 Kb)
PagedPool 0 Usage: 2507 ( 10028 Kb)
PagedPool 1 Usage: 508 ( 2032 Kb)
PagedPool 2 Usage: 500 ( 2000 Kb)
PagedPool Usage: 3515 ( 14060 Kb)
PagedPool Maximum: 65536 ( 262144 Kb)
Session Commit: 121 ( 484 Kb)
Shared Commit: 1628 ( 6512 Kb)
Special Pool: 0 ( 0 Kb)
Shared Process: 1915 ( 7660 Kb)
PagedPool Commit: 3515 ( 14060 Kb)
Driver Commit: 1125 ( 4500 Kb)
Committed pages: 24517 ( 98068 Kb)
Commit limit: 122544 ( 490176 Kb)

Total Private: 16052 ( 64208 Kb)
040c svchost.exe 2912 ( 11648 Kb)
07a4 vmtoolsd.exe 1831 ( 7324 Kb)
0638 explorer.exe 1807 ( 7228 Kb)
0270 winlogon.exe 1605 ( 6420 Kb)
06a0 vmtoolsd.exe 1195 ( 4780 Kb)
02b0 lsass.exe 930 ( 3720 Kb)
0590 spoolsv.exe 926 ( 3704 Kb)
035c svchost.exe 677 ( 2708 Kb)
0118 wmiprvse.exe 664 ( 2656 Kb)
02a4 services.exe 491 ( 1964 Kb)
0784 cmd.exe 477 ( 1908 Kb)
03ac svchost.exe 431 ( 1724 Kb)
0258 csrss.exe 421 ( 1684 Kb)
0680 VMwareTray.exe 411 ( 1644 Kb)
0490 svchost.exe 398 ( 1592 Kb)
0444 svchost.exe 293 ( 1172 Kb)
0520 alg.exe 276 ( 1104 Kb)
034c vmacthlp.exe 141 ( 564 Kb)
037c wscntfy.exe 117 ( 468 Kb)
0218 smss.exe 42 ( 168 Kb)
0004 System 7 ( 28 Kb)
kd> !sysptes

System PTE Information
Total System Ptes 264690
SysPtes list of size 1 has 371 free
SysPtes list of size 2 has 87 free
SysPtes list of size 4 has 33 free
SysPtes list of size 8 has 31 free
SysPtes list of size 16 has 50 free

starting PTE: c0788000
ending PTE: c07c6f88

free blocks: 1 total free: 257400 largest free block: 232448

kd> !memusage 8

*** CacheSize too low - increasing to 25 MB

Max cache size is : 26816512 bytes (0x664c KB)
Total memory in cache : 25864 bytes (0x1a KB)
Number of regions cached: 143
337 full reads broken into 340 partial reads
counts: 196 cached/144 uncached, 57.65% cached
bytes : 45965 cached/17856 uncached, 72.02% cached
** Transition PTEs are implicitly decoded
** Prototype PTEs are implicitly decoded loading PFN database loading (100% complete) Compiling memory usage data (99% Complete).
Zeroed: 85361 (341444 kb)
Free: 0 ( 0 kb)
Standby: 11383 ( 45532 kb)
Modified: 1464 ( 5856 kb)
ModifiedNoWrite: 0 ( 0 kb)
Active/Valid: 32749 (130996 kb)
Transition: 0 ( 0 kb)
Bad: 0 ( 0 kb)
Unknown: 0 ( 0 kb)
TOTAL: 130957 (523828 kb)

kd> !process 82175020
PROCESS 82175020 SessionId: 0 Cid: 0784 Peb: 7ffd5000 ParentCid: 0638
DirBase: 046c0260 ObjectTable: e1f944f8 HandleCount: 30.
Image: cmd.exe
VadRoot 82181e00 Vads 52 Clone 0 Private 130. Modified 0. Locked 0.
DeviceMap e1ab1730
Token e2054d48
ElapsedTime 00:00:11.140
UserTime 00:00:00.015
KernelTime 00:00:00.015
QuotaPoolUsage[PagedPool] 58252
QuotaPoolUsage[NonPagedPool] 2080
Working Set Sizes (now,min,max) (580, 50, 345) (2320KB, 200KB, 1380KB)
PeakWorkingSetSize 580
VirtualSize 28 Mb
PeakVirtualSize 35 Mb
PageFaultCount 611
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 477

THREAD 82174020 Cid 0784.0788 Teb: 7ffdf000 Win32Thread: e1ee16a8 WAIT: (WrLpcReply) UserMode Non-Alertable
82174214 Semaphore Limit 0x1
Waiting for reply to LPC MessageId 00006a1f:
Current LPC port e1ef0468
Not impersonating
DeviceMap e1ab1730
Owning Process 0 Image:
Attached Process 82175020 Image: cmd.exe
Wait Start TickCount 2455 Ticks: 589 (0:00:00:09.203)
Context Switch Count 92 LargeStack
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address 0x4ad05046
Start Address 0x7c8106f5
Stack Init f7498000 Current f7497c50 Base f7498000 Limit f7494000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr
f7497c68 80500cd6 nt!KiSwapContext+0x2e (FPO: [Uses EBP] [0,0,4])
f7497c74 804f9d62 nt!KiSwapThread+0x46 (FPO: [0,0,0])
f7497c9c 805986cb nt!KeWaitForSingleObject+0x1c2 (FPO: [Non-Fpo])
f7497d50 8053d638 nt!NtRequestWaitReplyPort+0x63d (FPO: [Non-Fpo])
f7497d50 7c90e4f4 nt!KiFastCallEntry+0xf8 (FPO: [0,0] TrapFrame @ f7497d64)
0013fc6c 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])

kd> !vad 82181e00
VAD level start end commit
8230c170 ( 1) 10 10 1 Private READWRITE
82181e00 ( 0) 20 20 1 Private READWRITE
821d0300 ( 4) 30 3f 5 Private READWRITE
81d87e00 ( 3) 40 13f 256 Private READWRITE
82301fa8 ( 2) 140 142 0 Mapped READONLY Pagefile-backed section
821ccd70 ( 4) 150 24f 10 Private READWRITE
81d8d308 ( 3) 250 25f 6 Private READWRITE
8203fbc8 ( 5) 260 26f 0 Mapped READWRITE Pagefile-backed section
81f6b8c8 ( 4) 270 285 0 Mapped READONLY \WINDOWS\system32\unicode.nls
822bf4a0 ( 6) 290 2d0 0 Mapped READONLY \WINDOWS\system32\locale.nls
822bf410 ( 5) 2e0 320 0 Mapped READONLY \WINDOWS\system32\sortkey.nls
8203fb38 ( 6) 330 335 0 Mapped READONLY \WINDOWS\system32\sorttbls.nls
8203f278 ( 8) 340 407 0 Mapped EXECUTE_READ Pagefile-backed section
82303230 ( 9) 410 410 1 Private READWRITE
81f6e350 (10) 420 420 1 Private READWRITE
82043748 (11) 430 431 0 Mapped READONLY Pagefile-backed section
81d82e30 (13) 440 44f 4 Private READWRITE
820437d8 (12) 450 451 0 Mapped READONLY Pagefile-backed section
82043808 (13) 460 461 0 Mapped READONLY Pagefile-backed section
822f7078 ( 7) 470 47f 8 Private READWRITE
81f65218 ( 8) 480 48f 4 Private READWRITE
81f62828 ( 9) 490 492 0 Mapped READONLY \WINDOWS\system32\ctype.nls
8203cfe8 (10) 4a0 4df 3 Private READWRITE
81f630a8 (11) 4e0 5e2 0 Mapped READONLY Pagefile-backed section
82306290 (12) 5f0 8ef 0 Mapped EXECUTE_READ Pagefile-backed section
81f62b60 (13) 8f0 8f0 1 Private READWRITE
81d8a718 ( 1) 4ad00 4ad60 30 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\cmd.exe
81f62718 ( 6) 5ad70 5ada7 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\uxtheme.dll
822b3de8 ( 5) 5cb70 5cb95 20 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\shimeng.dll
82043778 ( 7) 5d090 5d129 3 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\comctl32.dll
8203f248 ( 6) 6f880 6fa49 9 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\AppPatch\AcGenral.dll
823033c0 ( 8) 769c0 76a73 3 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\userenv.dll
82301a18 ( 7) 76b40 76b6c 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\winmm.dll
82301aa8 ( 9) 77120 771aa 4 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\oleaut32.dll
82043718 (10) 773d0 774d2 2 Mapped Exe EXECUTE_WRITECOPY

\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.5512_x-ww_35d4ce83\comctl32.dll
82301a78 ( 8) 774e0 7761c 8 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\ole32.dll
82301b18 ( 9) 77be0 77bf4 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\msacm32.dll
82301a48 (10) 77c00 77c07 1 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\version.dll
81d82d00 ( 4) 77c10 77c67 7 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\msvcrt.dll
82040330 ( 6) 77dd0 77e6a 5 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\advapi32.dll
8217ac68 ( 7) 77e70 77f01 1 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\rpcrt4.dll
822b0160 ( 5) 77f10 77f58 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\gdi32.dll
82303310 ( 7) 77f60 77fd5 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\shlwapi.dll
822b3150 ( 6) 77fe0 77ff0 1 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\secur32.dll
82301f78 ( 3) 7c800 7c8f5 5 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\kernel32.dll
821e9a28 ( 2) 7c900 7c9ae 5 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\ntdll.dll
823032e0 ( 6) 7c9c0 7d1d6 30 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\shell32.dll
81de3cd0 ( 5) 7e410 7e4a0 2 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\user32.dll
8203fb98 ( 4) 7f6f0 7f7ef 0 Mapped EXECUTE_READ Pagefile-backed section
81d8a688 ( 3) 7ffb0 7ffd3 0 Mapped READONLY Pagefile-backed section
823043d0 ( 4) 7ffd5 7ffd5 1 Private READWRITE
82306440 ( 5) 7ffdf 7ffdf 1 Private READWRITE

Total VADs: 52 average level: 7 maximum depth: 13

Here are various numbers with filter driver loaded doing FltReadFile for min(128K, filesize) bytes in PostOpCreate for all files. After system booted up, I was able to start only one instance of cmd.exe and then it failed to launch any process due to lack of memory.

kd> !vm

Virtual Memory Usage
Physical Memory: 130940 ( 523760 Kb)

NO PAGING FILE*********

Available Pages: 83625 ( 334500 Kb)
ResAvail Pages: 96579 ( 386316 Kb)
Locked IO Pages: 42 ( 168 Kb)
Free System PTEs: 257670 ( 1030680 Kb)
Free NP PTEs: 28139 ( 112556 Kb)
Free Special NP: 0 ( 0 Kb)
Modified Pages: 3394 ( 13576 Kb)
Modified PF Pages: 3394 ( 13576 Kb)
NonPagedPool Usage: 0 ( 0 Kb)
NonPagedPoolNx Usage: 1219 ( 4876 Kb)
NonPagedPool Max: 32768 ( 131072 Kb)
PagedPool 0 Usage: 2327 ( 9308 Kb)
PagedPool 1 Usage: 484 ( 1936 Kb)
PagedPool 2 Usage: 478 ( 1912 Kb)
PagedPool Usage: 3289 ( 13156 Kb)
PagedPool Maximum: 65536 ( 262144 Kb)
Session Commit: 121 ( 484 Kb)
Shared Commit: 1594 ( 6376 Kb)
Special Pool: 0 ( 0 Kb)
Shared Process: 1764 ( 7056 Kb)
PagedPool Commit: 3289 ( 13156 Kb)
Driver Commit: 1125 ( 4500 Kb)
Committed pages: 119883 ( 479532 Kb)
Commit limit: 122502 ( 490008 Kb)

2254 commit requests have failed

Total Private: 111604 ( 446416 Kb)
0428 svchost.exe 13978 ( 55912 Kb)
010c vmtoolsd.exe 9945 ( 39780 Kb)
0610 explorer.exe 9872 ( 39488 Kb)
027c winlogon.exe 8692 ( 34768 Kb)
01f4 wmiprvse.exe 7681 ( 30724 Kb)
02b4 lsass.exe 7581 ( 30324 Kb)
066c spoolsv.exe 7524 ( 30096 Kb)
0370 svchost.exe 7345 ( 29380 Kb)
06b4 vmtoolsd.exe 7143 ( 28572 Kb)
0510 svchost.exe 6606 ( 26424 Kb)
03c8 svchost.exe 6380 ( 25520 Kb)
06ac VMwareTray.exe 5443 ( 21772 Kb)
04b8 svchost.exe 5422 ( 21688 Kb)
0350 vmacthlp.exe 2408 ( 9632 Kb)
02a8 services.exe 2078 ( 8312 Kb)
0264 csrss.exe 1629 ( 6516 Kb)
00dc cmd.exe 1465 ( 5860 Kb)
021c smss.exe 231 ( 924 Kb)
0004 System 181 ( 724 Kb)
kd> !sysptes

System PTE Information
Total System Ptes 264690
SysPtes list of size 1 has 371 free
SysPtes list of size 2 has 87 free
SysPtes list of size 4 has 53 free
SysPtes list of size 8 has 31 free
SysPtes list of size 16 has 35 free

starting PTE: c0788000
ending PTE: c07c6f88

free blocks: 1 total free: 257670 largest free block: 232448

kd> !memusage 8

CacheSize too low - increasing to 25 MB

Max cache size is : 26816512 bytes (0x664c KB)
Total memory in cache : 21764 bytes (0x16 KB)
Number of regions cached: 116
303 full reads broken into 306 partial reads
counts: 189 cached/117 uncached, 61.76% cached
bytes : 45873 cached/15252 uncached, 75.05% cached
Transition PTEs are implicitly decoded
* Prototype PTEs are implicitly decoded loading PFN database loading (100% complete) Compiling memory usage data (99% Complete).
Zeroed: 82728 (330912 kb)
Free: 0 ( 0 kb)
Standby: 897 ( 3588 kb)
Modified: 3394 ( 13576 kb)
ModifiedNoWrite: 0 ( 0 kb)
Active/Valid: 43938 (175752 kb)
Transition: 0 ( 0 kb)
Bad: 0 ( 0 kb)
Unknown: 0 ( 0 kb)
TOTAL: 130957 (523828 kb)

kd> !process 81e17be0
PROCESS 81e17be0 SessionId: 0 Cid: 00dc Peb: 7ffdf000 ParentCid: 0610
DirBase: 057801a0 ObjectTable: e1b919a8 HandleCount: 17.
Image: cmd.exe
VadRoot 8204f210 Vads 32 Clone 0 Private 60. Modified 34. Locked 0.
DeviceMap e15510b0
Token e1ac9778
ElapsedTime 00:00:19.234
UserTime 00:00:00.015
KernelTime 00:00:00.046
QuotaPoolUsage[PagedPool] 27292
QuotaPoolUsage[NonPagedPool] 1280
Working Set Sizes (now,min,max) (1220, 50, 345) (4880KB, 200KB, 1380KB)
PeakWorkingSetSize 2315
VirtualSize 13 Mb
PeakVirtualSize 21 Mb
PageFaultCount 2655
MemoryPriority BACKGROUND
BasePriority 8
CommitCharge 1465

THREAD 8202bbe8 Cid 00dc.00e0 Teb: 7ffde000 Win32Thread: e1b6a338 WAIT: (WrLpcReply) UserMode Non-Alertable
8202bddc Semaphore Limit 0x1
Waiting for reply to LPC MessageId 000066f4:
Current LPC port e1c9a5f0
Not impersonating
DeviceMap e15510b0
Owning Process 0 Image:
Attached Process 81e17be0 Image: cmd.exe
Wait Start TickCount 5450 Ticks: 1220 (0:00:00:19.062)
Context Switch Count 75 LargeStack
UserTime 00:00:00.000
KernelTime 00:00:00.046
Win32 Start Address 0x4ad05046
Start Address 0x7c8106f5
Stack Init f72f1000 Current f72f0c50 Base f72f1000 Limit f72ed000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr
f72f0c68 80500cd6 nt!KiSwapContext+0x2e (FPO: [Uses EBP] [0,0,4])
f72f0c74 804f9d62 nt!KiSwapThread+0x46 (FPO: [0,0,0])
f72f0c9c 805986cb nt!KeWaitForSingleObject+0x1c2 (FPO: [Non-Fpo])
f72f0d50 8053d638 nt!NtRequestWaitReplyPort+0x63d (FPO: [Non-Fpo])
f72f0d50 7c90e4f4 nt!KiFastCallEntry+0xf8 (FPO: [0,0] TrapFrame @ f72f0d64)
0013fc6c 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])

kd> !vad 8204f210
VAD level start end commit
820f0778 ( 1) 10 10 1 Private READWRITE
8204f210 ( 0) 20 20 1 Private READWRITE
82109fe8 ( 4) 30 3f 5 Private READWRITE
8209ae58 ( 3) 40 13f 256 Private READWRITE
820e7880 ( 2) 140 142 0 Mapped READONLY Pagefile-backed section
820f7b68 ( 4) 150 24f 5 Private READWRITE
821b5bf0 ( 3) 250 25f 6 Private READWRITE
81e1be18 ( 5) 260 26f 0 Mapped READWRITE Pagefile-backed section
822d7588 ( 4) 270 285 0 Mapped READONLY \WINDOWS\system32\unicode.nls
821d77e8 ( 6) 290 2d0 0 Mapped READONLY \WINDOWS\system32\locale.nls
820e9dd0 ( 5) 2e0 320 0 Mapped READONLY \WINDOWS\system32\sortkey.nls
82310250 ( 6) 330 335 0 Mapped READONLY \WINDOWS\system32\sorttbls.nls
820dabf8 ( 7) 340 342 0 Mapped READONLY \WINDOWS\system32\ctype.nls
820dac28 ( 8) 350 417 0 Mapped EXECUTE_READ Pagefile-backed section
81e1bda8 ( 9) 420 522 0 Mapped READONLY Pagefile-backed section
82039218 (11) 530 530 1 Private READWRITE
81e1afa8 (10) 540 83f 0 Mapped EXECUTE_READ Pagefile-backed section
8227f120 (11) 840 840 1 Private READWRITE
81e1af88 (12) 850 850 1 Private READWRITE
82318508 ( 1) 4ad00 4ad60 126 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\cmd.exe
820f3a98 ( 4) 77c10 77c67 90 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\msvcrt.dll
81e25bf0 ( 6) 77dd0 77e6a 156 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\advapi32.dll
820d7390 ( 7) 77e70 77f01 145 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\rpcrt4.dll
820f0e68 ( 5) 77f10 77f58 73 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\gdi32.dll
820dbbf8 ( 6) 77fe0 77ff0 16 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\secur32.dll
82248558 ( 3) 7c800 7c8f5 245 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\kernel32.dll
8209cd40 ( 2) 7c900 7c9ae 179 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\ntdll.dll
8230b858 ( 5) 7e410 7e4a0 144 Mapped Exe EXECUTE_WRITECOPY \WINDOWS\system32\user32.dll
82217d60 ( 4) 7f6f0 7f7ef 0 Mapped EXECUTE_READ Pagefile-backed section
81e21298 ( 3) 7ffb0 7ffd3 0 Mapped READONLY Pagefile-backed section
82099b88 ( 5) 7ffde 7ffde 1 Private READWRITE
81e1bdd8 ( 4) 7ffdf 7ffdf 1 Private READWRITE

Total VADs: 32 average level: 6 maximum depth: 12

Thanks.
-Prasad


NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

@pavel, is this because I am calling FltReadFile even before image is mapped?

Also, I don’t have any pagefile. Hence it will be backed by extra physical RAM?

In any case why should this cause spike on per process basis when private bytes of the processes arr not going up?

I am travelling ATM and only have limited internet access on cellphone. I will try the windbg ommand you mention once i am back on Monday.

-Prasad.

Sent from Smartphone. Excuse typos.

@Dejan, I am sure if I understand your question. I have given various memory numbers with and without driver scenarios and was hoping experts on the list to explain the shoot up and potential underlying causes.

On other note, if you notice, in the state where i was not able to start process due to lack og memory the numbers show there was ample memory. Look at zero list.

-Prasad

.
Sent from smartphone. Excuse typos.

Yep, I looked at the memory output after I sent the reply… it’s not the pool tags, something else is causing committed pages.
I don’t have any idea here.

xxxxx@vmware.com wrote:

@Dejan, I am sure if I understand your question. I have given various memory numbers with and without driver scenarios and was hoping experts on the list to explain the shoot up and potential underlying causes.

On other note, if you notice, in the state where i was not able to start process due to lack og memory the numbers show there was ample memory. Look at zero list.

-Prasad

.
Sent from smartphone. Excuse typos.


Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
http://www.alfasp.com
File system audit, security and encryption kits.

> is this because I am calling FltReadFile even before image is mapped?

I think what matters is whether the file has an active data section. After calling FltReadFile you can set a breakpoint on MiFlushDataSection (which takes file object as a parameter) and dump the file object’s data control area (File->SectionObjectPointer->DataSectionObject) using !ca. If it has any section references or mapped views it will cause the image to be backed by pagefile. This assumes you’re using XP, things are a bit different on other releases.

Also, I don’t have any pagefile. Hence it will be backed by extra physical RAM?

Yes.

In any case why should this cause spike on per process basis when private bytes of the processes are not going up?

Private bytes (aka commit charge) are in fact going up, as confirmed by both !vm and !vad output. This is the reason you’re getting memory allocation failures despite having lots of available physical memory.

@Pavel, I had used vmmap tool from sysinternals to check the breakup of the commit charge for the processes and most of the extra memory was reported to come from “shared” working set and hence I am not sure how can this be private? At the same time, the tool was reporting most of that extra memory attributed towards “Page tables” memory type which is also mysterious since Page tables just cannot take that much of memory.

Even if the private bytes of the processes are going up, why should starting a new process fail when there is ample physical memory?

Thanks.
-Prasad

Hello Pavel,

You were absolutely correct with your analysis! With my filter driver loaded, FloppyMedia bit is indeed set. I am attaching relevant windbg output later in this post for one of the core file ntdll.dll.

Given this, I have few follow-up questions.

  1. What is this FloppyMedia bit? Is there any documentation on this? I searched google and cannot find anything relevant.
  2. When FloppyMedia bit is set, does this mean that all the processes are getting separate physical pages for all the shared DLLs viz. kernel32.dll, ntdll.dll etc and hence causing per process memory shoot-up? The various counters and tools seem to be misleading in this regard e.g. vmmap tool reports that the shared DLL pages as shared whereas actually they are not. Instead, vmmap reports the additional memory attributed towards page tables.
  3. The file object (ntdll in this case) on which I am making FltReadFile call is opened in readonly mode. Hence, there is no question of file being modified. Given this, why does Windows still give a copy of the pages for all processes and not share it?
  4. I observed this behavior on Windows XP and Windows 2003. However, on Vista, I don't see FloppyMedia bit being set and hence I don't see the memory shoot up either. Does this mean, that, this is an issue which is fixed in Vista and above?
  5. What options do I have to solve this issue? I can think of following options.
    a. Always do a non-cached read. I believe this will have performance impact and would like to avoid it.
    b. Do a cached read only if the image is mapped. I can probably do this by checking SectionObjectPointers->ImageSectionObject != NULL? However, it won't buy me much because I need to read the file only on first PostOpCreate for the file and not subsequently. Hence, effectively, I will end up doing non-cached read for all image files.
    c. Is there a way to detect in advance if the file being opened is a image file apart from looking at file extension and if so map it as image before initiating a read on it?
    d. Can I use FltReadFile with FLTFL_IO_OPERATION_PAGING|FLTFL_IO_OPERATION_SYNCHRONOUS_PAGING? This also seems to solve the memory shoot up issue.

Below is the windbg output for ntdll.dll.

Without my filter driver loaded, the relevant windbg output shows the following.

kd> !memusage
...
...
Control Valid Standby Dirty Shared Locked PageTables name
...
...
85e13280 0 4 0 0 0 0 mapped_file( ntdll.dll )
8619f2f8 340 420 0 316 0 0 mapped_file( ntdll.dll )
...
...

kd> !ca 85e13280

ControlArea @ 85e13280
Segment e1539598 Flink 85c80eac Blink 85e13974
Section Ref 0 Pfn Ref 1 Mapped Views 0
User Ref 0 WaitForDel 0 Flush Count 0
File Object 85e13310 ModWriteCount 0 System Views 0
WritableRefs 0
Flags (80) File

File: \WINDOWS\system32\ntdll.dll

Segment @ e1539598
Type nt!_MAPPED_FILE_SEGMENT not found.

kd> !ca 8619f2f8

ControlArea @ 8619f2f8
Segment e1008188 Flink 00000000 Blink 00000000
Section Ref 1 Pfn Ref be Mapped Views 1b
User Ref 1c WaitForDel 0 Flush Count 0
File Object 86010b00 ModWriteCount 0 System Views 0
WritableRefs 0
Flags (81000a0) Image File DebugSymbolsLoaded Accessed

File: \WINDOWS\system32\ntdll.dll

Segment @ e1008188
ControlArea 8619f2f8 BasedAddress 7c800000
Total Ptes c0
WriteUserRef 0 SizeOfSegment c0000
Committed 0 PTE Template 8619f3b000000420
Based Addr 7c800000 Image Base 0
Image Commit 6 Image Info e10087c8
ProtoPtes e10081c8

Subsection 1 @ 8619f330
ControlArea 8619f2f8 Starting Sector 0 Number Of Sectors 2
Base Pte e10081c8 Ptes In Subsect 1 Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

Subsection 2 @ 8619f350
ControlArea 8619f2f8 Starting Sector 2 Number Of Sectors 42e
Base Pte e10081d0 Ptes In Subsect 86 Unused Ptes 0
Flags 31 Sector Offset 0 Protection 3

Subsection 3 @ 8619f370
ControlArea 8619f2f8 Starting Sector 430 Number Of Sectors 1b
Base Pte e1008600 Ptes In Subsect 6 Unused Ptes 0
Flags 51 Sector Offset 0 Protection 5

Subsection 4 @ 8619f390
ControlArea 8619f2f8 Starting Sector 44b Number Of Sectors 172
Base Pte e1008630 Ptes In Subsect 2f Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

Subsection 5 @ 8619f3b0
ControlArea 8619f2f8 Starting Sector 5bd Number Of Sectors 1a
Base Pte e10087a8 Ptes In Subsect 4 Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

With my filter driver loaded, the relevant windbg output shows the following.

kd> !memusage
...
...
Control Valid Standby Dirty Shared Locked PageTables name
...
...

861a06c8 760 0 0 748 0 0 mapped_file( ntdll.dll )
85c5aca0 0 252 0 0 0 0 mapped_file( ntdll.dll )
...
...

kd> !ca 861a06c8

ControlArea @ 861a06c8
Segment e1437990 Flink 00000000 Blink 00000000
Section Ref 1 Pfn Ref be Mapped Views 1a
User Ref 1b WaitForDel 0 Flush Count 0
File Object 8605a858 ModWriteCount 0 System Views 0
WritableRefs 0
Flags (81040a0) Image File FloppyMedia DebugSymbolsLoaded Accessed

File: \WINDOWS\system32\ntdll.dll

Segment @ e1437990
ControlArea 861a06c8 BasedAddress 7c800000
Total Ptes c0
WriteUserRef 0 SizeOfSegment c0000
Committed 0 PTE Template 861a078000000420
Based Addr 7c800000 Image Base 0
Image Commit 6 Image Info e1437fd0
ProtoPtes e14379d0

Subsection 1 @ 861a0700
ControlArea 861a06c8 Starting Sector 0 Number Of Sectors 2
Base Pte e14379d0 Ptes In Subsect 1 Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

Subsection 2 @ 861a0720
ControlArea 861a06c8 Starting Sector 2 Number Of Sectors 42e
Base Pte e14379d8 Ptes In Subsect 86 Unused Ptes 0
Flags 31 Sector Offset 0 Protection 3

Subsection 3 @ 861a0740
ControlArea 861a06c8 Starting Sector 430 Number Of Sectors 1b
Base Pte e1437e08 Ptes In Subsect 6 Unused Ptes 0
Flags 51 Sector Offset 0 Protection 5

Subsection 4 @ 861a0760
ControlArea 861a06c8 Starting Sector 44b Number Of Sectors 172
Base Pte e1437e38 Ptes In Subsect 2f Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

Subsection 5 @ 861a0780
ControlArea 861a06c8 Starting Sector 5bd Number Of Sectors 1a
Base Pte e1437fb0 Ptes In Subsect 4 Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

kd> !ca 85c5aca0

ControlArea @ 85c5aca0
Segment e14053c8 Flink 85f23c7c Blink 85d9077c
Section Ref 0 Pfn Ref 3f Mapped Views 0
User Ref 0 WaitForDel 0 Flush Count 0
File Object 8605a858 ModWriteCount 0 System Views 0
WritableRefs 0
Flags (8008080) File WasPurged Accessed

File: \WINDOWS\system32\ntdll.dll

Segment @ e14053c8
Type nt!_MAPPED_FILE_SEGMENT not found.

Thanks.
-Prasad

Hello Pavel and others,

Do you have any suggestions on my earlier post? I would like to avoid doing non-cached reads as far as possible.

I confirmed Pavel’s theory again by deeper inspection of the code from the debugger on Windows XP SP3.

  1. MiCreateImageFileMap is the function where the FloppyMedia bit is getting set depending upon result of MiFlushDataSection. Here are the snippets.

MiCreateImageFileMap

8059f984 ff7508 push dword ptr [ebp+8]
8059f987 e828a2f6ff call nt!MiFlushDataSection (80509bb4)

8059f990 8945d0 mov dword ptr [ebp-30h],eax

8059fd4b 33c9 xor ecx,ecx
8059fd4d 41 inc ecx

8059fd72 394dd0 cmp dword ptr [ebp-30h],ecx
8059fd75 894220 mov dword ptr [edx+20h],eax
8059fd78 7438 je nt!MiCreateImageFileMap+0x4dc (8059fdb2)

8059fdb2 8b45f8 mov eax,dword ptr [ebp-8]
8059fdb5 80482140 or byte ptr [eax+21h],40h

  1. MiMapViewOfImageSection is the function where the FloppyMedia bit is getting checked and if it’s set the pages are marked dirty using MiSetPageModified. I think, this is resulting in separate physical pages being allocated to all processes for all the DLLs?

805a66d4 f6422140 test byte ptr [edx+21h],40h
805a66d8 0f849d000000 je nt!MiMapViewOfImageSection+0x3b1 (805a677b) [br=0]

805a6743 ff75fc push dword ptr [ebp-4]
805a6746 56 push esi
805a6747 e8a4e2f6ff call nt!MiSetPageModified (805149f0)

On Vista (where I don’t see the memory shootup) I don’t see MiCreateImageFileMap setting the FloppyMedia bit.

Thanks.
-Prasad

Did you measure any performance impact when doing non-cached reads ? Do you read the file multiple times or just once (or few times)? The reason i’m asking is because in a previous message you said “I am reading 128K bytes/filesize bytes whichever is lower for all the files”. If you only read 128k max and you only read once then the cache manager won’t help.

Moreover, because you’re reading executables as data (and creating a data section for them) it’s unlikely that there is anyone else in the system that is benefiting from the cache being set up and so the overhead of initializing caching for the file won’t amortize over time. Basically, using cached IO can even be detrimental to performance if you’re the only one doing it and if you’re only accessing each file only few times. If you have a very specific access pattern or only touch a limited range of the file you might be better off just implementing your own caching.

Thanks,
Alex.

Hi Alex,

Thanks for your comments.

I haven’t measured the impact, however, it seems logical to believe that there will be performance impact?

Yes, I do read 128K/filesize bytes to begin with, however, subsequent access pattern on the file is similar to any AV solution and it is possible that same file regions may be read again at a later time.
The files (e.g. shared DLLs) that I have read will be used by O/S as well right? Hence, if I cached read ntdll, won’t it benefit the O/S as it executes ntdll code subsequently?

Thanks.
-Prasad

On XP the answer is no. A data section and an image section for the same file have different page contents because images have different in-memory layout. So accessing the same image through a data section and then through an image section does twice the amount of IO and puts two differently formatted copies of image pages into memory.

On Vista and later, the memory manager has an optimization that can often avoid disk IO when somebody tries to fault an image page from disk and corresponding data page(s) are already resident. But there will still be two copies of the image in memory, which is wasteful.

FloppyMedia is a bit that means the entire image needs to be copied to pagefile-backed memory, instead of being backed by the image file on disk. Its original purpose was to allow programs on CDs or floppies to run after their media has been ejected. You can also link the program with /swaprun:net option and the system will copy the image to the pagefile when it is run from network.

In your case the image is getting copied to the pagefile because the memory manager thinks there might be user writeable references to the image file (e.g. writable handles, sections or mapped views), and it wants to avoid the possibility of an executable being modified after it's been loaded. If the file is actually opened read-only, Vista and win7 can detect that and avoid the copy, but I believe XP can't do that because it lacks some of the required plumbing between filesystems and the memory manager.

Executables backed by the pagefile should still share physical pages between processes (unless a process tries to modify a page, in which case it will get a private copy).

I agree with Alex and others who suggested using non-cached IO here. It seems like the simplest and safest approach.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@vmware.com
Sent: Tuesday, October 04, 2011 4:15 AM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] FltReadFile from file system filter driver and memory usage...

Hello Pavel,

You were absolutely correct with your analysis! With my filter driver loaded, FloppyMedia bit is indeed set. I am attaching relevant windbg output later in this post for one of the core file ntdll.dll.

Given this, I have few follow-up questions.

  1. What is this FloppyMedia bit? Is there any documentation on this? I searched google and cannot find anything relevant.
  2. When FloppyMedia bit is set, does this mean that all the processes are getting separate physical pages for all the shared DLLs viz. kernel32.dll, ntdll.dll etc and hence causing per process memory shoot-up? The various counters and tools seem to be misleading in this regard e.g. vmmap tool reports that the shared DLL pages as shared whereas actually they are not. Instead, vmmap reports the additional memory attributed towards page tables.
  3. The file object (ntdll in this case) on which I am making FltReadFile call is opened in readonly mode. Hence, there is no question of file being modified. Given this, why does Windows still give a copy of the pages for all processes and not share it?
  4. I observed this behavior on Windows XP and Windows 2003. However, on Vista, I don't see FloppyMedia bit being set and hence I don't see the memory shoot up either. Does this mean, that, this is an issue which is fixed in Vista and above?
  5. What options do I have to solve this issue? I can think of following options.
    a. Always do a non-cached read. I believe this will have performance impact and would like to avoid it.
    b. Do a cached read only if the image is mapped. I can probably do this by checking SectionObjectPointers->ImageSectionObject != NULL? However, it won't buy me much because I need to read the file only on first PostOpCreate for the file and not subsequently. Hence, effectively, I will end up doing non-cached read for all image files.
    c. Is there a way to detect in advance if the file being opened is a image file apart from looking at file extension and if so map it as image before initiating a read on it?
    d. Can I use FltReadFile with FLTFL_IO_OPERATION_PAGING|FLTFL_IO_OPERATION_SYNCHRONOUS_PAGING? This also seems to solve the memory shoot up issue.

Below is the windbg output for ntdll.dll.

Without my filter driver loaded, the relevant windbg output shows the following.

kd> !memusage
...
...
Control Valid Standby Dirty Shared Locked PageTables name ...
...
85e13280 0 4 0 0 0 0 mapped_file( ntdll.dll )
8619f2f8 340 420 0 316 0 0 mapped_file( ntdll.dll )
...
...

kd> !ca 85e13280

ControlArea @ 85e13280
Segment e1539598 Flink 85c80eac Blink 85e13974
Section Ref 0 Pfn Ref 1 Mapped Views 0
User Ref 0 WaitForDel 0 Flush Count 0
File Object 85e13310 ModWriteCount 0 System Views 0
WritableRefs 0
Flags (80) File

File: \WINDOWS\system32\ntdll.dll

Segment @ e1539598
Type nt!_MAPPED_FILE_SEGMENT not found.

kd> !ca 8619f2f8

ControlArea @ 8619f2f8
Segment e1008188 Flink 00000000 Blink 00000000
Section Ref 1 Pfn Ref be Mapped Views 1b
User Ref 1c WaitForDel 0 Flush Count 0
File Object 86010b00 ModWriteCount 0 System Views 0
WritableRefs 0
Flags (81000a0) Image File DebugSymbolsLoaded Accessed

File: \WINDOWS\system32\ntdll.dll

Segment @ e1008188
ControlArea 8619f2f8 BasedAddress 7c800000
Total Ptes c0
WriteUserRef 0 SizeOfSegment c0000
Committed 0 PTE Template 8619f3b000000420
Based Addr 7c800000 Image Base 0
Image Commit 6 Image Info e10087c8
ProtoPtes e10081c8

Subsection 1 @ 8619f330
ControlArea 8619f2f8 Starting Sector 0 Number Of Sectors 2
Base Pte e10081c8 Ptes In Subsect 1 Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

Subsection 2 @ 8619f350
ControlArea 8619f2f8 Starting Sector 2 Number Of Sectors 42e
Base Pte e10081d0 Ptes In Subsect 86 Unused Ptes 0
Flags 31 Sector Offset 0 Protection 3

Subsection 3 @ 8619f370
ControlArea 8619f2f8 Starting Sector 430 Number Of Sectors 1b
Base Pte e1008600 Ptes In Subsect 6 Unused Ptes 0
Flags 51 Sector Offset 0 Protection 5

Subsection 4 @ 8619f390
ControlArea 8619f2f8 Starting Sector 44b Number Of Sectors 172
Base Pte e1008630 Ptes In Subsect 2f Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

Subsection 5 @ 8619f3b0
ControlArea 8619f2f8 Starting Sector 5bd Number Of Sectors 1a
Base Pte e10087a8 Ptes In Subsect 4 Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

With my filter driver loaded, the relevant windbg output shows the following.

kd> !memusage
...
...
Control Valid Standby Dirty Shared Locked PageTables name ...
...

861a06c8 760 0 0 748 0 0 mapped_file( ntdll.dll )
85c5aca0 0 252 0 0 0 0 mapped_file( ntdll.dll )
...
...

kd> !ca 861a06c8

ControlArea @ 861a06c8
Segment e1437990 Flink 00000000 Blink 00000000
Section Ref 1 Pfn Ref be Mapped Views 1a
User Ref 1b WaitForDel 0 Flush Count 0
File Object 8605a858 ModWriteCount 0 System Views 0
WritableRefs 0
Flags (81040a0) Image File FloppyMedia DebugSymbolsLoaded Accessed

File: \WINDOWS\system32\ntdll.dll

Segment @ e1437990
ControlArea 861a06c8 BasedAddress 7c800000
Total Ptes c0
WriteUserRef 0 SizeOfSegment c0000
Committed 0 PTE Template 861a078000000420
Based Addr 7c800000 Image Base 0
Image Commit 6 Image Info e1437fd0
ProtoPtes e14379d0

Subsection 1 @ 861a0700
ControlArea 861a06c8 Starting Sector 0 Number Of Sectors 2
Base Pte e14379d0 Ptes In Subsect 1 Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

Subsection 2 @ 861a0720
ControlArea 861a06c8 Starting Sector 2 Number Of Sectors 42e
Base Pte e14379d8 Ptes In Subsect 86 Unused Ptes 0
Flags 31 Sector Offset 0 Protection 3

Subsection 3 @ 861a0740
ControlArea 861a06c8 Starting Sector 430 Number Of Sectors 1b
Base Pte e1437e08 Ptes In Subsect 6 Unused Ptes 0
Flags 51 Sector Offset 0 Protection 5

Subsection 4 @ 861a0760
ControlArea 861a06c8 Starting Sector 44b Number Of Sectors 172
Base Pte e1437e38 Ptes In Subsect 2f Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

Subsection 5 @ 861a0780
ControlArea 861a06c8 Starting Sector 5bd Number Of Sectors 1a
Base Pte e1437fb0 Ptes In Subsect 4 Unused Ptes 0
Flags 11 Sector Offset 0 Protection 1

kd> !ca 85c5aca0

ControlArea @ 85c5aca0
Segment e14053c8 Flink 85f23c7c Blink 85d9077c
Section Ref 0 Pfn Ref 3f Mapped Views 0
User Ref 0 WaitForDel 0 Flush Count 0
File Object 8605a858 ModWriteCount 0 System Views 0
WritableRefs 0
Flags (8008080) File WasPurged Accessed

File: \WINDOWS\system32\ntdll.dll

Segment @ e14053c8
Type nt!_MAPPED_FILE_SEGMENT not found.

Thanks.
-Prasad


NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:

To unsubscribe, visit the List Server section of OSR Online at ListServer/Forum

As I mentioned in another reply, most of the physical pages should still be shared (and backed by the pagefile) in this case so VMMap and other tools will count them as part of the shared working set.

However, VM usage (commit charge) and physical usage (working set) are orthogonal concepts. You can have pages that are shared and at the same time charged against process commit. One example of this is a regular copy-on-write mapping. Something similar happens in your case: all processes share the same pagefile-backed pages, but each process gets charged separately for the image commit. (As far as I can tell, this is an artifact of XP’s implementation… Vista and later charge commit once when the image section is converted to pagefile backing, rather than charging it for every process that maps the image).

I agree this is strange. Might be a VMMap issue.

Private bytes is the same thing as process commit charge, so when they grow the total system commit charge also increases. When system commit approaches the limit, memory allocations begin to fail even if there is still plenty of physical memory available. For a more detailed explanation of how memory commitment works in Windows, watch this video by Mark Russinovich:

http://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/WCL405

Discussion of system commit starts around 57:50.

Hello Pavel,

Thanks for the clarifications. Your and everybody’s inputs have been extremely helpful in resolving this mystery. So, essentially, the summary of the issue is:

  1. All the image files are getting backed by physical memory (since I have disabled page file for experimentation) instead of the disk image.
  2. Although, the pages are shared across processes, they are charged on per process basis and are also added up to system commit charge.
  3. When system commit charge reaches the size of physical memory (since I have disabled page file for experimentation), the new allocations fail despite of having ample free physical memory.

Now, I have last two questions on the topic.

  1. Can I use paged reads instead of non-cached reads?
  2. Can I do non-cached reads for pre-vista Windows flavors and do cached reads otherwise?

Thanks.
-Prasad

Hello,

I eventually implemented following solution for this issue.

For Windows XP/2003, in PostOpCreate, I check if FO->SectionObject->ImageSectionObject is non-NULL. If so, I perform cached read, otherwise I perform non-cached read. If I am end up doing non-cached reads on the initial region of the file, I parse those bytes to see if it has IMAGE_DOS_HEADER and IMAGE_NT_HEADER (match MZ and PE signatures). If it doesn’t have those
signatures, I mark the file as a data file in the stream context. In PostOpCreate, I also check this flag to decide whether to do cached/non-cached reads. Basically, I do cached reads if either ImageSectionObject is non-NULL OR streamcontext says that it’s a data file. Does anybody see any issue with this approach?

I am also going to perform some performance experiments in following scenarios and then choose the final approach.

  1. Without the fix for memory shootup.
  2. With the solution that does cached/non-cached reads depending upon if image section is mapped OR file is identified as data file in the stream context.
  3. With the solution that always does non-cached reads.

Thanks.
-Prasad