Hi Lyndon,
Indeed, my concern is that rather large number of dirty pages (more
dirty than active, in fact) which suggests that for some reason they are
not being written back. I have found locked dirty pages in precisely
this situation before - where the reference count is elevated.
While I understand the explanation you gave of the lock/unlock sequence,
it’s clear that *something* is not properly being unlocked in this case
as well - that’s quite clear from the other information. If this is a
reproducible case, have you added logic to count the number of
lock/unlock operations to ensure they match up?
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Lyndon J Clarke
Sent: Sunday, April 17, 2005 1:08 PM
To: ntfsd redirect
Subject: Re:[ntfsd] Re:fcb and [non] page pool
Hi Tony
Oh well you see I do know this file; I do happen to know the name but it
is
unimportant. I dont keep filenames in non-paged pool anywhere; the
filter
keeps filenames in paged pool.
Please can you expand on the “dirty count”? Do you mean the 260140KB
Dirty
in the !memusage output?
Regards your guess which is much appreciated the !memusage shows with
respect to this file just 1924KB Locked. This is more or less what I
would
expect since that code as written is supposed to have no more than 33 *
64KB
locked at the same time. So are you saying there can be pages locked
down
which !memusage does not count as Locked?
The file in question is mapped into the address space of a companion
service
process at virtual address BaseAddress and is used as a linear array of
64KB
data items. This how the code as written “locks” one such data item
described by a descriptor pointer p (this code executes in the context
of
the companion service process) …
p->Size = 64 * 1024;
p->UserAddress = BaseAddress + Index * Size;
p->Mdl = IoAllocateMdl(p->UserAddress, p->Size, FALSE, FALSE, NULL);
MmInitializeMdl(p->Mdl, p->UserAddress, Size);
MmProbeAndLockPages(p->Mdl, KernelMode, IoWriteAccess);
p->SystemAddress = MmGetSystemAddressForMdlSafe(p->Mdl,
HighPagePriority);
This is how the code as written “unlocks” one such data item described
by a
descriptor pointer p (some of this code executes in arbitrary context
and
some of the code executes in the context of the companion service
process as
indicated) …
MmUnlockPages(p->Mdl); // executes in arbitrary context
IoFreeMdl (p->Mdl); // executes in context of companion service
process
I have monitored the Mdl Flags field of all of the descriptors which
still
have a valid Mdl and these always have the expected values; ie both
MDL_MAPPED_TO_SYSTEM_VA and MDL_PAGES_LOCKED are set when the mdl is
expected to be “locked” and neither bit is set when the mdl is expected
to
be “unlocked”.
Cheers
Lyndon
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Lyndon,
Well, it is clear to see who the culprit is. Are you doing anything
with that particular file (81721f90) elsewhere in your filter? The dirty
count is clearly off the charts and is probably why everyone’s trying to
get these pages to go away.
My guess: someone has an MDL locking down these pages and is not
unlocking them. That will lead to this condition and would show exactly
this state. The key then is to figure out what this file is (you don’t
happen to store the file name away in any of your own data structures,
perhaps in a chunk of non-paged pool?)
Hope that helps.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Lyndon J Clarke
Sent: Sunday, April 17, 2005 9:55 AM
To: ntfsd redirect
Subject: Re:[ntfsd] Re:fcb and [non] page pool
Hi Tony
First of all here is the modified page writer … I couldnt get much out
of
him … I put the mapped page writer stack in the O/P …
THREAD 81dfa200 Cid 0004.0060 Teb: 00000000 Win32Thread:
00000000
WAIT: (WrPageOut) KernelMode Non-Alertable
8057fc28 NotificationEvent
81dfa278 NotificationTimer
Not impersonating
DeviceMap e1003928
Owning Process 81dff818 Image: System
Wait Start TickCount 1327449 Ticks: 2
(0:00:00:00.031)
Context Switch Count 1845453
UserTime 00:00:00.0000
KernelTime 00:04:22.0765
Start Address nt!MiModifiedPageWriter (0x8069c730)
Stack Init f891b000 Current f891accc Base f891b000 Limit
f8918000
Call 0
Priority 17 BasePriority 8 PriorityDecrement 0
ChildEBP RetAddr Args to Child
f891ace4 804edb2b 81dfa2a0 81dfa200 8057fc28
nt!KiSwapContext+0x26
(FPO: [EBP 0xf891ad0c] [0,0,4])
f891ad0c 804ed790 807490a0 00000000 00000000
nt!KiSwapThread+0x280
(FPO: [Non-Fpo])
f891ad40 8050e5ad 8057fc28 00000013 00000000
nt!KeWaitForSingleObject+0x249 (FPO: [Non-Fpo])
f891ad78 8069c8a7 81dfa200 00000000 00000018
nt!MiModifiedPageWriterWorker+0x1a8 (FPO: [EBP 0xf891adac] [0,4,0])
f891adac 80596ffe 00000000 00000000 00000000
nt!MiModifiedPageWriter+0x177 (FPO: [Non-Fpo])
f891addc 805008c6 8069c730 00000000 00000000
nt!PspSystemThreadStartup+0x2e (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16
… and then here is !vm … am running iostress in hct11.2 …
1: kd> !vm
Virtual Memory Usage
Physical Memory: 130816 ( 523264 Kb)
Paging File Name paged out
Current: 1479372Kb Free Space: 1398672Kb
Minimum: 786432Kb Maximum: 1572864Kb
Available Pages: 93 ( 372 Kb)
ResAvail Pages: 10836 ( 43344 Kb)
Locked IO Pages: 869 ( 3476 Kb)
Free System PTEs: 38182 ( 152728 Kb)
Free NP PTEs: 28646 ( 114584 Kb)
Free Special NP: 12080 ( 48320 Kb)
Modified Pages: 65562 ( 262248 Kb)
Modified PF Pages: 0 ( 0 Kb)
NonPagedPool Usage: 2872 ( 11488 Kb)
NonPagedPool Max: 32768 ( 131072 Kb)
PagedPool 0 Usage: 1447 ( 5788 Kb)
PagedPool 1 Usage: 394 ( 1576 Kb)
PagedPool 2 Usage: 396 ( 1584 Kb)
PagedPool 3 Usage: 386 ( 1544 Kb)
PagedPool 4 Usage: 400 ( 1600 Kb)
PagedPool Usage: 3023 ( 12092 Kb)
PagedPool Maximum: 50176 ( 200704 Kb)
Shared Commit: 777 ( 3108 Kb)
Special Pool: 1443 ( 5772 Kb)
Shared Process: 2257 ( 9028 Kb)
PagedPool Commit: 3023 ( 12092 Kb)
Driver Commit: 1687 ( 6748 Kb)
Committed pages: 121407 ( 485628 Kb)
Commit limit: 494294 ( 1977176 Kb)
5 commit requests have failed
Total Private: 110120 ( 440480 Kb)
0fc8 eatres.exe 65545 ( 262180 Kb)
0950 eatres.exe 19716 ( 78864 Kb)
0488 svchost.exe 3103 ( 12412 Kb)
025c winlogon.exe 2946 ( 11784 Kb)
0294 lsass.exe 2017 ( 8068 Kb)
0780 test.exe 1741 ( 6964 Kb)
00e8 fsstress.exe 1526 ( 6104 Kb)
008c explorer.exe 1357 ( 5428 Kb)
0244 csrss.exe 1320 ( 5280 Kb)
056c spoolsv.exe 1011 ( 4044 Kb)
0420 svchost.exe 847 ( 3388 Kb)
047c svchost.exe 645 ( 2580 Kb)
0d38 ssparsef.exe 636 ( 2544 Kb)
0f18 ios_unbufw.exe 617 ( 2468 Kb)
05a4 msdtc.exe 444 ( 1776 Kb)
01f4 wmiprvse.exe 424 ( 1696 Kb)
03dc wmiprvse.exe 391 ( 1564 Kb)
03c8 cmd.exe 368 ( 1472 Kb)
0288 services.exe 367 ( 1468 Kb)
0204 cmd.exe 364 ( 1456 Kb)
0180 cmd.exe 364 ( 1456 Kb)
0688 cmd.exe 359 ( 1436 Kb)
04dc cmd.exe 359 ( 1436 Kb)
0150 cmd.exe 359 ( 1436 Kb)
00f8 cmd.exe 359 ( 1436 Kb)
059c cmd.exe 353 ( 1412 Kb)
0370 svchost.exe 331 ( 1324 Kb)
0704 dfssvc.exe 330 ( 1320 Kb)
0330 svchost.exe 257 ( 1028 Kb)
0144 HCTReboot.exe 228 ( 912 Kb)
062c GhostStartServi 183 ( 732 Kb)
0494 vmstress.exe 179 ( 716 Kb)
0774 lotsfile.exe 159 ( 636 Kb)
0764 dks.exe 129 ( 516 Kb)
0614 svchost.exe 121 ( 484 Kb)
0130 GhostStartTrayA 116 ( 464 Kb)
00e4 ios_fscmprsn.ex 115 ( 460 Kb)
020c irpmonui.exe 74 ( 296 Kb)
0d90 lookaside.exe 71 ( 284 Kb)
0684 svchost.exe 71 ( 284 Kb)
0d20 lookaside.exe 67 ( 268 Kb)
038c iosysmon.exe 56 ( 224 Kb)
076c frag.exe 47 ( 188 Kb)
0214 smss.exe 41 ( 164 Kb)
0004 System 7 ( 28 Kb)
03d8 cmd.exe 0 ( 0 Kb)
02d8 net.exe 0 ( 0 Kb)
01ec rundll32.exe 0 ( 0 Kb)
00c0 verifier.exe 0 ( 0 Kb)
… so for completeness here is a bit of !memusage and friend …
1: kd> !memusage
*CacheSize too low - increasing to 25 MB
Max cache size is : 26787840 bytes (0x6630 KB)
Total memory in cache : 231261 bytes (0xe2 KB)
Number of regions cached: 19818
136930 full reads broken into 137972 partial reads
counts: 16586 cached/121386 uncached, 12.02% cached
bytes : 42746 cached/594290 uncached, 6.71% cached
Transition PTEs are implicitly decoded
loading PFN database
loading (100% complete)
Compiling memory usage data (99% Complete).
Zeroed: 6 ( 24 kb)
Free: 0 ( 0 kb)
Standby: 87 ( 348 kb)
Modified: 65798 (263192 kb)
ModifiedNoWrite: 13 ( 52 kb)
Active/Valid: 63632 (254528 kb)
Transition: 1298 ( 5192 kb)
Unknown: 0 ( 0 kb)
TOTAL: 130834 (523336 kb)
Building kernel map
Finished building kernel map
Scanning PFN database - (100% complete)
Usage Summary (in Kb):
Control Valid Standby Dirty Shared Locked PageTables name
81581348 80 1924 260140 0 1924 0 Name Not Available
8124aaa8 0 0 2828 0 932 0 mapped_file(
$ConvertToNonresident )
81acee68 0 0 4 0 0 0 mapped_file( $BitMap )
81c9ae68 0 0 4 0 0 0 mapped_file( $BitMap )
81d234d0 196 0 0 108 0 0 Name Not Available
81bb7c40 32 0 0 0 0 0 Name Not Available
81c06078 56 0 0 20 0 0 Name Not Available
81c9ce68 0 0 4 0 0 0 mapped_file( $BitMap )
81bd8008 92 0 0 48 0 0 Name Not Available
81d1ab68 0 0 4 0 0 0 mapped_file( $BitMap )
81cf4790 64 0 128 0 0 0 Name Not Available
81cc0c80 4 0 0 0 0 0 Name Not Available
81c85aa8 0 0 4 0 0 0 mapped_file( $LogFile )
81766b30 16 0 0 0 0 0 Name Not Available
81c98e68 0 0 96 0 0 0 mapped_file( $Mft )
81c08400 112 0 0 68 0 0 Name Not Available
81be8008 112 0 0 52 0 0 Name Not Available
81bdadb0 4 0 0 0 0 0 Name Not Available
81c8cc38 4 0 0 0 0 0 Name Not Available
81268348 0 0 4 0 0 0 mapped_file(
$ConvertToNonresident )
81bb2958 16 0 0 0 0 0 Name Not Available
81ad5ae8 16 0 0 0 0 0 Name Not Available
81775af8 16 0 0 0 0 0 Name Not Available
81c04740 12 0 0 0 0 0 Name Not Available
81482120 8 0 0 0 0 0 Name Not Available
8152c150 0 0 4 0 4 0 mapped_file(
$ConvertToNonresident )
8153a148 16 0 0 0 0 0 Name Not Available
81c4c3c8 48 0 0 16 0 0 Name Not Available
81bd5008 0 0 4 0 0 0 mapped_file(
$ConvertToNonresident )
81d207e8 0 0 4 0 0 0 mapped_file( $LogFile )
81c4f7a0 12 0 0 0 0 0 Name Not Available
81c081c8 4 0 0 0 0 0 Name Not Available
817d0108 12 0 0 0 0 0 mapped_file( fastprox.dll
)
81c131d8 12 0 0 0 0 0 Name Not Available
81ca12c8 0 0 4 0 0 0 mapped_file( $LogFile )
81ca51a8 24 0 0 0 0 0 Name Not Available
81bfead8 12 0 0 0 0 0 Name Not Available
81bf72f0 12 0 0 0 0 0 Name Not Available
81ab7008 4 0 0 0 0 0 Name Not Available
8146d368 4 0 0 0 0 0 Name Not Available
81cc63e0 20 0 0 0 0 0 Name Not Available
81c4d160 20 0 0 0 0 0 Name Not Available
8146d290 0 0 4 0 4 0 mapped_file(
$ConvertToNonresident )
81457e00 16 0 0 0 0 0 Name Not Available
812f89c0 0 0 4 0 4 0 Name Not Available
81c85188 16 0 0 0 0 0 Name Not Available
81524d78 4 0 0 0 0 0 Name Not Available
81349008 8 0 0 0 0 0 Name Not Available
81c08080 4 0 0 0 0 0 Name Not Available
81bda4b0 12 0 0 0 0 0 Name Not Available
81bdeab8 4 0 0 0 0 0 Name Not Available
81c0e008 8 0 0 0 0 0 Name Not Available
81c4d988 4 0 0 4 0 0 Name Not Available
814609d8 4 0 0 0 0 0 Name Not Available
8150bc60 8 0 0 0 0 0 Name Not Available
81c493b8 16 0 0 0 0 0 Name Not Available
81ca0d60 8 0 0 0 0 0 Name Not Available
81c8aa28 4 0 0 0 0 0 Name Not Available
81bbec08 4 0 0 0 0 0 Name Not Available
81482008 4 0 0 0 0 0 Name Not Available
81ac8e68 0 0 4 0 0 0 mapped_file( $LogFile )
81be5b98 4 0 0 0 0 0 Name Not Available
81bfa5d8 16 0 0 0 0 0 Name Not Available
81c84530 4 0 0 0 0 0 Name Not Available
81c0b460 4 0 0 0 0 0 Name Not Available
81ca9508 8 0 0 0 0 0 Name Not Available
81c21008 4 0 0 0 0 0 Name Not Available
81bda1b0 16 0 0 8 0 0 Name Not Available
81be4b60 4 0 0 0 0 0 Name Not Available
81ca62e0 8 0 0 0 0 0 Name Not Available
81c98ad8 4 0 0 4 0 0 mapped_file( unicode.nls
)
00000000 492 0 0 392 0 0 PagedPool
-------- 141436 0 0 ----- ----- 160 pagefile section
(18183)
-------- 78580 8 0 ----- ----- 96 pagefile section (a5de)
-------- 8 0 0 ----- ----- 8 pagefile section (39)
-------- 96 0 0 ----- ----- 68 pagefile section (fd47)
-------- 112 0 0 ----- ----- 76 pagefile section (f770)
-------- 88 0 0 ----- ----- 28 pagefile section (18687)
-------- 8 0 0 ----- ----- 8 pagefile section (6fa)
-------- 160 0 0 ----- ----- 60 pagefile section (aa35)
-------- 132 0 0 ----- ----- 60 pagefile section (1ac84)
-------- 96 0 0 ----- ----- 32 pagefile section (ad9a)
-------- 128 0 0 ----- ----- 48 pagefile section (1b69e)
-------- 20 0 0 ----- ----- 20 pagefile section (27e0)
-------- 24 0 0 ----- ----- 20 pagefile section (1b99a)
-------- 88 0 0 ----- ----- 32 pagefile section (b2b4)
-------- 80 0 0 ----- ----- 32 pagefile section (19e80)
-------- 116 0 0 ----- ----- 56 pagefile section (b11f)
-------- 120 0 0 ----- ----- 36 pagefile section (1b14c)
-------- 136 0 0 ----- ----- 56 pagefile section (175a8)
-------- 36 0 0 ----- ----- 20 pagefile section (1ab53)
-------- 116 0 0 ----- ----- 44 pagefile section (1aa44)
-------- 108 0 0 ----- ----- 36 pagefile section (ea45)
-------- 108 0 0 ----- ----- 64 pagefile section (1070e)
-------- 108 0 0 ----- ----- 36 pagefile section (1b42f)
-------- 100 0 0 ----- ----- 40 pagefile section (1b7a8)
-------- 76 0 0 ----- ----- 48 pagefile section (1707f)
-------- 28 8 0 ----- ----- 20 pagefile section (eccb)
-------- 96 0 0 ----- ----- 52 pagefile section (a5c0)
-------- 104 0 0 ----- ----- 36 pagefile section (b227)
-------- 116 0 0 ----- ----- 40 pagefile section (1b14f)
-------- 116 0 0 ----- ----- 56 pagefile section (199e0)
-------- 112 0 0 ----- ----- 28 pagefile section (f726)
-------- 88 0 0 ----- ----- 36 pagefile section (e876)
-------- 88 0 0 ----- ----- 28 pagefile section (1faa2)
-------- 36 0 0 ----- ----- 20 pagefile section (8518)
-------- 28 0 0 ----- ----- 20 pagefile section (5139)
-------- 132 0 0 ----- ----- 40 pagefile section (1caa3)
-------- 96 0 0 ----- ----- 32 pagefile section (1bc17)
-------- 36 0 0 ----- ----- 20 pagefile section (19a15)
-------- 36 0 0 ----- ----- 20 pagefile section (197b1)
-------- 36 0 0 ----- ----- 20 pagefile section (ab52)
-------- 36 0 0 ----- ----- 20 pagefile section (1d663)
-------- 36 0 0 ----- ----- 20 pagefile section (ad27)
-------- 36 0 0 ----- ----- 20 pagefile section (ee99)
-------- 36 0 0 ----- ----- 20 pagefile section (ec73)
-------- 36 0 0 ----- ----- 20 pagefile section (1a6ea)
-------- 8 0 0 ----- ----- 8 pagefile section (ecac)
-------- 8 0 0 ----- ----- 8 pagefile section (15743)
-------- 8 0 0 ----- ----- 8 pagefile section (f58f)
-------- 116 0 0 ----- ----- 92 pagefile section (1a7fe)
-------- 56 0 0 ----- ----- 40 pagefile section (1a495)
-------- 28 0 0 ----- ----- 24 pagefile section (1c68c)
-------- 40 40 0 ----- 0 ----- driver ( ntoskrnl.exe )
-------- 0 8 0 ----- 0 ----- driver ( hal.dll )
-------- 0 0 0 ----- 0 ----- driver ( kdcom.dll )
-------- 0 0 0 ----- 0 ----- driver ( BOOTVID.dll )
-------- 0 0 0 ----- 0 ----- driver ( ACPI.sys )
-------- 0 0 0 ----- 0 ----- driver ( WMILIB.SYS )
-------- 0 0 0 ----- 0 ----- driver ( pci.sys )
-------- 0 0 0 ----- 0 ----- driver ( isapnp.sys )
-------- 0 0 0 ----- 0 ----- driver ( pciide.sys )
-------- 0 0 0 ----- 0 ----- driver ( PCIIDEX.SYS )
-------- 0 0 0 ----- 0 ----- driver ( intelide.sys )
-------- 0 0 0 ----- 0 ----- driver ( MountMgr.sys )
-------- 0 0 0 ----- 0 ----- driver ( ftdisk.sys )
-------- 0 0 0 ----- 0 ----- driver ( dmload.sys )
-------- 0 0 0 ----- 0 ----- driver ( dmio.sys )
-------- 0 0 0 ----- 0 ----- driver ( volsnap.sys )
-------- 0 0 0 ----- 0 ----- driver ( PartMgr.sys )
-------- 0 0 0 ----- 0 ----- driver ( atapi.sys )
-------- 0 0 0 ----- 0 ----- driver ( symmpi.sys )
-------- 0 0 0 ----- 0 ----- driver ( SCSIPORT.SYS )
-------- 0 0 0 ----- 0 ----- driver ( disk.sys )
-------- 0 0 0 ----- 0 ----- driver ( CLASSPNP.SYS )
-------- 0 0 0 ----- 0 ----- driver ( Dfs.sys )
-------- 0 0 0 ----- 0 ----- driver ( KSecDD.sys )
-------- 0 0 0 ----- 0 ----- driver ( Ntfs.sys )
-------- 0 0 0 ----- 0 ----- driver ( NDIS.sys )
-------- 0 0 0 ----- 0 ----- driver ( Mup.sys )
-------- 0 0 0 ----- 0 ----- driver ( crcdisk.sys )
-------- 0 0 0 ----- 0 ----- driver ( agp440.sys )
-------- 0 0 0 ----- 0 ----- driver ( processr.sys )
-------- 0 0 0 ----- 0 ----- driver ( usbuhci.sys )
-------- 0 0 0 ----- 0 ----- driver ( USBPORT.SYS )
-------- 0 0 0 ----- 0 ----- driver ( usbehci.sys )
-------- 0 0 0 ----- 0 ----- driver ( ati2mpad.sys )
-------- 0 0 0 ----- 0 ----- driver ( VIDEOPRT.SYS )
-------- 0 0 0 ----- 0 ----- driver ( watchdog.sys )
-------- 0 0 0 ----- 0 ----- driver ( n100325.sys )
-------- 0 0 0 ----- 0 ----- driver ( e1000325.sys )
-------- 0 0 0 ----- 0 ----- driver ( fdc.sys )
-------- 0 0 0 ----- 0 ----- driver ( i8042prt.sys )
-------- 0 0 0 ----- 0 ----- driver ( kbdclass.sys )
-------- 0 0 0 ----- 0 ----- driver ( mouclass.sys )
-------- 0 0 0 ----- 0 ----- driver ( serial.sys )
-------- 0 0 0 ----- 0 ----- driver ( serenum.sys )
-------- 0 0 0 ----- 0 ----- driver ( parport.sys )
-------- 0 0 0 ----- 0 ----- driver ( cdrom.sys )
-------- 0 0 0 ----- 0 ----- driver ( redbook.sys )
-------- 0 0 0 ----- 0 ----- driver ( ks.sys )
-------- 0 0 0 ----- 0 ----- driver ( smwdm.sys )
-------- 0 0 0 ----- 0 ----- driver ( portcls.sys )
-------- 0 0 0 ----- 0 ----- driver ( drmk.sys )
-------- 0 0 0 ----- 0 ----- driver ( aeaudio.sys )
-------- 0 0 0 ----- 0 ----- driver ( audstub.sys )
-------- 0 0 0 ----- 0 ----- driver ( rasl2tp.sys )
-------- 0 0 0 ----- 0 ----- driver ( ndistapi.sys )
-------- 0 0 0 ----- 0 ----- driver ( ndiswan.sys )
-------- 0 0 0 ----- 0 ----- driver ( raspppoe.sys )
-------- 0 0 0 ----- 0 ----- driver ( raspptp.sys )
-------- 0 0 0 ----- 0 ----- driver ( TDI.SYS )
-------- 0 0 0 ----- 0 ----- driver ( ptilink.sys )
-------- 0 0 0 ----- 0 ----- driver ( raspti.sys )
-------- 0 0 0 ----- 0 ----- driver ( rdpdr.sys )
-------- 0 0 0 ----- 0 ----- driver ( termdd.sys )
-------- 0 0 0 ----- 0 ----- driver ( swenum.sys )
-------- 0 0 0 ----- 0 ----- driver ( update.sys )
-------- 0 0 0 ----- 0 ----- driver ( NDProxy.SYS )
-------- 0 0 0 ----- 0 ----- driver ( usbhub.sys )
-------- 0 0 0 ----- 0 ----- driver ( USBD.SYS )
-------- 0 0 0 ----- 0 ----- driver ( flpydisk.sys )
-------- 0 0 0 ----- 0 ----- driver ( Fs_Rec.SYS )
-------- 0 0 0 ----- 0 ----- driver ( Null.SYS )
-------- 0 0 0 ----- 0 ----- driver ( Beep.SYS )
-------- 0 0 0 ----- 0 ----- driver ( ghpciscan.sys )
-------- 0 0 0 ----- 0 ----- driver ( vga.sys )
-------- 0 0 0 ----- 0 ----- driver ( mnmdd.SYS )
-------- 0 0 0 ----- 0 ----- driver ( RDPCDD.sys )
-------- 0 0 0 ----- 0 ----- driver ( Msfs.SYS )
-------- 0 0 0 ----- 0 ----- driver ( Npfs.SYS )
-------- 0 0 0 ----- 0 ----- driver ( rasacd.sys )
-------- 0 0 0 ----- 0 ----- driver ( ipsec.sys )
-------- 0 0 0 ----- 0 ----- driver ( msgpc.sys )
-------- 0 0 0 ----- 0 ----- driver ( tcpip.sys )
-------- 0 0 0 ----- 0 ----- driver ( netbt.sys )
-------- 0 0 0 ----- 0 ----- driver ( wanarp.sys )
-------- 0 0 0 ----- 0 ----- driver ( netbios.sys )
-------- 0 0 0 ----- 0 ----- driver ( rdbss.sys )
-------- 0 0 0 ----- 0 ----- driver ( mrxsmb.sys )
-------- 0 0 0 ----- 0 ----- driver ( Fips.SYS )
-------- 0 0 0 ----- 0 ----- driver ( Fastfat.SYS )
-------- 0 0 0 ----- 0 ----- driver ( dump_atapi.sys )
-------- 0 0 0 ----- 0 ----- driver ( dump_WMILIB.SYS
)
-------- 0 0 0 ----- 0 ----- driver ( win32k.sys )
-------- 0 0 0 ----- 0 ----- driver ( Dxapi.sys )
-------- 0 0 0 ----- 0 ----- driver ( dxg.sys )
-------- 0 0 0 ----- 0 ----- driver ( dxgthk.sys )
-------- 0 0 0 ----- 0 ----- driver ( ati2drad.dll )
-------- 0 0 0 ----- 0 ----- driver ( afd.sys )
-------- 0 0 0 ----- 0 ----- driver ( ndisuio.sys )
-------- 0 0 0 ----- 0 ----- driver ( parvdm.sys )
-------- 0 0 0 ----- 0 ----- driver ( Aspi32.SYS )
-------- 0 0 0 ----- 0 ----- driver ( srv.sys )
-------- 0 0 0 ----- 0 ----- driver ( wdmaud.sys )
-------- 0 0 0 ----- 0 ----- driver ( sysaudio.sys )
-------- 0 0 0 ----- 0 ----- driver ( MYDRV.SYS )
-------- 0 0 0 ----- 0 ----- driver ( Cdfs.SYS )
-------- 0 0 0 ----- 0 ----- driver ( vcdrom.sys )
-------- 0 0 0 ----- 0 ----- driver ( Udfs.SYS )
-------- 76 200 0 ----- 12 ----- ( Paged Pool )
-------- 0 0 0 ----- 0 ----- ( Kernel Stacks )
-------- 976 84 0 ----- 0 ----- ( NonPaged Pool )
1: kd> !ca 81581348
ControlArea @81581348
Segment: e18e3660 Flink 0 Blink 0
Section Ref 1 Pfn Ref 10000 Mapped Views 1
User Ref 2 WaitForDel 0 Flush Count 0
File Object 81721f90 ModWriteCount 0 System Views 0
Flags (1000080) File HadUserReference
File Name paged out
Segment @ e18e3660:
Type MAPPED_FILE_SEGMENT not found.
Cheers
Lyndon
“Tony Mason” wrote in message news:xxxxx@ntfsd…
Hi Lyndon,
So you have the mapped page writer and lazy writer waiting to obtain the
FCB for a file that is currently performing an extending write (the MCB
package is used to manage allocation, which is why I say this is an
extending write).
The MCB package itself is faulting (presumably pulling in more of the
bitmap) and the memory manager is blocking on its own internal lock to
ensure that there are enough pages (MiEnsureAvailablePagesOrWait).
Hey, I talk about this in file systems class! I say “anything that
might block, memory allocation, worker threads, or synchronization
objects all have to be in a locking hierarchy”. The locking hierarchy
between these three is : FS first, then CC, then MM. But since MM is
now blocking on its own resource (essentially a blocking allocator) it
has violated that lock hierarchy. Of course, that leads to a deadlock.
I suspect this is a rather rare circumstance: Mm and Cc both have
threads blocked on the SAME file that is doing an extending write and
needs to fault in more of the bitmap. I’m wondering where the modified
page writer is (the only other thread that could possibly get you out of
this situation). I’m not sure if this is the precise deadlock that led
to the creation of the mapped page writer in the first place but it fits
the general description that I vaguely recall from the depths of
time…
Two more things to look at: find the mapped page writer and figure out
why it isn’t running or if it is running, try to find out why it isn’t
clearing enough pages. The second thing is to use “!vm” to see how the
page usage looks.
From what you’re showing here, this doesn’t strike me as an obvious
filter driver level bug but rather an OS-level issue that can probably
arise in these extreme circumstances anyway.
Of course, if you want I can look at a dump of this system and write it
up as an NT Insider article - I always love deadlock examples!
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Lyndon J Clarke
Sent: Sunday, April 17, 2005 8:44 AM
To: ntfsd redirect
Subject: Re:[ntfsd] Re:fcb and [non] page pool
Tony
The thread I showed in the O/P is the mapped page writer. It’s waiting
for
an exclusive lock on a resource. I guess this is the FCB resource since
…
f891e68c bae1529c 813cbd88 81547c01 bae3a03c
nt!ExAcquireResourceExclusiveLite+0x8b (FPO: [Non-Fpo])
f891e698 bae3a03c 81547c10 e1c44008 81547c01
Ntfs!NtfsAcquireResourceExclusive+0x1d (FPO: [3,0,0])
f891e6bc bae3a114 81547c01 e1c44008 e1c440d0
Ntfs!NtfsAcquireExclusiveFcb+0x40 (FPO: [Non-Fpo])
… hence I was looking to dig into the FCB … there is another
exclusive
waiter … do we recognise him? …
f88de2c0 804edb2b 81dfd0c0 81dfd020 8148d348
nt!KiSwapContext+0x26
(FPO: [EBP 0xf88de2e8] [0,0,4])
f88de2e8 804ed790 81dfd020 813cbd88 00000000
nt!KiSwapThread+0x280
(FPO: [Non-Fpo])
f88de31c 8051c0fb 8148d348 0000001b 00000000
nt!KeWaitForSingleObject+0x249 (FPO: [Non-Fpo])
f88de358 804f62f6 00000000 e1c44008 f88de964
nt!ExpWaitForResource+0xd3 (FPO: [Non-Fpo])
f88de378 bae1529c 813cbd88 f88de901 bae3a03c
nt!ExAcquireResourceExclusiveLite+0x8b (FPO: [Non-Fpo])
f88de384 bae3a03c f88de964 e1c44008 f88de901
Ntfs!NtfsAcquireResourceExclusive+0x1d (FPO: [3,0,0])
f88de3a8 bae3a114 f88de901 e1c44008 e1c440d0
Ntfs!NtfsAcquireExclusiveFcb+0x40 (FPO: [Non-Fpo])
f88de3c0 bae24dd9 f88de964 e1c440d0 8153e9e8
Ntfs!NtfsAcquireExclusiveScb+0x14 (FPO: [2,0,0])
f88de574 bae204c8 f88de964 8153e9e8 e1c440d0
Ntfs!NtfsPrepareBuffers+0xcb2 (FPO: [Non-Fpo])
f88de750 bae17a2a f88de964 8153e9e8 e1c440d0
Ntfs!NtfsNonCachedIo+0x4f7 (FPO: [Non-Fpo])
f88de954 bae17c84 f88de964 8153e9e8 0108070a
Ntfs!NtfsCommonWrite+0x18a0 (FPO: [Non-Fpo])
f88dead0 806a7630 81b28718 8153e9e8 80748a30
Ntfs!NtfsFsdWrite+0x16a
(FPO: [Non-Fpo])
f88deb00 805264de f7344334 f7344334 81583908
nt!IovCallDriver+0x110
(FPO: [Non-Fpo])
f88deb08 f7344334 81583908 806a7630 81583908
nt!IofCallDriver+0xe
(FPO: [0,0,0])
f88deb10 806a7630 81583908 8153e9e8 812fd870
MYDRV!MyDispatch+0x2e
(FPO: [2,0,0]) (CONV: stdcall)
f88deb40 805264de 804f91ca 804f91ca f88deb84
nt!IovCallDriver+0x110
(FPO: [Non-Fpo])
f88deb48 804f91ca f88deb84 f88ded54 8124aaa8
nt!IofCallDriver+0xe
(FPO: [0,0,0])
f88deb5c 804fcf44 812fd808 f88deb84 f88dec00
nt!IoSynchronousPageWrite+0xad (FPO: [Non-Fpo])
f88dec30 804fc6ad e1200e20 e1200e60 e1200e60
nt!MiFlushSectionInternal+0x3c4 (FPO: [Non-Fpo])
f88dec78 804fbfb9 8124aaa8 f88decc8 00010000
nt!MmFlushSection+0x1fe
(FPO: [Non-Fpo])
f88ded00 804fdffa 00010000 00000000 00000001
nt!CcFlushCache+0x37d
(FPO: [Non-Fpo])
f88ded40 804f223a 81dfd020 80582d80 81dfe2a0
nt!CcWriteBehind+0x116
(FPO: [Non-Fpo])
f88ded80 804eeabb 81dfe2a0 00000000 81dfd020
nt!CcWorkerThread+0x12c
(FPO: [Non-Fpo])
f88dedac 80596ffe 81dfe2a0 00000000 00000000
nt!ExpWorkerThread+0xe9
(FPO: [Non-Fpo])
f88deddc 805008c6 804ee9f0 00000000 00000000
nt!PspSystemThreadStartup+0x2e (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000
nt!KiThreadStartup+0x16
… and the thread that holds this resource is here …
f72ce3a0 804edb2b 813c79d0 813c7930 8057af50
nt!KiSwapContext+0x26
(FPO: [EBP 0xf72ce3c8] [0,0,4])
f72ce3c8 804ed790 00000000 813c7964 8057fe60
nt!KiSwapThread+0x280
(FPO: [Non-Fpo])
f72ce3fc 8052ffde 8057af50 00000008 00000000
nt!KeWaitForSingleObject+0x249 (FPO: [Non-Fpo])
f72ce444 8052ccce 00000001 fffff000 00000000
nt!MiEnsureAvailablePageOrWait+0x1e6 (FPO: [EBP 0xf72ce46c] [1,8,0])
f72ce46c 804fed93 e1ffbcc8 c0387fec 00000000
nt!MiResolveTransitionFault+0x390 (FPO: [Non-Fpo])
f72ce4f0 804f4c21 00000021 e1ffbcc8 c0387fec
nt!MiDispatchFault+0x638 (FPO: [Non-Fpo])
f72ce54c 804e2dfc 00000000 e1ffbcc8 00000000
nt!MmAccessFault+0x5ca
(FPO: [Non-Fpo])
f72ce54c 804f8bd2 00000000 e1ffbcc8 00000000 nt!KiTrap0E+0xc8
(FPO:
[0,0] TrapFrame @ f72ce564)
f72ce5dc 804f8cad 00000000 00000000 00000000
nt!FsRtlLookupBaseMcbEntry+0x26 (FPO: [Non-Fpo])
f72ce62c bae16a69 e1ffbcb8 00000000 00000000
nt!FsRtlLookupLargeMcbEntry+0x3d (FPO: [Non-Fpo])
f72ce66c bae49784 e1c44160 00000000 00000000
Ntfs!NtfsLookupNtfsMcbEntry+0x99 (FPO: [Non-Fpo])
f72ce6ec bae4a8b4 e1c44160 00000000 00000000
Ntfs!NtfsBuildMappingPairs+0x44 (FPO: [Non-Fpo])
f72ce8c4 bae4af91 81492ef8 e1c440d0 f72ce8f4
Ntfs!NtfsAddAttributeAllocation+0x8f5 (FPO: [Non-Fpo])
f72ce988 bae19b49 81492ef8 81cfbf90 e1c440d0
Ntfs!NtfsAddAllocation+0x391 (FPO: [Non-Fpo])
f72ceb94 bae17c84 81492ef8 83514e70 80748a30
Ntfs!NtfsCommonWrite+0x1252 (FPO: [Non-Fpo])
f72cec08 806a7630 81b28718 83514e70 80748a30
Ntfs!NtfsFsdWrite+0x16a
(FPO: [Non-Fpo])
f72cec38 805264de f7344334 f7344334 81583908
nt!IovCallDriver+0x110
(FPO: [Non-Fpo])
f72cec40 f7344334 81583908 806a7630 81583908
nt!IofCallDriver+0xe
(FPO: [0,0,0])
f72cec48 806a7630 81583908 83514e70 83514e70
MYFDRV!MyDispatch+0x2e
(FPO: [2,0,0]) (CONV: stdcall)
f72cec78 805264de 80585208 80585208 83514fdc
nt!IovCallDriver+0x110
(FPO: [Non-Fpo])
f72cec80 80585208 83514fdc 00000000 83514e70
nt!IofCallDriver+0xe
(FPO: [0,0,0])
f72cec94 8058c236 81583908 83514e70 81cfbf90
nt!IopSynchronousServiceTail+0x6f (FPO: [Non-Fpo])
f72ced38 804dfd24 000007ac 00000000 00000000
nt!NtWriteFile+0x5e0
(FPO: [Non-Fpo])
f72ced38 7ffe0304 000007ac 00000000 00000000
nt!KiSystemService+0xd0
(FPO: [0,0] TrapFrame @ f72ced64)
0006d094 00000000 00000000 00000000 00000000
SharedUserData!SystemCallStub+0x4 (FPO: [0,0,0])
… you can see this system is in a but of trouble since the mapped page
writer and other thread cant proceed because this thread cant proceed
due to
a shortage of free pages 
I was wondering if you had any insight you can share in this situatios?
Cheers
Lyndon
“Tony Mason” wrote in message news:xxxxx@ntfsd…
So long as the paging file path has all of its code and data non-paged
it is fine. The other structures CAN come from paged pool. For
example, if you look at FastFat it has a NON_PAGED_FCB structure and an
FCB structure, with the latter being allowed to come from paged pool for
anything except the paging file (which MUST all be non-paged).
Max is right that the dispatcher objects must be in non-paged memory.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Saturday, April 16, 2005 3:37 PM
To: ntfsd redirect
Subject: Re: Re:[ntfsd] fcb and [non] page pool
Usually, these ERESOURCEs are also in the FCB, just after the
header.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: “Lyndon J Clarke”
Newsgroups: ntfsd
To: “Windows File Systems Devs Interest List”
Sent: Saturday, April 16, 2005 8:27 PM
Subject: Re:[ntfsd] fcb and [non] page pool
> Hey Maxim
>
> Take a look at your ntifs.h there might be a surprise?
>
> typedef struct _FSRTL_COMMON_FCB_HEADER {
> [snip]
> PERESOURCE Resource;
> PERESOURCE PagingIoResource;
> [snip]
> } FSRTL_COMMON_FCB_HEADER;
>
> So anyway I [had] believe[d] that FCB cannot be in page pool - did I
go
> wrong somewhere in windbag in the (partial) session I copied in the
OP?
>
> Thanks
> Lyndon
>
> “Maxim S. Shatskih” wrote in message
> news:xxxxx@ntfsd…
> >> This might be a dumb question … but is it feasible for an fcb, as
in
> >> the
> >> thing pointed to by FileObject->FsContext, to be allocated from
paged
> >> pool?
> >
> > No. FCB contains 2 ERESOURCE structures, which in turn contain
events
> > within
> > them.
> >
> > You cannot have KEVENT in pageable memory - the dispatcher will
crash
> > while
> > awakening the thread waiting on such KEVENT.
> >
> > Maxim Shatskih, Windows DDK MVP
> > StorageCraft Corporation
> > xxxxx@storagecraft.com
> > http://www.storagecraft.com
> >
> >
>
>
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
