Hello
I started getting PFN_LIST_CORRUPT BSOD.
So looking at http://osronline.com/article.cfm?article=334. I confirmed *for now* (from source code walkthrough) that my MDL handling might not be the issue. Then I did the following
I enabled DV on my drivers and red-did the test.
I now get PROCESS_HAS_LOCKED_PAGES BSOD.
Below is its its winddbg.hlp
********************
“…Otherwise, set HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory
Management\TrackLockedPages to a DWORD 1 value and reboot. Then the system
will save stack traces so the guilty driver can be easily identified.
When you enable this flag, if the driver commits the error again you will
see a different bugcheck - DRIVER_LEFT_LOCKED_PAGES_IN_PROCESS (0xCB) -
which can identify the offending driver(s).
…”
*********************
So I created and set HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory
Management\TrackLockedPages=1.
Ran the test again. Was hoping to see DRIVER_LEFT_LOCKED_PAGES_IN_PROCESS (0xCB) BSOD, but again got the same PROCESS_HAS_LOCKED_PAGES BSOD.
Inlined belwo is the mini-dump.
-
Why I Di dnot get DRIVER_LEFT_LOCKED_PAGES_IN_PROCESS (0xCB)?
-
Is it becuase of DriverVerifier (DV)?
-
I tried to list verifier logs below and search the process as instructed, but they do not show up anything.
Is it becuase of minidump? -
In all BSODs I see CLASSPNP.SYS as the faulty one.
Can I believe that? what is that driver? Any known issues? -
To use TrackLockedPages do I need to dsiable DV and need a full dump?
-
Any other resources which shwo step by step
debug of Debugging PFN_LIST_CORRUPT/PROCESS_HAS_LOCKED_PAGES BSODs ?
For now will test with DV disabled and TrackLockedPages=1.
Let me knwo any pointers that woudl help to proceed on this.
******************************************************
Microsoft (R) Windows Debugger Version 6.9.0003.113 X86
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [C:_dumps\PROCESS_HAS_LOCKED_PAGES\Mini100208-02.dmp]
Mini Kernel Dump File: Only registers and stack trace are available
Symbol search path is: srv*c:_symbols*http://msdl.microsoft.com/download/symbols;C:_symbols_xp_exts;
Executable search path is: srv*c:_symbols*http://msdl.microsoft.com/download/symbols;C:_symbols_xp_exts;
Windows Server 2008 Kernel Version 6001 (Service Pack 1) MP (4 procs) Free x64
Product: Server, suite: Enterprise TerminalServer SingleUserTS
Built by: 6001.18000.amd64fre.longhorn_rtm.080118-1840
Kernel base = 0xfffff8000160d000 PsLoadedModuleList = 0xfffff800017d2db0
Debug session time: Thu Oct 2 12:26:40.750 2008 (GMT-7)
System Uptime: 0 days 0:46:23.531
Loading Kernel Symbols
…
Loading User Symbols
Loading unloaded module list
…
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
Use !analyze -v to get detailed debugging information.
BugCheck 76, {1, fffffa800b797910, 0, fffffa80069aaab0}
Probably caused by : CLASSPNP.SYS ( CLASSPNP!TransferPktComplete+1a0 )
Followup: MachineOwner
Processing initial command ‘.symopt-0x4’
3: kd> .symopt-0x4
Symbol options are 0x30233:
0x00000001 - SYMOPT_CASE_INSENSITIVE
0x00000002 - SYMOPT_UNDNAME
0x00000010 - SYMOPT_LOAD_LINES
0x00000020 - SYMOPT_OMAP_FIND_NEAREST
0x00000200 - SYMOPT_FAIL_CRITICAL_ERRORS
0x00010000 - SYMOPT_AUTO_PUBLICS
0x00020000 - SYMOPT_NO_IMAGE_SEARCH
3: kd> .reload /f
Loading Kernel Symbols
…Unable to load image mcupdate.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for mcupdate.dll
…Unable to load image CI.dll, Win32 error 0n2
*** WARNING: Unable to verify timestamp for CI.dll
…Unable to load image msrpc.sys, Win32 error 0n2
*** WARNING: Unable to verify timestamp for msrpc.sys
.Unable to load image qd260x64.sys, Win32 error 0n2
*** WARNING: Unable to verify timestamp for qd260x64.sys
…Unable to load image Fs_Rec.SYS, Win32 error 0n2
*** WARNING: Unable to verify timestamp for Fs_Rec.SYS
.Unable to load image Null.SYS, Win32 error 0n2
*** WARNING: Unable to verify timestamp for Null.SYS
…Unable to load image Msfs.SYS, Win32 error 0n2
*** WARNING: Unable to verify timestamp for Msfs.SYS
…Unable to load image dump_ataport.sys, Win32 error 0n2
*** WARNING: Unable to verify timestamp for dump_ataport.sys
…Unable to load image spsys.sys, Win32 error 0n2
*** WARNING: Unable to verify timestamp for spsys.sys
…Unable to load image npf.sys, Win32 error 0n2
*** WARNING: Unable to verify timestamp for npf.sys
…Unable to load image secdrv.SYS, Win32 error 0n2
*** WARNING: Unable to verify timestamp for secdrv.SYS
…
Loading User Symbols
Loading unloaded module list
…
3: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
PROCESS_HAS_LOCKED_PAGES (76)
Caused by a driver not cleaning up completely after an I/O.
Issue a !search over all of physical memory for the current process pointer.
This will yield at least one MDL which points to it. Then do another !search
for each MDL found, this will yield the irp(s) that point to it, revealing
which driver is leaking the pages.
Otherwise, set HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory
Management\TrackLockedPages to a DWORD 1 value and reboot. Then the system
will save stack traces so the guilty driver can be easily identified.
When you enable this flag, if the driver commits the error again you will
see a different bugcheck - DRIVER_LEFT_LOCKED_PAGES_IN_PROCESS (0xCB) -
which can identify the offending driver(s).
Note that on newer versions of Windows (XPSP2 or WS03 and above), if you hit a
breakpoint in MmCleanProcessAddressSpace indicating the process is leaking
pages, after you press ‘g’, memory management will automatically enable locked
page tracking (like the registry key description above) until the system is
rebooted in order to help find the culprit.
Arguments:
Arg1: 0000000000000001, 0
Arg2: fffffa800b797910, process address
Arg3: 0000000000000000, number of locked pages
Arg4: fffffa80069aaab0, pointer to driver stacks (if enabled) or 0 if not.
Debugging Details:
DEFAULT_BUCKET_ID: DRIVER_FAULT_0x76
CUSTOMER_CRASH_COUNT: 2
BUGCHECK_STR: 0x76
PROCESS_NAME: System
CURRENT_IRQL: 2
LAST_CONTROL_TRANSFER: from fffff800016ec62f to fffff80001662390
STACK_TEXT:
fffffa60017ff6f8 fffff800016ec62f : 0000000000000076 0000000000000001 fffffa800b797910 0000000000000000 : nt!KeBugCheckEx
fffffa60017ff700 fffff8000160e2cb : 0000000000000002 000000000000000c fffffa800b797910 fffffa800b797940 : nt!MiFreeMdlTracker+0xaf
fffffa60017ff760 fffff8000166455a : fffffa800058a310 000082fe00003266 0000000000000000 0000000000000000 : nt! ?? ::FNODOBFM::string'+0x1fade fffffa60017ff800 fffff80001a698c3 : fffff980029b8dc0 fffff80001679e01 fffffa80069ace90 fffffa60017ff9a0 : nt!IopfCompleteRequest+0x16a fffffa60017ff8c0 fffffa6000fc9730 : fffffa60017ffb50 fffff80001a67383 0000000000000002 fffffa800940a010 : nt!IovCompleteRequest+0x43 fffffa60017ff9a0 fffff80001a6f3f6 : 0000000000000000 0000000000000000 fffff980026beea0 fffffa60017ffb78 : CLASSPNP!TransferPktComplete+0x1a0 fffffa60017ffa20 fffff80001664705 : fffff980026beea0 0000000000000000 fffffa800940ac00 fffff980026befbb : nt!IovpLocalCompletionRoutine+0x116 fffffa60017ffa60 fffff80001a698c3 : fffff980026beea0 fffff80001a6b101 fffffa800b8264b8 fffffa8008f4313a : nt!IopfCompleteRequest+0x315 fffffa60017ffb20 fffffa60025723ee : fffffa8007a1b1a0 fffff98001cb6dc0 fffffa8008ff2020 fffffa800940acc0 : nt!IovCompleteRequest+0x43 fffffa60017ffc00 fffffa6002572ee2 : fffffa8000000000 00000000000000fe fffffa8009008b30 fffffa8008ff07c0 : storport!RaidUnitCompleteRequest+0x1fe fffffa60017ffce0 fffff8000166b9d7 : fffff98001efafb0 fffffa60017db580 fffff980074f0fa0 0000000000000000 : storport!RaidpAdapterDpcRoutine+0x32 fffffa60017ffd10 fffff8000166cb72 : fffffa6002572eb0 fffffa60017d8180 0000000000000000 fffffa60017e1d40 : nt!KiRetireDpcList+0x117 fffffa60017ffd80 fffff8000183a5c0 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : nt!KiIdleLoop+0x62 fffffa60017ffdb0 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 00000000`00000000 : nt!zzz_AsmCodeRange_End+0x4
STACK_COMMAND: kb
FOLLOWUP_IP:
CLASSPNP!TransferPktComplete+1a0
fffffa60`00fc9730 837c243002 cmp dword ptr [rsp+30h],2
SYMBOL_STACK_INDEX: 5
SYMBOL_NAME: CLASSPNP!TransferPktComplete+1a0
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: CLASSPNP
IMAGE_NAME: CLASSPNP.SYS
DEBUG_FLR_IMAGE_TIMESTAMP: 479198a5
FAILURE_BUCKET_ID: X64_0x76_System_CLASSPNP!TransferPktComplete+1a0
BUCKET_ID: X64_0x76_System_CLASSPNP!TransferPktComplete+1a0
Followup: MachineOwner
3: kd> ~1
Can’t switch processors on a kernel triage dump
3: kd> ~2
Can’t switch processors on a kernel triage dump
3: kd> ~3
3: kd> k
Child-SP RetAddr Call Site
fffffa60017ff6f8 fffff800016ec62f nt!KeBugCheckEx
fffffa60017ff700 fffff8000160e2cb nt!MiFreeMdlTracker+0xaf
fffffa60017ff760 fffff8000166455a nt! ?? ::FNODOBFM::string'+0x1fade fffffa60017ff800 fffff80001a698c3 nt!IopfCompleteRequest+0x16a fffffa60017ff8c0 fffffa6000fc9730 nt!IovCompleteRequest+0x43 fffffa60017ff9a0 fffff80001a6f3f6 CLASSPNP!TransferPktComplete+0x1a0 fffffa60017ffa20 fffff80001664705 nt!IovpLocalCompletionRoutine+0x116 fffffa60017ffa60 fffff80001a698c3 nt!IopfCompleteRequest+0x315 fffffa60017ffb20 fffffa60025723ee nt!IovCompleteRequest+0x43 fffffa60017ffc00 fffffa6002572ee2 storport!RaidUnitCompleteRequest+0x1fe fffffa60017ffce0 fffff8000166b9d7 storport!RaidpAdapterDpcRoutine+0x32 fffffa60017ffd10 fffff8000166cb72 nt!KiRetireDpcList+0x117 fffffa60017ffd80 fffff8000183a5c0 nt!KiIdleLoop+0x62 fffffa60017ffdb0 00000000`00000000 nt!zzz_AsmCodeRange_End+0x4
3: kd> !verifier 1
fffff800017d3700: Unable to get verifier list.
3: kd> !verifier 80 3
Error: incorrect value of nt!VfPoolTracesLength = 0000000000000000
3: kd> !verifier 80
Error: incorrect value of nt!VfPoolTracesLength = 0000000000000000
3: kd> !verifier 0x2
fffff800017d3700: Unable to get verifier list.
3: kd> .symfix
No downstream store given, using C:\Program Files\Debugging Tools for Windows (x86)\sym
3: kd> !search fffffa800b797910
Warning: unable to dereference MmPfnDatabase at fffff80001836250.
Search: READ_PVOID error
Search: READ_PVOID error
Searching PFNs in range 0000000000000000 - 0000000000000000 for [FFFFFA800B797910 - FFFFFA800B797910]
Pfn Offset Hit Va Pte
Search: READ_PVOID error
Search: READ_ULONG error
Search error: cannot allocate system memory descriptor
******************************************************