Hello,
I am new to windbg and have a basic understanding of windows internals. What I need now is some practice. I managed to get a dump from a freezed server by sending a NMI.
So I started with first basic step.
0: kd> !analyze -v -hang
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
NMI_HARDWARE_FAILURE (80)
This is typically due to a hardware malfunction. The hardware supplier should
be called.
Arguments:
Arg1: 004f4454
Arg2: 00000000
Arg3: 00000000
Arg4: 00000000
Debugging Details:
Scanning for threads blocked on locks …
DEFAULT_BUCKET_ID: DRIVER_FAULT
PROCESS_NAME: Idle
CURRENT_IRQL: 1c
BLOCKED_THREAD: 8089d8c0
BLOCKING_THREAD: 86cb43e8
LOCK_ADDRESS: 8455538c – (!locks 8455538c)
Resource @ 0x8455538c Exclusively owned
Contention Count = 1
NumberOfExclusiveWaiters = 1
Threads: 86cb43e8-01<*>
Threads Waiting On Exclusive Access:
87c2a2f0
1 total locks, 1 locks currently held
BUGCHECK_STR: LOCK_HELD
LAST_CONTROL_TRANSFER: from 80833491 to 8088db2e
FAULTING_THREAD: 86cb43e8
STACK_TEXT:
b550c2c0 80833491 86cb43e8 86cb440c 00000000 nt!KiSwapContext+0x26
b550c2ec 8082ffe0 8d1cf33b d660b15c b550c384 nt!KiSwapThread+0x2e5
b550c314 8087d832 00000000 b550c698 b550c69c nt!KeWaitForGate+0x152
d660b15c 00000000 dbf83968 00000000 00000000 nt!ExfAcquirePushLockShared+0xf0
STACK_COMMAND: .thread 0xffffffff86cb43e8 ; kb
FOLLOWUP_IP:
nt!KiSwapContext+26
8088db2e 8b2c24 mov ebp,dword ptr [esp]
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: nt!KiSwapContext+26
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: nt
IMAGE_NAME: ntkrpamp.exe
DEBUG_FLR_IMAGE_TIMESTAMP: 4b7a90ad
FAILURE_BUCKET_ID: LOCK_HELD_nt!KiSwapContext+26
BUCKET_ID: LOCK_HELD_nt!KiSwapContext+26
Followup: MachineOwner
0: kd> !thread 86cb43e8
THREAD 86cb43e8 Cid 0a34.2fd8 Teb: 7ff53000 Win32Thread: 00000000 GATEWAIT
IRP List:
869d9350: (0006,0094) Flags: 00000000 Mdl: 00000000
Not impersonating
DeviceMap d66038f0
Owning Process 8ab907e8 Image: svchost.exe
Wait Start TickCount 206449713 Ticks: 6563547 (1:04:29:15.421)
Context Switch Count 8
UserTime 00:00:00.015
KernelTime 00:00:00.000
Win32 Start Address 0x7654d935
Start Address 0x77e617ec
Stack Init b550d000 Current b550c2a8 Base b550d000 Limit b550a000 Call 0
Priority 14 BasePriority 8 PriorityDecrement 5
ChildEBP RetAddr Args to Child
b550c2c0 80833491 86cb43e8 86cb440c 00000000 nt!KiSwapContext+0x26 (FPO: [Uses EBP] [0,0,4])
b550c2ec 8082ffe0 8d1cf33b d660b15c b550c384 nt!KiSwapThread+0x2e5 (FPO: [Non-Fpo])
b550c314 8087d832 00000000 b550c698 b550c69c nt!KeWaitForGate+0x152 (FPO: [Non-Fpo])
d660b15c 00000000 dbf83968 00000000 00000000 nt!ExfAcquirePushLockShared+0xf0 (FPO: [Non-Fpo])
0: kd> !thread 87c2a2f0
THREAD 87c2a2f0 Cid 0a34.40e0 Teb: 00000000 Win32Thread: 00000000 WAIT: (Unknown) KernelMode Non-Alertable
866c2958 SynchronizationEvent
87c2a368 NotificationTimer
Not impersonating
DeviceMap d66038f0
Owning Process 8ab907e8 Image: svchost.exe
Wait Start TickCount 213013029 Ticks: 231 (0:00:00:03.609)
Context Switch Count 25641
UserTime 00:00:00.000
KernelTime 00:00:00.000
Start Address termdd!_IcaDriverThread (0xf764c218)
Stack Init 8bec0000 Current 8bebf430 Base 8bec0000 Limit 8bebd000 Call 0
Priority 16 BasePriority 8 PriorityDecrement 0
ChildEBP RetAddr Args to Child
8bebf448 80833491 87c2a2f0 87c2a398 00000003 nt!KiSwapContext+0x26 (FPO: [Uses EBP] [0,0,4])
8bebf474 80829a82 87c2a2f0 8455538c 00000000 nt!KiSwapThread+0x2e5 (FPO: [Non-Fpo])
8bebf4bc 8087cc21 866c2958 0000001b 00000000 nt!KeWaitForSingleObject+0x346 (FPO: [Non-Fpo])
8bebf4f8 8087ce3b 00000000 8723e4fc 80a5e401 nt!ExpWaitForResource+0xd5 (FPO: [Non-Fpo])
8bebf518 8087ce67 8455538c 00000001 8bebf548 nt!ExAcquireResourceExclusiveLite+0x8d (FPO: [Non-Fpo])
8bebf528 f764bf33 8455538c 881ebce0 881eba38 nt!ExEnterCriticalRegionAndAcquireResourceExclusive+0x19 (FPO: [Non-Fpo])
8bebf548 b4e6e457 8723e4fc 881ebcec ffffffff termdd!IcaWaitForSingleObject+0x95 (FPO: [Non-Fpo])
8bebf55c b4e6ddd8 881eba38 84555380 8723e4e8 TDTCP!DeviceWaitForRead+0x19 (FPO: [Non-Fpo])
8bebfd90 f764c265 000dc008 00000000 87c2a2f0 TDTCP!TdInputThread+0x17e (FPO: [Non-Fpo])
8bebfdac 80949c5a 881eba38 00000000 00000000 termdd!_IcaDriverThread+0x4d (FPO: [Non-Fpo])
8bebfddc 8088e0c2 f764c218 841e8468 00000000 nt!PspSystemThreadStartup+0x2e (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16
So my basic understanding is, Thread 87c2a2f0 is waiting on 86cb43e8. This is the lock:
0: kd> !locks 8455538c
Resource @ 0x8455538c Exclusively owned
Contention Count = 1
NumberOfExclusiveWaiters = 1
Threads: 86cb43e8-01<*>
Threads Waiting On Exclusive Access:
87c2a2f0
1 total locks, 1 locks currently held
But where is Thread 86cb43e8 waiting for? I would expect to see this thread as a waiting thread in !locks (if it is a deadlock):
0: kd> !locks
**** DUMP OF ALL RESOURCE OBJECTS ****
KD: Scanning for held locks…
Resource @ nt!CmpRegistryLock (0x808a48c0) Shared 156 owning threads
Contention Count = 3964
Threads: 87cd3020-01<*> 889387a0-01<*> 8b778b40-01<*> 87d70db0-01<*>
87c0a388-01<*> 8ad5a6c0-01<*> 860d98e0-01<*> 8b777660-01<*>
862f71d0-01<*> 87889780-01<*> 8927f558-01<*> 884abdb0-01<*>
8aede780-01<*> 89b16ce8-01<*> 86aafd38-01<*> 84f495d0-01<*>
859e4020-01<*> 866971f0-01<*> 8563cdb0-01<*> 85e53020-01<*>
86d49240-01<*> 872a1d88-01<*> 8777d190-01<*> 885b9498-01<*>
85104b68-01<*> 8683adb0-01<*> 89843db0-01<*> 867e63e8-01<*>
8b7798d0-01<*> 89590db0-01<*> 89204198-01<*> 89608c18-01<*>
89a4a6c8-01<*> 8635c240-01<*> 86ec78b0-01<*> 850acad8-01<*>
88661db0-01<*> 86a43890-01<*> 87247900-01<*> 8650c0d8-01<*>
886045b8-01<*> 89888db0-01<*> 87739430-01<*> 8523d370-01<*>
868f9278-01<*> 899ecdb0-01<*> 89b85020-01<*> 85d6e700-01<*>
896f3db0-01<*> 88a74020-01<*> 87bec058-01<*> 88ea99e8-01<*>
86a21020-01<*> 89879db0-01<*> 88340db0-01<*> 89af7020-01<*>
8ac3fb18-01<*> 856d5c78-01<*> 87291108-01<*> 856c9d18-01<*>
861f0860-01<*> 857c0db0-01<*> 86dd2db0-01<*> 84f3b358-01<*>
875146c0-01<*> 861b2db0-01<*> 84f43d88-01<*> 87b4d3f0-01<*>
888a24c0-01<*> 864afa50-01<*> 8927a5d8-01<*> 84061db0-01<*>
85b393d0-01<*> 87b4a548-01<*> 85769020-01<*> 89963698-01<*>
88f52db0-01<*> 8705dcf8-01<*> 86295db0-01<*> 87056588-01<*>
85454640-01<*> 8911e0a0-01<*> 89b0e020-01<*> 88b6e2f0-01<*>
88633ba0-01<*> 896b4db0-01<*> 88a23db0-01<*> 844caa50-01<*>
852ed480-01<*> 85828110-01<*> 869a8020-01<*> 860c1168-01<*>
895f8db0-01<*> 844fe200-01<*> 86242020-01<*> 86bac550-01<*>
86a20670-01<*> 8737fdb0-01<*> 8552da28-01<*> 86cb43e8-01<*>
89656ca8-01<*> 89258178-01<*> 8600e158-01<*> 85f46db0-01<*>
885e2488-01<*> 88b96db0-01<*> 88e84020-01<*> 84f5d020-01<*>
876a0158-01<*> 88320778-01<*> 8781f7d0-01<*> 89891db0-01<*>
87794bc8-01<*> 85427020-01<*> 85294020-01<*> 86c1dc88-01<*>
89b8f880-01<*> 89c4a140-01<*> 8985eb40-01<*> 89bd8a80-01<*>
89bcd020-01<*> 899d2b18-01<*> 89a815e8-01<*> 8ad45db0-01<*>
899b8b18-01<*> 89a9ca90-01<*> 8987d698-01<*> 89bd1020-01<*>
85105db0-01<*> 8ac66378-01<*> 89a7f350-01<*> 89f105e8-01<*>
89a16ca0-01<*> 89890540-01<*> 89aa59f0-01<*> 89bae378-01<*>
89965880-01<*> 89870bf0-01<*> 89a89700-01<*> 86f2adb0-01<*>
850e5db0-01<*> 871c80a8-01<*> 8adabb20-01<*> 8988bdb0-01<*>
8870fcc8-01<*> 8aefe7e8-01<*> 89894db0-01<*> 8ae6f2b8-01<*>
88160298-01<*> 890a9db0-01<*> 89b82b18-01<*> 86f1f4b0-01<*>
85a33db0-01<*> 8810d778-01<*> 8895c6d0-01<*> 86f12d40-01<*>
KD: Scanning for held locks…
Resource @ 0x8add092c Shared 1 owning threads
Threads: 89b53880-01<*>
KD: Scanning for held locks…
Resource @ 0x89f06964 Exclusively owned
Contention Count = 45
NumberOfExclusiveWaiters = 2
Threads: 89f077e8-01<*>
Threads Waiting On Exclusive Access:
8b775b40 8b775db0
KD: Scanning for held locks…
Resource @ 0x89c4babc Exclusively owned
Contention Count = 48
Threads: 8af3edb0-01<*>
KD: Scanning for held locks…
Resource @ 0x850b9014 Exclusively owned
Contention Count = 1
Threads: 866971f0-01<*>
KD: Scanning for held locks.
Resource @ 0x87d680e4 Exclusively owned
Contention Count = 3
Threads: 8777d190-01<*>
KD: Scanning for held locks…
Resource @ 0x851ca4d4 Exclusively owned
Contention Count = 2
NumberOfExclusiveWaiters = 1
Threads: 89608c18-01<*>
Threads Waiting On Exclusive Access:
898b28a8
KD: Scanning for held locks.
Resource @ 0x86642014 Exclusively owned
Contention Count = 2
Threads: 88661db0-01<*>
KD: Scanning for held locks.
Resource @ 0x867e29f4 Exclusively owned
Contention Count = 3
Threads: 8650c0d8-01<*>
KD: Scanning for held locks.
Resource @ 0x84f0771c Exclusively owned
Contention Count = 3
NumberOfExclusiveWaiters = 1
Threads: 8523d370-01<*>
Threads Waiting On Exclusive Access:
853dd1e0
KD: Scanning for held locks.
Resource @ 0x8637c154 Exclusively owned
Contention Count = 1
Threads: 868f9278-01<*>
KD: Scanning for held locks.
Resource @ 0x8450b0ac Exclusively owned
Contention Count = 2
Threads: 87bec058-01<*>
KD: Scanning for held locks.
Resource @ 0x8602338c Exclusively owned
Contention Count = 2
NumberOfExclusiveWaiters = 1
Threads: 88ea99e8-01<*>
Threads Waiting On Exclusive Access:
8951aa68
KD: Scanning for held locks.
Resource @ 0x86f3a284 Exclusively owned
Contention Count = 2
Threads: 89879db0-01<*>
KD: Scanning for held locks…
Resource @ 0x873cc524 Exclusively owned
Contention Count = 3
Threads: 856c9d18-01<*>
KD: Scanning for held locks.
Resource @ 0x858de95c Exclusively owned
Contention Count = 1
Threads: 875146c0-01<*>
KD: Scanning for held locks.
Resource @ 0x877f359c Exclusively owned
Contention Count = 3
Threads: 861b2db0-01<*>
KD: Scanning for held locks.
Resource @ 0x879b2664 Exclusively owned
Contention Count = 4
NumberOfExclusiveWaiters = 1
Threads: 888a24c0-01<*>
Threads Waiting On Exclusive Access:
88165a18
KD: Scanning for held locks.
Resource @ 0x85ffa844 Exclusively owned
Contention Count = 1
Threads: 864afa50-01<*>
KD: Scanning for held locks.
Resource @ 0x88d199f4 Exclusively owned
Contention Count = 2
NumberOfExclusiveWaiters = 1
Threads: 8927a5d8-01<*>
Threads Waiting On Exclusive Access:
89f32700
KD: Scanning for held locks.
Resource @ 0x8834595c Exclusively owned
Contention Count = 1
Threads: 84061db0-01<*>
KD: Scanning for held locks.
Resource @ 0x8820d9f4 Exclusively owned
Contention Count = 3
Threads: 85b393d0-01<*>
KD: Scanning for held locks.
Resource @ 0x880b88d4 Exclusively owned
Contention Count = 2
Threads: 87b4a548-01<*>
KD: Scanning for held locks.
Resource @ 0x860e746c Exclusively owned
Contention Count = 1
Threads: 85769020-01<*>
KD: Scanning for held locks.
Resource @ 0x8601f564 Exclusively owned
Contention Count = 1
Threads: 89963698-01<*>
KD: Scanning for held locks.
Resource @ 0x866a39f4 Exclusively owned
Contention Count = 2
NumberOfExclusiveWaiters = 1
Threads: 88f52db0-01<*>
Threads Waiting On Exclusive Access:
875e7c48
KD: Scanning for held locks.
Resource @ 0x871a09f4 Exclusively owned
Contention Count = 1
Threads: 86295db0-01<*>
KD: Scanning for held locks.
Resource @ 0x88c259f4 Exclusively owned
Contention Count = 3
Threads: 87056588-01<*>
KD: Scanning for held locks.
Resource @ 0x85a0c9f4 Exclusively owned
Contention Count = 2
NumberOfExclusiveWaiters = 1
Threads: 85454640-01<*>
Threads Waiting On Exclusive Access:
89995db0
KD: Scanning for held locks.
Resource @ 0x85a4f3ac Exclusively owned
Contention Count = 3
NumberOfExclusiveWaiters = 1
Threads: 8911e0a0-01<*>
Threads Waiting On Exclusive Access:
867a7d40
KD: Scanning for held locks.
Resource @ 0x87529014 Exclusively owned
Contention Count = 4
NumberOfExclusiveWaiters = 1
Threads: 88b6e2f0-01<*>
Threads Waiting On Exclusive Access:
86e815a0
KD: Scanning for held locks.
Resource @ 0x88b2b354 Exclusively owned
Contention Count = 2
Threads: 88633ba0-01<*>
KD: Scanning for held locks.
Resource @ 0x8624642c Exclusively owned
Contention Count = 2
Threads: 896b4db0-01<*>
KD: Scanning for held locks.
Resource @ 0x883e29f4 Exclusively owned
Contention Count = 5
NumberOfExclusiveWaiters = 1
Threads: 88a23db0-01<*>
Threads Waiting On Exclusive Access:
84ed5db0
KD: Scanning for held locks.
Resource @ 0x8710e69c Exclusively owned
Contention Count = 3
Threads: 844caa50-01<*>
KD: Scanning for held locks.
Resource @ 0x8639173c Exclusively owned
Contention Count = 1
Threads: 852ed480-01<*>
KD: Scanning for held locks.
Resource @ 0x8861c14c Exclusively owned
Contention Count = 3
Threads: 85828110-01<*>
KD: Scanning for held locks.
Resource @ 0x87efa0e4 Exclusively owned
Contention Count = 2
Threads: 869a8020-01<*>
KD: Scanning for held locks.
Resource @ 0x865c49f4 Exclusively owned
Contention Count = 1
Threads: 860c1168-01<*>
KD: Scanning for held locks.
Resource @ 0x87ab59f4 Exclusively owned
Contention Count = 2
NumberOfExclusiveWaiters = 1
Threads: 895f8db0-01<*>
Threads Waiting On Exclusive Access:
88dd5a60
KD: Scanning for held locks.
Resource @ 0x888d45ac Exclusively owned
Threads: 844fe200-01<*>
KD: Scanning for held locks.
Resource @ 0x84fcc5ec Exclusively owned
Contention Count = 4
Threads: 86242020-01<*>
KD: Scanning for held locks.
Resource @ 0x88af73b4 Exclusively owned
Contention Count = 2
NumberOfExclusiveWaiters = 1
Threads: 8737fdb0-01<*>
Threads Waiting On Exclusive Access:
86242488
KD: Scanning for held locks.
Resource @ 0x863a6134 Exclusively owned
Contention Count = 2
NumberOfExclusiveWaiters = 1
Threads: 86bac550-01<*>
Threads Waiting On Exclusive Access:
887bbdb0
KD: Scanning for held locks.
Resource @ 0x84f3f9f4 Exclusively owned
Contention Count = 1
Threads: 86a20670-01<*>
KD: Scanning for held locks.
Resource @ 0x86ec19f4 Exclusively owned
Contention Count = 4
NumberOfExclusiveWaiters = 1
Threads: 8552da28-01<*>
Threads Waiting On Exclusive Access:
858083e0
KD: Scanning for held locks.
Resource @ 0x8455538c Exclusively owned
Contention Count = 1
NumberOfExclusiveWaiters = 1
Threads: 86cb43e8-01<*>
Threads Waiting On Exclusive Access:
87c2a2f0
KD: Scanning for held locks.
Resource @ 0x889ea08c Exclusively owned
Contention Count = 3
Threads: 89656ca8-01<*>
KD: Scanning for held locks.
Resource @ 0x875d589c Exclusively owned
Contention Count = 3
Threads: 89258178-01<*>
KD: Scanning for held locks.
Resource @ 0x85f4a014 Exclusively owned
Contention Count = 3
Threads: 8600e158-01<*>
KD: Scanning for held locks.
Resource @ 0x841f09f4 Exclusively owned
Contention Count = 1
Threads: 88b96db0-01<*>
KD: Scanning for held locks.
Resource @ 0x87e9a1c4 Exclusively owned
Contention Count = 3
Threads: 88e84020-01<*>
KD: Scanning for held locks.
Resource @ 0x869c6014 Exclusively owned
Contention Count = 1
Threads: 876a0158-01<*>
KD: Scanning for held locks.
Resource @ 0x8682f0d4 Exclusively owned
Contention Count = 2
Threads: 88320778-01<*>
KD: Scanning for held locks…
185868 total locks, 54 locks currently held
But there is no such thread.
But what I see is this:
Resource @ nt!CmpRegistryLock (0x808a48c0) Shared 156 owning threads
Contention Count = 3964
which leads me to this:
http://www.dumpanalysis.org/blog/index.php/2007/11/02/crash-dump-analysis-patterns-part-13c/
Therefore we see at least two problems in this memory dump: excessive paged pool usage and high thread contention around registry resource slowing down if not halting the system.
But why do I see this? Is this the RCA or just a symptom? Is my approach to troubleshoot this hang/freeze wrong?
I hope that someone finds the time, to point me to the right direction - thx in advance…
Bjoern
PS: !VM does not show any Excessive usage or allocations errors (output can be posted if needed).
PS2: No Thread is in ready state:
0: kd> !ready
Processor 0: No threads in READY state
Processor 1: No threads in READY state
Processor 2: No threads in READY state
Processor 3: No threads in READY state
PS3: All CPUs seems to be idle
0: kd> kb
ChildEBP RetAddr Args to Child
808977bc 80a60df7 00000080 004f4454 00000000 nt!KeBugCheckEx+0x1b
8089780c 8088a766 00000000 00000000 8089a9ca hal!HalHandleNMI+0x1a5
8089780c 80a5c84b 00000000 00000000 8089a9ca nt!KiTrap02+0x136
8089a570 f75c9ca2 badb0d00 001c5d93 6ae8859b hal!HalpClockInterrupt+0xf7
WARNING: Stack unwind information not available. Following frames may be wrong.
8089a600 8088de52 00000000 0000000e 00000000 intelppm+0x2ca2
8089a604 00000000 0000000e 00000000 00000000 nt!KiIdleLoop+0xa
0: kd> ~1
1: kd> kb
ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be wrong.
f78bad50 8088de52 00000000 0000000e f789f497 intelppm+0x2ca2
*** ERROR: Module load completed but symbols could not be loaded for vmscsi.sys
f78bad5c f789f497 000010c1 00000002 f78c6d90 nt!KiIdleLoop+0xa
f78bad6c f789ff70 00000002 8b28d00c 897fae3c vmscsi+0x497
f78c6ddc 80a60456 884b2f02 8b479b02 f78c6e0c vmscsi+0xf70
f78c6dec 80a5e44b f72377ae 00000000 00000000 hal!KfLowerIrql+0x62
f78c6df0 f72377ae 00000000 00000000 00000001 hal!KfReleaseSpinLock+0xb
f78c6e0c 80a5eb4a 8b479a40 86473138 86329760 SCSIPORT!SpReceiveScatterGather+0x33b
f78c6e54 8b2a3020 00479b02 f78c6e80 80a5fae6 hal!HalpAllocateAdapterCallback+0xf4
f78c6e88 b6b7f876 00000000 f773f120 00000000 0x8b2a3020
f78c6ea0 00000000 b6b43402 808b4302 f78c6ec0 tcpip!TcpMppEventHandler+0x7d
1: kd> ~2
2: kd> kb
ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be wrong.
f78c2d50 8088de52 00000000 0000000e f789f497 intelppm+0x2ca2
f78c2d5c f789f497 000010c1 00000002 f78ced90 nt!KiIdleLoop+0xa
f78c2d6c f789ff70 00000002 8b28d00c 897fae3c vmscsi+0x497
f78ceddc 80a60456 8740a402 8b479b02 f78cee0c vmscsi+0xf70
f78cedec 80a5e44b f72377ae 00000000 00000000 hal!KfLowerIrql+0x62
f78cedf0 f72377ae 00000000 00000000 00000001 hal!KfReleaseSpinLock+0xb
f78cee0c 80a5eb4a 8b479a40 897831f8 866cdea8 SCSIPORT!SpReceiveScatterGather+0x33b
f78cee54 8b2a3020 00479b02 f78cee80 80a5fae6 hal!HalpAllocateAdapterCallback+0xf4
f78cee88 b6b7f876 00000000 f7747120 00000000 0x8b2a3020
f78ceea0 00000000 b6b43402 808b4302 f78ceec0 tcpip!TcpMppEventHandler+0x7d
2: kd> ~3
3: kd> kb
ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be wrong.
f78cad50 8088de52 00000000 0000000e 00000000 intelppm+0x2ca2
f78cad54 00000000 0000000e 00000000 00000000 nt!KiIdleLoop+0xa