INetCfgClassSetup::DeInstall causes deadlock in NDIS

NTDEV
1

This problem happens when we try to uninstall our NDIS 5.1 IM driver (based on the Passthru sample). The analysis shows that thread 81c53428 is blocked because of a mutex acquired by NDIS in our thread which is waiting on a notification event. I am not sure who is supposed to signal this event. Problem was reproduced on VMware 7.0 but we have reports of it happening on physical machines as well. It happens rarely but often enough that we would like to do something about it. Thanks.

===================================================

Microsoft (R) Windows Debugger Version 6.11.0001.404 X86
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\MEMORY.DMP]
Kernel Complete Dump File: Full address space is available

Executable search path is:
Windows XP Kernel Version 2600 (Service Pack 3) MP (2 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 2600.xpsp.080413-2111
Machine Name:
Kernel base = 0x804d7000 PsLoadedModuleList = 0x8055d720
Debug session time: Thu Mar 3 09:57:27.413 2011 (GMT-8)
System Uptime: 0 days 2:07:31.093
Loading Kernel Symbols
…
…
Loading User Symbols
…
Loading unloaded module list
…
1: kd> !stacks 2 NDIS!
Proc.Thread .Thread Ticks ThreadState Blocker
[81f9f5f0 System]
4.000074 81e42a28 0000040 Blocked nt!KiSwapContext+0x2f
nt!KiSwapThread+0x8a
nt!KeRemoveQueue+0x22a
NDIS!ndisWorkerThread+0x30
nt!PspSystemThreadStartup+0x34
nt!KiThreadStartup+0x16
*** ERROR: Module load completed but symbols could not be loaded for vmci.sys
*** ERROR: Module load completed but symbols could not be loaded for vmdebug.sys
*** ERROR: Module load completed but symbols could not be loaded for vmmemctl.sys

[81ad8020 smss.exe]

[81cc3b60 csrss.exe]

[81a2ea68 winlogon.exe]

[819d6358 services.exe]

[81d45290 lsass.exe]

[81b83020 vmacthlp.exe]

[81adb020 svchost.exe]

[81ab7da0 svchost.exe]

[819b77b0 svchost.exe]
ec.0001c4 81c53428 000361f Blocked nt!KiSwapContext+0x2f
nt!KiSwapThread+0x8a
nt!KeWaitForSingleObject+0x1c2
NDIS!ndisHandleUModePnPOp+0x19
NDIS!ndisHandlePnPRequest+0x163
NDIS!ndisDispatchRequest+0x78
nt!IopfCallDriver+0x31
nt!IovCallDriver+0xa0
nt!IopSynchronousServiceTail+0x70
nt!IopXxxControlFile+0x5c5
nt!NtDeviceIoControlFile+0x2a
nt!KiFastCallEntry+0xfc
ntdll!KiFastSystemCallRet

[81add8c8 svchost.exe]

[81adf500 svchost.exe]

[81acc6c0 spoolsv.exe]

[819403c0 explorer.exe]

[8192d198 VMwareTray.exe]

[818f3ab0 VMwareUser.exe]

[819193c0 vmtoolsd.exe]

[818b8638 VMUpgradeHelper]

[8192fb28 alg.exe]

[819e1da0 cmd.exe]

[8191a020 setup.exe]
8a4.000968 81adf798 0003638 Blocked nt!KiSwapContext+0x2f
nt!KiSwapThread+0x8a
nt!KeWaitForSingleObject+0x1c2
NDIS!ndisPnPNotifyBinding+0x59
NDIS!ndisPnPNotifyAllTransports+0x44
NDIS!NdisIMNotifyPnPEvent+0x29
psched!ClPnPEventHandler+0x3a
NDIS!ndisPnPNotifyBinding+0x3f
NDIS!ndisPnPNotifyAllTransports+0x44
NDIS!NdisIMNotifyPnPEvent+0x29
passthru!PtPNPHandler+0x93
NDIS!ndisUnbindProtocol+0xff
NDIS!ndisHandleProtocolUnbindNotification+0xce
NDIS!ndisHandleUModePnPOp+0x9a
NDIS!ndisHandlePnPRequest+0x163
NDIS!ndisDispatchRequest+0x78
nt!IopfCallDriver+0x31
nt!IovCallDriver+0xa0
nt!IopSynchronousServiceTail+0x70
nt!IopXxxControlFile+0x5c5
nt!NtDeviceIoControlFile+0x2a
nt!KiFastCallEntry+0xfc
ntdll!KiFastSystemCallRet
ntdll!NtDeviceIoControlFile+0xc
kernel32!DeviceIoControl+0xdd
netcfgx!NdisHandlePnPEvent+0x14e
netcfgx!HrPnpBindOrUnbind+0xa6
netcfgx!CRegistryBindingsContext::PnpBindOrUnbindBindPaths+0x101
netcfgx!CModifyContext::ApplyChanges+0x343
netcfgx!CModifyContext::HrApplyIfOkOrCancel+0x2d
netcfgx!CModifyContext::HrPopRecursionDepth+0x20
netcfgx!CModifyContext::HrRemoveComponentIfNotReferenced+0xe9
netcfgx!CImplINetCfgClass::DeInstall+0xa1

Threads Processed: 301
1: kd> !thread 81adf798 7
THREAD 81adf798 Cid 08a4.0968 Teb: 7ffde000 Win32Thread: e18fe9e0 WAIT: (Executive) KernelMode Non-Alertable
f010b950 NotificationEvent
IRP List:
832faf68: (0006,0094) Flags: 40000030 Mdl: 00000000
Not impersonating
DeviceMap e1f03820
Owning Process 0 Image:
Attached Process 8191a020 Image: setup.exe
Wait Start TickCount 475790 Ticks: 13880 (0:00:03:36.875)
Context Switch Count 3019 LargeStack
UserTime 00:00:00.109
KernelTime 00:00:01.000
Win32 Start Address setup!ILT+21035(_wWinMainCRTStartup) (0x0049b230)
Start Address kernel32!BaseProcessStartThunk (0x7c8106f5)
Stack Init f010c000 Current f010b8dc Base f010c000 Limit f0107000 Call 0
Priority 10 BasePriority 8 PriorityDecrement 2 DecrementCount 16
ChildEBP RetAddr Args to Child
f010b8f4 80503836 81adf808 81adf798 804fb068 nt!KiSwapContext+0x2f (FPO: [Uses EBP] [0,0,4])
f010b900 804fb068 f010b97c 81a2d440 81a05d90 nt!KiSwapThread+0x8a (FPO: [0,0,0])
f010b928 baae93e6 00000000 00000000 00000000 nt!KeWaitForSingleObject+0x1c2 (FPO: [5,5,4])
f010b968 baae9511 f010ba44 81af32f8 f010ba44 NDIS!ndisPnPNotifyBinding+0x59 (FPO: [0,6,4])
f010b9cc baae9605 81b6f918 00000002 00000000 NDIS!ndisPnPNotifyAllTransports+0x44 (FPO: [4,20,4])
f010b9e4 f93e0e54 81b6f918 f010ba44 81c28f28 NDIS!NdisIMNotifyPnPEvent+0x29 (FPO: [2,0,0])
f010b9fc baae93cc 81cddc70 00000000 00000002 psched!ClPnPEventHandler+0x3a (FPO: [2,1,4])
f010ba30 baae9511 f010bb7c 81b815d0 00000000 NDIS!ndisPnPNotifyBinding+0x3f (FPO: [0,6,4])
f010ba94 baae9605 81b6a130 00000002 00000000 NDIS!ndisPnPNotifyAllTransports+0x44 (FPO: [4,20,4])
f010baac f99a6933 81b6a130 f010bb30 00000002 NDIS!NdisIMNotifyPnPEvent+0x29 (FPO: [2,0,0])
f010bac8 baae67ab 822b0e90 f010bb30 819ef860 passthru!PtPNPHandler+0x93 (FPO: [Non-Fpo]) (CONV: stdcall)
f010bb98 baae69b3 819ef860 00000000 81d6e400 NDIS!ndisUnbindProtocol+0xff (FPO: [4,45,4])
f010bbcc baadef5f 832faf68 818ac728 00000099 NDIS!ndisHandleProtocolUnbindNotification+0xce (FPO: [0,4,4])
f010bbdc baadeee6 00000000 832faf68 832fafd8 NDIS!ndisHandleUModePnPOp+0x9a (FPO: [0,0,0])
f010bc00 baadea10 81e42cb0 81e42dd8 832faf00 NDIS!ndisHandlePnPRequest+0x163 (FPO: [0,4,4])
f010bc1c 804ef18f 81e42cb0 832faf68 806e6428 NDIS!ndisDispatchRequest+0x78 (FPO: [2,2,4])
f010bc2c 80658128 81b3b070 806e6410 832faf68 nt!IopfCallDriver+0x31 (FPO: [0,0,0])
f010bc50 8057f982 832fafd8 81b3b070 832faf68 nt!IovCallDriver+0xa0 (FPO: [0,4,0])
f010bc64 805807f7 81e42cb0 832faf68 81b3b070 nt!IopSynchronousServiceTail+0x70 (FPO: [7,0,4])
f010bd00 80579274 00000194 00000000 00000000 nt!IopXxxControlFile+0x5c5 (FPO: [Non-Fpo])
f010bd34 8054161c 00000194 00000000 00000000 nt!NtDeviceIoControlFile+0x2a (FPO: [10,0,0])
f010bd34 7c90e4f4 00000194 00000000 00000000 nt!KiFastCallEntry+0xfc (FPO: [0,0] TrapFrame @ f010bd64)
0012f22c 7c90d26c 7c801675 00000194 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
0012f230 7c801675 00000194 00000000 00000000 ntdll!NtDeviceIoControlFile+0xc (FPO: [10,0,0])
0012f290 7566113d 00000194 00170008 00166db8 kernel32!DeviceIoControl+0xdd (FPO: [Non-Fpo])
0012f2f8 756200c1 00000001 00000002 0012f328 netcfgx!NdisHandlePnPEvent+0x14e (FPO: [7,13,0])
0012f364 75620333 00000001 00000002 00156870 netcfgx!HrPnpBindOrUnbind+0xa6 (FPO: [Non-Fpo])
0012f79c 75618f3d 00000002 00157020 0012f828 netcfgx!CRegistryBindingsContext::PnpBindOrUnbindBindPaths+0x101 (FPO: [3,261,4])
0012f82c 75619602 00166288 00156fdc 00166220 netcfgx!CModifyContext::ApplyChanges+0x343 (FPO: [0,28,4])
0012f840 756196e3 00000001 7561c013 00168abc netcfgx!CModifyContext::HrApplyIfOkOrCancel+0x2d (FPO: [1,0,4])
0012f848 7561c013 00168abc 00166220 00000000 netcfgx!CModifyContext::HrPopRecursionDepth+0x20 (FPO: [0,0,0])
0012f85c 7561aa6c 00000000 0012f970 00000000 netcfgx!CModifyContext::HrRemoveComponentIfNotReferenced+0xe9 (FPO: [3,0,4])
0012f87c 004a5def 00168abc 001662e8 0012f970 netcfgx!CImplINetCfgClass::DeInstall+0xa1 (FPO: [4,0,4])

1: kd> .thread /p /r 81c53428
Implicit thread is now 81c53428
Implicit process is now 819b77b0
Loading User Symbols
…
…
…
1: kd> !thread 81c53428 7
THREAD 81c53428 Cid 00ec.01c4 Teb: 7ffd7000 Win32Thread: 00000000 WAIT: (Executive) KernelMode Non-Alertable
baadcb40 Mutant - owning thread 81adf798
IRP List:
82fbaf68: (0006,0094) Flags: 40000030 Mdl: 00000000
Not impersonating
DeviceMap e1002a60
Owning Process 0 Image:
Attached Process 819b77b0 Image: svchost.exe
Wait Start TickCount 475815 Ticks: 13855 (0:00:03:36.484)
Context Switch Count 3408
UserTime 00:00:00.000
KernelTime 00:00:00.703
Win32 Start Address dhcpcsvc!MediaSenseDetectionLoop (0x7d4b98e0)
Start Address kernel32!BaseThreadStartThunk (0x7c8106e9)
Stack Init f09a2000 Current f09a1b6c Base f09a2000 Limit f099f000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr Args to Child
f09a1b84 80503836 81c53498 81c53428 804fb068 nt!KiSwapContext+0x2f (FPO: [Uses EBP] [0,0,4])
f09a1b90 804fb068 00000000 818fd178 baadcb40 nt!KiSwapThread+0x8a (FPO: [0,0,0])
f09a1bb8 baadef09 00000000 00000000 00000000 nt!KeWaitForSingleObject+0x1c2 (FPO: [5,5,4])
f09a1bdc baadeee6 00000000 82fbaf68 82fbafd8 NDIS!ndisHandleUModePnPOp+0x19 (FPO: [0,0,0])
f09a1c00 baadea10 81e42cb0 81e42dd8 82fbaf00 NDIS!ndisHandlePnPRequest+0x163 (FPO: [0,4,4])
f09a1c1c 804ef18f 81e42cb0 82fbaf68 806e6428 NDIS!ndisDispatchRequest+0x78 (FPO: [2,2,4])
f09a1c2c 80658128 819cadc0 806e6410 82fbaf68 nt!IopfCallDriver+0x31 (FPO: [0,0,0])
f09a1c50 8057f982 82fbafd8 819cadc0 82fbaf68 nt!IovCallDriver+0xa0 (FPO: [0,4,0])
f09a1c64 805807f7 81e42cb0 82fbaf68 819cadc0 nt!IopSynchronousServiceTail+0x70 (FPO: [7,0,4])
f09a1d00 80579274 00000a48 00000000 00000000 nt!IopXxxControlFile+0x5c5 (FPO: [Non-Fpo])
f09a1d34 8054161c 00000a48 00000000 00000000 nt!NtDeviceIoControlFile+0x2a (FPO: [10,0,0])
f09a1d34 7c90e4f4 00000a48 00000000 00000000 nt!KiFastCallEntry+0xfc (FPO: [0,0] TrapFrame @ f09a1d64)
00adfb24 7c90d26c 7c801675 00000a48 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
00adfb28 7c801675 00000a48 00000000 00000000 ntdll!NtDeviceIoControlFile+0xc (FPO: [10,0,0])
00adfb88 7d4b6885 00000a48 00170008 001185f8 kernel32!DeviceIoControl+0xdd (FPO: [Non-Fpo])
00adfbf0 7d4b6726 00000001 00000003 00adfc30 dhcpcsvc!NdisHandlePnPEvent+0x14e (FPO: [7,13,0])
00adfec4 7d4b6666 000a74d0 00000000 0000007f dhcpcsvc!TcpIpNotifyRouterDiscoveryOption+0xc9 (FPO: [3,169,4])
00adfedc 7d4b698b 000a72b8 00000000 000a72b8 dhcpcsvc!DhcpSetRouterDiscoverOption+0x2a (FPO: [1,1,0])
00adfef8 7d4bc8f6 00000000 00000000 00000000 dhcpcsvc!DhcpSetAllStackParameters+0xe7 (FPO: [2,1,4])
00adff30 7d4c221e 4d6fd5ae 00000000 00000000 dhcpcsvc!SetDhcpConfigurationForNIC+0x231 (FPO: [6,7,4])
00adff60 7d4bcade 000a72b8 00000000 00000103 dhcpcsvc!DhcpDestroyContextEx+0x89 (FPO: [2,1,0])
00adff80 7d4b99df 000a4614 0000ffff 00002b13 dhcpcsvc!ProcessAdapterBindingEvent+0x116 (FPO: [3,1,4])
00adffb4 7c80b713 00000000 00001800 00001002 dhcpcsvc!MediaSenseDetectionLoop+0x145 (FPO: [0,5,0])
00adffec 00000000 7d4b98e0 00000000 00000000 kernel32!BaseThreadStart+0x37 (FPO: [Non-Fpo])

1: kd> !address baadcb40
baad6000 - 0002d000
Usage KernelSpaceUsageImage
ImageName NDIS.sys

1: kd> ln baadcb40
(baadcb40) NDIS!ndisPnPMutex | (baadcb60) NDIS!ndisGlobalOpenListLock
Exact matches:
NDIS!ndisPnPMutex =

1: kd> !address f010b950
f0106000 - 00006000
Usage KernelSpaceUsageKernelStack
KernelStack 81adf798 : 8a4.968

1: kd> dt -r nt!_KEVENT f010b950
+0x000 Header : _DISPATCHER_HEADER
+0x000 Type : 0 ‘’
+0x001 Absolute : 0xab ‘’
+0x002 Size : 0x4 ‘’
+0x003 Inserted : 0xba ‘’
+0x004 SignalState : 0
+0x008 WaitListHead : _LIST_ENTRY [0x81adf808 - 0x81adf808]
+0x000 Flink : 0x81adf808 _LIST_ENTRY [0xf010b958 - 0xf010b958]
+0x004 Blink : 0x81adf808 _LIST_ENTRY [0xf010b958 - 0xf010b958]

1: kd> !thread 81e42a28 7
THREAD 81e42a28 Cid 0004.0074 Teb: 00000000 Win32Thread: 00000000 WAIT: (WrQueue) KernelMode Non-Alertable
baadc3dc Unknown
Not impersonating
DeviceMap e1002a60
Owning Process 0 Image:
Attached Process 81f9f5f0 Image: System
Wait Start TickCount 489606 Ticks: 64 (0:00:00:01.000)
Context Switch Count 618492
UserTime 00:00:00.000
KernelTime 00:00:23.093
Start Address NDIS!ndisWorkerThread (0xbaadcb85)
Stack Init f9c7b000 Current f9c7ad40 Base f9c7b000 Limit f9c78000 Call 0
Priority 13 BasePriority 8 PriorityDecrement 5 DecrementCount 16
ChildEBP RetAddr Args to Child
f9c7ad58 80503836 81e42a98 81e42a28 804fcc86 nt!KiSwapContext+0x2f (FPO: [Uses EBP] [0,0,4])
f9c7ad64 804fcc86 81a2a888 00000000 00000000 nt!KiSwapThread+0x8a (FPO: [0,0,0])
f9c7ad90 baadcbbd 00000000 00000000 00000000 nt!KeRemoveQueue+0x22a (FPO: [3,6,4])
f9c7adac 805cff64 00000830 00000000 00000000 NDIS!ndisWorkerThread+0x30 (FPO: [1,0,0])
f9c7addc 805460de baadcb85 00000000 00000000 nt!PspSystemThreadStartup+0x34 (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

1: kd> !address baadc3dc
baad6000 - 0002d000
Usage KernelSpaceUsageImage
ImageName NDIS.sys

1: kd> ln baadc3dc
(baadc3dc) NDIS!ndisWorkerQueue | (baadc408) NDIS!PoolAgingTicks
Exact matches:
NDIS!ndisWorkerQueue =

2

Thanks for the detailed debug output :slight_smile:

The first thread, 81e42a28, is a harmless background worker thread and you can ignore it.

The second thread, 81c53428, is a PNP operation triggered by a background service (looks like DHCP client, in this case). The OS will normally install/uninstall tunnel adapters based on IP address acquisition. As you already noticed, that thread is blocked on ndisPnPMutex, which is a global lock protecting all PNP operations. The mutex would be owned by the third thread (81adf798) so it makes sense that it hasn’t been acquired yet. Also note that this second thread (81c53428) has not yet acquired any locks or whatnot, so it’s doubtful that it plays a role in hanging the third thread. Probably it’s just a victim.

Now the third thread, 81adf798, is the interesting one. We sent you a NetEventQueryRemoveDevice in preparation for unbind. Then you passed it up to the next IM driver, PACER. In turn, PACER passed it up to the protocol over it (maybe TCPIP, but the dump doesn’t say). This overlying protocol driver hasn’t yet completed the PNP event. So you need to continue your investigation to figure out where *that* event went.

You can use !ndiskd.miniport to figure out who is bound above PACER (aka PSCHED). If it’s not TCPIP, then obviously you need to follow up with its owners about NetEventQueryRemoveDevice. If it is TCPIP (as is likely) then I can tell you that TCPIP itself doesn’t do anything in response to NetEventQueryRemoveDevice. However, it has to give that event to TDI clients, and one of them may have pended the event and gotten stuck. Are there any suspicious looking threads not in ndis? (E.g., in mrxsmb or something like that).

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Thursday, March 03, 2011 12:00 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] INetCfgClassSetup::DeInstall causes deadlock in NDIS

This problem happens when we try to uninstall our NDIS 5.1 IM driver (based on the Passthru sample). The analysis shows that thread 81c53428 is blocked because of a mutex acquired by NDIS in our thread which is waiting on a notification event. I am not sure who is supposed to signal this event. Problem was reproduced on VMware 7.0 but we have reports of it happening on physical machines as well. It happens rarely but often enough that we would like to do something about it. Thanks.

===================================================

Microsoft (R) Windows Debugger Version 6.11.0001.404 X86 Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\MEMORY.DMP]
Kernel Complete Dump File: Full address space is available

Executable search path is:
Windows XP Kernel Version 2600 (Service Pack 3) MP (2 procs) Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS Built by: 2600.xpsp.080413-2111 Machine Name:
Kernel base = 0x804d7000 PsLoadedModuleList = 0x8055d720 Debug session time: Thu Mar 3 09:57:27.413 2011 (GMT-8) System Uptime: 0 days 2:07:31.093 Loading Kernel Symbols …
…
Loading User Symbols
…
Loading unloaded module list
…
1: kd> !stacks 2 NDIS!
Proc.Thread .Thread Ticks ThreadState Blocker
[81f9f5f0 System]
4.000074 81e42a28 0000040 Blocked nt!KiSwapContext+0x2f
nt!KiSwapThread+0x8a
nt!KeRemoveQueue+0x22a
NDIS!ndisWorkerThread+0x30
nt!PspSystemThreadStartup+0x34
nt!KiThreadStartup+0x16
*** ERROR: Module load completed but symbols could not be loaded for vmci.sys
*** ERROR: Module load completed but symbols could not be loaded for vmdebug.sys
*** ERROR: Module load completed but symbols could not be loaded for vmmemctl.sys

[81ad8020 smss.exe]

[81cc3b60 csrss.exe]

[81a2ea68 winlogon.exe]

[819d6358 services.exe]

[81d45290 lsass.exe]

[81b83020 vmacthlp.exe]

[81adb020 svchost.exe]

[81ab7da0 svchost.exe]

[819b77b0 svchost.exe]
ec.0001c4 81c53428 000361f Blocked nt!KiSwapContext+0x2f
nt!KiSwapThread+0x8a
nt!KeWaitForSingleObject+0x1c2
NDIS!ndisHandleUModePnPOp+0x19
NDIS!ndisHandlePnPRequest+0x163
NDIS!ndisDispatchRequest+0x78
nt!IopfCallDriver+0x31
nt!IovCallDriver+0xa0
nt!IopSynchronousServiceTail+0x70
nt!IopXxxControlFile+0x5c5
nt!NtDeviceIoControlFile+0x2a
nt!KiFastCallEntry+0xfc
ntdll!KiFastSystemCallRet

[81add8c8 svchost.exe]

[81adf500 svchost.exe]

[81acc6c0 spoolsv.exe]

[819403c0 explorer.exe]

[8192d198 VMwareTray.exe]

[818f3ab0 VMwareUser.exe]

[819193c0 vmtoolsd.exe]

[818b8638 VMUpgradeHelper]

[8192fb28 alg.exe]

[819e1da0 cmd.exe]

[8191a020 setup.exe]
8a4.000968 81adf798 0003638 Blocked nt!KiSwapContext+0x2f
nt!KiSwapThread+0x8a
nt!KeWaitForSingleObject+0x1c2
NDIS!ndisPnPNotifyBinding+0x59
NDIS!ndisPnPNotifyAllTransports+0x44
NDIS!NdisIMNotifyPnPEvent+0x29
psched!ClPnPEventHandler+0x3a
NDIS!ndisPnPNotifyBinding+0x3f
NDIS!ndisPnPNotifyAllTransports+0x44
NDIS!NdisIMNotifyPnPEvent+0x29
passthru!PtPNPHandler+0x93
NDIS!ndisUnbindProtocol+0xff
NDIS!ndisHandleProtocolUnbindNotification+0xce
NDIS!ndisHandleUModePnPOp+0x9a
NDIS!ndisHandlePnPRequest+0x163
NDIS!ndisDispatchRequest+0x78
nt!IopfCallDriver+0x31
nt!IovCallDriver+0xa0
nt!IopSynchronousServiceTail+0x70
nt!IopXxxControlFile+0x5c5
nt!NtDeviceIoControlFile+0x2a
nt!KiFastCallEntry+0xfc
ntdll!KiFastSystemCallRet
ntdll!NtDeviceIoControlFile+0xc
kernel32!DeviceIoControl+0xdd
netcfgx!NdisHandlePnPEvent+0x14e
netcfgx!HrPnpBindOrUnbind+0xa6
netcfgx!CRegistryBindingsContext::PnpBindOrUnbindBindPaths+0x101
netcfgx!CModifyContext::ApplyChanges+0x343
netcfgx!CModifyContext::HrApplyIfOkOrCancel+0x2d
netcfgx!CModifyContext::HrPopRecursionDepth+0x20
netcfgx!CModifyContext::HrRemoveComponentIfNotReferenced+0xe9
netcfgx!CImplINetCfgClass::DeInstall+0xa1

Threads Processed: 301
1: kd> !thread 81adf798 7
THREAD 81adf798 Cid 08a4.0968 Teb: 7ffde000 Win32Thread: e18fe9e0 WAIT: (Executive) KernelMode Non-Alertable
f010b950 NotificationEvent
IRP List:
832faf68: (0006,0094) Flags: 40000030 Mdl: 00000000 Not impersonating
DeviceMap e1f03820
Owning Process 0 Image:
Attached Process 8191a020 Image: setup.exe
Wait Start TickCount 475790 Ticks: 13880 (0:00:03:36.875)
Context Switch Count 3019 LargeStack
UserTime 00:00:00.109
KernelTime 00:00:01.000
Win32 Start Address setup!ILT+21035(_wWinMainCRTStartup) (0x0049b230) Start Address kernel32!BaseProcessStartThunk (0x7c8106f5) Stack Init f010c000 Current f010b8dc Base f010c000 Limit f0107000 Call 0 Priority 10 BasePriority 8 PriorityDecrement 2 DecrementCount 16
ChildEBP RetAddr Args to Child
f010b8f4 80503836 81adf808 81adf798 804fb068 nt!KiSwapContext+0x2f (FPO: [Uses EBP] [0,0,4])
f010b900 804fb068 f010b97c 81a2d440 81a05d90 nt!KiSwapThread+0x8a (FPO: [0,0,0])
f010b928 baae93e6 00000000 00000000 00000000 nt!KeWaitForSingleObject+0x1c2 (FPO: [5,5,4])
f010b968 baae9511 f010ba44 81af32f8 f010ba44 NDIS!ndisPnPNotifyBinding+0x59 (FPO: [0,6,4]) f010b9cc baae9605 81b6f918 00000002 00000000 NDIS!ndisPnPNotifyAllTransports+0x44 (FPO: [4,20,4])
f010b9e4 f93e0e54 81b6f918 f010ba44 81c28f28 NDIS!NdisIMNotifyPnPEvent+0x29 (FPO: [2,0,0]) f010b9fc baae93cc 81cddc70 00000000 00000002 psched!ClPnPEventHandler+0x3a (FPO: [2,1,4])
f010ba30 baae9511 f010bb7c 81b815d0 00000000 NDIS!ndisPnPNotifyBinding+0x3f (FPO: [0,6,4])
f010ba94 baae9605 81b6a130 00000002 00000000 NDIS!ndisPnPNotifyAllTransports+0x44 (FPO: [4,20,4]) f010baac f99a6933 81b6a130 f010bb30 00000002 NDIS!NdisIMNotifyPnPEvent+0x29 (FPO: [2,0,0])
f010bac8 baae67ab 822b0e90 f010bb30 819ef860 passthru!PtPNPHandler+0x93 (FPO: [Non-Fpo]) (CONV: stdcall)
f010bb98 baae69b3 819ef860 00000000 81d6e400 NDIS!ndisUnbindProtocol+0xff (FPO: [4,45,4]) f010bbcc baadef5f 832faf68 818ac728 00000099 NDIS!ndisHandleProtocolUnbindNotification+0xce (FPO: [0,4,4]) f010bbdc baadeee6 00000000 832faf68 832fafd8 NDIS!ndisHandleUModePnPOp+0x9a (FPO: [0,0,0])
f010bc00 baadea10 81e42cb0 81e42dd8 832faf00 NDIS!ndisHandlePnPRequest+0x163 (FPO: [0,4,4]) f010bc1c 804ef18f 81e42cb0 832faf68 806e6428 NDIS!ndisDispatchRequest+0x78 (FPO: [2,2,4]) f010bc2c 80658128 81b3b070 806e6410 832faf68 nt!IopfCallDriver+0x31 (FPO: [0,0,0])
f010bc50 8057f982 832fafd8 81b3b070 832faf68 nt!IovCallDriver+0xa0 (FPO: [0,4,0])
f010bc64 805807f7 81e42cb0 832faf68 81b3b070 nt!IopSynchronousServiceTail+0x70 (FPO: [7,0,4])
f010bd00 80579274 00000194 00000000 00000000 nt!IopXxxControlFile+0x5c5 (FPO: [Non-Fpo])
f010bd34 8054161c 00000194 00000000 00000000 nt!NtDeviceIoControlFile+0x2a (FPO: [10,0,0])
f010bd34 7c90e4f4 00000194 00000000 00000000 nt!KiFastCallEntry+0xfc (FPO: [0,0] TrapFrame @ f010bd64) 0012f22c 7c90d26c 7c801675 00000194 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
0012f230 7c801675 00000194 00000000 00000000 ntdll!NtDeviceIoControlFile+0xc (FPO: [10,0,0])
0012f290 7566113d 00000194 00170008 00166db8 kernel32!DeviceIoControl+0xdd (FPO: [Non-Fpo])
0012f2f8 756200c1 00000001 00000002 0012f328 netcfgx!NdisHandlePnPEvent+0x14e (FPO: [7,13,0])
0012f364 75620333 00000001 00000002 00156870 netcfgx!HrPnpBindOrUnbind+0xa6 (FPO: [Non-Fpo]) 0012f79c 75618f3d 00000002 00157020 0012f828 netcfgx!CRegistryBindingsContext::PnpBindOrUnbindBindPaths+0x101 (FPO: [3,261,4]) 0012f82c 75619602 00166288 00156fdc 00166220 netcfgx!CModifyContext::ApplyChanges+0x343 (FPO: [0,28,4])
0012f840 756196e3 00000001 7561c013 00168abc netcfgx!CModifyContext::HrApplyIfOkOrCancel+0x2d (FPO: [1,0,4])
0012f848 7561c013 00168abc 00166220 00000000 netcfgx!CModifyContext::HrPopRecursionDepth+0x20 (FPO: [0,0,0]) 0012f85c 7561aa6c 00000000 0012f970 00000000 netcfgx!CModifyContext::HrRemoveComponentIfNotReferenced+0xe9 (FPO: [3,0,4]) 0012f87c 004a5def 00168abc 001662e8 0012f970 netcfgx!CImplINetCfgClass::DeInstall+0xa1 (FPO: [4,0,4])

1: kd> .thread /p /r 81c53428
Implicit thread is now 81c53428
Implicit process is now 819b77b0
Loading User Symbols
…
…
…
1: kd> !thread 81c53428 7
THREAD 81c53428 Cid 00ec.01c4 Teb: 7ffd7000 Win32Thread: 00000000 WAIT: (Executive) KernelMode Non-Alertable
baadcb40 Mutant - owning thread 81adf798 IRP List:
82fbaf68: (0006,0094) Flags: 40000030 Mdl: 00000000 Not impersonating
DeviceMap e1002a60
Owning Process 0 Image:
Attached Process 819b77b0 Image: svchost.exe
Wait Start TickCount 475815 Ticks: 13855 (0:00:03:36.484)
Context Switch Count 3408
UserTime 00:00:00.000
KernelTime 00:00:00.703
Win32 Start Address dhcpcsvc!MediaSenseDetectionLoop (0x7d4b98e0) Start Address kernel32!BaseThreadStartThunk (0x7c8106e9) Stack Init f09a2000 Current f09a1b6c Base f09a2000 Limit f099f000 Call 0 Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr Args to Child
f09a1b84 80503836 81c53498 81c53428 804fb068 nt!KiSwapContext+0x2f (FPO: [Uses EBP] [0,0,4])
f09a1b90 804fb068 00000000 818fd178 baadcb40 nt!KiSwapThread+0x8a (FPO: [0,0,0])
f09a1bb8 baadef09 00000000 00000000 00000000 nt!KeWaitForSingleObject+0x1c2 (FPO: [5,5,4]) f09a1bdc baadeee6 00000000 82fbaf68 82fbafd8 NDIS!ndisHandleUModePnPOp+0x19 (FPO: [0,0,0])
f09a1c00 baadea10 81e42cb0 81e42dd8 82fbaf00 NDIS!ndisHandlePnPRequest+0x163 (FPO: [0,4,4]) f09a1c1c 804ef18f 81e42cb0 82fbaf68 806e6428 NDIS!ndisDispatchRequest+0x78 (FPO: [2,2,4]) f09a1c2c 80658128 819cadc0 806e6410 82fbaf68 nt!IopfCallDriver+0x31 (FPO: [0,0,0])
f09a1c50 8057f982 82fbafd8 819cadc0 82fbaf68 nt!IovCallDriver+0xa0 (FPO: [0,4,0])
f09a1c64 805807f7 81e42cb0 82fbaf68 819cadc0 nt!IopSynchronousServiceTail+0x70 (FPO: [7,0,4])
f09a1d00 80579274 00000a48 00000000 00000000 nt!IopXxxControlFile+0x5c5 (FPO: [Non-Fpo])
f09a1d34 8054161c 00000a48 00000000 00000000 nt!NtDeviceIoControlFile+0x2a (FPO: [10,0,0])
f09a1d34 7c90e4f4 00000a48 00000000 00000000 nt!KiFastCallEntry+0xfc (FPO: [0,0] TrapFrame @ f09a1d64)
00adfb24 7c90d26c 7c801675 00000a48 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
00adfb28 7c801675 00000a48 00000000 00000000 ntdll!NtDeviceIoControlFile+0xc (FPO: [10,0,0])
00adfb88 7d4b6885 00000a48 00170008 001185f8 kernel32!DeviceIoControl+0xdd (FPO: [Non-Fpo])
00adfbf0 7d4b6726 00000001 00000003 00adfc30 dhcpcsvc!NdisHandlePnPEvent+0x14e (FPO: [7,13,0])
00adfec4 7d4b6666 000a74d0 00000000 0000007f dhcpcsvc!TcpIpNotifyRouterDiscoveryOption+0xc9 (FPO: [3,169,4]) 00adfedc 7d4b698b 000a72b8 00000000 000a72b8 dhcpcsvc!DhcpSetRouterDiscoverOption+0x2a (FPO: [1,1,0])
00adfef8 7d4bc8f6 00000000 00000000 00000000 dhcpcsvc!DhcpSetAllStackParameters+0xe7 (FPO: [2,1,4])
00adff30 7d4c221e 4d6fd5ae 00000000 00000000 dhcpcsvc!SetDhcpConfigurationForNIC+0x231 (FPO: [6,7,4])
00adff60 7d4bcade 000a72b8 00000000 00000103 dhcpcsvc!DhcpDestroyContextEx+0x89 (FPO: [2,1,0])
00adff80 7d4b99df 000a4614 0000ffff 00002b13 dhcpcsvc!ProcessAdapterBindingEvent+0x116 (FPO: [3,1,4])
00adffb4 7c80b713 00000000 00001800 00001002 dhcpcsvc!MediaSenseDetectionLoop+0x145 (FPO: [0,5,0]) 00adffec 00000000 7d4b98e0 00000000 00000000 kernel32!BaseThreadStart+0x37 (FPO: [Non-Fpo])

1: kd> !address baadcb40
baad6000 - 0002d000
Usage KernelSpaceUsageImage
ImageName NDIS.sys

1: kd> ln baadcb40
(baadcb40) NDIS!ndisPnPMutex | (baadcb60) NDIS!ndisGlobalOpenListLock
Exact matches:
NDIS!ndisPnPMutex =

1: kd> !address f010b950
f0106000 - 00006000
Usage KernelSpaceUsageKernelStack
KernelStack 81adf798 : 8a4.968

1: kd> dt -r nt!_KEVENT f010b950
+0x000 Header : _DISPATCHER_HEADER
+0x000 Type : 0 ‘’
+0x001 Absolute : 0xab ‘’
+0x002 Size : 0x4 ‘’
+0x003 Inserted : 0xba ‘’
+0x004 SignalState : 0
+0x008 WaitListHead : _LIST_ENTRY [0x81adf808 - 0x81adf808]
+0x000 Flink : 0x81adf808 _LIST_ENTRY [0xf010b958 - 0xf010b958]
+0x004 Blink : 0x81adf808 _LIST_ENTRY [0xf010b958 - 0xf010b958]

1: kd> !thread 81e42a28 7
THREAD 81e42a28 Cid 0004.0074 Teb: 00000000 Win32Thread: 00000000 WAIT: (WrQueue) KernelMode Non-Alertable
baadc3dc Unknown
Not impersonating
DeviceMap e1002a60
Owning Process 0 Image:
Attached Process 81f9f5f0 Image: System
Wait Start TickCount 489606 Ticks: 64 (0:00:00:01.000)
Context Switch Count 618492
UserTime 00:00:00.000
KernelTime 00:00:23.093
Start Address NDIS!ndisWorkerThread (0xbaadcb85) Stack Init f9c7b000 Current f9c7ad40 Base f9c7b000 Limit f9c78000 Call 0 Priority 13 BasePriority 8 PriorityDecrement 5 DecrementCount 16
ChildEBP RetAddr Args to Child
f9c7ad58 80503836 81e42a98 81e42a28 804fcc86 nt!KiSwapContext+0x2f (FPO: [Uses EBP] [0,0,4])
f9c7ad64 804fcc86 81a2a888 00000000 00000000 nt!KiSwapThread+0x8a (FPO: [0,0,0])
f9c7ad90 baadcbbd 00000000 00000000 00000000 nt!KeRemoveQueue+0x22a (FPO: [3,6,4]) f9c7adac 805cff64 00000830 00000000 00000000 NDIS!ndisWorkerThread+0x30 (FPO: [1,0,0]) f9c7addc 805460de baadcb85 00000000 00000000 nt!PspSystemThreadStartup+0x34 (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

1: kd> !address baadc3dc
baad6000 - 0002d000
Usage KernelSpaceUsageImage
ImageName NDIS.sys

1: kd> ln baadc3dc
(baadc3dc) NDIS!ndisWorkerQueue | (baadc408) NDIS!PoolAgingTicks
Exact matches:
NDIS!ndisWorkerQueue =

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

3

PSCHED has bindings to ndisuio, tcpip6, and tcpip.

Here is a suspicious looking thread:

1: kd> !thread 81f9dda8
THREAD 81f9dda8 Cid 0004.0024 Teb: 00000000 Win32Thread: 00000000 WAIT: (Executive) KernelMode Non-Alertable
81cc1b84 NotificationEvent
Not impersonating
DeviceMap e1002a60
Owning Process 0 Image:
Attached Process 81f9f5f0 Image: System
Wait Start TickCount 475771 Ticks: 13899 (0:00:03:37.171)
Context Switch Count 516
UserTime 00:00:00.000
KernelTime 00:00:00.046
Start Address nt!ExpWorkerThread (0x8053867e)
Stack Init f9c27000 Current f9c26be4 Base f9c27000 Limit f9c24000 Call 0
Priority 13 BasePriority 12 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr Args to Child
f9c26bfc 80503836 81f9de18 81f9dda8 804fb068 nt!KiSwapContext+0x2f (FPO: [Uses EBP] [0,0,4])
f9c26c08 804fb068 00000000 81cc19e8 f0ccc98c nt!KiSwapThread+0x8a (FPO: [0,0,0])
f9c26c30 f0ccb14e 00000000 00000000 00000000 nt!KeWaitForSingleObject+0x1c2 (FPO: [5,5,4])
f9c26c6c f0cd3125 81cc19e8 00000001 81aa9338 netbt!NbtDestroyDevice+0x1fc (FPO: [2,5,4])
f9c26d10 f9b1cd80 00000002 81b50438 81c4e008 netbt!TdiBindHandler+0x11f (FPO: [3,35,0])
f9c26d3c f9b1d2f5 f9b1e408 81c4e008 00000000 TDI!TdiNotifyPnpClientList+0xd0 (FPO: [3,3,4])
f9c26d60 f9b1b3e4 f9b1e420 00d2db08 f9b1e430 TDI!TdiExecuteRequest+0x19b (FPO: [2,1,4])
f9c26d7c 8053876d 81d2db08 00000000 81f9dda8 TDI!CTEpEventHandler+0x32 (FPO: [1,0,4])
f9c26dac 805cff64 f9b1e420 00000000 00000000 nt!ExpWorkerThread+0xef (FPO: [1,6,0])
f9c26ddc 805460de 8053867e 00000001 00000000 nt!PspSystemThreadStartup+0x34 (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

4

I agree; it does look pretty suspicious. If you see this issue occurs particularly with your IM driver, it could be because you are leaking one of netbt’s packets. However, we’ve fixed bugs like this in the OS itself before, so it’s also possible that this is a bug in netbt. It’s hard to say just from looking at the callstack.

If you want to rule out a leak in your driver’s handling of packets, you can instrument your driver to count the number of packets that it is asked to send, the number of packets it pends, send completes, receives, receive pends, and receive returns. Then do the math to make sure the packet counters add up.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Friday, March 04, 2011 3:17 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] INetCfgClassSetup::DeInstall causes deadlock in NDIS

PSCHED has bindings to ndisuio, tcpip6, and tcpip.

Here is a suspicious looking thread:

1: kd> !thread 81f9dda8
THREAD 81f9dda8 Cid 0004.0024 Teb: 00000000 Win32Thread: 00000000 WAIT: (Executive) KernelMode Non-Alertable
81cc1b84 NotificationEvent
Not impersonating
DeviceMap e1002a60
Owning Process 0 Image:
Attached Process 81f9f5f0 Image: System
Wait Start TickCount 475771 Ticks: 13899 (0:00:03:37.171)
Context Switch Count 516
UserTime 00:00:00.000
KernelTime 00:00:00.046
Start Address nt!ExpWorkerThread (0x8053867e) Stack Init f9c27000 Current f9c26be4 Base f9c27000 Limit f9c24000 Call 0 Priority 13 BasePriority 12 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr Args to Child
f9c26bfc 80503836 81f9de18 81f9dda8 804fb068 nt!KiSwapContext+0x2f (FPO: [Uses EBP] [0,0,4])
f9c26c08 804fb068 00000000 81cc19e8 f0ccc98c nt!KiSwapThread+0x8a (FPO: [0,0,0])
f9c26c30 f0ccb14e 00000000 00000000 00000000 nt!KeWaitForSingleObject+0x1c2 (FPO: [5,5,4]) f9c26c6c f0cd3125 81cc19e8 00000001 81aa9338 netbt!NbtDestroyDevice+0x1fc (FPO: [2,5,4])
f9c26d10 f9b1cd80 00000002 81b50438 81c4e008 netbt!TdiBindHandler+0x11f (FPO: [3,35,0]) f9c26d3c f9b1d2f5 f9b1e408 81c4e008 00000000 TDI!TdiNotifyPnpClientList+0xd0 (FPO: [3,3,4])
f9c26d60 f9b1b3e4 f9b1e420 00d2db08 f9b1e430 TDI!TdiExecuteRequest+0x19b (FPO: [2,1,4]) f9c26d7c 8053876d 81d2db08 00000000 81f9dda8 TDI!CTEpEventHandler+0x32 (FPO: [1,0,4]) f9c26dac 805cff64 f9b1e420 00000000 00000000 nt!ExpWorkerThread+0xef (FPO: [1,6,0]) f9c26ddc 805460de 8053867e 00000001 00000000 nt!PspSystemThreadStartup+0x34 (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer