PNPD Rebalance Test failing : DPC timeout

Hi,

My driver architecture is VBD+NDIS. I am initiating two miniport instances over VBD. When I run the PNPD Rebalance Test it hang around somewhere and leads to DPC timeout.

I see the problem only if two instances of miniport driver is loaded. If we disable one of them and then run pnpd test it runs fine.

Everytime it seems to failing during NDISMALLOCATESHARED memory.

Below is stack please help me to solve the issue.

regards
rohit
/****************************************************/

PNPFILTR: FilterCompletionRoutine Completed, event set.
PNPFILTR: Time taken for IRP_MN_START_DEVICE to get processed for the PDO FFFFFA8001AAEA20 is 873 milliseconds
PNPFILTR: Start: Test Event cleared
PNPFILTR: Irp recorded, sequence count = 8
PNPFILTR: Completing Start request with status == 0x0000000000000000 PDO = 0xFFFFFA8001AAEA20
PNPFILTR: Received IRP_MN_START_DEVICE for stack with PDO 0xFFFFFA8001F934F0
PNPFILTR: Received IRP_MN_QUERY_CAPABILITIES which I am simply passing down
TunQs=4 ToeQs=0 CtrlQs=0 FlQs=4
PNPFILTR: Received IRP_MN_QUERY_INTERFACE which I am simply passing down
PNPFILTR: FilterCompletionRoutine Completed, event set.
PNPFILTR: Time taken for IRP_MN_START_DEVICE to get processed for the PDO FFFFFA8001F934F0 is 358 milliseconds
PNPFILTR: Completing Start request with status == 0x0000000000000000 PDO = 0xFFFFFA8001F934F0
PNPFILTR: Received IRP_MN_START_DEVICE for stack with PDO 0xFFFFFA8001F94060
PNPFILTR: Received IRP_MN_QUERY_CAPABILITIES which I am simply passing down
PNPFILTR: Received IRP_MN_QUERY_INTERFACE which I am simply passing down
Assertion: *** DPC execution time exceeds system limit
This is NOT a break in update time
This is a BUG in a DPC routine
Perform a stack trace to find the culprit
The time out limit will be doubled on continuation
Use gh to continue!!

nt! ?? ::FNODOBFM::string'+0x4ebe: fffff800016284b3 cd2c int 2Ch
1: kd> !locks
**** DUMP OF ALL RESOURCE OBJECTS ****
KD: Scanning for held locks.

Resource @ nt!CmpRegistryLock (0xfffff800017fdf80) Shared 1 owning threads
Contention Count = 3
Threads: fffffa80043b4b60-01<*>
KD: Scanning for held locks.

Resource @ nt!IopDeviceTreeLock (0xfffff8000187e500) Shared 1 owning threads
Threads: fffffa80018b2680-01<*>
KD: Scanning for held locks.

Resource @ nt!PiEngineLock (0xfffff8000187e400) Exclusively owned
Contention Count = 9
NumberOfExclusiveWaiters = 1
Threads: fffffa80018b2680-01<*>
Threads Waiting On Exclusive Access:
fffffa80018b1680

KD: Scanning for held locks…

Resource @ NDIS!SharedMemoryResource (0xfffff88000eaee70) Exclusively owned
Threads: fffffa80018b2680-01<*>
KD: Scanning for held locks…1: kd> !locks -p 0xfffff88000eaee70
**** Dump Resource Performance Data ****

00000000: No resource performance data available
1: kd> !locks -d 0xfffff88000eaee70

Resource @ NDIS!SharedMemoryResource (0xfffff88000eaee70) Exclusively owned
Threads: fffffa80018b2680-01<*>
1 total locks, 1 locks currently held
1: kd> !locks -v 0xfffff88000eaee70

Resource @ NDIS!SharedMemoryResource (0xfffff88000eaee70) Exclusively owned
Threads: fffffa80018b2680-01<*>

THREAD fffffa80018b2680 Cid 0004.0048 Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 0
Not impersonating
DeviceMap fffff8a000008c10
Owning Process fffffa8001892040 Image: System
Attached Process N/A Image: N/A
Wait Start TickCount 13786 Ticks: 644 (0:00:00:10.046)
Context Switch Count 1783
UserTime 00:00:00.000
KernelTime 00:00:13.728
Win32 Start Address nt!ExpWorkerThread (0xfffff80001689050)
Stack Init fffff88001fb5db0 Current fffff88001fb4b10
Base fffff88001fb6000 Limit fffff88001fb0000 Call 0
Priority 15 BasePriority 12 UnusualBoost 2 ForegroundBoost 0 IoPriority 2 PagePriority 5
*** ERROR: Module load completed but symbols could not be loaded for pnpfiltr.sys
Child-SP RetAddr Call Site
fffff88001fb49a0 fffff8000172dcb7 nt!KeSignalCallDpcSynchronize+0x70
fffff88001fb49d0 fffff80001738e0e nt!MiDoStackCopy+0x57
fffff88001fb4a20 fffff80001747bec nt!KeGenericCallDpc+0xfe
fffff88001fb4ad0 fffff80001751b4e nt!MiSwapStackPage+0x9ac
fffff88001fb4bd0 fffff80001752ba0 nt!MiClaimPhysicalRun+0x5fe
fffff88001fb4c90 fffff8000178cc15 nt!MiFindContiguousPages+0xb30
fffff88001fb4db0 fffff8000178dbb3 nt!MiAllocateContiguousMemory+0x2a5
fffff88001fb4e40 fffff80001beb2a0 nt!MmAllocateContiguousMemorySpecifyCache+0x53
fffff88001fb4e80 fffff88000ef8eea hal!HalAllocateCommonBuffer+0x40
fffff88001fb4ec0 fffff88003236cc5 NDIS!NdisMAllocateSharedMemory+0x1aa
fffff88001fb4f20 fffff8800321e203 cht4ndis!T4Nic_AllocTxDefragNbPool+0x255 [d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_utils.c @ 1542]
fffff88001fb4fc0 fffff8800321cab5 cht4ndis!_NicSetupTxQue+0x333 [d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_nic.c @ 6042]
fffff88001fb5070 fffff88003213f6e cht4ndis!T4Nic_NicSetupQues+0x925 [d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_nic.c @ 5194]
fffff88001fb5120 fffff88000eee0a5 cht4ndis!T4Nic_MpInitializeEx+0x3ee [d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_main.c @ 1306]
fffff88001fb5190 fffff88000eed793 NDIS!ndisMInitializeAdapter+0x695
fffff88001fb5550 fffff88000eef86c NDIS!ndisInitializeAdapter+0x113
fffff88001fb55b0 fffff88000eed466 NDIS!ndisPnPStartDevice+0xac
fffff88001fb5610 fffff880015f456f NDIS!ndisPnPDispatch+0x246
fffff88001fb56b0 fffff880015f9d40 pnpfiltr+0x156f
fffff88001fb5710 fffff880015f9c87 pnpfiltr+0x6d40
fffff88001fb5760 fffff80001a39bde pnpfiltr+0x6c87
fffff88001fb5790 fffff800017740ed nt!PnpAsynchronousCall+0xce
fffff88001fb57d0 fffff80001a44926 nt!PnpStartDevice+0x11d
fffff88001fb5890 fffff80001a44b20 nt!PnpStartDeviceNode+0x156
fffff88001fb5920 fffff80001a67ddb nt!PipProcessRestartPhase1+0x30
fffff88001fb5950 fffff80001a6852a nt!PipProcessDevNodeTree+0x1cb
fffff88001fb5bc0 fffff8000177c351 nt!PiProcessResourceRequirementsChanged+0xca
fffff88001fb5c10 fffff80001689161 nt!PnpDeviceActionWorker+0x331
fffff88001fb5cb0 fffff8000191f166 nt!ExpWorkerThread+0x111
fffff88001fb5d40 fffff8000165a486 nt!PspSystemThreadStartup+0x5a
fffff88001fb5d80 0000000000000000 nt!KxStartSystemThread+0x16

The !locks output isn’t interesting in this case, that is only useful if
you’re working with an ERESOURCE deadlock. Here the ASSERT is telling you
that a processor is spinning for too long, so you have a livelock situation.
More interesting would be the output of !running -it.

-scott


Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc.
http://www.osronline.com

Hope to see you at the next OSR kernel debugging class February 14th in
Columbia, MD!

wrote in message news:xxxxx@ntdev…

Hi,

My driver architecture is VBD+NDIS. I am initiating two miniport instances
over VBD. When I run the PNPD Rebalance Test it hang around somewhere and
leads to DPC timeout.

I see the problem only if two instances of miniport driver is loaded. If we
disable one of them and then run pnpd test it runs fine.

Everytime it seems to failing during NDISMALLOCATESHARED memory.

Below is stack please help me to solve the issue.

regards
rohit
/****************************************************/

PNPFILTR: FilterCompletionRoutine Completed, event set.
PNPFILTR: Time taken for IRP_MN_START_DEVICE to get processed for the PDO
FFFFFA8001AAEA20 is 873 milliseconds
PNPFILTR: Start: Test Event cleared
PNPFILTR: Irp recorded, sequence count = 8
PNPFILTR: Completing Start request with status == 0x0000000000000000 PDO =
0xFFFFFA8001AAEA20
PNPFILTR: Received IRP_MN_START_DEVICE for stack with PDO 0xFFFFFA8001F934F0
PNPFILTR: Received IRP_MN_QUERY_CAPABILITIES which I am simply passing down
TunQs=4 ToeQs=0 CtrlQs=0 FlQs=4
PNPFILTR: Received IRP_MN_QUERY_INTERFACE which I am simply passing down
PNPFILTR: FilterCompletionRoutine Completed, event set.
PNPFILTR: Time taken for IRP_MN_START_DEVICE to get processed for the PDO
FFFFFA8001F934F0 is 358 milliseconds
PNPFILTR: Completing Start request with status == 0x0000000000000000 PDO =
0xFFFFFA8001F934F0
PNPFILTR: Received IRP_MN_START_DEVICE for stack with PDO 0xFFFFFA8001F94060
PNPFILTR: Received IRP_MN_QUERY_CAPABILITIES which I am simply passing down
PNPFILTR: Received IRP_MN_QUERY_INTERFACE which I am simply passing down
Assertion: *** DPC execution time exceeds system limit
This is NOT a break in update time
This is a BUG in a DPC routine
Perform a stack trace to find the culprit
The time out limit will be doubled on continuation
Use gh to continue!!

nt! ?? ::FNODOBFM::string'+0x4ebe: fffff800016284b3 cd2c int 2Ch
1: kd> !locks
**** DUMP OF ALL RESOURCE OBJECTS ****
KD: Scanning for held locks.

Resource @ nt!CmpRegistryLock (0xfffff800017fdf80) Shared 1 owning
threads
Contention Count = 3
Threads: fffffa80043b4b60-01<*>
KD: Scanning for held locks.

Resource @ nt!IopDeviceTreeLock (0xfffff8000187e500) Shared 1 owning
threads
Threads: fffffa80018b2680-01<*>
KD: Scanning for held locks.

Resource @ nt!PiEngineLock (0xfffff8000187e400) Exclusively owned
Contention Count = 9
NumberOfExclusiveWaiters = 1
Threads: fffffa80018b2680-01<*>
Threads Waiting On Exclusive Access:
fffffa80018b1680

KD: Scanning for held locks…

Resource @ NDIS!SharedMemoryResource (0xfffff88000eaee70) Exclusively
owned
Threads: fffffa80018b2680-01<*>
KD: Scanning for held
locks…1:
kd> !locks -p 0xfffff88000eaee70
**** Dump Resource Performance Data ****

00000000: No resource performance data available
1: kd> !locks -d 0xfffff88000eaee70

Resource @ NDIS!SharedMemoryResource (0xfffff88000eaee70) Exclusively
owned
Threads: fffffa80018b2680-01<*>
1 total locks, 1 locks currently held
1: kd> !locks -v 0xfffff88000eaee70

Resource @ NDIS!SharedMemoryResource (0xfffff88000eaee70) Exclusively
owned
Threads: fffffa80018b2680-01<*>

THREAD fffffa80018b2680 Cid 0004.0048 Teb: 0000000000000000
Win32Thread: 0000000000000000 RUNNING on processor 0
Not impersonating
DeviceMap fffff8a000008c10
Owning Process fffffa8001892040 Image: System
Attached Process N/A Image: N/A
Wait Start TickCount 13786 Ticks: 644 (0:00:00:10.046)
Context Switch Count 1783
UserTime 00:00:00.000
KernelTime 00:00:13.728
Win32 Start Address nt!ExpWorkerThread (0xfffff80001689050)
Stack Init fffff88001fb5db0 Current fffff88001fb4b10
Base fffff88001fb6000 Limit fffff88001fb0000 Call 0
Priority 15 BasePriority 12 UnusualBoost 2 ForegroundBoost 0 IoPriority
2 PagePriority 5
*** ERROR: Module load completed but symbols could not be loaded for
pnpfiltr.sys
Child-SP RetAddr Call Site
fffff88001fb49a0 fffff8000172dcb7 nt!KeSignalCallDpcSynchronize+0x70
fffff88001fb49d0 fffff80001738e0e nt!MiDoStackCopy+0x57
fffff88001fb4a20 fffff80001747bec nt!KeGenericCallDpc+0xfe
fffff88001fb4ad0 fffff80001751b4e nt!MiSwapStackPage+0x9ac
fffff88001fb4bd0 fffff80001752ba0 nt!MiClaimPhysicalRun+0x5fe
fffff88001fb4c90 fffff8000178cc15 nt!MiFindContiguousPages+0xb30
fffff88001fb4db0 fffff8000178dbb3 nt!MiAllocateContiguousMemory+0x2a5
fffff88001fb4e40 fffff80001beb2a0
nt!MmAllocateContiguousMemorySpecifyCache+0x53
fffff88001fb4e80 fffff88000ef8eea hal!HalAllocateCommonBuffer+0x40
fffff88001fb4ec0 fffff88003236cc5
NDIS!NdisMAllocateSharedMemory+0x1aa
fffff88001fb4f20 fffff8800321e203
cht4ndis!T4Nic_AllocTxDefragNbPool+0x255
[d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_utils.c @ 1542]
fffff88001fb4fc0 fffff8800321cab5 cht4ndis!_NicSetupTxQue+0x333
[d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_nic.c @ 6042]
fffff88001fb5070 fffff88003213f6e cht4ndis!T4Nic_NicSetupQues+0x925
[d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_nic.c @ 5194]
fffff88001fb5120 fffff88000eee0a5 cht4ndis!T4Nic_MpInitializeEx+0x3ee
[d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_main.c @ 1306]
fffff88001fb5190 fffff88000eed793 NDIS!ndisMInitializeAdapter+0x695
fffff88001fb5550 fffff88000eef86c NDIS!ndisInitializeAdapter+0x113
fffff88001fb55b0 fffff88000eed466 NDIS!ndisPnPStartDevice+0xac
fffff88001fb5610 fffff880015f456f NDIS!ndisPnPDispatch+0x246
fffff88001fb56b0 fffff880015f9d40 pnpfiltr+0x156f
fffff88001fb5710 fffff880015f9c87 pnpfiltr+0x6d40
fffff88001fb5760 fffff80001a39bde pnpfiltr+0x6c87
fffff88001fb5790 fffff800017740ed nt!PnpAsynchronousCall+0xce
fffff88001fb57d0 fffff80001a44926 nt!PnpStartDevice+0x11d
fffff88001fb5890 fffff80001a44b20 nt!PnpStartDeviceNode+0x156
fffff88001fb5920 fffff80001a67ddb nt!PipProcessRestartPhase1+0x30
fffff88001fb5950 fffff80001a6852a nt!PipProcessDevNodeTree+0x1cb
fffff88001fb5bc0 fffff8000177c351
nt!PiProcessResourceRequirementsChanged+0xca
fffff88001fb5c10 fffff80001689161 nt!PnpDeviceActionWorker+0x331
fffff88001fb5cb0 fffff8000191f166 nt!ExpWorkerThread+0x111
fffff88001fb5d40 fffff8000165a486 nt!PspSystemThreadStartup+0x5a
fffff88001fb5d80 0000000000000000 nt!KxStartSystemThread+0x16

Scott

Thanks for your reply.

Here is input of !running -it

: kd> !running -it
System Processors: (000000000000000f)
Idle Processors: (0000000000000000) (0000000000000000) (0000000000000000)
(0000000000000000)
Prcbs Current Next
0 fffff800017f4e80 fffffa80018b2680

Child-SP RetAddr Call Site
fffff88001fb49a0 fffff8000172dcb7 nt!KeSignalCallDpcSynchronize+0x70
fffff88001fb49d0 fffff80001738e0e nt!MiDoStackCopy+0x57
fffff88001fb4a20 fffff80001747bec nt!KeGenericCallDpc+0xfe
fffff88001fb4ad0 fffff80001751b4e nt!MiSwapStackPage+0x9ac
fffff88001fb4bd0 fffff80001752ba0 nt!MiClaimPhysicalRun+0x5fe
fffff88001fb4c90 fffff8000178cc15 nt!MiFindContiguousPages+0xb30
fffff88001fb4db0 fffff8000178dbb3 nt!MiAllocateContiguousMemory+0x2a5
fffff88001fb4e40 fffff80001beb2a0
nt!MmAllocateContiguousMemorySpecifyCache+0x53
fffff88001fb4e80 fffff88000ef8eea hal!HalAllocateCommonBuffer+0x40
fffff88001fb4ec0 fffff88003236cc5 NDIS!NdisMAllocateSharedMemory+0x1aa
fffff88001fb4f20 fffff8800321e203 cht4ndis!T4Nic_AllocTxDefragNbPool+0x255
[d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_utils.c @ 1542]
fffff88001fb4fc0 fffff8800321cab5 cht4ndis!_NicSetupTxQue+0x333
[d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_nic.c @ 6042]
fffff88001fb5070 fffff88003213f6e cht4ndis!T4Nic_NicSetupQues+0x925
[d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_nic.c @ 5194]
fffff88001fb5120 fffff88000eee0a5 cht4ndis!T4Nic_MpInitializeEx+0x3ee
[d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_main.c @ 1306]
fffff88001fb5190 fffff88000eed793 NDIS!ndisMInitializeAdapter+0x695
fffff88001fb5550 fffff88000eef86c NDIS!ndisInitializeAdapter+0x113
fffff88001fb55b0 fffff88000eed466 NDIS!ndisPnPStartDevice+0xac
fffff88001fb5610 fffff880015f456f NDIS!ndisPnPDispatch+0x246
fffff88001fb56b0 fffff880015f9d40 pnpfiltr+0x156f
fffff88001fb5710 fffff880015f9c87 pnpfiltr+0x6d40
fffff88001fb5760 fffff80001a39bde pnpfiltr+0x6c87
fffff88001fb5790 fffff800017740ed nt!PnpAsynchronousCall+0xce
fffff88001fb57d0 fffff80001a44926 nt!PnpStartDevice+0x11d
fffff88001fb5890 fffff80001a44b20 nt!PnpStartDeviceNode+0x156
fffff88001fb5920 fffff80001a67ddb nt!PipProcessRestartPhase1+0x30
fffff88001fb5950 fffff80001a6852a nt!PipProcessDevNodeTree+0x1cb
fffff88001fb5bc0 fffff8000177c351
nt!PiProcessResourceRequirementsChanged+0xca
fffff88001fb5c10 fffff80001689161 nt!PnpDeviceActionWorker+0x331
fffff88001fb5cb0 fffff8000191f166 nt!ExpWorkerThread+0x111
fffff88001fb5d40 fffff8000165a486 nt!PspSystemThreadStartup+0x5a
fffff88001fb5d80 0000000000000000 nt!KxStartSystemThread+0x16
1 fffff880009bf180 fffffa80043b4b60

Child-SP RetAddr Call Site
fffff88001e1bc30 fffff80001688e18 nt! ?? ::FNODOBFM::string'+0x4ebe fffff88001e1bcc0 fffff80001688ba1 nt!KeUpdateRunTime+0xb8 fffff88001e1bcf0 fffff80001721dc0 nt!KiSecondaryClockInterrupt+0x131 fffff88001e1be80 fffff8000172dcb7 nt!KeSignalCallDpcSynchronize+0x70 fffff88001e1beb0 fffff800016875dc nt!MiDoStackCopy+0x57 fffff88001e1bf00 fffff80001682065 nt!KiRetireDpcList+0x1bc fffff88001e1bfb0 fffff80001681e7c nt!KxRetireDpcList+0x5 fffff88004ee62e0 fffff800016c765c nt!KiDispatchInterruptContinue fffff88004ee6310 fffff80001957453 nt!KiDpcInterrupt+0xcc fffff88004ee64a0 fffff80001956f69 nt!CmpCheckKeyBodyAccess+0x73 fffff88004ee6530 fffff80001952fb5 nt!CmpDoOpen+0x319 fffff88004ee65f0 fffff80001972764 nt!CmpParseKey+0x4d5 fffff88004ee68c0 fffff80001977876 nt!ObpLookupObjectName+0x585 fffff88004ee69c0 fffff80001956bec nt!ObOpenObjectByName+0x306 fffff88004ee6a90 fffff80001958d3b nt!CmOpenKey+0x28a fffff88004ee6be0 fffff8000167b153 nt!NtOpenKeyEx+0xf fffff88004ee6c20 0000000077ab0e0a nt!KiSystemServiceCopyEnd+0x13 0000000003ffe9b8 0000000000000000 ntdll!NtOpenKeyEx+0xa 2 fffff88001e5c180 fffffa80018b1b60 ................ Child-SP RetAddr Call Site fffff88001e8b070 fffff80001736898 nt!KxWaitForSpinLockAndAcquire+0x20 fffff88001e8b0a0 fffff80001aff6ce nt!KeFreezeExecution+0xc9 fffff88001e8b150 fffff800016e0a5f nt!KdEnterDebugger+0x6e fffff88001e8b190 fffff80001afe03f nt! ?? ::FNODOBFM::string’+0x193ef
fffff88001e8b1d0 fffff800016b61a6 nt!KdpTrap+0x2f
fffff88001e8b220 fffff8000167b542 nt!KiDispatchException+0x126
fffff88001e8b8c0 fffff8000167aaed nt!KiExceptionDispatch+0xc2
fffff88001e8baa0 fffff800016284b3 nt!KiRaiseAssertion+0xed
fffff88001e8bc30 fffff80001688e18 nt! ?? ::FNODOBFM::string'+0x4ebe fffff88001e8bcc0 fffff80001688ba1 nt!KeUpdateRunTime+0xb8 fffff88001e8bcf0 fffff80001721dbe nt!KiSecondaryClockInterrupt+0x131 fffff88001e8be80 fffff8000172dcb7 nt!KeSignalCallDpcSynchronize+0x6e fffff88001e8beb0 fffff800016875dc nt!MiDoStackCopy+0x57 fffff88001e8bf00 fffff80001682065 nt!KiRetireDpcList+0x1bc fffff88001e8bfb0 fffff80001681e7c nt!KxRetireDpcList+0x5 fffff88001fc2600 fffff800016c765c nt!KiDispatchInterruptContinue fffff88001fc2630 fffff8000195490a nt!KiDpcInterrupt+0xcc fffff88001fc27c0 fffff80001952d81 nt!CmpBuildHashStackAndLookupCache+0x10a fffff88001fc2ba0 fffff80001972764 nt!CmpParseKey+0x2a1 fffff88001fc2e70 fffff80001977876 nt!ObpLookupObjectName+0x585 fffff88001fc2f70 fffff80001956bec nt!ObOpenObjectByName+0x306 fffff88001fc3040 fffff80001959e12 nt!CmOpenKey+0x28a fffff88001fc3190 fffff8000167b153 nt!NtOpenKey+0x12 fffff88001fc31d0 fffff800016776f0 nt!KiSystemServiceCopyEnd+0x13 fffff88001fc3368 fffff88000defed9 nt!KiServiceLinkage fffff88001fc3370 fffff88000defaa9 mrxsmb!MRxSmbReadMiscellaneousRegistryParameters+0x249 fffff88001fc35f0 fffff88000def5c4 mrxsmb!SmbCeGetConfigurationInformation+0x29 fffff88001fc36a0 fffff88001923b2f mrxsmb!MRxSmbPnPBindingHandler+0x194 fffff88001fc3810 fffff88001924089 TDI!TdiNotifyPnpClientList+0x11f fffff88001fc3870 fffff880019243e1 TDI!TdiExecuteRequest+0x429 fffff88001fc38c0 fffff88001924d36 TDI!TdiHandleSerializedRequest+0x2a1 fffff88001fc3930 fffff8800190b1b4 TDI!TdiRegisterDeviceObject+0xa6 fffff88001fc3960 fffff8800190e511 tdx!TdxCreateAndRegisterDeviceObject+0x524 fffff88001fc3b50 fffff88001905965 tdx!TdxProcessInterfaceChangeRoutine+0x401 fffff88001fc3c30 fffff80001975541 tdx!NetiopIoWorkItemRoutine+0x35 fffff88001fc3c80 fffff80001689161 nt!IopProcessWorkItem+0x3d fffff88001fc3cb0 fffff8000191f166 nt!ExpWorkerThread+0x111 fffff88001fc3d40 fffff8000165a486 nt!PspSystemThreadStartup+0x5a fffff88001fc3d80 0000000000000000 nt!KxStartSystemThread+0x16 3 fffff88001ecc180 fffffa80040525a0 fffffa80045a7660 ................ Child-SP RetAddr Call Site fffff88001efaa48 fffff880028f2512 cht4vbd!READ_REGISTER_ULONG+0xc [c:\winddk\7600.16385.1\inc\ddk\wdm.h @ 12796] fffff88001efaa50 fffff880028f5b78 cht4vbd!t4_read_reg+0x22 [d:\t4\dev\t4\windows\src\kernel\osdep\adapter.h @ 263] fffff88001efaa90 fffff880028faf22 cht4vbd!t4_slow_intr_handler+0x18 [d:\t4\dev\t4\common\t4_hw.c @ 2057] fffff88001efaad0 fffff80001677af9 cht4vbd!NICEvtInterruptIsr+0x92 [d:\t4\dev\t4\windows\src\kernel\vbd\nic_isrdpc.c @ 160] fffff88001efab20 fffff800016778d8 nt!KiScanInterruptObjectList+0x69 fffff88001efab70 fffff80001720dbb nt!KiChainedDispatch+0x128 fffff88001efad00 fffff80001aff5da nt!KeThawExecution+0x26b fffff88001efad80 fffff80001afe182 nt!KdExitDebugger+0x7a fffff88001efadb0 fffff80001afe617 nt!KdpPrint+0x122 fffff88001efae00 fffff800016b61a6 nt! ?? ::DFIOBLLK::string’+0x157
fffff88001efae50 fffff8000167b542 nt!KiDispatchException+0x126
fffff88001efb4f0 fffff8000167abf3 nt!KiExceptionDispatch+0xc2
fffff88001efb6d0 fffff80001673f85 nt!KiDebugServiceTrap+0xf3
fffff88001efb868 fffff800016bb495 nt!DebugPrint+0x15
fffff88001efb870 fffff800017344bc nt! ?? ::FNODOBFM::string'+0xc612 fffff88001efbb20 fffff880028ee180 nt!DbgPrint+0x3c fffff88001efbb60 fffff880028f8488 cht4vbd!TraceEvents+0xd0 [d:\t4\dev\t4\windows\src\kernel\vbd\t4busenum.c @ 2039] fffff88001efbdb0 fffff880028f7ce3 cht4vbd!t4_os_portmod_changed+0x28 [d:\t4\dev\t4\windows\src\kernel\vbd\nic_common.c @ 123] fffff88001efbde0 fffff880028fb0be cht4vbd!t4_handle_fw_rpl+0x213 [d:\t4\dev\t4\common\t4_hw.c @ 4394] fffff88001efbe60 fffff880011398a7 cht4vbd!NICEvtInterruptDpc+0x11e [d:\t4\dev\t4\windows\src\kernel\vbd\nic_isrdpc.c @ 247] fffff88001efbed0 fffff800016875dc Wdf01000!FxInterrupt::_InterruptDpcThunk+0x8f fffff88001efbf00 fffff80001682065 nt!KiRetireDpcList+0x1bc fffff88001efbfb0 fffff80001681e7c nt!KxRetireDpcList+0x5 fffff88005b988b0 fffff800016c7793 nt!KiDispatchInterruptContinue fffff88005b988e0 fffff80001677da2 nt!KiDpcInterruptBypass+0x13 fffff88005b988f0 fffff80001673084 nt!KiInterruptDispatch+0x212 fffff88005b98a88 fffff80001924317 nt!memcmp+0xb4 fffff88005b98a90 fffff80001923e12 nt!EtwpFindGuidEntryByGuid+0x73 fffff88005b98ad0 fffff800019247d8 nt!EtwpRegisterUMGuid+0x66 fffff88005b98b40 fffff8000167b153 nt!NtTraceControl+0x198 fffff88005b98bb0 0000000077ab16fa nt!KiSystemServiceCopyEnd+0x13 000000000048eff8 00000000`00000000 ntdll!ZwTraceControl+0xa

regards
rohit

On Tue, Jan 18, 2011 at 10:04 PM, Scott Noone wrote:

> The !locks output isn’t interesting in this case, that is only useful if
> you’re working with an ERESOURCE deadlock. Here the ASSERT is telling you
> that a processor is spinning for too long, so you have a livelock situation.
> More interesting would be the output of !running -it.
>
> -scott
>
> –
> Scott Noone
> Consulting Associate and Chief System Problem Analyst
> OSR Open Systems Resources, Inc.
> http://www.osronline.com
>
> Hope to see you at the next OSR kernel debugging class February 14th in
> Columbia, MD!
>
> wrote in message news:xxxxx@ntdev…
>
>
> Hi,
>
> My driver architecture is VBD+NDIS. I am initiating two miniport instances
> over VBD. When I run the PNPD Rebalance Test it hang around somewhere and
> leads to DPC timeout.
>
> I see the problem only if two instances of miniport driver is loaded. If we
> disable one of them and then run pnpd test it runs fine.
>
> Everytime it seems to failing during NDISMALLOCATESHARED memory.
>
> Below is stack please help me to solve the issue.
>
> regards
> rohit
> / *******************************************/
>
> PNPFILTR: FilterCompletionRoutine Completed, event set.
> PNPFILTR: Time taken for IRP_MN_START_DEVICE to get processed for the PDO
> FFFFFA8001AAEA20 is 873 milliseconds
> PNPFILTR: Start: Test Event cleared
> PNPFILTR: Irp recorded, sequence count = 8
> PNPFILTR: Completing Start request with status == 0x0000000000000000 PDO =
> 0xFFFFFA8001AAEA20
> PNPFILTR: Received IRP_MN_START_DEVICE for stack with PDO
> 0xFFFFFA8001F934F0
> PNPFILTR: Received IRP_MN_QUERY_CAPABILITIES which I am simply passing
> down
> TunQs=4 ToeQs=0 CtrlQs=0 FlQs=4
> PNPFILTR: Received IRP_MN_QUERY_INTERFACE which I am simply passing down
> PNPFILTR: FilterCompletionRoutine Completed, event set.
> PNPFILTR: Time taken for IRP_MN_START_DEVICE to get processed for the PDO
> FFFFFA8001F934F0 is 358 milliseconds
> PNPFILTR: Completing Start request with status == 0x0000000000000000 PDO =
> 0xFFFFFA8001F934F0
> PNPFILTR: Received IRP_MN_START_DEVICE for stack with PDO
> 0xFFFFFA8001F94060
> PNPFILTR: Received IRP_MN_QUERY_CAPABILITIES which I am simply passing
> down
> PNPFILTR: Received IRP_MN_QUERY_INTERFACE which I am simply passing down
> Assertion:
DPC execution time exceeds system limit
> This is NOT a break in update time
> This is a BUG in a DPC routine
> Perform a stack trace to find the culprit
> The time out limit will be doubled on continuation
> Use gh to continue!!
>
> nt! ?? ::FNODOBFM::string'+0x4ebe:<br>&gt; fffff800016284b3 cd2c int 2Ch
> 1: kd> !locks
> DUMP OF ALL RESOURCE OBJECTS
> KD: Scanning for held locks.
>
> Resource @ nt!CmpRegistryLock (0xfffff800017fdf80) Shared 1 owning
> threads
> Contention Count = 3
> Threads: fffffa80043b4b60-01<
>
> KD: Scanning for held locks.
>
> Resource @ nt!IopDeviceTreeLock (0xfffff8000187e500) Shared 1 owning
> threads
> Threads: fffffa80018b2680-01<
>
> KD: Scanning for held locks.
>
> Resource @ nt!PiEngineLock (0xfffff8000187e400) Exclusively owned
> Contention Count = 9
> NumberOfExclusiveWaiters = 1
> Threads: fffffa80018b2680-01<
>
> Threads Waiting On Exclusive Access:
> fffffa80018b1680
>
> KD: Scanning for held locks…
>
> Resource @ NDIS!SharedMemoryResource (0xfffff88000eaee70) Exclusively
> owned
> Threads: fffffa80018b2680-01<
>
> KD: Scanning for held
> locks…1:
> kd> !locks -p 0xfffff88000eaee70
> Dump Resource Performance Data
>
> 00000000: No resource performance data available
> 1: kd> !locks -d 0xfffff88000eaee70
>
> Resource @ NDIS!SharedMemoryResource (0xfffff88000eaee70) Exclusively
> owned
> Threads: fffffa80018b2680-01<
>
> 1 total locks, 1 locks currently held
> 1: kd> !locks -v 0xfffff88000eaee70
>
> Resource @ NDIS!SharedMemoryResource (0xfffff88000eaee70) Exclusively
> owned
> Threads: fffffa80018b2680-01<
>
>
> THREAD fffffa80018b2680 Cid 0004.0048 Teb: 0000000000000000
> Win32Thread: 0000000000000000 RUNNING on processor 0
> Not impersonating
> DeviceMap fffff8a000008c10
> Owning Process fffffa8001892040 Image: System
> Attached Process N/A Image: N/A
> Wait Start TickCount 13786 Ticks: 644 (0:00:00:10.046)
> Context Switch Count 1783
> UserTime 00:00:00.000
> KernelTime 00:00:13.728
> Win32 Start Address nt!ExpWorkerThread (0xfffff80001689050)
> Stack Init fffff88001fb5db0 Current fffff88001fb4b10
> Base fffff88001fb6000 Limit fffff88001fb0000 Call 0
> Priority 15 BasePriority 12 UnusualBoost 2 ForegroundBoost 0 IoPriority
> 2 PagePriority 5
> *** ERROR: Module load completed but symbols could not be loaded for
> pnpfiltr.sys
> Child-SP RetAddr Call Site
> fffff88001fb49a0 fffff8000172dcb7 nt!KeSignalCallDpcSynchronize+0x70
> fffff88001fb49d0 fffff80001738e0e nt!MiDoStackCopy+0x57
> fffff88001fb4a20 fffff80001747bec nt!KeGenericCallDpc+0xfe
> fffff88001fb4ad0 fffff80001751b4e nt!MiSwapStackPage+0x9ac
> fffff88001fb4bd0 fffff80001752ba0 nt!MiClaimPhysicalRun+0x5fe
> fffff88001fb4c90 fffff8000178cc15 nt!MiFindContiguousPages+0xb30
> fffff88001fb4db0 fffff8000178dbb3 nt!MiAllocateContiguousMemory+0x2a5
> fffff88001fb4e40 fffff80001beb2a0
> nt!MmAllocateContiguousMemorySpecifyCache+0x53
> fffff88001fb4e80 fffff88000ef8eea hal!HalAllocateCommonBuffer+0x40
> fffff88001fb4ec0 fffff88003236cc5 NDIS!NdisMAllocateSharedMemory+0x1aa
> fffff88001fb4f20 fffff8800321e203
> cht4ndis!T4Nic_AllocTxDefragNbPool+0x255
> [d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_utils.c @ 1542]
> fffff88001fb4fc0 fffff8800321cab5 cht4ndis!_NicSetupTxQue+0x333
> [d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_nic.c @ 6042]
> fffff88001fb5070 fffff88003213f6e cht4ndis!T4Nic_NicSetupQues+0x925
> [d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_nic.c @ 5194]
> fffff88001fb5120 fffff88000eee0a5 cht4ndis!T4Nic_MpInitializeEx+0x3ee
> [d:\t4\dev\t4\windows\src\kernel\nic\ndis6x\cht4nic_main.c @ 1306]
> fffff88001fb5190 fffff88000eed793 NDIS!ndisMInitializeAdapter+0x695
> fffff88001fb5550 fffff88000eef86c NDIS!ndisInitializeAdapter+0x113
> fffff88001fb55b0 fffff88000eed466 NDIS!ndisPnPStartDevice+0xac
> fffff88001fb5610 fffff880015f456f NDIS!ndisPnPDispatch+0x246
> fffff88001fb56b0 fffff880015f9d40 pnpfiltr+0x156f
> fffff88001fb5710 fffff880015f9c87 pnpfiltr+0x6d40
> fffff88001fb5760 fffff80001a39bde pnpfiltr+0x6c87
> fffff88001fb5790 fffff800017740ed nt!PnpAsynchronousCall+0xce
> fffff88001fb57d0 fffff80001a44926 nt!PnpStartDevice+0x11d
> fffff88001fb5890 fffff80001a44b20 nt!PnpStartDeviceNode+0x156
> fffff88001fb5920 fffff80001a67ddb nt!PipProcessRestartPhase1+0x30
> fffff88001fb5950 fffff80001a6852a nt!PipProcessDevNodeTree+0x1cb
> fffff88001fb5bc0 fffff8000177c351
> nt!PiProcessResourceRequirementsChanged+0xca
> fffff88001fb5c10 fffff80001689161 nt!PnpDeviceActionWorker+0x331
> fffff88001fb5cb0 fffff8000191f166 nt!ExpWorkerThread+0x111
> fffff88001fb5d40 fffff8000165a486 nt!PspSystemThreadStartup+0x5a
> fffff88001fb5d80 0000000000000000 nt!KxStartSystemThread+0x16
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Your ISR may be stuck in a loop, either because the device is powered down and all its registers are read as FFFFFFFF, or for some other reason. Another processor is trying to flush DPCs and times out.