DPC_WATCHDOG_VIOLATION (133/1) in Storport virtual miniport

Hi All,

I’m working on a Storport virtual miniport.
Now it’s occured that it throws bugchecks DPC_WATCHDOG_VIOLATION (133/1) under tests with high load.

I’ve read the following thread https://community.osr.com/discussion/comment/291047/#Comment_291047 but found no ideas how to pin out the problem in our case.

It seems that the bugcheck occures in a DPC that being scheduled on our call to StorPortNotification(RequestComplete, pDevExt, pSrb), which is called from a callback that runs at DISPATCH_LEVEL too.

Here is the crash dump analyzer’s output:

Windows 10 Kernel Version 14393 MP (56 procs) Free x64
Product: Server, suite: TerminalServer SingleUserTS
Built by: 14393.2828.amd64fre.rs1_release_inmarket.190216-1457

DPC_WATCHDOG_VIOLATION (133)
The DPC watchdog detected a prolonged run time at an IRQL of DISPATCH_LEVEL or above.
Arguments:
Arg1: 0000000000000001, The system cumulatively spent an extended period of time at
DISPATCH_LEVEL or above. The offending component can usually be
identified with a stack trace.
Arg2: 0000000000001e00, The watchdog period.
Arg3: fffff800ff420540, cast to nt!DPC_WATCHDOG_GLOBAL_TRIAGE_BLOCK, which contains
additional information regarding the cumulative timeout
Arg4: 0000000000000000

KEY_VALUES_STRING: 1

TIMELINE_ANALYSIS: 1

DUMP_CLASS: 1

DUMP_QUALIFIER: 401

BUILD_VERSION_STRING: 14393.2828.amd64fre.rs1_release_inmarket.190216-1457

SYSTEM_MANUFACTURER: Dell Inc.

SYSTEM_PRODUCT_NAME: PowerEdge R730

SYSTEM_SKU: SKU=NotProvided;ModelName=PowerEdge R730

BIOS_VENDOR: Dell Inc.

BIOS_VERSION: 2.8.0

BIOS_DATE: 005/17/2018

BASEBOARD_MANUFACTURER: Dell Inc.

BASEBOARD_PRODUCT: 0599V5

BASEBOARD_VERSION: A06

DUMP_TYPE: 1

BUGCHECK_P1: 1

BUGCHECK_P2: 1e00

BUGCHECK_P3: fffff800ff420540

BUGCHECK_P4: 0

DPC_TIMEOUT_TYPE: DPC_QUEUE_EXECUTION_TIMEOUT_EXCEEDED

CPU_COUNT: 38

CPU_MHZ: 7d0

CPU_VENDOR: GenuineIntel

CPU_FAMILY: 6

CPU_MODEL: 3f

CPU_STEPPING: 2

CPU_MICROCODE: 6,3f,2,0 (F,M,S,R) SIG: 3D’00000000 (cache) 3D’00000000 (init)

DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT

BUGCHECK_STR: 0x133

PROCESS_NAME: System

CURRENT_IRQL: d

ANALYSIS_SESSION_HOST: SA

ANALYSIS_SESSION_TIME: 03-07-2019 21:59:22.0422

ANALYSIS_VERSION: 10.0.17134.1 amd64fre

LAST_CONTROL_TRANSFER: from fffff800ff12600a to fffff800ff1d8940

STACK_TEXT:
ffff9481cbce9d88 fffff800ff12600a : 0000000000000133 0000000000000001 0000000000001e00 fffff800ff420540 : nt!KeBugCheckEx
ffff9481cbce9d90 fffff800ff123258 : 000204bb7770f843 0000000000000000 0000000000000000 fffff78000000320 : nt!KeAccumulateTicks+0x42a
ffff9481cbce9df0 fffff800ff00a4e5 : ffff81014128f500 ffff81014128f500 0000000000000001 ffff81014128f500 : nt!KeClockInterruptNotify+0xb8
ffff9481cbce9f40 fffff800ff0d8566 : ffffec389c851cbd 0000000000000000 0000000fffffffff 0000000000000003 : hal!HalpTimerClockIpiRoutine+0x15
ffff9481cbce9f70 fffff800ff1da13a : ffff9481cbcdac20 ffff9481cb517180 ffff9481cb566180 0000000000000000 : nt!KiCallInterruptServiceRoutine+0x106
ffff9481cbce9fb0 fffff800ff1da627 : ffff9481cbcdc000 fffff800ff0e2d2b ffff810149c66340 000000000000000d : nt!KiInterruptSubDispatchNoLockNoEtw+0xea
ffff9481cbcdaba0 fffff800ff009c2f : ffffffffffd151b8 ffff890f79bdd001 ffff9481cb517180 0000000000000000 : nt!KiInterruptDispatchNoLockNoEtw+0x37
ffff9481cbcdad30 fffff800ff0edac0 : ffff81014873a690 ffff890f79bdd080 0000000000000000 ffff890f797bbc60 : hal!HalSendSoftwareInterrupt+0xaf
ffff9481cbcdada0 fffff800ff0ed5b3 : fffff18283d82d00 0000000000000001 ffff890f00000001 ffff810100000000 : nt!KiDeferredReadyThread+0x280
ffff9481cbcdae40 fffff800ff0eb39c : fffff18283d82d80 0000000000000000 ffff890f79bdd1c0 00000000a000000c : nt!KiExitDispatcher+0x133
ffff9481cbcdaeb0 fffff800ff778f65 : fffff18200000001 ffff9481cbcdb000 0000000000000000 0000000000000000 : nt!IopfCompleteRequest+0x9bc
ffff9481cbcdafd0 fffff80c376145cb : 0000000000000000 0000000000001000 fffff18283d82d80 ffff81014a03ba60 : nt!IovCompleteRequest+0x1c1
ffff9481cbcdb0b0 fffff800ff779593 : ffff9481cbcdb4d8 fffff1827af02e50 ffff9481cbcdb4d8 fffff1827af02f68 : CLASSPNP!TransferPktComplete+0x4ab
ffff9481cbcdb300 fffff800ff0eaaf2 : fffff1827af02e50 ffff810100000001 ffff9481cbcdb419 ffffffffffffffff : nt!IovpLocalCompletionRoutine+0x16f
ffff9481cbcdb360 fffff800ff778f65 : fffff1827af02e50 ffff9481cbcdb4f9 ffff810149f89e40 ffff81014a0270d8 : nt!IopfCompleteRequest+0x112
ffff9481cbcdb480 fffff80c36e06e0e : 0000000000000000 0000000000000000 fffff1827af02e50 0000000000000000 : nt!IovCompleteRequest+0x1c1
ffff9481cbcdb560 fffff80c36e0682c : ffff81014a057990 0000000000000000 ffff9481cbcdb730 00000000001dcd61 : storport!RaidCompleteRequestEx+0x8e
ffff9481cbcdb630 fffff80c36e05c3a : 0000000000000000 ffff8101466761a0 ffff81014a057990 0000000000000000 : storport!RaidUnitCompleteRequest+0xabc
ffff9481cbcdb7b0 fffff800ff0f0385 : ffff9481cbcdb990 ffff9481cb566180 ffff810146676050 0000000000000000 : storport!RaidpAdapterDpcRoutine+0x10a
ffff9481cbcdb890 fffff800ff0ef910 : ffff890f00000000 ffff890f790c9180 0000000000140001 0000000000000000 : nt!KiExecuteAllDpcs+0x335
ffff9481cbcdb9e0 fffff800ff1dbf9a : ffffffff00000000 ffff9481cb566180 ffff9481cb576e00 ffff810149cc0080 : nt!KiRetireDpcList+0x910
ffff9481cbcdbc60 0000000000000000 : ffff9481cbcdc000 ffff9481cbcd6000 0000000000000000 0000000000000000 : nt!KiIdleLoop+0x5a

THREAD_SHA1_HASH_MOD_FUNC: a86d7ec77ab9a723b3f722778cbbcf1afea16235

THREAD_SHA1_HASH_MOD_FUNC_OFFSET: ac68da73957282477d41a8e44cf44b4296644ba8

THREAD_SHA1_HASH_MOD: 9e41cd2e595f99de4e64f4b4152f3f08791883d7

FOLLOWUP_IP:
CLASSPNP!TransferPktComplete+4ab
fffff80c`376145cb 4183fc02 cmp r12d,2

FAULT_INSTR_CODE: 2fc8341

SYMBOL_STACK_INDEX: c

SYMBOL_NAME: CLASSPNP!TransferPktComplete+4ab

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: CLASSPNP

IMAGE_NAME: CLASSPNP.SYS

DEBUG_FLR_IMAGE_TIMESTAMP: 58ba5893

STACK_COMMAND: .thread ; .cxr ; kb

BUCKET_ID_FUNC_OFFSET: 4ab

FAILURE_BUCKET_ID: 0x133_VRF_ISR_CLASSPNP!TransferPktComplete

BUCKET_ID: 0x133_VRF_ISR_CLASSPNP!TransferPktComplete

PRIMARY_PROBLEM_CLASS: 0x133_VRF_ISR_CLASSPNP!TransferPktComplete

TARGET_TIME: 2019-03-07T16:15:18.000Z

OSBUILD: 14393

OSSERVICEPACK: 0

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK: 272

PRODUCT_TYPE: 3

OSPLATFORM_TYPE: x64

OSNAME: Windows 10

OSEDITION: Windows 10 Server TerminalServer SingleUserTS

OS_LOCALE:

USER_LCID: 0

OSBUILD_TIMESTAMP: 2019-02-17 03:56:01

BUILDDATESTAMP_STR: 190216-1457

BUILDLAB_STR: rs1_release_inmarket

BUILDOSVER_STR: 10.0.14393.2828.amd64fre.rs1_release_inmarket.190216-1457

ANALYSIS_SESSION_ELAPSED_TIME: 95b

ANALYSIS_SOURCE: KM

FAILURE_ID_HASH_STRING: km:0x133_vrf_isr_classpnp!transferpktcomplete

FAILURE_ID_HASH: {d26a3ea4-fbeb-e820-2562-ef4c6fa8fb78}

Thank you in advance for help or any ideas,

Best regards,
Valeriy Glushkov
www.starwind.com

I prototyped a solution for our internal driver signing processes last year. It took me just under a workweek to get something going (upload a signed CAB, wait for the process to finish, downloaded the CAB of signed files and store them in our artifact control). Not production quality code, mind, but enough to show that it could work. This was with me having very little REST experience and absolutely no prior experience with Azure and its terminology, so I had to figure out the whole Azure AD App business and how to interface to all of it from Python.