In one of our KMDF USB drivers we have a bulk continuous reader. It’s triggered in EvtDeviceFileCreate and halted in either EvtD0Exit or EvtIoCleanup.
Recently we failed the 10,000 open/close test in Device Path Exerciser (or rather, never finished it), presumably due to some sort of race between open and close.
Inside EvtDeviceFileCreate I call WdfIoTargetStart() on the bulk WDFIOTARGET, and inside EvtD0Exit or EvtIoCleanup I call WdfIoTargetStop() with argument WdfIoTargetLeaveSentIoPending.
Basically, what the DebugView log was showing was when you have a few apps calling CreateFile()/CloseHandle() on the device object simultaneously, eventually the call to WdfIoTargetStop() blocks forever. When this happens the calling application is blocked and can’t be killed, and the driver fails to unload.
Also, this only started when we exposed a composite device configuration on our device, so we were sitting on the generic parent. I’m wondering if this may have introduced sufficient time delay (in passing URBs to the device, and back) to start causing this, but not really sure.
I protected EvtDeviceFileCreate(), EvtIoCleanup() and EvtD0Exit() all with the same WDFWAITLOCK and this cleared the problem right up, but thought I would report my findings as WdfIoTargetStop() is not supposed to block in that case.
WdfIoTargetStop is not reentrant. Only one thread can be calling it at once which is why the WDFWAITLOCK cleared things up. It appears the docs don’t have this restriction, I will get that fixed. This has nothing to do with there being a continuous reader.
Also, for a continuous reader, LeaveSentIoPending is ignored, we treat WdfIoTargetSTop(x) on a continuous reader as WdfIoTarget(CancelSentIo).
Also, to for clarity, EvtDeviceD0Entry/D0Exit are not synchronized with file object callbacks (no matter what level of synchronization you choose). Instead of the WDFWAITLOCK you can set your device to idle out while in S0 (IdleCannotWakeFromS0) with a super short timeout (1ms) and then in EvtDeviceFileCreate() call WdfDeviceStopIdle(TRUE) and in EvtFileCleanup() call WdfDeviceResumeIdle(). You want to do this for 2 reasons
- by using the idle functionality you don’t have to synchronize your WDFIOTARGET state with D0Entry/Exit with a WDFWAITLOCK
- since EvtFileCleanup() is not coordinated with power state it can be called after you device has been surprise removed which means you are touching your hardware after it is gone. Maybe not such a big deal for USB b/c it is protocol based, but if you are relying on allocations/structures initialized in EvtDevicePrepareHardware, EvtDeviceReleaseHardware may have already run and freed them.
The KMDF serial example initially had this mistake where it tried to touch hw in file cleanup() but we fixed this for KMDF v1.5 (the WDK).
Hope that helps
d