Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results


More Info on Driver Writing and Debugging

The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.

Check out The OSR Learning Library at:

Before Posting...

Please check out the Community Guidelines in the Announcements and Administration Category.

Deleting Control Devices before the FDO

PlusquamPlusquam Member Posts: 2

Hi, I am debugging a KMDF-based PNP driver that creates dozens of Control Devices (CDOs) per FDO for legacy compatibility reasons. The driver creates the CDOs in EvtDevicePrepareHardware with WdfDeviceCreate and deletes them in EvtDeviceReleaseHardware with WdfObjectDelete. Apparently, WDF waits forever for the state machine of the m_DefaultQueueForCreates if a IRP_MJ_CREATE is completed on a CDO that is about to be deleted.

If your driver creates control device objects (which do not support PnP or power management), and if the driver also creates framework device objects that support PnP and power management, the driver must eventually call WdfObjectDelete at IRQL = PASSIVE_LEVEL to delete the control device objects.

If the driver creates both types of device objects, the operating system cannot unload your driver until the driver has deleted the control device objects.

However, the driver must not delete the control device objects until after the framework has deleted the other device objects. To determine when the framework has deleted the other device objects, your driver should provide EvtCleanupCallback functions for those objects.

The last paragraph irritates me: to my understanding the paragraph means that the only valid point to delete the CDOs is the EvtCleanupCallback of the FDO (as demonstrated in the toastDrv sideband sample), because (1) the CDO must outlive the FDO and (2) the PNP manager will not unload the driver if DOs remain after the FDO is cleaned up, as discussed in

My current understanding contradicts (1): CDOs and FDOs are unrelated and therefore have independent lifetimes, so the driver may delete CDOs at any time. Is this correct or does the paragraph hint at a restriction that I am not aware of? In case it is relevant: the FDO and each CDO have their own WdfIoQueue and all Ioctls are completed in the IrpPreprocessCallback.

Thanks for helping.


  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 8,499

    Ah! You have entered that space which is the cross-product of PnP, NON-PnP, and KMDF. Things get... tricky.

    There is a positively wonderful (if ancient) thread on this topic that will explain to you what's going on. See it here.


    Peter Viscarola

  • PlusquamPlusquam Member Posts: 2

    I linked to that thread in the original post. The primary topic of the thread is that PNP drivers will not unload if the last open handle is on a CDO. This is a known and accepted issue of the driver that I work on. I am more concerned with the documentation. Is the statement “However, the driver must not delete the control device objects until after the framework has deleted the other device objects.” correct? If this is actually a requirement of WDF, I would at least consider moving the deletes of the CDOs from EvtDeviceReleaseHardware to EvtCleanupCallback, although deleting in EvtDeviceReleaseHardware apparently worked fine for many years. The only related information in the linked thread is that Doron_Holan states “The correct pattern is to delete in the last fdo's cleanup”.

    Anyhow, the problem I have does not appear to be related with the time when the CDOs are deleted. Instead WDF has (in my opinion) a bug that it blocks forever if the last reference on a device is released by a WdfRequestComplete in the EvtDeviceFileCreate callback of a CDO. The result is not just that the driver does not unload, but that a process, or in the case I have even another kernel service, hangs forever in the EvtFileCreate. Consequently, the system does not shutdown anymore and with Driver Verifier enabled, bluescreens. In that case I get the following callstack:

    Wdf01000!_FX_DRIVER_GLOBALS::WaitForSignal+0xc6 [minkernel\wdf\framework\shared\object\globals.cpp @ 1972]
    Wdf01000!FxIoQueue::Dispose+0xdb [minkernel\wdf\framework\shared\irphandlers\io\fxioqueue.cpp @ 552]
    Wdf01000!FxObject::DisposeChildrenWorker+0x1e68e [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 1212]
    Wdf01000!FxObject::PerformDisposingDisposeChildrenLocked+0x35 [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 846]
    Wdf01000!FxObject::PerformEarlyDisposeWorkerAndUnlock+0x6a [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 926]
    Wdf01000!FxObject::PerformEarlyDispose+0x8d [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 592]
    Wdf01000!FxObject::DisposeChildrenWorker+0x1e629 [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 1191]
    Wdf01000!FxObject::DeleteObject+0x26d [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 124]
    Wdf01000!FxPkgGeneral::DecrementOpenHandleCount+0x23 [minkernel\wdf\framework\shared\irphandlers\general\fxpkggeneral.cpp @ 1983]
    Wdf01000!FxRequest::CompleteInternal+0x1e855 [minkernel\wdf\framework\shared\core\fxrequest.cpp @ 862]
    Wdf01000!imp_WdfRequestComplete+0x8b [minkernel\wdf\framework\shared\core\fxrequestapi.cpp @ 436]
    cdoBug!WdfRequestComplete+0x45 [C:\Program Files (x86)\Windows Kits\10\Include\wdf\kmdf\1.15\wdfrequest.h @ 996]
    cdoBug!cdoBugEvtFileCreate+0x22 [cdoBug.c @ 14]
    Wdf01000!FxPkgGeneral::OnCreate+0xa68 [minkernel\wdf\framework\shared\irphandlers\general\fxpkggeneral.cpp @ 1302]
    Wdf01000!FxPkgGeneral::Dispatch+0x44c [minkernel\wdf\framework\shared\irphandlers\general\fxpkggeneral.cpp @ 785]
    Wdf01000!FxDevice::DispatchWithLock+0x112 [minkernel\wdf\framework\shared\core\fxdevice.cpp @ 1430]

    Relevant member variables of FxIoQueue are:
    m_Deleted = 0, m_Disposing = 1, m_Dispatching = 0, m_DriverIoCount = 1, m_Queue.m_RequestCount = 0

    To my understanding the bug is that FxIoQueue::DispatchEvents is called with m_DriverIoCount = 1, resulting in a totalIoCount=1 that prevents the FxIoQueue::DispatchEvents from setting the m_FinishDisposing event.

    For reference, the driver written to trigger the issue is:

    typedef struct {
    PKTHREAD thread;
    volatile BOOLEAN threadShallRun;
    void cdoBugEvtFileCreate(WDFDEVICE d, WDFREQUEST r, WDFFILEOBJECT fo) {
    WdfRequestComplete(r, STATUS_UNSUCCESSFUL);
    void cdoBugThreadRoutine(void *ctxVoid) {
    DECLARE_CONST_UNICODE_STRING(ntDevName, L"\Device\cdoBug");
    DEVICE_CTX *ctx = ctxVoid;
    while (ctx->threadShallRun) {
    devInit = WdfControlDeviceInitAllocate( WdfGetDriver(), &SDDL_DEVOBJ_SYS_ALL_ADM_RWX_WORLD_RWX_RES_RWX);
    if (!devInit) {
    NTSTATUS status = WdfDeviceInitAssignName(devInit, &ntDevName);
    if (!NT_SUCCESS(status)) {
    WDF_FILEOBJECT_CONFIG_INIT(&foc, cdoBugEvtFileCreate, NULL, NULL);
    WdfDeviceInitSetFileObjectConfig(devInit, &foc, WDF_NO_OBJECT_ATTRIBUTES);
    WDFDEVICE cdo;
    status = WdfDeviceCreate(&devInit, WDF_NO_OBJECT_ATTRIBUTES, &cdo);
    if (!NT_SUCCESS(status)) {
    li.QuadPart = WDF_REL_TIMEOUT_IN_MS(10);
    KeDelayExecutionThread(Executive, FALSE, &li);
    DEVICE_CTX *ctx = GetDeviceCtx(device);
    ctx->threadShallRun = TRUE;
    HANDLE hThread = 0;
    InitializeObjectAttributes(&objAttr, NULL, OBJ_KERNEL_HANDLE, NULL, NULL);
    NTSTATUS status = PsCreateSystemThread(&hThread, THREAD_ALL_ACCESS, &objAttr, 0, NULL, cdoBugThreadRoutine, ctx);
    if (NT_SUCCESS(status)) {
    status = ObReferenceObjectByHandle(hThread, SYNCHRONIZE, NULL, KernelMode, &ctx->thread, NULL);
    return status;
    DEVICE_CTX *ctx = GetDeviceCtx(device);
    ctx->threadShallRun = FALSE;
    KeWaitForSingleObject(ctx->thread, Executive, KernelMode, FALSE, NULL);
    ctx->thread = NULL;
    return STATUS_SUCCESS;
    NTSTATUS cdoBugEvtDeviceAdd(WDFDRIVER d, PWDFDEVICE_INIT deviceInit) {
    callbacks.EvtDeviceD0Entry = cdoBugEvtD0Entry;
    callbacks.EvtDeviceD0Exit = cdoBugEvtD0Exit;
    WdfDeviceInitSetPnpPowerEventCallbacks(deviceInit, &callbacks);
    WDFDEVICE device;
    return WdfDeviceCreate(&deviceInit, &attributes, &device);
    NTSTATUS DriverEntry(PDRIVER_OBJECT driverObject,
    PUNICODE_STRING registryPath) {
    WDF_DRIVER_CONFIG_INIT(&config, cdoBugEvtDeviceAdd);
    return WdfDriverCreate(driverObject, registryPath, WDF_NO_OBJECT_ATTRIBUTES, &config, WDF_NO_HANDLE);

    and a single userspace process calls CreateFile(L"\\\\.\\globalroot\\Device\\cdoBug", GENERIC_READ, 0, NULL, OPEN_EXISTING, 0, NULL); in an infinite loop.

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Developing Minifilters 24 May 2021 Live, Online
Writing WDF Drivers 14 June 2021 Live, Online
Internals & Software Drivers 27 September 2021 Live, Online
Kernel Debugging 15 November 2021 Live, Online