Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTDEV
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


Waiting for worker thread at driver unload

owsows Member Posts: 1

Hi,

I have a driver that share some traits with the nonpnp sample from the WDK. It's a KMDF non-PnP driver that creates a single CDO.

It has a worker thread that's created using PsCreateSystemThread(). Work items are queued into a regular doubly-linked list and the thread is woken using a notification event. This is working just as intended during normal operation.

There is also a lock used for the work item queue. The lock protects the wake-up event as well, to avoid any race conditions.

However, when attempting to unload the driver I find that the worker thread is not being scheduled as expected.

At the time of unloading, the worker is suspended inside KeWaitForSingleObject(), waiting for the wake-up event to be signaled:

kd> !thread 0xFFFFBC888D44C080
THREAD ffffbc888d44c080  Cid 0004.206c  Teb: 0000000000000000 Win32Thread: 0000000000000000 READY on processor 0
Not impersonating
DeviceMap                 ffff910571c13ae0
Owning Process            ffffbc8884662040       Image:         System
Attached Process          N/A            Image:         N/A
Wait Start TickCount      244327         Ticks: 1842018 (0:07:59:41.531)
Context Switch Count      20             IdealProcessor: 0             
UserTime                  00:00:00.000
KernelTime                00:00:00.015
Win32 Start Address mydriver!DispatchWorker (0xfffff8047363a420)
Stack Init ffff8c0fa265ac90 Current ffff8c0fa265a800
Base ffff8c0fa265b000 Limit ffff8c0fa2655000 Call 0000000000000000
Priority 9 BasePriority 8 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP          RetAddr           : Args to Child                                                           : Call Site
ffff8c0f`a265a840 fffff804`740b968d : fffff804`00000001 00000000`fffffffe fffff804`00000008 00000000`00000002 : nt!KiSwapContext+0x76
ffff8c0f`a265a980 fffff804`740b8514 : ffffbc88`8d44c080 fffff804`741df215 00000000`00000010 00000000`00000000 : nt!KiSwapThread+0xbfd
ffff8c0f`a265aa20 fffff804`740b7cb5 : ffffbc88`84175e90 00000008`00000000 fffff804`75303900 00000000`00000000 : nt!KiCommitThreadWait+0x144
ffff8c0f`a265aac0 fffff804`7363a47a : ffff9105`762636c8 00000000`00000000 ffff9105`7f2bc100 00000000`00000000 : nt!KeWaitForSingleObject+0x255
ffff8c0f`a265aba0 fffff804`74133135 : ffff9105`76263690 fffff804`7363a420 ffff9105`76263690 ffff84d7`2cc84b81 : mydriver!DispatchWorker+0x5a [C:\work\mydriver\mydriver.cpp @ 151] 
ffff8c0f`a265ac10 fffff804`741de9a8 : fffff804`73325180 ffffbc88`8d44c080 fffff804`741330e0 00000000`00000064 : nt!PspSystemThreadStartup+0x55
ffff8c0f`a265ac60 00000000`00000000 : ffff8c0f`a265b000 ffff8c0f`a2655000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x28

A system thread has called into the device object clean-up handler, and has just executed the following code:

WdfWaitLockAcquire(context->QueueLock, NULL);

KeSetEvent(&context->ExitWorker, 0, FALSE);
KeSetEvent(&context->WakeUpWorker, 0, FALSE);

WdfWaitLockRelease(context->QueueLock);

KeWaitForSingleObject(&context->DispatchWorker, Executive, KernelMode, FALSE, NULL);

Where DispatchWorker was acquired much earlier like so:

status = ObReferenceObjectByHandle
(
    threadHandle,
    THREAD_ALL_ACCESS,
    NULL,
    KernelMode,
    (PVOID *)&context->DispatchWorker,
    NULL
);

As the clean-up code starts waiting on the worker thread, the whole system dead-locks.

Here are the details for the system thread:

kd> !thread
THREAD ffffbc888474e040  Cid 0004.0058  Teb: 0000000000000000 Win32Thread: 0000000000000000 RUNNING on processor 0
Not impersonating
DeviceMap                 ffff910571c13ae0
Owning Process            ffffbc8884662040       Image:         System
Attached Process          N/A            Image:         N/A
Wait Start TickCount      261835         Ticks: 0
Context Switch Count      164696         IdealProcessor: 0             
UserTime                  00:00:00.000
KernelTime                00:00:08.484
Win32 Start Address nt!ExpWorkerThread (0xfffff80474098840)
Stack Init ffff8c0fa009ec90 Current ffff8c0fa009d7a0
Base ffff8c0fa009f000 Limit ffff8c0fa0099000 Call 0000000000000000
Priority 13 BasePriority 12 PriorityDecrement 0 IoPriority 2 PagePriority 5
Child-SP          RetAddr           : Args to Child                                                           : Call Site
ffff8c0f`a009e910 fffff804`7535c457 : 00004377`74a463e8 ffff8c0f`a009e958 ffff8c0f`a009e958 64206c65`6e6e7501 : mydriver!CleanupDevice+0x70 [C:\work\mydriver\mydriver.cpp @ 197] 
ffff8c0f`a009e950 fffff804`75353779 : ffffbc88`8b5b9c10 ffffbc88`8cba55b0 ffffbc88`8b5b9c30 fffff804`741e554c : Wdf01000!FxObject::CallCleanupCallbacks+0x4f [minkernel\wdf\framework\shared\object\fxobject.cpp @ 354] 
(Inline Function) --------`-------- : --------`-------- --------`-------- --------`-------- --------`-------- : Wdf01000!FxObject::CallCleanup+0x16 (Inline Function @ fffff804`75353779) [minkernel\wdf\framework\shared\inc\private\common\fxobject.hpp @ 831] 
ffff8c0f`a009e980 fffff804`75326c5b : ffffbc88`8b5b9c10 ffffbc88`8cba55b0 ffffbc88`8bf5bb08 00001f80`01083a55 : Wdf01000!FxDevice::Dispose+0x39 [minkernel\wdf\framework\shared\core\fxdevice.cpp @ 1306] 
ffff8c0f`a009e9b0 fffff804`7530821d : ffffbc88`8cba55b0 ffffbc88`8a69fb60 ffffbc88`8b5b9c58 00000000`00000000 : Wdf01000!FxObject::DisposeChildrenWorker+0x1e5bb [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 1212] 
(Inline Function) --------`-------- : --------`-------- --------`-------- --------`-------- --------`-------- : Wdf01000!FxObject::PerformDisposingDisposeChildrenLocked+0x25 (Inline Function @ fffff804`7530821d) [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 846] 
(Inline Function) --------`-------- : --------`-------- --------`-------- --------`-------- --------`-------- : Wdf01000!FxObject::PerformEarlyDisposeWorkerAndUnlock+0x43 (Inline Function @ fffff804`7530821d) [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 924] 
(Inline Function) --------`-------- : --------`-------- --------`-------- --------`-------- --------`-------- : Wdf01000!FxObject::PerformEarlyDispose+0x65 (Inline Function @ fffff804`7530821d) [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 553] 
(Inline Function) --------`-------- : --------`-------- --------`-------- --------`-------- --------`-------- : Wdf01000!FxObject::DisposeChildrenWorker+0x161 (Inline Function @ fffff804`7530821d) [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 1191] 
(Inline Function) --------`-------- : --------`-------- --------`-------- --------`-------- --------`-------- : Wdf01000!FxObject::DeleteWorkerAndUnlock+0x1a0 (Inline Function @ fffff804`7530821d) [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 968] 
ffff8c0f`a009e9f0 fffff804`753555d9 : ffffbc88`8bf90e00 00004377`75960498 00000000`00000000 00000000`00000001 : Wdf01000!FxObject::DeleteObject+0x26d [minkernel\wdf\framework\shared\object\fxobjectstatemachine.cpp @ 124] 
ffff8c0f`a009ea70 fffff804`753559c8 : 00004377`75960498 00000000`00000000 00000000`00000080 ffffbc88`846acca0 : Wdf01000!FxDriver::DeleteObject+0x9 [minkernel\wdf\framework\shared\inc\private\common\fxdriver.hpp @ 375] 
ffff8c0f`a009eaa0 fffff804`7363b82e : ffffbc88`8474e040 fffff804`7470d9c0 ffffbc88`846acca0 00000000`00010282 : Wdf01000!FxDriver::Unload+0xe8 [minkernel\wdf\framework\shared\core\fxdriver.cpp @ 201] 
ffff8c0f`a009eb00 fffff804`747fbb13 : 00000000`00000000 00000000`00000000 00000000`00000300 fffff804`7447aa60 : mydriver!FxStubDriverUnload+0x1e [d:\w8rtm\minkernel\wdf\framework\kmdf\src\dynamic\stub\stub.cpp @ 158] 
ffff8c0f`a009eb30 fffff804`74098945 : ffffbc88`846acca0 fffff804`7462f900 ffffbc88`8474e040 ffffbc88`0000000c : nt!IopLoadUnloadDriver+0xee153
ffff8c0f`a009eb70 fffff804`74133135 : ffffbc88`8474e040 00000000`00000080 ffffbc88`84662040 00000000`00000000 : nt!ExpWorkerThread+0x105
ffff8c0f`a009ec10 fffff804`741de9a8 : fffff804`73325180 ffffbc88`8474e040 fffff804`741330e0 00000000`00000000 : nt!PspSystemThreadStartup+0x55
ffff8c0f`a009ec60 00000000`00000000 : ffff8c0f`a009f000 ffff8c0f`a0099000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x28

The only thing I notice that seems a little off is the raised priority of the system thread.

Is this a case of "don't use KeWaitForSingleObject() in an arbitrary context"? If so, what is a better design to solve this?

TIA

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Writing WDF Drivers 7 Dec 2020 LIVE ONLINE
Internals & Software Drivers 25 Jan 2021 LIVE ONLINE
Developing Minifilters 8 March 2021 LIVE ONLINE