Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results
The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.
Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/
Hi, guys.
I have very strange deadlock and I am completely misunderstanding what's going on.
Three threads try to acquire resource, but none of them succeeds:
Resource @ 0xffffe00153341ab0 Shared 1 owning threads Contention Count = 7 NumberOfSharedWaiters = 1 NumberOfExclusiveWaiters = 2 Threads: ffffe00152e9c080-01<*> Threads Waiting On Exclusive Access: ffffe001525386c0 ffffe00151ce54c0
According to this picture, the thread ffffe00152e9c080 is resource owner an it also one of shared waiters.
But how is that even possible? The thread has acquired the resource and then waits for it again?
Here is complete output of '!locks' command:
0: kd> !locks **** DUMP OF ALL RESOURCE OBJECTS **** KD: Scanning for held locks...... Resource @ 0xffffe0014f56ea00 Shared 1 owning threads Contention Count = 137 NumberOfSharedWaiters = 102 NumberOfExclusiveWaiters = 1 Threads: ffffe00151ce54c0-01<*> ffffe0015335a080-01 ffffe00151b7f2c0-01 ffffe00151380080-01 ffffe00152907040-01 ffffe00152c68080-01 ffffe001532d0080-01 ffffe0015247d280-01 ffffe00152686080-01 ffffe00150f07080-01 ffffe0015120f780-01 ffffe0014dc34700-01 ffffe001519be1c0-01 ffffe00150d62040-01 ffffe0014f02f080-01 ffffe00151ab8840-01 ffffe00153650840-01 ffffe00150ee1040-01 ffffe001526aa080-01 ffffe0014f05f840-01 ffffe00151b0b840-01 ffffe00151233840-01 ffffe00152e55080-01 ffffe00151abc080-01 ffffe001519f85c0-01 ffffe00151d1a080-01 ffffe00151dde840-01 ffffe00152efd080-01 ffffe0014dd21040-01 ffffe00151ef1840-01 ffffe0015347b080-01 ffffe00151a62080-01 ffffe00151b1e080-01 ffffe00151bce080-01 ffffe001516da080-01 ffffe00152583840-01 ffffe001524a4080-01 ffffe00151a6e080-01 ffffe00151e37080-01 ffffe00151d1e080-01 ffffe0015266b080-01 ffffe001523b9080-01 ffffe00152489840-01 ffffe001532ce840-01 ffffe00153374080-01 ffffe00153392840-01 ffffe00152e7c080-01 ffffe00151723080-01 ffffe001518d1840-01 ffffe0014dc5d840-01 ffffe00152691840-01 ffffe001533c8840-01 ffffe001528d9840-01 ffffe00152fba080-01 ffffe00153372080-01 ffffe001529ec840-01 ffffe001520cc500-01 ffffe00150df6040-01 ffffe001517ee840-01 ffffe00151c03840-01 ffffe00150f61080-01 ffffe001525ae080-01 ffffe001524ec080-01 ffffe00152e3a840-01 ffffe00150ece080-01 ffffe0014dc87040-01 ffffe00150f01040-01 ffffe0014ef71080-01 ffffe0014f1f1040-01 ffffe00151b826c0-01 ffffe00152594080-01 ffffe001526b1080-01 ffffe001528da840-01 ffffe001527fc080-01 ffffe00151cee080-01 ffffe001511ad480-01 ffffe00151763080-01 ffffe0014f200080-01 ffffe001523b5080-01 ffffe00153581080-01 ffffe00151be5080-01 ffffe00152f02080-01 ffffe00151339080-01 ffffe00152a49080-01 ffffe0014ef62080-01 ffffe001526a7080-01 ffffe00152836080-01 ffffe0015171e080-01 ffffe00152e9e080-01 ffffe00152aa0040-01 ffffe00152980080-01 ffffe00150ebd040-01 ffffe0015188f080-01 ffffe001532cf080-01 ffffe0014f034080-01 ffffe001525a0080-01 ffffe001511ea080-01 ffffe0014efcd840-01 ffffe001516e9080-01 ffffe00150f50080-01 ffffe001516ea080-01 ffffe00153537080-01 ffffe00152c87080-01<*> Threads Waiting On Exclusive Access: ffffe00152c77080 KD: Scanning for held locks............ Resource @ 0xffffe00153341ab0 Shared 1 owning threads Contention Count = 7 NumberOfSharedWaiters = 1 NumberOfExclusiveWaiters = 2 Threads: ffffe00152e9c080-01<*> Threads Waiting On Exclusive Access: ffffe001525386c0 ffffe00151ce54c0 KD: Scanning for held locks........ 24712 total locks, 2 locks currently held
As you can see, the thread ffffe00152e9c080 is not involved in any other locks (but somehow magically it blocks the whole system).
CMKD extension (https://codemachine.com/downloads/cmkd.html) confirms that all three threads wait on the same resource 0xffffe00153341ab0,
see 1st parameter of ExAcquireResourceXxxLite and ExpWaitForResource:
.thread ffffe00152e9c080;!stack -p;.thread ffffe001525386c0;!stack -p;.thread ffffe00151ce54c0;!stack -p 03 ffffd0010d96ba40 fffff8019bcdc562 nt!ExpWaitForResource+674 Parameter[0] = ffffe00153341ab0 Parameter[1] = ffffd0010d96bb40 Parameter[2] = (unknown) Parameter[3] = (unknown) 04 ffffd0010d96bb00 fffff8003e592cea nt!ExAcquireResourceSharedLite+412 Parameter[0] = ffffe00153341ab0 Parameter[1] = 0000000000000001 Parameter[2] = (unknown) Parameter[3] = (unknown) ... 03 ffffd0010d130bb0 fffff8019bcdbf9f nt!ExpWaitForResource+674 Parameter[0] = ffffe00153341ab0 Parameter[1] = ffffe00152d07c30 Parameter[2] = (unknown) Parameter[3] = (unknown) 04 ffffd0010d130c70 fffff8003e4bb317 nt!ExAcquireResourceExclusiveLite+1cf Parameter[0] = ffffe00153341ab0 Parameter[1] = ffffc001e023c150 Parameter[2] = 0000000000000001 Parameter[3] = (unknown) ... 03 ffffd0010d1599b0 fffff8019bcdbf9f nt!ExpWaitForResource+674 Parameter[0] = ffffe00153341ab0 Parameter[1] = ffffe00152d07c30 Parameter[2] = (unknown) Parameter[3] = (unknown) 04 ffffd0010d159a70 fffff8003e5947bd nt!ExAcquireResourceExclusiveLite+1cf Parameter[0] = ffffe00153341ab0 Parameter[1] = 0000000000000001 Parameter[2] = (unknown) Parameter[3] = (unknown)
Upcoming OSR Seminars | ||
---|---|---|
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead! | ||
Kernel Debugging | 30 January 2023 | Live, Online |
Developing Minifilters | 20 March 2023 | Live, Online |
Internals & Software Drivers | 17 April 2023 | Live, Online |
Writing WDF Drivers | 22 May 2023 | Live, Online |
Comments
More information about threads and their I/O:
THREAD ffffe00152e9c080
THREAD ffffe001525386c0
THREAD ffffe00151ce54c0
I also found that "Resource @ 0xffffe00153341ab0" is a 'PagingIoResource' member of FSRTL_ADVANCED_FCB_HEADER of these FILE_OBJECTs
(all 'FsContext' point to the same FCB; see above):
A few words about my driver (MyDriver.sys): it just a file monitor. This driver intercepts some file operations and then, in separate
thread, pulls summary information to user-mode application. Nothing unusual. Two special cases are post-create and post-cleanup callbacks:
sometimes, driver calls FltQueryInformationFile in these callbacks to obtain file size (see MyDriver!queryFileSize in the stack of
the first thread). IoGetTopLevelIrp check is present.
The "Driver Verifier" did not help (no violations was found).
Thanks in advance for any help.
Definitely looks weird. Is it possible to look at a copy of the dump file?
-scott
OSR