@rod_widdowson said:
My go to behavior is pretty “brute force and ignorance” but FWIW:
.logappend
to a file (the next two commands product a lot of output will be long)!locks
!process 0 7
(this will take a fair chunk of time)Then open the log file in an editor where it’s easy to search.
9 times out of 10 the output of
!locks
will lead you to a deadlock then its just a matter of working out what you are holding that Kaspersky is taking umbrage to.From your description it feels like something is blocked while holding the system volume locked - all the other threads will be backed up behind that - it’s a matter of working out why that thread isn’t moving
If
!locks
doesn’t help you are going to have to spelunk through all the threads (the output of the!process
command) looking for candidates for a deadlock -look for threads with your driver on the stack or with kaspersky on the stack.
!stacks 2 <yourdrvier.sys>
just lists the threads where you are on the stack and can be helpful but I find that I usually need to be able to peruse all the threads at some stage and so doing it up front saves me time in the long run.
Hi Rod, thank you for the reply.
I am looking through the output of the commands you suggested as i am writing this, but as a side note, i found our driver and kaspersky’s klfle in the stack of one of the cpu cores, although i am not sure whether its related to the problem or not, but in this stack we are calling FltCreateFile and then kaspersky is calling FltGetFileNameInformation, probably on the filename that we just tried to open. And we are opening the file using FILE_SHARE_READ/WRITE/DELETE and also with the READ+WRITE+SYNCHRONIZE access and FILE_SYNCHRONOUS_IO_NONALERT flag, and the file is a local file and not a network file.
nt!KxReleaseQueuedSpinLock+0x23
nt!ExpReleaseResourceForThreadLite+0x1f6
NTFS!NtfsReleaseFcb+0x4b
NTFS!NtfsCommonQueryInformation$fin$0+0x6c
nt!_C_specific_handler+0x18e
nt!RtlpExecuteHandlerForUnwind+0xd
nt!RtlUnwindEx+0x432
nt!_C_specific_handler+0xe2
nt!RtlpExecuteHandlerForException+0xd
nt!RtlDispatchException+0x421
nt!RtlRaiseStatus+0x4e
NTFS!NtfsRaiseStatusInternal+0x6c
NTFS!NtfsCommonQueryInformation+0xbfc
NTFS!NtfsFsdDispatchSwitch+0xcc
NTFS!NtfsFsdDispatchWait+0x40
FLTMGR!FltpQueryInformationFile+0x112
FLTMGR!FltpGetFileName+0x30c
FLTMGR!FltpGetOpenedFileName+0x19
FLTMGR!FltpCallOpenedFileNameHandler+0x2b
FLTMGR!FltpExpandFilePathWorker+0x445
FLTMGR!FltpExpandFilePath+0x1a
FLTMGR!FltpGetNormalizedFileNameWorker+0x112
FLTMGR!FltpGetNormalizedFileName+0x1a
FLTMGR!FltpCreateFileNameInformation+0x32c
FLTMGR!HandleStreamListNotSupported+0x115
FLTMGR!FltpGetFileNameInformation+0x5e7
FLTMGR!FltGetFileNameInformation+0x1b0
luafv!LuafvGenerateFileName+0x4c
FLTMGR!FltpCallOpenedFileNameHandler+0x89
FLTMGR!FltpGetNormalizedFileNameWorker+0x2f
FLTMGR!FltpGetNormalizedFileName+0x1a
FLTMGR!FltpCreateFileNameInformation+0x32c
FLTMGR!HandleStreamListNotSupported+0x113
FLTMGR!FltpGetFileNameInformation+0x5e7
FLTMGR!FltGetFileNameInformation+0x1b0
wcifs!WcGenerateFileName+0x44
FLTMGR!FltpCallOpenedFileNameHandler+0x89
FLTMGR!FltpGetNormalizedFileNameWorker+0x2f
FLTMGR!FltpGetNormalizedFileName+0x1a
FLTMGR!FltpCreateFileNameInformation+0x32c
FLTMGR!HandleStreamListNotSupported+0x115
FLTMGR!FltpGetFileNameInformation+0x5e1
FLTMGR!FltGetFileNameInformation+0x1b0
klfle+0x610f0 ----> kaspersky
klfle+0x57950 ----> kaspersky
FLTMGR!FltpPerformPreCallbacks+0x2ea
FLTMGR!FltpPassThroughInternal+0x88
FLTMGR!FltpCreate+0x2e1
nt!IopParseDevice+0x168f
nt!ObpLookupObjectName+0x8af
nt!ObOpenObjectByNameEx+0x1dd
nt!IopCreateFile+0x860
nt!IoCreateFileEx+0x115
FLTMGR!FltpCreateFile+0x1cd
FLTMGR!FltCreateFile+0x8d
ourdriver+0x220a ----> ourdriver
output of locks:
3: kd> !locks
**** DUMP OF ALL RESOURCE OBJECTS ****
KD: Scanning for held locks......
Resource @ 0xffffb408db5f5a08 Shared 4 owning threads
Contention Count = 95
NumberOfSharedWaiters = 46
NumberOfExclusiveWaiters = 1
Threads: ffffb408e3bbc080-01<*> ffffb408de8d3080-01<*> ffffb408de8e4080-01<*> ffffb408e36d7800-01<*>
ffffb408e3751080-01 ffffb408e390d080-01 ffffb408e225e800-01 ffffb408e3448040-01
ffffb408de94f800-01 ffffb408def17080-01 ffffb408debb5800-01 ffffb408df0af080-01
ffffb408e3b68800-01 ffffb408dbdbe080-01 ffffb408df282080-01 ffffb408e1d58080-01
ffffb408e3802080-01 ffffb408e3be3800-01 ffffb408e1f6e040-01 ffffb408e1bd4080-01
ffffb408e2bf2800-01 ffffb408e367d080-01 ffffb408e3af5800-01 ffffb408dec8a800-01
ffffb408e4515080-01 ffffb408e414a800-01 ffffb408e373f800-01 ffffb408e38d8080-01
ffffb408df748080-01 ffffb408dd561080-01 ffffb408d9cd3080-01 ffffb408e1bf1080-01
ffffb408e3928080-01 ffffb408e39c5800-01 ffffb408e41dc080-01 ffffb408e4103800-01
ffffb408e374d040-01 ffffb408df3cc080-01 ffffb408e3768040-01 ffffb408e4973040-01
ffffb408df4a6740-01 ffffb408dea8b080-01 ffffb408df702080-01 ffffb408dd6b9080-01
ffffb408e14f7080-01 ffffb408df996800-01 ffffb408e1e9c800-01 ffffb408df63c080-01
ffffb408e3363180-01 ffffb408dddc4080-01<*>
Threads Waiting On Exclusive Access:
ffffb408e3642800
KD: Scanning for held locks....................................................................................................
Resource @ 0xffffb408dd3ec940 Exclusively owned
Contention Count = 3
NumberOfExclusiveWaiters = 1
Threads: ffffb408e2bf2800-01<*>
Threads Waiting On Exclusive Access:
ffffb408ddf47080
KD: Scanning for held locks.........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Resource @ 0xffffb408e35b8310 Exclusively owned
Contention Count = 11
NumberOfSharedWaiters = 3
NumberOfExclusiveWaiters = 1
Threads: ffffb408e3bbc080-02<*> ffffb408de8d3080-01 ffffb408de8e4080-01 ffffb408e262b040-01<*>
Threads Waiting On Exclusive Access:
ffffb408e36d7800
KD: Scanning for held locks.........................................................................................................
Resource @ 0xffffb408dab1fd60 Exclusively owned
Threads: ffffb408e3bbc080-01<*>
Resource @ 0xffffb408dab1fe10 Exclusively owned
Threads: ffffb408e3bbc080-01<*>
KD: Scanning for held locks.....................................
32227 total locks, 5 locks currently held