Possible DeadLock

Hello guys!

I got a legacy filter running on Windows XP/2K3 and with an antivirus.
A specific process is causing a possible deadlock that freezes the machine for a while. After a time, the machine returns its normal way.
Once the machine has returned the issue doesn?t occur in the same session of the user.
If I disable the AVXPTO antivirus filter, the issue doesn?t occur.

I detected the process that causes it througth output debug from my filter.
Tha machine freezes exactly when I call ZwCreateFile() from an IRP_MJ_CREATE function handle.

This the only thread from the proccess I mentioned:

kd> !thread 814beb28
THREAD 814beb28 Cid 0bc0.0bc4 Teb: 7ffdf000 Win32Thread: e17e7830 WAIT: (Executive) KernelMode Non-Alertable
ba624220 SynchronizationEvent
IRP List:
81432bf8: (0006,01fc) Flags: 00000884 Mdl: 00000000
8174d868: (0006,01fc) Flags: 00000884 Mdl: 00000000
Not impersonating
DeviceMap e13645f0
Owning Process 0 Image:
Attached Process 814beda0 Image: XPTO.exe
Wait Start TickCount 9066 Ticks: 3875 (0:00:01:00.546)
Context Switch Count 1135 LargeStack
UserTime 00:00:00.031
KernelTime 00:00:00.218
Win32 Start Address 0x00401494
Start Address 0x7c810705
Stack Init ba6256e0 Current ba6241ac Base ba626000 Limit ba621000 Call ba6256ec
Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr Args to Child
ba6241c4 80500cf0 814beb98 814beb28 804f9d72 nt!KiSwapContext+0x2e (FPO: [Uses EBP] [0,0,4])
ba6241d0 804f9d72 bad8ff28 e199b01c 00000000 nt!KiSwapThread+0x46 (FPO: [0,0,0])
ba6241f8 bad69b8f 00000000 00000000 00000000 nt!KeWaitForSingleObject+0x1c2 (FPO: [5,5,4])
WARNING: Stack unwind information not available. Following frames may be wrong.
ba624234 bad600a4 e199b01c 00000001 00000001 AVXPTO+0xcb8f <======== This is the av
ba6242ac f101a80d 00000000 00020089 016242ec AVXPTO+0x30a4 <======== This is the av
ba624340 804ee129 817a8828 81432bf8 81432bf8 AVXPTO2!AVXPTOQueryFullName+0x5b13 <======== This is the av
ba6243c0 80578688 81818018 814e4614 ba624568 nt!IopfCallDriver+0x31 (FPO: [0,0,0])
ba6244a0 805b4d3c 81818030 00000000 814e4570 nt!IopParseDevice+0xa12 (FPO: [Non-Fpo])
ba624528 805b10e5 00000000 ba624568 00000040 nt!ObpLookupObjectName+0x56a (FPO: [11,19,4])
ba62457c 8056b295 00000000 00000000 535c6c00 nt!ObOpenObjectByName+0xeb (FPO: [7,5,4])
ba6245f8 8056bc0c ba6247e4 80100000 ba624778 nt!IopCreateFile+0x407 (FPO: [Non-Fpo])
ba624654 8056e31e ba6247e4 80100000 ba624778 nt!IoCreateFile+0x8e (FPO: [14,3,0])
ba624694 8053d658 ba6247e4 80100000 ba624778 nt!NtCreateFile+0x30 (FPO: [11,0,0])
ba624694 804fe09d ba6247e4 80100000 ba624778 nt!KiFastCallEntry+0xf8 (FPO: [0,0] TrapFrame @ ba6246c8)
ba624738 f99f0979 ba6247e4 80100000 ba624778 nt!ZwCreateFile+0x11 (FPO: [11,0,0]) <======== MyFilter calls ZwCreateFile
ba6247b8 f99f2946 ba6247e4 81670cf4 80000000 MyFilter!verifierOpenFile+0xaa (FPO: [Non-Fpo]) <======== Another func from MyFilter
ba6247ec f99f4d28 81a0c7f8 f99f410c 814c7360 MyFilter!verifierValidateSE+0xb0 <======== Another func from MyFilter
ba625058 804ee129 81a63bd0 8174d868 00000021 MyFilter!MyFilterCreate+0x21e (FPO: [Non-Fpo]) <======== This is my MJ_CREATE handle func
ba625068 f101a9d5 00000000 81918790 81a1eb00 nt!IopfCallDriver+0x31 (FPO: [0,0,0])
ba6250d0 804ee129 817a8828 8174d868 8174d868 AVXPTO2!AVXPTOQueryFullName+0x5cdb <======== This is the av
ba625150 80578688 81818018 814c7c44 ba6252f8 nt!IopfCallDriver+0x31 (FPO: [0,0,0])
ba625230 805b4d3c 81818030 00000000 814c7ba0 nt!IopParseDevice+0xa12 (FPO: [Non-Fpo])
ba6252b8 805b10e5 00000000 ba6252f8 00000040 nt!ObpLookupObjectName+0x56a (FPO: [11,19,4])
ba62530c 8056b295 00000000 00000000 3e419001 nt!ObOpenObjectByName+0xeb (FPO: [7,5,4])
ba625388 8056bc0c 0012ecb4 00100001 0012ec58 nt!IopCreateFile+0x407 (FPO: [Non-Fpo])
ba6253e4 8056f433 0012ecb4 00100001 0012ec58 nt!IoCreateFile+0x8e (FPO: [14,3,0])
ba625424 8053d658 0012ecb4 00100001 0012ec58 nt!NtOpenFile+0x27 (FPO: [6,0,0])
ba625424 7c90e514 0012ecb4 00100001 0012ec58 nt!KiFastCallEntry+0xf8 (FPO: [0,0] TrapFrame @ ba625444)
0012ef24 00000000 00000000 00000000 00000000 0x7c90e514

My filter always run with AV without problem but with a specific version of the AV I got the problem related above.

Can I assume that in this call “ba6241f8 bad69b8f 00000000 00000000 00000000 nt!KeWaitForSingleObject+0x1c2 (FPO: [5,5,4])” is the problem?

After a while with the machine freezed I look at the debug info and the thread is still showing the same information.

How can I do to extend this investigation?

Thanks

Ismael

> How can I do to extend this investigation?

Some ideas?

  • Does the AV have a user mode bit? If so look to see whether it has any
    IOs wedged against locks of yours. If not extend your search for all
    threads.

  • This deadlock clears itself right? So look for what it is that releases
    the object ("ba w 4 on the KEVENT is where I’d start).

    - What part of pool is the KEVENT in? Can you make any guesses as to the
    function of that blob?

Hello Rod,

I will extend the investigation in this way.

Thanks for suggestions.

On Sat, Jul 24, 2010 at 8:24 AM, Rod Widdowson wrote:

> How can I do to extend this investigation?
>>
>
> Some ideas?
>
> - Does the AV have a user mode bit? If so look to see whether it has any
> IOs wedged against locks of yours. If not extend your search for all
> threads.
>
> - This deadlock clears itself right? So look for what it is that releases
> the object ("ba w 4 on the KEVENT is where I’d start).
>
> - What part of pool is the KEVENT in? Can you make any guesses as to the
> function of that blob?
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Hi, from the callstack you provided i suppose your antivirus files filter driver doesn’t supports recursion, as you can see you have AVXPTO! two times on the stack first on NtOpenFile and second after you call NtCreateFile (it handles the IRP_MJ_CREATE in both cases).

To avoid this situation you should use FltCreateFile, so you will avoid this problem, or you can try to create a system thread that calls ZwCreateFile.

> To avoid this situation you should use FltCreateFile

OP has a legacy so FltCreate is not available. Also FltCreate doesn’t stop
recursion - it just makes it less likely, but its real benefit is functional
isolation.

On legacy filters you could use the IoCreateFileSpecifyDeviceObjectHint routine to send a create request only to the filters below a specified device object and to the file system. http://msdn.microsoft.com/en-us/library/ff548289(VS.85).aspx