Norton AntiVirus deadlock with file encryption

NTFSD
1

Hi all,

My file encryption driver is currently suffering a dead-lock problem
when used with Norton AntiVirus 2005 or a service packed Norton 2003.
This problem was not occurring with the original version of Norton
2003, if the scanner is disabled from the tray icon or with other
Anti-Virus products.

My driver is receiving a IRP_MJ_CREATE request for an encrypted file.
During the create processing, a message is passed to my Win32
component (via a completed IOCTL IRP). The Win32 component is then
responsible for decrypting the file encryption key, asking the user to
insert their smartcard, etc. The original create request is held,
pending the completion of the Win32 processing.

During the Win32 processing, the application makes COM calls,
resulting in system dlls being accessed. The thread then appears to
deadlock on the dll file open request, waiting for the original create
request to complete.

It looks as though Norton AntiVirus is sometimes only allowing one
file to be scanned at a time.

Can anyone confirm that Norton AntiVirus imposes such a limitation?
Am I correct in assuming that it should be safe to perform such
processing in a Win32 application?

Norton appears to be swapping the kernel stacks. Does anyone know of a
way to see the kernel stack as it was before Norton swaps the stacks?
I can manually decode the Win32 stack associated with the thread, so I
have some idea of what is being processed, but it would be really
helpful if I could see the original kernel stack.

The only way that I can see to fix this problem is to remove the Win32
callback. However this severely limits the functionality of my
encryption product (no calls to CryptoAPI, no user dialog prompts, no
storage on smartcards or HSMs). Does anyone have any better
suggestions? (Other than not using Norton :slight_smile:

I have checked that my application is not deadlocking itself. Only one
encrypted file is being requested. The file being opened by the RPC
mechanism is rpcss.dll (and this isn’t encrypted). My driver is only
processing a single IRP on a single thread (see the trace below).

Thanks in advance,

Andy Larter

========== This is a dump of the original create request ==========
Its interesting that the request is coming from svchost, not explorer
as I was expecting

THREAD ff826020 Cid 02f0.04a0 Teb: 7ffdc000 Win32Thread: 00000000
WAIT: (Executive) KernelMode Non-Alertable
ff82648c SynchronizationEvent
IRP List:
ff508aa8: (0006,01b4) Flags: 00000884 Mdl: 00000000
ff508c68: (0006,01b4) Flags: 00000884 Mdl: 00000000
Impersonation token: e187f030 (Level Impersonation)
DeviceMap e14ad718
Owning Process ffae72f0 Image: svchost.exe
Wait Start TickCount 20160 Ticks: 2622 (0:00:00:26.257)
Context Switch Count 120
UserTime 00:00:00.0010
KernelTime 00:00:00.0040
Start Address kernel32!BaseThreadStartThunk (0x77e802f4)
Stack Init f636e000 Current f636d198 Base f636e000 Limit f636b000 Call
0
Priority 9 BasePriority 8 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr Args to Child
f636d1b0 804ea71c ff826090 ff826020 804ea5a3 nt!KiSwapContext+0x2e
(FPO: [EBP 0xf636d1e4] [0,0,4])
f636d1bc 804ea5a3 00000000 80ef0900 80dcc8d8 nt!KiSwapThread+0x44
(FPO: [0,0,2])
f636d1e4 f91191a2 00000000 00000000 00000000
nt!KeWaitForSingleObject+0x1c0 (FPO: [Non-Fpo])
f636d210 f911b51f 000004d5 ff5c8338 ffa3cd34
FileSafe!CPendingUserJob::WaitForCompletion+0x6d (FPO: [Non-Fpo])
(CONV: thiscall)
f636d268 f9109da3 f636d4a0 f636d338 f636d328
FileSafe!CUserConnectionProcessing::ScheduleDemonTask_OpenFile+0x189
(FPO: [Non-Fpo])
f636d350 f9108c58 ffb026b0 f636d400 f636d4a0
FileSafe!DriverCreateHandler_Open+0x50e (FPO: [Non-Fpo]) (CONV:
stdcall)
f636d4b4 f9118529 ffb026b0 00000000 80ef0900
FileSafe!DriverCreateHandler+0xb47 (FPO: [Non-Fpo]) (CONV: cdecl)
f636d4f0 f911dcbe 00000246 ffb026b0 e188dc90
FileSafe!CIRPContext::CallRoutine+0x7d (FPO: [Non-Fpo]) (CONV:
thiscall)
f636d514 804ec04f 80f1d428 ff508aa8 f636d578
FileSafe!Filter_FSDCreate+0xae (FPO: [Non-Fpo]) (CONV: stdcall)
f636d524 f7a6422b f636d578 f604d500 00000000 nt!IopfCallDriver+0x31
(FPO: [0,0,1])
WARNING: Stack unwind information not available. Following frames may
be wrong.
f636d688 8057069c 80f52620 00000000 ffa54f30
SYMEVENT!SYMEvent_GetVMDataPtr+0x886b
f636d70c 80572d6b 00000000 f636d74c 00000040
nt!ObpLookupObjectName+0x56a (FPO: [Non-Fpo])
f636d760 80574a10 00000000 00000000 f636d800
nt!ObOpenObjectByName+0xe9 (FPO: [Non-Fpo])
f636d7dc 80574ac1 ff504dac 80100180 ff504df0 nt!IopCreateFile+0x407
(FPO: [Non-Fpo])
f636d824 80578f0d ff504dac 80100180 ff504df0 nt!IoCreateFile+0x36
(FPO: [Non-Fpo])
f636d864 804d4e91 ff504dac 80100180 ff504df0 nt!NtCreateFile+0x2e
(FPO: [Non-Fpo])
f636d864 8050b15b ff504dac 80100180 ff504df0 nt!KiSystemService+0xc4
(FPO: [0,0] TrapFrame @ f636d898)
f636d908 f5c5e8b1 ff504dac 80100180 ff504df0 nt!ZwCreateFile+0x11
(FPO: [11,0,0])
f636d958 f5c36253 ff504f88 f5c36040 f636d970 SAVRT+0x2f8b1
e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 SAVRT+0x7253
e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 0xe1500580
f7a5aa00 f7a61750 f7a617f0 f7a5b9c0 f7a61840 0xe1500580
f7a63670 ffffe058 082444f6 56097401 00261be8
SYMEVENT!SYMEvent_GetVMDataPtr+0x5d90
e8f18b56 00000000 00000000 00000000 00000000 0xffffe058

========== This is a dump of the locked thread ==========

THREAD ff508338 Cid 03c4.010c Teb: 7ffd7000 Win32Thread: 00000000
WAIT: (Executive) KernelMode Non-Alertable
ff801d40 Mutant - owning thread ff826020
IRP List:
ff508160: (0006,01b4) Flags: 00000884 Mdl: 00000000
Not impersonating
DeviceMap e14ad718
Owning Process ff6188b8 Image: ASEFDA~1.EXE
Wait Start TickCount 20162 Ticks: 2620 (0:00:00:26.237)
Context Switch Count 1
UserTime 00:00:00.0000
KernelTime 00:00:00.0000
Start Address kernel32!BaseThreadStartThunk (0x77e802f4)
Win32 Start Address ole32!CRpcThreadCache::RpcWorkerThreadEntry
(0x771debfc)
Stack Init f5932000 Current f5931920 Base f5932000 Limit f592f000 Call
0
Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 0
ChildEBP RetAddr Args to Child
f5931938 804ea71c ff5083a8 ff508338 804ea5a3 nt!KiSwapContext+0x2e
(FPO: [EBP 0xf593196c] [0,0,4])
f5931944 804ea5a3 e188dc98 ff801d20 00000001 nt!KiSwapThread+0x44
(FPO: [0,0,2])
f593196c f5c39c3d 00000000 00000000 00000000
nt!KeWaitForSingleObject+0x1c0 (FPO: [Non-Fpo])
WARNING: Stack unwind information not available. Following frames may
be wrong.
e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 SAVRT+0xac3d
e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 0xe1500580
f7a5aa00 f7a61750 f7a617f0 f7a5b9c0 f7a61840 0xe1500580
f7a63670 ffffe058 082444f6 56097401 00261be8
SYMEVENT!SYMEvent_GetVMDataPtr+0x5d90
e8f18b56 00000000 00000000 00000000 00000000 0xffffe058

2

> encryption product (no calls to CryptoAPI, no user dialog prompts, no

storage on smartcards or HSMs).

No need in CryptoAPI in the kernel - take the sources from OpenSSL and use
them. Just mention this in the product documentation.

Asking the user while having the CREATE pended in the kernel mode is a bad
idea. Just fail such CREATE and send the async (not blocking the CREATE path)
notification to user mode than the encrypted files were opened without the
smartcard or password. The user app will suggest smartcard insertion or
password entering.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

3

Thanks for the reply. We already use our own crypto engine, but we are
getting pressure from customers to use certificates from external
sources (Active Directory and Entrust). I didn’t want to prevent such
integration in the future.

There seems to be two ways to handle removed smartcards:

The technique you suggest is to fail the requests. This is the most
secure method, but may lead to strange behavior in the users
applications. Word for instance gets very upset if some of the file
operations are failed.

The other method would be to ignore smart card removal and always
cache the keys. This isn’t as secure as I would like, but would
probably lead to a more reliable system.

Neither of these methods is ideal. We would also be prevented from
using sensitive keys. It’s annoying that our current system has been
working fine for the last two years.

I was hoping someone could identify this as a limitation with Norton
AntiVirus, rather than a design issue. After all, I can’t believe we
are the only people making use of Win32 processing during
IRP_MJ_CREATE handling.

Cheers Maxim.

  • Andy Larter

On Fri, 24 Jun 2005 20:47:39 +0400, “Maxim S. Shatskih”
wrote:

>> encryption product (no calls to CryptoAPI, no user dialog prompts, no
>> storage on smartcards or HSMs).
>
>No need in CryptoAPI in the kernel - take the sources from OpenSSL and use
>them. Just mention this in the product documentation.
>
>Asking the user while having the CREATE pended in the kernel mode is a bad
>idea. Just fail such CREATE and send the async (not blocking the CREATE path)
>notification to user mode than the encrypted files were opened without the
>smartcard or password. The user app will suggest smartcard insertion or
>password entering.
>
>Maxim Shatskih, Windows DDK MVP
>StorageCraft Corporation
>xxxxx@storagecraft.com
>http://www.storagecraft.com
>

4

Is there anyone on this mailing list who is *not* writing an encryption
driver? :slight_smile:

----- Original Message -----

On Fri, 24 Jun 2005 20:47:39 +0400, “Maxim S. Shatskih”
wrote:

>> encryption product (no calls to CryptoAPI, no user dialog prompts, no
>> storage on smartcards or HSMs).
>
>No need in CryptoAPI in the kernel - take the sources from OpenSSL and use
>them. Just mention this in the product documentation.
>
>Asking the user while having the CREATE pended in the kernel mode is a bad
>idea. Just fail such CREATE and send the async (not blocking the CREATE
path)
>notification to user mode than the encrypted files were opened without the
>smartcard or password. The user app will suggest smartcard insertion or
>password entering.
>
>Maxim Shatskih, Windows DDK MVP
>StorageCraft Corporation
>xxxxx@storagecraft.com
>http://www.storagecraft.com
>

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@netlib.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

5

Me :slight_smile:
----- Original Message -----
From: “Neil Weicher”
To: “Windows File Systems Devs Interest List”
Sent: Friday, June 24, 2005 10:11 AM
Subject: Re:[ntfsd] Norton AntiVirus deadlock with file encryption

Is there anyone on this mailing list who is not writing an encryption
driver? :slight_smile:

----- Original Message -----

On Fri, 24 Jun 2005 20:47:39 +0400, “Maxim S. Shatskih”
wrote:

>> encryption product (no calls to CryptoAPI, no user dialog prompts, no
>> storage on smartcards or HSMs).
>
>No need in CryptoAPI in the kernel - take the sources from OpenSSL and use
>them. Just mention this in the product documentation.
>
>Asking the user while having the CREATE pended in the kernel mode is a bad
>idea. Just fail such CREATE and send the async (not blocking the CREATE
path)
>notification to user mode than the encrypted files were opened without the
>smartcard or password. The user app will suggest smartcard insertion or
>password entering.
>
>Maxim Shatskih, Windows DDK MVP
>StorageCraft Corporation
>xxxxx@storagecraft.com
>http://www.storagecraft.com
>

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@netlib.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@rocketdivision.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

NOD32 1.1129 (20050605) Information

This message was checked by NOD32 antivirus system.
http://www.eset.com

6

There is always one who has to be different! :wink:

On Fri, 24 Jun 2005 10:15:07 -0700, “Jamey Kirby”
wrote:

>Me :slight_smile:
>----- Original Message -----
>From: “Neil Weicher”
>To: “Windows File Systems Devs Interest List”
>Sent: Friday, June 24, 2005 10:11 AM
>Subject: Re:[ntfsd] Norton AntiVirus deadlock with file encryption
>
>
>Is there anyone on this mailing list who is not writing an encryption
>driver? :slight_smile:
>
>----- Original Message -----
>
>On Fri, 24 Jun 2005 20:47:39 +0400, “Maxim S. Shatskih”
> wrote:
>
>>> encryption product (no calls to CryptoAPI, no user dialog prompts, no
>>> storage on smartcards or HSMs).
>>
>>No need in CryptoAPI in the kernel - take the sources from OpenSSL and use
>>them. Just mention this in the product documentation.
>>
>>Asking the user while having the CREATE pended in the kernel mode is a bad
>>idea. Just fail such CREATE and send the async (not blocking the CREATE
>path)
>>notification to user mode than the encrypted files were opened without the
>>smartcard or password. The user app will suggest smartcard insertion or
>>password entering.
>>
>>Maxim Shatskih, Windows DDK MVP
>>StorageCraft Corporation
>>xxxxx@storagecraft.com
>>http://www.storagecraft.com
>>
>
>
>—
>Questions? First check the IFS FAQ at
>https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@netlib.com
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>—
>Questions? First check the IFS FAQ at
>https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@rocketdivision.com
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>NOD32 1.1129 (20050605) Information
>
>This message was checked by NOD32 antivirus system.
>http://www.eset.com
>
>

7

Me too :-o

“Jamey Kirby” wrote in message
news:xxxxx@ntfsd…
> Me :slight_smile:
> ----- Original Message -----
> From: “Neil Weicher”
> To: “Windows File Systems Devs Interest List”
> Sent: Friday, June 24, 2005 10:11 AM
> Subject: Re:[ntfsd] Norton AntiVirus deadlock with file encryption
>
>
> Is there anyone on this mailing list who is not writing an encryption
> driver? :slight_smile:
>
> ----- Original Message -----
>
> On Fri, 24 Jun 2005 20:47:39 +0400, “Maxim S. Shatskih”
> wrote:
>
>>> encryption product (no calls to CryptoAPI, no user dialog prompts, no
>>> storage on smartcards or HSMs).
>>
>>No need in CryptoAPI in the kernel - take the sources from OpenSSL and use
>>them. Just mention this in the product documentation.
>>
>>Asking the user while having the CREATE pended in the kernel mode is a bad
>>idea. Just fail such CREATE and send the async (not blocking the CREATE
> path)
>>notification to user mode than the encrypted files were opened without the
>>smartcard or password. The user app will suggest smartcard insertion or
>>password entering.
>>
>>Maxim Shatskih, Windows DDK MVP
>>StorageCraft Corporation
>>xxxxx@storagecraft.com
>>http://www.storagecraft.com
>>
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@netlib.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@rocketdivision.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> NOD32 1.1129 (20050605) Information
>
> This message was checked by NOD32 antivirus system.
> http://www.eset.com
>
>
>

8

> The technique you suggest is to fail the requests. This is the most

secure method, but may lead to strange behavior in the users
applications. Word for instance gets very upset if some of the file
operations are failed.

Fail CREATE only.

AntiVirus, rather than a design issue. After all, I can’t believe we
are the only people making use of Win32 processing during
IRP_MJ_CREATE handling.

IIRC this was causing problems for everybody.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

9

I don’t have to since firmware in our Momentus FDE model encrypts the entire
drive. :slight_smile:

The personal opinion of
Gary G. Little

“Neil Weicher” wrote in message news:xxxxx@ntfsd…
> Is there anyone on this mailing list who is not writing an encryption
> driver? :slight_smile:
>
> ----- Original Message -----
>
> On Fri, 24 Jun 2005 20:47:39 +0400, “Maxim S. Shatskih”
> wrote:
>
>>> encryption product (no calls to CryptoAPI, no user dialog prompts, no
>>> storage on smartcards or HSMs).
>>
>>No need in CryptoAPI in the kernel - take the sources from OpenSSL and use
>>them. Just mention this in the product documentation.
>>
>>Asking the user while having the CREATE pended in the kernel mode is a bad
>>idea. Just fail such CREATE and send the async (not blocking the CREATE
> path)
>>notification to user mode than the encrypted files were opened without the
>>smartcard or password. The user app will suggest smartcard insertion or
>>password entering.
>>
>>Maxim Shatskih, Windows DDK MVP
>>StorageCraft Corporation
>>xxxxx@storagecraft.com
>>http://www.storagecraft.com
>>
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@netlib.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>

10

NTFS switches stacks also. Microsoft put support in the OS to do stack
switches for NTFS. Too many people writing encryption drivers? Maybe.

“Andy Larter” wrote in message
news:xxxxx@ntfsd…
> Hi all,
>
> My file encryption driver is currently suffering a dead-lock problem
> when used with Norton AntiVirus 2005 or a service packed Norton 2003.
> This problem was not occurring with the original version of Norton
> 2003, if the scanner is disabled from the tray icon or with other
> Anti-Virus products.
>
> My driver is receiving a IRP_MJ_CREATE request for an encrypted file.
> During the create processing, a message is passed to my Win32
> component (via a completed IOCTL IRP). The Win32 component is then
> responsible for decrypting the file encryption key, asking the user to
> insert their smartcard, etc. The original create request is held,
> pending the completion of the Win32 processing.
>
> During the Win32 processing, the application makes COM calls,
> resulting in system dlls being accessed. The thread then appears to
> deadlock on the dll file open request, waiting for the original create
> request to complete.
>
> It looks as though Norton AntiVirus is sometimes only allowing one
> file to be scanned at a time.
>
> Can anyone confirm that Norton AntiVirus imposes such a limitation?
> Am I correct in assuming that it should be safe to perform such
> processing in a Win32 application?
>
> Norton appears to be swapping the kernel stacks. Does anyone know of a
> way to see the kernel stack as it was before Norton swaps the stacks?
> I can manually decode the Win32 stack associated with the thread, so I
> have some idea of what is being processed, but it would be really
> helpful if I could see the original kernel stack.
>
> The only way that I can see to fix this problem is to remove the Win32
> callback. However this severely limits the functionality of my
> encryption product (no calls to CryptoAPI, no user dialog prompts, no
> storage on smartcards or HSMs). Does anyone have any better
> suggestions? (Other than not using Norton :slight_smile:
>
> I have checked that my application is not deadlocking itself. Only one
> encrypted file is being requested. The file being opened by the RPC
> mechanism is rpcss.dll (and this isn’t encrypted). My driver is only
> processing a single IRP on a single thread (see the trace below).
>
>
> Thanks in advance,
>
> Andy Larter
>
>
>
>
> ========== This is a dump of the original create request ==========
> Its interesting that the request is coming from svchost, not explorer
> as I was expecting
>
> THREAD ff826020 Cid 02f0.04a0 Teb: 7ffdc000 Win32Thread: 00000000
> WAIT: (Executive) KernelMode Non-Alertable
> ff82648c SynchronizationEvent
> IRP List:
> ff508aa8: (0006,01b4) Flags: 00000884 Mdl: 00000000
> ff508c68: (0006,01b4) Flags: 00000884 Mdl: 00000000
> Impersonation token: e187f030 (Level Impersonation)
> DeviceMap e14ad718
> Owning Process ffae72f0 Image: svchost.exe
> Wait Start TickCount 20160 Ticks: 2622 (0:00:00:26.257)
> Context Switch Count 120
> UserTime 00:00:00.0010
> KernelTime 00:00:00.0040
> Start Address kernel32!BaseThreadStartThunk (0x77e802f4)
> Stack Init f636e000 Current f636d198 Base f636e000 Limit f636b000 Call
> 0
> Priority 9 BasePriority 8 PriorityDecrement 0 DecrementCount 16
> ChildEBP RetAddr Args to Child
> f636d1b0 804ea71c ff826090 ff826020 804ea5a3 nt!KiSwapContext+0x2e
> (FPO: [EBP 0xf636d1e4] [0,0,4])
> f636d1bc 804ea5a3 00000000 80ef0900 80dcc8d8 nt!KiSwapThread+0x44
> (FPO: [0,0,2])
> f636d1e4 f91191a2 00000000 00000000 00000000
> nt!KeWaitForSingleObject+0x1c0 (FPO: [Non-Fpo])
> f636d210 f911b51f 000004d5 ff5c8338 ffa3cd34
> FileSafe!CPendingUserJob::WaitForCompletion+0x6d (FPO: [Non-Fpo])
> (CONV: thiscall)
> f636d268 f9109da3 f636d4a0 f636d338 f636d328
> FileSafe!CUserConnectionProcessing::ScheduleDemonTask_OpenFile+0x189
> (FPO: [Non-Fpo])
> f636d350 f9108c58 ffb026b0 f636d400 f636d4a0
> FileSafe!DriverCreateHandler_Open+0x50e (FPO: [Non-Fpo]) (CONV:
> stdcall)
> f636d4b4 f9118529 ffb026b0 00000000 80ef0900
> FileSafe!DriverCreateHandler+0xb47 (FPO: [Non-Fpo]) (CONV: cdecl)
> f636d4f0 f911dcbe 00000246 ffb026b0 e188dc90
> FileSafe!CIRPContext::CallRoutine+0x7d (FPO: [Non-Fpo]) (CONV:
> thiscall)
> f636d514 804ec04f 80f1d428 ff508aa8 f636d578
> FileSafe!Filter_FSDCreate+0xae (FPO: [Non-Fpo]) (CONV: stdcall)
> f636d524 f7a6422b f636d578 f604d500 00000000 nt!IopfCallDriver+0x31
> (FPO: [0,0,1])
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> f636d688 8057069c 80f52620 00000000 ffa54f30
> SYMEVENT!SYMEvent_GetVMDataPtr+0x886b
> f636d70c 80572d6b 00000000 f636d74c 00000040
> nt!ObpLookupObjectName+0x56a (FPO: [Non-Fpo])
> f636d760 80574a10 00000000 00000000 f636d800
> nt!ObOpenObjectByName+0xe9 (FPO: [Non-Fpo])
> f636d7dc 80574ac1 ff504dac 80100180 ff504df0 nt!IopCreateFile+0x407
> (FPO: [Non-Fpo])
> f636d824 80578f0d ff504dac 80100180 ff504df0 nt!IoCreateFile+0x36
> (FPO: [Non-Fpo])
> f636d864 804d4e91 ff504dac 80100180 ff504df0 nt!NtCreateFile+0x2e
> (FPO: [Non-Fpo])
> f636d864 8050b15b ff504dac 80100180 ff504df0 nt!KiSystemService+0xc4
> (FPO: [0,0] TrapFrame @ f636d898)
> f636d908 f5c5e8b1 ff504dac 80100180 ff504df0 nt!ZwCreateFile+0x11
> (FPO: [11,0,0])
> f636d958 f5c36253 ff504f88 f5c36040 f636d970 SAVRT+0x2f8b1
> e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 SAVRT+0x7253
> e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 0xe1500580
> f7a5aa00 f7a61750 f7a617f0 f7a5b9c0 f7a61840 0xe1500580
> f7a63670 ffffe058 082444f6 56097401 00261be8
> SYMEVENT!SYMEvent_GetVMDataPtr+0x5d90
> e8f18b56 00000000 00000000 00000000 00000000 0xffffe058
>
>
> ========== This is a dump of the locked thread ==========
>
> THREAD ff508338 Cid 03c4.010c Teb: 7ffd7000 Win32Thread: 00000000
> WAIT: (Executive) KernelMode Non-Alertable
> ff801d40 Mutant - owning thread ff826020
> IRP List:
> ff508160: (0006,01b4) Flags: 00000884 Mdl: 00000000
> Not impersonating
> DeviceMap e14ad718
> Owning Process ff6188b8 Image: ASEFDA~1.EXE
> Wait Start TickCount 20162 Ticks: 2620 (0:00:00:26.237)
> Context Switch Count 1
> UserTime 00:00:00.0000
> KernelTime 00:00:00.0000
> Start Address kernel32!BaseThreadStartThunk (0x77e802f4)
> Win32 Start Address ole32!CRpcThreadCache::RpcWorkerThreadEntry
> (0x771debfc)
> Stack Init f5932000 Current f5931920 Base f5932000 Limit f592f000 Call
> 0
> Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 0
> ChildEBP RetAddr Args to Child
> f5931938 804ea71c ff5083a8 ff508338 804ea5a3 nt!KiSwapContext+0x2e
> (FPO: [EBP 0xf593196c] [0,0,4])
> f5931944 804ea5a3 e188dc98 ff801d20 00000001 nt!KiSwapThread+0x44
> (FPO: [0,0,2])
> f593196c f5c39c3d 00000000 00000000 00000000
> nt!KeWaitForSingleObject+0x1c0 (FPO: [Non-Fpo])
> WARNING: Stack unwind information not available. Following frames may
> be wrong.
> e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 SAVRT+0xac3d
> e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 0xe1500580
> f7a5aa00 f7a61750 f7a617f0 f7a5b9c0 f7a61840 0xe1500580
> f7a63670 ffffe058 082444f6 56097401 00261be8
> SYMEVENT!SYMEvent_GetVMDataPtr+0x5d90
> e8f18b56 00000000 00000000 00000000 00000000 0xffffe058
>
>

11

Can you name the exact calls used by NTFS?

From what I know, the only way of stack switching MS uses is
Ex/IoQueueWorkItem, which yes - widely used inside the FSDs.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “David J. Craig”
Newsgroups: ntfsd
To: “Windows File Systems Devs Interest List”
Sent: Saturday, June 25, 2005 11:07 AM
Subject: Re:[ntfsd] Norton AntiVirus deadlock with file encryption

> NTFS switches stacks also. Microsoft put support in the OS to do stack
> switches for NTFS. Too many people writing encryption drivers? Maybe.
>
> “Andy Larter” wrote in message
> news:xxxxx@ntfsd…
> > Hi all,
> >
> > My file encryption driver is currently suffering a dead-lock problem
> > when used with Norton AntiVirus 2005 or a service packed Norton 2003.
> > This problem was not occurring with the original version of Norton
> > 2003, if the scanner is disabled from the tray icon or with other
> > Anti-Virus products.
> >
> > My driver is receiving a IRP_MJ_CREATE request for an encrypted file.
> > During the create processing, a message is passed to my Win32
> > component (via a completed IOCTL IRP). The Win32 component is then
> > responsible for decrypting the file encryption key, asking the user to
> > insert their smartcard, etc. The original create request is held,
> > pending the completion of the Win32 processing.
> >
> > During the Win32 processing, the application makes COM calls,
> > resulting in system dlls being accessed. The thread then appears to
> > deadlock on the dll file open request, waiting for the original create
> > request to complete.
> >
> > It looks as though Norton AntiVirus is sometimes only allowing one
> > file to be scanned at a time.
> >
> > Can anyone confirm that Norton AntiVirus imposes such a limitation?
> > Am I correct in assuming that it should be safe to perform such
> > processing in a Win32 application?
> >
> > Norton appears to be swapping the kernel stacks. Does anyone know of a
> > way to see the kernel stack as it was before Norton swaps the stacks?
> > I can manually decode the Win32 stack associated with the thread, so I
> > have some idea of what is being processed, but it would be really
> > helpful if I could see the original kernel stack.
> >
> > The only way that I can see to fix this problem is to remove the Win32
> > callback. However this severely limits the functionality of my
> > encryption product (no calls to CryptoAPI, no user dialog prompts, no
> > storage on smartcards or HSMs). Does anyone have any better
> > suggestions? (Other than not using Norton :slight_smile:
> >
> > I have checked that my application is not deadlocking itself. Only one
> > encrypted file is being requested. The file being opened by the RPC
> > mechanism is rpcss.dll (and this isn’t encrypted). My driver is only
> > processing a single IRP on a single thread (see the trace below).
> >
> >
> > Thanks in advance,
> >
> > Andy Larter
> >
> >
> >
> >
> > ========== This is a dump of the original create request ==========
> > Its interesting that the request is coming from svchost, not explorer
> > as I was expecting
> >
> > THREAD ff826020 Cid 02f0.04a0 Teb: 7ffdc000 Win32Thread: 00000000
> > WAIT: (Executive) KernelMode Non-Alertable
> > ff82648c SynchronizationEvent
> > IRP List:
> > ff508aa8: (0006,01b4) Flags: 00000884 Mdl: 00000000
> > ff508c68: (0006,01b4) Flags: 00000884 Mdl: 00000000
> > Impersonation token: e187f030 (Level Impersonation)
> > DeviceMap e14ad718
> > Owning Process ffae72f0 Image: svchost.exe
> > Wait Start TickCount 20160 Ticks: 2622 (0:00:00:26.257)
> > Context Switch Count 120
> > UserTime 00:00:00.0010
> > KernelTime 00:00:00.0040
> > Start Address kernel32!BaseThreadStartThunk (0x77e802f4)
> > Stack Init f636e000 Current f636d198 Base f636e000 Limit f636b000 Call
> > 0
> > Priority 9 BasePriority 8 PriorityDecrement 0 DecrementCount 16
> > ChildEBP RetAddr Args to Child
> > f636d1b0 804ea71c ff826090 ff826020 804ea5a3 nt!KiSwapContext+0x2e
> > (FPO: [EBP 0xf636d1e4] [0,0,4])
> > f636d1bc 804ea5a3 00000000 80ef0900 80dcc8d8 nt!KiSwapThread+0x44
> > (FPO: [0,0,2])
> > f636d1e4 f91191a2 00000000 00000000 00000000
> > nt!KeWaitForSingleObject+0x1c0 (FPO: [Non-Fpo])
> > f636d210 f911b51f 000004d5 ff5c8338 ffa3cd34
> > FileSafe!CPendingUserJob::WaitForCompletion+0x6d (FPO: [Non-Fpo])
> > (CONV: thiscall)
> > f636d268 f9109da3 f636d4a0 f636d338 f636d328
> > FileSafe!CUserConnectionProcessing::ScheduleDemonTask_OpenFile+0x189
> > (FPO: [Non-Fpo])
> > f636d350 f9108c58 ffb026b0 f636d400 f636d4a0
> > FileSafe!DriverCreateHandler_Open+0x50e (FPO: [Non-Fpo]) (CONV:
> > stdcall)
> > f636d4b4 f9118529 ffb026b0 00000000 80ef0900
> > FileSafe!DriverCreateHandler+0xb47 (FPO: [Non-Fpo]) (CONV: cdecl)
> > f636d4f0 f911dcbe 00000246 ffb026b0 e188dc90
> > FileSafe!CIRPContext::CallRoutine+0x7d (FPO: [Non-Fpo]) (CONV:
> > thiscall)
> > f636d514 804ec04f 80f1d428 ff508aa8 f636d578
> > FileSafe!Filter_FSDCreate+0xae (FPO: [Non-Fpo]) (CONV: stdcall)
> > f636d524 f7a6422b f636d578 f604d500 00000000 nt!IopfCallDriver+0x31
> > (FPO: [0,0,1])
> > WARNING: Stack unwind information not available. Following frames may
> > be wrong.
> > f636d688 8057069c 80f52620 00000000 ffa54f30
> > SYMEVENT!SYMEvent_GetVMDataPtr+0x886b
> > f636d70c 80572d6b 00000000 f636d74c 00000040
> > nt!ObpLookupObjectName+0x56a (FPO: [Non-Fpo])
> > f636d760 80574a10 00000000 00000000 f636d800
> > nt!ObOpenObjectByName+0xe9 (FPO: [Non-Fpo])
> > f636d7dc 80574ac1 ff504dac 80100180 ff504df0 nt!IopCreateFile+0x407
> > (FPO: [Non-Fpo])
> > f636d824 80578f0d ff504dac 80100180 ff504df0 nt!IoCreateFile+0x36
> > (FPO: [Non-Fpo])
> > f636d864 804d4e91 ff504dac 80100180 ff504df0 nt!NtCreateFile+0x2e
> > (FPO: [Non-Fpo])
> > f636d864 8050b15b ff504dac 80100180 ff504df0 nt!KiSystemService+0xc4
> > (FPO: [0,0] TrapFrame @ f636d898)
> > f636d908 f5c5e8b1 ff504dac 80100180 ff504df0 nt!ZwCreateFile+0x11
> > (FPO: [11,0,0])
> > f636d958 f5c36253 ff504f88 f5c36040 f636d970 SAVRT+0x2f8b1
> > e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 SAVRT+0x7253
> > e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 0xe1500580
> > f7a5aa00 f7a61750 f7a617f0 f7a5b9c0 f7a61840 0xe1500580
> > f7a63670 ffffe058 082444f6 56097401 00261be8
> > SYMEVENT!SYMEvent_GetVMDataPtr+0x5d90
> > e8f18b56 00000000 00000000 00000000 00000000 0xffffe058
> >
> >
> > ========== This is a dump of the locked thread ==========
> >
> > THREAD ff508338 Cid 03c4.010c Teb: 7ffd7000 Win32Thread: 00000000
> > WAIT: (Executive) KernelMode Non-Alertable
> > ff801d40 Mutant - owning thread ff826020
> > IRP List:
> > ff508160: (0006,01b4) Flags: 00000884 Mdl: 00000000
> > Not impersonating
> > DeviceMap e14ad718
> > Owning Process ff6188b8 Image: ASEFDA~1.EXE
> > Wait Start TickCount 20162 Ticks: 2620 (0:00:00:26.237)
> > Context Switch Count 1
> > UserTime 00:00:00.0000
> > KernelTime 00:00:00.0000
> > Start Address kernel32!BaseThreadStartThunk (0x77e802f4)
> > Win32 Start Address ole32!CRpcThreadCache::RpcWorkerThreadEntry
> > (0x771debfc)
> > Stack Init f5932000 Current f5931920 Base f5932000 Limit f592f000 Call
> > 0
> > Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 0
> > ChildEBP RetAddr Args to Child
> > f5931938 804ea71c ff5083a8 ff508338 804ea5a3 nt!KiSwapContext+0x2e
> > (FPO: [EBP 0xf593196c] [0,0,4])
> > f5931944 804ea5a3 e188dc98 ff801d20 00000001 nt!KiSwapThread+0x44
> > (FPO: [0,0,2])
> > f593196c f5c39c3d 00000000 00000000 00000000
> > nt!KeWaitForSingleObject+0x1c0 (FPO: [Non-Fpo])
> > WARNING: Stack unwind information not available. Following frames may
> > be wrong.
> > e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 SAVRT+0xac3d
> > e14fa528 e1500580 f7a59480 e21b6f40 f7a59484 0xe1500580
> > f7a5aa00 f7a61750 f7a617f0 f7a5b9c0 f7a61840 0xe1500580
> > f7a63670 ffffe058 082444f6 56097401 00261be8
> > SYMEVENT!SYMEvent_GetVMDataPtr+0x5d90
> > e8f18b56 00000000 00000000 00000000 00000000 0xffffe058
> >
> >
>
>
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

12

> Norton appears to be swapping the kernel stacks. Does anyone know of a

way to see the kernel stack as it was before Norton swaps the stacks?

This is true. But I think they must soon stop using that
technique, because the SP1 for 2003 server forbids this technique
( I’ve read about it in the BookOfSP1 document), but yet it seems
to be for 64-bit editions only.

After all, I can’t believe we
are the only people making use of Win32 processing during
IRP_MJ_CREATE handling.

You are not. We use ver similar approach.

Try to remove the deadlock this way:

When the user mode app receives the notification, try to open the
file again from that user mode app. (just CreateFile - CloseHandle).
This should generally deadlock the system too
I think if you’ll be able to make THIS working, you will not have
a problem with NAV either.

If you want, we can discuss this in bigger detail, but I would
prefer to do this off-list, the best on ICQ or MSN. My
ICQ number is 153292074.

L.

13

Thanks for the response guys.

My problem is that failing the request in the open processing is the
simple case to fix. The real problem comes when a user creates a new
encrypted file. (That is, the user saves a file into an encrypted
folder, rather than using our applications to encrypt an existing
clear file). In this case, the workstation may not have the
appropriate keys and will have to communicate with a remote key server
to retrieve the keys then check the trust. It must do this before
returning the Create IRP. Currently I can only do this processing from
Win32. (As a last resort I could port this code to the driver. Yuck!).

Before I do that, I’m going to investigate installing another
DEVICE_OBJECT to the top of the file system stacks after the virus
scanner is started. If I can do the key processing before the
Anti-Virus driver gets the IRPs I should be able to avoid the
deadlock. My driver will still use the low-level DO for handling block
encryption, file renames, etc, but it will not need to block the
request.

Thanks.

  • Andy Larter
14

I’m a bit late to this thread however, this may be relavent (or not).

We had a similar deadlock problem with NAV a year or so ago. It turned out
the main culprit was an internal change to APC handling and locks in WS03.
There’s been a bunch written on this topic (guarded regions). Also, the
Norton network filter (SYMTDI.SYS) was (erroneously) calling ZwCreateFile
with kernel special APCs disabled. This caused the NAV filter to wait
forever for an event from a lower filter which cannot be signaled due to an
undetected (by checking IRQL) call with ACPs disabled. Worse yet,
KeAreApcsDisabled in WS03 SP0 returns false – New function in SP1:
KeAreAllApcsDisabled

Check the version of SYMTDI.SYS (if this is on your system) and look for a
date of June 2004 or later. Anything earlier, IIRC, has this problem.

Any Norton folks around here?

HTH, /ted

-----Original Message-----
From: Andy Larter [mailto:xxxxx@ArmourSoft.com]
Sent: Monday, June 27, 2005 9:55 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Norton AntiVirus deadlock with file encryption

Thanks for the response guys.

My problem is that failing the request in the open processing is the simple
case to fix. The real problem comes when a user creates a new encrypted
file. (That is, the user saves a file into an encrypted folder, rather than
using our applications to encrypt an existing clear file). In this case, the
workstation may not have the appropriate keys and will have to communicate
with a remote key server to retrieve the keys then check the trust. It must
do this before returning the Create IRP. Currently I can only do this
processing from Win32. (As a last resort I could port this code to the
driver. Yuck!).

Before I do that, I’m going to investigate installing another DEVICE_OBJECT
to the top of the file system stacks after the virus scanner is started. If
I can do the key processing before the Anti-Virus driver gets the IRPs I
should be able to avoid the deadlock. My driver will still use the low-level
DO for handling block encryption, file renames, etc, but it will not need to
block the request.

Thanks.

  • Andy Larter

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To unsubscribe
send a blank email to xxxxx@lists.osr.com

15

Thanks for the info Ted. I hadn't considered looking for APC problems.

I've installed a checked version of the kernel/hal onto my test
machine. I now get a bug check during the loading of Norton. It looks
like the SymTDI driver is not calling KeLeaveCriticalRegion on exit
from an IOCTL call. This is without my driver running.

I have checked the SYMTDI.SYS file and it appears to be a very recent
version. So perhaps they have reintroduced a previous bug.

Interestingly, I can't get Norton 2003 to install on a checked XP or
checked XP Sp1. The install quits without error mid-install. Norton
2005 will install, but throws a IRP_NOT_LESS_OR_EQUAL bug on loading
SavRT. Good to know Symantec use checked builds for testing. :frowning:

I'll try and raise a bug report with Symantec and see what happens.

Cheers all.

  • Andy

If anyone is interested, the bug check analysis is as follows:

APC_INDEX_MISMATCH (1)
This is a kernel internal error which can occur only on a checked
build.
The most common reason to see such a bugcheck would occur when a
filesystem had a mismatched number of KeEnterCriticalRegion calls
compared
to KeLeaveCriticalRegion calls. This check is made on exit from a
system
call.
Arguments:
Arg1: 805dada2, address of system function (system call)
Arg2: 00010000, Thread->ApcStateIndex << 8 | Previous ApcStateIndex
Arg3: ffffffff, Thread->KernelApcDicable
Arg4: 00000000, Previous KernelApcDisable

Debugging Details:

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0x1

LAST_CONTROL_TRANSFER: from 804fc9ae to 80579b54

STACK_TEXT:
f7ace908 804fc9ae 00000003 00000000 00000001
nt!RtlpBreakWithStatusInstruction
f7ace954 804fd2f1 00000003 f7aced58 00000000
nt!KiBugCheckDebugBreak+0x19
f7aced1c 804fd7f7 00000001 805dada2 00010000 nt!KeBugCheck2+0x43c
f7aced3c 80591d75 00000001 805dada2 00010000 nt!KeBugCheckEx+0x19
f7aced3c 7ffe0304 00000001 805dada2 00010000 nt!KiServiceExit2+0x1d6
0071f160 77f7e7df 77e73fcb 00000480 00000000
SharedUserData!SystemCallStub+0x4
0071f164 77e73fcb 00000480 00000000 00000000
ntdll!NtDeviceIoControlFile+0xc
0071f1c4 10009a9c 00000480 83022227 0071f214
kernel32!DeviceIoControl+0xdd
WARNING: Stack unwind information not available. Following frames may
be wrong.
00000000 00000000 00000000 00000000 00000000 ccTrust!Ordinal10+0x7bbc

16

> Interestingly, I can’t get Norton 2003 to install on a checked XP or

checked XP Sp1. The install quits without error mid-install. Norton
2005 will install, but throws a IRP_NOT_LESS_OR_EQUAL bug on loading
SavRT. Good to know Symantec use checked builds for testing. :frowning:

Try this:
Install a normal build of the operating system, then install the AV
and all what you need to test. After all works, replace the kernel
and HAL by their checked version (there is an article at osr about
this).

L.