Resolving 0x7F bugcheck, exception_double_fault

OSR_Community_User · March 18, 2002, 7:44am

Hi guys,

During stress-test with our driver loaded together with an anti-virus
driver (Symantec Norton 2002 in this case), we have experienced some
bugchecks after a couple of days, mostly 0x7F
(UNEXPECTED_KERNEL_MODE_TRAP). The debugger tells us that the
declaration and initialization of a parameter on the local stack is the
problem, however I find that hard to believe. Who can explain the
situation for me ?

What I’m curious about, are the following lines from an IRP stack trace
(full details are listed below):

[3, 0] 0 e0 81427020 81424038 80531396-8462e414 Success Error
Cancel
\FileSystem\Ntfs nt!IovpInternalCompletionTrap
Args: 00001000 00000000 00001000 00000000
[3, 0] 0 e0 81307020 81424038 80531396-8462e438 Success Error
Cancel
\Driver\ThTrack nt!IovpInternalCompletionTrap
Args: 00001000 00000000 00001000 00000000

What is the cause of the completion routine appearing in both Ntfs and
my driver (ThTrack) ? Any hints or tips how to track down the cause of
the problem using WinDbg ?

Below you’ll find some more details.

Best,
Bartjan Wattel.

Some details: (ThTrac2k is our driver):

************************************************************************
*******
*
*
* Bugcheck Analysis
*
*
*
************************************************************************
*******

UNEXPECTED_KERNEL_MODE_TRAP (7f)
This means a trap occurred in kernel mode, and it’s a trap of a kind
that the kernel isn’t allowed to have/catch (bound trap) or that
is always instant death (double fault). The first number in the
bugcheck parens is the number of the trap (8 = double fault, etc)
Consult an Intel x86 family manual to learn more about what these
traps are. Here is a *portion* of those codes:
If kv shows a taskGate
use .tss on the part before the colon, then kv.
Else if kv shows a trapframe
use .trap on that value
Else
.trap on the appropriate frame will show where the trap was
taken
(on x86, this will be the ebp that goes with the procedure
KiTrap)
Endif
kb will then show the corrected stack.
Arguments:
Arg1: 00000008, EXCEPTION_DOUBLE_FAULT
Arg2: 00000000
Arg3: 00000000
Arg4: 00000000

Debugging Details:

*** ERROR: Symbol file could not be found. Defaulted to export symbols
for SYMEVENT.SYS -

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 7F

LAST_CONTROL_TRANSFER: from 8042bef7 to 80455994

Looking at the IRP that is passed to my dispatch routine, I get the
following:

kd> !irp 8462e2a8
Irp is active with 9 stacks 7 is current (= 0x8462e3f0)
Mdl = 871ba8e0 Thread 810d2020: Irp stack trace.
cmd flg cl Device File Completion-Context
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 10 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000

[3, 0] 0 e0 81427020 81424038 80531396-8462e414 Success Error
Cancel
\FileSystem\Ntfs nt!IovpInternalCompletionTrap
Args: 00001000 00000000 00001000 00000000
[3, 0] 0 e0 81307020 81424038 80531396-8462e438 Success Error
Cancel
\Driver\ThTrack nt!IovpInternalCompletionTrap
Args: 00001000 00000000 00001000 00000000
[3, 0] 0 0 812701e0 81424038 00000000-00000000
\Driver\SymEvent
Args: 00001000 00000000 00001000 00000000

kd> !thread
THREAD 810d2020 Cid 468.440 Teb: 7ffdb000 Win32Thread: a6c21ec0
RUNNING
IRP List:
83b572a8: (0006,01b4) Flags: 40000884 Mdl: 00000000
841c72a8: (0006,01b4) Flags: 40000884 Mdl: 00000000
a58d7e48: (0006,01b4) Flags: 40000884 Mdl: 00000000
Not impersonating
Owning Process 81124d60
WaitTime (seconds) 19606726
Context Switch Count 111499 LargeStack
UserTime 0:00:00.0090
KernelTime 0:00:13.0719
Start Address KERNEL32!BaseThreadStartThunk (0x77e87532)
Win32 Start Address msvcrt!_beginthreadex (0x7800a361)
Stack Init eef64000 Current eef61ae4 Base eef64000 Limit eef61000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16

ChildEBP RetAddr Args to Child
804700a4 8042bef7 00000004 ffdff408 804703b8
nt!RtlpBreakWithStatusInstruction
804700d4 8042c438 00000004 00000000 00000000
nt!KiBugCheckDebugBreak+0x31
80470460 804669be 0000007f 00000008 00000000 nt!KeBugCheckEx+0x5d7
80470460 ef6e0817 0000007f 00000008 00000000 nt!KiTrap08+0x3e
eef61164 ef6e1028 81307020 819872a8 80063124
ThTrac2k!ThTrackHookRoutine+0x11
eef6117c 80530510 eef63df0 819872a8 819872a8
ThTrac2k!ThTrackDispatch+0x4f
eef611c8 8052fcd5 aee47fa0 819872a8 81308450
nt!IovSpecialIrpCallDriver+0xcd
eef611e4 ef29d065 812701e0 80063124 80530510 nt!IovCallDriver+0x31
eef6123c 80420964 00000000 00000000 80062f10 SYMEVENT+0x1065
eef61250 80440a0d 81422668 87b5c8a0 87b5c880 nt!IoPageRead+0xb1
eef61290 8044a0c4 00000001 a83fdf1c c02a0ff4 nt!MiDispatchFault+0x23d
eef612dc 80467cc6 00000001 00000000 00000000 nt!MmAccessFault+0x67b
eef612dc bfed6d71 00000001 00000000 00000000 nt!KiTrap0E+0xc3
eef61380 bfed789d a83f9ee8 00000002 00000000
Ntfs!NtfsLookupNtfsMcbEntry+0x16c
eef6148c bfed7f8b 8121abc8 a83f9e48 00000002
Ntfs!NtfsLookupAllocation+0x58
eef6165c bfed6e87 8121abc8 8462e2a8 a83f9e48
Ntfs!NtfsPrepareBuffers+0x25e
eef61830 bfed744d 8121abc8 8462e2a8 a83f9e48 Ntfs!NtfsNonCachedIo+0x121
eef61a48 bfed8a3b 8121abc8 8462e2a8 00000001 Ntfs!NtfsCommonRead+0xeee
eef61ae4 80530510 81427020 8462e2a8 8462e2a8 Ntfs!NtfsFsdRead+0x201
eef61b30 8052fcd5 8462e40c 8462e430 8462e2a8
nt!IovSpecialIrpCallDriver+0xcd
eef61b4c ef6e0e2a 81307020 81307020 812f9dd0 nt!IovCallDriver+0x31
eef61ce0 ef6e1028 81307020 8462e2a8 80063124
ThTrac2k!ThTrackHookRoutine+0x624
eef61cf8 80530510 eef63df0 8462e2a8 8462e2a8
ThTrac2k!ThTrackDispatch+0x4f
eef61d44 8052fcd5 aee47fa0 8462e2a8 81308450
nt!IovSpecialIrpCallDriver+0xcd
eef61d60 ef29d065 812701e0 80063124 80530510 nt!IovCallDriver+0x31
eef61db8 80420964 00000000 00000000 80062f10 SYMEVENT+0x1065
eef61dcc 80440a0d 81424038 871ba8e0 871ba8c0 nt!IoPageRead+0xb1
eef61e0c 8044a0c4 00000000 c2441000 c0309104 nt!MiDispatchFault+0x23d
eef61e58 80467cc6 00000000 00000000 00000000 nt!MmAccessFault+0x67b
eef61e58 80413a2b 00000000 00000000 00000000 nt!KiTrap0E+0xc3
eef61f28 bfeee6b5 81424038 eef61f5c 00001000 nt!CcMapData+0xd9
eef61f4c bfef0573 80f2e768 a83f9e48 00001000 Ntfs!NtfsMapStream+0x4b
eef61f7c bfeefafc 80f2e768 0000000b 00000002 Ntfs!ReadIndexBuffer+0x8b
eef61fa8 bff4b8c3 80f2e768 eef6202c eef6211c
Ntfs!FindFirstIndexEntry+0x1be
eef620b4 bff45124 80f2e768 a83f9e48 eef6211c Ntfs!NtOfsFindRecord+0xbd
eef62124 bff44f7f 80f2e768 814270f0 00000162
Ntfs!MapSecurityIdToSecurityDescriptorHeaderUnsafe+0x32
eef62178 bfefd0c3 80f2e768 814270f0 00000162
Ntfs!NtfsCacheSharedSecurityBySecurityId+0xb6
eef62220 bfefb3aa 80f2e768 00000001 e3d28ca8
Ntfs!NtfsUpdateFcbInfoFromDisk+0x209
eef6232c bfefb091 80f2e768 83b572a8 83b573f0 Ntfs!NtfsOpenFile+0x3f6
eef62684 bfeef855 80f2e768 83b572a8 eef626e4 Ntfs!NtfsCommonCreate+0xf31

kd> !irp 83b572a8
Irp is active with 9 stacks 7 is current (= 0x83b573f0)
No Mdl Thread 810d2020: Irp stack trace.
cmd flg cl Device File Completion-Context
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 10 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000

[0, 0] 0 e0 81427020 859f8a88 80531396-83b57414 Success Error
Cancel
\FileSystem\Ntfs nt!IovpInternalCompletionTrap
Args: eef62bc4 01000050 00010080 00000000
[0, 0] 0 e0 81307020 859f8a88 ef29d1b3-eef629b4 Success Error
Cancel
\Driver\ThTrack SYMEVENT
Args: eef62bc4 01000050 00010080 00000000
[0, 0] 0 0 812701e0 859f8a88 00000000-00000000
\Driver\SymEvent
Args: eef62bc4 01000050 00010080 00000000
kd> !object 859f8a88
Object: 859f8a88 Type: (81490820) File
ObjectHeader: 859f8a70
HandleCount: 0 PointerCount: 1
Directory Object: 00000000 Name:
\PROGRA~1\COMMON~1\SYMANT~1\VIRUSD~1\definfo.dat {HarddiskVolume5}
kd> !irp 841c72a8
Irp is active with 9 stacks 8 is current (= 0x841c7414)
No Mdl Thread 810d2020: Irp stack trace.
cmd flg cl Device File Completion-Context
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 10 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000

[0, 0] 0 e0 81307020 8677a268 ef29d1b3-eef63140 Success Error
Cancel
\Driver\ThTrack SYMEVENT
Args: eef63350 01000960 00070080 00000000
[0, 0] 0 0 812701e0 8677a268 00000000-00000000
\Driver\SymEvent
Args: eef63350 01000960 00070080 00000000
kd> !object 8677a268
Object: 8677a268 Type: (81490820) File
ObjectHeader: 8677a250
HandleCount: 0 PointerCount: 1
Directory Object: 00000000 Name:
\PROGRA~1\COMMON~1\SYMANT~1\VIRUSD~1\definfo.dat {HarddiskVolume5}
kd> !irp a58d7e48
Irp is active with 9 stacks 9 is current (= 0xa58d7fd8)
No Mdl Thread 810d2020: Irp stack trace.
cmd flg cl Device File Completion-Context
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 10 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000

[0, 0] 0 0 812701e0 8227ad08 00000000-00000000
\Driver\SymEvent
Args: eef63a88 01000060 00070000 00000000
kd> !object 8227ad08
Object: 8227ad08 Type: (81490820) File
ObjectHeader: 8227acf0
HandleCount: 0 PointerCount: 1
Directory Object: 00000000 Name:
\PROGRA~1\COMMON~1\SYMANT~1\VIRUSD~1\definfo.dat {HarddiskVolume5}

OSR_Community_User · March 18, 2002, 8:43am

Bartjan,

This looks like a stack overflow (note that the EBP for your last frame is
eef61164 and the stack limit for the thread is eef61000.) I don’t know how
much stack space you are using in this particular call, either.

However, I note that the !thread command lists the current stack at eef61ae4
but (interestingly enough) that’s quite a long way up in the stack trace
shown:

eef61ae4 80530510 81427020 8462e2a8 8462e2a8 Ntfs!NtfsFsdRead+0x201

I thought that a bit odd.

Stack overflow conditions are not uncommon, especially with multiple file
system filter drivers in the stack - and this particular stack looks like a
potential overflow condition.

As for the fact that the same completion routine is in each stack location,
that’s not important in this case - the completion routine is
IovpInternalCompletionTrap, which (by the prefix) is an I/O Manager private
routine that is part of the verifier (hence Iovp = I/O, verifier,
private/internal). All that says is you were running driver verifier.

The debugger documents suggest otherwise, however:

0x00000008, or Double Fault, is when an exception occurs while trying to
call the handler for a prior exception. Normally, the two exceptions can be
handled serially. However, there are several exceptions that cannot be
handled serially, and in this situation the processor signals a double
fault. This is almost always caused by hardware problems

You are in an exception - a page fault - and thus there are certain
conditions that would cause a second fault to trap out (oddly enough, not a
page fault, which would just call KiTrap0E again).

I’m teaching a debug class this week - if you want, I’d be happy to take the
dump in to class as an example!

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Bartjan Wattel [mailto:xxxxx@zeelandnet.nl]
Sent: Monday, March 18, 2002 7:41 AM
To: File Systems Developers
Subject: [ntfsd] Resolving 0x7F bugcheck, exception_double_fault

Hi guys,

During stress-test with our driver loaded together with an anti-virus
driver (Symantec Norton 2002 in this case), we have experienced some
bugchecks after a couple of days, mostly 0x7F
(UNEXPECTED_KERNEL_MODE_TRAP). The debugger tells us that the
declaration and initialization of a parameter on the local stack is the
problem, however I find that hard to believe. Who can explain the
situation for me ?

What I’m curious about, are the following lines from an IRP stack trace
(full details are listed below):

[3, 0] 0 e0 81427020 81424038 80531396-8462e414 Success Error
Cancel
\FileSystem\Ntfs nt!IovpInternalCompletionTrap
Args: 00001000 00000000 00001000 00000000
[3, 0] 0 e0 81307020 81424038 80531396-8462e438 Success Error
Cancel
\Driver\ThTrack nt!IovpInternalCompletionTrap
Args: 00001000 00000000 00001000 00000000

What is the cause of the completion routine appearing in both Ntfs and
my driver (ThTrack) ? Any hints or tips how to track down the cause of
the problem using WinDbg ?

Below you’ll find some more details.

Best,
Bartjan Wattel.

Some details: (ThTrac2k is our driver):

************************************************************************
*******
*
*
* Bugcheck Analysis
*
*
*
************************************************************************
*******

UNEXPECTED_KERNEL_MODE_TRAP (7f)
This means a trap occurred in kernel mode, and it’s a trap of a kind
that the kernel isn’t allowed to have/catch (bound trap) or that
is always instant death (double fault). The first number in the
bugcheck parens is the number of the trap (8 = double fault, etc)
Consult an Intel x86 family manual to learn more about what these
traps are. Here is a *portion* of those codes:
If kv shows a taskGate
use .tss on the part before the colon, then kv.
Else if kv shows a trapframe
use .trap on that value
Else
.trap on the appropriate frame will show where the trap was
taken
(on x86, this will be the ebp that goes with the procedure
KiTrap)
Endif
kb will then show the corrected stack.
Arguments:
Arg1: 00000008, EXCEPTION_DOUBLE_FAULT
Arg2: 00000000
Arg3: 00000000
Arg4: 00000000

Debugging Details:

*** ERROR: Symbol file could not be found. Defaulted to export symbols
for SYMEVENT.SYS -

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 7F

LAST_CONTROL_TRANSFER: from 8042bef7 to 80455994

Looking at the IRP that is passed to my dispatch routine, I get the
following:

kd> !irp 8462e2a8
Irp is active with 9 stacks 7 is current (= 0x8462e3f0)
Mdl = 871ba8e0 Thread 810d2020: Irp stack trace.
cmd flg cl Device File Completion-Context
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 10 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000

[3, 0] 0 e0 81427020 81424038 80531396-8462e414 Success Error
Cancel
\FileSystem\Ntfs nt!IovpInternalCompletionTrap
Args: 00001000 00000000 00001000 00000000
[3, 0] 0 e0 81307020 81424038 80531396-8462e438 Success Error
Cancel
\Driver\ThTrack nt!IovpInternalCompletionTrap
Args: 00001000 00000000 00001000 00000000
[3, 0] 0 0 812701e0 81424038 00000000-00000000
\Driver\SymEvent
Args: 00001000 00000000 00001000 00000000

kd> !thread
THREAD 810d2020 Cid 468.440 Teb: 7ffdb000 Win32Thread: a6c21ec0
RUNNING
IRP List:
83b572a8: (0006,01b4) Flags: 40000884 Mdl: 00000000
841c72a8: (0006,01b4) Flags: 40000884 Mdl: 00000000
a58d7e48: (0006,01b4) Flags: 40000884 Mdl: 00000000
Not impersonating
Owning Process 81124d60
WaitTime (seconds) 19606726
Context Switch Count 111499 LargeStack
UserTime 0:00:00.0090
KernelTime 0:00:13.0719
Start Address KERNEL32!BaseThreadStartThunk (0x77e87532)
Win32 Start Address msvcrt!_beginthreadex (0x7800a361)
Stack Init eef64000 Current eef61ae4 Base eef64000 Limit eef61000 Call 0
Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16

ChildEBP RetAddr Args to Child
804700a4 8042bef7 00000004 ffdff408 804703b8
nt!RtlpBreakWithStatusInstruction
804700d4 8042c438 00000004 00000000 00000000
nt!KiBugCheckDebugBreak+0x31
80470460 804669be 0000007f 00000008 00000000 nt!KeBugCheckEx+0x5d7
80470460 ef6e0817 0000007f 00000008 00000000 nt!KiTrap08+0x3e
eef61164 ef6e1028 81307020 819872a8 80063124
ThTrac2k!ThTrackHookRoutine+0x11
eef6117c 80530510 eef63df0 819872a8 819872a8
ThTrac2k!ThTrackDispatch+0x4f
eef611c8 8052fcd5 aee47fa0 819872a8 81308450
nt!IovSpecialIrpCallDriver+0xcd
eef611e4 ef29d065 812701e0 80063124 80530510 nt!IovCallDriver+0x31
eef6123c 80420964 00000000 00000000 80062f10 SYMEVENT+0x1065
eef61250 80440a0d 81422668 87b5c8a0 87b5c880 nt!IoPageRead+0xb1
eef61290 8044a0c4 00000001 a83fdf1c c02a0ff4 nt!MiDispatchFault+0x23d
eef612dc 80467cc6 00000001 00000000 00000000 nt!MmAccessFault+0x67b
eef612dc bfed6d71 00000001 00000000 00000000 nt!KiTrap0E+0xc3
eef61380 bfed789d a83f9ee8 00000002 00000000
Ntfs!NtfsLookupNtfsMcbEntry+0x16c
eef6148c bfed7f8b 8121abc8 a83f9e48 00000002
Ntfs!NtfsLookupAllocation+0x58
eef6165c bfed6e87 8121abc8 8462e2a8 a83f9e48
Ntfs!NtfsPrepareBuffers+0x25e
eef61830 bfed744d 8121abc8 8462e2a8 a83f9e48 Ntfs!NtfsNonCachedIo+0x121
eef61a48 bfed8a3b 8121abc8 8462e2a8 00000001 Ntfs!NtfsCommonRead+0xeee
eef61ae4 80530510 81427020 8462e2a8 8462e2a8 Ntfs!NtfsFsdRead+0x201
eef61b30 8052fcd5 8462e40c 8462e430 8462e2a8
nt!IovSpecialIrpCallDriver+0xcd
eef61b4c ef6e0e2a 81307020 81307020 812f9dd0 nt!IovCallDriver+0x31
eef61ce0 ef6e1028 81307020 8462e2a8 80063124
ThTrac2k!ThTrackHookRoutine+0x624
eef61cf8 80530510 eef63df0 8462e2a8 8462e2a8
ThTrac2k!ThTrackDispatch+0x4f
eef61d44 8052fcd5 aee47fa0 8462e2a8 81308450
nt!IovSpecialIrpCallDriver+0xcd
eef61d60 ef29d065 812701e0 80063124 80530510 nt!IovCallDriver+0x31
eef61db8 80420964 00000000 00000000 80062f10 SYMEVENT+0x1065
eef61dcc 80440a0d 81424038 871ba8e0 871ba8c0 nt!IoPageRead+0xb1
eef61e0c 8044a0c4 00000000 c2441000 c0309104 nt!MiDispatchFault+0x23d
eef61e58 80467cc6 00000000 00000000 00000000 nt!MmAccessFault+0x67b
eef61e58 80413a2b 00000000 00000000 00000000 nt!KiTrap0E+0xc3
eef61f28 bfeee6b5 81424038 eef61f5c 00001000 nt!CcMapData+0xd9
eef61f4c bfef0573 80f2e768 a83f9e48 00001000 Ntfs!NtfsMapStream+0x4b
eef61f7c bfeefafc 80f2e768 0000000b 00000002 Ntfs!ReadIndexBuffer+0x8b
eef61fa8 bff4b8c3 80f2e768 eef6202c eef6211c
Ntfs!FindFirstIndexEntry+0x1be
eef620b4 bff45124 80f2e768 a83f9e48 eef6211c Ntfs!NtOfsFindRecord+0xbd
eef62124 bff44f7f 80f2e768 814270f0 00000162
Ntfs!MapSecurityIdToSecurityDescriptorHeaderUnsafe+0x32
eef62178 bfefd0c3 80f2e768 814270f0 00000162
Ntfs!NtfsCacheSharedSecurityBySecurityId+0xb6
eef62220 bfefb3aa 80f2e768 00000001 e3d28ca8
Ntfs!NtfsUpdateFcbInfoFromDisk+0x209
eef6232c bfefb091 80f2e768 83b572a8 83b573f0 Ntfs!NtfsOpenFile+0x3f6
eef62684 bfeef855 80f2e768 83b572a8 eef626e4 Ntfs!NtfsCommonCreate+0xf31

kd> !irp 83b572a8
Irp is active with 9 stacks 7 is current (= 0x83b573f0)
No Mdl Thread 810d2020: Irp stack trace.
cmd flg cl Device File Completion-Context
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 10 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000

[0, 0] 0 e0 81427020 859f8a88 80531396-83b57414 Success Error
Cancel
\FileSystem\Ntfs nt!IovpInternalCompletionTrap
Args: eef62bc4 01000050 00010080 00000000
[0, 0] 0 e0 81307020 859f8a88 ef29d1b3-eef629b4 Success Error
Cancel
\Driver\ThTrack SYMEVENT
Args: eef62bc4 01000050 00010080 00000000
[0, 0] 0 0 812701e0 859f8a88 00000000-00000000
\Driver\SymEvent
Args: eef62bc4 01000050 00010080 00000000
kd> !object 859f8a88
Object: 859f8a88 Type: (81490820) File
ObjectHeader: 859f8a70
HandleCount: 0 PointerCount: 1
Directory Object: 00000000 Name:
\PROGRA~1\COMMON~1\SYMANT~1\VIRUSD~1\definfo.dat {HarddiskVolume5}
kd> !irp 841c72a8
Irp is active with 9 stacks 8 is current (= 0x841c7414)
No Mdl Thread 810d2020: Irp stack trace.
cmd flg cl Device File Completion-Context
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 10 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000

[0, 0] 0 e0 81307020 8677a268 ef29d1b3-eef63140 Success Error
Cancel
\Driver\ThTrack SYMEVENT
Args: eef63350 01000960 00070080 00000000
[0, 0] 0 0 812701e0 8677a268 00000000-00000000
\Driver\SymEvent
Args: eef63350 01000960 00070080 00000000
kd> !object 8677a268
Object: 8677a268 Type: (81490820) File
ObjectHeader: 8677a250
HandleCount: 0 PointerCount: 1
Directory Object: 00000000 Name:
\PROGRA~1\COMMON~1\SYMANT~1\VIRUSD~1\definfo.dat {HarddiskVolume5}
kd> !irp a58d7e48
Irp is active with 9 stacks 9 is current (= 0xa58d7fd8)
No Mdl Thread 810d2020: Irp stack trace.
cmd flg cl Device File Completion-Context
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 0 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000
[0, 0] 0 10 00000000 00000000 00000000-00000000

Args: 00000000 00000000 00000000 00000000

[0, 0] 0 0 812701e0 8227ad08 00000000-00000000
\Driver\SymEvent
Args: eef63a88 01000060 00070000 00000000
kd> !object 8227ad08
Object: 8227ad08 Type: (81490820) File
ObjectHeader: 8227acf0
HandleCount: 0 PointerCount: 1
Directory Object: 00000000 Name:
\PROGRA~1\COMMON~1\SYMANT~1\VIRUSD~1\definfo.dat {HarddiskVolume5}

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%

OSR_Community_User · March 18, 2002, 9:00am

Tony,

Thanks for the quick answer. A stack overflow might be a good guess
because the debugger indicates a “push ebx” instruction as the point of
failure (which at first I found a bit odd, but now it makes sence). Now,
what is *the* solution to cope with this once and for all ? I can try to
tidy up my code with respect to stack variables, but I don’t believe
this is a 100% good solution. Would fiddling around with
IoGetRemainingStackSize be a “clean” solution ?

I know there was some discussion on the list a few weeks ago about stack
consumption of Symantec’s SymEvent driver. Does anybody of the list has
any experience with interoperability with Symantec’s SysEvent.Sys (NAV
2002) ?

As for the driver debugging example, I need to discuss this internally.
I’ll be in touch.

–
Bartjan.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: maandag 18 maart 2002 14:40
To: File Systems Developers
Subject: [ntfsd] RE: Resolving 0x7F bugcheck, exception_double_fault

Bartjan,

This looks like a stack overflow (note that the EBP for your
last frame is eef61164 and the stack limit for the thread is
eef61000.) I don’t know how much stack space you are using
in this particular call, either.

However, I note that the !thread command lists the current
stack at eef61ae4 but (interestingly enough) that’s quite a
long way up in the stack trace
shown:

eef61ae4 80530510 81427020 8462e2a8 8462e2a8 Ntfs!NtfsFsdRead+0x201

I thought that a bit odd.

Stack overflow conditions are not uncommon, especially with
multiple file system filter drivers in the stack - and this
particular stack looks like a potential overflow condition.

As for the fact that the same completion routine is in each
stack location, that’s not important in this case - the
completion routine is IovpInternalCompletionTrap, which (by
the prefix) is an I/O Manager private routine that is part of
the verifier (hence Iovp = I/O, verifier, private/internal).
All that says is you were running driver verifier.

The debugger documents suggest otherwise, however:

0x00000008, or Double Fault, is when an exception occurs
while trying to call the handler for a prior exception.
Normally, the two exceptions can be handled serially.
However, there are several exceptions that cannot be handled
serially, and in this situation the processor signals a
double fault. This is almost always caused by hardware problems

You are in an exception - a page fault - and thus there are
certain conditions that would cause a second fault to trap
out (oddly enough, not a page fault, which would just call
KiTrap0E again).

I’m teaching a debug class this week - if you want, I’d be
happy to take the dump in to class as an example!

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

OSR_Community_User · March 18, 2002, 9:51am

Barjan,

First, to confirm this diagnosis, you need to do:

kd> kv

and then, using the last trap frame on the stack (the one for KiTrap08):

kd> .trap

That will show you what the ESP was at the time of the trap. If it was
caused by a stack overflow, you are going to see the ESP right on the edge
of the frame.

As for resolving stack overflow conditions, there are no “perfect”
solutions. At the recent Plugfest they gave everyone a t-shirt which was -
surprise - a stack overflow condition! The file systems folks have since
“worked around” this particular condition, but the technique they use
(posting to the FsRtl stack overflow thread) isn’t very good for your
situation (because you aren’t the file system, it isn’t safe for you to do
that!)

The techniques you could try:

- Minimizing stack usage. This is ugly for other obvious reasons.
- Test for stack overflow conditions and, if present, attempt to handle it.
This could mean failing the operation (although not a good idea when paging
in FS data structure), or posting to your own driver’s stack overflow thread
(this can cause locking/deadlock issues and problems).

Others who have dealt with this situation might have other techniques to
suggest.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Bartjan Wattel [mailto:xxxxx@zeelandnet.nl]
Sent: Monday, March 18, 2002 8:57 AM
To: File Systems Developers
Subject: [ntfsd] RE: Resolving 0x7F bugcheck, exception_double_fault

Tony,

Thanks for the quick answer. A stack overflow might be a good guess
because the debugger indicates a “push ebx” instruction as the point of
failure (which at first I found a bit odd, but now it makes sence). Now,
what is the solution to cope with this once and for all ? I can try to
tidy up my code with respect to stack variables, but I don’t believe
this is a 100% good solution. Would fiddling around with
IoGetRemainingStackSize be a “clean” solution ?

I know there was some discussion on the list a few weeks ago about stack
consumption of Symantec’s SymEvent driver. Does anybody of the list has
any experience with interoperability with Symantec’s SysEvent.Sys (NAV
2002) ?

As for the driver debugging example, I need to discuss this internally.
I’ll be in touch.

–
Bartjan.

OSR_Community_User · March 18, 2002, 12:20pm

Thanks Tony.

BTW: I know I would miss something because I didn’t appear at the
february Plugfest… But I didn’t know I would miss the t-shirt…

I’ll stick to debugging and hope to solve the problem.

Bartjan.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: maandag 18 maart 2002 15:48
To: File Systems Developers
Subject: [ntfsd] RE: Resolving 0x7F bugcheck, exception_double_fault

Barjan,

First, to confirm this diagnosis, you need to do:

kd> kv

and then, using the last trap frame on the stack (the one for
KiTrap08):

kd> .trap
>
> That will show you what the ESP was at the time of the trap.
> If it was caused by a stack overflow, you are going to see
> the ESP right on the edge of the frame.
>
> As for resolving stack overflow conditions, there are no
> “perfect” solutions. At the recent Plugfest they gave
> everyone a t-shirt which was - surprise - a stack overflow
> condition! The file systems folks have since “worked around”
> this particular condition, but the technique they use
> (posting to the FsRtl stack overflow thread) isn’t very good
> for your situation (because you aren’t the file system, it
> isn’t safe for you to do
> that!)
>
> The techniques you could try:
>
> - Minimizing stack usage. This is ugly for other obvious reasons.
> - Test for stack overflow conditions and, if present, attempt
> to handle it. This could mean failing the operation (although
> not a good idea when paging in FS data structure), or posting
> to your own driver’s stack overflow thread (this can cause
> locking/deadlock issues and problems).
>
> Others who have dealt with this situation might have other
> techniques to suggest.
>
> Regards,
>
> Tony
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com
>
>
> -----Original Message-----
> From: Bartjan Wattel [mailto:xxxxx@zeelandnet.nl]
> Sent: Monday, March 18, 2002 8:57 AM
> To: File Systems Developers
> Subject: [ntfsd] RE: Resolving 0x7F bugcheck, exception_double_fault
>
> Tony,
>
> Thanks for the quick answer. A stack overflow might be a good
> guess because the debugger indicates a “push ebx” instruction
> as the point of failure (which at first I found a bit odd,
> but now it makes sence). Now, what is the solution to cope
> with this once and for all ? I can try to tidy up my code
> with respect to stack variables, but I don’t believe this is
> a 100% good solution. Would fiddling around with
> IoGetRemainingStackSize be a “clean” solution ?
>
> I know there was some discussion on the list a few weeks ago
> about stack consumption of Symantec’s SymEvent driver. Does
> anybody of the list has any experience with interoperability
> with Symantec’s SysEvent.Sys (NAV
> 2002) ?
>
> As for the driver debugging example, I need to discuss this
> internally. I’ll be in touch.
>
> –
> Bartjan.
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@zeelandnet.nl
> To unsubscribe send a blank email to %%email.unsub%%
>

OSR_Community_User · March 21, 2002, 4:15pm

Bartjan,

There is no 100% solution for solving stack overflow problems. The best
answer for now is what Tony suggested, use as little stack as possible
when calling down to lower level filters (you can fix this by doing your
real work in subroutines) and possibly queuing to a worker thread when
most of the stack is overflowed

It is important to point out that using the verifier exacerbates this
problem because is also consumes stack. We have seen situations where
you overflow with the verifier but work fine without it.

Neal Christiansen

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Bartjan Wattel [mailto:xxxxx@zeelandnet.nl]
Sent: Monday, March 18, 2002 05:57 AM
To: File Systems Developers
Subject: [ntfsd] RE: Resolving 0x7F bugcheck, exception_double_fault

Tony,

Thanks for the quick answer. A stack overflow might be a good guess
because the debugger indicates a “push ebx” instruction as the point of
failure (which at first I found a bit odd, but now it makes sence). Now,
what is *the* solution to cope with this once and for all ? I can try to
tidy up my code with respect to stack variables, but I don’t believe
this is a 100% good solution. Would fiddling around with
IoGetRemainingStackSize be a “clean” solution ?

I know there was some discussion on the list a few weeks ago about stack
consumption of Symantec’s SymEvent driver. Does anybody of the list has
any experience with interoperability with Symantec’s SysEvent.Sys (NAV
2002) ?

As for the driver debugging example, I need to discuss this internally.
I’ll be in touch.

–
Bartjan.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: maandag 18 maart 2002 14:40
To: File Systems Developers
Subject: [ntfsd] RE: Resolving 0x7F bugcheck, exception_double_fault

Bartjan,

This looks like a stack overflow (note that the EBP for your
last frame is eef61164 and the stack limit for the thread is
eef61000.) I don’t know how much stack space you are using
in this particular call, either.

However, I note that the !thread command lists the current
stack at eef61ae4 but (interestingly enough) that’s quite a
long way up in the stack trace
shown:

eef61ae4 80530510 81427020 8462e2a8 8462e2a8 Ntfs!NtfsFsdRead+0x201

I thought that a bit odd.

Stack overflow conditions are not uncommon, especially with
multiple file system filter drivers in the stack - and this
particular stack looks like a potential overflow condition.

As for the fact that the same completion routine is in each
stack location, that’s not important in this case - the
completion routine is IovpInternalCompletionTrap, which (by
the prefix) is an I/O Manager private routine that is part of
the verifier (hence Iovp = I/O, verifier, private/internal).
All that says is you were running driver verifier.

The debugger documents suggest otherwise, however:

0x00000008, or Double Fault, is when an exception occurs
while trying to call the handler for a prior exception.
Normally, the two exceptions can be handled serially.
However, there are several exceptions that cannot be handled
serially, and in this situation the processor signals a
double fault. This is almost always caused by hardware problems

You are in an exception - a page fault - and thus there are
certain conditions that would cause a second fault to trap
out (oddly enough, not a page fault, which would just call
KiTrap0E again).

I’m teaching a debug class this week - if you want, I’d be
happy to take the dump in to class as an example!

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%