Driver Verifier Fault injections causing BSOD

NTFSD
1

Hi friends, I’m working on a file system filter driver which picks up the file path on user activity and gives it to a third party driver.I had been testing my filter driver with driver verifier kept on and it works perfectly without any problem. But the problem comes when i integrate the third party driver with my driver and the verifier kept on. Everything goes file untill the fault injections starts hitting the driver hard and the third party driver crashes with UNEXPECTED_KERNEL_MODE_TRAP (7f)

Arguments:
Arg1: 00000008, EXCEPTION_DOUBLE_FAULT
Arg2: 00000000
Arg3: 00000000
Arg4: 00000000

As the third party driver is not mine ,i dont have the source code to debug it. So i want to
detect the fault injections before hand and switch off the third party driver before it crashes. Is there any event to detect fault injection condition? what exactly happens when the verifier hits with fault injections ? Thanks for your time.

2

What is missing here is a stack trace. This bugcheck usually means you have
run out of stack space. Routines with a lot of local data declared or
recursive functions are common causes.

//Daniel

wrote in message news:xxxxx@ntfsd…
> Hi friends, I’m working on a file system filter driver which picks up the
> file path on user activity and gives it to a third party driver.I had been
> testing my filter driver with driver verifier kept on and it works
> perfectly without any problem. But the problem comes when i integrate the
> third party driver with my driver and the verifier kept on. Everything
> goes file untill the fault injections starts hitting the driver hard and
> the third party driver crashes with UNEXPECTED_KERNEL_MODE_TRAP (7f)
>
> Arguments:
> Arg1: 00000008, EXCEPTION_DOUBLE_FAULT
> Arg2: 00000000
> Arg3: 00000000
> Arg4: 00000000
>
> As the third party driver is not mine ,i dont have the source code to
> debug it. So i want to
> detect the fault injections before hand and switch off the third party
> driver before it crashes. Is there any event to detect fault injection
> condition? what exactly happens when the verifier hits with fault
> injections ? Thanks for your time.
>
>

3

Yes daniel the third party driver uses recursive functions.I tried using IoGetRemainingStackSize() to check stack size before passing the file to the thrid party driver, but that didn’t help. Is there any way to determine how much exactly that third party driver uses from the stack? so that i can check the limit before passing the file into that driver… Here is the stack trace

0: kd> kb
ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be wrong.
f74ff254 804fe507 0000008e c0000005 f7358ae6 nt!KeBugCheckEx+0x1b
f74ff61c 80541075 f74ff638 00000000 f74ff68c nt!KeRaiseUserException+0xc29
f74ff6ac f734770e ffa00a07 f74ff6ec 00000000 nt!Kei386EoiHelper+0x1d9
f74ff708 f7356f45 ffa00190 ffbde7c8 ffa00a7c VBENGNT!EDK_ReceiveMsg+0x9328e
f74ff730 f736119e ffa00a7c 00000000 81094044 VBENGNT!EDK_ReceiveMsg+0xa2ac5
f74ff774 f735db3f ff875010 81094044 00000000 VBENGNT!EDK_ReceiveMsg+0xacd1e
00000000 00000000 00000000 00000000 00000000 VBENGNT!EDK_ReceiveMsg+0xa96bf

4

Also this is another BSOD i get when faults injected.

KERNEL_MODE_EXCEPTION_NOT_HANDLED (8e)
This is a very common bugcheck. Usually the exception address pinpoints
the driver/function that caused the problem. Always note this address
as well as the link date of the driver/image that contains this address.
Some common problems are exception code 0x80000003. This means a hard
coded breakpoint or assertion was hit, but this system was booted
/NODEBUG. This is not supposed to happen as developers should never have
hardcoded breakpoints in retail code, but ...
If this happens, make sure a debugger gets connected, and the
system is booted /DEBUG. This will let us see why this breakpoint is
happening.
Arguments:
Arg1: c0000005, The exception code that was not handled
Arg2: f72e0ae6, The address that the exception occurred at
Arg3: f750768c, Trap Frame
Arg4: 00000000

Debugging Details:

***** Kernel symbols are WRONG. Please fix symbols to do analysis.

*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: nt!_KPRCB ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: nt!_KPRCB ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: kernel32!pNlsUserInfo ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: kernel32!pNlsUserInfo ***
*** ***
*************************************************************************

MODULE_NAME: VBENGNT

FAULTING_MODULE: 804d7000 nt

DEBUG_FLR_IMAGE_TIMESTAMP: 4846b66f

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at "0x%08lx" referenced memory at "0x%08lx". The memory could not be "%s".

FAULTING_IP:
VBENGNT!EDK_ReceiveMsg+a4666
f72e0ae6 803800 cmp byte ptr [eax],0

TRAP_FRAME: f750768c -- (.trap 0xfffffffff750768c)
ErrCode = 00000000
eax=6e656700 ebx=00000000 ecx=80070012 edx=ff9e0a7c esi=80fdcc38 edi=ff9e0190
eip=f72e0ae6 esp=f7507700 ebp=ff9e0190 iopl=0 nv up ei pl nz na pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010206
VBENGNT!EDK_ReceiveMsg+0xa4666:
f72e0ae6 803800 cmp byte ptr [eax],0 ds:0023:6e656700=??
Resetting default scope

DEFAULT_BUCKET_ID: WRONG_SYMBOLS

BUGCHECK_STR: 0x8E

LAST_CONTROL_TRANSFER: from 804fe507 to 804f9c37

STACK_TEXT:
WARNING: Stack unwind information not available. Following frames may be wrong.
f7507254 804fe507 0000008e c0000005 f72e0ae6 nt!KeBugCheckEx+0x1b
f750761c 80541075 f7507638 00000000 f750768c nt!KeRaiseUserException+0xc29
f75076ac f72cf70e ff9e0a07 f75076ec 00000000 nt!Kei386EoiHelper+0x1d9
f7507708 f72def45 ff9e0190 80fdcc38 ff9e0a7c VBENGNT!EDK_ReceiveMsg+0x9328e
f7507730 f72e919e ff9e0a7c 00000000 81075044 VBENGNT!EDK_ReceiveMsg+0xa2ac5
f7507774 f72e5b3f ff855010 81075044 00000000 VBENGNT!EDK_ReceiveMsg+0xacd1e
00000000 00000000 00000000 00000000 00000000 VBENGNT!EDK_ReceiveMsg+0xa96bf

STACK_COMMAND: kb

FOLLOWUP_IP:
VBENGNT!EDK_ReceiveMsg+a4666
f72e0ae6 803800 cmp byte ptr [eax],0

SYMBOL_STACK_INDEX: 0

SYMBOL_NAME: VBENGNT!EDK_ReceiveMsg+a4666

FOLLOWUP_NAME: MachineOwner

IMAGE_NAME: VBENGNT.SYS

BUCKET_ID: WRONG_SYMBOLS

Followup: MachineOwner

5

wrote in message news:xxxxx@ntfsd…
> Yes daniel the third party driver uses recursive functions.I tried using
> IoGetRemainingStackSize() to check stack size before passing the file to
> the thrid party driver, but that didn’t help. Is there any way to
> determine how much exactly that third party driver uses from the stack? so
> that i can check the limit before passing the file into that driver…
> Here is the stack trace
>

If the amount of stack space used by the driver is variable (and not fixed),
for example because it does a recursive directory search then I think there
is not much you can do as you don’t know how much stack is going to be
consumed beforehand. You should ask him to fix his driver, he is likely to
run in trouble with more people than you.

//Daniel

6

>perfectly without any problem. But the problem comes when i integrate the third party driver with my

driver and the verifier kept on. Everything goes file untill the fault injections starts hitting the driver hard
and the third party driver crashes with UNEXPECTED_KERNEL_MODE_TRAP (7f)

Kernel stack overflow.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

7

> ***** Kernel symbols are WRONG. Please fix symbols to do analysis.

Please fix your kernel symbols issue.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

8

Since it’s third party driver , it’s not possible for me to get symbols. The stack flow is occuring in the third party driver , it’s not in my control to fix the stack overflow. I was just curious to know if there was any way to know the stack usage by that driver,but now it seems it’s not possible. I’ll better report this to the driver developer.

9

Try “knf”, or do the math from the frame pointer differences yourself.

? S

-----Original Message-----
From: xxxxx@gmail.com
Sent: Sunday, January 11, 2009 02:43
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] Driver Verifier Fault injections causing BSOD

Yes daniel the third party driver uses recursive functions.I tried using IoGetRemainingStackSize() to check stack size before passing the file to the thrid party driver, but that didn’t help. Is there any way to determine how much exactly that third party driver uses from the stack? so that i can check the limit before passing the file into that driver… Here is the stack trace

0: kd> kb
ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be wrong.
f74ff254 804fe507 0000008e c0000005 f7358ae6 nt!KeBugCheckEx+0x1b
f74ff61c 80541075 f74ff638 00000000 f74ff68c nt!KeRaiseUserException+0xc29
f74ff6ac f734770e ffa00a07 f74ff6ec 00000000 nt!Kei386EoiHelper+0x1d9
f74ff708 f7356f45 ffa00190 ffbde7c8 ffa00a7c VBENGNT!EDK_ReceiveMsg+0x9328e
f74ff730 f736119e ffa00a7c 00000000 81094044 VBENGNT!EDK_ReceiveMsg+0xa2ac5
f74ff774 f735db3f ff875010 81094044 00000000 VBENGNT!EDK_ReceiveMsg+0xacd1e
00000000 00000000 00000000 00000000 00000000 VBENGNT!EDK_ReceiveMsg+0xa96bf

—
NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@valhallalegends.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

10

Thanks Ken. Could you please give an example of calculating frame pointer differences from the above stack trace ? Thanks for your time.

11

On 1/12/09, xxxxx@gmail.com wrote:
>
> Thanks Ken. Could you please give an example of calculating frame pointer
> differences from the above stack trace ? Thanks for your time.

0:000> knf
# Memory ChildEBP RetAddr
00 0006fc94 7c921639 ntdll!LdrpInitializeProcess+0xfff
01 88 0006fd1c 7c90eac7 ntdll!_LdrpInitialize+0x183
02 00000000 00000000 ntdll!KiUserApcDispatcher+0x7
0:000> ? 6fd1c - 6fc94
Evaluate expression: 136 = 00000088
0:000> k
ChildEBP RetAddr
0006fc94 7c921639 ntdll!LdrpInitializeProcess+0xfff
0006fd1c 7c90eac7 ntdll!_LdrpInitialize+0x183
00000000 00000000 ntdll!KiUserApcDispatcher+0x7

just subtract the differences from childebp’s on stack trace

some times this can be a problem with frame pointer omission whatever things

regards

raj_r

12

Thanks raj,that was very much helpfull.