ExRaiseHardError & ZwAllocateVirtualMemory...

Hi,

One of my test machines crashed with bug check 0x0a, in ExRaiseHardError,
while handling a delayed write failure.
The file on which delayed write failed (the file name in the passed to
ExRaiseHardError) is a file on my FS driver. while delayed write failure is
one of my problems, I’m interested in finding out why the system crashed
while reporting this error. From the stack trace and parameters, it doesn’t
look like it was trying to access anything related to the file object or any
of the associated structure.

From the dump, it looks like ExRaiseHardError called ZwAllocateVirtualMemory
with a NULL BaseAddress value (not the pointer) and ZwAllocateVirtualMemory
returned STATUS_SUCCESS, with BaseAddress value as 0x550000. When
ExRaiseHardError accessed this, system bug checked.

Has anyone else seen similar crash? Any idea why ZwAllocateVirtualMemory
would return an invalid address? Are there any extension commands that can
provide more details about the allocations made by ZwAllocateVirtualMemory
and related functions (like !pool etc.)?

0: kd> !analyze -v
*******************************************************************************
*
*
* Bugcheck Analysis
*
*
*
*******************************************************************************

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 00550000, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000001, value 0 = read operation, 1 = write operation
Arg4: 809bbeee, address which referenced memory

Debugging Details:

WRITE_ADDRESS: 00550000

CURRENT_IRQL: 0

FAULTING_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx*4],edx

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8098b193 to 809bbeee

TRAP_FRAME: f7906c6c – (.trap fffffffff7906c6c)
ErrCode = 00000002
eax=00000000 ebx=00000001 ecx=00000000 edx=00550014 esi=00550000
edi=0055003c
eip=809bbeee esp=f7906ce0 ebp=f7906d4c iopl=0 nv up di pl nz na po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010006
nt!ExRaiseHardError+0xc5:
809bbeee 89148e mov [esi+ecx*4],edx
ds:0023:00550000=???
Resetting default scope

STACK_TEXT:
f7906d4c 8098b193 c0000222 00000001 00000001 nt!ExRaiseHardError+0xc5
f7906d80 8083f72e 00000000 00000000 8a78a8d0 nt!IopHardErrorThread+0x53
f7906dac 8092ccff 00000000 00000000 00000000 nt!ExpWorkerThread+0xeb
f7906ddc 80841a96 8083f671 00000001 00000000 nt!PspSystemThreadStartup+0x2e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

FOLLOWUP_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx*4],edx

Thanks,

  • Hrishikesh.

The address is probably a legitimate address (did you try “!pte” on that
address?) I suspect you’ll find it is a demand zero pte, which means a
new page must be allocated on first write (note this is a write
operation) but returns zero on read.

However, because you are at elevated IRQL (irql = 0xff is the special
way they report “interrupts disabled”) the page fault cannot be
processed and the system crashes to advise you of this error/issue.
Why the IRQL is elevated is not clear to me from the information that
you provided.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 2:13 PM
To: ntfsd redirect
Subject: [ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Hi,

One of my test machines crashed with bug check 0x0a, in
ExRaiseHardError,
while handling a delayed write failure.
The file on which delayed write failed (the file name in the passed to
ExRaiseHardError) is a file on my FS driver. while delayed write failure
is
one of my problems, I’m interested in finding out why the system crashed

while reporting this error. From the stack trace and parameters, it
doesn’t
look like it was trying to access anything related to the file object or
any
of the associated structure.

From the dump, it looks like ExRaiseHardError called
ZwAllocateVirtualMemory
with a NULL BaseAddress value (not the pointer) and
ZwAllocateVirtualMemory
returned STATUS_SUCCESS, with BaseAddress value as 0x550000. When
ExRaiseHardError accessed this, system bug checked.

Has anyone else seen similar crash? Any idea why ZwAllocateVirtualMemory

would return an invalid address? Are there any extension commands that
can
provide more details about the allocations made by
ZwAllocateVirtualMemory
and related functions (like !pool etc.)?

0: kd> !analyze -v
************************************************************************
*******
*
*
* Bugcheck Analysis
*
*
*
************************************************************************
*******

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address
at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 00550000, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000001, value 0 = read operation, 1 = write operation
Arg4: 809bbeee, address which referenced memory

Debugging Details:

WRITE_ADDRESS: 00550000

CURRENT_IRQL: 0

FAULTING_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx*4],edx

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8098b193 to 809bbeee

TRAP_FRAME: f7906c6c – (.trap fffffffff7906c6c)
ErrCode = 00000002
eax=00000000 ebx=00000001 ecx=00000000 edx=00550014 esi=00550000
edi=0055003c
eip=809bbeee esp=f7906ce0 ebp=f7906d4c iopl=0 nv up di pl nz na
po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010006
nt!ExRaiseHardError+0xc5:
809bbeee 89148e mov [esi+ecx*4],edx
ds:0023:00550000=???
Resetting default scope

STACK_TEXT:
f7906d4c 8098b193 c0000222 00000001 00000001 nt!ExRaiseHardError+0xc5
f7906d80 8083f72e 00000000 00000000 8a78a8d0 nt!IopHardErrorThread+0x53
f7906dac 8092ccff 00000000 00000000 00000000 nt!ExpWorkerThread+0xeb
f7906ddc 80841a96 8083f671 00000001 00000000
nt!PspSystemThreadStartup+0x2e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

FOLLOWUP_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx*4],edx

Thanks,

  • Hrishikesh.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Thanks Tony.

Running !irql reports IRQL 0, so I guess 0xff was set when it entered the
bug check code.

0: kd> !irql
Debugger saved IRQL for processor 0x0 – 0 (LOW_LEVEL)

Running !pte on this address gives

0: kd> !pte 0x550000
VA 00550000
PDE at C0300004 PTE at C0001540
contains 00000000

Can you please tell me how can I check if it’s a demand zero PTE? Does the
“contains 00000000” indicate demand zero PTE? 0xc0001540 is not a valid
address in the dump.

Thanks,

  • Hrishikesh.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
The address is probably a legitimate address (did you try “!pte” on that
address?) I suspect you’ll find it is a demand zero pte, which means a
new page must be allocated on first write (note this is a write
operation) but returns zero on read.

However, because you are at elevated IRQL (irql = 0xff is the special
way they report “interrupts disabled”) the page fault cannot be
processed and the system crashes to advise you of this error/issue.
Why the IRQL is elevated is not clear to me from the information that
you provided.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 2:13 PM
To: ntfsd redirect
Subject: [ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Hi,

One of my test machines crashed with bug check 0x0a, in
ExRaiseHardError,
while handling a delayed write failure.
The file on which delayed write failed (the file name in the passed to
ExRaiseHardError) is a file on my FS driver. while delayed write failure
is
one of my problems, I’m interested in finding out why the system crashed

while reporting this error. From the stack trace and parameters, it
doesn’t
look like it was trying to access anything related to the file object or
any
of the associated structure.

From the dump, it looks like ExRaiseHardError called
ZwAllocateVirtualMemory
with a NULL BaseAddress value (not the pointer) and
ZwAllocateVirtualMemory
returned STATUS_SUCCESS, with BaseAddress value as 0x550000. When
ExRaiseHardError accessed this, system bug checked.

Has anyone else seen similar crash? Any idea why ZwAllocateVirtualMemory

would return an invalid address? Are there any extension commands that
can
provide more details about the allocations made by
ZwAllocateVirtualMemory
and related functions (like !pool etc.)?

0: kd> !analyze -v
***********************************************************



Bugcheck Analysis



**************************************************************


IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address
at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 00550000, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000001, value 0 = read operation, 1 = write operation
Arg4: 809bbeee, address which referenced memory

Debugging Details:
------------------

WRITE_ADDRESS: 00550000

CURRENT_IRQL: 0

FAULTING_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx
4],edx

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8098b193 to 809bbeee

TRAP_FRAME: f7906c6c – (.trap fffffffff7906c6c)
ErrCode = 00000002
eax=00000000 ebx=00000001 ecx=00000000 edx=00550014 esi=00550000
edi=0055003c
eip=809bbeee esp=f7906ce0 ebp=f7906d4c iopl=0 nv up di pl nz na
po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010006
nt!ExRaiseHardError+0xc5:
809bbeee 89148e mov [esi+ecx
4],edx
ds:0023:00550000=???
Resetting default scope

STACK_TEXT:
f7906d4c 8098b193 c0000222 00000001 00000001 nt!ExRaiseHardError+0xc5
f7906d80 8083f72e 00000000 00000000 8a78a8d0 nt!IopHardErrorThread+0x53
f7906dac 8092ccff 00000000 00000000 00000000 nt!ExpWorkerThread+0xeb
f7906ddc 80841a96 8083f671 00000001 00000000
nt!PspSystemThreadStartup+0x2e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

FOLLOWUP_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx
4],edx

Thanks,
- Hrishikesh.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

KeBugCheckEx explicitly looks to see if interrupts are disabled and
“reports” the IRQL as 0xff (an invalid IRQL value). That’s independent
of the last call to KeRaiseIrql.

The PDE itself isn’t valid - there’s nothing within that 4MB region of
memory, so the PTE has no meaning.

Alas, from the information you’ve dug out, it isn’t clear to me what is
happening. I’ve relied upon your comment that ESI is the value returned
from ZwAllocateVirtualMemory, although I do find it odd that it would be
called in system process context (it should work, but it is odd) unless
the memory allocated was in a *different* process context than the
current one (and finding the PDE/PTE in that case is considerably more
work.)

With a crash dump (or live crash) there would be lots of things to
examine - the disassembly, the context of the operation, the original
thread/process that generated the hard error, etc. But WinDBG over SMTP
isn’t terribly efficient, either.

Perhaps someone else will have some further insight to share.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 3:35 PM
To: ntfsd redirect
Subject: Re:[ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Thanks Tony.

Running !irql reports IRQL 0, so I guess 0xff was set when it entered
the
bug check code.

0: kd> !irql
Debugger saved IRQL for processor 0x0 – 0 (LOW_LEVEL)

Running !pte on this address gives

0: kd> !pte 0x550000
VA 00550000
PDE at C0300004 PTE at C0001540
contains 00000000

Can you please tell me how can I check if it’s a demand zero PTE? Does
the
“contains 00000000” indicate demand zero PTE? 0xc0001540 is not a valid
address in the dump.

Thanks,

  • Hrishikesh.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
The address is probably a legitimate address (did you try “!pte” on that
address?) I suspect you’ll find it is a demand zero pte, which means a
new page must be allocated on first write (note this is a write
operation) but returns zero on read.

However, because you are at elevated IRQL (irql = 0xff is the special
way they report “interrupts disabled”) the page fault cannot be
processed and the system crashes to advise you of this error/issue.
Why the IRQL is elevated is not clear to me from the information that
you provided.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 2:13 PM
To: ntfsd redirect
Subject: [ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Hi,

One of my test machines crashed with bug check 0x0a, in
ExRaiseHardError,
while handling a delayed write failure.
The file on which delayed write failed (the file name in the passed to
ExRaiseHardError) is a file on my FS driver. while delayed write failure
is
one of my problems, I’m interested in finding out why the system crashed

while reporting this error. From the stack trace and parameters, it
doesn’t
look like it was trying to access anything related to the file object or
any
of the associated structure.

From the dump, it looks like ExRaiseHardError called
ZwAllocateVirtualMemory
with a NULL BaseAddress value (not the pointer) and
ZwAllocateVirtualMemory
returned STATUS_SUCCESS, with BaseAddress value as 0x550000. When
ExRaiseHardError accessed this, system bug checked.

Has anyone else seen similar crash? Any idea why ZwAllocateVirtualMemory

would return an invalid address? Are there any extension commands that
can
provide more details about the allocations made by
ZwAllocateVirtualMemory
and related functions (like !pool etc.)?

0: kd> !analyze -v
***********************************************************



Bugcheck Analysis



**************************************************************


IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address
at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 00550000, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000001, value 0 = read operation, 1 = write operation
Arg4: 809bbeee, address which referenced memory

Debugging Details:
------------------

WRITE_ADDRESS: 00550000

CURRENT_IRQL: 0

FAULTING_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx
4],edx

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8098b193 to 809bbeee

TRAP_FRAME: f7906c6c – (.trap fffffffff7906c6c)
ErrCode = 00000002
eax=00000000 ebx=00000001 ecx=00000000 edx=00550014 esi=00550000
edi=0055003c
eip=809bbeee esp=f7906ce0 ebp=f7906d4c iopl=0 nv up di pl nz na
po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010006
nt!ExRaiseHardError+0xc5:
809bbeee 89148e mov [esi+ecx
4],edx
ds:0023:00550000=???
Resetting default scope

STACK_TEXT:
f7906d4c 8098b193 c0000222 00000001 00000001 nt!ExRaiseHardError+0xc5
f7906d80 8083f72e 00000000 00000000 8a78a8d0 nt!IopHardErrorThread+0x53
f7906dac 8092ccff 00000000 00000000 00000000 nt!ExpWorkerThread+0xeb
f7906ddc 80841a96 8083f671 00000001 00000000
nt!PspSystemThreadStartup+0x2e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

FOLLOWUP_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx
4],edx

Thanks,
- Hrishikesh.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Hi Tony,

Here is how I came to the conclusion ZwAllocateVirtualAddress returned wrong
value.
Please let me know any errors you find in the disassembly/any other
comments.

0: kd> kv
ChildEBP RetAddr Args to Child
f7906c6c 809bbeee badb0d00 00550014 00000000 nt!KiTrap0E+0x2a7 (FPO: [0,0]
TrapFrame @ f7906c6c)
f7906d4c 8098b193 c0000222 00000001 00000001 nt!ExRaiseHardError+0xc5 (FPO:
[Non-Fpo])
f7906d80 8083f72e 00000000 00000000 8a78a8d0 nt!IopHardErrorThread+0x53
(FPO: [Non-Fpo])
f7906dac 8092ccff 00000000 00000000 00000000 nt!ExpWorkerThread+0xeb (FPO:
[Non-Fpo])
f7906ddc 80841a96 8083f671 00000001 00000000 nt!PspSystemThreadStartup+0x2e
(FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

0: kd> .trap f7906c6c
ErrCode = 00000002
eax=00000000 ebx=00000001 ecx=00000000 edx=00550014 esi=00550000
edi=0055003c
eip=809bbeee esp=f7906ce0 ebp=f7906d4c iopl=0 nv up di pl nz na po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010006
nt!ExRaiseHardError+0xc5:
809bbeee 89148e mov [esi+ecx*4],edx
ds:0023:00550000=???

0: kd> dd f7906d4c
f7906d4c f7906d80 8098b193 c0000222 00000001
f7906d5c 00000001 f7906d78 00000007 f7906d74
f7906d6c 808b70dc 808addc0 00000000 8848f3a4
f7906d7c 8083f707 f7906dac 8083f72e 00000000
f7906d8c 00000000 8a78a8d0 00000000 00000000
f7906d9c 00000000 00000001 00000000 8098b140
f7906dac f7906ddc 8092ccff 00000000 00000000
f7906dbc 00000000 00000000 f7906db8 00000000

Here is the disassembly
0: kd> u nt!ExRaiseHardError 0x809bbf04
nt!ExRaiseHardError:
809bbe29 6a50 push 0x50
809bbe2b 6898228780 push 0x80872298
809bbe30 e8d030e8ff call nt!_SEH_prolog (8083ef05)
809bbe35 803decc58a8000 cmp byte ptr [nt!ExpTooLateForErrors
(808ac5ec)],0x0 ; @0x808ac5ec == 0
809bbe3c 7410 jz nt!ExRaiseHardError+0x25 (809bbe4e)
809bbe3e 8b451c mov eax,[ebp+0x1c]
809bbe41 c70001000000 mov dword ptr [eax],0x1
809bbe47 33c0 xor eax,eax
809bbe49 e95a010000 jmp nt!ExRaiseHardError+0x175 (809bbfa8)
809bbe4e 33f6 xor esi,esi
; ESI == 0
809bbe50 8975e4 mov [ebp-0x1c],esi
; Var1 = 0
809bbe53 8b4514 mov eax,[ebp+0x14]
; EAX == f7906d78 (Pointer to Parameters Array)
809bbe56 3bc6 cmp eax,esi
; if (pArray == NULL)
809bbe58 0f8402010000 je nt!ExRaiseHardError+0x12d (809bbf60)
809bbe5e 8b5d10 mov ebx,[ebp+0x10]
; EBX = 1 (Unicodestring parameter mask)
809bbe61 3bde cmp ebx,esi
; if (UnicodestringMask == 0)
809bbe63 0f84f4000000 je nt!ExRaiseHardError+0x12a (809bbf5d)
;
809bbe69 c745e044000000 mov dword ptr [ebp-0x20],0x44
; Var2 = ‘D’
809bbe70 33c9 xor ecx,ecx
; ECX = 0
809bbe72 39750c cmp [ebp+0xc],esi
; if (NumParameters == 0)
809bbe75 7627 jbe nt!ExRaiseHardError+0x75 (809bbe9e)
;
809bbe77 33d2 xor edx,edx
; EDX = 0
809bbe79 42 inc edx
; EDX = 1
809bbe7a d3e2 shl edx,cl
; EDX <<= CL (EDX remains 1)
809bbe7c 85d3 test ebx,edx
; if (EBX & EDX)
809bbe7e 7418 jz nt!ExRaiseHardError+0x6f (809bbe98)
809bbe80 8b1488 mov edx,[eax+ecx*4]
; EDX <= first DWORD from Params array
809bbe83 8b3a mov edi,[edx]
; EDI <= Actual parameter
809bbe85 897ccda0 mov [ebp+ecx*8-0x60],edi
; Var3 <- Pointer to UNICODE_STRING
809bbe89 8b5204 mov edx,[edx+0x4]
; EDX <- Pointer to Buffer
809bbe8c 8954cda4 mov [ebp+ecx*8-0x5c],edx
; Var4 <- Ptr to Buffer
809bbe90 0fb754cda2 movzx edx,word ptr [ebp+ecx*8-0x5e]
; EDX <- sign extended Maximum Length (0xf2)
809bbe95 0155e0 add [ebp-0x20],edx
; Var2 += 0xf2 => 0x44 + 0xf2 = 0x136
809bbe98 41 inc ecx
; Next parameter
809bbe99 3b4d0c cmp ecx,[ebp+0xc]
; ECX == NumParameters
809bbe9c 72d9 jb nt!ExRaiseHardError+0x4e (809bbe77)
;
809bbe9e 6a04 push 0x4
; PAGE_READWRITE (Protect)
809bbea0 6800100000 push 0x1000
; MEM_COMMIT (AllocationType)
809bbea5 8d45e0 lea eax,[ebp-0x20]
;
809bbea8 50 push eax
; &Var2 (RegionSize) == 0x136
809bbea9 56 push esi
; 0 (ZeroBits)
809bbeaa 8d45e4 lea eax,[ebp-0x1c]
;
809bbead 50 push eax
; &Var1 (BaseAddress) == 0
809bbeae 6aff push 0xff
; 0xff?? -1?? CurrentProcess??
809bbeb0 e88201e8ff call nt!ZwAllocateVirtualMemory (8083c037)
;
809bbeb5 3bc6 cmp eax,esi
; Status < 0 (!NT_SUCCESS)
809bbeb7 0f8ceb000000 jl nt!ExRaiseHardError+0x175 (809bbfa8)
; must have been successful (fault addr < 809bbfa8
809bbebd 8b55e4 mov edx,[ebp-0x1c]
; BaseAddress returned (0x00550000 ???)
809bbec0 8bf2 mov esi,edx
; ESI <- EDX <- BaseAddr <- 0x00550000
809bbec2 8975d0 mov [ebp-0x30],esi
; Var5 <- BaseAddress
809bbec5 83c214 add edx,0x14
; EDX += 0x14
809bbec8 8955cc mov [ebp-0x34],edx
; Var6 <- BaseAddress + 0x14 ???
809bbecb 8d7a28 lea edi,[edx+0x28]
; EDI <- BaseAddress + 0x14 + 0x28
809bbece 897ddc mov [ebp-0x24],edi
; Var7 <- &(BaseAddress + 0x3C) ???
809bbed1 33c9 xor ecx,ecx
; ECX <- 0
809bbed3 894dfc mov [ebp-0x4],ecx
; Var8 <- 0
809bbed6 894dd8 mov [ebp-0x28],ecx
; Var9 <- 0
809bbed9 3b4d0c cmp ecx,[ebp+0xc]
; if (0 == NumParameters)
809bbedc 7379 jnb nt!ExRaiseHardError+0x124 (809bbf57)
809bbede 33c0 xor eax,eax
; EAX = 0
809bbee0 40 inc eax
; EAX = 1
809bbee1 d3e0 shl eax,cl
; EAX <<= ECX (EAX == 1)
809bbee3 85c3 test ebx,eax
; if (EAX & EBX) - i.e. (1 & 1)
809bbee5 7450 jz nt!ExRaiseHardError+0x10e (809bbf37)
;
809bbee7 8bc1 mov eax,ecx
; EAX <- ECX; EAX == 0
809bbee9 c1e003 shl eax,0x3
; EAX <<= 3; EAX == 0
809bbeec 03d0 add edx,eax
; EDX += EAX; EDX == 0x00550014
809bbeee 89148e mov [esi+ecx*4],edx
; *ESI <- EDX <========= FAULT!!!
809bbef1 8d4c05a2 lea ecx,[ebp+eax-0x5e]
809bbef5 894dd4 mov [ebp-0x2c],ecx
809bbef8 0fb709 movzx ecx,word ptr [ecx]
809bbefb 8b7405a4 mov esi,[ebp+eax-0x5c]
809bbeff 8bd9 mov ebx,ecx
809bbf01 c1e902 shr ecx,0x2

And the data returned by ZwAllocateVirtualMemory

0: kd> dd 0xf7906d4c-0x1c
f7906d30 00550000 f7906ce0 8a78a8d0 f7906dcc
f7906d40 8083b9bc 80872298 00000000 f7906d80
f7906d50 8098b193 c0000222 00000001 00000001
f7906d60 f7906d78 00000007 f7906d74 808b70dc
f7906d70 808addc0 00000000 8848f3a4 8083f707
f7906d80 f7906dac 8083f72e 00000000 00000000
f7906d90 8a78a8d0 00000000 00000000 00000000
f7906da0 00000001 00000000 8098b140 f7906ddc

Thanks,

  • Hrishikesh.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
KeBugCheckEx explicitly looks to see if interrupts are disabled and
“reports” the IRQL as 0xff (an invalid IRQL value). That’s independent
of the last call to KeRaiseIrql.

The PDE itself isn’t valid - there’s nothing within that 4MB region of
memory, so the PTE has no meaning.

Alas, from the information you’ve dug out, it isn’t clear to me what is
happening. I’ve relied upon your comment that ESI is the value returned
from ZwAllocateVirtualMemory, although I do find it odd that it would be
called in system process context (it should work, but it is odd) unless
the memory allocated was in a different process context than the
current one (and finding the PDE/PTE in that case is considerably more
work.)

With a crash dump (or live crash) there would be lots of things to
examine - the disassembly, the context of the operation, the original
thread/process that generated the hard error, etc. But WinDBG over SMTP
isn’t terribly efficient, either.

Perhaps someone else will have some further insight to share.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 3:35 PM
To: ntfsd redirect
Subject: Re:[ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Thanks Tony.

Running !irql reports IRQL 0, so I guess 0xff was set when it entered
the
bug check code.

0: kd> !irql
Debugger saved IRQL for processor 0x0 – 0 (LOW_LEVEL)

Running !pte on this address gives

0: kd> !pte 0x550000
VA 00550000
PDE at C0300004 PTE at C0001540
contains 00000000

Can you please tell me how can I check if it’s a demand zero PTE? Does
the
“contains 00000000” indicate demand zero PTE? 0xc0001540 is not a valid
address in the dump.

Thanks,
- Hrishikesh.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
The address is probably a legitimate address (did you try “!pte” on that
address?) I suspect you’ll find it is a demand zero pte, which means a
new page must be allocated on first write (note this is a write
operation) but returns zero on read.

However, because you are at elevated IRQL (irql = 0xff is the special
way they report “interrupts disabled”) the page fault cannot be
processed and the system crashes to advise you of this error/issue.
Why the IRQL is elevated is not clear to me from the information that
you provided.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 2:13 PM
To: ntfsd redirect
Subject: [ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Hi,

One of my test machines crashed with bug check 0x0a, in
ExRaiseHardError,
while handling a delayed write failure.
The file on which delayed write failed (the file name in the passed to
ExRaiseHardError) is a file on my FS driver. while delayed write failure
is
one of my problems, I’m interested in finding out why the system crashed

while reporting this error. From the stack trace and parameters, it
doesn’t
look like it was trying to access anything related to the file object or
any
of the associated structure.

From the dump, it looks like ExRaiseHardError called
ZwAllocateVirtualMemory
with a NULL BaseAddress value (not the pointer) and
ZwAllocateVirtualMemory
returned STATUS_SUCCESS, with BaseAddress value as 0x550000. When
ExRaiseHardError accessed this, system bug checked.

Has anyone else seen similar crash? Any idea why ZwAllocateVirtualMemory

would return an invalid address? Are there any extension commands that
can
provide more details about the allocations made by
ZwAllocateVirtualMemory
and related functions (like !pool etc.)?

0: kd> !analyze -v
***********************************************************



Bugcheck Analysis



**************************************************************


IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address
at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 00550000, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000001, value 0 = read operation, 1 = write operation
Arg4: 809bbeee, address which referenced memory

Debugging Details:
------------------

WRITE_ADDRESS: 00550000

CURRENT_IRQL: 0

FAULTING_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx
4],edx

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8098b193 to 809bbeee

TRAP_FRAME: f7906c6c – (.trap fffffffff7906c6c)
ErrCode = 00000002
eax=00000000 ebx=00000001 ecx=00000000 edx=00550014 esi=00550000
edi=0055003c
eip=809bbeee esp=f7906ce0 ebp=f7906d4c iopl=0 nv up di pl nz na
po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010006
nt!ExRaiseHardError+0xc5:
809bbeee 89148e mov [esi+ecx
4],edx
ds:0023:00550000=???
Resetting default scope

STACK_TEXT:
f7906d4c 8098b193 c0000222 00000001 00000001 nt!ExRaiseHardError+0xc5
f7906d80 8083f72e 00000000 00000000 8a78a8d0 nt!IopHardErrorThread+0x53
f7906dac 8092ccff 00000000 00000000 00000000 nt!ExpWorkerThread+0xeb
f7906ddc 80841a96 8083f671 00000001 00000000
nt!PspSystemThreadStartup+0x2e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

FOLLOWUP_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx
4],edx

Thanks,
- Hrishikesh.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

I understand the logic you follow here. The ZwAllocateVirtualMemory
call is passed -1 (push 0xff, sign extends and -1 is the
NtCurrentProcess() macro value.) So this IS allocating space in the
current process. This is the system process, which is (again) fine.

I’m suspicious of the IRP - that’s where the information is derived to
tell the OS how to handle the hard error. You might want to look around
and see if you can find the original IRP. Do you call IoRaiseHardError
within your driver? If not, someone ELSE has raised it.

If this were a UP system I’d be pretty sure you’d find the original
request (low priority thread posts to high priority work queue and
pre-empts) but on MP it is quite possible that the originating thread
has continued onwards. The next step is to look at the other threads
running and then after that the ready threads and then after that use
“!irpfind” (a good thing to do before you head home for the day - it is
VERY slow generally.)

What you want to see is the IRP in question - how was it constructed,
how did it point. If it makes you feel better, I’ve reported bugs in
hard error posting (in the SCSI disk class driver code) so I know there
are real errors down these paths (think about it - rare condition and
the system crashes. How many peoplereally do research this?)

I wish I could give you more to go on here.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 5:11 PM
To: ntfsd redirect
Subject: Re:[ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Hi Tony,

Here is how I came to the conclusion ZwAllocateVirtualAddress returned
wrong
value.
Please let me know any errors you find in the disassembly/any other
comments.

0: kd> kv
ChildEBP RetAddr Args to Child
f7906c6c 809bbeee badb0d00 00550014 00000000 nt!KiTrap0E+0x2a7 (FPO:
[0,0]
TrapFrame @ f7906c6c)
f7906d4c 8098b193 c0000222 00000001 00000001 nt!ExRaiseHardError+0xc5
(FPO:
[Non-Fpo])
f7906d80 8083f72e 00000000 00000000 8a78a8d0 nt!IopHardErrorThread+0x53
(FPO: [Non-Fpo])
f7906dac 8092ccff 00000000 00000000 00000000 nt!ExpWorkerThread+0xeb
(FPO:
[Non-Fpo])
f7906ddc 80841a96 8083f671 00000001 00000000
nt!PspSystemThreadStartup+0x2e
(FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

0: kd> .trap f7906c6c
ErrCode = 00000002
eax=00000000 ebx=00000001 ecx=00000000 edx=00550014 esi=00550000
edi=0055003c
eip=809bbeee esp=f7906ce0 ebp=f7906d4c iopl=0 nv up di pl nz na
po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010006
nt!ExRaiseHardError+0xc5:
809bbeee 89148e mov [esi+ecx*4],edx
ds:0023:00550000=???

0: kd> dd f7906d4c
f7906d4c f7906d80 8098b193 c0000222 00000001
f7906d5c 00000001 f7906d78 00000007 f7906d74
f7906d6c 808b70dc 808addc0 00000000 8848f3a4
f7906d7c 8083f707 f7906dac 8083f72e 00000000
f7906d8c 00000000 8a78a8d0 00000000 00000000
f7906d9c 00000000 00000001 00000000 8098b140
f7906dac f7906ddc 8092ccff 00000000 00000000
f7906dbc 00000000 00000000 f7906db8 00000000

Here is the disassembly
0: kd> u nt!ExRaiseHardError 0x809bbf04
nt!ExRaiseHardError:
809bbe29 6a50 push 0x50
809bbe2b 6898228780 push 0x80872298
809bbe30 e8d030e8ff call nt!_SEH_prolog (8083ef05)
809bbe35 803decc58a8000 cmp byte ptr [nt!ExpTooLateForErrors
(808ac5ec)],0x0 ; @0x808ac5ec == 0
809bbe3c 7410 jz nt!ExRaiseHardError+0x25 (809bbe4e)
809bbe3e 8b451c mov eax,[ebp+0x1c]
809bbe41 c70001000000 mov dword ptr [eax],0x1
809bbe47 33c0 xor eax,eax
809bbe49 e95a010000 jmp nt!ExRaiseHardError+0x175 (809bbfa8)
809bbe4e 33f6 xor esi,esi
; ESI == 0
809bbe50 8975e4 mov [ebp-0x1c],esi
; Var1 = 0
809bbe53 8b4514 mov eax,[ebp+0x14]
; EAX == f7906d78 (Pointer to Parameters Array)
809bbe56 3bc6 cmp eax,esi
; if (pArray == NULL)
809bbe58 0f8402010000 je nt!ExRaiseHardError+0x12d (809bbf60)
809bbe5e 8b5d10 mov ebx,[ebp+0x10]
; EBX = 1 (Unicodestring parameter mask)
809bbe61 3bde cmp ebx,esi
; if (UnicodestringMask == 0)
809bbe63 0f84f4000000 je nt!ExRaiseHardError+0x12a (809bbf5d)
;
809bbe69 c745e044000000 mov dword ptr [ebp-0x20],0x44
; Var2 = ‘D’
809bbe70 33c9 xor ecx,ecx
; ECX = 0
809bbe72 39750c cmp [ebp+0xc],esi
; if (NumParameters == 0)
809bbe75 7627 jbe nt!ExRaiseHardError+0x75 (809bbe9e)
;
809bbe77 33d2 xor edx,edx
; EDX = 0
809bbe79 42 inc edx
; EDX = 1
809bbe7a d3e2 shl edx,cl
; EDX <<= CL (EDX remains 1)
809bbe7c 85d3 test ebx,edx
; if (EBX & EDX)
809bbe7e 7418 jz nt!ExRaiseHardError+0x6f (809bbe98)
809bbe80 8b1488 mov edx,[eax+ecx*4]
; EDX <= first DWORD from Params array
809bbe83 8b3a mov edi,[edx]
; EDI <= Actual parameter
809bbe85 897ccda0 mov [ebp+ecx*8-0x60],edi
; Var3 <- Pointer to UNICODE_STRING
809bbe89 8b5204 mov edx,[edx+0x4]
; EDX <- Pointer to Buffer
809bbe8c 8954cda4 mov [ebp+ecx*8-0x5c],edx
; Var4 <- Ptr to Buffer
809bbe90 0fb754cda2 movzx edx,word ptr [ebp+ecx*8-0x5e]
; EDX <- sign extended Maximum Length (0xf2)
809bbe95 0155e0 add [ebp-0x20],edx
; Var2 += 0xf2 => 0x44 + 0xf2 = 0x136
809bbe98 41 inc ecx
; Next parameter
809bbe99 3b4d0c cmp ecx,[ebp+0xc]
; ECX == NumParameters
809bbe9c 72d9 jb nt!ExRaiseHardError+0x4e (809bbe77)
;
809bbe9e 6a04 push 0x4
; PAGE_READWRITE (Protect)
809bbea0 6800100000 push 0x1000
; MEM_COMMIT (AllocationType)
809bbea5 8d45e0 lea eax,[ebp-0x20]
;
809bbea8 50 push eax
; &Var2 (RegionSize) == 0x136
809bbea9 56 push esi
; 0 (ZeroBits)
809bbeaa 8d45e4 lea eax,[ebp-0x1c]
;
809bbead 50 push eax
; &Var1 (BaseAddress) == 0
809bbeae 6aff push 0xff
; 0xff?? -1?? CurrentProcess??
809bbeb0 e88201e8ff call nt!ZwAllocateVirtualMemory (8083c037)
;
809bbeb5 3bc6 cmp eax,esi
; Status < 0 (!NT_SUCCESS)
809bbeb7 0f8ceb000000 jl nt!ExRaiseHardError+0x175 (809bbfa8)
; must have been successful (fault addr < 809bbfa8
809bbebd 8b55e4 mov edx,[ebp-0x1c]
; BaseAddress returned (0x00550000 ???)
809bbec0 8bf2 mov esi,edx
; ESI <- EDX <- BaseAddr <- 0x00550000
809bbec2 8975d0 mov [ebp-0x30],esi
; Var5 <- BaseAddress
809bbec5 83c214 add edx,0x14
; EDX += 0x14
809bbec8 8955cc mov [ebp-0x34],edx
; Var6 <- BaseAddress + 0x14 ???
809bbecb 8d7a28 lea edi,[edx+0x28]
; EDI <- BaseAddress + 0x14 + 0x28
809bbece 897ddc mov [ebp-0x24],edi
; Var7 <- &(BaseAddress + 0x3C) ???
809bbed1 33c9 xor ecx,ecx
; ECX <- 0
809bbed3 894dfc mov [ebp-0x4],ecx
; Var8 <- 0
809bbed6 894dd8 mov [ebp-0x28],ecx
; Var9 <- 0
809bbed9 3b4d0c cmp ecx,[ebp+0xc]
; if (0 == NumParameters)
809bbedc 7379 jnb nt!ExRaiseHardError+0x124 (809bbf57)
809bbede 33c0 xor eax,eax
; EAX = 0
809bbee0 40 inc eax
; EAX = 1
809bbee1 d3e0 shl eax,cl
; EAX <<= ECX (EAX == 1)
809bbee3 85c3 test ebx,eax
; if (EAX & EBX) - i.e. (1 & 1)
809bbee5 7450 jz nt!ExRaiseHardError+0x10e (809bbf37)
;
809bbee7 8bc1 mov eax,ecx
; EAX <- ECX; EAX == 0
809bbee9 c1e003 shl eax,0x3
; EAX <<= 3; EAX == 0
809bbeec 03d0 add edx,eax
; EDX += EAX; EDX == 0x00550014
809bbeee 89148e mov [esi+ecx*4],edx
; *ESI <- EDX <========= FAULT!!!
809bbef1 8d4c05a2 lea ecx,[ebp+eax-0x5e]
809bbef5 894dd4 mov [ebp-0x2c],ecx
809bbef8 0fb709 movzx ecx,word ptr [ecx]
809bbefb 8b7405a4 mov esi,[ebp+eax-0x5c]
809bbeff 8bd9 mov ebx,ecx
809bbf01 c1e902 shr ecx,0x2

And the data returned by ZwAllocateVirtualMemory

0: kd> dd 0xf7906d4c-0x1c
f7906d30 00550000 f7906ce0 8a78a8d0 f7906dcc
f7906d40 8083b9bc 80872298 00000000 f7906d80
f7906d50 8098b193 c0000222 00000001 00000001
f7906d60 f7906d78 00000007 f7906d74 808b70dc
f7906d70 808addc0 00000000 8848f3a4 8083f707
f7906d80 f7906dac 8083f72e 00000000 00000000
f7906d90 8a78a8d0 00000000 00000000 00000000
f7906da0 00000001 00000000 8098b140 f7906ddc

Thanks,

  • Hrishikesh.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
KeBugCheckEx explicitly looks to see if interrupts are disabled and
“reports” the IRQL as 0xff (an invalid IRQL value). That’s independent
of the last call to KeRaiseIrql.

The PDE itself isn’t valid - there’s nothing within that 4MB region of
memory, so the PTE has no meaning.

Alas, from the information you’ve dug out, it isn’t clear to me what is
happening. I’ve relied upon your comment that ESI is the value returned
from ZwAllocateVirtualMemory, although I do find it odd that it would be
called in system process context (it should work, but it is odd) unless
the memory allocated was in a different process context than the
current one (and finding the PDE/PTE in that case is considerably more
work.)

With a crash dump (or live crash) there would be lots of things to
examine - the disassembly, the context of the operation, the original
thread/process that generated the hard error, etc. But WinDBG over SMTP
isn’t terribly efficient, either.

Perhaps someone else will have some further insight to share.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 3:35 PM
To: ntfsd redirect
Subject: Re:[ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Thanks Tony.

Running !irql reports IRQL 0, so I guess 0xff was set when it entered
the
bug check code.

0: kd> !irql
Debugger saved IRQL for processor 0x0 – 0 (LOW_LEVEL)

Running !pte on this address gives

0: kd> !pte 0x550000
VA 00550000
PDE at C0300004 PTE at C0001540
contains 00000000

Can you please tell me how can I check if it’s a demand zero PTE? Does
the
“contains 00000000” indicate demand zero PTE? 0xc0001540 is not a valid
address in the dump.

Thanks,
- Hrishikesh.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
The address is probably a legitimate address (did you try “!pte” on that
address?) I suspect you’ll find it is a demand zero pte, which means a
new page must be allocated on first write (note this is a write
operation) but returns zero on read.

However, because you are at elevated IRQL (irql = 0xff is the special
way they report “interrupts disabled”) the page fault cannot be
processed and the system crashes to advise you of this error/issue.
Why the IRQL is elevated is not clear to me from the information that
you provided.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 2:13 PM
To: ntfsd redirect
Subject: [ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Hi,

One of my test machines crashed with bug check 0x0a, in
ExRaiseHardError,
while handling a delayed write failure.
The file on which delayed write failed (the file name in the passed to
ExRaiseHardError) is a file on my FS driver. while delayed write failure
is
one of my problems, I’m interested in finding out why the system crashed

while reporting this error. From the stack trace and parameters, it
doesn’t
look like it was trying to access anything related to the file object or
any
of the associated structure.

From the dump, it looks like ExRaiseHardError called
ZwAllocateVirtualMemory
with a NULL BaseAddress value (not the pointer) and
ZwAllocateVirtualMemory
returned STATUS_SUCCESS, with BaseAddress value as 0x550000. When
ExRaiseHardError accessed this, system bug checked.

Has anyone else seen similar crash? Any idea why ZwAllocateVirtualMemory

would return an invalid address? Are there any extension commands that
can
provide more details about the allocations made by
ZwAllocateVirtualMemory
and related functions (like !pool etc.)?

0: kd> !analyze -v
***********************************************************



Bugcheck Analysis



**************************************************************


IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address
at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 00550000, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000001, value 0 = read operation, 1 = write operation
Arg4: 809bbeee, address which referenced memory

Debugging Details:
------------------

WRITE_ADDRESS: 00550000

CURRENT_IRQL: 0

FAULTING_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx
4],edx

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8098b193 to 809bbeee

TRAP_FRAME: f7906c6c – (.trap fffffffff7906c6c)
ErrCode = 00000002
eax=00000000 ebx=00000001 ecx=00000000 edx=00550014 esi=00550000
edi=0055003c
eip=809bbeee esp=f7906ce0 ebp=f7906d4c iopl=0 nv up di pl nz na
po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010006
nt!ExRaiseHardError+0xc5:
809bbeee 89148e mov [esi+ecx
4],edx
ds:0023:00550000=???
Resetting default scope

STACK_TEXT:
f7906d4c 8098b193 c0000222 00000001 00000001 nt!ExRaiseHardError+0xc5
f7906d80 8083f72e 00000000 00000000 8a78a8d0 nt!IopHardErrorThread+0x53
f7906dac 8092ccff 00000000 00000000 00000000 nt!ExpWorkerThread+0xeb
f7906ddc 80841a96 8083f671 00000001 00000000
nt!PspSystemThreadStartup+0x2e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

FOLLOWUP_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx
4],edx

Thanks,
- Hrishikesh.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Thanks for the information, Tony.

My driver doesn’t call IoRaiseHardError. I’ll try to find out the IRP as you
suggested.

Thanks,

  • Hrishikesh.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
I understand the logic you follow here. The ZwAllocateVirtualMemory
call is passed -1 (push 0xff, sign extends and -1 is the
NtCurrentProcess() macro value.) So this IS allocating space in the
current process. This is the system process, which is (again) fine.

I’m suspicious of the IRP - that’s where the information is derived to
tell the OS how to handle the hard error. You might want to look around
and see if you can find the original IRP. Do you call IoRaiseHardError
within your driver? If not, someone ELSE has raised it.

If this were a UP system I’d be pretty sure you’d find the original
request (low priority thread posts to high priority work queue and
pre-empts) but on MP it is quite possible that the originating thread
has continued onwards. The next step is to look at the other threads
running and then after that the ready threads and then after that use
“!irpfind” (a good thing to do before you head home for the day - it is
VERY slow generally.)

What you want to see is the IRP in question - how was it constructed,
how did it point. If it makes you feel better, I’ve reported bugs in
hard error posting (in the SCSI disk class driver code) so I know there
are real errors down these paths (think about it - rare condition and
the system crashes. How many peoplereally do research this?)

I wish I could give you more to go on here.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 5:11 PM
To: ntfsd redirect
Subject: Re:[ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Hi Tony,

Here is how I came to the conclusion ZwAllocateVirtualAddress returned
wrong
value.
Please let me know any errors you find in the disassembly/any other
comments.

0: kd> kv
ChildEBP RetAddr Args to Child
f7906c6c 809bbeee badb0d00 00550014 00000000 nt!KiTrap0E+0x2a7 (FPO:
[0,0]
TrapFrame @ f7906c6c)
f7906d4c 8098b193 c0000222 00000001 00000001 nt!ExRaiseHardError+0xc5
(FPO:
[Non-Fpo])
f7906d80 8083f72e 00000000 00000000 8a78a8d0 nt!IopHardErrorThread+0x53
(FPO: [Non-Fpo])
f7906dac 8092ccff 00000000 00000000 00000000 nt!ExpWorkerThread+0xeb
(FPO:
[Non-Fpo])
f7906ddc 80841a96 8083f671 00000001 00000000
nt!PspSystemThreadStartup+0x2e
(FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

0: kd> .trap f7906c6c
ErrCode = 00000002
eax=00000000 ebx=00000001 ecx=00000000 edx=00550014 esi=00550000
edi=0055003c
eip=809bbeee esp=f7906ce0 ebp=f7906d4c iopl=0 nv up di pl nz na
po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010006
nt!ExRaiseHardError+0xc5:
809bbeee 89148e mov [esi+ecx4],edx
ds:0023:00550000=???

0: kd> dd f7906d4c
f7906d4c f7906d80 8098b193 c0000222 00000001
f7906d5c 00000001 f7906d78 00000007 f7906d74
f7906d6c 808b70dc 808addc0 00000000 8848f3a4
f7906d7c 8083f707 f7906dac 8083f72e 00000000
f7906d8c 00000000 8a78a8d0 00000000 00000000
f7906d9c 00000000 00000001 00000000 8098b140
f7906dac f7906ddc 8092ccff 00000000 00000000
f7906dbc 00000000 00000000 f7906db8 00000000

Here is the disassembly
0: kd> u nt!ExRaiseHardError 0x809bbf04
nt!ExRaiseHardError:
809bbe29 6a50 push 0x50
809bbe2b 6898228780 push 0x80872298
809bbe30 e8d030e8ff call nt!_SEH_prolog (8083ef05)
809bbe35 803decc58a8000 cmp byte ptr [nt!ExpTooLateForErrors
(808ac5ec)],0x0 ; @0x808ac5ec == 0
809bbe3c 7410 jz nt!ExRaiseHardError+0x25 (809bbe4e)
809bbe3e 8b451c mov eax,[ebp+0x1c]
809bbe41 c70001000000 mov dword ptr [eax],0x1
809bbe47 33c0 xor eax,eax
809bbe49 e95a010000 jmp nt!ExRaiseHardError+0x175 (809bbfa8)
809bbe4e 33f6 xor esi,esi
; ESI == 0
809bbe50 8975e4 mov [ebp-0x1c],esi
; Var1 = 0
809bbe53 8b4514 mov eax,[ebp+0x14]
; EAX == f7906d78 (Pointer to Parameters Array)
809bbe56 3bc6 cmp eax,esi
; if (pArray == NULL)
809bbe58 0f8402010000 je nt!ExRaiseHardError+0x12d (809bbf60)
809bbe5e 8b5d10 mov ebx,[ebp+0x10]
; EBX = 1 (Unicodestring parameter mask)
809bbe61 3bde cmp ebx,esi
; if (UnicodestringMask == 0)
809bbe63 0f84f4000000 je nt!ExRaiseHardError+0x12a (809bbf5d)
;
809bbe69 c745e044000000 mov dword ptr [ebp-0x20],0x44
; Var2 = ‘D’
809bbe70 33c9 xor ecx,ecx
; ECX = 0
809bbe72 39750c cmp [ebp+0xc],esi
; if (NumParameters == 0)
809bbe75 7627 jbe nt!ExRaiseHardError+0x75 (809bbe9e)
;
809bbe77 33d2 xor edx,edx
; EDX = 0
809bbe79 42 inc edx
; EDX = 1
809bbe7a d3e2 shl edx,cl
; EDX <<= CL (EDX remains 1)
809bbe7c 85d3 test ebx,edx
; if (EBX & EDX)
809bbe7e 7418 jz nt!ExRaiseHardError+0x6f (809bbe98)
809bbe80 8b1488 mov edx,[eax+ecx
4]
; EDX <= first DWORD from Params array
809bbe83 8b3a mov edi,[edx]
; EDI <= Actual parameter
809bbe85 897ccda0 mov [ebp+ecx8-0x60],edi
; Var3 <- Pointer to UNICODE_STRING
809bbe89 8b5204 mov edx,[edx+0x4]
; EDX <- Pointer to Buffer
809bbe8c 8954cda4 mov [ebp+ecx
8-0x5c],edx
; Var4 <- Ptr to Buffer
809bbe90 0fb754cda2 movzx edx,word ptr [ebp+ecx8-0x5e]
; EDX <- sign extended Maximum Length (0xf2)
809bbe95 0155e0 add [ebp-0x20],edx
; Var2 += 0xf2 => 0x44 + 0xf2 = 0x136
809bbe98 41 inc ecx
; Next parameter
809bbe99 3b4d0c cmp ecx,[ebp+0xc]
; ECX == NumParameters
809bbe9c 72d9 jb nt!ExRaiseHardError+0x4e (809bbe77)
;
809bbe9e 6a04 push 0x4
; PAGE_READWRITE (Protect)
809bbea0 6800100000 push 0x1000
; MEM_COMMIT (AllocationType)
809bbea5 8d45e0 lea eax,[ebp-0x20]
;
809bbea8 50 push eax
; &Var2 (RegionSize) == 0x136
809bbea9 56 push esi
; 0 (ZeroBits)
809bbeaa 8d45e4 lea eax,[ebp-0x1c]
;
809bbead 50 push eax
; &Var1 (BaseAddress) == 0
809bbeae 6aff push 0xff
; 0xff?? -1?? CurrentProcess??
809bbeb0 e88201e8ff call nt!ZwAllocateVirtualMemory (8083c037)
;
809bbeb5 3bc6 cmp eax,esi
; Status < 0 (!NT_SUCCESS)
809bbeb7 0f8ceb000000 jl nt!ExRaiseHardError+0x175 (809bbfa8)
; must have been successful (fault addr < 809bbfa8
809bbebd 8b55e4 mov edx,[ebp-0x1c]
; BaseAddress returned (0x00550000 ???)
809bbec0 8bf2 mov esi,edx
; ESI <- EDX <- BaseAddr <- 0x00550000
809bbec2 8975d0 mov [ebp-0x30],esi
; Var5 <- BaseAddress
809bbec5 83c214 add edx,0x14
; EDX += 0x14
809bbec8 8955cc mov [ebp-0x34],edx
; Var6 <- BaseAddress + 0x14 ???
809bbecb 8d7a28 lea edi,[edx+0x28]
; EDI <- BaseAddress + 0x14 + 0x28
809bbece 897ddc mov [ebp-0x24],edi
; Var7 <- &(BaseAddress + 0x3C) ???
809bbed1 33c9 xor ecx,ecx
; ECX <- 0
809bbed3 894dfc mov [ebp-0x4],ecx
; Var8 <- 0
809bbed6 894dd8 mov [ebp-0x28],ecx
; Var9 <- 0
809bbed9 3b4d0c cmp ecx,[ebp+0xc]
; if (0 == NumParameters)
809bbedc 7379 jnb nt!ExRaiseHardError+0x124 (809bbf57)
809bbede 33c0 xor eax,eax
; EAX = 0
809bbee0 40 inc eax
; EAX = 1
809bbee1 d3e0 shl eax,cl
; EAX <<= ECX (EAX == 1)
809bbee3 85c3 test ebx,eax
; if (EAX & EBX) - i.e. (1 & 1)
809bbee5 7450 jz nt!ExRaiseHardError+0x10e (809bbf37)
;
809bbee7 8bc1 mov eax,ecx
; EAX <- ECX; EAX == 0
809bbee9 c1e003 shl eax,0x3
; EAX <<= 3; EAX == 0
809bbeec 03d0 add edx,eax
; EDX += EAX; EDX == 0x00550014
809bbeee 89148e mov [esi+ecx
4],edx
; *ESI <- EDX <========= FAULT!!!
809bbef1 8d4c05a2 lea ecx,[ebp+eax-0x5e]
809bbef5 894dd4 mov [ebp-0x2c],ecx
809bbef8 0fb709 movzx ecx,word ptr [ecx]
809bbefb 8b7405a4 mov esi,[ebp+eax-0x5c]
809bbeff 8bd9 mov ebx,ecx
809bbf01 c1e902 shr ecx,0x2

And the data returned by ZwAllocateVirtualMemory

0: kd> dd 0xf7906d4c-0x1c
f7906d30 00550000 f7906ce0 8a78a8d0 f7906dcc
f7906d40 8083b9bc 80872298 00000000 f7906d80
f7906d50 8098b193 c0000222 00000001 00000001
f7906d60 f7906d78 00000007 f7906d74 808b70dc
f7906d70 808addc0 00000000 8848f3a4 8083f707
f7906d80 f7906dac 8083f72e 00000000 00000000
f7906d90 8a78a8d0 00000000 00000000 00000000
f7906da0 00000001 00000000 8098b140 f7906ddc

Thanks,
- Hrishikesh.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
KeBugCheckEx explicitly looks to see if interrupts are disabled and
“reports” the IRQL as 0xff (an invalid IRQL value). That’s independent
of the last call to KeRaiseIrql.

The PDE itself isn’t valid - there’s nothing within that 4MB region of
memory, so the PTE has no meaning.

Alas, from the information you’ve dug out, it isn’t clear to me what is
happening. I’ve relied upon your comment that ESI is the value returned
from ZwAllocateVirtualMemory, although I do find it odd that it would be
called in system process context (it should work, but it is odd) unless
the memory allocated was in a different process context than the
current one (and finding the PDE/PTE in that case is considerably more
work.)

With a crash dump (or live crash) there would be lots of things to
examine - the disassembly, the context of the operation, the original
thread/process that generated the hard error, etc. But WinDBG over SMTP
isn’t terribly efficient, either.

Perhaps someone else will have some further insight to share.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 3:35 PM
To: ntfsd redirect
Subject: Re:[ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Thanks Tony.

Running !irql reports IRQL 0, so I guess 0xff was set when it entered
the
bug check code.

0: kd> !irql
Debugger saved IRQL for processor 0x0 – 0 (LOW_LEVEL)

Running !pte on this address gives

0: kd> !pte 0x550000
VA 00550000
PDE at C0300004 PTE at C0001540
contains 00000000

Can you please tell me how can I check if it’s a demand zero PTE? Does
the
“contains 00000000” indicate demand zero PTE? 0xc0001540 is not a valid
address in the dump.

Thanks,
- Hrishikesh.

“Tony Mason” wrote in message news:xxxxx@ntfsd…
The address is probably a legitimate address (did you try “!pte” on that
address?) I suspect you’ll find it is a demand zero pte, which means a
new page must be allocated on first write (note this is a write
operation) but returns zero on read.

However, because you are at elevated IRQL (irql = 0xff is the special
way they report “interrupts disabled”) the page fault cannot be
processed and the system crashes to advise you of this error/issue.
Why the IRQL is elevated is not clear to me from the information that
you provided.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the next OSR File Systems class in
Boston, MA April 24-27, 2006.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Hrishikesh Vidwans
Sent: Friday, January 06, 2006 2:13 PM
To: ntfsd redirect
Subject: [ntfsd] ExRaiseHardError & ZwAllocateVirtualMemory…

Hi,

One of my test machines crashed with bug check 0x0a, in
ExRaiseHardError,
while handling a delayed write failure.
The file on which delayed write failed (the file name in the passed to
ExRaiseHardError) is a file on my FS driver. while delayed write failure
is
one of my problems, I’m interested in finding out why the system crashed

while reporting this error. From the stack trace and parameters, it
doesn’t
look like it was trying to access anything related to the file object or
any
of the associated structure.

From the dump, it looks like ExRaiseHardError called
ZwAllocateVirtualMemory
with a NULL BaseAddress value (not the pointer) and
ZwAllocateVirtualMemory
returned STATUS_SUCCESS, with BaseAddress value as 0x550000. When
ExRaiseHardError accessed this, system bug checked.

Has anyone else seen similar crash? Any idea why ZwAllocateVirtualMemory

would return an invalid address? Are there any extension commands that
can
provide more details about the allocations made by
ZwAllocateVirtualMemory
and related functions (like !pool etc.)?

0: kd> !analyze -v
***********************************************************



Bugcheck Analysis



**************************************************************


IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address
at
an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 00550000, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000001, value 0 = read operation, 1 = write operation
Arg4: 809bbeee, address which referenced memory

Debugging Details:
------------------

WRITE_ADDRESS: 00550000

CURRENT_IRQL: 0

FAULTING_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx
4],edx

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xA

LAST_CONTROL_TRANSFER: from 8098b193 to 809bbeee

TRAP_FRAME: f7906c6c – (.trap fffffffff7906c6c)
ErrCode = 00000002
eax=00000000 ebx=00000001 ecx=00000000 edx=00550014 esi=00550000
edi=0055003c
eip=809bbeee esp=f7906ce0 ebp=f7906d4c iopl=0 nv up di pl nz na
po
nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000
efl=00010006
nt!ExRaiseHardError+0xc5:
809bbeee 89148e mov [esi+ecx
4],edx
ds:0023:00550000=???
Resetting default scope

STACK_TEXT:
f7906d4c 8098b193 c0000222 00000001 00000001 nt!ExRaiseHardError+0xc5
f7906d80 8083f72e 00000000 00000000 8a78a8d0 nt!IopHardErrorThread+0x53
f7906dac 8092ccff 00000000 00000000 00000000 nt!ExpWorkerThread+0xeb
f7906ddc 80841a96 8083f671 00000001 00000000
nt!PspSystemThreadStartup+0x2e
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

FOLLOWUP_IP:
nt!ExRaiseHardError+c5
809bbeee 89148e mov [esi+ecx
4],edx

Thanks,
- Hrishikesh.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com