Hoping for a bit of guidance here from folk with more experience of using wow64exts.
OK. So. Driver developer here: when I fire up WinDbg it's normally to do some kernel-mode debugging. However, sometimes when people's machines lock up, they either generate a full memory dump or we attach WinDbg and pull one down the wire.
Often the cause of the lockup lies in the (32 bit) user mode code, so ideally we'd like to analyze these dumps and see the 32 bit stacks even if the dump came from a 64 bit box. But I can't seem to get 'em.
I've been given one such dump to look at to try and *find* the 32 bit stacks.
Well I can do a !process 0 0 and see the 32 bit process there - fine. I can dump its thread stacks (lots of wow64cpu!blah at the bottom... okay...).
0: kd> .process fffffa80079e4b30
Implicit process is now fffffa80`079e4b30
0: kd> !wow64exts.sw
Switched to 32bit mode
0: kd:x86> k
^ Current scope machine type mismatch error in 'k'
0: kd:x86> !wow64exts.k
The context is partially valid. Only x86 user-mode context is available.
Walking 64bit Stack...
*** Stack trace for last set context - .thread/.cxr resets it
Child-SP RetAddr Call Site
fffff880`07740710 fffff800`02ad5752 nt!KiSwapContext+0x7a
fffff880`07740850 fffff800`02ad78af nt!KiCommitThreadWait+0x1d2
fffff880`077408e0 fffff800`02aacfc8 nt!KeWaitForSingleObject+0x19f
fffff880`07740980 fffff800`02aab009 nt!KiSuspendThread+0x74
fffff880`077409c0 fffff800`02ad595d nt!KiDeliverApc+0x211
fffff880`07740a40 fffff800`02ad78af nt!KiCommitThreadWait+0x3dd
fffff880`07740ad0 fffff800`02dc9db2 nt!KeWaitForSingleObject+0x19f
fffff880`07740b70 fffff800`02acd853 nt!NtWaitForSingleObject+0xb2
fffff880`07740be0 00000000`74cb2dd9 nt!KiSystemServiceCopyEnd+0x13
00000000`0008ebf8 00000000`74cb2bcd 0x74cb2dd9
00000000`0008ec00 00000000`77220190 0x74cb2bcd
00000000`0008ec08 00000000`74d20023 0x77220190
00000000`0008ec10 00000000`002cc5a8 0x74d20023
00000000`0008ec18 00000000`0018fff0 0x2cc5a8
00000000`0008ec20 00000000`0008ec50 0x18fff0
00000000`0008ec28 00000000`00390008 0x8ec50
00000000`0008ec30 00000000`00000000 0x390008
Walking 32bit Stack...
^ Current scope machine type mismatch error in 'k '
!wow64exts.k: Execute failed: 0x80040205
?? Current scope machine type mismatch error in 'k ' ??
I suspect there is some key fact I'm not aware of here.
I did have a look in the windbg help but as I interpreted it, if you're grabbing a *full* memory dump it doesn't matter whether the machine running windbg - or the version of windbg being run - is 64 bit or not. So it must be something else - any help gratefully received.
0 ·
Comments
the current implementation or a limitation. I'll demonstrate what I'm seeing
on a live dump as it ends better than the crash dump version
I force a context switch to a 32bit process in WinDBG via .process /i, in
this case I picked 32bit IE:
0: kd> !process -1 0
PROCESS fffffa8002455340
SessionId: 1 Cid: 0aa0 Peb: 7efdf000 ParentCid: 0444
DirBase: 1a2a3000 ObjectTable: fffff8a002e36e40 HandleCount: 327.
Image: iexplore.exe
I then switch to a thread in the process and attempt to do !wow64exts.k,
which meets the same fate as yours:
0: kd> !wow64exts.k
Walking 64bit Stack...
*** Stack trace for last set context - .thread/.cxr resets it
Child-SP RetAddr Call Site
fffff880`0473a660 fffff800`026c2052 nt!KiSwapContext+0x7a
fffff880`0473a7a0 fffff800`026c41af nt!KiCommitThreadWait+0x1d2
fffff880`0473a830 fffff800`026d94ef nt!KeWaitForSingleObject+0x19f
fffff880`0473a8d0 fffff800`029be736 nt!AlpcpSignalAndWait+0x8f
fffff880`0473a980 fffff800`029bc6c0 nt!AlpcpReceiveSynchronousReply+0x46
fffff880`0473a9e0 fffff800`029b9c3b nt!AlpcpProcessSynchronousRequest+0x33d
fffff880`0473ab00 fffff800`026ba153 nt!NtAlpcSendWaitReceivePort+0x1ab
fffff880`0473abb0 00000000`77b6070a nt!KiSystemServiceCopyEnd+0x13
00000000`0371e758 00000000`7581e2f0 ntdll!ZwAlpcSendWaitReceivePort+0xa
00000000`0371e760 00000000`7580cf87 wow64!whNtAlpcSendWaitReceivePort+0x64
00000000`0371e7c0 00000000`7579276d wow64!Wow64SystemServiceEx+0xd7
00000000`0371f080 00000000`7580d07e
wow64cpu!TurboDispatchJumpAddressEnd+0x24
00000000`0371f140 00000000`7580c549 wow64!RunCpuSimulation+0xa
00000000`0371f190 00000000`77b8d177 wow64!Wow64LdrpInitialize+0x429
00000000`0371f6e0 00000000`77b4308e ntdll! ?? ::FNODOBFM::`string'+0x2bfe4
00000000`0371f750 00000000`00000000 ntdll!LdrInitializeThunk+0xe
Walking 32bit Stack...
^ Current scope machine type mismatch error in 'k '
!wow64exts.k: Execute failed: 0x80040205
If you hook a debugger up to this, the stack walk is failing by raising an
exception:
0:003> kc
Call Site
KERNELBASE!RaiseException
dbgeng!ReportError
dbgeng!WrapParseStackCmd
dbgeng!ProcessCommands
dbgeng!ProcessCommandsAndCatch
dbgeng!Execute
dbgeng!DebugClient::ExecuteWide
dbgeng!DebugClient::Execute
wow64exts!k
A call to GetExplicitContext fails, so DbgEng is reporting error 0x1038 (not
sure what that stands for).
After playing with it a bit more, I had a theory that the problem has to do
with the context at the time of the last debugger event. I think that
because we're in 64bit code the 32bit stack walk command is getting confused
and refusing to run. So, I set a breakpoint in the wow64 layer, hit Go,
tried again, and the command works like a charm:
0: kd> ba e1 wow64!RunCpuSimulation
0: kd> g
Breakpoint 7 hit
wow64!RunCpuSimulation:
0033:00000000`7580d074 4883ec48 sub rsp,48h
1: kd> .reload /user
Loading User Symbols
.....
Loading Wow64 Symbols
................................................................
........
1: kd> !wow64exts.k
Walking 64bit Stack...
Child-SP RetAddr Call Site
00000000`0119d778 00000000`75808a40 wow64!RunCpuSimulation
00000000`0119d780 00000000`757d29ee
wow64!Wow64KiUserCallbackDispatcher+0x204
00000000`0119dad0 00000000`77b5fdf5 wow64win!whcbfnDWORD+0xe2
00000000`0119e4c0 00000000`757dfbda ntdll!KiUserCallbackDispatcherContinue
00000000`0119e548 00000000`757bab53 wow64win!NtUserPeekMessage+0xa
00000000`0119e550 00000000`7580cf87 wow64win!whNtUserPeekMessage+0x37
00000000`0119e5c0 00000000`7579276d wow64!Wow64SystemServiceEx+0xd7
00000000`0119ee80 00000000`7580d07e
wow64cpu!TurboDispatchJumpAddressEnd+0x24
00000000`0119ef40 00000000`7580c549 wow64!RunCpuSimulation+0xa
00000000`0119ef90 00000000`77b8d177 wow64!Wow64LdrpInitialize+0x429
00000000`0119f4e0 00000000`77b4308e ntdll! ?? ::FNODOBFM::`string'+0x2bfe4
00000000`0119f550 00000000`00000000 ntdll!LdrInitializeThunk+0xe
Walking 32bit Stack...
ChildEBP RetAddr
02dcfb08 729f03d2 ntdll_77cf0000!KiUserCallbackDispatcher
02dcfb90 72a00446 IEFRAME!CTabWindow::_TabWindowThreadProc+0x24e
02dcfc48 761849bd IEFRAME!LCIETab_ThreadProc+0x2c1
02dcfc58 76673677 iertutil!CIsoScope::RegisterThread+0xab
02dcfc64 77d29d72 kernel32!BaseThreadInitThunk+0xe
02dcfca4 77d29d45 ntdll_77cf0000!__RtlUserThreadStart+0x70
02dcfcbc 00000000 ntdll_77cf0000!_RtlUserThreadStart+0x1b
This doesn't bode overly well for doing this from a crash dump though I
don't play with this all that often, so maybe there's a workaround (or I'm
just insane and doing something wrong also).
-scott
--
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com
<[email protected]> wrote in message news:[email protected]
> Hoping for a bit of guidance here from folk with more experience of using
> wow64exts.
>
> OK. So. Driver developer here: when I fire up WinDbg it's normally to do
> some kernel-mode debugging. However, sometimes when people's machines lock
> up, they either generate a full memory dump or we attach WinDbg and pull
> one down the wire.
> Often the cause of the lockup lies in the (32 bit) user mode code, so
> ideally we'd like to analyze these dumps and see the 32 bit stacks even if
> the dump came from a 64 bit box. But I can't seem to get 'em.
>
> I've been given one such dump to look at to try and *find* the 32 bit
> stacks.
> Well I can do a !process 0 0 and see the 32 bit process there - fine. I
> can dump its thread stacks (lots of wow64cpu!blah at the bottom...
> okay...).
>
> 0: kd> .process fffffa80079e4b30
> Implicit process is now fffffa80`079e4b30
> 0: kd> !wow64exts.sw
> Switched to 32bit mode
> 0: kd:x86> k
> ^ Current scope machine type mismatch error in 'k'
> 0: kd:x86> !wow64exts.k
> The context is partially valid. Only x86 user-mode context is available.
> Walking 64bit Stack...
> *** Stack trace for last set context - .thread/.cxr resets it
> Child-SP RetAddr Call Site
> fffff880`07740710 fffff800`02ad5752 nt!KiSwapContext+0x7a
> fffff880`07740850 fffff800`02ad78af nt!KiCommitThreadWait+0x1d2
> fffff880`077408e0 fffff800`02aacfc8 nt!KeWaitForSingleObject+0x19f
> fffff880`07740980 fffff800`02aab009 nt!KiSuspendThread+0x74
> fffff880`077409c0 fffff800`02ad595d nt!KiDeliverApc+0x211
> fffff880`07740a40 fffff800`02ad78af nt!KiCommitThreadWait+0x3dd
> fffff880`07740ad0 fffff800`02dc9db2 nt!KeWaitForSingleObject+0x19f
> fffff880`07740b70 fffff800`02acd853 nt!NtWaitForSingleObject+0xb2
> fffff880`07740be0 00000000`74cb2dd9 nt!KiSystemServiceCopyEnd+0x13
> 00000000`0008ebf8 00000000`74cb2bcd 0x74cb2dd9
> 00000000`0008ec00 00000000`77220190 0x74cb2bcd
> 00000000`0008ec08 00000000`74d20023 0x77220190
> 00000000`0008ec10 00000000`002cc5a8 0x74d20023
> 00000000`0008ec18 00000000`0018fff0 0x2cc5a8
> 00000000`0008ec20 00000000`0008ec50 0x18fff0
> 00000000`0008ec28 00000000`00390008 0x8ec50
> 00000000`0008ec30 00000000`00000000 0x390008
> Walking 32bit Stack...
> ^ Current scope machine type mismatch error in 'k '
> !wow64exts.k: Execute failed: 0x80040205
>
>
> ?? Current scope machine type mismatch error in 'k ' ??
>
> I suspect there is some key fact I'm not aware of here.
>
> I did have a look in the windbg help but as I interpreted it, if you're
> grabbing a *full* memory dump it doesn't matter whether the machine
> running windbg - or the version of windbg being run - is 64 bit or not. So
> it must be something else - any help gratefully received.
>
-scott
OSR
Thanks,
Pavel
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Scott Noone
Sent: Wednesday, August 25, 2010 11:34 AM
To: Kernel Debugging Interest List
Subject: Re:[windbg] wow64exts, or, where are the 32 bit stacks
OK, there's certainly something wrong here. I'm not sure if this is a bug in the current implementation or a limitation. I'll demonstrate what I'm seeing on a live dump as it ends better than the crash dump version
I force a context switch to a 32bit process in WinDBG via .process /i, in this case I picked 32bit IE:
0: kd> !process -1 0
PROCESS fffffa8002455340
SessionId: 1 Cid: 0aa0 Peb: 7efdf000 ParentCid: 0444
DirBase: 1a2a3000 ObjectTable: fffff8a002e36e40 HandleCount: 327.
Image: iexplore.exe
I then switch to a thread in the process and attempt to do !wow64exts.k, which meets the same fate as yours:
0: kd> !wow64exts.k
Walking 64bit Stack...
*** Stack trace for last set context - .thread/.cxr resets it
Child-SP RetAddr Call Site
fffff880`0473a660 fffff800`026c2052 nt!KiSwapContext+0x7a
fffff880`0473a7a0 fffff800`026c41af nt!KiCommitThreadWait+0x1d2
fffff880`0473a830 fffff800`026d94ef nt!KeWaitForSingleObject+0x19f
fffff880`0473a8d0 fffff800`029be736 nt!AlpcpSignalAndWait+0x8f
fffff880`0473a980 fffff800`029bc6c0 nt!AlpcpReceiveSynchronousReply+0x46
fffff880`0473a9e0 fffff800`029b9c3b nt!AlpcpProcessSynchronousRequest+0x33d
fffff880`0473ab00 fffff800`026ba153 nt!NtAlpcSendWaitReceivePort+0x1ab
fffff880`0473abb0 00000000`77b6070a nt!KiSystemServiceCopyEnd+0x13
00000000`0371e758 00000000`7581e2f0 ntdll!ZwAlpcSendWaitReceivePort+0xa
00000000`0371e760 00000000`7580cf87 wow64!whNtAlpcSendWaitReceivePort+0x64
00000000`0371e7c0 00000000`7579276d wow64!Wow64SystemServiceEx+0xd7
00000000`0371f080 00000000`7580d07e
wow64cpu!TurboDispatchJumpAddressEnd+0x24
00000000`0371f140 00000000`7580c549 wow64!RunCpuSimulation+0xa
00000000`0371f190 00000000`77b8d177 wow64!Wow64LdrpInitialize+0x429
00000000`0371f6e0 00000000`77b4308e ntdll! ?? ::FNODOBFM::`string'+0x2bfe4
00000000`0371f750 00000000`00000000 ntdll!LdrInitializeThunk+0xe Walking 32bit Stack...
^ Current scope machine type mismatch error in 'k '
!wow64exts.k: Execute failed: 0x80040205
mode stack for that thread. It's all a bit hairy though, now !wow64exts.k
won't show the x64 stack and getting back to the real stack isn't overly
intuitive:
1: kd> .thread /w fffffa8001b00660
Implicit thread is now fffffa80`01b00660
x86 context set
1: kd:x86> !wow64exts.k
Walking 64bit Stack...
^ Current scope machine type mismatch error in 'k '
!wow64exts.k: Execute failed: 0x80040205
1: kd:x86> kc
*** Stack trace for last set context - .thread/.cxr resets it
ntdll_77cf0000!NtRemoveIoCompletion
KERNELBASE!GetQueuedCompletionStatus
Dxtrans!TMThreadProc
kernel32!BaseThreadInitThunk
ntdll_77cf0000!__RtlUserThreadStart
ntdll_77cf0000!_RtlUserThreadStart
1: kd:x86> .thread fffffa8001b00660
Implicit thread is now fffffa80`01b00660
1: kd:x86> kc
*** Stack trace for last set context - .thread/.cxr resets it
WARNING: Frame IP not in any known module. Following frames may be wrong.
0x0
1: kd:x86> .effmach #
Effective machine: x64 (AMD64)
1: kd> kc
^ Current scope machine type mismatch error in 'kc'
1: kd> .thread fffffa8001b00660
Implicit thread is now fffffa80`01b00660
1: kd> kc
*** Stack trace for last set context - .thread/.cxr resets it
Call Site
nt!KiSwapContext
nt!KiCommitThreadWait
nt!KeRemoveQueueEx
nt!IoRemoveIoCompletion
nt!NtRemoveIoCompletion
nt!KiSystemServiceCopyEnd
wow64cpu!CpupSyscallStub
wow64cpu!RemoveIoCompletionFault
wow64!RunCpuSimulation
wow64!Wow64LdrpInitialize
ntdll!_LdrpInitialize
ntdll!LdrInitializeThunk
So, the magic incantation to get back from .thread /w appears to be:
.effmach #
.thread <thread>
Is that correct?
Thanks!
-scott
"Pavel Lebedynskiy" <[email protected]> wrote in message
news:[email protected]
> Try .thread /w.
>
> Thanks,
> Pavel
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Scott Noone
> Sent: Wednesday, August 25, 2010 11:34 AM
> To: Kernel Debugging Interest List
> Subject: Re:[windbg] wow64exts, or, where are the 32 bit stacks
>
> OK, there's certainly something wrong here. I'm not sure if this is a bug
> in the current implementation or a limitation. I'll demonstrate what I'm
> seeing on a live dump as it ends better than the crash dump version
>
> I force a context switch to a 32bit process in WinDBG via .process /i, in
> this case I picked 32bit IE:
>
> 0: kd> !process -1 0
> PROCESS fffffa8002455340
> SessionId: 1 Cid: 0aa0 Peb: 7efdf000 ParentCid: 0444
> DirBase: 1a2a3000 ObjectTable: fffff8a002e36e40 HandleCount: 327.
> Image: iexplore.exe
>
> I then switch to a thread in the process and attempt to do !wow64exts.k,
> which meets the same fate as yours:
>
> 0: kd> !wow64exts.k
> Walking 64bit Stack...
> *** Stack trace for last set context - .thread/.cxr resets it
> Child-SP RetAddr Call Site
> fffff880`0473a660 fffff800`026c2052 nt!KiSwapContext+0x7a
> fffff880`0473a7a0 fffff800`026c41af nt!KiCommitThreadWait+0x1d2
> fffff880`0473a830 fffff800`026d94ef nt!KeWaitForSingleObject+0x19f
> fffff880`0473a8d0 fffff800`029be736 nt!AlpcpSignalAndWait+0x8f
> fffff880`0473a980 fffff800`029bc6c0 nt!AlpcpReceiveSynchronousReply+0x46
> fffff880`0473a9e0 fffff800`029b9c3b
> nt!AlpcpProcessSynchronousRequest+0x33d
> fffff880`0473ab00 fffff800`026ba153 nt!NtAlpcSendWaitReceivePort+0x1ab
> fffff880`0473abb0 00000000`77b6070a nt!KiSystemServiceCopyEnd+0x13
> 00000000`0371e758 00000000`7581e2f0 ntdll!ZwAlpcSendWaitReceivePort+0xa
> 00000000`0371e760 00000000`7580cf87 wow64!whNtAlpcSendWaitReceivePort+0x64
> 00000000`0371e7c0 00000000`7579276d wow64!Wow64SystemServiceEx+0xd7
> 00000000`0371f080 00000000`7580d07e
> wow64cpu!TurboDispatchJumpAddressEnd+0x24
> 00000000`0371f140 00000000`7580c549 wow64!RunCpuSimulation+0xa
> 00000000`0371f190 00000000`77b8d177 wow64!Wow64LdrpInitialize+0x429
> 00000000`0371f6e0 00000000`77b4308e ntdll! ?? ::FNODOBFM::`string'+0x2bfe4
> 00000000`0371f750 00000000`00000000 ntdll!LdrInitializeThunk+0xe Walking
> 32bit Stack...
> ^ Current scope machine type mismatch error in 'k '
> !wow64exts.k: Execute failed: 0x80040205
>
-scott
OSR
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Scott Noone
Sent: Friday, August 27, 2010 7:38 AM
To: Kernel Debugging Interest List
Subject: Re:[windbg] Re:wow64exts, or, where are the 32 bit stacks
So, the magic incantation to get back from .thread /w appears to be:
.effmach #
.thread <thread>
Is that correct?
day...
In the end, I guess it would be nice if !wow64exts.k was updated to hide all
of this from us.
-scott
--
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com
"Pavel Lebedynskiy" <[email protected]> wrote in message
news:[email protected]
> I think that's correct, but I'm not sure. It always takes me a few tries
> to get this to work.
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Scott Noone
> Sent: Friday, August 27, 2010 7:38 AM
> To: Kernel Debugging Interest List
> Subject: Re:[windbg] Re:wow64exts, or, where are the 32 bit stacks
>
> So, the magic incantation to get back from .thread /w appears to be:
>
> .effmach #
> .thread <thread>
>
> Is that correct?
>
-scott
OSR
kd> .thread blah
Implicit thread is now blah
kd> .thread /w blah
Implicit thread is now blah
x86 context set
... .and lo! a 32 bit call stack!
(Having previously
kd>.process /p /r blahblahprocaddress )
That's brilliant - now I know how to coax the information out... oh happy day! Thanks Pavel.
I know this an old thread but this problem still happens all the time. When .thread /w doesn't work, I do the following:
kd> .thread ETHREAD
kd> !teb
kd> dds StackLimit StackBase
The stack values are plucked from the TEB32 section.