Hmmm, well those ideas make sense, but this always happens, it isn’t a part time thing. As I step through a function the stack changes, sometimes displaying more or less levels.
How do I make sure symbol load notification is on?
Also if I do this:
Kd> k
Child-SP RetAddr Call Site
000000000b29d690 00000000
02a0f360 mydll!myfunction+0x70 [my.cpp @ 128]
000000000b29d698 00000000
00000000 0x2a0f360
kd> u 2a0f360
0000000002a0f360 0000 add byte ptr [rax],al 00000000
02a0f362 0000 add byte ptr [rax],al
0000000002a0f364 0000 add byte ptr [rax],al 00000000
02a0f366 0000 add byte ptr [rax],al
It’s pretty obvious that isn’t the actual return address. It looks like the PDBs aren’t matching the stack properly, but they are the correct PDBs.
It’s also worth pointing out my host machine is x86 hosting an x64 VM. So I am running Windbg x86.
-Jeff
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Skywing
Sent: Monday, June 22, 2009 4:22 PM
To: Kernel Debugging Interest List
Subject: RE:[windbg] Weird stack
You should just use k instead of kv on amd64 as that is completely unreadable. (Especially on a handheld screen!)
That being said, my guess is that you have an intervening frame for which unwind information couldn’t be pulled.
Unwinding on amd64 requires reading unwind metadata for each frame. This can be problematic in kd because of things becoming paged out.
- If part of the loaded module list is paged out, or you did not load user mode symbols beforehand, then the debugger won’t have a binary for a given frame’s RIP and unwinding will break* because no unwind metadata can be found.
- If symbols cannot be loaded and the unwind metadata for a binary is paged out (not uncommon), then unwinding will break* because no unwind metadata can be found.
- If you don’t have symbol load notifications enabled (or otherwise weren’t there to get the notification at the time a particular binary was loaded), and the image headers are paged out, then the debugger won’t be able to get unwind metadata and unwinding will break*.
- If the caller was JIT’d code, then unwinding will break* in kernel mode as the kernel debugger doesn’t support loading the JIT unwind helper DLL (AFAIK).
*: I believe the debugger will treat these scenarios according to the specification for a leaf function on amd64. This is going to be incorrect for a function that isn’t a leaf function.
In your case, we see mydll!myfunction1 returning to 00000000002ad368 (your first example), or mydll!myfunc3 00000000
0750d9e8 (your second example), neither of which the debugger has associated with a loaded module. That means the debugger probably gets lost at those respective frames because it can’t access the unwind table for those functions.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jeff Curless
Sent: Monday, June 22, 2009 12:56 PM
To: Kernel Debugging Interest List
Subject: RE:[windbg] Weird stack
Here is more. It appears to definitely be something to do with my PDBs as the Microsoft ones from the symbol server look correct…
Child-SP RetAddr : Args to Child : Call Site
00 000000000750d9b0 00000000
0750d9e8 : 000000000750d9e8 00000000
0750da78 000000000750d9d8 cccccccc
cccccccc : mydll!myfunc3
01 000000000750d9b8 00000000
0750d9e8 : 000000000750da78 00000000
0750d9d8 cccccccccccccccc 00000000
07ab0d28 : 0x750d9e8
02 000000000750d9c0 00000000
0750da78 : 000000000750d9d8 cccccccc
cccccccc 0000000007ab0d28 cccccc00
cccccccc : 0x750d9e8
03 000000000750d9c8 00000000
0750d9d8 : cccccccccccccccc 00000000
07ab0d28 cccccc00cccccccc 00000001
802b0268 : 0x750da78
04 000000000750d9d0 cccccccc
cccccccc : 0000000007ab0d28 cccccc00
cccccccc 00000001802b0268 cccccccc
cccccccc : 0x750d9d8
05 000000000750d9d8 00000000
07ab0d28 : cccccc00cccccccc 00000001
802b0268 cccccccccccccccc cccccccc
cccccccc : 0xcccccccccccccccc 06 00000000
0750d9e0 cccccc00cccccccc : 00000001
802b0268 cccccccccccccccc cccccccc
cccccccc cccccccccccccccc : 0x7ab0d28 07 00000000
0750d9e8 00000001802b0268 : cccccccc
cccccccc cccccccccccccccc cccccccc
cccccccc fffffffffffffffe : 0xcccccc00
cccccccc
08 000000000750d9f0 cccccccc
cccccccc : cccccccccccccccc cccccccc
cccccccc fffffffffffffffe 00000000
0750d9e8 : mydll!myfunc2
09 000000000750d9f8 cccccccc
cccccccc : cccccccccccccccc ffffffff
fffffffe 000000000750d9e8 00000000
0750d9e8 : 0xcccccccccccccccc 0a 00000000
0750da00 cccccccccccccccc : ffffffff
fffffffe 000000000750d9e8 00000000
0750d9e8 cccccccc00000000 : 0xcccccccc
cccccccc
0b 000000000750da08 ffffffff
fffffffe : 000000000750d9e8 00000000
0750d9e8 cccccccc00000000 cccccccc
cccccccc : 0xcccccccccccccccc 0c 00000000
0750da10 000000000750d9e8 : 00000000
0750d9e8 cccccccc00000000 cccccccc
cccccccc cccccccccccccccc : 0xffffffff
fffffffe
0d 000000000750da18 00000000
0750d9e8 : cccccccc00000000 cccccccc
cccccccc cccccccccccccccc cccccccc
cccccccc : 0x750d9e8
0e 000000000750da20 cccccccc
00000000 : cccccccccccccccc cccccccc
cccccccc cccccccccccccccc 00000000
0750dac0 : 0x750d9e8
0f 000000000750da28 cccccccc
cccccccc : cccccccccccccccc cccccccc
cccccccc 000000000750dac0 00000001
800247f0 : 0xcccccccc00000000 10 00000000
0750da30 cccccccccccccccc : cccccccc
cccccccc 000000000750dac0 00000001
800247f0 0000000000f6fd60 : 0xcccccccc
cccccccc
11 000000000750da38 cccccccc
cccccccc : 000000000750dac0 00000001
800247f0 0000000000f6fd60 00000000
0750da78 : 0xcccccccccccccccc 12 00000000
0750da40 000000000750dac0 : 00000001
800247f0 0000000000f6fd60 00000000
0750da78 cccccccccccccccc : 0xcccccccc
cccccccc
13 000000000750da48 00000001
800247f0 : 0000000000f6fd60 00000000
0750da78 cccccccccccccccc cccccccc
cccccccc : 0x750dac0
14 000000000750da50 00000000
00f6fd60 : 000000000750da78 cccccccc
cccccccc cccccccccccccccc cccccccc
cccccccc : mydll!myfunc1
15 000000000750da58 00000000
0750da78 : cccccccccccccccc cccccccc
cccccccc cccccccccccccccc 00000000
07ab09a8 : 0xf6fd60
16 000000000750da60 cccccccc
cccccccc : cccccccccccccccc cccccccc
cccccccc 0000000007ab09a8 cccccccc
cccccccc : 0x750da78
17 000000000750da68 cccccccc
cccccccc : cccccccccccccccc 00000000
07ab09a8 cccccccccccccccc cccccccc
cccccccc : 0xcccccccccccccccc 18 00000000
0750da70 cccccccccccccccc : 00000000
07ab09a8 cccccccccccccccc cccccccc
cccccccc cccccccccccccccc : 0xcccccccc
cccccccc
19 000000000750da78 00000000
07ab09a8 : cccccccccccccccc cccccccc
cccccccc cccccccccccccccc cccccccc
cccccccc : 0xcccccccccccccccc 1a 00000000
0750da80 cccccccccccccccc : cccccccc
cccccccc cccccccccccccccc cccccccc
cccccccc cccccccccccccccc : 0x7ab09a8 1b 00000000
0750da88 cccccccccccccccc : cccccccc
cccccccc cccccccccccccccc cccccccc
cccccccc fffffffffffffffe : 0xcccccccc
cccccccc
1c 000000000750da90 cccccccc
cccccccc : cccccccccccccccc cccccccc
cccccccc fffffffffffffffe cccccccc
cccccccc : 0xcccccccccccccccc 1d 00000000
0750da98 cccccccccccccccc : cccccccc
cccccccc fffffffffffffffe cccccccc
cccccccc cccccccccccccccc : 0xcccccccc
cccccccc
1e 000000000750daa0 cccccccc
cccccccc : fffffffffffffffe cccccccc
cccccccc cccccccccccccccc 00000000
07a99680 : 0xcccccccccccccccc 1f 00000000
0750daa8 fffffffffffffffe : cccccccc
cccccccc cccccccccccccccc 00000000
07a99680 000007fefdaf528f : 0xcccccccc
cccccccc
20 000000000750dab0 cccccccc
cccccccc : cccccccccccccccc 00000000
07a99680 000007fefdaf528f 00000000
00f6b760 : 0xfffffffffffffffe 21 00000000
0750dab8 cccccccccccccccc : 00000000
07a99680 000007fefdaf528f 00000000
00f6b760 0000000005e9b827 : 0xcccccccc
cccccccc
22 000000000750dac0 00000000
07a99680 : 000007fefdaf528f 00000000
00f6b760 0000000005e9b827 00000000
052f62b4 : 0xcccccccccccccccc 23 00000000
0750dac8 000007fefdaf528f : 00000000
00f6b760 0000000005e9b827 00000000
052f62b4 0000000000000400 : 0x7a99680 24 00000000
0750dad0 000007fefdaf54d0 : 00000000
0750df90 000007fe05e9b827 00000000
052f62b4 000000000750de80 : OLEAUT32!IDispatch_Invoke_Stub+0xcf 25 00000000
0750db70 000007fefdbd599c : 000007fe
fdb40082 000000000750df90 000007fe
fdb43a50 000007fefdb426d8 : OLEAUT32!IDispatch_RemoteInvoke_Thunk+0x60 26 00000000
0750dbe0 000007fefdc64442 : 00000000
00000000 0000000007ab0798 00000000
07a96200 000007fefdb625c0 : RPCRT4!NdrStubCall2+0xc0b 27 00000000
0750e1e0 000007fefdafb24e : 00000000
00000001 0000000000000000 00000000
07ab07a8 0000000007ab0770 : RPCRT4!CStdStubBuffer_Invoke+0x9a 28 00000000
0750e210 000007fefda18809 : 00000000
00000000 0000000000000000 00000000
00105f40 00000000001107a0 : OLEAUT32!CStubWrapper::Invoke+0x9e 29 00000000
0750e240 000007fefda1877f : 00000000
0010f8a0 0000000000117534 00000000
001107a0 00000001801f7e48 : ole32!SyncStubInvoke+0x5d 2a 00000000
0750e2b0 000007fefd8e85ab : 00000000
0010f8a0 0000000005334ca0 00000000
07a0d720 0000000000000001 : ole32!StubInvoke+0xdb 2b 00000000
0750e360 000007fefd8e9bf5 : cccccccc
00000000 0000000000117534 00000000
07ab0770 000000000010f8a0 : ole32!CCtxComChnl::ContextInvoke+0x19f 2c 00000000
0750e4e0 000007fefda18a83 : 00000000
05334ca0 0000000000000000 00000000
07a0d720 0000000000000000 : ole32!STAInvoke+0x91 2d 00000000
0750e530 000007fefda183b7 : 00000000
d0908070 0000000005334ca0 00000000
0010fa30 00000000052f62b0 : ole32!AppInvoke+0x193 2e 00000000
0750e5a0 000007fefda18b15 : 00000000
0010f810 0000000000000400 00000000
00000000 0000000000000000 : ole32!ComInvokeWithLockAndIPID+0x407 2f 00000000
0750e720 000007fefd8e9ad9 : 00000000
00105f40 0000000000000000 00000000
000ff008 000000000010f810 : ole32!ComInvoke+0x85 30 00000000
0750e750 000007fefd8e9982 : 00000000
05334ca0 000000000010f818 00000000
00000400 0000000000000000 : ole32!ThreadDispatch+0x29 31 00000000
0750e780 0000000076f5d24a : 00000000
00000624 00000000000006cc 00000000
00000000 0000000007a4fea0 : ole32!ThreadWndProc+0xaa 32 00000000
0750e800 0000000076f5d39e : 00000000
0750e980 000007fefd8e98d8 00000000
00000000 0000000001068ae0 : USER32!UserCallWinProcCheckWow+0x1ad 33 00000000
0750e8c0 000007fef67e7a79 : 00000000
052e9500 00000000052e9500 000007fe
fd8e98d8 0000000000000100 : USER32!DispatchMessageWorker+0x389 34 00000000
0750e940 000007fef68805ee : 00000000
000f2a60 0000000000000000 00000000
00000000 0000000003440880 : IEFRAME!CBrowserFrame::FrameMessagePump+0x411 35 00000000
0750ea00 000007fef685bd65 : 00000000
00000000 0000000000062801 00000000
052e9500 0000000000000000 : IEFRAME!BrowserThreadProc+0x14d 36 00000000
0750eaa0 000007fef685bdcf : 10127334
00000001 0000000000000000 00000000
00000000 0000000000000000 : IEFRAME!BrowserNewThreadProc+0x98 37 00000000
0750eae0 0000000076e3495d : 00000000
00000000 0000000000000000 00000000
00000000 0000000000000000 : IEFRAME!SHOpenFolderWindow+0x1df 38 00000000
0750fb90 0000000077038791 : 00000000
00000000 0000000000000000 00000000
00000000 0000000000000000 : kernel32!BaseThreadInitThunk+0xd 39 00000000
0750fbc0 0000000000000000 : 00000000
00000000 0000000000000000 00000000
00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d
Thanks,
-Jeff
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jeff Curless
Sent: Monday, June 22, 2009 3:47 PM
To: Kernel Debugging Interest List
Subject: [windbg] Weird stack
When I am kernel debugging on x64 and I am looking at a user-mode stack I get a lot of funky output:
RetAddr : Args to Child : Call Site
00000001800247f0 : 00000000
002ad368 cccccccccccccccc cccccccc
cccccccc cccccccccccccccc : mydll!myfunction2 00000000
002ad368 : cccccccccccccccc cccccccc
cccccccc cccccccccccccccc cccccccc
cccccccc : mydll!myfunction1
cccccccccccccccc : cccccccc
cccccccc cccccccccccccccc cccccccc
cccccccc 0000000000000000 : 0x2ad368 cccccccc
cccccccc : cccccccccccccccc cccccccc
cccccccc 0000000000000000 cccccccc
cccccccc : 0xcccccccccccccccc cccccccc
cccccccc : cccccccccccccccc 00000000
00000000 cccccccccccccccc cccccccc
cccccccc : 0xcccccccccccccccc cccccccc
cccccccc : 0000000000000000 cccccccc
cccccccc cccccccccccccccc cccccccc
cccccccc : 0xcccccccccccccccc 00000000
00000000 : cccccccccccccccc cccccccc
cccccccc cccccccccccccccc cccccccc
cccccccc : 0xcccccccc`cccccccc
This is a debug build of user-mode code, with matching PDBs. Has anyone else seen this? It makes debugging a lot more of a pain to say the least…
Newest WinDbg, PDBs built with newest VS2008, newest service pack. VMWare VM running Windows Vista x64, SP1.
Thanks!
-Jeff
WINDBG is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
WINDBG is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
WINDBG is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer