Hmmm, well those ideas make sense, but this always happens, it isn’t a part time thing. As I step through a function the stack changes, sometimes displaying more or less levels.
How do I make sure symbol load notification is on?
Also if I do this:
Kd> k
Child-SP RetAddr Call Site
000000000b29d690 0000000002a0f360 mydll!myfunction+0x70 [my.cpp @ 128]
000000000b29d698 0000000000000000 0x2a0f360
kd> u 2a0f360
0000000002a0f360 0000 add byte ptr [rax],al 0000000002a0f362 0000 add byte ptr [rax],al
0000000002a0f364 0000 add byte ptr [rax],al 0000000002a0f366 0000 add byte ptr [rax],al
It’s pretty obvious that isn’t the actual return address. It looks like the PDBs aren’t matching the stack properly, but they are the correct PDBs.
It’s also worth pointing out my host machine is x86 hosting an x64 VM. So I am running Windbg x86.
-Jeff
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Skywing
Sent: Monday, June 22, 2009 4:22 PM
To: Kernel Debugging Interest List
Subject: RE:[windbg] Weird stack
You should just use k instead of kv on amd64 as that is completely unreadable. 
(Especially on a handheld screen!)
That being said, my guess is that you have an intervening frame for which unwind information couldn’t be pulled.
Unwinding on amd64 requires reading unwind metadata for each frame. This can be problematic in kd because of things becoming paged out.
- If part of the loaded module list is paged out, or you did not load user mode symbols beforehand, then the debugger won’t have a binary for a given frame’s RIP and unwinding will break* because no unwind metadata can be found.
- If symbols cannot be loaded and the unwind metadata for a binary is paged out (not uncommon), then unwinding will break* because no unwind metadata can be found.
- If you don’t have symbol load notifications enabled (or otherwise weren’t there to get the notification at the time a particular binary was loaded), and the image headers are paged out, then the debugger won’t be able to get unwind metadata and unwinding will break*.
- If the caller was JIT’d code, then unwinding will break* in kernel mode as the kernel debugger doesn’t support loading the JIT unwind helper DLL (AFAIK).
*: I believe the debugger will treat these scenarios according to the specification for a leaf function on amd64. This is going to be incorrect for a function that isn’t a leaf function.
In your case, we see mydll!myfunction1 returning to 00000000002ad368 (your first example), or mydll!myfunc3 000000000750d9e8 (your second example), neither of which the debugger has associated with a loaded module. That means the debugger probably gets lost at those respective frames because it can’t access the unwind table for those functions.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jeff Curless
Sent: Monday, June 22, 2009 12:56 PM
To: Kernel Debugging Interest List
Subject: RE:[windbg] Weird stack
Here is more. It appears to definitely be something to do with my PDBs as the Microsoft ones from the symbol server look correct…
Child-SP RetAddr : Args to Child : Call Site
00 000000000750d9b0 000000000750d9e8 : 000000000750d9e8 000000000750da78 000000000750d9d8 cccccccccccccccc : mydll!myfunc3
01 000000000750d9b8 000000000750d9e8 : 000000000750da78 000000000750d9d8 cccccccccccccccc 0000000007ab0d28 : 0x750d9e8
02 000000000750d9c0 000000000750da78 : 000000000750d9d8 cccccccccccccccc 0000000007ab0d28 cccccc00cccccccc : 0x750d9e8
03 000000000750d9c8 000000000750d9d8 : cccccccccccccccc 0000000007ab0d28 cccccc00cccccccc 00000001802b0268 : 0x750da78
04 000000000750d9d0 cccccccccccccccc : 0000000007ab0d28 cccccc00cccccccc 00000001802b0268 cccccccccccccccc : 0x750d9d8
05 000000000750d9d8 0000000007ab0d28 : cccccc00cccccccc 00000001802b0268 cccccccccccccccc cccccccccccccccc : 0xcccccccccccccccc 06 000000000750d9e0 cccccc00cccccccc : 00000001802b0268 cccccccccccccccc cccccccccccccccc cccccccccccccccc : 0x7ab0d28 07 000000000750d9e8 00000001802b0268 : cccccccccccccccc cccccccccccccccc cccccccccccccccc fffffffffffffffe : 0xcccccc00cccccccc
08 000000000750d9f0 cccccccccccccccc : cccccccccccccccc cccccccccccccccc fffffffffffffffe 000000000750d9e8 : mydll!myfunc2
09 000000000750d9f8 cccccccccccccccc : cccccccccccccccc fffffffffffffffe 000000000750d9e8 000000000750d9e8 : 0xcccccccccccccccc 0a 000000000750da00 cccccccccccccccc : fffffffffffffffe 000000000750d9e8 000000000750d9e8 cccccccc00000000 : 0xcccccccccccccccc
0b 000000000750da08 fffffffffffffffe : 000000000750d9e8 000000000750d9e8 cccccccc00000000 cccccccccccccccc : 0xcccccccccccccccc 0c 000000000750da10 000000000750d9e8 : 000000000750d9e8 cccccccc00000000 cccccccccccccccc cccccccccccccccc : 0xfffffffffffffffe
0d 000000000750da18 000000000750d9e8 : cccccccc00000000 cccccccccccccccc cccccccccccccccc cccccccccccccccc : 0x750d9e8
0e 000000000750da20 cccccccc00000000 : cccccccccccccccc cccccccccccccccc cccccccccccccccc 000000000750dac0 : 0x750d9e8
0f 000000000750da28 cccccccccccccccc : cccccccccccccccc cccccccccccccccc 000000000750dac0 00000001800247f0 : 0xcccccccc00000000 10 000000000750da30 cccccccccccccccc : cccccccccccccccc 000000000750dac0 00000001800247f0 0000000000f6fd60 : 0xcccccccccccccccc
11 000000000750da38 cccccccccccccccc : 000000000750dac0 00000001800247f0 0000000000f6fd60 000000000750da78 : 0xcccccccccccccccc 12 000000000750da40 000000000750dac0 : 00000001800247f0 0000000000f6fd60 000000000750da78 cccccccccccccccc : 0xcccccccccccccccc
13 000000000750da48 00000001800247f0 : 0000000000f6fd60 000000000750da78 cccccccccccccccc cccccccccccccccc : 0x750dac0
14 000000000750da50 0000000000f6fd60 : 000000000750da78 cccccccccccccccc cccccccccccccccc cccccccccccccccc : mydll!myfunc1
15 000000000750da58 000000000750da78 : cccccccccccccccc cccccccccccccccc cccccccccccccccc 0000000007ab09a8 : 0xf6fd60
16 000000000750da60 cccccccccccccccc : cccccccccccccccc cccccccccccccccc 0000000007ab09a8 cccccccccccccccc : 0x750da78
17 000000000750da68 cccccccccccccccc : cccccccccccccccc 0000000007ab09a8 cccccccccccccccc cccccccccccccccc : 0xcccccccccccccccc 18 000000000750da70 cccccccccccccccc : 0000000007ab09a8 cccccccccccccccc cccccccccccccccc cccccccccccccccc : 0xcccccccccccccccc
19 000000000750da78 0000000007ab09a8 : cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc : 0xcccccccccccccccc 1a 000000000750da80 cccccccccccccccc : cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc : 0x7ab09a8 1b 000000000750da88 cccccccccccccccc : cccccccccccccccc cccccccccccccccc cccccccccccccccc fffffffffffffffe : 0xcccccccccccccccc
1c 000000000750da90 cccccccccccccccc : cccccccccccccccc cccccccccccccccc fffffffffffffffe cccccccccccccccc : 0xcccccccccccccccc 1d 000000000750da98 cccccccccccccccc : cccccccccccccccc fffffffffffffffe cccccccccccccccc cccccccccccccccc : 0xcccccccccccccccc
1e 000000000750daa0 cccccccccccccccc : fffffffffffffffe cccccccccccccccc cccccccccccccccc 0000000007a99680 : 0xcccccccccccccccc 1f 000000000750daa8 fffffffffffffffe : cccccccccccccccc cccccccccccccccc 0000000007a99680 000007fefdaf528f : 0xcccccccccccccccc
20 000000000750dab0 cccccccccccccccc : cccccccccccccccc 0000000007a99680 000007fefdaf528f 0000000000f6b760 : 0xfffffffffffffffe 21 000000000750dab8 cccccccccccccccc : 0000000007a99680 000007fefdaf528f 0000000000f6b760 0000000005e9b827 : 0xcccccccccccccccc
22 000000000750dac0 0000000007a99680 : 000007fefdaf528f 0000000000f6b760 0000000005e9b827 00000000052f62b4 : 0xcccccccccccccccc 23 000000000750dac8 000007fefdaf528f : 0000000000f6b760 0000000005e9b827 00000000052f62b4 0000000000000400 : 0x7a99680 24 000000000750dad0 000007fefdaf54d0 : 000000000750df90 000007fe05e9b827 00000000052f62b4 000000000750de80 : OLEAUT32!IDispatch_Invoke_Stub+0xcf 25 000000000750db70 000007fefdbd599c : 000007fefdb40082 000000000750df90 000007fefdb43a50 000007fefdb426d8 : OLEAUT32!IDispatch_RemoteInvoke_Thunk+0x60 26 000000000750dbe0 000007fefdc64442 : 0000000000000000 0000000007ab0798 0000000007a96200 000007fefdb625c0 : RPCRT4!NdrStubCall2+0xc0b 27 000000000750e1e0 000007fefdafb24e : 0000000000000001 0000000000000000 0000000007ab07a8 0000000007ab0770 : RPCRT4!CStdStubBuffer_Invoke+0x9a 28 000000000750e210 000007fefda18809 : 0000000000000000 0000000000000000 0000000000105f40 00000000001107a0 : OLEAUT32!CStubWrapper::Invoke+0x9e 29 000000000750e240 000007fefda1877f : 000000000010f8a0 0000000000117534 00000000001107a0 00000001801f7e48 : ole32!SyncStubInvoke+0x5d 2a 000000000750e2b0 000007fefd8e85ab : 000000000010f8a0 0000000005334ca0 0000000007a0d720 0000000000000001 : ole32!StubInvoke+0xdb 2b 000000000750e360 000007fefd8e9bf5 : cccccccc00000000 0000000000117534 0000000007ab0770 000000000010f8a0 : ole32!CCtxComChnl::ContextInvoke+0x19f 2c 000000000750e4e0 000007fefda18a83 : 0000000005334ca0 0000000000000000 0000000007a0d720 0000000000000000 : ole32!STAInvoke+0x91 2d 000000000750e530 000007fefda183b7 : 00000000d0908070 0000000005334ca0 000000000010fa30 00000000052f62b0 : ole32!AppInvoke+0x193 2e 000000000750e5a0 000007fefda18b15 : 000000000010f810 0000000000000400 0000000000000000 0000000000000000 : ole32!ComInvokeWithLockAndIPID+0x407 2f 000000000750e720 000007fefd8e9ad9 : 0000000000105f40 0000000000000000 00000000000ff008 000000000010f810 : ole32!ComInvoke+0x85 30 000000000750e750 000007fefd8e9982 : 0000000005334ca0 000000000010f818 0000000000000400 0000000000000000 : ole32!ThreadDispatch+0x29 31 000000000750e780 0000000076f5d24a : 0000000000000624 00000000000006cc 0000000000000000 0000000007a4fea0 : ole32!ThreadWndProc+0xaa 32 000000000750e800 0000000076f5d39e : 000000000750e980 000007fefd8e98d8 0000000000000000 0000000001068ae0 : USER32!UserCallWinProcCheckWow+0x1ad 33 000000000750e8c0 000007fef67e7a79 : 00000000052e9500 00000000052e9500 000007fefd8e98d8 0000000000000100 : USER32!DispatchMessageWorker+0x389 34 000000000750e940 000007fef68805ee : 00000000000f2a60 0000000000000000 0000000000000000 0000000003440880 : IEFRAME!CBrowserFrame::FrameMessagePump+0x411 35 000000000750ea00 000007fef685bd65 : 0000000000000000 0000000000062801 00000000052e9500 0000000000000000 : IEFRAME!BrowserThreadProc+0x14d 36 000000000750eaa0 000007fef685bdcf : 1012733400000001 0000000000000000 0000000000000000 0000000000000000 : IEFRAME!BrowserNewThreadProc+0x98 37 000000000750eae0 0000000076e3495d : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : IEFRAME!SHOpenFolderWindow+0x1df 38 000000000750fb90 0000000077038791 : 0000000000000000 0000000000000000 0000000000000000 0000000000000000 : kernel32!BaseThreadInitThunk+0xd 39 000000000750fbc0 0000000000000000 : 0000000000000000 0000000000000000 0000000000000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d
Thanks,
-Jeff
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jeff Curless
Sent: Monday, June 22, 2009 3:47 PM
To: Kernel Debugging Interest List
Subject: [windbg] Weird stack
When I am kernel debugging on x64 and I am looking at a user-mode stack I get a lot of funky output:
RetAddr : Args to Child : Call Site
00000001800247f0 : 00000000002ad368 cccccccccccccccc cccccccccccccccc cccccccccccccccc : mydll!myfunction2 00000000002ad368 : cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc : mydll!myfunction1
cccccccccccccccc : cccccccccccccccc cccccccccccccccc cccccccccccccccc 0000000000000000 : 0x2ad368 cccccccccccccccc : cccccccccccccccc cccccccccccccccc 0000000000000000 cccccccccccccccc : 0xcccccccccccccccc cccccccccccccccc : cccccccccccccccc 0000000000000000 cccccccccccccccc cccccccccccccccc : 0xcccccccccccccccc cccccccccccccccc : 0000000000000000 cccccccccccccccc cccccccccccccccc cccccccccccccccc : 0xcccccccccccccccc 0000000000000000 : cccccccccccccccc cccccccccccccccc cccccccccccccccc cccccccccccccccc : 0xcccccccc`cccccccc
This is a debug build of user-mode code, with matching PDBs. Has anyone else seen this? It makes debugging a lot more of a pain to say the least…
Newest WinDbg, PDBs built with newest VS2008, newest service pack. VMWare VM running Windows Vista x64, SP1.
Thanks!
-Jeff
WINDBG is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
WINDBG is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
WINDBG is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
