Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.More Info on Driver Writing and Debugging
The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.
Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/
COM <-> named pipe debugging in vmware workstation
OSR_Community_User
Member Posts: 110,217
in WINDBG
I successfully connect a target macine running inside VirtualPC to
a host-based windbg through a named pipe.
Now I try to do the same on a VMWare workstation 5.0.
I redirect virtual's COM1 or COM2 to \\.\pipe\dbgpipe which is
what I use for VPC but no matter what I do with other VMWare's
COM port settings (client vs server, app or VM on the other end,
yield on poll), host-based windbg does not see the target
[VMWare virtual] machine but is happy when the target runs
inside a VPC.
boot.ini's are identical on both virtual machines, with records like
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="win + COM1 +
ntbtlog.txt" /fastdetect /debug /debugport=com1
/baudrate=115200 /bootlog
VMWare on-line help brought nothing new. My guess is that
it should work, what can be possibly wrong with a named pipe?
Did anyone have this problem and solved it (or did not have this
VMWare-specific problem at all, for that matter)?
Maybe, it's some security policy setting that I'm missing?
Any hints would be greatly appreciated.
Regards,
Alex Shvedov
a host-based windbg through a named pipe.
Now I try to do the same on a VMWare workstation 5.0.
I redirect virtual's COM1 or COM2 to \\.\pipe\dbgpipe which is
what I use for VPC but no matter what I do with other VMWare's
COM port settings (client vs server, app or VM on the other end,
yield on poll), host-based windbg does not see the target
[VMWare virtual] machine but is happy when the target runs
inside a VPC.
boot.ini's are identical on both virtual machines, with records like
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="win + COM1 +
ntbtlog.txt" /fastdetect /debug /debugport=com1
/baudrate=115200 /bootlog
VMWare on-line help brought nothing new. My guess is that
it should work, what can be possibly wrong with a named pipe?
Did anyone have this problem and solved it (or did not have this
VMWare-specific problem at all, for that matter)?
Maybe, it's some security policy setting that I'm missing?
Any hints would be greatly appreciated.
Regards,
Alex Shvedov
Howdy, Stranger!
| Upcoming OSR Seminars | ||
|---|---|---|
| OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead! | ||
| Kernel Debugging | 13-17 May 2024 | Live, Online |
| Developing Minifilters | 1-5 Apr 2024 | Live, Online |
| Internals & Software Drivers | 11-15 Mar 2024 | Live, Online |
| Writing WDF Drivers | 26 Feb - 1 Mar 2024 | Live, Online |
Comments
the virtual COM port in the virtual machine to be a named pipe, as you did.
Then I opened WInDbg and specified to connect via a named pipe, the only way
I managed this was by running WinDbg from a command line specifying the
following:
start windbg -W Pipe -b -k com:pipe,port=\\.\pipe\dbgpipe,resets=0
Note that this will also break on entry into the system, I do this to ensure
connectivity. It also uses the workspace profile I created called Pipe,
which contains the symbol paths etc.
I have encountered an issue with VPC that requires me to restart the
debugger whenever I restart the virtual machine. This is not always the case
with VMWare but it does happen.
Pete
Kernel Drivers
Windows Filesystem and Device Driver Consulting
www.KernelDrivers.com
(303)546-0300
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of xxxxx@Home
Sent: Tuesday, July 19, 2005 1:25 PM
To: Kernel Debugging Interest List
Subject: [windbg] COM <-> named pipe debugging in vmware workstation
I successfully connect a target macine running inside VirtualPC to
a host-based windbg through a named pipe.
Now I try to do the same on a VMWare workstation 5.0.
I redirect virtual's COM1 or COM2 to \\.\pipe\dbgpipe which is
what I use for VPC but no matter what I do with other VMWare's
COM port settings (client vs server, app or VM on the other end,
yield on poll), host-based windbg does not see the target
[VMWare virtual] machine but is happy when the target runs
inside a VPC.
boot.ini's are identical on both virtual machines, with records like
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="win + COM1 +
ntbtlog.txt" /fastdetect /debug /debugport=com1
/baudrate=115200 /bootlog
VMWare on-line help brought nothing new. My guess is that
it should work, what can be possibly wrong with a named pipe?
Did anyone have this problem and solved it (or did not have this
VMWare-specific problem at all, for that matter)?
Maybe, it's some security policy setting that I'm missing?
Any hints would be greatly appreciated.
Regards,
Alex Shvedov
---
You are currently subscribed to windbg as: [email protected]
To unsubscribe send a blank email to [email protected]
* On Tue, Jul 19, 2005 at 03:25:09PM -0600 Peter Scott wrote:
> I have encountered an issue with VPC that requires me to restart the
> debugger whenever I restart the virtual machine. This is not always
> the case with VMWare but it does happen.
With VMWare, I have the following issue: When I boot the virtual machine
(XP), the debugger is not allowed to be started (or the VM will not boot
at all). I have to wait until after the VM has started the bootloader, I
have selected the debug flags, pressed enter, and the VM's screen
dimensions have changed.
HTH,
Spiro.
--
Spiro R. Trikaliotis [email protected]
Dipl.-Inform. http://www.trikaliotis.net/
OvG-Universitaet, IVS.EUK Fon: +49-391-67-12566
PF 4120, 39016 Magdeburg Fax: +49-391-67-11161
> command line specifying the following:
> start windbg -W Pipe -b -k com:pipe,port=\\.\pipe\dbgpipe,resets=0
Thanks for replies, Peter and Spiro, at least I know that it _may_ work as
I expected.
Strange, very strange...
My arrangements are exactly the same for both VPC (no
problems) and VMWare, I also figured out that there is no way
- for now, at least - to save the info about COM port being
redirected to a pipe in windbg's workspace so that
we have to use prosthetics. In my case it is a dbgpipe.cmd
wich contains (I split the line)
"C:\Program Files\Debugging Tools for Windows\windbg.exe"
-y srv*D:\SYMBOLS*http://msdl.microsoft.com/download/symbols
-k com:pipe,port=\\.\pipe\dbgpipe,resets=0,reconnect
(as you see, no break on boot here, it _may_ be saved in a
workspace; symbols in this script are extras also).
Yet again, this script works ok with VPC but not with VMWare,
and I can not spot any difference whatsoever...
The VMWare-based target sits for some notable time, say
30 secs, probably waiting for windbg response, the port state
is reported by VMWare as "connected" (to what ???), and then
the boot continues as usual, and windbg does not see anything.
Mystery. And I thought that all named pipes are created equal:-)
> I have encountered an issue with VPC that requires me to restart the
> debugger whenever I restart the virtual machine.
Never ever happened to me, I run as many reboots as I want keeping
windbg running. One more mystery...
> With VMWare, I have the following issue: When I boot the virtual machine
> (XP), the debugger is not allowed to be started (or the VM will not boot
> at all). I have to wait until after the VM has started the bootloader, I
> have selected the debug flags, pressed enter, and the VM's screen
> dimensions have changed.
Can you tell me what are the VMWare's and target's port settings?
client/server, other end is a VM/other end is an app, yield on poll?
Speed seems not to matter (or does it?).
Anyway, I'll try it your way, normally I start windbg way before the
target, and it's ok for VPC.
Regards,
Alex Shvedov
----- Original Message -----
From: "Peter Scott" <[email protected]>
To: "Kernel Debugging Interest List" <[email protected]>
Sent: Tuesday, July 19, 2005 5:25 PM
Subject: RE: [windbg] COM <-> named pipe debugging in vmware workstation
>
> I use this method for debugging all the time. To get it working, I created
> the virtual COM port in the virtual machine to be a named pipe, as you
> did.
> Then I opened WInDbg and specified to connect via a named pipe, the only
> way
> I managed this was by running WinDbg from a command line specifying the
> following:
>
> start windbg -W Pipe -b -k com:pipe,port=\\.\pipe\dbgpipe,resets=0
>
> Note that this will also break on entry into the system, I do this to
> ensure
> connectivity. It also uses the workspace profile I created called Pipe,
> which contains the symbol paths etc.
>
> I have encountered an issue with VPC that requires me to restart the
> debugger whenever I restart the virtual machine. This is not always the
> case
> with VMWare but it does happen.
>
> Pete
>
>
> Kernel Drivers
> Windows Filesystem and Device Driver Consulting
> www.KernelDrivers.com
> (303)546-0300
>
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of xxxxx@Home
> Sent: Tuesday, July 19, 2005 1:25 PM
> To: Kernel Debugging Interest List
> Subject: [windbg] COM <-> named pipe debugging in vmware workstation
>
> I successfully connect a target macine running inside VirtualPC to
> a host-based windbg through a named pipe.
>
> Now I try to do the same on a VMWare workstation 5.0.
>
> I redirect virtual's COM1 or COM2 to \\.\pipe\dbgpipe which is
> what I use for VPC but no matter what I do with other VMWare's
> COM port settings (client vs server, app or VM on the other end,
> yield on poll), host-based windbg does not see the target
> [VMWare virtual] machine but is happy when the target runs
> inside a VPC.
>
> boot.ini's are identical on both virtual machines, with records like
>
> multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="win + COM1 +
> ntbtlog.txt" /fastdetect /debug /debugport=com1
> /baudrate=115200 /bootlog
>
> VMWare on-line help brought nothing new. My guess is that
> it should work, what can be possibly wrong with a named pipe?
>
> Did anyone have this problem and solved it (or did not have this
> VMWare-specific problem at all, for that matter)?
>
> Maybe, it's some security policy setting that I'm missing?
>
> Any hints would be greatly appreciated.
>
>
> Regards,
> Alex Shvedov
>
>
>
> ---
> You are currently subscribed to windbg as: [email protected]
> To unsubscribe send a blank email to [email protected]
>
>
>
> ---
> You are currently subscribed to windbg as: [email protected]
> To unsubscribe send a blank email to [email protected]
>
xxxxx@Home <[email protected]> schrieb:
> The VMWare-based target sits for some notable time, say 30 secs,
> probably waiting for windbg response, the port state is reported by
> VMWare as "connected" (to what ???), and then the boot continues as
> usual, and windbg does not see anything.
This is exactly the behaviour I had with older WinDBGs. With newer ones,
the target just doesn't sit 30seconds or so, but forever.
Thus, my work-around is like this: Don't start WinDBG until the target
waits. Now, start WinDBG and press Ctrl+Break (this is important, the
automatic break-in does NOT work!). After this, I have control over the
target.
Regards,
Spiro.
--
Spiro R. Trikaliotis
http://www.trikaliotis.net/
> waits. Now, start WinDBG and press Ctrl+Break (this is important, the
> automatic break-in does NOT work!). After this, I have control over the
> target.
Thanks.
The Ctrl+Break adds to the idea I extracted from your previous post.
My guess is that VMWare virtual starts its COM pipe as a _server_
so it waits for someone to send in a request.
Stupid, as compared to VPC, and - if I'm right - directly contradicts
VM's own on-line help.
> This is exactly the behaviour I had with older WinDBGs.
I had all this with windbg 6.4, just got the latest tools with 6.5.3.7,
let's see...
Regards,
Alex Shvedov
----- Original Message -----
From: "Spiro Trikaliotis" <[email protected]>
To: "Kernel Debugging Interest List" <[email protected]>
Sent: Saturday, July 23, 2005 4:04 AM
Subject: Re: [windbg] COM <-> named pipe debugging in vmware workstation
> Hello Alex,
>
> xxxxx@Home <[email protected]> schrieb:
>
>> The VMWare-based target sits for some notable time, say 30 secs,
>> probably waiting for windbg response, the port state is reported by
>> VMWare as "connected" (to what ???), and then the boot continues as
>> usual, and windbg does not see anything.
>
> This is exactly the behaviour I had with older WinDBGs. With newer ones,
> the target just doesn't sit 30seconds or so, but forever.
>
> Thus, my work-around is like this: Don't start WinDBG until the target
> waits. Now, start WinDBG and press Ctrl+Break (this is important, the
> automatic break-in does NOT work!). After this, I have control over the
> target.
>
> Regards,
> Spiro.
>
> --
> Spiro R. Trikaliotis
> http://www.trikaliotis.net/
>
> ---
> You are currently subscribed to windbg as: [email protected]
> To unsubscribe send a blank email to [email protected]
>
time with VMWare. Not the case with VPC. I have the initial BP set and it
hits it each time on the VMWare virtual machine.
OK, every 10-15 reboots of a VMWare virtual pc I need to restart Windbg, I
call that pretty good.
Pete
Kernel Drivers
Windows Filesystem and Device Driver Consulting
www.KernelDrivers.com
(303)546-0300
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of xxxxx@Home
Sent: Saturday, July 23, 2005 8:32 AM
To: Kernel Debugging Interest List
Subject: Re: [windbg] COM <-> named pipe debugging in vmware workstation
> Thus, my work-around is like this: Don't start WinDBG until the target
> waits. Now, start WinDBG and press Ctrl+Break (this is important, the
> automatic break-in does NOT work!). After this, I have control over the
> target.
Thanks.
The Ctrl+Break adds to the idea I extracted from your previous post.
My guess is that VMWare virtual starts its COM pipe as a _server_
so it waits for someone to send in a request.
Stupid, as compared to VPC, and - if I'm right - directly contradicts
VM's own on-line help.
> This is exactly the behaviour I had with older WinDBGs.
I had all this with windbg 6.4, just got the latest tools with 6.5.3.7,
let's see...
Regards,
Alex Shvedov
----- Original Message -----
From: "Spiro Trikaliotis" <[email protected]>
To: "Kernel Debugging Interest List" <[email protected]>
Sent: Saturday, July 23, 2005 4:04 AM
Subject: Re: [windbg] COM <-> named pipe debugging in vmware workstation
> Hello Alex,
>
> xxxxx@Home <[email protected]> schrieb:
>
>> The VMWare-based target sits for some notable time, say 30 secs,
>> probably waiting for windbg response, the port state is reported by
>> VMWare as "connected" (to what ???), and then the boot continues as
>> usual, and windbg does not see anything.
>
> This is exactly the behaviour I had with older WinDBGs. With newer ones,
> the target just doesn't sit 30seconds or so, but forever.
>
> Thus, my work-around is like this: Don't start WinDBG until the target
> waits. Now, start WinDBG and press Ctrl+Break (this is important, the
> automatic break-in does NOT work!). After this, I have control over the
> target.
>
> Regards,
> Spiro.
>
> --
> Spiro R. Trikaliotis
> http://www.trikaliotis.net/
>
> ---
> You are currently subscribed to windbg as: [email protected]
> To unsubscribe send a blank email to [email protected]
>
---
You are currently subscribed to windbg as: [email protected]
To unsubscribe send a blank email to [email protected]
> time with VMWare. Not the case with VPC.
Mystery. I am happy with VPC 100%.
As I already said, I thought that all named pipes are created equal.
They are not, some named pipes are more named than others:-)
> initial BP set and it
> hits it each time on the VMWare virtual machine.
This is a secondary problem; in my case windbg hears
VPC all right in all cases, including target reboots, and
does not hear VMW at all.
I stop the stuff by a DbgBreakPoint in my DriverEntry
(in debug/checked build, of course).
Why on earth should it behave differently, I wonder?
It's just a named pipe, nothing more, after all.
Mystery.
A pro po: starting windbg when wmv is waiting, as Spiro
suggested, does help, but wmv restart is missed by windbg
once again. VMWare does insists on being a server?
Regards,
Alex Shvedov
----- Original Message -----
From: "Peter Scott" <[email protected]>
To: "Kernel Debugging Interest List" <[email protected]>
Sent: Saturday, July 23, 2005 10:00 PM
Subject: RE: [windbg] COM <-> named pipe debugging in vmware workstation
>
> Just to let you know, I posted my command line Windbg, and it works every
> time with VMWare. Not the case with VPC. I have the initial BP set and it
> hits it each time on the VMWare virtual machine.
>
> OK, every 10-15 reboots of a VMWare virtual pc I need to restart Windbg, I
> call that pretty good.
>
> Pete
>
> Kernel Drivers
> Windows Filesystem and Device Driver Consulting
> www.KernelDrivers.com
> (303)546-0300
>
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of xxxxx@Home
> Sent: Saturday, July 23, 2005 8:32 AM
> To: Kernel Debugging Interest List
> Subject: Re: [windbg] COM <-> named pipe debugging in vmware workstation
>
>> Thus, my work-around is like this: Don't start WinDBG until the target
>> waits. Now, start WinDBG and press Ctrl+Break (this is important, the
>> automatic break-in does NOT work!). After this, I have control over the
>> target.
> Thanks.
> The Ctrl+Break adds to the idea I extracted from your previous post.
> My guess is that VMWare virtual starts its COM pipe as a _server_
> so it waits for someone to send in a request.
> Stupid, as compared to VPC, and - if I'm right - directly contradicts
> VM's own on-line help.
>
>> This is exactly the behaviour I had with older WinDBGs.
> I had all this with windbg 6.4, just got the latest tools with 6.5.3.7,
> let's see...
>
> Regards,
> Alex Shvedov
>
>
> ----- Original Message -----
> From: "Spiro Trikaliotis" <[email protected]>
> To: "Kernel Debugging Interest List" <[email protected]>
> Sent: Saturday, July 23, 2005 4:04 AM
> Subject: Re: [windbg] COM <-> named pipe debugging in vmware workstation
>
>
>> Hello Alex,
>>
>> xxxxx@Home <[email protected]> schrieb:
>>
>>> The VMWare-based target sits for some notable time, say 30 secs,
>>> probably waiting for windbg response, the port state is reported by
>>> VMWare as "connected" (to what ???), and then the boot continues as
>>> usual, and windbg does not see anything.
>>
>> This is exactly the behaviour I had with older WinDBGs. With newer ones,
>> the target just doesn't sit 30seconds or so, but forever.
>>
>> Thus, my work-around is like this: Don't start WinDBG until the target
>> waits. Now, start WinDBG and press Ctrl+Break (this is important, the
>> automatic break-in does NOT work!). After this, I have control over the
>> target.
>>
>> Regards,
>> Spiro.
>>
>> --
>> Spiro R. Trikaliotis
>> http://www.trikaliotis.net/
>>
>> ---
>> You are currently subscribed to windbg as: [email protected]
>> To unsubscribe send a blank email to [email protected]
>>
>
>
> ---
> You are currently subscribed to windbg as: [email protected]
> To unsubscribe send a blank email to [email protected]
>
>
>
> ---
> You are currently subscribed to windbg as: [email protected]
> To unsubscribe send a blank email to [email protected]
>
* On Sat, Jul 23, 2005 at 10:32:02AM -0400 xxxxx@Home wrote:
> My guess is that VMWare virtual starts its COM pipe as a _server_ so
> it waits for someone to send in a request.
> Stupid, as compared to VPC, and - if I'm right - directly contradicts
> VM's own on-line help.
Well, at least in my VMWare, I can tell VMWare if I want this side to be
a server or an application... Look into the settings of your serial
port.
Anyway, I am not sure how this would change anything?
* On Sat, Jul 23, 2005 at 11:45:08PM -0400 xxxxx@Home wrote:
> I stop the stuff by a DbgBreakPoint in my DriverEntry (in
> debug/checked build, of course).
You mean instead of an initial breakpoint? Yes, this works for me, too.
> A pro po: starting windbg when wmv is waiting, as Spiro suggested,
> does help, but wmv restart is missed by windbg once again. VMWare
> does insists on being a server?
I don't remeber for a VMWare restart, but I remember that going back to
a snapshot in VMWare, you get an error that the pipe could not be opened
(as it already is open).
Perhaps, this is somehow related?
Regards,
Spiro.
--
Spiro R. Trikaliotis
http://www.trikaliotis.net/
I'm not using virtual anything.
--
The personal opinion of
Gary G. Little
"Peter Scott" wrote in message
news:xxxxx@windbg...
>
> Just to let you know, I posted my command line Windbg, and it works every
> time with VMWare. Not the case with VPC. I have the initial BP set and it
> hits it each time on the VMWare virtual machine.
>
> OK, every 10-15 reboots of a VMWare virtual pc I need to restart Windbg, I
> call that pretty good.
>
> Pete
>
> Kernel Drivers
> Windows Filesystem and Device Driver Consulting
> www.KernelDrivers.com
> (303)546-0300
>
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of xxxxx@Home
> Sent: Saturday, July 23, 2005 8:32 AM
> To: Kernel Debugging Interest List
> Subject: Re: [windbg] COM <-> named pipe debugging in vmware workstation
>
>> Thus, my work-around is like this: Don't start WinDBG until the target
>> waits. Now, start WinDBG and press Ctrl+Break (this is important, the
>> automatic break-in does NOT work!). After this, I have control over the
>> target.
> Thanks.
> The Ctrl+Break adds to the idea I extracted from your previous post.
> My guess is that VMWare virtual starts its COM pipe as a _server_
> so it waits for someone to send in a request.
> Stupid, as compared to VPC, and - if I'm right - directly contradicts
> VM's own on-line help.
>
>> This is exactly the behaviour I had with older WinDBGs.
> I had all this with windbg 6.4, just got the latest tools with 6.5.3.7,
> let's see...
>
> Regards,
> Alex Shvedov
>
>
> ----- Original Message -----
> From: "Spiro Trikaliotis"
> To: "Kernel Debugging Interest List"
> Sent: Saturday, July 23, 2005 4:04 AM
> Subject: Re: [windbg] COM <-> named pipe debugging in vmware workstation
>
>
>> Hello Alex,
>>
>> xxxxx@Home schrieb:
>>
>>> The VMWare-based target sits for some notable time, say 30 secs,
>>> probably waiting for windbg response, the port state is reported by
>>> VMWare as "connected" (to what ???), and then the boot continues as
>>> usual, and windbg does not see anything.
>>
>> This is exactly the behaviour I had with older WinDBGs. With newer ones,
>> the target just doesn't sit 30seconds or so, but forever.
>>
>> Thus, my work-around is like this: Don't start WinDBG until the target
>> waits. Now, start WinDBG and press Ctrl+Break (this is important, the
>> automatic break-in does NOT work!). After this, I have control over the
>> target.
>>
>> Regards,
>> Spiro.
>>
>> --
>> Spiro R. Trikaliotis
>> http://www.trikaliotis.net/
>>
>> ---
>> You are currently subscribed to windbg as: [email protected]
>> To unsubscribe send a blank email to [email protected]
>>
>
>
> ---
> You are currently subscribed to windbg as: [email protected]
> To unsubscribe send a blank email to [email protected]
>
>
>