Windbg and VMWare ESXi

(1) I have Windbg running with VMWare by using 2 vms. One just to run Windbg and the other is a target. Both are using are using serial as Windbg’s transport mechanism. VMWare “converts” the serial driver to a named-pipe between the two VMs. Cool and works. And I can be somewhere else on the Internet and work with both VMs with VMWare Infrastructure tool that lets work with both machines by “redirecting” video, keyboard and mouse to the web based tool. Real Cool.
BUT.
(2) I do not like “wasting” a vm for just Windbg. I tried using Windbg on my machine where I am running the VMWare Infrastructure and connect with the parameters to windbg. I am trying windbg -k com:pipe,port=\machinename\pipe\namepipe of serial port. I can not get this to work and I have tried different combinations of parameters. Anybody succeed in this?

I’m not familiar with VMWare Infrastructure, but it sounds like you just want to connect windbg to a
virtual machine? If so, that definitely works using the command line that you appear to be using,
though you might have some security issues.

What kind of error messages are you getting (and where)?

Is the VMWare host XP or Vista?

mm

xxxxx@earthlink.net wrote:

(1) I have Windbg running with VMWare by using 2 vms. One just to run Windbg and the other is a target. Both are using are using serial as Windbg’s transport mechanism. VMWare “converts” the serial driver to a named-pipe between the two VMs. Cool and works. And I can be somewhere else on the Internet and work with both VMs with VMWare Infrastructure tool that lets work with both machines by “redirecting” video, keyboard and mouse to the web based tool. Real Cool.
BUT.
(2) I do not like “wasting” a vm for just Windbg. I tried using Windbg on my machine where I am running the VMWare Infrastructure and connect with the parameters to windbg. I am trying windbg -k com:pipe,port=\machinename\pipe\namepipe of serial port. I can not get this to work and I have tried different combinations of parameters. Anybody succeed in this?

Martin O’Brien wrote:

I’m not familiar with VMWare Infrastructure, but it sounds like you just
want to connect windbg to a virtual machine? If so, that definitely
works using the command line that you appear to be using, though you
might have some security issues.

What kind of error messages are you getting (and where)?

Is the VMWare host XP or Vista?

It is sort of Linux ( = ESX server).

–pa

xxxxx@earthlink.net wrote:
> (1) I have Windbg running with VMWare by using 2 vms. One just to run
> Windbg and the other is a target. Both are using are using serial as
> Windbg’s transport mechanism. VMWare “converts” the serial driver to a
> named-pipe between the two VMs. Cool and works. And I can be
> somewhere else on the Internet and work with both VMs with VMWare
> Infrastructure tool that lets work with both machines by “redirecting”
> video, keyboard and mouse to the web based tool. Real Cool.
> BUT.
> (2) I do not like “wasting” a vm for just Windbg. I tried using
> Windbg on my machine where I am running the VMWare Infrastructure and
> connect with the parameters to windbg. I am trying windbg -k
> com:pipe,port=\machinename\pipe\namepipe of serial port. I can not
> get this to work and I have tried different combinations of
> parameters. Anybody succeed in this?

Ah, rebranding at work.

Thanks,

mm

Pavel A. wrote:

Martin O’Brien wrote:
> I’m not familiar with VMWare Infrastructure, but it sounds like you
> just want to connect windbg to a virtual machine? If so, that
> definitely works using the command line that you appear to be using,
> though you might have some security issues.
>
> What kind of error messages are you getting (and where)?
>
> Is the VMWare host XP or Vista?

It is sort of Linux ( = ESX server).

–pa

>
> xxxxx@earthlink.net wrote:
>> (1) I have Windbg running with VMWare by using 2 vms. One just to run
>> Windbg and the other is a target. Both are using are using serial as
>> Windbg’s transport mechanism. VMWare “converts” the serial driver to
>> a named-pipe between the two VMs. Cool and works. And I can be
>> somewhere else on the Internet and work with both VMs with VMWare
>> Infrastructure tool that lets work with both machines by
>> “redirecting” video, keyboard and mouse to the web based tool. Real
>> Cool.
>> BUT.
>> (2) I do not like “wasting” a vm for just Windbg. I tried using
>> Windbg on my machine where I am running the VMWare Infrastructure and
>> connect with the parameters to windbg. I am trying windbg -k
>> com:pipe,port=\machinename\pipe\namepipe of serial port. I can not
>> get this to work and I have tried different combinations of
>> parameters. Anybody succeed in this?

Named pipe mode like that only works if the pipe can be connected to via CIFS, i.e. you’re hosting the host on Windows. No dice for ESXi or VMware Server on Linux.

  • S

-----Original Message-----
From: xxxxx@earthlink.net
Sent: Monday, March 30, 2009 06:34
To: Windows System Software Devs Interest List
Subject: [ntdev] Windbg and VMWare ESXi

(1) I have Windbg running with VMWare by using 2 vms. One just to run Windbg and the other is a target. Both are using are using serial as Windbg’s transport mechanism. VMWare “converts” the serial driver to a named-pipe between the two VMs. Cool and works. And I can be somewhere else on the Internet and work with both VMs with VMWare Infrastructure tool that lets work with both machines by “redirecting” video, keyboard and mouse to the web based tool. Real Cool.
BUT.
(2) I do not like “wasting” a vm for just Windbg. I tried using Windbg on my machine where I am running the VMWare Infrastructure and connect with the parameters to windbg. I am trying windbg -k com:pipe,port=\machinename\pipe\namepipe of serial port. I can not get this to work and I have tried different combinations of parameters. Anybody succeed in this?


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Hi Ken:

Let me reiterate.
(1) If we running VMWare where Windows is the host then this will work with the Windbg connection of name-pipes at the command prompt. Now if we use Linux as the Host then the command line name-pipes for Windbg will not work.

(2) If we use ESXi which is a hypervisor style of virtualization then this will not work.

(3) So therefore the only way this will work with VMware’s technology is where Windows is the host.

Is the above correct?

This is a function of whether the VM host is on Windows or not. ESXi is out because it’s not hosted on Windows.

WinDbg’s named pipe support expects to connect to a remote NPFS over CIFS/SMB. Linux doesn’t support this (neither does the ESXi operating environment), and furthermore, pipes work somewhat differently internally for Linux anyway.

N.B. Other virtualization solutions (i.e. Hyper-V) hosted on Windows also support serial over named pipe support that is compatible with the debugger’s KDCOM capabilities.

Basically, this is a shortcut that takes advantage of the fact that the host supports exporting a serial port as a Win32 named pipe. If that isn’t available, then you need to run two guests with connected serial ports.

  • S

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@earthlink.net
Sent: Monday, March 30, 2009 12:56 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Windbg and VMWare ESXi

Hi Ken:

Let me reiterate.
(1) If we running VMWare where Windows is the host then this will work with the Windbg connection of name-pipes at the command prompt. Now if we use Linux as the Host then the command line name-pipes for Windbg will not work.

(2) If we use ESXi which is a hypervisor style of virtualization then this will not work.

(3) So therefore the only way this will work with VMware’s technology is where Windows is the host.

Is the above correct?


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Thanks Ken:

I get the idea. If we are using ESXi we have use the two vms at this point for kernel debugging. Interesting to note that the Hyper-V technology supports the NPFS CIFS/SMB solution for kernel debugging.

Hyper-V is hosted on Windows (“standalone” Hyper-V runs on a stripped down version of Srv08 Server Core), so it inherits the capability to do that.

  • S

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@earthlink.net
Sent: Monday, March 30, 2009 1:42 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Windbg and VMWare ESXi

Thanks Ken:

I get the idea. If we are using ESXi we have use the two vms at this point for kernel debugging. Interesting to note that the Hyper-V technology supports the NPFS CIFS/SMB solution for kernel debugging.


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

>WinDbg’s named pipe support expects to connect to a remote NPFS over CIFS/SMB.

No.

NPFS does not mandate CIFS/SMD. Actually, \.\pipe.… names are not involved in SMB.


Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com