NET Debugging Issue

I have a setup where I am doing NET debugging on ROOT OS and HV both.

Obviously I am using separate ports (54323 for kernel and 54324 for HV)

I believe I have setup the connection correctly.
I see the initial connect to windbg on my host (bootdebug connection) and
then I suddenly see the flood of below message in italics.

*KDNET received an out of sequence ping packet.The target machine restarted
without notifying the debugger.Forcing a debugger reconnect…*

I see that HV do connects on target is and still connected to WinDBG.
But not the root OS.

I fired wireshark and I see the UDP packets are being casted on the network.

Has anybody every faced an issue like this?
Any plausible explanation for this behavior?

-Deepak