Storage stack on Hyper-V

NTDEV
1

Hello All,

I have been trying to learn about Hyper-V and storage stack and how the storage stack in host OS interacts with VMs. However, I have not been able to find much documention. I am also trying to understand how MPIO works under Hyper-V?

Any pointers would be highly appreciated. Thank you!

2

Wrt, storage stack.

This might not be enough but it talks little bit about how the call goes.
http://blogs.msdn.com/b/tvoellm/archive/2008/01/02/hyper-v-integration-components-and-enlightenments.aspx

So basically, there is a storvsp that calls into vmbusr in the root and vmbusr communictes with vmbus in the guest.

3

Thanks for the pointer.

I want to figure out with MPIO/DSM installed on host having multiple VMs, how does the IO traffic gets routed from VM via the host storage stack?

I have searched every where but did not find much.

4

>routed from VM via the host storage stack?

Guest:

  • virtual SCSI controller (STORVSC driver, based on STORPORT and thus incompatible with XP). It sends the requests down the VMBus.
  • for emulated IDE disks, the per-port emulation is used
  • also, StorFlt driver helps emulated IDE disks, by sending requests down the VMBus, so the slow per-port emulation is only used before the guest’s storage stack - and thus StorFlt - is active.
  • VMBus sends the request to partition 0 using hypervisor calls of “send message”.

Host:

  • STORVSP driver consumes the HVCall messages and executes them by sending them down.
  • below, the VHD format parser is used
  • and below, the host’s filesystem is used (can even be SMB client in 2012).

MPIO is below the host’s filesystem. Hyper-V actually does not know much on MPIO :slight_smile:

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

5

Thanks a lot Maxim, this is what I had been looking, if you know of a MS document that explains this, can you please share?

Is the stack any different for passthrough disks?

I have a followup question then, since now we have NPIV based HBAs, so each VM can have a virtual HBA port and talk with storage, and see LUNs from the storage on VM (by creating virtual HBAs in your VMs and connecting them to virtual SANs). I can understand that VM itself is VHD or VHDX, and then for simple configuration you have;

VM IO -> HVCall->VHD parser->host FS->HBA->SAN

but for VMs that can see LUNs on SAN, how would the stack look like in those cases?

Thanks for your time, I really appreciate it. Thank you!

6

As far as I know, those fibre channel traffic goes through lunparser instead of vhd parser.

7

Virtual hard disks are stored as files. MPIO and DSMs do the same thing with virtual hard disk files (VHDs) that they do with any other file. If you want to study the behavior of this, load up any shared disk and watch it go in the debugger, or with tracing. The upshot is that the behavior will depend a lot on the DSM and the HBA.

  • Jake Oshins
    Hyper-V Team
    Microsoft

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.com
Sent: Sunday, November 18, 2012 11:14 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Storage stack on Hyper-V

Thanks for the pointer.

I want to figure out with MPIO/DSM installed on host having multiple VMs, how does the IO traffic gets routed from VM via the host storage stack?

I have searched every where but did not find much.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

8

Just a little correction here:

The hypervisor itself mostly isn’t involved in VMBus messages. There’s typically no hypercall at all, and when there is, it’s just a signal to post an interrupt to the opposite endpoint to get VMBus running again.

  • Jake Oshins
    Hyper-V Team
    Microsoft

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Monday, November 19, 2012 2:29 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Storage stack on Hyper-V

routed from VM via the host storage stack?

Guest:

  • virtual SCSI controller (STORVSC driver, based on STORPORT and thus incompatible with XP). It sends the requests down the VMBus.
  • for emulated IDE disks, the per-port emulation is used
  • also, StorFlt driver helps emulated IDE disks, by sending requests down the VMBus, so the slow per-port emulation is only used before the guest’s storage stack - and thus StorFlt - is active.
  • VMBus sends the request to partition 0 using hypervisor calls of “send message”.

Host:

  • STORVSP driver consumes the HVCall messages and executes them by sending them down.
  • below, the VHD format parser is used
  • and below, the host’s filesystem is used (can even be SMB client in 2012).

MPIO is below the host’s filesystem. Hyper-V actually does not know much on MPIO :slight_smile:

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer