Hello All,
I am implementing the SR-IOV driver for our network adapter…
I was going through the SR-IOV documentation… I have few queries, can some body help me in understanding :
The doc says when we get a OID OID_NIC_SWITCH_CREATE_VPORT for creating a non default Vport :–> When the PF miniport driver is issued the OID request, the driver allocates the hardware and software resources associated with the specified nondefault VPort.
- NDIS_NIC_SWITCH_VPORT_PARAMETER->NumQueuePairs: Do PF driver need to create queues for the VF driver ?
- what is the use of all the members specified in the structure NDIS_NIC_SWITCH_VPORT_PARAMETER w.r.t to PF driver ?
- if answer for question 1 is true …then Why do PF driver need to create the queues for the non default Vport? [VF driver any ways has to create queue for TX and RX].
-Thanks
Think about this this way. The guest OS cannot be trusted. It might be
running a rootkit. Or the hoster may be selling computing time to a tenant
you decides to try to grab more resources than he’s supposed to have.
Your PF driver must ensure that nothing the VF driver does can allow the VF
to adversely impact other VMs on the machine. This means that it must be
impossible for the VF to ask for more queues than it has been allotted. (In
fact, it makes little sense to allow a VF to configure its own queues.) As
another example, it must also be impossible for the VF to send more
interrupts than policy allows.
If your hardware allows a VF to configure itself, choosing resources for
itself, your design probably can’t be successfully turned into a secure
product. Our tests, for example, will take over a VF in a non-privileged VM
and write random patterns to the VF’s memory-mapped I/O space. If this
impacts networking through the PF, or any other VF, you’ll fail.
Jake Oshins
Windows Kernel Team
The message offers no warranties and confers no rights.
wrote in message news:xxxxx@ntdev…
Hello All,
I am implementing the SR-IOV driver for our network adapter…
I was going through the SR-IOV documentation… I have few queries, can some
body help me in understanding :
The doc says when we get a OID OID_NIC_SWITCH_CREATE_VPORT for creating a
non default Vport :–> When the PF miniport driver is issued the OID
request, the driver allocates the hardware and software resources associated
with the specified nondefault VPort.
- NDIS_NIC_SWITCH_VPORT_PARAMETER->NumQueuePairs: Do PF driver need to
create queues for the VF driver ?
- what is the use of all the members specified in the structure
NDIS_NIC_SWITCH_VPORT_PARAMETER w.r.t to PF driver ?
- if answer for question 1 is true …then Why do PF driver need to create
the queues for the non default Vport? [VF driver any ways has to create
queue for TX and RX].
-Thanks
thanks Jake…
But if PF driver allocates queues and hardware resources, how will VF driver come to know about these resources ? for example we allocate a shared memory[with the updated struct for shared memory i.e provide the port id] ?
In windows VF driver documentation there is not much information on this… it just provides the details for MpInitialize function and registry settings…
-Thank you…
There is no memory shared between the PF and the VF (at least not for
control structures.) This would be another sort of security hole. If you
need to communicate between them, use these functions:
NdisMReadConfigBlock
NdisMWriteConfigBlock
NdisMInvalidatedConfigBlock
You can see them documented on the “SR-IOV NDIS Functions” page, which is
the first hit here:
http://www.bing.com/search?q=SR-IOV+NDIS+Functions&qs=n&form=QBLH&pq=sr-iov+ndis+functions&sc=0-0&sp=-1&sk=
(The ever-so-helpful MSDN people are always changing the links so that
anything we post here will be obsolete before we turn around. That’s why
I’ve given you a search link.)
Jake Oshins
Windows Kernel Team
The message offers no warranties and confers no rights.
wrote in message news:xxxxx@ntdev…
thanks Jake…
But if PF driver allocates queues and hardware resources, how will VF driver
come to know about these resources ? for example we allocate a shared
memory[with the updated struct for shared memory i.e provide the port id] ?
In windows VF driver documentation there is not much information on this…
it just provides the details for MpInitialize function and registry
settings…
-Thank you…
Oh, and given the questions that you’re asking, I strongly recommend that
you pause and read this book:
http://www.bing.com/search?q=writing+secure+code+second+edition&qs=AS&form=QBRE&pq=writing+secure+code+second&sc=3-26&sp=1&sk=
Jake Oshins
Windows Kernel Team
The message offers no warranties and confers no rights.
“Jake Oshins” wrote in message news:xxxxx@ntdev…
There is no memory shared between the PF and the VF (at least not for
control structures.) This would be another sort of security hole. If you
need to communicate between them, use these functions:
NdisMReadConfigBlock
NdisMWriteConfigBlock
NdisMInvalidatedConfigBlock
You can see them documented on the “SR-IOV NDIS Functions” page, which is
the first hit here:
http://www.bing.com/search?q=SR-IOV+NDIS+Functions&qs=n&form=QBLH&pq=sr-iov+ndis+functions&sc=0-0&sp=-1&sk=
(The ever-so-helpful MSDN people are always changing the links so that
anything we post here will be obsolete before we turn around. That’s why
I’ve given you a search link.)
Jake Oshins
Windows Kernel Team
The message offers no warranties and confers no rights.
wrote in message news:xxxxx@ntdev…
thanks Jake…
But if PF driver allocates queues and hardware resources, how will VF driver
come to know about these resources ? for example we allocate a shared
memory[with the updated struct for shared memory i.e provide the port id] ?
In windows VF driver documentation there is not much information on this…
it just provides the details for MpInitialize function and registry
settings…
-Thank you…
Jake i guess my question has not reached to you correctly i guess…
If i go back in this conversation,
PF driver allocates queues even for VF[i.e in MS terms VPORT]. As this statement is true,
Then, if a VM wants to send the data …
-
As the queues are created by PF, if VF wants to send/receive the data[tx/rx], then we need to use the PF/VF back channel communication APIs, so that VF first writes to the config block and PF reads it and then writes to the specified VPORT Queue which will latter tx the packets through the adapter…
Is it correct ?
-
Is it that the queues memory which are created in the PF using the NDIS shared memory API, are accessible to the VF driver and when VM wants to send data it writes to this shared memory?
No. I still don’t think that I’ve completely understood your question. But
I’ll describe what might be the answer to it.
The PF must control resource allocation. And it must prevent the VF from
doing anything that would impact other workloads running on the same host.
But once the VF has been configured (by the PF driver) then the control
registers for the queues are mapped into the VF’s BAR space. At that point,
network packets can be sent and received by the VF with no interaction with
the PF. This is the entire goal of SR-IOV. If you need to interact with
the PF driver to send and receive packets, then you might as well turn off
SR-IOV and just use VMQ, as we did in Windows Server 2008 R2.
Our miscommunication probably stems from the use of the word “queue,” which
can have many meanings. I meant it to mean “control structure in the
hardware of your NIC which has a filtering engine that can separate packets
arriving for one MAC address/VLAN pair from another MAC address/VLAN pair
which will store and forward these packets to the correct client in
software.” Perhaps you meant it to mean “control structures in software
which track packets in main memory.”
To set up a receive buffer in the guest OS which the hardware filters
target, you do call the same NDIS common buffer functions that you would
with any NDIS miniport. You then program the VF hardware to target that
common buffer, from the VF miniport.
Jake Oshins
Windows Kernel Team
This message offers no warranties and confers no rights.
wrote in message news:xxxxx@ntdev…
Jake i guess my question has not reached to you correctly i guess…
If i go back in this conversation,
PF driver allocates queues even for VF[i.e in MS terms VPORT]. As this
statement is true,
Then, if a VM wants to send the data …
-
As the queues are created by PF, if VF wants to send/receive the
data[tx/rx], then we need to use the PF/VF back channel communication APIs,
so that VF first writes to the config block and PF reads it and then writes
to the specified VPORT Queue which will latter tx the packets through the
adapter…
Is it correct ?
-
Is it that the queues memory which are created in the PF using the NDIS
shared memory API, are accessible to the VF driver and when VM wants to send
data it writes to this shared memory?
I am not really getting confused with the word “queue”…
OK let me explain it by using the MS OID calls
- When PF driver get the call for OID_NIC_SWITCH_ALLOCATE_VF, PF driver get the parameters like NIC name Mac address/Mac address length for VF… This OID is clear, PF driver needs to make necessary calls to expose a Virtual Function and set the value of the VF MAC address etc.
[No Doubts in this]
************************************************
My confusion is with the OID OID_NIC_SWITCH_CREATE_VPORT.
**************************************************
2 when PF driver gets the call for the OID OID_NIC_SWITCH_CREATE_VPORT, PF driver gets the parameters as defined in the structure NDIS_NIC_SWITCH_VPORT_PARAMETERS. If you see this struct it describes the NumQueuePairs,InterruptModeration,ProcessorAffinity.
The description of
NumQueuePairs:
One or more queue pairs are configured on a nondefault VPort through an OID method request of OID_NIC_SWITCH_CREATE_VPORT.
******************************
MY QUESTION IS : Do the above description mean that PF driver needs to create the queues for the VF or is it that the value of this parameter has to be some how passed to the VF driver so that VF driver reads this and creates the Tx/Rx queue based on this parameter count ?
When i mean queue i mean the hardware queue …
-Thanks