Multifunction PCI Express Devices?

Hi

We are thinking about implementing firmware that exports few functions (MAC and serial) and of course the corresponding driver.

I’ve read about bus drivers and multifunction drivers, and I’m not sure whether they both relate to the same thing or not.
What’s the difference between bus device and multifunction device?

I also haven’t found anything about PCI express multifunction devices.
Does PCI express multifunction devices exist?

Thanks,
Tal

*>>>What’s the difference between bus device and multifunction device?*
Devices are connected to bus based on type of device, e.g. pci(e)
devices connected PCI(e) bus,
Bus helps in discovering devices connected to it, identifying its
functions. In pre-boot environment BIOS identifies all the buses and
devices connected to it and prepares table of all discovered devices, ACPI
tables.
In case of pci(e) , devices can be multi-function. As per the specification
PCI(e) can incorporate 8 different functions. Each function has its own
configuration space.

*>>>>I also haven’t found anything about PCI express multifunction devices.*
*>>>>Does PCI express multifunction devices exist?*
PCI(e) devices can be multi-functional.

Regards,
Mohan

On Mon, Feb 4, 2013 at 1:04 PM, wrote:

> Hi
>
> We are thinking about implementing firmware that exports few functions
> (MAC and serial) and of course the corresponding driver.
>
> I’ve read about bus drivers and multifunction drivers, and I’m not sure
> whether they both relate to the same thing or not.
> What’s the difference between bus device and multifunction device?
>
> I also haven’t found anything about PCI express multifunction devices.
> Does PCI express multifunction devices exist?
>
> Thanks,
> Tal
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Conformant PCI(e) multifunction devices implement each ‘function’ as a
separate PCI(e) function on the physical device. You should too. You do not
want to attempt to implement a resource sharing bus driver for windows if
you can possibly avoid it.

Mark Roddy

On Mon, Feb 4, 2013 at 2:34 AM, wrote:

> Hi
>
> We are thinking about implementing firmware that exports few functions
> (MAC and serial) and of course the corresponding driver.
>
> I’ve read about bus drivers and multifunction drivers, and I’m not sure
> whether they both relate to the same thing or not.
> What’s the difference between bus device and multifunction device?
>
> I also haven’t found anything about PCI express multifunction devices.
> Does PCI express multifunction devices exist?
>
> Thanks,
> Tal
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

PCIe was expanded about five years ago to support devices with up to 256 functions, and many more if the device supports I/O virtualization. Support for that went into Windows 8 and Server 2012.

  • Jake Oshins
    (Microsoft representative to the PCI SIG on topics of virtualization)
    Windows Kernel Team

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Mohan Tarole
Sent: Monday, February 4, 2013 5:54 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Multifunction PCI Express Devices?

>>What’s the difference between bus device and multifunction device?
Devices are connected to bus based on type of device, e.g. pci(e) devices connected PCI(e) bus,
Bus helps in discovering devices connected to it, identifying its functions. In pre-boot environment BIOS identifies all the buses and devices connected to it and prepares table of all discovered devices, ACPI tables.
In case of pci(e) , devices can be multi-function. As per the specification PCI(e) can incorporate 8 different functions. Each function has its own configuration space.

>>>I also haven’t found anything about PCI express multifunction devices.
>>>Does PCI express multifunction devices exist?
PCI(e) devices can be multi-functional.

I would put it this way. If your device has multiple interfaces, for example multiple ports in a NIC, you’ll need to split that out so that Windows perceives your adapter as multiple devices. You can either do that using the PCI spec, exposing multiple functions in which case Microsoft’s PCI.sys will be the bus driver that exposes multiple device, or you can do it by exposing a single function to PCI.sys and then supplying your own bus driver which further splits your adapter up into different logical devices.

If you choose to use Microsoft’s bus driver, then you’ve got to conform to all the picayune details of the PCI and PCI Express specs for multi-function devices and all the things implied by those specs within Microsoft’s driver model.

If you choose to use your own bus driver, then you can mostly define your own usage model and you have a lot more flexibility. But, as Mark notes, this comes with a software burden that, while manageable, isn’t trivial.

Many people run into problems by designing multi-function devices where there are interdependencies between the functions. Just last week, somebody came to this forum and asked (and I’m paraphrasing) “how can I make a driver that uses two of my functions?” This really isn’t practical. If there’s any interaction *at all* between your functions, then you’re usually better off making a bus driver and not asking Microsoft’s bus driver to try to manage your device.

If, however, your functions really can stand alone with no software coordination, using PCI.sys as your bus driver (by exposing multiple functions) then you’ll have much less software to write. Usually, when I’m talking to hardware designers, I ask them this question:

“Can I move function 0 through D3 and back to D0, involving the full internal reset of function 0 while I’m using function 1?” If they answer no, they’re doomed, and they would have been better off not exposing multiple functions.

The next question after that one is this: “Can I upgrade the device driver for function 0 while I’m using function 1?” Again, a “no” answer means that you’d have been better off using a single function.

With this in mind, if you can still affect the design of your device, by all means, try to make it work without needing your own bus driver. That path will be simpler.

  • Jake Oshins
    (sometime PCI guy)
    Windows Kernel Team

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Mark Roddy
Sent: Monday, February 4, 2013 6:47 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Multifunction PCI Express Devices?

Conformant PCI(e) multifunction devices implement each ‘function’ as a separate PCI(e) function on the physical device. You should too. You do not want to attempt to implement a resource sharing bus driver for windows if you can possibly avoid it.

Mark Roddy

On Mon, Feb 4, 2013 at 2:34 AM, wrote:
Hi

We are thinking about implementing firmware that exports few functions (MAC and serial) and of course the corresponding driver.

I’ve read about bus drivers and multifunction drivers, and I’m not sure whether they both relate to the same thing or not.
What’s the difference between bus device and multifunction device?

I also haven’t found anything about PCI express multifunction devices.
Does PCI express multifunction devices exist?

Thanks,
Tal

>If, however, your functions really can stand alone with no software coordination, using PCI.sys as your bus driver (by exposing multiple functions) then you’ll have much less software to write.

There are some chips where the functions share a common engine, and common firmware space. And that way lies madness… No sane developer would make such architecture, but sometimes customers dictate making that shit.

Some 10-15 years ago I’ve had to deal with a PCI ADSL modem chip, which (by request of a prospective big OEM customer, known for their PC and servers, who later merged with another big OEM) had two PCI functions, one controlled the embedded controller, another controlled the data path. That was a hell to deal with.