PCI bus numbering

How the system determines and allocates bus numbers to its express port bridges and switches?
Is the BIOS allocates the bus number resource and transfer its allocation to the OS or the OS allocates this resource according to the BIOS scan results?
Is there a way to ask the OS to change the bus number allocation of some of the express ports?
Is there a way to write a filter driver that intercept the bus number allocating and ask to change this resource allocation?

OS.

Why? What fundamental problem are you trying to solve?

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Wednesday, July 29, 2009 8:19 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] PCI bus numbering

How the system determines and allocates bus numbers to its express port bridges and switches?
Is the BIOS allocates the bus number resource and transfer its allocation to the OS or the OS allocates this resource according to the BIOS scan results?
Is there a way to ask the OS to change the bus number allocation of some of the express ports?
Is there a way to write a filter driver that intercept the bus number allocating and ask to change this resource allocation?

OS.


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

> How the system determines and allocates bus numbers to its express port bridges and switches?

The OS can get this info simply by scanning, i.e. writing/reading PCI configuration registers and checking all possible combinations of bus,device and function numbers, effectively establishing that “Function X of Device Y on Bus Z is present”. Then it can get the additional info about the target device via PCI Configuration space. Please note that some info obtained from PCI configuration space (for example, interrupt pin) may be invalid for a given HAL, so that the OS can obtain it from BIOS tables (and, if BIOS allows it, can even change it).

Is there a way to ask the OS to change the bus number allocation of some of the express ports?

It is not the OS but BIOS who establishes this mapping…

Anton Bassov

replay to Doron Holan,
The fundamental problem that I’m trying to solve is device manager error code 31.
We have a PCIe express card that act as a bridge 1 (upstream) to 2 (downstream).
If the machine (laptop) is booting while the express card in unplugged and only after the machine is booted then I plugged in the express card then I’m getting the above error.
I’m assuming that it is happened because that OS can not allocate bus numbers to these downstream.
By the way, the error code 31 is appearing on the downstream bridges. The upstream PCI driver is loading OK. I think that the OS reserved one bus number extra for express slot.
I’m trying to see if I can change the bus numbering allocation.
Maybe by create a communication with ACPI (with evaluate method).

Anton, Interrupt Pin is never invalid. Interrupt Line is pretty
meaningless, and I think that’s what you meant.

Furthermore, Windows will number PCI busses if they have no number when the
BIOS hands off to the OS. We’ve toyed many times with dynamically
rebalancing bus numbers and the result is always that too many things break
when we do. We’re currently thinking about this again as we think about the
ramifications of the SR-IOV specification.

At a high level, I think that this is a symptom of a larger issue in PC bus
architecture. When we added PCI support to Windows 95 OSR 2 and Windows NT
3.51, we essentially locked out the creation of new PnP mechanisms for new
backplane busses. Hardware designers couldn’t afford to invent a new bus
which didn’t run with existing software. So every new bus (proprietary or
otherwise) has to look more or less backward compatible with PCI from a
software perspective. PCI Express and HyperTransport are two easily
accessible examples, though there are many others. And the bottom line is
that we’ve found that if we mess with bus numbering, we expose the
incompatible aspects of these busses (or chipsets) which the hardware
designers swore were invisible to software. So we leave bus numbers (and
other resources too) the way the BIOS set them up.


Jake Oshins
Hyper-V I/O Architect
(general PnP guy and member of the PCI SIG’s I/O Virtualization
subcommittee)
Windows Kernel Group

This post implies no warranties and confers no rights.


wrote in message news:xxxxx@ntdev…
>> How the system determines and allocates bus numbers to its express port
>> bridges and switches?
>
> The OS can get this info simply by scanning, i.e. writing/reading PCI
> configuration registers and checking all possible combinations of
> bus,device and function numbers, effectively establishing that “Function X
> of Device Y on Bus Z is present”. Then it can get the additional info
> about the target device via PCI Configuration space. Please note that
> some info obtained from PCI configuration space (for example, interrupt
> pin) may be invalid for a given HAL, so that the OS can obtain it from
> BIOS tables (and, if BIOS allows it, can even change it).
>
>> Is there a way to ask the OS to change the bus number allocation of some
>> of the express ports?
>
> It is not the OS but BIOS who establishes this mapping…
>
>
> Anton Bassov
>
>
>
>
>
>

ACPI won’t solve your problem. Fundamentally, nothing will, except for
calling the guy who made your BIOS with a plea to get him to leave more
space in his PCI bus numbering algorithm. If you renumber the busses
yourself, you’ll not only mess with the PnP manager in really nasty ways but
you’ll probably run into the same problems we have. You can get it working
on one machine. But it’s really difficult to get it working across the
entire PC ecosystem.


Jake Oshins
Hyper-V I/O Architect (former ACPI guy)
Windows Kernel Group

This post implies no warranties and confers no rights.


wrote in message news:xxxxx@ntdev…
> replay to Doron Holan,
> The fundamental problem that I’m trying to solve is device manager error
> code 31.
> We have a PCIe express card that act as a bridge 1 (upstream) to 2
> (downstream).
> If the machine (laptop) is booting while the express card in unplugged and
> only after the machine is booted then I plugged in the express card then
> I’m getting the above error.
> I’m assuming that it is happened because that OS can not allocate bus
> numbers to these downstream.
> By the way, the error code 31 is appearing on the downstream bridges. The
> upstream PCI driver is loading OK. I think that the OS reserved one bus
> number extra for express slot.
> I’m trying to see if I can change the bus numbering allocation.
> Maybe by create a communication with ACPI (with evaluate method).
>
>

> Anton, Interrupt Pin is never invalid. Interrupt Line is pretty meaningless, and I think that’s what you meant.

Indeed, I mean interrupt line…

The problem is that we just seem to use terms line and pin interchangeably these days, although Line and Pin are two different registers in PCI configuration space. I think this is mainly due to the fact that these days the term “interrupt line” often refers to IOAPIC pin, so that statements like “IRQ 27” (meaning pin 3 of IOAPIC 2) are not that uncommon…

Anton Bassov

I’d guess that your BIOS is numbering buses very conservatively, and so there’s only one bus number for any given root port. When you hot plug your ExpressCard, two bus numbers are required, but there’s not enough space for two, so you get your code 31 error. If you boot your computer with the ExpressCard already attached, it’s likely this will fix the problem, since the BIOS knows that it needs to leave room for it. Also, it’s possible you might have options in your BIOS settings to leave larger bus number gaps for hot plugging.

I’ve had some limited success with creating a filter driver to renumber buses, but this is during startup, and before PCI.sys loads. I have no experience with trying to renumber a hot plugged device, but renumbering buses after the PCI bus driver loads seems more difficult.

If you can’t find a BIOS option to leave a larger bus number gap, and you really must have hot plugging, you could possibly create a filter driver that runs during startup that writes to the root port’s configuration space to forward more than one bus number. Then when you hot plug, it will have enough space.

Daniel Marcotte:
To whom your filter driver was attached? to ACPI?
How did you access to the PCI configuration space?
What do you mean by that you had limited success? What didn’t you success to do?

First off, keep in mind that this sort of thing is definitely not recommended. In fact, it’s discouraged. Secondly, if you do this wrong, the computer might not be able to boot, and you’d have to reimage the harddrive. So whatever you’re using for testing your code, you should have a good image saved in advance.

I used a class filter driver on the System Class, however there are other options available for what you’re doing. Essentially, to renumber your buses, you really just need to load before PCI.sys, and then you can do all of your renumberings in the DriverEntry. You can access what you need in config space through the CF8 and CFC registers. My success in this process is limited because it doesn’t work on every computer. If it works on a particular system, it will always work on it, however there’s no guarantee that it will work for another system. Typically, I won’t see any code 31 errors, but I’ve been known to run into code 12 errors, which means that resources could not be assigned. This is all in Windows XP.

In Vista, a filter driver can complicate things, especially when trying to renumber buses, because Vista has been known to renumber things on it’s own. If you are trying to hotplug your card in Vista, you have other options other than creating a filter driver, such as using the registry key HackFlags.

Are you using XP or Vista, or are you trying to use both?