I have a KMDF driver and for certain reasons, I need the PCIe domain number from the PCIe address. However, when I call WdfDeviceQueryProperty(DevicePropertyLocationInformation), I get the following:
PCI bus 4, device 0, function 0
Is there any way to obtain the domain number as well?
Thanks,
((&->
Does the endpoint device, or even Windows, even KNOW about the domain? Don’t they just know that the devices are in the “system” domain and that’s all they know?
Isn’t it up to an interdomain switch or something to understand any other topology?
This isn’t something I’ve dealt with before,
Peter
OSR
@OSRDrivers
This really depend on what you mean by “domain.” It’s not a term defined within the PCI specs, or at least not one that I’ve ever run across. (Please educate me if you can find it in there.)
I think that you’re asking about the PCI segment number, which for the audience, defines the scope of a family of embedded, PCIe and/or conventional PCI buses that have a common number space. There can be more than one “root” PCI bus in the system that has a bus number of zero. The segment number distinguishes these from each other.
So, if my reinterpretation above is correct, the number you’re looking for is the upper 24 bits of “bus number” in all the Windows APIs and data structures that have a bus number.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@Yahoo.com
Sent: Thursday, March 20, 2014 8:00 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Query the PCIe domain number
I have a KMDF driver and for certain reasons, I need the PCIe domain number from the PCIe address. However, when I call WdfDeviceQueryProperty(DevicePropertyLocationInformation), I get the following:
PCI bus 4, device 0, function 0
Is there any way to obtain the domain number as well?
Thanks,
((&->
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
The only time I’ve ever even heard the term “domain” and “PCIe” together in the same sentence was with regard to using a PCIe bus to connect multiple systems together:
http:
Peter
OSR
@OSRDrivers</http:>
Thanks, Peter.
That doc defines “domain” as, and I’m paraphrasing, “a root complex and all the endpoints that are attached to it, perhaps through intermediary switches.” Another way of paraphrasing it would be “the space across which a PCIe transaction can take place.” And, of course, there can be multiples of these in any machine.
Domains in this sense don’t have numbers. The set of buses within them have numbers. Windows doesn’t, however, have any mechanism to expose the root complex associated with a particular PCIe endpoint. You might be able to infer it from user mode using the same APIs that Device Manager uses to identify the root complex itself and work up the tree to that from an endpoint.
I’m honestly not sure what good that would do you. You might try to infer that peer to peer traffic is possible within a domain and only within the domain, though that’s often not true. You might try to infer that traffic is unbuffered within the domain, and that’s probably closer to true, statistically speaking.
I still think that the OP meant “segment number.”
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Thursday, March 20, 2014 10:57 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Query the PCIe domain number
The only time I’ve ever even heard the term “domain” and “PCIe” together in the same sentence was with regard to using a PCIe bus to connect multiple systems together:
http:
Peter
OSR
@OSRDrivers
—
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</http:>
Peter,
Using the definition of “domain” derived from the deck you forwarded to me, the Windows PCI driver does know about it and track it, but only if the BIOS hands control of PCIe to Windows. If the BIOS keeps control of Express, then Windows doesn’t attempt to set up all the tracking structures to hook things into the notion of a “root complex.”
Even when the BIOS does grant control to Windows, this mostly gets used for error tracking and reporting. Not much else that’s interesting to software is at the scope of a root complex.
We are, by the way, seeing a lot more machines lately that hand control of Express to Windows that that I added features to Windows 8 and Windows Server 2012 (runtime power management and SR-IOV support) which actually depend on getting that control.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Thursday, March 20, 2014 9:44 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Query the PCIe domain number
Does the endpoint device, or even Windows, even KNOW about the domain? Don’t they just know that the devices are in the “system” domain and that’s all they know?
Isn’t it up to an interdomain switch or something to understand any other topology?
This isn’t something I’ve dealt with before,
Peter
OSR
@OSRDrivers
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Thanks Jake… Interesting as always. Much appreciated,
Peter
OSR
@OSRDrivers
Peter & Jake:
This is a Linux-ish driver which has a port to Windows. The Linux people call it a “domain”, but this is the “segment” to which you refer. Whatever it is called, it is the PCIe address portion above the bus number:
ssss:bb:dd:ff.x
ssss =segment
bb = bus
dd = device
ff = function
Since there is no way to get the segment number in Windows, we just initialized the segment to 0. That worked perfect until we recently encountered an IvyTower system. Now the problem is that there are devices on different segments which collide at the bb:dd:ff level. Without the segment number, we cannot tell them apart. I could differentiate between the devices with the device instance, but that would change some of the methodology of the driver. It would be easy to just read the segment number and then there would be minimal changes to the code. However, it looks like I’m gonna havta do this the hard way.
((&->
And from what Jake said, take the bus number and use the high order 24 bits
as the segment number and the lower bits as the bus number.
Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@Yahoo.com
Sent: Thursday, March 20, 2014 3:05 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Query the PCIe domain number
Peter & Jake:
This is a Linux-ish driver which has a port to Windows. The Linux people
call it a “domain”, but this is the “segment” to which you refer. Whatever
it is called, it is the PCIe address portion above the bus number:
ssss:bb:dd:ff.x
ssss =segment
bb = bus
dd = device
ff = function
Since there is no way to get the segment number in Windows, we just
initialized the segment to 0. That worked perfect until we recently
encountered an IvyTower system. Now the problem is that there are devices
on different segments which collide at the bb:dd:ff level. Without the
segment number, we cannot tell them apart. I could differentiate between
the devices with the device instance, but that would change some of the
methodology of the driver. It would be easy to just read the segment number
and then there would be minimal changes to the code. However, it looks like
I’m gonna havta do this the hard way.
((&->
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
Thanks, Don, I’ll give that a shot.
Just as a follow up, in addition to Jake’s info, I also found some info about the PCIe segment in relation to ACPI. Here is some sample code:
// Get the BusNumber
status = WdfDeviceQueryProperty( device,
DevicePropertyBusNumber,
sizeof(ULONG),
(PVOID)&busNumber,
&length);
if ( !NT_SUCCESS(status) )
{
error_print(“Failed to get PCIe bus number.”);
}
// NOTE: https://lkml.org/lkml/2013/12/5/256 says: the “change to
// definition of the “Bus” field into the recently released ACPI
// Spec 5.0a section 18.3.2.3-5: … The Bus is encoded in bits 0-7.
// For systems that expose multiple PCI segment groups, the segment
// number is encoded in bits 8-23 and bits 24-31 must be zero.”
devContext->pcie_bus = (uint8_t)busNumber;
devContext->pcie_segment = (busNumber >> 8) & 0xFFFF;