Querying PCI Config Space.

NTDEV
1

Hi,

Is there a method to get the PCI config space for a device from user mode?

I have seen people using the filter driver approach. What I want is a method to query the bus driver.

Thanks,

Kevin.

2

xxxxx@gmail.com wrote:

Is there a method to get the PCI config space for a device from user mode?

No.

I have seen people using the filter driver approach. What I want is a method to query the bus driver.

That requires a filter driver. The bus driver does not expose an
interface to user mode.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

3

Nothing from user mode. What problem are you trying to solve?

d

Bent from my phone

From: xxxxx@gmail.commailto:xxxxx
Sent: ?5/?30/?2013 1:13 PM
To: Windows System Software Devs Interest Listmailto:xxxxx
Subject: [ntdev] Querying PCI Config Space.

Hi,

Is there a method to get the PCI config space for a device from user mode?

I have seen people using the filter driver approach. What I want is a method to query the bus driver.

Thanks,

Kevin.

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>

4

I’m trying to get/set device properties, something like Intel PCI Express Configuraton Utility.

Thanks,

Kevin.

5

Perhaps what you are looking for is expressed as device properties on the devnode.

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Thursday, May 30, 2013 2:21 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Querying PCI Config Space.

I’m trying to get/set device properties, something like Intel PCI Express Configuraton Utility.

Thanks,

Kevin.

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

6

I want to dump the complete config space. devonode has only certain properties.

Thanks,

Kevin.

7

You can use a driver to a access IO Ports and dump config space
No dia 30 de Mai de 2013 22:50, escreveu:

> I want to dump the complete config space. devonode has only certain
> properties.
>
> Thanks,
>
> Kevin.
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>

8

The PCI bus driver owns the resources needed to touch config space. If you attempt to use them without coordination with PCI, you can easily trash the system

d

Bent from my phone

From: Helder Danielmailto:xxxxx
Sent: ?5/?30/?2013 3:36 PM
To: Windows System Software Devs Interest Listmailto:xxxxx
Subject: RE:[ntdev] Querying PCI Config Space.

You can use a driver to a access IO Ports and dump config space

No dia 30 de Mai de 2013 22:50, > escreveu:
I want to dump the complete config space. devonode has only certain properties.

Thanks,

Kevin.

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

— NTDEV is sponsored by OSR OSR is HIRING!! See http://www.osr.com/careers For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>

9

As Doron has already pointed out those registers are owned by the PCI
bus driver. I’ve spent a number of weeks debugging a problem a client’s
customer claimed was our driver failing, when it was a piece of shit
software that did what you are proposing. Do not go there.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

“Helder Daniel” wrote in message news:xxxxx@ntdev:

> You can use a driver to a access IO Ports and dump config space
> No dia 30 de Mai de 2013 22:50, escreveu:
>
> > I want to dump the complete config space. devonode has only certain
> > properties.
> >
> > Thanks,
> >
> > Kevin.
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > OSR is HIRING!! See http://www.osr.com/careers
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at
> > http://www.osronline.com/page.cfm?name=ListServer
> >
> >

10

I was thinking only reading the configure space.

But I see your point.

Addressing config space means changing the config address register to read
data. And this will mess with PCI bus driver.

No dia 30 de Mai de 2013 23:54, “Don Burn” escreveu:

> As Doron has already pointed out those registers are owned by the PCI bus
> driver. I’ve spent a number of weeks debugging a problem a client’s
> customer claimed was our driver failing, when it was a piece of shit
> software that did what you are proposing. Do not go there.
>
>
> Don Burn
> Windows Filesystem and Driver Consulting
> Website: http://www.windrvr.com
> Blog: http://msmvps.com/blogs/ WinDrvr http:
>
>
>
>
> “Helder Daniel” wrote in message news:xxxxx@ntdev:
>
> You can use a driver to a access IO Ports and dump config space
>> No dia 30 de Mai de 2013 22:50, escreveu:
>>
>> > I want to dump the complete config space. devonode has only certain
>> > properties.
>> >
>> > Thanks,
>> >
>> > Kevin.
>> >
>> > —
>> > NTDEV is sponsored by OSR
>> >
>> > OSR is HIRING!! See http://www.osr.com/careers
>> >
>> > For our schedule of WDF, WDM, debugging and other seminars visit:
>> > http://www.osr.com/seminars
>> >
>> > To unsubscribe, visit the List Server section of OSR Online at
>> > http://www.osronline.com/page. cfm?name=ListServerhttp:
>> >
>> >
>>
>
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.**cfm?name=ListServerhttp:
>
></http:></http:></http:>

11

the correct way to do this is to write a pci bus filter driver that
attaches to all of the pdos enumerated by pci.sys. You can then, carefully,
use documented apis to access config space for each device.

Mark Roddy

On Thu, May 30, 2013 at 7:24 PM, Helder Daniel wrote:

> I was thinking only reading the configure space.
>
> But I see your point.
>
> Addressing config space means changing the config address register to read
> data. And this will mess with PCI bus driver.
>
>
>
> No dia 30 de Mai de 2013 23:54, “Don Burn” escreveu:
>
> As Doron has already pointed out those registers are owned by the PCI bus
>> driver. I’ve spent a number of weeks debugging a problem a client’s
>> customer claimed was our driver failing, when it was a piece of shit
>> software that did what you are proposing. Do not go there.
>>
>>
>> Don Burn
>> Windows Filesystem and Driver Consulting
>> Website: http://www.windrvr.com
>> Blog: http://msmvps.com/blogs/ WinDrvr http:
>>
>>
>>
>>
>> “Helder Daniel” wrote in message news:xxxxx@ntdev:
>>
>> You can use a driver to a access IO Ports and dump config space
>>> No dia 30 de Mai de 2013 22:50, escreveu:
>>>
>>> > I want to dump the complete config space. devonode has only certain
>>> > properties.
>>> >
>>> > Thanks,
>>> >
>>> > Kevin.
>>> >
>>> > —
>>> > NTDEV is sponsored by OSR
>>> >
>>> > OSR is HIRING!! See http://www.osr.com/careers
>>> >
>>> > For our schedule of WDF, WDM, debugging and other seminars visit:
>>> > http://www.osr.com/seminars
>>> >
>>> > To unsubscribe, visit the List Server section of OSR Online at
>>> > http://www.osronline.com/page. cfm?name=ListServerhttp:
>>> >
>>> >
>>>
>>
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> OSR is HIRING!! See http://www.osr.com/careers
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.**cfm?name=ListServerhttp:
>>
>> — NTDEV is sponsored by OSR OSR is HIRING!! See
> http://www.osr.com/careers For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
></http:></http:></http:>

12

On 31-May-2013 15:50, Mark Roddy wrote:

the correct way to do this is to write a pci bus filter driver that
attaches to all of the pdos enumerated by pci.sys. You can then,
carefully, use documented apis to access config space for each device.

One hardware engineer whom I know, did the correct thing:
When this guy heard that Window will not let driver access
the config space, he simply added another BAR and mirrored the config
space to it. That made everyone happy.
– pa

13

windows gives any driver in the devnode access to the config space, it just
doesn’t do that for random drivers not in the immediate device stack of the
hardware pdo.

Mark Roddy

On Fri, May 31, 2013 at 5:58 PM, Pavel A. wrote:

> On 31-May-2013 15:50, Mark Roddy wrote:
>
>> the correct way to do this is to write a pci bus filter driver that
>> attaches to all of the pdos enumerated by pci.sys. You can then,
>> carefully, use documented apis to access config space for each device.
>>
>
> One hardware engineer whom I know, did the correct thing:
> When this guy heard that Window will not let driver access
> the config space, he simply added another BAR and mirrored the config
> space to it. That made everyone happy.
> – pa
>
>
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.**cfm?name=ListServerhttp:
></http:>

14

> When this guy heard that Window will not let driver access

the config space, he simply added another BAR and mirrored the config
space to it. That made everyone happy.

He did the correct thing for sure, but Windows does allow the driver to access the config space, but only the config space of its own device.

Nevertheless, he is still correct. The good old PPT file by MS of “Windows PCI Hardware Guidelines” (I’ve read one in around year 2000) says: “no control/status registers in the config space”.

After all, the config space can be very slow.

–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

15

Pavel, it’s more than clear to me why your hardware guy added a BAR that mirrors config space. I know that Windows and other software stacks don’t offer direct, unfettered, lightweight access to every single part of configuration space. I also know that the PCI SIG has defined things in ways where things that shouldn’t be in configuration space are required to be there, and that makes the first problem more vexing.

But you all should also know that it’s specifically because of implementations like the one you describe that we’ve concluded that we cannot securely assign entire PCIe devices to a Hyper-V guest VM. We support SR-IOV for networking in Hyper-V, and in order to build that, we needed to get our hypervisor’s physical interrupt delivery and I/O MMU code working well enough that we could productively engage the networking device vendors’ driver teams. So we built whole-device assignment (not SR-IOV VF assignment,) in the lab, so that we could run existing device drivers while we tested the lower layers.

The very first device we got working was one that mirrors its config space into its first memory BAR. (We noticed this entirely accidentally, by the way. There’s nothing in the PCI spec that would tell us.) That got us thinking, and we started working through the threat model for a device where the driver, running in an untrusted VM, can change its own BAR values, change its own power state, arm its own wake signal, change its response to bus errors, etc. The results were sobering. I’m reasonably certain that I know how to attack any machine that assigns such a device to an untrusted VM. And the kicker is that it’s not possible to know when you’re doing so unless you talk to the vendor of the device and ask them if their device is insecure, something no vendor I’ve met seems to understand, let alone admit.

  • Jake Oshins
    (Hyper-V guy)
    Windows Kernel Team

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Mark Roddy
Sent: Saturday, June 1, 2013 4:11 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Querying PCI Config Space.

windows gives any driver in the devnode access to the config space, it just doesn’t do that for random drivers not in the immediate device stack of the hardware pdo.

Mark Roddy

On Fri, May 31, 2013 at 5:58 PM, Pavel A. > wrote:
On 31-May-2013 15:50, Mark Roddy wrote:
the correct way to do this is to write a pci bus filter driver that
attaches to all of the pdos enumerated by pci.sys. You can then,
carefully, use documented apis to access config space for each device.

One hardware engineer whom I know, did the correct thing:
When this guy heard that Window will not let driver access
the config space, he simply added another BAR and mirrored the config space to it. That made everyone happy.
– pa

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

— NTDEV is sponsored by OSR OSR is HIRING!! See http://www.osr.com/careers For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

16

>That got us thinking, and we started working through the threat model for a device where the driver, running in an untrusted VM, can change its own BAR values, change its own power
state, arm its own wake signal, change its response to bus errors, etc.

Does IOMMU allow to remap DMA without the guest and the device cooperation? Does device assignment require MSI(-X) interrupts only (level-triggered are verboten)?

17

Yes, the I/O MMUs remaps both DMA and interrupts, assuming that the chipset itself supports ACS (see the PCI SIG for a definition) such that all DMA is forced through the I/O MMU and not to peer devices. We don’t allow level-triggered interrupts, but even that would be possible if we wanted to bother.

But the point of my message is that there are lots of attack vectors that have little to do with DMA or interrupts, ones that an I/O MMU won’t help you with.

  • Jake

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@broadcom.com
Sent: Monday, June 3, 2013 10:58 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Querying PCI Config Space.

That got us thinking, and we started working through the threat model for a device where the driver, running in an untrusted VM, can change its own BAR values, change its own power
state, arm its own wake signal, change its response to bus errors, etc.

Does IOMMU allow to remap DMA without the guest and the device cooperation? Does device assignment require MSI(-X) interrupts only (level-triggered are verboten)?

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

18

>We don’t allow level-triggered interrupts, but even that would be possible if we wanted to bother.

I think level-triggered interrupt allow for interrupt storm and the shared interrupt sabotage too easily. They cannot be trusted for an untrusted guest. Actually, I don’t even know how a level-triggered interrupt could even be shared between devices that could belong to different VMs.

19

> response to bus errors, etc. The results were sobering. I’m reasonably

certain that I know how to attack any machine that assigns such a device
to an untrusted VM. And the kicker is that it’s not possible to know when
you’re doing so unless you talk to the vendor of the device and ask them
if their device is insecure, something no vendor I’ve met seems to
understand, let alone admit.

  • Jake Oshins
    (Hyper-V guy)
    Windows Kernel Team

Thank you for that fascinating information. I am going to add it to my
catalog of vulnerabilities.

However, woudn’t an insecure device be readily detected because it is
holding tight to its blanket?
joe

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Mark Roddy
Sent: Saturday, June 1, 2013 4:11 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Querying PCI Config Space.

windows gives any driver in the devnode access to the config space, it
just doesn’t do that for random drivers not in the immediate device stack
of the hardware pdo.

Mark Roddy

On Fri, May 31, 2013 at 5:58 PM, Pavel A.
> wrote:
> On 31-May-2013 15:50, Mark Roddy wrote:
> the correct way to do this is to write a pci bus filter driver that
> attaches to all of the pdos enumerated by pci.sys. You can then,
> carefully, use documented apis to access config space for each device.
>
> One hardware engineer whom I know, did the correct thing:
> When this guy heard that Window will not let driver access
> the config space, he simply added another BAR and mirrored the config
> space to it. That made everyone happy.
> – pa
>
>
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> — NTDEV is sponsored by OSR OSR is HIRING!! See
> http://www.osr.com/careers For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

20

Jake, thank you for very interesting information.

I don’t know SR-IOV enough and don’t understand your
message completely, but from the paranoic POV,
if a rogue device can break out of IOMMU
“jail” on the host side - the driver can stay out of
the game, at least, it can look innocent.
The device (firmware) can do things itself.
– pa

On 03-Jun-2013 19:57, Jake Oshins wrote:

Pavel, it’s more than clear to me why your hardware guy added a BAR that
mirrors config space. I know that Windows and other software stacks
don’t offer direct, unfettered, lightweight access to every single part
of configuration space. I also know that the PCI SIG has defined things
in ways where things that shouldn’t be in configuration space are
required to be there, and that makes the first problem more vexing.

But you all should also know that it’s specifically because of
implementations like the one you describe that we’ve concluded that we
cannot securely assign entire PCIe devices to a Hyper-V guest VM. We
support SR-IOV for networking in Hyper-V, and in order to build that, we
needed to get our hypervisor’s physical interrupt delivery and I/O MMU
code working well enough that we could productively engage the
networking device vendors’ driver teams. So we built whole-device
assignment (not SR-IOV VF assignment,) in the lab, so that we could run
existing device drivers while we tested the lower layers.

The very first device we got working was one that mirrors its config
space into its first memory BAR. (We noticed this entirely
accidentally, by the way. There’s nothing in the PCI spec that would
tell us.) That got us thinking, and we started working through the
threat model for a device where the driver, running in an untrusted VM,
can change its own BAR values, change its own power state, arm its own
wake signal, change its response to bus errors, etc. The results were
sobering. I’m reasonably certain that I know how to attack any machine
that assigns such a device to an untrusted VM. And the kicker is that
it’s not possible to know when you’re doing so unless you talk to the
vendor of the device and ask them if their device is insecure, something
no vendor I’ve met seems to understand, let alone admit.

  • Jake Oshins

(Hyper-V guy)

Windows Kernel Team

*From:*xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] *On Behalf Of *Mark Roddy
*Sent:* Saturday, June 1, 2013 4:11 AM
*To:* Windows System Software Devs Interest List
*Subject:* Re: [ntdev] Querying PCI Config Space.

windows gives any driver in the devnode access to the config space, it
just doesn’t do that for random drivers not in the immediate device
stack of the hardware pdo.

Mark Roddy

On Fri, May 31, 2013 at 5:58 PM, Pavel A. > mailto:xxxxx> wrote:
>
> On 31-May-2013 15:50, Mark Roddy wrote:
>
> the correct way to do this is to write a pci bus filter driver that
> attaches to all of the pdos enumerated by pci.sys. You can then,
> carefully, use documented apis to access config space for each
> device.
>
> One hardware engineer whom I know, did the correct thing:
> When this guy heard that Window will not let driver access
> the config space, he simply added another BAR and mirrored the
> config space to it. That made everyone happy.
> – pa
></mailto:xxxxx>