How MSI address register gets set in PCIe devices?

NTDEV
1

Hi all,
I am writing a Windows driver in KMDF (windows 7) for 2 devices. One is for a Pci express storage device(device 1) and the other for Altera Stratix 4 PCI express board(device 2).

For Device 1 (Pci express storage device) :

When I set the Interrupt disable bit in command register(Disables legacy interrupts) and MSI Enable bit in the MSI capabilities register from my driver and then try to install the driver, I can see that the system is setting the MSI address register in the protocol analyzer(Pcie protocol analyzer was switched on during the driver installation). As expected, I am getting the MSI data to the address specified by the system.

I followed the same procedure of setting the MSI for Altera Stratix 4 Pci express board.

For Device 2(Altera Stratix 4 PCI express board):

I cannot see the MSI address register getting set in Protocol analyzer (Pcie protocol analyzer was switched on during the driver installation). MSI data(00000000h) was being sent to a default location(00000000h). Now I tried writing some valid physical address and data into the MSI address and MSI data registers, I was getting the MSI data to the address location specified by me.

My Questions:

  1. Who is setting the MSI address register in case of device 1?
  2. If Msi address register is being set by the system in device 1, why is the same setting not happening for device 2(Altera Stratix 4 PCI express board)?
2

As a general rule, you should consider any register defined by the PCI
specification in configuration space, either in the common header or in
capability structure, as owned by the Windows PCI driver. Don’t touch.
There are a couple of minor exceptions to this. But setting these things in
your driver will put the hardware out of synch with the things the PCI
driver is doing on the rest of the bus, and to your device. It may update
the very same registers you just changed with their old values, for
instance.

In this case, the entire interrupt management mechanisms are managed by
pci.sys. If your INF says that you want MSI, they’ll be enabled if and only
if the processor and surrounding chipset also support them (and if you’re
using Vista or later – Server 2003 and earlier simply can’t support MSI.)

Jake Oshins
Windows Kernel Team

The message offers no warranties and confers no rights.

wrote in message news:xxxxx@ntdev…

Hi all,
I am writing a Windows driver in KMDF (windows 7) for 2 devices. One is for
a Pci express storage device(device 1) and the other for Altera Stratix 4
PCI express board(device 2).

For Device 1 (Pci express storage device) :

When I set the Interrupt disable bit in command register(Disables legacy
interrupts) and MSI Enable bit in the MSI capabilities register from my
driver and then try to install the driver, I can see that the system is
setting the MSI address register in the protocol analyzer(Pcie protocol
analyzer was switched on during the driver installation). As expected, I am
getting the MSI data to the address specified by the system.

I followed the same procedure of setting the MSI for Altera Stratix 4 Pci
express board.

For Device 2(Altera Stratix 4 PCI express board):

I cannot see the MSI address register getting set in Protocol analyzer (Pcie
protocol analyzer was switched on during the driver installation). MSI
data(00000000h) was being sent to a default location(00000000h). Now I tried
writing some valid physical address and data into the MSI address and MSI
data registers, I was getting the MSI data to the address location specified
by me.

My Questions:

  1. Who is setting the MSI address register in case of device 1?
  2. If Msi address register is being set by the system in device 1, why is
    the same setting not happening for device 2(Altera Stratix 4 PCI express
    board)?
3

@Jake Oshins

  1. The INF file settings for both the devices with respect to MSI capabilities are same and I have presented few lines of the settings made in the INF file below:

HKR,MSICapablity, 0x00010001, 0x0f
HKR, ?Interrupt Management?, 0x00000010
HKR, ?Interrupt Management\MessageSignaledInterruptProperties?, 0x00000010
HKR, ?Interrupt Management\MessageSignaledInterruptProperties?, MSISupported, 0x00010001, 1
HKR, ?Interrupt Management\MessageSignaledInterruptProperties?, MessageNumberLimit, 0x00010001,8
HKR, ?Interrupt Management\Affinity Policy?, 0x00000010
HKR, ?Interrupt Management\Affinity Policy?, DevicePolicy, 0x00010001, 0x02.

  1. I tried installing both the devices on the same processor and also the OS in use is windows 7. But the system is setting the MSI address register for device 1(PCI express storage device), but not for device 2 (Altera Stratix 4 PCI express board).

Since the INF file settings with respect to MSI, processor and OS in use are same for both the devices, do I have to follow some different techniques in case of Altera stratix 4 Pci express board so that the MSI address register is set by the system??

Thanks

4

You say you installed them on the same “processor.” Do you mean two
different machines with the same type of processor? If so, try swapping
them and see whether the problem travels with the adapter or with the
motherboard. If it travels with the adapter, look at configuration space
for the device. You’ve got a problem somewhere.

Jake Oshins
Windows Kernel Team

The message offers no warranties and confers no rights.

wrote in message news:xxxxx@ntdev…

@Jake Oshins

  1. The INF file settings for both the devices with respect to MSI
    capabilities are same and I have presented few lines of the settings made in
    the INF file below:

HKR,MSICapablity, 0x00010001, 0x0f
HKR, ?Interrupt Management?, 0x00000010
HKR, ?Interrupt Management\MessageSignaledInterruptProperties?, 0x00000010
HKR, ?Interrupt Management\MessageSignaledInterruptProperties?,
MSISupported, 0x00010001, 1
HKR, ?Interrupt Management\MessageSignaledInterruptProperties?,
MessageNumberLimit, 0x00010001,8
HKR, ?Interrupt Management\Affinity Policy?, 0x00000010
HKR, ?Interrupt Management\Affinity Policy?, DevicePolicy, 0x00010001, 0x02.

  1. I tried installing both the devices on the same processor and also the OS
    in use is windows 7. But the system is setting the MSI address register for
    device 1(PCI express storage device), but not for device 2 (Altera Stratix 4
    PCI express board).

Since the INF file settings with respect to MSI, processor and OS in use are
same for both the devices, do I have to follow some different techniques in
case of Altera stratix 4 Pci express board so that the MSI address register
is set by the system??

Thanks

5

@Jake Oshins:
I meant that I worked on the same machines.

Another piece of information is that I dumped the configuration space of both the devices after driver installation.
For device 1 (PCI express storage device), in the MSI address register field I could see the same physical address (set by the system) as that seen in the protocol analyzer.

For device 2 (Altera Stratix 4 PCI express board), in the MSI address register field I could see 00000000h. If I explicitly set the MSI address, then I can see the address set by me in that MSI address field. Is this non-setting of the MSI field in case of device 2 related to any additional settings that I am missing from the configuration space side?

6

Is the PCIe core which you are using in Altera Stratix 4 PCI express board supports MSI and is it enabled ? Please check that.

7

@Amogha varsha:

Yes the PCIe core for Altera Stratix 4 PCIe board supports MSI and the MSI related settings are enabled while generating the IP. Is there any setting in the configuration space or any other settings from the PC side which tells the system to set MSI address register in the configuration space?

8

Arvind,

Enabling of MSI in hw and specifying in driver inf files are sufficient.

One thing I would like to confirm is what is the setting in messagecontrol.multiplemessagecapable in MSI capability structure of pci config space.

Another thing you can try is by giving less number of MessageNumberLimit in driver inf file. For example 4.

9

If the processor supports MSI (as exposed by the datasheet, identified
through CPUID,) the chipset supports MSI (as exposed by an internal errata
list,) the driver supports MSI (as exposed by the registry, set through the
INF) and the hardware supports either MSI or MSI-X (as exposed through the
capability structure list in config space) then the OS will program those
registers.

Jake Oshins
Windows Kernel Team

The message offers no warranties and confers no rights.

wrote in message news:xxxxx@ntdev…

@Amogha varsha:

Yes the PCIe core for Altera Stratix 4 PCIe board supports MSI and the MSI
related settings are enabled while generating the IP. Is there any setting
in the configuration space or any other settings from the PC side which
tells the system to set MSI address register in the configuration space?

10

@Amoghavrsha:

messagecontrol.multiplemessagecapable in MSI capability structure of pci config space is set to 4. Message number limit as of now is 8. I will try to reduce it to 4 and check. Will this have any bearing on MSI address register setting?

11

No.

@Jake, in the current scenario, the pcie device is requesting for 16 messages and the driver inf says the limit as 8 messages so which will be considered for allocation by the bus driver?

@Arvind,

I think the requested number of messages is not allocated by the pci bus driver and hence MSI is not enabled.

12

@Amoghavarsha:

I meant that the device is requesting 4 MSIs (multiple message capable field is set to “010”). And the Message number limit is set to 8. Do you see any problem now?

13

Thats the same question which I have just asked Jake in my previous post.

I think device is capable of 4 messages and you are requesting for more (8 messages) and hence the request fails and MSI is not enabled.

14

@Amogha varsha:

Device 1 (PCIe express storage device)----------->Working case (MSI address register is getting set by the system)

MultipleMessageCapable = 16 MSIs.
MultipleMessageEnable = 8 MSIs.

Message Number Limit = 8 (inf file)

Device 2 (Altera Stratix 4 Pcie express board)----------->NON-Working case (MSI address register is NOT getting set by the system)

MultipleMessageCapable = 4 MSIs.
MultipleMessageEnable = 1 MSI.

Message Number Limit = 8 (inf file)

Do you see any problem with the MSI capability register settings in both the devices?