I’ve recently had a hardware vendor make an assertion that I dispute,
but before I make a fool of myself with them, I’d like to do so here. 
This vendor is asserting that MSI interrupts in PCIe are queued up.
That is, if he fires three MSI interrupts before my ISR gets a chance to
run, it won’t matter, because my ISR will always be fired three times.
Is that possibly correct? With standard PCI interrupts, that wasn’t the
case. If a second interrupt came in while the first was pending, there
was no way to know. You’re asserting a signal that was already asserted.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Tim,
I am far from being an expert on MSI, but I am afraid you just misunderstood one another…
Don’t forget that one of the basic concepts of MSI is to allow the same device to have multiple ISRs
that are serviced via the different vectors. It is impossible under the legacy model, because interrupts are signaled via a pin with a strict device-pin-GSI - interrupt vector correspondence. Therefore, ISR has to check the reason why device had requested CPU’s attention. However, once MSI is just a memory write, you can have multiple ISRs, i.e. one ISR for data arrival, another ISR for DMA completion, etc for the same device. These ISRs may be serviced via different vectors, and, hence, will be treated as interrupts that are totally independent from one another. In this sense, the vendor is correct - if the same device has raised 3 independent interrupts for 3 different reasons and the target device’s driver has multiple ISRs , all 3 ISRs will be invoked…
If device is serviced by a single interrupt vector, there is a (slim) chance of the second interrupt firing while the first one is being serviced. This behavior is, indeed, consistent with the semantics of edge-triggered interrupts. Therefore, it is the Os’s responsibility to ensure that ISR does not re-enter itself. I would rather refer you to the Linux sources (namely, to handle_edge_irq()) here…
if he fires three MSI interrupts before my ISR gets a chance to run, it won’t matter, because my
ISR will always be fired three times
Well, the above behavior seems to simply defeat the very definition of edge-triggered interrupts, right…
Anton Bassov
Queued where? Maybe in the PCIe interrupt code in his FPGA?
But three identical messages fired at the root complex being buffered before the ISR gets to run, and then the ISR running three successive times? Gee, I doubt it. But I’m not close to being an expert on Express either. How would the root complex size the buffer for this?
You’ve checked the Express spec, I assume?
You’ve got me curious now. Please be sure to post back here if you find the answer off list.
Peter
OSR
@OSRDrivers
Ha!
At the bottom of page 263 of the book PCI System Architecture by Anderson and Shanley, item 10:
“If the device writes the same message multiple times, only one of those messages is guaranteed to be serviced. If all of them must be serviced, the device must not generate the same message again until the driver services the earlier one,”
That’s reasonably close to definitive, given many people read Anderson and Shanley but never read the actual spec.
Peter
OSR
@OSRDrivers
Also see MSDN at http://msdn.microsoft.com/en-us/library/windows/hardware/ff547940(v=vs.85).aspx
…
Note that if the system receives multiple identical interrupts over a short time interval, it can combine these into a single call to InterruptMessageService. The routine must be written to handle multiple identical interrupts within a single call.
…
So, one way or the other your client needs to rethink their hardware design.
Peter
OSR
@OSRDrivers