I have a server with 6 dual port FibreChannel HBA’s in it (12 PCIe functions / device instances).
Each device instance can have up to 32 MSI-X interrupt vectors as defined in the HBA’s PCI config space.
When I load the OEM driver for the devices, device manager shows each was assigned 32 IRQs under the resources tab, for a total of 12*32 = 384 MSI-X interrupts among all the instances.
When I load my KMDF driver, 8 of the instances are given 32 vectors, and the remaining 4 are given only 1. This is with no MessageNumberLimit set in the registry.
If I disable 3 of the remaining instances and play around with the MessageNumberLimit on the 9th instance, I find I can get the total number of MSI-X interrupts up to about 273 or so.
The first 8 are assigned 32 and the 9th can have it’s MessageNumberLimit set to 17 and it will be assigned 17 IRQs. If I bump it up to 18 (total of 274), I seem to go over some threshold and it only get’s assigned 1 MSI-X vector.
If I set the MessageNumberLimit to 16 for each device instance, then all 12 instances are assigned 16 MSI-X vectors.
With all of that said, my question is:
Why can the OEM driver have 32 MSI-X vectors assigned to each of the 12 instances but my KMDF driver apparently cannot?
I assume the OEM driver is an older WDM based driver, but I don’t know that for sure.
Is there some sort of limit built in to the KMDF/WDF framework?
Specs:
Server is a SuperMicro with two 8 core Intel Xeon Silver 4215R CPUs.
Motherboard: X11DPX-T
32 GB of RAM
Windows Server 2019.
FibreChannel HBAs are PCIe 3.0
KMDF is 1.19
Thanks for any light you can shed on this discrepancy,
Erik