Interrupts never trigger ISR

I’ve been developing a KMDF driver for a new custom multiport serial device that uses the Xilinx XDMA for DMA. Mostly it’s fine, but for some reason I am completely unable to get an interrupt. I follow all the steps in Microsoft’s guides for creating an ISR DPC, I have compared it to a previous product we developed that works just fine. The interrupt resource shows up appropriately in PrepareHardware, I get no errors in the process at any step that indicates that the interrupt shouldn’t fire. The interrupts trigger just fine in Linux. I had the firmware developer make a temporary register that should cause an interrupt storm when I set a specific bit. I even downloaded some trial software that allowed me to see that interrupts were happening, but the ISR itself never triggers. I have DbgPrints in InterruptEnable and InterruptDisable telling me that the interrupt should be enabled. Nothing I do will cause the ISR to fire.

Here’s the code in DeviceAdd:

WDF_INTERRUPT_CONFIG_INIT(&interruptConfig, newproject_isr, NULL);
interruptConfig.ShareVector = WdfTrue;
interruptConfig.EvtInterruptEnable = EvtWdfInterruptEnable;
interruptConfig.EvtInterruptDisable = EvtWdfInterruptDisable;
WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&interruptAttributes, NEWPROJECT_PORT);

status = WdfInterruptCreate(port->device, &interruptConfig, &interruptAttributes, &port->interrupt);
if (!NT_SUCCESS(status)) {
	TraceEvents(TRACE_LEVEL_ERROR, TRACE_DEVICE, "WdfInterruptCreate failed %x\n", status);
	return 0;
}

At no point in any step of the process do I get an error.

Is there some magic trick I’m overlooking? Some pitfall I may not know about? Everything suggests the isr should fire, and the first thing I do in the isr is a debug print, and I never see a single one.

I’m tearing my hair out here, thanks.

ShareVector = WdfTrue - if this is intentional, then the ISR can be chained at the end of a queue. Is there a device that improperly claims the interrupt ahead of your device? Windows-driver-samples\setup\devcon implements a resources switch that dumps the IRQ:

Windows-driver-samples\setup\devcon\x64\Debug\devcon.exe help resources
Devcon Resources Command
Lists hardware resources of devices with the specified hardware or instance ID.
Valid on local and remote computers.
devcon.exe [-m:\\<machine>] resources <id> [<id>...]
Examples of <id>:
 *              - All devices
 PCI\*          - PCI devices with wildcards  (* matches anything)

That does not appear to be the issue. There are no other devices in the system, and even when I force it to WdfFalse (which only allows one port to be generated) it still behaves the same way. Also, devcon shows only the filter driver and the port drivers sharing the irq (19). Nothing else is using irq 19.

Thanks for the suggestion. Any other ideas?

Standard PCI interrupts (non-MSI) are always shared. The only question I have is about ‘port->device’. Is this a ‘multiport’ device, with a parent device and multiple child devices?

!arbiter 4 + !idt are a starting point.

Here's the output of both. I am unfamiliar with both arbiter and idt, and arbiter seems to have some strange indentation patterns (which I only mention because I'm unsure if it's relevant, if it is I can provide more context). fsccxmc-windows is my driver. It is a multiport device with 6 ports, 3 that will be part of fsccxmc-windows and 3 that will be part of a different driver. Neither of the drivers isrs trigger under any circumstance, and the other driver is currently uninstalled to reduce noise.

!arbiter 4
...
DEVNODE ffffbc05c4d58010 (PCI\VEN_10EE&DEV_7021&SUBSYS_000710EE&REV_00\4&1286464&0&0008)
Interrupt Arbiter "Mf Arbiter" at ffffa98c3c7e97e8
Allocated ranges:
0000000000000000 - 0000000000000012       00000000 
0000000000000013 - 0000000000000013 S
0000000000000013 - 0000000000000013 S     ffffbc05c7a35d40  (fsccxmc-windows)
0000000000000013 - 0000000000000013 S     ffffbc05c7a24d40  (fsccxmc-windows)
0000000000000013 - 0000000000000013 S     ffffbc05c7a13d40  (fsccxmc-windows)
0000000000000014 - ffffffffffffffff       00000000 
Possible allocation:
< none >

!idt:
...
70:	fffff80323415640 serial!SerialCIsrSw (KINTERRUPT ffff87011fb79640)
71:	fffff80323415648 fsccxmc_windows+0x5880 (KMDF) (KINTERRUPT ffff87011fb79280)
fsccxmc_windows+0x5880 (KMDF) (KINTERRUPT ffff87011fb79140)
fsccxmc_windows+0x5880 (KMDF) (KINTERRUPT ffff87011fb79000)
81:	fffff803234156c8 dxgkrnl!DpiFdoMessageInterruptRoutine (KINTERRUPT ffff87012e87fdc0)
...

Here's the output of both. I am unfamiliar with both arbiter and idt, and arbiter seems to have some strange indentation patterns (which I only mention because I'm unsure if it's relevant, if it is I can provide more context). fsccxmc-windows is my driver. It is a multiport device with 6 ports, 3 that will be part of fsccxmc-windows and 3 that will be part of a different driver. Neither of the drivers isrs trigger under any circumstance, and the other driver is currently uninstalled to reduce noise.

!arbiter 4
...
DEVNODE ffffbc05c4d58010 (PCI\VEN_10EE&DEV_7021&SUBSYS_000710EE&REV_00\4&1286464&0&0008)
Interrupt Arbiter "Mf Arbiter" at ffffa98c3c7e97e8
Allocated ranges:
0000000000000000 - 0000000000000012       00000000 
0000000000000013 - 0000000000000013 S
0000000000000013 - 0000000000000013 S     ffffbc05c7a35d40  (fsccxmc-windows)
0000000000000013 - 0000000000000013 S     ffffbc05c7a24d40  (fsccxmc-windows)
0000000000000013 - 0000000000000013 S     ffffbc05c7a13d40  (fsccxmc-windows)
0000000000000014 - ffffffffffffffff       00000000 
Possible allocation:
< none >

!idt:
...
70:	fffff80323415640 serial!SerialCIsrSw (KINTERRUPT ffff87011fb79640)
71:	fffff80323415648 fsccxmc_windows+0x5880 (KMDF) (KINTERRUPT ffff87011fb79280)
fsccxmc_windows+0x5880 (KMDF) (KINTERRUPT ffff87011fb79140)
fsccxmc_windows+0x5880 (KMDF) (KINTERRUPT ffff87011fb79000)
81:	fffff803234156c8 dxgkrnl!DpiFdoMessageInterruptRoutine (KINTERRUPT ffff87012e87fdc0)
...

Neither of those seem to indicate a problem, but I'm not familiar with either. This is a multiport device that has filter inf and 2 drivers, one being fsccxmc-windows (3 ports) and the other is current uninstalled to reduce spam. Neither the ISRs ever seem to trigger, regardless of which I have installed/uninstalled at any given time. I did notice !arbiter 4 has some odd indention. I only mention it because I'm unsure of its relevance. If it's relevant I can provide more context.

Output of !arbiter 4:
...
DEVNODE ffffd4025256e050 (PCI\VEN_10EE&DEV_7021&SUBSYS_000710EE&REV_00\4&1286464&0&0008)
Interrupt Arbiter "Mf Arbiter" at ffffc30d3dfe9368
Allocated ranges:
0000000000000000 - 0000000000000012 00000000
0000000000000013 - 0000000000000013 S
0000000000000013 - 0000000000000013 S ffffd40255f24d40 (fsccxmc-windows)
0000000000000013 - 0000000000000013 S ffffd40255f02d40 (fsccxmc-windows)
0000000000000013 - 0000000000000013 S ffffd40255eefd40 (fsccxmc-windows)
0000000000000014 - ffffffffffffffff 00000000
Possible allocation:
< none >

Output of !idt:
...
70: fffff80540015640 serial!SerialCIsrSw (KINTERRUPT ffff9c010a58b640)
71: fffff80540015648 fsccxmc_windows!fsccxmc_isr (KMDF) (KINTERRUPT ffff9c010a58b280)
fsccxmc_windows!fsccxmc_isr (KMDF) (KINTERRUPT ffff9c010a58b140)
fsccxmc_windows!fsccxmc_isr (KMDF) (KINTERRUPT ffff9c010a58b000)
81: fffff805400156c8 dxgkrnl!DpiFdoMessageInterruptRoutine (KINTERRUPT ffff9c010c1bfdc0)
...

Some low hanging fruits:

  • compare the ports’ PCI Configuration Space between Windows and Linux.
  • !pci 145 <Bus> <Device> <Function> for endpoint, bridge if present.
1 Like

I spent a little time trying to process the configuration space, and had a surprisingly difficult time finding information about what the Status and Command registers represent, but at least those two seem to indicate that there shouldn’t be any issues. I’m thinking the next step might be to implement a function in my driver that uses SetBusData in the BUS_INTERFACE_STANDARD structure to try and forcefully trigger an interrupt. Does that seem appropriate? The firmware dev implemented a register that should do that very thing, but it never seems to trigger the ISR in Windows.

I should mention that we’re currently using the Xilinx vendor/devid, but that will be changed before we ship anything. I figured that was implied, but maybe it was worth saying out loud.

0: kd> !pci 145 1 0 0

PCI Configuration Space (Segment:0000 Bus:01 Device:00 Function:00)
Common Header:
    00: VendorID       10ee Xilinx Corporation
    02: DeviceID       7021
    04: Command        0006 MemSpaceEn BusInitiate 
    06: Status         0010 CapList 
    08: RevisionID     00
    09: ProgIF         01 16450 Compatible
    0a: SubClass       00 Serial Controller
    0b: BaseClass      07 Simple Communications Controller
    0c: CacheLineSize  0010 BurstDisabled 
    0d: LatencyTimer   00
    0e: HeaderType     00
    0f: BIST           00
    10: BAR0           f7a00000
    14: BAR1           f78f0000
    18: BAR2           f7900000
    1c: BAR3           00000000
    20: BAR4           00000000
    24: BAR5           00000000
    28: CBCISPtr       00000000
    2c: SubSysVenID    10ee
    2e: SubSysID       0007
    30: ROMBAR         00000000
    34: CapPtr         40
    3c: IntLine        13
    3d: IntPin         04
    3e: MinGnt         00
    3f: MaxLat         00
Device Private:
    40: 00037001 00000008 00807005 00000000
    50: 00000000 00000000 00000000 00000000
    60: 00007011 00000000 00000000 00000000
    70: 00020010 012c8023 00092900 0043f042
    80: 10420040 00000000 00000000 00000000
    90: 00000000 00000016 00000000 00000006
    a0: 00010002 00000000 00000000 00000000
    b0: 00000000 00000000 00000000 00000000
    c0: 00000000 00000000 00000000 00000000
    d0: 00000000 00000000 00000000 00000000
    e0: 00000000 00000000 00000000 00000000
    f0: 00000000 00000000 00000000 00000000
Capabilities:
    40: CapID          01 PwrMgmt Capability
    41: NextPtr        70
    42: PwrMgmtCap     0003 Version=3
    44: PwrMgmtCtrl    0008 DataScale:0 DataSel:0 D0 

    70: CapID          10 PCI Express Capability
    71: NextPtr        00
    72: Express Caps   0002 (ver. 2) Type:Endpoint
    74: Device Caps    012c8023
    78: Device Control 2900 bcre/flr MRR:512 NS ap pf ET MP:128 ro ur fe nf ce
    7a: Device Status  0009 tp ap UR fe nf CE 
    7c: Link Caps      0043f042
    80: Link Control   0040 es CC rl ld RCB:64 ASPM:None 
    82: Link Status    1042 SCC lt lte NLW:x4 LS:2.5 
    94: DeviceCaps2    00000016 CTR:6 CTDIS arifwd aor aoc32 aoc64 cas128 noro ltr TPH:0 OBFF:0 extfmt eetlp EETLPMax:0
    98: DeviceControl2 0000 CTVal:0 ctdis arifwd aor aoeb idoreq idocom ltr OBFF:0 eetlp
    84: Slot Caps      00000000 
    88: Slot Control   0000 pcc PI:?? AI:?? hpi cc pde mrls pfd ab 
    8a: Slot Status    0000 dlsc eis pds hpi cc pdc ms pfd ab 
    8c: Root Control   0000 pmei fs nfs cs 
    8e: Root Caps      0000 
    90: Root Status    00000000 pmep pmes ID:0 

Enhanced Capabilities:
    100: CapID         0001 Advanced Error Reporting Capability
         Version       1
         NextPtr       1f0
    104: UncorrectableErrorStatus   00000000  dlpe sde ptlp fcpe ct ca mtlp ecrc ur acsv uie mcbtlp aeb tpb
    108: UncorrectableErrorMask     00400000  dlpe sde ptlp fcpe ct ca mtlp ecrc ur acsv UIE mcbtlp aeb tpb
    10c: UncorrectableErrorSeverity 00462030  DLPE SDE ptlp FCPE ct ca MTLP ecrc ur acsv UIE mcbtlp aeb tpb
    110: CorrectableErrorStatus     00002000  re btlp bdllp rnr rtt ANFE cie hlo
    114: CorrectableErrorMask       0000e000  re btlp bdllp rnr rtt ANFE CIE HLO
    118: CapabilitiesAndControl     00000000  FirstErr:0 ecrcgcap ecrcgen ecrcccap ecrccen mhrcap mhren tplp
    11c: HeaderLog:    00 00 00 00  00 00 00 00  00 00 00 00  00 00 00 00  
         Fmt           0 (3 DW Header, no data)
         Type          0 (MRd)
         Length        0 (0 bytes)
         RID           0:0:0
         Tag           0
         Attributes    0 (Default Ordering)
         Traffic Class 0
         Address       00000000

    1f0: CapID         0002 Virtual Channel Capability
         Version       1
         NextPtr       000
    01f4: Port VC Capability 1        00000000
    01f8: Port VC Capability 2        31000000
    01fc: Port VC Control             0000
    01fe: Port VC Status              0000
    0200: VC Resource[0] Cap          00000000
    0204: VC Resource[0] Control      80000001
    020a: VC Resource[0] Status       0000