Comments

• Tim_Roberts Member - All Emails Posts: 14,348

  November 2021

  Why don't you want to read the "interrupt status" register? That's what it's for, for gosh sakes.

  In general, all of the interrupts for a single device are routed to the same CPU, so multiple MSI interrupts should be serialized. I doubt there is any guarantee of that.

  Tim Roberts, [email protected]
  Providenza & Boekelheide, Inc.

  1 · Share on Facebook
• Peter_Viscarola_(OSR) Administrator Posts: 8,996

  November 2021 edited November 2021

  Why don't you want to read the "interrupt status" register

  Because it's a serializing operation. One of the goals of MSI/MSI-X is to make it so that you don't need to touch ANY device registers in your ISR (or, in many cases, any time after device setup/initialization). This avoids the slow-downs due to serialization. In high -speed devices/transfers, like the OP, it's usually one of my goals to not touch any device registers in the ISR. You use the MSI message number (for example) to tell you which set of memory resident data structures you need to process in your ISR (and/or DPC).

  all of the interrupts for a single device are routed to the same CPU

  Hmmmm... really? I've never heard this in all the years I've been writing drivers... and I'm not sure it matches my experience. Windows definitely connects devices to all CPUs... and I was under the impression that the ICH/PCH had "magic" to determine which CPU to which to route the interrupt from a given device (it's in one of the "colored" Intel books, IIRC).

  It's definitely not the case for MSI-X where the device can actually CHOOSE the CPU to which to target the interrupt.

  The message code that we get in the ISR should give me the source of the interrupt, so that I do not need to go and read a status register from the hardware

  Well, yes or no, depending. The prototype for your ISR is:

  BOOLEAN 
  DeviceInterrupt_EvtInterruptIsr(
      _In_ WDFINTERRUPT Interrupt,
      _In_ ULONG MessageID
      )
  

  Right? So, for MSI type interrupts, you just look at the MessageID in your ISR to determine which interrupt is being services. You'll get one Interrupt resource (passed to you in EvtDevicePrepareHardware) and you need to create one WDFINTERRUPT for each message thats assigned to you for that single Interrupt resource (see the u.MessageInterrupt.Raw.MessageCount field of the raw resource descriptor). The ISRs that you connect to this (one) interrupt resource will be called for all your MSIs. You differentiate using the MessageID field.

  Except... you specifically said MSI-X in your title, not MSI.

  For MSI-X things are slightly different. You'll get one interrupt resource (passed to you in EvtDevicePrepareHardware) for each MSI-X interrupt that you're granted. Therefore the ISR that's called will imply which interrupt was generated. You will need to connect these interrupts (calling WdfInterruptCreate) from your EvtDevicePrepareHardware Event Processing Callback.

  The sort of basic documentation for this is here... it could definitely be more helpful, as it leaves a lot of pragmatic things unaddressed. But my comments above should fill those missing things in sufficiently for you.

  Peter

  ETA: Clarify that you need to call WdfInterruptCreate for each MSI, which was not clear in the original post

  Post edited by Peter_Viscarola_(OSR) on November 2021

  Peter Viscarola
  OSR
  @OSRDrivers

  0 · Share on Facebook
• ajitabhs Member Posts: 73

  November 2021

  Thank you peter. This is really helpful. I will go through the documentation on this and will also look at samples. If you can recommend any samples that will be great.

  0 · Share on Facebook
• Peter_Viscarola_(OSR) Administrator Posts: 8,996

  November 2021 edited December 2021

  I'm not sure why you're looking for a sample. There's really nothing to see, particularly in the ISR.

  The only even slightly tricky item is calling WdfInterruptCreate from your EvtDevicePrepareHardware, which is badly described in the docs, but which might want to look something like this:

              //
              // We expect MSI-X resources only.  The device does not fall-back to an LBI. 
              // NOTE: THE CODE BELOW DOES NOT WORK FOR MSI
              //
              case CmResourceTypeInterrupt: {
  
                  MyTracePrint(INFO,"Resource %lu: Interrupt\n", i);
  
                  if(totalInterruptsFound < MY_DEVICE_EXPECTED_NUMBER_OF_INTERRUPTS) {
  
                      resourceRaw = WdfCmResourceListGetDescriptor(Resources, i);
  
                      if(resourceTrans->Flags & CM_RESOURCE_INTERRUPT_MESSAGE) {
  
                          WDF_INTERRUPT_CONFIG interruptConfig;
  
                          MyTracePrint(VERBOSE, "\t\tInt type: MSI/MSI-X\n");
                          MyTracePrint(VERBOSE, "\t\tMessages: %d\n", resourceRaw->u.MessageInterrupt.Raw.MessageCount)
  
                          //
                          // Create WDFINTERRUPT object thereby connecting to the interrupt.
                          //
                          WDF_INTERRUPT_CONFIG_INIT(&interruptConfig,
                                                    MyIsr,
                                                    MyDpcForIsr);
  
                          interruptConfig.EvtInterruptEnable  = MyInterruptEnable;
                          interruptConfig.EvtInterruptDisable = MyInterruptDisable;
                          interruptConfig.InterruptTranslated = resourceTrans;
                          interruptConfig.InterruptRaw        = resourceRaw;
  
                          status = WdfInterruptCreate(devContext->WdfDevice,
                                              &interruptConfig,
                                              WDF_NO_OBJECT_ATTRIBUTES,
                                              &devContext->InterruptObjects[interruptResourcesFound]);
  
                          if(!NT_SUCCESS(status)) {
  
                              MyTracePrint(ERROR, "ConnectInterrupt failed? Status = 0x%lx\n", status);
  
                              goto done;
                          }
  
                      } else {
  
                          MyTracePrint(ERROR, "UNEXPECTED Int type LBI found\n");
  
                          //
                          // Win32: ERROR_INVALID_PARAMETER
                          //
                          status = STATUS_DEVICE_CONFIGURATION_ERROR;
  
                          goto done;
                      }
  
                      interruptResourcesFound++;
  
                      totalInterruptsFound += resourceRaw->u.MessageInterrupt.Raw.MessageCount;
  
                  } else {
  
                      MyTracePrint(INFO,"********UNEXPECTED INTERRUPT RESOURCE FOUND (not used)... %d interrupts already found.\n", totalInterruptsFound);
  
                  }
  
                  break;
              }
  
  

  ETA: Clarify that the code above is intended ONLY for MSI-X, and will not work for MSI (where you need to create on WDFINTERRUPT for each message you receive, unlike in MSI-X where you create one WDFINTERRUPT for each Interrupt Resource you receive)

  Post edited by Peter_Viscarola_(OSR) on December 2021

  Peter Viscarola
  OSR
  @OSRDrivers

  0 · Share on Facebook
• ajitabhs Member Posts: 73

  November 2021

  Thanks Peter. Really appreciate the explanation.

  0 · Share on Facebook