Re: WDF, multiple MSI/MSIX interrupts, and hibernate/rebalance.

The documenation says to create the WDFINTERRUPT objects in
EvtDriverDeviceAdd. It was my understanding that they need to be
present before PrepareHardware. It’s been a while since I had access
to the device I was using for MSI (never got it working at that time
due to a system issue) so I could be remembering that last part wrong.

Beverly

On 1/11/07, Doron Holan wrote:
> I have to read your email later in greater detail (and hopefully Jake
> reads it as well), but let me clear up a couple of low hanging pieces of
> fruit
>
> 1) you should enumerate the WDFINTERRUPTs connect info in
> EvtDevicePrepareHardware. The connect info will remain the same until
> EvtDeviceReleaseHardware, so there is no need to evaluate them each time
> you enter D0.
>
> 2) Hibernate/resume will not cause rebalance, so the info reported will
> not change in this case. Like I pointed out in 1),
> EvtDeviceReleaseHardware will be called when the interrupts information
> is “released” (during a rebalance for instance) and then
> EvtDevicePrepareHardware will called again with the new connect
> information.
>
> d
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Martin Harvey
> Sent: Thursday, January 11, 2007 9:10 AM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] WDF, multiple MSI/MSIX interrupts, and
> hibernate/rebalance.
>
> Hi folks,
>
> The quick question:
>
> What is the approved method for enumerating the WDF_INTERRUPT_INFO
> structures and working out which (of several) WDFINTERRUPT’s is actually
>
> connected? After a hibernate/rebalance, if MSI interrupts have fallen
> back to using fewer message numbers than before, the actual WDFINTERRUPT
>
> that gets connected appears to be arbitrary.
>
> The detailed version:
>
> I’ve recently been doing WHQL testing on a driver I’ve written and we’ve
>
> encountered problems with some of the test cases indicating that I’m not
>
> initialising or using interrupts correctly. I’ve got a good idea of
> what’s wrong, but I need some help to correctly resolve the bug, and do
> the right thing.
>
> The hardware is a PCIe card, which supports 4 MSI interrupts, but it can
>
> also fall back to legacy / line based interrupts.
>
> - I’m using WdfInterruptCreate in EvtDeviceAdd.
> - In EvtD0Entry I’m then iterating through the interrupts using
> WdfInterruptGetInfo, trying to work out how many are connected, and what
>
> sort they are.
>
> - The problem we’re seeing is: On the first interrupt we get after
> resuming (after hibernate or rebalance), the data in our interrupt
> context leads us (via some pointers) into our own datastructures, but
> these pointers are stale (or wrong), and everything falls apart. I think
>
> this is because I’m making silly assumptions about which if the
> WDFINTERRUPT objects is actually connected.
>
> I’ve seen this problem:
>
> 1. On PnP rebalance on a Vista machine supporting MSI interrupts.
> 2. Resuming from a hibernate on a 2k3 machine which does not support MSI
>
> interrupts, where the driver is running several instances of our device.
>
> What I think is happening:
>
> - On initial device addition and start, we see two cases:
>
> 1) Line based interrupts. I create 4 WDFINTERRUPT objects OK, and then
> in d0_entry, I query the WDF_INTERRUPT_INFO, and see that either:
>
> a) We get 4 message signalled interrupts with different message numbers,
>
> and can use all 4.
> b) If the O.S. does not support it, we see all the interrupt objects
> indicating not message signalled.
>
> in case b) I assume the interrupt that is actually connected is the
> first.
>
> I am aware that there is a case where you ask for N MSI interrupts and
> only get one,
>
> - On rebalance / hibernate it gets more complicated:
>
> On resumption. I do the same query process, and sometimes see
>
> 1. MSI interrupts where they are all set to have the same message
> number.
> 2. For line based interrupts the interrupt vector is set to be the same
> for all the interrupts.
>
> I think the bad assumption I’m making is that in both of these cases,
> I assume “OK, we’ve had to fall back to only one interrupt (MSI or Line
> based) it must be the first one”, and then I assume that the
> WDFINTERRUPT that is actually connected is going to be the first one I
> originally asked for, and hence I only end up re-initializing the
> interrupt context for that first WDFINTERRUPT object, and then get a
> nasty surprise when the 3rd (or nth) in my array of WDFINTERRUPTS fires
> off instead (complete with stale context from before the
> hibernate/rebalance!)
>
> What is the approved method for enumerating the WDF_INTERRUPT_INFO
> structures?
>
> MH.
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>

Yes, you create them in AddDevice. You look at (or enumerate in the
response below) the assigned resources in PrepareHardware

d

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Beverly Brown
Sent: Thursday, January 11, 2007 10:03 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] WDF, multiple MSI/MSIX interrupts, and
hibernate/rebalance.

The documenation says to create the WDFINTERRUPT objects in
EvtDriverDeviceAdd. It was my understanding that they need to be
present before PrepareHardware. It’s been a while since I had access
to the device I was using for MSI (never got it working at that time
due to a system issue) so I could be remembering that last part wrong.

Beverly

On 1/11/07, Doron Holan wrote:
> I have to read your email later in greater detail (and hopefully Jake
> reads it as well), but let me clear up a couple of low hanging pieces
of
> fruit
>
> 1) you should enumerate the WDFINTERRUPTs connect info in
> EvtDevicePrepareHardware. The connect info will remain the same until
> EvtDeviceReleaseHardware, so there is no need to evaluate them each
time
> you enter D0.
>
> 2) Hibernate/resume will not cause rebalance, so the info reported
will
> not change in this case. Like I pointed out in 1),
> EvtDeviceReleaseHardware will be called when the interrupts
information
> is “released” (during a rebalance for instance) and then
> EvtDevicePrepareHardware will called again with the new connect
> information.
>
> d
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Martin Harvey
> Sent: Thursday, January 11, 2007 9:10 AM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] WDF, multiple MSI/MSIX interrupts, and
> hibernate/rebalance.
>
> Hi folks,
>
> The quick question:
>
> What is the approved method for enumerating the WDF_INTERRUPT_INFO
> structures and working out which (of several) WDFINTERRUPT’s is
actually
>
> connected? After a hibernate/rebalance, if MSI interrupts have fallen
> back to using fewer message numbers than before, the actual
WDFINTERRUPT
>
> that gets connected appears to be arbitrary.
>
> The detailed version:
>
> I’ve recently been doing WHQL testing on a driver I’ve written and
we’ve
>
> encountered problems with some of the test cases indicating that I’m
not
>
> initialising or using interrupts correctly. I’ve got a good idea of
> what’s wrong, but I need some help to correctly resolve the bug, and
do
> the right thing.
>
> The hardware is a PCIe card, which supports 4 MSI interrupts, but it
can
>
> also fall back to legacy / line based interrupts.
>
> - I’m using WdfInterruptCreate in EvtDeviceAdd.
> - In EvtD0Entry I’m then iterating through the interrupts using
> WdfInterruptGetInfo, trying to work out how many are connected, and
what
>
> sort they are.
>
> - The problem we’re seeing is: On the first interrupt we get after
> resuming (after hibernate or rebalance), the data in our interrupt
> context leads us (via some pointers) into our own datastructures, but
> these pointers are stale (or wrong), and everything falls apart. I
think
>
> this is because I’m making silly assumptions about which if the
> WDFINTERRUPT objects is actually connected.
>
> I’ve seen this problem:
>
> 1. On PnP rebalance on a Vista machine supporting MSI interrupts.
> 2. Resuming from a hibernate on a 2k3 machine which does not support
MSI
>
> interrupts, where the driver is running several instances of our
device.
>
> What I think is happening:
>
> - On initial device addition and start, we see two cases:
>
> 1) Line based interrupts. I create 4 WDFINTERRUPT objects OK, and then
> in d0_entry, I query the WDF_INTERRUPT_INFO, and see that either:
>
> a) We get 4 message signalled interrupts with different message
numbers,
>
> and can use all 4.
> b) If the O.S. does not support it, we see all the interrupt objects
> indicating not message signalled.
>
> in case b) I assume the interrupt that is actually connected is the
> first.
>
> I am aware that there is a case where you ask for N MSI interrupts and
> only get one,
>
> - On rebalance / hibernate it gets more complicated:
>
> On resumption. I do the same query process, and sometimes see
>
> 1. MSI interrupts where they are all set to have the same message
> number.
> 2. For line based interrupts the interrupt vector is set to be the
same
> for all the interrupts.
>
> I think the bad assumption I’m making is that in both of these
cases,
> I assume “OK, we’ve had to fall back to only one interrupt (MSI or
Line
> based) it must be the first one”, and then I assume that the
> WDFINTERRUPT that is actually connected is going to be the first one I
> originally asked for, and hence I only end up re-initializing the
> interrupt context for that first WDFINTERRUPT object, and then get a
> nasty surprise when the 3rd (or nth) in my array of WDFINTERRUPTS
fires
> off instead (complete with stale context from before the
> hibernate/rebalance!)
>
> What is the approved method for enumerating the WDF_INTERRUPT_INFO
> structures?
>
> MH.
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer