Interrupt not received

NTDEV
1

Hi,

I am developing a device driver for an Altera Arria GX II FPGA (PCI-e).
A driver I have developed for linux seems to work without any problems on all the tested computers.

However, when developing a Windows 7 x64 driver with KMDF, I seem to have some problems.

We use a MSI-interrupt (edge-triggered) and its purpose is to let us know when a DMA transaction is complete.

Though, the driver works perfectly on two amd64 processor systems, it does not work very well on Intel core i5 systems (two computers tested). We have also noticed that some interrupts may be delayed by a minute or two on a third amd64 system.

On the Intel core i5 system:
When testing, its waiting after the first DMA transaction(using ReadFile/WriteFile function in application). The DMA works fine and the DMA register on the HW shows both that DMA transaction is done and that the interrupt has fired, but the ISR routine is not triggered.

When developing the driver I have looked at AMCC5933 and Pcidrv sample files from WinDDK.

My configuration looks like this:

EvtDeviceAdd:

// Configure Interrupt object
WDF_INTERRUPT_CONFIG_INIT(&interruptConfig,
EvtInterruptIsr,
EvtInterruptDpc);

interruptConfig.EvtInterruptEnable = EvtInterruptEnable;
interruptConfig.EvtInterruptDisable = EvtInterruptDisable;

status = WdfInterruptCreate(devExt->Device,
&interruptConfig,
WDF_NO_OBJECT_ATTRIBUTES,
&devExt->Interrupt);

EvtInterruptEnable:
Enabling Interrupt in DMA register.
EvtInterruptDisable:
Disabling Interrupt in DMA register.
EvtInterruptIsr:
Checks DMA register if we have interrupt, if so, we clear it and signals DPC routine. return TRUE, otherwise return FALSE.

Any guidance appreciated.

2

> … DMA register on the HW shows both that DMA transaction is done
and that the interrupt has fired, but the ISR routine is not triggered.

Do you check for this in your ISR, or using a bus sniffer?
Is what you see that the ISR is called, but DPC delayed?
– pa

On 03-Nov-2011 17:16, xxxxx@limsim.com wrote:

Hi,

I am developing a device driver for an Altera Arria GX II FPGA (PCI-e).
A driver I have developed for linux seems to work without any problems on all the tested computers.

However, when developing a Windows 7 x64 driver with KMDF, I seem to have some problems.

We use a MSI-interrupt (edge-triggered) and its purpose is to let us know when a DMA transaction is complete.

Though, the driver works perfectly on two amd64 processor systems, it does not work very well on Intel core i5 systems (two computers tested). We have also noticed that some interrupts may be delayed by a minute or two on a third amd64 system.

On the Intel core i5 system:
When testing, its waiting after the first DMA transaction(using ReadFile/WriteFile function in application). The DMA works fine and the DMA register on the HW shows both that DMA transaction is done and that the interrupt has fired, but the ISR routine is not triggered.

When developing the driver I have looked at AMCC5933 and Pcidrv sample files from WinDDK.

My configuration looks like this:

EvtDeviceAdd:

// Configure Interrupt object
WDF_INTERRUPT_CONFIG_INIT(&interruptConfig,
EvtInterruptIsr,
EvtInterruptDpc);

interruptConfig.EvtInterruptEnable = EvtInterruptEnable;
interruptConfig.EvtInterruptDisable = EvtInterruptDisable;

status = WdfInterruptCreate(devExt->Device,
&interruptConfig,
WDF_NO_OBJECT_ATTRIBUTES,
&devExt->Interrupt);

EvtInterruptEnable:
Enabling Interrupt in DMA register.
EvtInterruptDisable:
Disabling Interrupt in DMA register.
EvtInterruptIsr:
Checks DMA register if we have interrupt, if so, we clear it and signals DPC routine. return TRUE, otherwise return FALSE.

Any guidance appreciated.

3

Hi,
The ISR is not called, I test this by reading temporarily from the
DMA-register in a for-loop after activating DMA transaction in the
ProgramDma function that is triggered after calling
WdfDmaTransactionExecute.(I trigger the WdfDmaTransactionExecute
function which starts the ProgramDma function, this fills the
DMA-register with the needed data: addresses,length and
control-start-bit, which starts the operation. Before returning TRUE, I
have inserted this small for-loop to check that the DMA transaction
actually begin, and it does.)

This all happens before the ISR is supposed to be triggered.
I could of course do this test with the NIOS II, by looping a command in
the HW to see that that specific register is behaving like that.

On Thu, 2011-11-03 at 19:24 +0200, Pavel A wrote:

> … DMA register on the HW shows both that DMA transaction is done
and that the interrupt has fired, but the ISR routine is not triggered.

Do you check for this in your ISR, or using a bus sniffer?
Is what you see that the ISR is called, but DPC delayed?
– pa

On 03-Nov-2011 17:16, xxxxx@limsim.com wrote:
> Hi,
>
> I am developing a device driver for an Altera Arria GX II FPGA (PCI-e).
> A driver I have developed for linux seems to work without any problems on all the tested computers.
>
> However, when developing a Windows 7 x64 driver with KMDF, I seem to have some problems.
>
> We use a MSI-interrupt (edge-triggered) and its purpose is to let us know when a DMA transaction is complete.
>
> Though, the driver works perfectly on two amd64 processor systems, it does not work very well on Intel core i5 systems (two computers tested). We have also noticed that some interrupts may be delayed by a minute or two on a third amd64 system.
>
> On the Intel core i5 system:
> When testing, its waiting after the first DMA transaction(using ReadFile/WriteFile function in application). The DMA works fine and the DMA register on the HW shows both that DMA transaction is done and that the interrupt has fired, but the ISR routine is not triggered.
>
> When developing the driver I have looked at AMCC5933 and Pcidrv sample files from WinDDK.
>
> My configuration looks like this:
>
> EvtDeviceAdd:
>
> // Configure Interrupt object
> WDF_INTERRUPT_CONFIG_INIT(&interruptConfig,
> EvtInterruptIsr,
> EvtInterruptDpc);
>
> interruptConfig.EvtInterruptEnable = EvtInterruptEnable;
> interruptConfig.EvtInterruptDisable = EvtInterruptDisable;
>
> status = WdfInterruptCreate(devExt->Device,
> &interruptConfig,
> WDF_NO_OBJECT_ATTRIBUTES,
> &devExt->Interrupt);
>
> EvtInterruptEnable:
> Enabling Interrupt in DMA register.
> EvtInterruptDisable:
> Disabling Interrupt in DMA register.
> EvtInterruptIsr:
> Checks DMA register if we have interrupt, if so, we clear it and signals DPC routine. return TRUE, otherwise return FALSE.
>
>
> Any guidance appreciated.
>

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

4

Okay, I just made some interesting discoveries.

When I am waiting for an interrupt (with a simple read/write single
DMA-packet test-program), reinstalling or updating a random
driver/chipset (either deliberate or by Windows update) will make my
interrupt appear and ISR routine is fired. This successfully release the
program from the device. But at the same time I may get a couple of
“ghost” interrupts as well, probably depending on how long I have waited
for the interrupt. (The driver responds with return FALSE in ISR to
these “ghost” interrupts. Is this behavior correct?) However, I have not
yet been able to figure out what causes the interrupt from not being
pointed to my drivers ISR routine, as it does on a AMD64 platform.

–pfbach

On Fri, 2011-11-04 at 02:19 +0100, Per Ferdinand Bach wrote:

Hi,
The ISR is not called, I test this by reading temporarily from the
DMA-register in a for-loop after activating DMA transaction in the
ProgramDma function that is triggered after calling
WdfDmaTransactionExecute.(I trigger the WdfDmaTransactionExecute
function which starts the ProgramDma function, this fills the
DMA-register with the needed data: addresses,length and
control-start-bit, which starts the operation. Before returning TRUE, I
have inserted this small for-loop to check that the DMA transaction
actually begin, and it does.)

This all happens before the ISR is supposed to be triggered.
I could of course do this test with the NIOS II, by looping a command in
the HW to see that that specific register is behaving like that.

On Thu, 2011-11-03 at 19:24 +0200, Pavel A wrote:
> > … DMA register on the HW shows both that DMA transaction is done
> and that the interrupt has fired, but the ISR routine is not triggered.
>
> Do you check for this in your ISR, or using a bus sniffer?
> Is what you see that the ISR is called, but DPC delayed?
> – pa
>
>
> On 03-Nov-2011 17:16, xxxxx@limsim.com wrote:
> > Hi,
> >
> > I am developing a device driver for an Altera Arria GX II FPGA (PCI-e).
> > A driver I have developed for linux seems to work without any problems on all the tested computers.
> >
> > However, when developing a Windows 7 x64 driver with KMDF, I seem to have some problems.
> >
> > We use a MSI-interrupt (edge-triggered) and its purpose is to let us know when a DMA transaction is complete.
> >
> > Though, the driver works perfectly on two amd64 processor systems, it does not work very well on Intel core i5 systems (two computers tested). We have also noticed that some interrupts may be delayed by a minute or two on a third amd64 system.
> >
> > On the Intel core i5 system:
> > When testing, its waiting after the first DMA transaction(using ReadFile/WriteFile function in application). The DMA works fine and the DMA register on the HW shows both that DMA transaction is done and that the interrupt has fired, but the ISR routine is not triggered.
> >
> > When developing the driver I have looked at AMCC5933 and Pcidrv sample files from WinDDK.
> >
> > My configuration looks like this:
> >
> > EvtDeviceAdd:
> >
> > // Configure Interrupt object
> > WDF_INTERRUPT_CONFIG_INIT(&interruptConfig,
> > EvtInterruptIsr,
> > EvtInterruptDpc);
> >
> > interruptConfig.EvtInterruptEnable = EvtInterruptEnable;
> > interruptConfig.EvtInterruptDisable = EvtInterruptDisable;
> >
> > status = WdfInterruptCreate(devExt->Device,
> > &interruptConfig,
> > WDF_NO_OBJECT_ATTRIBUTES,
> > &devExt->Interrupt);
> >
> > EvtInterruptEnable:
> > Enabling Interrupt in DMA register.
> > EvtInterruptDisable:
> > Disabling Interrupt in DMA register.
> > EvtInterruptIsr:
> > Checks DMA register if we have interrupt, if so, we clear it and signals DPC routine. return TRUE, otherwise return FALSE.
> >
> >
> > Any guidance appreciated.
> >
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

5

What else is on that interrupt? Depending on where you are in the chain, your ISR could be called during the processing of anothers interrupt.

Gary Little
H (952) 223-1349
C (952) 454-4629
xxxxx@comcast.net

On Nov 4, 2011, at 12:44 PM, Per Ferdinand Bach wrote:

Okay, I just made some interesting discoveries.

When I am waiting for an interrupt (with a simple read/write single
DMA-packet test-program), reinstalling or updating a random
driver/chipset (either deliberate or by Windows update) will make my
interrupt appear and ISR routine is fired. This successfully release the
program from the device. But at the same time I may get a couple of
“ghost” interrupts as well, probably depending on how long I have waited
for the interrupt. (The driver responds with return FALSE in ISR to
these “ghost” interrupts. Is this behavior correct?) However, I have not
yet been able to figure out what causes the interrupt from not being
pointed to my drivers ISR routine, as it does on a AMD64 platform.

–pfbach

On Fri, 2011-11-04 at 02:19 +0100, Per Ferdinand Bach wrote:
> Hi,
> The ISR is not called, I test this by reading temporarily from the
> DMA-register in a for-loop after activating DMA transaction in the
> ProgramDma function that is triggered after calling
> WdfDmaTransactionExecute.(I trigger the WdfDmaTransactionExecute
> function which starts the ProgramDma function, this fills the
> DMA-register with the needed data: addresses,length and
> control-start-bit, which starts the operation. Before returning TRUE, I
> have inserted this small for-loop to check that the DMA transaction
> actually begin, and it does.)
>
> This all happens before the ISR is supposed to be triggered.
> I could of course do this test with the NIOS II, by looping a command in
> the HW to see that that specific register is behaving like that.
>
> On Thu, 2011-11-03 at 19:24 +0200, Pavel A wrote:
>>> … DMA register on the HW shows both that DMA transaction is done
>> and that the interrupt has fired, but the ISR routine is not triggered.
>>
>> Do you check for this in your ISR, or using a bus sniffer?
>> Is what you see that the ISR is called, but DPC delayed?
>> – pa
>>
>>
>> On 03-Nov-2011 17:16, xxxxx@limsim.com wrote:
>>> Hi,
>>>
>>> I am developing a device driver for an Altera Arria GX II FPGA (PCI-e).
>>> A driver I have developed for linux seems to work without any problems on all the tested computers.
>>>
>>> However, when developing a Windows 7 x64 driver with KMDF, I seem to have some problems.
>>>
>>> We use a MSI-interrupt (edge-triggered) and its purpose is to let us know when a DMA transaction is complete.
>>>
>>> Though, the driver works perfectly on two amd64 processor systems, it does not work very well on Intel core i5 systems (two computers tested). We have also noticed that some interrupts may be delayed by a minute or two on a third amd64 system.
>>>
>>> On the Intel core i5 system:
>>> When testing, its waiting after the first DMA transaction(using ReadFile/WriteFile function in application). The DMA works fine and the DMA register on the HW shows both that DMA transaction is done and that the interrupt has fired, but the ISR routine is not triggered.
>>>
>>> When developing the driver I have looked at AMCC5933 and Pcidrv sample files from WinDDK.
>>>
>>> My configuration looks like this:
>>>
>>> EvtDeviceAdd:
>>>
>>> // Configure Interrupt object
>>> WDF_INTERRUPT_CONFIG_INIT(&interruptConfig,
>>> EvtInterruptIsr,
>>> EvtInterruptDpc);
>>>
>>> interruptConfig.EvtInterruptEnable = EvtInterruptEnable;
>>> interruptConfig.EvtInterruptDisable = EvtInterruptDisable;
>>>
>>> status = WdfInterruptCreate(devExt->Device,
>>> &interruptConfig,
>>> WDF_NO_OBJECT_ATTRIBUTES,
>>> &devExt->Interrupt);
>>>
>>> EvtInterruptEnable:
>>> Enabling Interrupt in DMA register.
>>> EvtInterruptDisable:
>>> Disabling Interrupt in DMA register.
>>> EvtInterruptIsr:
>>> Checks DMA register if we have interrupt, if so, we clear it and signals DPC routine. return TRUE, otherwise return FALSE.
>>>
>>>
>>> Any guidance appreciated.
>>>
>>
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

6

Where do you mask and unmask the Interrupt register?

Igor Sharovar

7

They are not “ghost” interrupts; they are interrupts that share the same
IRQ with your device, and indeed, returning FALSE is the correct response.

I can’t help with the other half of the question.
joe

Okay, I just made some interesting discoveries.

When I am waiting for an interrupt (with a simple read/write single
DMA-packet test-program), reinstalling or updating a random
driver/chipset (either deliberate or by Windows update) will make my
interrupt appear and ISR routine is fired. This successfully release the
program from the device. But at the same time I may get a couple of
“ghost” interrupts as well, probably depending on how long I have waited
for the interrupt. (The driver responds with return FALSE in ISR to
these “ghost” interrupts. Is this behavior correct?) However, I have not
yet been able to figure out what causes the interrupt from not being
pointed to my drivers ISR routine, as it does on a AMD64 platform.

–pfbach

On Fri, 2011-11-04 at 02:19 +0100, Per Ferdinand Bach wrote:
> Hi,
> The ISR is not called, I test this by reading temporarily from the
> DMA-register in a for-loop after activating DMA transaction in the
> ProgramDma function that is triggered after calling
> WdfDmaTransactionExecute.(I trigger the WdfDmaTransactionExecute
> function which starts the ProgramDma function, this fills the
> DMA-register with the needed data: addresses,length and
> control-start-bit, which starts the operation. Before returning TRUE, I
> have inserted this small for-loop to check that the DMA transaction
> actually begin, and it does.)
>
> This all happens before the ISR is supposed to be triggered.
> I could of course do this test with the NIOS II, by looping a command in
> the HW to see that that specific register is behaving like that.
>
> On Thu, 2011-11-03 at 19:24 +0200, Pavel A wrote:
> > > … DMA register on the HW shows both that DMA transaction is done
> > and that the interrupt has fired, but the ISR routine is not
> triggered.
> >
> > Do you check for this in your ISR, or using a bus sniffer?
> > Is what you see that the ISR is called, but DPC delayed?
> > – pa
> >
> >
> > On 03-Nov-2011 17:16, xxxxx@limsim.com wrote:
> > > Hi,
> > >
> > > I am developing a device driver for an Altera Arria GX II FPGA
> (PCI-e).
> > > A driver I have developed for linux seems to work without any
> problems on all the tested computers.
> > >
> > > However, when developing a Windows 7 x64 driver with KMDF, I seem to
> have some problems.
> > >
> > > We use a MSI-interrupt (edge-triggered) and its purpose is to let us
> know when a DMA transaction is complete.
> > >
> > > Though, the driver works perfectly on two amd64 processor systems,
> it does not work very well on Intel core i5 systems (two computers
> tested). We have also noticed that some interrupts may be delayed by
> a minute or two on a third amd64 system.
> > >
> > > On the Intel core i5 system:
> > > When testing, its waiting after the first DMA transaction(using
> ReadFile/WriteFile function in application). The DMA works fine and
> the DMA register on the HW shows both that DMA transaction is done
> and that the interrupt has fired, but the ISR routine is not
> triggered.
> > >
> > > When developing the driver I have looked at AMCC5933 and Pcidrv
> sample files from WinDDK.
> > >
> > > My configuration looks like this:
> > >
> > > EvtDeviceAdd:
> > >
> > > // Configure Interrupt object
> > > WDF_INTERRUPT_CONFIG_INIT(&interruptConfig,
> > > EvtInterruptIsr,
> > > EvtInterruptDpc);
> > >
> > > interruptConfig.EvtInterruptEnable = EvtInterruptEnable;
> > > interruptConfig.EvtInterruptDisable = EvtInterruptDisable;
> > >
> > > status = WdfInterruptCreate(devExt->Device,
> > > &interruptConfig,
> > > WDF_NO_OBJECT_ATTRIBUTES,
> > > &devExt->Interrupt);
> > >
> > > EvtInterruptEnable:
> > > Enabling Interrupt in DMA register.
> > > EvtInterruptDisable:
> > > Disabling Interrupt in DMA register.
> > > EvtInterruptIsr:
> > > Checks DMA register if we have interrupt, if so, we clear it and
> signals DPC routine. return TRUE, otherwise return FALSE.
> > >
> > >
> > > Any guidance appreciated.
> > >
> >
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

8

Thnx Joseph, that was actually very helpful. It makes sence when I figured out my driver wasn’t the only device on that IRQ. I didn’t think drivers shared IRQ these days.

Gary, here is the setup of the IRQ that MyDriver is using:

IRQ 16 MyDriver OK
IRQ 16 2nd generation Intel(R) Core™ processor family PCI Express Controller - 0101 OK
IRQ 16 Intel(R) Management Engine Interface OK
IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root Port 1 - 1C10 OK
IRQ 16 VIA 1394 OHCI Compliant Host Controller OK
IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root Port 5 - 1C18 OK

Seems like I may have gotten an interrupt from one of the Intel drivers that I installed which made my driver think it was to continue (as the dma-register show that interrupt bit is high (a check in the ISR routine before taking any further action)).

Igor, I am not sure exacly what you mean by masking/unmasking the Interrupt register.

9

>Igor, I am not sure exacly what you mean by masking/unmasking the Interrupt register.
When you get an interrupt in your ISR you need to read the status interrupt register, clear and mask interrupt and schedule an interrupt DPC. On the DPC you need to process the interrupt. After processing you need to unmask interrupt. If you will not mask interrupt in ISR you may miss some interrupts on your DPC. For example, you clear an interrupt in ISR and schedule DPC. In the same time a hardware send the interrupt again and ISR, not DPC, will be run again. In this case DPC will miss the interrupt.

Igor Sharovar

10

See below…

Thnx Joseph, that was actually very helpful. It makes sence when I figured
out my driver wasn’t the only device on that IRQ. I didn’t think drivers
shared IRQ these days.
*****
It isn’t you choice, it is usually the BIOS/HAL combination that
establishes this. For “pure PCI” (not message-based interrupts) sharing
is still normal behavior.
*****

Gary, here is the setup of the IRQ that MyDriver is using:

IRQ 16 MyDriver OK
IRQ 16 2nd generation Intel(R) Core™ processor family PCI Express
Controller - 0101 OK
IRQ 16 Intel(R) Management Engine Interface OK
IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root Port
1 - 1C10 OK
IRQ 16 VIA 1394 OHCI Compliant Host Controller OK
IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root Port
5 - 1C18 OK

Seems like I may have gotten an interrupt from one of the Intel drivers
that I installed which made my driver think it was to continue (as the
dma-register show that interrupt bit is high (a check in the ISR routine
before taking any further action)).
*******
See the discussion below. Note that if you have not enabled for
interrupts, but your device signals “status done”, and you got to your ISR
because some other device interrupted, then it still works because
interrupts are level-triggered, and presumably when you reset the device
(there is usually a control bit called, for example, IACK, Interrupt
ACKnowledge, which you use to tell the device to release its interrupt
line). When you hit IACK, the I-am-interrupting or I/O-is-done bit will
be cleared.
********

Igor, I am not sure exacly what you mean by masking/unmasking the
Interrupt register.
*****
Most devices have an “interrupt enable” bit in the PCI control register
set. Typically, you set this bit to 1 if you want to allow the device to
interrupt. If it is not set, the device will not interrupt.

Interrupts can be enabled across different spans. For example, enable on
CreateFile and disable on Close. Or enable when an IRP is sent down, and
disable after the interrupt is handled. Or even, enable on AddDevice and
disable on IRP_MJ_PNP:IRP_MN_REMOVE (although this is highly unusual, very
risky, and rarely makes sense for most devices). Most devices power-up
with their IE bit set to 0, so they will not interrupt until you tell them
they are allowed to.

Most (well, all) PCI devices I’ve seen have some kind of
interrupt-has-occurred or I/O-Operation-Completed bit in a status
register. So what you do in your ISR is test this bit. If it isn’t your
device, you ignore EVERYTHING the device might be telling you, and return
FALSE immediately. You don’t care if it isn’t your device.

For debugging purposes, you might look at DMA counters or other state, but
such code would never be included in a product driver. If I’m doing
something like this, I tend to do something like put a timestamped entry
in an internal circular debug buffer, and have a debug-mode-only IOCTL
that delivers this buffer back to an app, so I can examine it (I usually
spend some time writing nice code to format and display it).

You can get away with lots of things in development that you must not even
consider putting in a product driver.
joe
********************************************

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

11

>>Igor, I am not sure exacly what you mean by masking/unmasking the

> Interrupt register.
When you get an interrupt in your ISR you need to read the status
interrupt register, clear and mask interrupt and schedule an interrupt
DPC. On the DPC you need to process the interrupt. After processing you
need to unmask interrupt. If you will not mask interrupt in ISR you may
miss some interrupts on your DPC. For example, you clear an interrupt in
ISR and schedule DPC. In the same time a hardware send the interrupt
again and ISR, not DPC, will be run again. In this case DPC will miss the
interrupt.
*******
Well, the real truth is more complex. For example, some devices
(streaming devices) will interrupt all the time, and these are the kinds
where you have to mask off the interrupt while you are in the DPC. But
many devices have a sequence
Start:
load transfer registers
enable device interrupt via its command register or other mechanism
it supports
hit device’s “start I/O” bit or whatever your device needs to start a
transfer
(device does transfer)
(device interrupts)
ISR: test status bit to see if it is your device
(not your device: return FALSE)
acknowledge the interrupt via IACK or similar mechanism
[maybe disable interrupts on device]
if(not much to do in ISR, but not done yet)
{
set up subsequent transfer
initiate transfer
return TRUE
}
else
{
queue up DPC
return TRUE
}
DPC: [disable interrupts on device]
process information that takes too much time to do in ISR
start next IRP from queue
complete current IRP

Note that an interrupt cannot be generated in this scenario until you
initiate transfer (“hit the GO bit”), so even if interrupts are left
enabled, there is no downside, because the device will not interrupt again
until it has be “re-initialized” to do the next transfer.

Some devices, however, require handling the interrupts which come in at a
predetermined rate; this is where you want to disable interrupts in the
ISR. Note that DPC latency can lead to data overruns.

For some devices, there is so little to do in the ISR that you can easily
fit it into the 10us budget you have for the ISR, and doesn’t require any
operations that are forbidden in an ISR, you can do it all in the ISR.
For example, for some devices, you read the data from the device, put it
in the buffer (non-DMA), and start the next transfer (transfers may be as
small as a single byte, or a group of multiple 32-bit words). As long as
you have the “headroom” to complete it in the ISR, you don’t need the DPC
except to start the next IRP and complete the current one.

On a machine that can execute 6 billion instructions per second (or
better), you can do a LOT in 10us, even allowing for the slow PCI bus
accesses.

If you are doing synchronous I/O, the round-trip time for an IRP can be
several hundred milliseconds, meaning there is no possible way your app
can keep up with your device. But note that if you use asynch I/O, the
cost of starting the next IRP is incredibly low, essentially the DPC
latency. For me to avoid the round-trip latency, I had to pump 40 IRPs
down, asynchronously. When a ReadFile completed, my app’s IOCP handler
thread simply issued another ReadFile. I was able to keep up with a
serious real-time device.
joe
*********

Igor Sharovar

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

12

> When you get an interrupt in your ISR you need to read the status interrupt register, clear and mask

interrupt and schedule an interrupt DPC. On the DPC you need to process the interrupt. After
processing you need to unmask interrupt

This is the most dumb SCSIPORT’s scheme. There are more capable schemes, where you never need to mask interrupts on the device.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

13

>This is the most dumb SCSIPORT’s scheme. There are more capable schemes, where you never >need to mask interrupts on the device.
Where does OP mention that this for SCSIPORT drivers? I just gave an example where his problem could be. And Joseph elaborated this. The real solution would be depend on how OP hardware works.

Igor Sharovar

14

Okay,

I still haven’t figured out this problem. The system does work on a
AMD64 processor system but not on a INTEL processor system, at least,
that is the only specific difference between those computers (as what I
have at hand to test here.)

To clearify, Interrupt is only enabled upon writing a control word to
the DMA-register when starting a DMA transaction.
Further, the interrupt is disabled in the ISR, and DPC is triggered. At
this time, no new interrupt (from device) will occur
until current I/O Request is completed, and a new IRP triggers a new
DMA-transaction. A very simple approach for now to test the system.

I have implemented an IOCTL function where I check the DMA-register and
it clearly shows that the interrupt is set after DMA transaction has
occured. However, ISR routine is not triggered.

I initially mentioned we use MSI interrupt, but according to
CM_PARTIAL_RESOURCE_DESCRIPTOR its a LineInterrupt.
As I understand LineInterrupts (Legacy) are level-triggered interrupts
so shouldn’t the resource descriptor flag tell me exacly that when
testing for CM_RESOURCE_INTERRUPT_LEVEL_SENSITIVE, in my case it does
not?

Are there any WDF structures or places I may check if the OS actually
has received the interrupt, kind of like /etc/interrupt on linux?

-pfbach

On Fri, 2011-11-04 at 23:40 -0400, xxxxx@flounder.com wrote:

See below…
> Thnx Joseph, that was actually very helpful. It makes sence when I figured
> out my driver wasn’t the only device on that IRQ. I didn’t think drivers
> shared IRQ these days.
*****
It isn’t you choice, it is usually the BIOS/HAL combination that
establishes this. For “pure PCI” (not message-based interrupts) sharing
is still normal behavior.
*****
>
> Gary, here is the setup of the IRQ that MyDriver is using:
>
> IRQ 16 MyDriver OK
> IRQ 16 2nd generation Intel(R) Core™ processor family PCI Express
> Controller - 0101 OK
> IRQ 16 Intel(R) Management Engine Interface OK
> IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root Port
> 1 - 1C10 OK
> IRQ 16 VIA 1394 OHCI Compliant Host Controller OK
> IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root Port
> 5 - 1C18 OK
>
> Seems like I may have gotten an interrupt from one of the Intel drivers
> that I installed which made my driver think it was to continue (as the
> dma-register show that interrupt bit is high (a check in the ISR routine
> before taking any further action)).
*******
See the discussion below. Note that if you have not enabled for
interrupts, but your device signals “status done”, and you got to your ISR
because some other device interrupted, then it still works because
interrupts are level-triggered, and presumably when you reset the device
(there is usually a control bit called, for example, IACK, Interrupt
ACKnowledge, which you use to tell the device to release its interrupt
line). When you hit IACK, the I-am-interrupting or I/O-is-done bit will
be cleared.
********
>
> Igor, I am not sure exacly what you mean by masking/unmasking the
> Interrupt register.
*****
Most devices have an “interrupt enable” bit in the PCI control register
set. Typically, you set this bit to 1 if you want to allow the device to
interrupt. If it is not set, the device will not interrupt.

Interrupts can be enabled across different spans. For example, enable on
CreateFile and disable on Close. Or enable when an IRP is sent down, and
disable after the interrupt is handled. Or even, enable on AddDevice and
disable on IRP_MJ_PNP:IRP_MN_REMOVE (although this is highly unusual, very
risky, and rarely makes sense for most devices). Most devices power-up
with their IE bit set to 0, so they will not interrupt until you tell them
they are allowed to.

Most (well, all) PCI devices I’ve seen have some kind of
interrupt-has-occurred or I/O-Operation-Completed bit in a status
register. So what you do in your ISR is test this bit. If it isn’t your
device, you ignore EVERYTHING the device might be telling you, and return
FALSE immediately. You don’t care if it isn’t your device.

For debugging purposes, you might look at DMA counters or other state, but
such code would never be included in a product driver. If I’m doing
something like this, I tend to do something like put a timestamped entry
in an internal circular debug buffer, and have a debug-mode-only IOCTL
that delivers this buffer back to an app, so I can examine it (I usually
spend some time writing nice code to format and display it).

You can get away with lots of things in development that you must not even
consider putting in a product driver.
joe
********************************************
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

15

Generally, there are several things you can investigate. Here are some
random ideas off the top of my head…

For example, an erroneous driver sitting ahead of yours in the ISR chain
for that level, that fails to return FALSE if its device did not
interrupt, will block your ISR from ever seeing the interrupt.
(Historically, I know of at least one case in which this happened, as
described to me; the developer spent considerable time discovering this,
reported the bug to the vendor, who pointed out that they had fixed that
bug a year before).

One way to test this is to physically rearrange the cards on the
backplane. Sometimes this will cause the ISR chain to be developed in a
different order. In old systems, in guaranteed the ISR chain would be
different; in modern systems, PNP and Power Management can cause the ISR
chain to be developed non-deterministically, particularly when there are
different times for completion of PNP and Power sequencing. But this is
an unreliable diagnostic technique, because different physical
arrangements today could still produce the same ISR chain. If it *does*
make a difference, you have a pretty good idea that there is a broken
drive in the chain, but if it doesn’t make a difference, you have just
wasted time.

This also means that if you do not have *exactly* the same devices in your
Intel machine as your AMD machine, then you have a different set of
drivers, and one of them might be broken. Maybe. It is always worth
checking out. Perhaps one way to do this would be to load a software-only
driver that loads after everything else, and adds itself to the ISR chain.
I believe this puts it at the head of the chain. It would see all the
interrupts, check your device, and always return FALSE, then you would
have to examine this information (note that DbgPrint/KdPrint in an ISR can
change system behavior to the level that some devices will begin to fail,
so my inclination would be to log it to a circular buffer. This is a
fairly drsstic technique, and someone else is free to say it is the
dumbest idea they’ve ever heard.

I discovered some years ago that if I failed to call IoConnectInterrupt
properly (I don’t know the KMDF equivalent), then I would see on the debug
console a message that there was an unhandled interrupt on line XXXX. So
if you don’t get one of these messages, it means the interrupt has been
fielded and there is *something* in the ISR chain.

Are you checking all the return codes from the DDI calls? Could it be
that one one configuration, some call fails and you didn’t notice? Just a
thought.

Note that on-motherboard PCI devices are going to change the picture, and
I suspect that the CPU type is not the only difference between the
motherboards.

Discovering what devices are on each ISR chain might be worth researching.
joe

Okay,

I still haven’t figured out this problem. The system does work on a
AMD64 processor system but not on a INTEL processor system, at least,
that is the only specific difference between those computers (as what I
have at hand to test here.)

To clearify, Interrupt is only enabled upon writing a control word to
the DMA-register when starting a DMA transaction.
Further, the interrupt is disabled in the ISR, and DPC is triggered. At
this time, no new interrupt (from device) will occur
until current I/O Request is completed, and a new IRP triggers a new
DMA-transaction. A very simple approach for now to test the system.

I have implemented an IOCTL function where I check the DMA-register and
it clearly shows that the interrupt is set after DMA transaction has
occured. However, ISR routine is not triggered.

I initially mentioned we use MSI interrupt, but according to
CM_PARTIAL_RESOURCE_DESCRIPTOR its a LineInterrupt.
As I understand LineInterrupts (Legacy) are level-triggered interrupts
so shouldn’t the resource descriptor flag tell me exacly that when
testing for CM_RESOURCE_INTERRUPT_LEVEL_SENSITIVE, in my case it does
not?

Are there any WDF structures or places I may check if the OS actually
has received the interrupt, kind of like /etc/interrupt on linux?

-pfbach

On Fri, 2011-11-04 at 23:40 -0400, xxxxx@flounder.com wrote:
> See below…
> > Thnx Joseph, that was actually very helpful. It makes sence when I
> figured
> > out my driver wasn’t the only device on that IRQ. I didn’t think
> drivers
> > shared IRQ these days.
> *****
> It isn’t you choice, it is usually the BIOS/HAL combination that
> establishes this. For “pure PCI” (not message-based interrupts) sharing
> is still normal behavior.
> *****
> >
> > Gary, here is the setup of the IRQ that MyDriver is using:
> >
> > IRQ 16 MyDriver OK
> > IRQ 16 2nd generation Intel(R) Core™ processor family PCI Express
> > Controller - 0101 OK
> > IRQ 16 Intel(R) Management Engine Interface OK
> > IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root
> Port
> > 1 - 1C10 OK
> > IRQ 16 VIA 1394 OHCI Compliant Host Controller OK
> > IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root
> Port
> > 5 - 1C18 OK
> >
> > Seems like I may have gotten an interrupt from one of the Intel
> drivers
> > that I installed which made my driver think it was to continue (as the
> > dma-register show that interrupt bit is high (a check in the ISR
> routine
> > before taking any further action)).
> *******
> See the discussion below. Note that if you have not enabled for
> interrupts, but your device signals “status done”, and you got to your
> ISR
> because some other device interrupted, then it still works because
> interrupts are level-triggered, and presumably when you reset the device
> (there is usually a control bit called, for example, IACK, Interrupt
> ACKnowledge, which you use to tell the device to release its interrupt
> line). When you hit IACK, the I-am-interrupting or I/O-is-done bit will
> be cleared.
> ********
> >
> > Igor, I am not sure exacly what you mean by masking/unmasking the
> > Interrupt register.
> *****
> Most devices have an “interrupt enable” bit in the PCI control register
> set. Typically, you set this bit to 1 if you want to allow the device
> to
> interrupt. If it is not set, the device will not interrupt.
>
> Interrupts can be enabled across different spans. For example, enable
> on
> CreateFile and disable on Close. Or enable when an IRP is sent down,
> and
> disable after the interrupt is handled. Or even, enable on AddDevice
> and
> disable on IRP_MJ_PNP:IRP_MN_REMOVE (although this is highly unusual,
> very
> risky, and rarely makes sense for most devices). Most devices power-up
> with their IE bit set to 0, so they will not interrupt until you tell
> them
> they are allowed to.
>
> Most (well, all) PCI devices I’ve seen have some kind of
> interrupt-has-occurred or I/O-Operation-Completed bit in a status
> register. So what you do in your ISR is test this bit. If it isn’t
> your
> device, you ignore EVERYTHING the device might be telling you, and
> return
> FALSE immediately. You don’t care if it isn’t your device.
>
> For debugging purposes, you might look at DMA counters or other state,
> but
> such code would never be included in a product driver. If I’m doing
> something like this, I tend to do something like put a timestamped entry
> in an internal circular debug buffer, and have a debug-mode-only IOCTL
> that delivers this buffer back to an app, so I can examine it (I usually
> spend some time writing nice code to format and display it).
>
> You can get away with lots of things in development that you must not
> even
> consider putting in a product driver.
> joe
> ********************************************
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at
> > http://www.osronline.com/page.cfm?name=ListServer
> >
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

16

A quick way to find out if the interrupt vector is shared is to attach
WinDbg and issue a !idt command. That will reveal if multiple ISRs are
registered for the same interrupt vector or not.

While this is a possibility, what I think is that the OP has a logic problem
somewhere.

EvtInterruptIsr:
Checks DMA register if we have interrupt, if so, we clear it and signals
DPC routine. return TRUE, otherwise return FALSE.

This sound as if new interrupts can occur while your are still busy
processing the last one in your DPC routine.

Posting the code of the ISR and DPC routine here might help the experts help
you further.

Regards,

Daniel Terhell
Resplendence Software Projects Sp
http://www.resplendence.com

wrote in message news:xxxxx@ntdev…
> For example, an erroneous driver sitting ahead of yours in the ISR chain
> for that level, that fails to return FALSE if its device did not
> interrupt, will block your ISR from ever seeing the interrupt.
> (Historically, I know of at least one case in which this happened, as
> described to me; the developer spent considerable time discovering this,
> reported the bug to the vendor, who pointed out that they had fixed that
> bug a year before).
>

17

Can it be indeed that you get line interrupt instead of MSI, but the device
tries to use MSI?
Then it won’t work, of course.
– pa

“Per Ferdinand Bach” wrote in message news:xxxxx@ntdev…
> Okay,
>
> I still haven’t figured out this problem. The system does work on a
> AMD64 processor system but not on a INTEL processor system, at least,
> that is the only specific difference between those computers (as what I
> have at hand to test here.)
>
> To clearify, Interrupt is only enabled upon writing a control word to
> the DMA-register when starting a DMA transaction.
> Further, the interrupt is disabled in the ISR, and DPC is triggered. At
> this time, no new interrupt (from device) will occur
> until current I/O Request is completed, and a new IRP triggers a new
> DMA-transaction. A very simple approach for now to test the system.
>
> I have implemented an IOCTL function where I check the DMA-register and
> it clearly shows that the interrupt is set after DMA transaction has
> occured. However, ISR routine is not triggered.
>
> I initially mentioned we use MSI interrupt, but according to
> CM_PARTIAL_RESOURCE_DESCRIPTOR its a LineInterrupt.
> As I understand LineInterrupts (Legacy) are level-triggered interrupts
> so shouldn’t the resource descriptor flag tell me exacly that when
> testing for CM_RESOURCE_INTERRUPT_LEVEL_SENSITIVE, in my case it does
> not?
>
> Are there any WDF structures or places I may check if the OS actually
> has received the interrupt, kind of like /etc/interrupt on linux?
>
>
> -pfbach
>
> On Fri, 2011-11-04 at 23:40 -0400, xxxxx@flounder.com wrote:
>> See below…
>> > Thnx Joseph, that was actually very helpful. It makes sence when I
>> > figured
>> > out my driver wasn’t the only device on that IRQ. I didn’t think
>> > drivers
>> > shared IRQ these days.
>>
>> It isn’t you choice, it is usually the BIOS/HAL combination that
>> establishes this. For “pure PCI” (not message-based interrupts) sharing
>> is still normal behavior.
>>
>> >
>> > Gary, here is the setup of the IRQ that MyDriver is using:
>> >
>> > IRQ 16 MyDriver OK
>> > IRQ 16 2nd generation Intel(R) Core™ processor family PCI Express
>> > Controller - 0101 OK
>> > IRQ 16 Intel(R) Management Engine Interface OK
>> > IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root
>> > Port
>> > 1 - 1C10 OK
>> > IRQ 16 VIA 1394 OHCI Compliant Host Controller OK
>> > IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root
>> > Port
>> > 5 - 1C18 OK
>> >
>> > Seems like I may have gotten an interrupt from one of the Intel drivers
>> > that I installed which made my driver think it was to continue (as the
>> > dma-register show that interrupt bit is high (a check in the ISR
>> > routine
>> > before taking any further action)).
>> *******
>> See the discussion below. Note that if you have not enabled for
>> interrupts, but your device signals “status done”, and you got to your
>> ISR
>> because some other device interrupted, then it still works because
>> interrupts are level-triggered, and presumably when you reset the device
>> (there is usually a control bit called, for example, IACK, Interrupt
>> ACKnowledge, which you use to tell the device to release its interrupt
>> line). When you hit IACK, the I-am-interrupting or I/O-is-done bit will
>> be cleared.
>>
>> >
>> > Igor, I am not sure exacly what you mean by masking/unmasking the
>> > Interrupt register.
>>
>> Most devices have an “interrupt enable” bit in the PCI control register
>> set. Typically, you set this bit to 1 if you want to allow the device to
>> interrupt. If it is not set, the device will not interrupt.
>>
>> Interrupts can be enabled across different spans. For example, enable on
>> CreateFile and disable on Close. Or enable when an IRP is sent down, and
>> disable after the interrupt is handled. Or even, enable on AddDevice and
>> disable on IRP_MJ_PNP:IRP_MN_REMOVE (although this is highly unusual,
>> very
>> risky, and rarely makes sense for most devices). Most devices power-up
>> with their IE bit set to 0, so they will not interrupt until you tell
>> them
>> they are allowed to.
>>
>> Most (well, all) PCI devices I’ve seen have some kind of
>> interrupt-has-occurred or I/O-Operation-Completed bit in a status
>> register. So what you do in your ISR is test this bit. If it isn’t your
>> device, you ignore EVERYTHING the device might be telling you, and return
>> FALSE immediately. You don’t care if it isn’t your device.
>>
>> For debugging purposes, you might look at DMA counters or other state,
>> but
>> such code would never be included in a product driver. If I’m doing
>> something like this, I tend to do something like put a timestamped entry
>> in an internal circular debug buffer, and have a debug-mode-only IOCTL
>> that delivers this buffer back to an app, so I can examine it (I usually
>> spend some time writing nice code to format and display it).
>>
>> You can get away with lots of things in development that you must not
>> even
>> consider putting in a product driver.
>> joe
>>*******************************
>> >
>> > —
>> > NTDEV is sponsored by OSR
>> >
>> > For our schedule of WDF, WDM, debugging and other seminars visit:
>> > http://www.osr.com/seminars
>> >
>> > To unsubscribe, visit the List Server section of OSR Online at
>> > http://www.osronline.com/page.cfm?name=ListServer
>> >
>>
>>
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>
>
>

18

> Can it be indeed that you get line interrupt instead of MSI, but the

device
tries to use MSI?
Then it won’t work, of course.
– pa

I agree. The presumption apparently was that it is *not* using MSI.
joe

“Per Ferdinand Bach” wrote in message
> news:xxxxx@ntdev…
>> Okay,
>>
>> I still haven’t figured out this problem. The system does work on a
>> AMD64 processor system but not on a INTEL processor system, at least,
>> that is the only specific difference between those computers (as what I
>> have at hand to test here.)
>>
>> To clearify, Interrupt is only enabled upon writing a control word to
>> the DMA-register when starting a DMA transaction.
>> Further, the interrupt is disabled in the ISR, and DPC is triggered. At
>> this time, no new interrupt (from device) will occur
>> until current I/O Request is completed, and a new IRP triggers a new
>> DMA-transaction. A very simple approach for now to test the system.
>>
>> I have implemented an IOCTL function where I check the DMA-register and
>> it clearly shows that the interrupt is set after DMA transaction has
>> occured. However, ISR routine is not triggered.
>>
>> I initially mentioned we use MSI interrupt, but according to
>> CM_PARTIAL_RESOURCE_DESCRIPTOR its a LineInterrupt.
>> As I understand LineInterrupts (Legacy) are level-triggered interrupts
>> so shouldn’t the resource descriptor flag tell me exacly that when
>> testing for CM_RESOURCE_INTERRUPT_LEVEL_SENSITIVE, in my case it does
>> not?
>>
>> Are there any WDF structures or places I may check if the OS actually
>> has received the interrupt, kind of like /etc/interrupt on linux?
>>
>>
>> -pfbach
>>
>> On Fri, 2011-11-04 at 23:40 -0400, xxxxx@flounder.com wrote:
>>> See below…
>>> > Thnx Joseph, that was actually very helpful. It makes sence when I
>>> > figured
>>> > out my driver wasn’t the only device on that IRQ. I didn’t think
>>> > drivers
>>> > shared IRQ these days.
>>>
>>> It isn’t you choice, it is usually the BIOS/HAL combination that
>>> establishes this. For “pure PCI” (not message-based interrupts)
>>> sharing
>>> is still normal behavior.
>>>
>>> >
>>> > Gary, here is the setup of the IRQ that MyDriver is using:
>>> >
>>> > IRQ 16 MyDriver OK
>>> > IRQ 16 2nd generation Intel(R) Core™ processor family PCI Express
>>> > Controller - 0101 OK
>>> > IRQ 16 Intel(R) Management Engine Interface OK
>>> > IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root
>>> > Port
>>> > 1 - 1C10 OK
>>> > IRQ 16 VIA 1394 OHCI Compliant Host Controller OK
>>> > IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root
>>> > Port
>>> > 5 - 1C18 OK
>>> >
>>> > Seems like I may have gotten an interrupt from one of the Intel
>>> drivers
>>> > that I installed which made my driver think it was to continue (as
>>> the
>>> > dma-register show that interrupt bit is high (a check in the ISR
>>> > routine
>>> > before taking any further action)).
>>> *******
>>> See the discussion below. Note that if you have not enabled for
>>> interrupts, but your device signals “status done”, and you got to your
>>> ISR
>>> because some other device interrupted, then it still works because
>>> interrupts are level-triggered, and presumably when you reset the
>>> device
>>> (there is usually a control bit called, for example, IACK, Interrupt
>>> ACKnowledge, which you use to tell the device to release its interrupt
>>> line). When you hit IACK, the I-am-interrupting or I/O-is-done bit
>>> will
>>> be cleared.
>>>
>>> >
>>> > Igor, I am not sure exacly what you mean by masking/unmasking the
>>> > Interrupt register.
>>>
>>> Most devices have an “interrupt enable” bit in the PCI control register
>>> set. Typically, you set this bit to 1 if you want to allow the device
>>> to
>>> interrupt. If it is not set, the device will not interrupt.
>>>
>>> Interrupts can be enabled across different spans. For example, enable
>>> on
>>> CreateFile and disable on Close. Or enable when an IRP is sent down,
>>> and
>>> disable after the interrupt is handled. Or even, enable on AddDevice
>>> and
>>> disable on IRP_MJ_PNP:IRP_MN_REMOVE (although this is highly unusual,
>>> very
>>> risky, and rarely makes sense for most devices). Most devices power-up
>>> with their IE bit set to 0, so they will not interrupt until you tell
>>> them
>>> they are allowed to.
>>>
>>> Most (well, all) PCI devices I’ve seen have some kind of
>>> interrupt-has-occurred or I/O-Operation-Completed bit in a status
>>> register. So what you do in your ISR is test this bit. If it isn’t
>>> your
>>> device, you ignore EVERYTHING the device might be telling you, and
>>> return
>>> FALSE immediately. You don’t care if it isn’t your device.
>>>
>>> For debugging purposes, you might look at DMA counters or other state,
>>> but
>>> such code would never be included in a product driver. If I’m doing
>>> something like this, I tend to do something like put a timestamped
>>> entry
>>> in an internal circular debug buffer, and have a debug-mode-only IOCTL
>>> that delivers this buffer back to an app, so I can examine it (I
>>> usually
>>> spend some time writing nice code to format and display it).
>>>
>>> You can get away with lots of things in development that you must not
>>> even
>>> consider putting in a product driver.
>>> joe
>>>*******************************
>>> >
>>> > —
>>> > NTDEV is sponsored by OSR
>>> >
>>> > For our schedule of WDF, WDM, debugging and other seminars visit:
>>> > http://www.osr.com/seminars
>>> >
>>> > To unsubscribe, visit the List Server section of OSR Online at
>>> > http://www.osronline.com/page.cfm?name=ListServer
>>> >
>>>
>>>
>>>
>>> —
>>> NTDEV is sponsored by OSR
>>>
>>> For our schedule of WDF, WDM, debugging and other seminars visit:
>>> http://www.osr.com/seminars
>>>
>>> To unsubscribe, visit the List Server section of OSR Online at
>>> http://www.osronline.com/page.cfm?name=ListServer
>>
>>
>>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

19

> Where does OP mention that this for SCSIPORT drivers?

SCSIPORT uses this scheme internally.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

20

Hi,

You are absolutely right.

The HW uses MSI which I figured out in linux with the command “lspci
-vvv”. Under the “Capabilities” section for the device I could see: “MSI
Enable-” before I loaded the linux driver and “MSI Enable+” after it was
loaded. So, my linux driver definetly use MSI.

It was stupid of me not to check CmResourceTypeInterrupt earlier,
as it would have told me what kind of interrupt was set up on
Windows.(Perhaps I would have been told the same thing with WinDbg and
the command !idt ?) Here is a code-snippet that might be of interest to
others:

for(i=0;i desc = WdfCmResourceListGetDescriptor(ResourcesTranslated,i);
switch(desc->Type){
case CmResourceTypeInterrupt:
if (desc->Flags & CM_RESOURCE_INTERRUPT_MESSAGE){
//Msi Interrupt
}else{
//Line Interrupt
}
if(desc->Flags & CM_RESOURCE_INTERRUPT_LEVEL_SENSITIVE){
//Level triggered.
}
if(desc->Flags & CM_RESOURCE_INTERRUPT_LEVEL_SENSITIVE){
//Edge triggered.
}
}
}

Check out :
http://msdn.microsoft.com/en-us/library/windows/hardware/ff541977%
28v=vs.85%29.aspx for more information regarding the info on the struct
CM_PARTIAL_RESOURCE_DESCRIPTOR.

In order to get MSI to work in WDF / KMDF, one have to add a section in
the inx/inf file, as follows:

[mydevice.NT.HW]
AddReg = mydevice_addreg

[mydevice_addreg]
HKR,Interrupt Management,0x00000010
HKR,Interrupt Management\MessageSignaledInterruptProperties,0x00000010
HKR,Interrupt Management
\MessageSignaledInterruptProperties,MSISupported,0x00010001,1

You have to specifically add MSI support to the registry of the Device’s
Hardware Key. Only then will the WDF framework actually use MSI.
See: http://msdn.microsoft.com/en-us/library/ff544246(v=VS.85).aspx

I find it a bit odd that I cannot find anything about this in the book
“Developing Drivers with the WDF” from MS that is 880 pages long…

Though, a good idea is to search for “Message-signaled interrupts” in
the Microsoft docs than just “MSI”…

So, I guess it was only plain luck that made it work on a few AMD
systems.

Thanks for the help guys.

-pfbach

On Tue, 2011-11-08 at 14:21 +0200, Pavel A. wrote:
> Can it be indeed that you get line interrupt instead of MSI, but the device
> tries to use MSI?
> Then it won’t work, of course.
> – pa
>
>
> “Per Ferdinand Bach” wrote in message news:xxxxx@ntdev…
> > Okay,
> >
> > I still haven’t figured out this problem. The system does work on a
> > AMD64 processor system but not on a INTEL processor system, at least,
> > that is the only specific difference between those computers (as what I
> > have at hand to test here.)
> >
> > To clearify, Interrupt is only enabled upon writing a control word to
> > the DMA-register when starting a DMA transaction.
> > Further, the interrupt is disabled in the ISR, and DPC is triggered. At
> > this time, no new interrupt (from device) will occur
> > until current I/O Request is completed, and a new IRP triggers a new
> > DMA-transaction. A very simple approach for now to test the system.
> >
> > I have implemented an IOCTL function where I check the DMA-register and
> > it clearly shows that the interrupt is set after DMA transaction has
> > occured. However, ISR routine is not triggered.
> >
> > I initially mentioned we use MSI interrupt, but according to
> > CM_PARTIAL_RESOURCE_DESCRIPTOR its a LineInterrupt.
> > As I understand LineInterrupts (Legacy) are level-triggered interrupts
> > so shouldn’t the resource descriptor flag tell me exacly that when
> > testing for CM_RESOURCE_INTERRUPT_LEVEL_SENSITIVE, in my case it does
> > not?
> >
> > Are there any WDF structures or places I may check if the OS actually
> > has received the interrupt, kind of like /etc/interrupt on linux?
> >
> >
> > -pfbach
> >
> > On Fri, 2011-11-04 at 23:40 -0400, xxxxx@flounder.com wrote:
> >> See below…
> >> > Thnx Joseph, that was actually very helpful. It makes sence when I
> >> > figured
> >> > out my driver wasn’t the only device on that IRQ. I didn’t think
> >> > drivers
> >> > shared IRQ these days.
> >>
> >> It isn’t you choice, it is usually the BIOS/HAL combination that
> >> establishes this. For “pure PCI” (not message-based interrupts) sharing
> >> is still normal behavior.
> >>
> >> >
> >> > Gary, here is the setup of the IRQ that MyDriver is using:
> >> >
> >> > IRQ 16 MyDriver OK
> >> > IRQ 16 2nd generation Intel(R) Core™ processor family PCI Express
> >> > Controller - 0101 OK
> >> > IRQ 16 Intel(R) Management Engine Interface OK
> >> > IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root
> >> > Port
> >> > 1 - 1C10 OK
> >> > IRQ 16 VIA 1394 OHCI Compliant Host Controller OK
> >> > IRQ 16 Intel(R) 6 Series/C200 Series Chipset Family PCI Express Root
> >> > Port
> >> > 5 - 1C18 OK
> >> >
> >> > Seems like I may have gotten an interrupt from one of the Intel drivers
> >> > that I installed which made my driver think it was to continue (as the
> >> > dma-register show that interrupt bit is high (a check in the ISR
> >> > routine
> >> > before taking any further action)).
> >> *******
> >> See the discussion below. Note that if you have not enabled for
> >> interrupts, but your device signals “status done”, and you got to your
> >> ISR
> >> because some other device interrupted, then it still works because
> >> interrupts are level-triggered, and presumably when you reset the device
> >> (there is usually a control bit called, for example, IACK, Interrupt
> >> ACKnowledge, which you use to tell the device to release its interrupt
> >> line). When you hit IACK, the I-am-interrupting or I/O-is-done bit will
> >> be cleared.
> >>
> >> >
> >> > Igor, I am not sure exacly what you mean by masking/unmasking the
> >> > Interrupt register.
> >>
> >> Most devices have an “interrupt enable” bit in the PCI control register
> >> set. Typically, you set this bit to 1 if you want to allow the device to
> >> interrupt. If it is not set, the device will not interrupt.
> >>
> >> Interrupts can be enabled across different spans. For example, enable on
> >> CreateFile and disable on Close. Or enable when an IRP is sent down, and
> >> disable after the interrupt is handled. Or even, enable on AddDevice and
> >> disable on IRP_MJ_PNP:IRP_MN_REMOVE (although this is highly unusual,
> >> very
> >> risky, and rarely makes sense for most devices). Most devices power-up
> >> with their IE bit set to 0, so they will not interrupt until you tell
> >> them
> >> they are allowed to.
> >>
> >> Most (well, all) PCI devices I’ve seen have some kind of
> >> interrupt-has-occurred or I/O-Operation-Completed bit in a status
> >> register. So what you do in your ISR is test this bit. If it isn’t your
> >> device, you ignore EVERYTHING the device might be telling you, and return
> >> FALSE immediately. You don’t care if it isn’t your device.
> >>
> >> For debugging purposes, you might look at DMA counters or other state,
> >> but
> >> such code would never be included in a product driver. If I’m doing
> >> something like this, I tend to do something like put a timestamped entry
> >> in an internal circular debug buffer, and have a debug-mode-only IOCTL
> >> that delivers this buffer back to an app, so I can examine it (I usually
> >> spend some time writing nice code to format and display it).
> >>
> >> You can get away with lots of things in development that you must not
> >> even
> >> consider putting in a product driver.
> >> joe
> >>*******************************
> >> >
> >> > —
> >> > NTDEV is sponsored by OSR
> >> >
> >> > For our schedule of WDF, WDM, debugging and other seminars visit:
> >> > http://www.osr.com/seminars
> >> >
> >> > To unsubscribe, visit the List Server section of OSR Online at
> >> > http://www.osronline.com/page.cfm?name=ListServer
> >> >
> >>
> >>
> >>
> >> —
> >> NTDEV is sponsored by OSR
> >>
> >> For our schedule of WDF, WDM, debugging and other seminars visit:
> >> http://www.osr.com/seminars
> >>
> >> To unsubscribe, visit the List Server section of OSR Online at
> >> http://www.osronline.com/page.cfm?name=ListServer
> >
> >
> >
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer