RE: Device Interrupt priority - Reviewing Jose Flores

Well thanks. This is something for the “MS Driver Survey”.
I need a DPC queue per available interrupt IRQL, handled
accordingly.

----- Original Message -----
From: “Maxim S. Shatskih”
To: “NT Developers Interest List”
Sent: Wednesday, December 11, 2002 10:17 AM
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

> > Somewhere I have read that all DPC’s for a particular processor
> within the
> > system are FIFO queued
> > to run at the same IRQL. This would mean that when a high priority
> ISR
> > queues a DPC, this DPC will
> > only come to run when all other DPC’s, even those queued by less
> priority
> > ISR’s, have finished.
> > Is this correct ?
>
> Yes.
>
> Max
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@Compaqnet.be
> To unsubscribe send a blank email to %%email.unsub%%
>

Unless the APIC has the ability to queue interrupts, that will be of limited
usefulness. The last machine I worked with that could queue interrupts by
hardware was the Sperry Univac 418III, way back in the seventies and
eighties. Anybody know any other systems that do it ?

Alberto.

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Wednesday, December 11, 2002 4:20 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

issues an STI. This leaves a small window within which an interrupt
could
theoretically be missed

It will not. It will be put on hold on PIC/APIC.

Max


You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to %%email.unsub%%

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

Hmm, aren’t PCI IRQs level triggered ?
With level triggered IRQ you never loose one.
But the term ‘loose’ depends on how you define it. Your hardware may
have some max. latency requirements that make you loose an event.
Norbert.

“If you have tried something and failed, you are vastly better off
than if you had tried nothing and succeeded.”
---- snip ----

Unless the APIC has the ability to queue interrupts, that will be of limited
usefulness. The last machine I worked with that could queue interrupts by
hardware was the Sperry Univac 418III, way back in the seventies and
eighties. Anybody know any other systems that do it ?

---- snip ----

The PCI bus will hold the IRQ signal till the driver’s ISR will reset
it. With any PIC, be it 8259 or APIC.

Max

----- Original Message -----
From: “Moreira, Alberto”
To: “NT Developers Interest List”
Sent: Wednesday, December 11, 2002 6:13 PM
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

> Unless the APIC has the ability to queue interrupts, that will be of
limited
> usefulness. The last machine I worked with that could queue
interrupts by
> hardware was the Sperry Univac 418III, way back in the seventies and
> eighties. Anybody know any other systems that do it ?
>
> Alberto.
>
>
> -----Original Message-----
> From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
> Sent: Wednesday, December 11, 2002 4:20 AM
> To: NT Developers Interest List
> Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose
Flores
>
>
> > issues an STI. This leaves a small window within which an
interrupt
> could
> > theoretically be missed
>
> It will not. It will be put on hold on PIC/APIC.
>
> Max
>
>
>
> —
> You are currently subscribed to ntdev as:
xxxxx@compuware.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
> The contents of this e-mail are intended for the named addressee
only. It
> contains information that may be confidential. Unless you are the
named
> addressee or an authorized designee, you may not copy or use it, or
disclose
> it to anyone else. If you received it in error please notify us
immediately
> and then destroy it.
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to %%email.unsub%%
>

Who will rise to the next challenge :

"Four PCI cards fire quasi at the same moment ( assume 100 ns time
difference ) an interrupt. Card1 , card2 and card3 have different IRQ’s ,
card4 shares the same IRQ with card3.

I would like to see a detailled flow of all handschaking actions between the
Cards <–> [PCI bus <– >] PIC/APIC <–> CPU with bus <–> OS/ISR’s in
both the “Lazy” and the “Strict Model”. The flow ends when the last ( fourth )
IRQ has been completely serviced.

Such a description would have an incredible value .

----- Original Message -----
From: “Maxim S. Shatskih”
To: “NT Developers Interest List”
Sent: Wednesday, December 11, 2002 7:20 PM
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

> The PCI bus will hold the IRQ signal till the driver’s ISR will reset
> it. With any PIC, be it 8259 or APIC.
>
> Max
>
> ----- Original Message -----
> From: “Moreira, Alberto”
> To: “NT Developers Interest List”
> Sent: Wednesday, December 11, 2002 6:13 PM
> Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores
>
>
> > Unless the APIC has the ability to queue interrupts, that will be of
> limited
> > usefulness. The last machine I worked with that could queue
> interrupts by
> > hardware was the Sperry Univac 418III, way back in the seventies and
> > eighties. Anybody know any other systems that do it ?
> >
> > Alberto.
> >
> >
> > -----Original Message-----
> > From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
> > Sent: Wednesday, December 11, 2002 4:20 AM
> > To: NT Developers Interest List
> > Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose
> Flores
> >
> >
> > > issues an STI. This leaves a small window within which an
> interrupt
> > could
> > > theoretically be missed
> >
> > It will not. It will be put on hold on PIC/APIC.
> >
> > Max
> >
> >
> >
> > —
> > You are currently subscribed to ntdev as:
> xxxxx@compuware.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> >
> > The contents of this e-mail are intended for the named addressee
> only. It
> > contains information that may be confidential. Unless you are the
> named
> > addressee or an authorized designee, you may not copy or use it, or
> disclose
> > it to anyone else. If you received it in error please notify us
> immediately
> > and then destroy it.
> >
> >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@Compaqnet.be
> To unsubscribe send a blank email to %%email.unsub%%
>

I thought the APIC in fact had the ability to queue one or perhaps two interrupts per source
if no target were ready to service the interrupt.

===========================
Mark Roddy
Consultant, Microsoft DDK MVP
Hollis Technology Solutions
xxxxx@hollistech.com
www.hollistech.com
603-321-1032

The PCI bus will hold the IRQ signal till the driver’s ISR will reset
it. With any PIC, be it 8259 or APIC.

Max

----- Original Message -----
From: “Moreira, Alberto”
> To: “NT Developers Interest List”
> Sent: Wednesday, December 11, 2002 6:13 PM
> Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores
>
>
> > Unless the APIC has the ability to queue interrupts, that will be of
> limited
> > usefulness. The last machine I worked with that could queue
> interrupts by
> > hardware was the Sperry Univac 418III, way back in the seventies and
> > eighties. Anybody know any other systems that do it ?
> >
> > Alberto.
> >
> >
> > -----Original Message-----
> > From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
> > Sent: Wednesday, December 11, 2002 4:20 AM
> > To: NT Developers Interest List
> > Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose
> Flores
> >
> >
> > > issues an STI. This leaves a small window within which an
> interrupt
> > could
> > > theoretically be missed
> >
> > It will not. It will be put on hold on PIC/APIC.
> >
> > Max
> >
> >
> >
> > —
> > You are currently subscribed to ntdev as:
> xxxxx@compuware.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> >
> > The contents of this e-mail are intended for the named addressee
> only. It
> > contains information that may be confidential. Unless you are the
> named
> > addressee or an authorized designee, you may not copy or use it, or
> disclose
> > it to anyone else. If you received it in error please notify us
> immediately
> > and then destroy it.
> >
> >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@hollistech.com
> To unsubscribe send a blank email to %%email.unsub%%

>"Four PCI cards fire quasi at the same moment ( assume 100 ns time

difference ) an interrupt. Card1 , card2 and card3 have different
IRQ’s ,
card4 shares the same IRQ with card3.
I would like to see a detailled flow of all handschaking actions
between the

I will omit cards1, 2, 3 and describe only cards 3 and 4.

  • card 3 starts to pull INTA# down. Note that the line is
    open-collector.
  • card 4 starts to pull INTA# down.
  • this pull-down causes the PIC to interrupt the CPU.
  • the PIC’s mask is raised to this IRQ.
  • the CPU, after possibly executing STI, starts to execute a vector
  • the NT kernel enumerates all drivers on these vectors
  • it calls ISR for card 3
  • the ISR writes something to card 3, which stops it from pulling
    INTA# down.
  • nevertheless, INTA# is still pulled down by card 4
  • card 3 ISR done, the NT kernel calls card 4’s ISR
  • the ISR writes something to card 4, which stops it from pulling
    INTA# down.
  • nobody more pulls INTA# down. It is pulled up by a resistor.
  • PIC returns to passive (no interrupt pending) mode
  • the NT kernel returns from the last ISR and executes IRET.

Max

Sorry, what is queuing an interrupt?
The PCI card just holds INTA# down till the driver’s ISR will reset it
from this state. So, no more interrupts from the same PCI card can
occur during this :slight_smile:

Max

----- Original Message -----
From: “Mark Roddy”
To: “NT Developers Interest List”
Sent: Wednesday, December 11, 2002 11:46 PM
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

> I thought the APIC in fact had the ability to queue one or perhaps
two interrupts per source
> if no target were ready to service the interrupt.
>
> ===========================
> Mark Roddy
> Consultant, Microsoft DDK MVP
> Hollis Technology Solutions
> xxxxx@hollistech.com
> www.hollistech.com
> 603-321-1032
>
>
> > The PCI bus will hold the IRQ signal till the driver’s ISR will
reset
> > it. With any PIC, be it 8259 or APIC.
> >
> > Max
> >
> > ----- Original Message -----
> > From: “Moreira, Alberto”
> > To: “NT Developers Interest List”
> > Sent: Wednesday, December 11, 2002 6:13 PM
> > Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose
Flores
> >
> >
> > > Unless the APIC has the ability to queue interrupts, that will
be of
> > limited
> > > usefulness. The last machine I worked with that could queue
> > interrupts by
> > > hardware was the Sperry Univac 418III, way back in the seventies
and
> > > eighties. Anybody know any other systems that do it ?
> > >
> > > Alberto.
> > >
> > >
> > > -----Original Message-----
> > > From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
> > > Sent: Wednesday, December 11, 2002 4:20 AM
> > > To: NT Developers Interest List
> > > Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose
> > Flores
> > >
> > >
> > > > issues an STI. This leaves a small window within which an
> > interrupt
> > > could
> > > > theoretically be missed
> > >
> > > It will not. It will be put on hold on PIC/APIC.
> > >
> > > Max
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntdev as:
> > xxxxx@compuware.com
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> > >
> > >
> > > The contents of this e-mail are intended for the named addressee
> > only. It
> > > contains information that may be confidential. Unless you are
the
> > named
> > > addressee or an authorized designee, you may not copy or use it,
or
> > disclose
> > > it to anyone else. If you received it in error please notify us
> > immediately
> > > and then destroy it.
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@hollistech.com
> > To unsubscribe send a blank email to %%email.unsub%%
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to %%email.unsub%%
>

> The PCI card just holds INTA# down till the driver’s ISR will reset it

How ? I thought the hardware signal INTA# is reset by the PIC. There
are no software instructions for that. The OS acknowledges the PIC
in different ways, see “Lazy” vs. “Strict”

Sorry Max, this is too popular. I like to see the cycle loop when the PIC
puts the “int”
instruction on the bus, when in the mean time other cards take the INTA#
down. I also
like to know the reaction of the PIC when priorities are changed and /or
interrupts
are masked off within the “Lazy” and “Strict” model, also when the PIC
cannot
push a new “int” and vector because of a cleared interrupt flag, even then
when it
has already been ACK’d by the processor, etc…
Note : everything I state here is more or less an assumption until I get a
non-popular
description

Christiaan

----- Original Message -----
From: “Maxim S. Shatskih”
To: “NT Developers Interest List”
Sent: Wednesday, December 11, 2002 10:52 PM
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

> >"Four PCI cards fire quasi at the same moment ( assume 100 ns time
> >difference ) an interrupt. Card1 , card2 and card3 have different
> IRQ’s ,
> >card4 shares the same IRQ with card3.
> >I would like to see a detailled flow of all handschaking actions
> between the
>
> I will omit cards1, 2, 3 and describe only cards 3 and 4.
>
> - card 3 starts to pull INTA# down. Note that the line is
> open-collector.
> - card 4 starts to pull INTA# down.
> - this pull-down causes the PIC to interrupt the CPU.
> - the PIC’s mask is raised to this IRQ.
> - the CPU, after possibly executing STI, starts to execute a vector
> - the NT kernel enumerates all drivers on these vectors
> - it calls ISR for card 3
> - the ISR writes something to card 3, which stops it from pulling
> INTA# down.
> - nevertheless, INTA# is still pulled down by card 4
> - card 3 ISR done, the NT kernel calls card 4’s ISR
> - the ISR writes something to card 4, which stops it from pulling
> INTA# down.
> - nobody more pulls INTA# down. It is pulled up by a resistor.
> - PIC returns to passive (no interrupt pending) mode
> - the NT kernel returns from the last ISR and executes IRET.
>
> Max
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@Compaqnet.be
> To unsubscribe send a blank email to %%email.unsub%%
>

PCI controllers are generally designed so that their interrupt signal output is “asserted” 'til some systems software (like a device driver, usually code in a first-level or second-level interrupt handler) performs a well-defined operation to clear the condition which caused the controller to “assert” the interrupt to start with.

So, I think the answer to this question is “by definition of a PCI 2.1 (and lesser version) controller level-sensitive interrupt mechanism”.

When PCI 2.2-2.3 becomes ubiquitous, we’ll be discussing “how does Message Signaled Interrupts” work.

Cheers

-----Original Message-----
From: Christiaan Ghijselinck [mailto:xxxxx@CompaqNet.be]
Sent: Wednesday, December 11, 2002 4:12 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

The PCI card just holds INTA# down till the driver’s ISR will reset it

How ? I thought the hardware signal INTA# is reset by the PIC. There
are no software instructions for that. The OS acknowledges the PIC
in different ways, see “Lazy” vs. “Strict”

> Note : everything I state here is more or less an assumption until I
get a

non-popular
description

You can find it on Intel’s site - for both 8259 and APIC.

Max

> > The PCI card just holds INTA# down till the driver’s ISR will
reset it

How ? I thought the hardware signal INTA# is reset by the PIC.

No.
The only way of resetting it is the card-dependent one (writing to
status or “IRQ disable” register), and only the card’s driver can (and
must) do this.

are no software instructions for that. The OS acknowledges the PIC

The only “acknowledgement” for PCI IRQs is the card-dependent one.

Max

Thanks, I know that these fine documents exist. But I would like to see an
even
detailled desciption on how the OS works with the CPU and PIC/APIC in a
timely
manner. This would allow us to link both desriptions, construct time related
x/y-axis
flow diagrams and getting a deep insight of what is possible/not allowed.
Much of
that hard problems that occur “one or two times a week/month” could be
avoided.

----- Original Message -----
From: “Maxim S. Shatskih”
To: “NT Developers Interest List”
Sent: Thursday, December 12, 2002 2:15 PM
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

> > Note : everything I state here is more or less an assumption until I
> get a
> > non-popular
> > description
>
> You can find it on Intel’s site - for both 8259 and APIC.
>
> Max
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@Compaqnet.be
> To unsubscribe send a blank email to %%email.unsub%%
>

> Much of

that hard problems that occur “one or two times a week/month” could
be
avoided.

NT never loses an interrupt. It can only deliver it a bit later then
you want, but not lose.

Max

I have understand that now. This brings me upon the idea how long this
could take when a driver acknownledges the card within a DPC ( just as
MS prescribes ) that WILL be queued at the bottom : 10 us, 50 us 100 us ?

To give you an idea, my NE2000 compatible driver “consumes” about
15 us on old ISA NIC. By teh way, is it a hard rule that a card should
hold
its INTA# as long as it does not receive a response ?

----- Original Message -----
From: “Maxim S. Shatskih”
To: “NT Developers Interest List”
Sent: Thursday, December 12, 2002 2:13 PM
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

> > > The PCI card just holds INTA# down till the driver’s ISR will
> reset it
> >
> > How ? I thought the hardware signal INTA# is reset by the PIC.
>
> No.
> The only way of resetting it is the card-dependent one (writing to
> status or “IRQ disable” register), and only the card’s driver can (and
> must) do this.
>
> > are no software instructions for that. The OS acknowledges the PIC
>
> The only “acknowledgement” for PCI IRQs is the card-dependent one.
>
> Max
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@Compaqnet.be
> To unsubscribe send a blank email to %%email.unsub%%
>

With hardware interrupt handling you traverse into the deepest, darkest secrets of systems programming.

I’ve broken down below into the following:

  1. Interrupts exist

  2. IO Bus Side Interrupt signaling

  3. IO Bus Interrupt topology

  4. Host side interrupt signaling

  5. Host side interrupt signaling evolution

  6. Software servicing of interrupts

  7. Common mistakes

  8. Live within the model

  9. Interrupts exist

The first realization is that interrupts exist at all. Basically their intent is to inform that something that you expect to happen or might have expected to happen may have actually happened.

  1. IO Bus Side Interrupt signaling

At the IO bus level (ISA, PCI), interrupts are dedicated signal traces from a device to the host bridge that attaches the IO bus to the remainder of the system (memory, CPUs, other IO buses). These signal traces can be thought of as unidirectional, in that the device drives the signal line, whereas the host bridge samples/responds to the signal line.

Whether or not the signal traces (interrupt lines) can be shared by more than one device is a function of the IO bus design. ISA buses do not provide sharing of these interrupt lines, whereas PCI does. In order to facilitate sharing devices employ level sensitive interrupts. By electrical design, there is a pull up or down in h/w on the signal so that if no device is asserting the signal it appears as deasserted. When a device wishes to signal a level sensitive interrupt, it drives or sinks the signal to the opposite state thus making the interrupt line asserted. Electrically, these shared signals are special because you can have two or more devices driving or sinking the current (WIRE-OR or WIRE-AND) without burning out the h/w circuits in the other devices that are also asserting the signal.

When a device has asserted a level sensitive interrupt, it keeps the interrupt line asserted until an IO bus transaction clears the interrupt condition. This IO bus transaction is normally initiated by software. Typically, s/w reads a device interrupt status register to determine the actual cause of the interrupt and then writes back the set of bits that the s/w serviced. So from a h/w perspective each device has 3 interrupt related registers. One is the interrupt status bits, indicating which interrupts conditions are occurring. An interrupt status mask which s/w can use to mask off particular interrupt status bits so that they do not cause an interrupt, and finally an interrupt acknowledge register. By design the h/w will assert its interrupt line if the result of anding the interrupt status bits with the interrupt mask register is non-zero. Software normally reads the interrupt status bits (sometimes there is an additional register called interrupt pending which is the anded result of the interrupt status and mask) to determine if its device is interrupting. If so the s/w employs whatever s/w logic is needed to service the condition. When s/w has finished servicing all the unmasked interrupt status conditions it does a write back to the interrupt acknowledge register specifying the bits that it has serviced. Thus causes the h/w to re-evaluate the conditions feeding into the interrupt status register. Note there is no-potential of missing an interrupt here as long as the software only acknowledges the interrupt conditions that it has actually serviced. The h/w will keep the interrupt line asserted while unmasked interrupt status bits are still valid.

So you may ask, why with level sensitive interrupts isn’t the processor being interrupted all the time? This is solved with an mechanism called EOI (End of Interrupt). The host bridge examines the interrupt lines and maintains state about each interrupt line. When it detects that a level triggered interrupt line is asserted it schedules a host bus transaction to inform the system about the interrupt. When this is internally scheduled, it marks the internal state for that interrupt line so that it doesn’t schedule a new host bus transaction to report that interrupt again until s/w does a host bus transaction to the host bridge to EOI that particular interrupt source. The EOI causes the host bridge to re-enable its internal logic associated with that interrupt line and re-examine the interrupt signal to determine if its still asserted. If the line is still asserted then the whole process starts over again.

On ISA buses, devices used Edge Triggered interrupts. The triggering edge could be a rising or falling edge of the signal (i.e. only one side of the signal transition signals the interrupt). PCI devices can also use edge triggered interrupts, but level triggered is preferred mechanism for PCI.

The drawback with Edge Triggered interrupts is that they cannot be easily shared amongst multiple devices. The ISA or PCI device that uses Edge Triggered interrupts will need different logic to create a signal interrupt line pulse (and not generate another one) until s/w has explicitly told it that it is done servicing interrupts. This is typically the folly of many of the edge triggered designs.

One of the advantages of the Edge Triggered interrupts is that the host bridge doesn’t need an EOI. It can accept another interrupt from an edge triggered device as soon as it has sent the previous host bus transaction to report the previous interrupt. If the edge triggered interrupt occurs again before the previous host bus transaction has completed, the behavior is host bridge dependent. Sometimes the host bridge can latch one additional edge, other host bridges report this as a spurious interrupt.

  1. IO Bus Interrupt topology

Since ISA devices use edge triggered interrupts and edge triggered interrupts aren’t easily shared, each ISA 16 bit bus slot has access to all interrupt signal lines. Thus each device needs to be configured to uniquely use just one of these interrupt lines. In the 1980’s this was done with jumpers. ISA plug and play came around and with some additional device logic the interrupt line selection could be done programmatically.

PCI defines four interrupt lines per device A, B, C and D. By default, if a PCI device only uses one interrupt it should only use interrupt A. The motherboard designer selects how the A, B, C and D interrupt lines from each PCI slot are routed to the host interrupt controller. Normally, the A(s) from each slot are routed to unique interrupt signal pins on the host interrupt controller. This mapping from of the interrupt lines from each slot to the host interrupt controller is known a priori by the system BIOS. This information is reported to the host OS during system boot.

You may have noticed in this section I started referring to the host interrupt controller instead of the host bridge. This is because in early systems the host interrupt controller was a unique set of devices from the host bridge. Thus interrupts are in essence handled (originally physically) logically on a different bus. However, as time as passed the two functions have merged to interoperate on top of the same infrastructure.

On original 16 bit ISA PCs, the host interrupt controller was 2 cascaded Intel 8259A Programmer Interrupt Controllers. One device was the master interrupt controller, the second was a slave (a feature of the 8259A). This master/slave design allowed up to 64 interrupts with 9 devices, but the original PC design only used two.

Each 8259A has 8 interrupt line inputs. Each could be configured as edge or level triggered. On the master device an interrupt line could be configured as cascaded (indicating that a second 8259A was driving that interrupt line input). Thus this cascading caused the loss of one interrupt line in exchange for 8 more, thus we had 15 interrupts in the original PC design.

Some of these interrupts were dedicated to system resources, timer, clock, math co-processor interrupt. Others ended up being available to be used as IO bus interrupt signal inputs.

  1. Host side interrupt signaling

Amazingly the 8259A(s) appeared as devices on the ISA bus itself, since they had programmable registers (e.g. interrupt masks, configuration, interrupt status, etc). Thus the ISA bus design included special transaction types that allowed the 8259A to tell the system (i.e. essentially the host bus transaction I mentioned above) about an interrupt that occurred. This was done with a two cycle approach. The master 8259 actually had an interrupt output that was wired directly to the CPU. When the CPU was ready and the interrupt was asserted it would generate an interrupt special cycle indicating that it was ready to receive the interrupt (at this point the 8259A would deassert its interrupt output). The CPU would follow this with an interrupt acknowledge read cycle, which caused either the master or slave 8259A to place the corresponding 8 bit interrupt vector onto the data bus corresponding to the interrupt that was to be serviced). Thus when the CPU had interrupts disabled it would not generate any special interrupt cycles, and interrupts were essentially held in the 8259A(s) until the CPU was ready to accept them.

  1. Host side interrupt signaling evolution

As time passed the performance of the 8259A, along with the number of supported interrupts became design limitations. This along with the integration of legacy ISA devices into SuperIO style chips allowed Intel to redesign the h/w mechanisms used to deliver host side interrupts while preserving the 8259A programming model. Eventually, the 8259A programming model was itself deemed antiquated, which gave rise to the APIC and SAPIC designs that people are really just starting to understand today.

So logically, APIC(s) have some additional capabilities that the 8259A(s) lacked. The most important (I think) is the ability to signal an interrupt to a particular processors or a group of processors, along with being able to specify interrupt priority.

If you research you will discover that Intel has evolved the way that host side interrupts are delivered to the processors. The Pentium III(s) had a special bus called the APIC bus. Each processor, and the host bridge would share this serial bus. Interprocessor interrupts were message types on this bus, along with interrupt messages from the host bridge. One of the neat things is that the APIC in the host bridge could be configured to deliver an interrupt to all processors (actually only one would service it), but the APIC devices in each processor would arbitrate amongst themselves to determine who was best suited to service the interrupt. There are a set of registers in the Pentium III used to control this selection which are under O/S control.

If you look at a Pentium III systems programmers manual you can see the programming details for the APIC unit in each processor. Each processor has one and it logically appears as device registers.

Logically the APIC maintains an interrupt bit for each interrupt condition that it has accepted that it has not dispatched internally. When the processor dispatches the interrupt (i.e. it vectors through the IDT), it clears that bit which allows the APIC to accept another interrupt for that vector.

S/W is responsible for knowing whether the corresponding interrupt was initiated by the an IO device and whether or not an EOI needs to be generated.

There is a special APIC transaction to generate an EOI.

I don’t believe there is any interrupt priority per se with h/w generated interrupts within the processor. This is managed by s/w using interrupt priority registers defined by the Pentium III.

Intel has continued to evolve the APIC architecture. The latest is SAPIC which preserves the programming model but gets rid of the separate serial APIC bus.

  1. Software servicing of interrupts

The nuances of handling priority and servicing interrupts are very OS dependent. I think Jake has done a nice job of explaining some of the performance tweaks in Windows to handle the slow access times to legacy 8259 based designs.

There are a few more details about Windows that I can add:

Each interrupt handler at a minimum needs to get its device to stop interrupting. Ideally your device will have a device level interrupt enable/disable (separate from the interrupt mask I mentioned above). Basically the device interrupt enable/disable is a MASK on the interrupt line before it leaves the device.

Your s/w should interrogate your device normally using a memory mapped or IO bus device access to determine if your device is interrupting (i.e. interrupt pending register is non-zero). If it is interrupting then disable the device interrupt enable for your device, schedule a DPC and return TRUE out of your interrupt handler.

By returning TRUE when your device is interrupting, Windows will short circuit the rest of the interrupt handler(s) that are sharing the same interrupt with your device (assuming that your device is a level sensitive interrupt).

Windows is also responsible for sending the EOI to acknowledge the interrupt to the host bridge.

  1. Common mistakes

If your device doesn’t have a device level interrupt enable/disable (separate from the interrupt mask I mentioned above), then make sure you protect changes to the interrupt mask using an interrupt spin-lock since your interrupt handler will need to modify the device interrupt mask directly.

The much more common mistake is that you designed your interrupt handler differently than I described in the previous paragraph. You are actually trying to service conditions, pass information to your DPC, schedule multiple DPC(s), etc. You are making the game much too complicated and inefficient. If you do your job right you will only have one instance of your DPC queued/running at any time. You want your DPC to be looking at the pending interrupt conditions, writing the interrupt acknowledge register and re-enabling the device level interrupt. If you play the game this way you will avoid many of the programming pitfalls that I’ve seen in interrupt handling design.

  1. Live within the model

There is no short answer to your question. It is motherboard, bus, host-bridge, BIOS, HAL, OS Version and Interrupt Handler dependent.

As you can tell, the underlying mechanisms have evolved significantly overtime. Unfortunately though the OS(s) don’t always provide us device driver writers with the additional programming interfaces to control interrupt mechanisms that may be available on the underlying h/w. The OS(s) themselves often live with the interrupt priorities and routing set up by the BIOS/HAL.

Hopefully, in the future Windows will provide documented access methods to facilitate priority and interrupt routing. Until then live within the model, or we are doomed not to interoperate.

Duane J. McCrory
InfiniCon Systems

-----Original Message-----
From: Christiaan Ghijselinck [mailto:xxxxx@CompaqNet.be]
Sent: Wednesday, December 11, 2002 5:30 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

Sorry Max, this is too popular. I like to see the cycle loop when the PIC
puts the “int”
instruction on the bus, when in the mean time other cards take the INTA#
down. I also
like to know the reaction of the PIC when priorities are changed and /or
interrupts
are masked off within the “Lazy” and “Strict” model, also when the PIC
cannot
push a new “int” and vector because of a cleared interrupt flag, even then
when it
has already been ACK’d by the processor, etc…
Note : everything I state here is more or less an assumption until I get a
non-popular
description

Christiaan

----- Original Message -----
From: “Maxim S. Shatskih”
To: “NT Developers Interest List”
Sent: Wednesday, December 11, 2002 10:52 PM
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

> >"Four PCI cards fire quasi at the same moment ( assume 100 ns time
> >difference ) an interrupt. Card1 , card2 and card3 have different
> IRQ’s ,
> >card4 shares the same IRQ with card3.
> >I would like to see a detailled flow of all handschaking actions
> between the
>
> I will omit cards1, 2, 3 and describe only cards 3 and 4.
>
> - card 3 starts to pull INTA# down. Note that the line is
> open-collector.
> - card 4 starts to pull INTA# down.
> - this pull-down causes the PIC to interrupt the CPU.
> - the PIC’s mask is raised to this IRQ.
> - the CPU, after possibly executing STI, starts to execute a vector
> - the NT kernel enumerates all drivers on these vectors
> - it calls ISR for card 3
> - the ISR writes something to card 3, which stops it from pulling
> INTA# down.
> - nevertheless, INTA# is still pulled down by card 4
> - card 3 ISR done, the NT kernel calls card 4’s ISR
> - the ISR writes something to card 4, which stops it from pulling
> INTA# down.
> - nobody more pulls INTA# down. It is pulled up by a resistor.
> - PIC returns to passive (no interrupt pending) mode
> - the NT kernel returns from the last ISR and executes IRET.
>
> Max
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@Compaqnet.be
> To unsubscribe send a blank email to %%email.unsub%%
>


You are currently subscribed to ntdev as: xxxxx@infiniconsys.com
To unsubscribe send a blank email to %%email.unsub%%

> I have understand that now. This brings me upon the idea how long
this

could take when a driver acknownledges the card within a DPC

No. Within the ISR. MS prescribes this to be the first thing done in
the ISR.

15 us on old ISA NIC. By teh way, is it a hard rule that a card
should
hold
its INTA# as long as it does not receive a response ?

Yes.
Again - this is on PCI’s level-triggered IRQs, ISA’s edge-triggered
use completely other rules.

Max

Probably not, but I saw cases were ALL interrupts were locked up, nothing
came
in anymore. And this occurred very sporadic.

----- Original Message -----
From: “Maxim S. Shatskih”
To: “NT Developers Interest List”
Sent: Thursday, December 12, 2002 4:23 PM
Subject: [ntdev] RE: Device Interrupt priority - Reviewing Jose Flores

> > Much of
> > that hard problems that occur “one or two times a week/month” could
> be
> > avoided.
>
> NT never loses an interrupt. It can only deliver it a bit later then
> you want, but not lose.
>
> Max
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@Compaqnet.be
> To unsubscribe send a blank email to %%email.unsub%%
>

>One of the advantages of the Edge Triggered interrupts is that the
host bridge doesn’t need

an EOI.

It needs a EOI in

mov al, 20h
out 20h, al

form.

Max