Message
----- Original Message -----
From: Moreira, Alberto
To: NT Developers Interest List
Sent: Wednesday, October 17, 2001 11:44 AM
Subject: [ntdev] RE: Number of processing DPC objects
I don’t have much recent experience with this, but if a DPC will drain the interrupt queue, the ISR itself can just table the interrupt and ready the hardware for another interrupt, right ? All the ISR has to do is to queue the interrupts, and the DPC will drain the queue when it gets dispatched. No need to disable-in-isr, unless your hardware has special needs; here I tend to agree with Mark, it may be a long time between the disable-in-isr and the enable-in-dpc, you may lose more peripheral bandwidth than you’re willing to. I like to look at ISRs as hardware
Except that each of those interrupts incurs additional overhead.
Actually, the disable-in-isr and the enable-in-dpc crowd is really asking for the ISR to occur at DISPATCH_LEVEL. If we could get that, there would be no point in scheduling the DPC at all, we could do all the work in the ISR. Since the ISR wouldn’t run as often, we’d have less interrupts (assuming hardware queuing). The only point in having a higher level ISR than DISPATCH_LEVEL is if you actually do something to the device in the ISR beyond ACK’ing the interrupt (or your OS/hardware doesn’t let you control this).
-DH
handlers, and DPC as OS babysitters - the ISR is short and sweet, gets enough information to get going and resets the hardware for the next one, while the DPC picks up what the ISR enqueued and passes it up to the OS at the speed the OS is willing to take it.
This reminds me of the Sperry DCP, it had a central processor and multiple peripheral processors. The peripheral processors would handle all interrupts, and the pp program would enqueue the request into a hardware queue in shared memory. That queue would have been previously “armed” so that a new request into the queue would cause a “software attention item” to be enqueued into yet another queue, typically, an OS dispatch queue. So, The pp enqueuing an interrupt item would automatically schedule the corresponding dpc, all by hardware. The dpc, when dispatched, would empty the interrupt queue and pass each item up to the OS. In between the time the first interrupt arived and the time the dpc got activated, other interrupts could come down that same queue, the pp would handle the hardware transparent to the dpc and to the os. I remember that when the Sperry Monitor product was developed, the real-time executive’s dispatcher loop had precisely 15 machine instructions.
But they don’t make them like that anymore. 
Another thing: if DMA setup is so critical, how about a common-buffer strategy ? You may be able to keep a single DMA transaction alive for a long time, switching buffers if necessary. In our ICD, for example, we had three or more DMA buffers, sometimes in high throughput conditions we’d have as much as 32Mb or more worth of active DMA buffering between our OpenGL renderer and the graphics chip. Also, with the Pentium SIMD extensions, moving data can be done quite fast, so even if you have to shuffle data buffers around, it may still be a feasible proposition.
Alberto.
-----Original Message-----
From: Dave Harvey [mailto:xxxxx@syssoftsol.com]
Sent: Tuesday, October 16, 2001 10:32 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Number of processing DPC objects
Having worked with devices that queue “interrupt events”, and generate an interrupt on
each “interrupt event”, the disable-in-isr enable-in-dpc approach works well if the DPC drains
the “interrupt event” queue. In this way, you get to amortize the interrupt overhead across
multiple “interrupt events”.
Mark, if the DMA setup time is so critical, why do this in the DPC instead of the ISR?
(I guess I object to this philosophy that says you should do the least possible in the
ISR being carried to the extreme where you move time critical code out of the ISR to
a DPC, even if the overhead of that code is small compared to the ISR overhead.)
Also, in your example, I’m confused by your description. If the two DMA channels are
independent, but share an interrupt, and you have a single CPU, what’s the difference
between (ISR=>DPC=>sched DMA=>EnableInt Then ISR=>DPC->sched DMA)
and two independent DPCs that may be queued at the same time? Or were you failing to
check to see if the other DMA channel is ready in the DPC and servicing it before
enabling the interrupt?
With multiple CPUs, this whole question becomes more interesting. The APIC will
run the ISRs on different CPUs, which isn’t always a win, since each ISR has to
drag the dirty cache between the CPUs. I’ve found that locking an ISR (and therefore
the DPC) to a CPU using interrupt affinity often yields better performance than
letting interrupts randomly select the CPU.
-DH
----- Original Message -----
From: Roddy, Mark
To: NT Developers Interest List
Sent: Tuesday, October 16, 2001 12:56 PM
Subject: [ntdev] RE: Number of processing DPC objects
If you do as suggested by Evan and if your hardware has N independent IO channels, then you are essentially serializing access to what would otherwise be concurrent data paths. For example I have a plx9054 pci device I fool around with at home. It has, at a minimum, two independent DMA channels. Using the disable-in-isr enable-in-dpc approach achieves almost exactly half the throughput of the deserialized dpc approach, up to some level of saturation where either channel is able to consume all the bandwidth by itself. (To be clearer, the effect is most noticeable where small transfers are performed and DMA setup time is the dominant factor. For large transfers the DMA transfer operation itself dominates and the setup time and serialization is not terribly important.) I agree with Vipul that the ISR ought to record the state, queue the dpc, and then re-enable the next interrupt. Generally this is not a lot of extra processing in the ISR - read some data, store it somewhere, and it allows for parallel performance on MP platforms for your driver lower edge.
-----Original Message-----
From: Vipul Jain [mailto:xxxxx@fvc.com]
Sent: Tuesday, October 16, 2001 12:31 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Number of processing DPC objects
I agree.
But I think the right approach would be disable the interrupt from ISR - as soon as you recognize the interrupt -, save the device register values in some data structure (may be device extension), Queue the DPC and then enable the interrupt once again.
- Vipul
-----Original Message-----
From: Evan Lloyd [mailto:xxxxx@eicon.com]
Sent: Tuesday, October 16, 2001 1:55 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Number of processing DPC objects
Mark,
You may not recommend this approach but the NDIS spec does:
"MiniportISR must disable interrupts to ensure that when MiniportHandleInterrupt runs, no information in the NIC will be overwritten by a subsequent interrupt. ".
In fact, either you turn off interrupts in the ISR and do the processing in the DPC, or you are obliged to move some of the event processing into the ISR (to at least convince the card to remove the current IRQ).
From a system perspective it is much better to do the processing in the DPC.
Regards,
Evan
-----Original Message-----
From: Roddy, Mark [mailto:xxxxx@stratus.com]
Sent: 15 October 2001 15:49
To: NT Developers Interest List
Subject: [ntdev] RE: Number of processing DPC objects
In addition to the concurrency of your dpc routine (Np) it is important to note that there is not a 1-1 relationship between IoRequestDpc and queued dpc objects. That is to say, as a single dpc object can be queued at most once, an attempt to queue a dpc object that is already queued will have no effect. An invocation of a dpc routine can therefore represent 0…* interrupt events. Zero, as some other invocation of your dpc routine may have consumed the interrupt event that this invocation was triggered by, and * as some unknown number of interrupt events may have occurred before this invocation of the dpc routine started to execute (and an unknown number of interrupt events may occur while the dpc routine executes.) I have seen more than one driver that find this so annoying that they turn off interrupts from their device before requesting a dpc, and turn interrupts back on at the end of the dpc routine. I wouldn’t recommend this approach, but it is not uncommon.
-----Original Message-----
From: Primoz Beltram [mailto:xxxxx@hermes.si]
Sent: Monday, October 15, 2001 10:29 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Number of processing DPC objects
With “only one pending DPC”, I meant “my drivers DPC object” (sorry for inexactness). I know that there could be others drivers DPC objects in the DPC queues. My interest is to clear how many “my drivers DPC objects” can be in processing on a MP machine. So far it seams that point 3. is true. Thanks.
Wbr Primoz
-----Original Message-----
From: Mark Roddy [mailto:xxxxx@hollistech.com]
Sent: Monday, October 15, 2001 12:42 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Number of processing DPC objects
-
correct.
-
not sure what you meant here. Each processor dpc queue can have an unbounded number of dpc objects queued, but each dpc object can only be on one queue at a time. Easy to express in UML, not so easy to express in english.
-
depends on what you meant by (2). Each dpc object can only be either queued or not-queued. So a single dpc object can be queued at most once. This holds true while a dpc routine is executing: the related dpc object is either queued or not queued, and it can be queued at most once. The dpc routines related to a single dpc object can be executing with a concurrency of Np where Np is the number of processors.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Primoz Beltram
Sent: Monday, October 15, 2001 6:10 AM
To: NT Developers Interest List
Subject: [ntdev] Number of processing DPC objects
I didn’t find no explicit answer in books nor in electronic media, regarding the question I have.
-
On a multiprocessor NT platform, each processor has its own DPC queue. Right?
-
Then: On each(!) DPC queue, there can be only one pending DPC object. Right?
-
Then: I can (theoretically) have NumOfProcessors*2 DPC objects in processing, (one in DPC_queue[processor] + one currently processing on processor)*NumOfProcessors.
Right?
Thanks in advance for any info.
Wbr Primoz
— You are currently subscribed to ntdev as: xxxxx@fvc.com To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com -----------------------------------------------
Primoz Beltram
e-mail: xxxxx@hermes.si
www: http://www.hermes.si
HERMES SoftLab, Office Nova Gorica
Erjavceva ul. 2, 5000 Nova Gorica, Slovenia
phone: (++386 5) 33 30 510, (++386 1) 58 65 710
fax: (++386 5) 33 32 656, (++386 1) 58 65 270
You are currently subscribed to ntdev as: xxxxx@hollistech.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
You are currently subscribed to ntdev as: xxxxx@hermes.si
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
You are currently subscribed to ntdev as: xxxxx@eicon.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
You are currently subscribed to ntdev as: xxxxx@syssoftsol.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
You are currently subscribed to ntdev as: xxxxx@syssoftsol.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
