DPC latency

NTDEV
1

Hello,

I’ve to provide some information on DPC latency to hardware dept. for
Windows2000\XP PCI device deisgn. I know this depends on many factors but I
need to make some realistic assumption (wide enough) so that hardware
engineers can decide how much RAM they will need to put into the device. The
device will do Tx\Rx at 400 KBytes/sec without interruption by DMA per
channel, there will be eight such channels in one system. I’ve measured in
the past that DPC happens from 5us to 5ms after leaving ISR routine by
interrupt driven driver. In this device there won’t be many interrupts as
DMA will be used. Any idea what should I quote- 20ms, 30ms (The larger the
bettter)? Please share your experience.

Thanks,
Hakim

2

Harkim -

20-30ms is high. My experience w/dual Pentiums running at 2.4 Ghz has been
5-10us.

Dean Gereaux

-----Original Message-----
From: Hakim [mailto:xxxxx@yahoo.ca]
Sent: Tuesday, March 29, 2005 9:26 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] DPC latency

Hello,

I’ve to provide some information on DPC latency to hardware dept. for
Windows2000\XP PCI device deisgn. I know this depends on many factors but I
need to make some realistic assumption (wide enough) so that hardware
engineers can decide how much RAM they will need to put into the device. The

device will do Tx\Rx at 400 KBytes/sec without interruption by DMA per
channel, there will be eight such channels in one system. I’ve measured in
the past that DPC happens from 5us to 5ms after leaving ISR routine by
interrupt driven driver. In this device there won’t be many interrupts as
DMA will be used. Any idea what should I quote- 20ms, 30ms (The larger the
bettter)? Please share your experience.

Thanks,
Hakim

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@amcc.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

3

We just had a machine show a 17ms interrupt latency (it also showed some
problems, which is why we started looking at it in the first place). Don’t
ask me how or why… It seemed to cure itself at some later point…

It’s reasonable to assume that ISR’s aren’t going to take much more than a
few hundred milliseconds on 99% of all machines and configurations.
However, some drivers do REALLY stupid things in their interrupt handler,
and may well cause all sorts of bad latency. Microsoft are on the warpath
on this, as more than a few hundred milliseconds of interrupt latency will
disrupt real-time audio and real-time video, unless you put up big buffers,
which isn’t a particularly good solution.

So what am I saying: Well, it depends on whether you want belt, braces and
a couple of safety-pins in there, or just something that will work in MOST
cases. What happens if you overflow one of your buffers? Does a nuclear
disaster happen, airplane falls out of the sky, some person calling on a
mobile gets cut off, or just a little bit of missing data from a
weather-station?

Is your hardware going to be installed on all sorts of different machines,
or can you give “selected models” that you recommend and have thoroughly
tested for compatibility, and refer the customer to this list if they are
having problems?

400KB * 8 * 0.05 => 160KB. So that’s a big ram if it’s built into the
device, but rather a small one if it’s an external RAM-device (DRAM or
SRAM). Maybe a solution is to use a 256KB or 512KB external SRAM? Should
give you plenty of space.

–
Mats

xxxxx@lists.osr.com wrote on 03/29/2005 06:26:02 PM:

Hello,

I’ve to provide some information on DPC latency to hardware dept. for
Windows2000\XP PCI device deisgn. I know this depends on many factors but
I
need to make some realistic assumption (wide enough) so that hardware
engineers can decide how much RAM they will need to put into the device.
The
device will do Tx\Rx at 400 KBytes/sec without interruption by DMA per
channel, there will be eight such channels in one system. I’ve measured
in
the past that DPC happens from 5us to 5ms after leaving ISR routine by
interrupt driven driver. In this device there won’t be many interrupts as

DMA will be used. Any idea what should I quote- 20ms, 30ms (The larger
the
bettter)? Please share your experience.

Thanks,
Hakim

Questions? First check the Kernel Driver FAQ at http://www.
osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@3dlabs.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

ForwardSourceID:NT0000F9FA

4

Yes, I forgot to mention about the interrupt latency also that I also saw in
the range of 8-10ms in an old machine of 500MHz without any extra
peripeheral on ISA, PCI buses except ours. That is another thing I need to
tell them. The device can not underrun or overrun at any moment and it will
be used to communicate with delicate equipments where 100% data accuracy is
needed, that’s why we are looking at big figures, putting 1MB RAM is no
problem.

Thanks,
Hakim

“Mats PETERSSON” wrote in message
news:xxxxx@ntdev…
>
>
>
>
>
> We just had a machine show a 17ms interrupt latency (it also showed some
> problems, which is why we started looking at it in the first place). Don’t
> ask me how or why… It seemed to cure itself at some later point…
>
> It’s reasonable to assume that ISR’s aren’t going to take much more than a
> few hundred milliseconds on 99% of all machines and configurations.
> However, some drivers do REALLY stupid things in their interrupt handler,
> and may well cause all sorts of bad latency. Microsoft are on the warpath
> on this, as more than a few hundred milliseconds of interrupt latency will
> disrupt real-time audio and real-time video, unless you put up big
> buffers,
> which isn’t a particularly good solution.
>
> So what am I saying: Well, it depends on whether you want belt, braces and
> a couple of safety-pins in there, or just something that will work in MOST
> cases. What happens if you overflow one of your buffers? Does a nuclear
> disaster happen, airplane falls out of the sky, some person calling on a
> mobile gets cut off, or just a little bit of missing data from a
> weather-station?
>
> Is your hardware going to be installed on all sorts of different machines,
> or can you give “selected models” that you recommend and have thoroughly
> tested for compatibility, and refer the customer to this list if they are
> having problems?
>
> 400KB * 8 * 0.05 => 160KB. So that’s a big ram if it’s built into the
> device, but rather a small one if it’s an external RAM-device (DRAM or
> SRAM). Maybe a solution is to use a 256KB or 512KB external SRAM? Should
> give you plenty of space.
>
> –
> Mats
>
> xxxxx@lists.osr.com wrote on 03/29/2005 06:26:02 PM:
>
>> Hello,
>>
>> I’ve to provide some information on DPC latency to hardware dept. for
>> Windows2000\XP PCI device deisgn. I know this depends on many factors but
> I
>> need to make some realistic assumption (wide enough) so that hardware
>> engineers can decide how much RAM they will need to put into the device.
> The
>> device will do Tx\Rx at 400 KBytes/sec without interruption by DMA per
>> channel, there will be eight such channels in one system. I’ve measured
> in
>> the past that DPC happens from 5us to 5ms after leaving ISR routine by
>> interrupt driven driver. In this device there won’t be many interrupts as
>
>> DMA will be used. Any idea what should I quote- 20ms, 30ms (The larger
> the
>> bettter)? Please share your experience.
>>
>> Thanks,
>> Hakim
>>
>>
>>
>> —
>> Questions? First check the Kernel Driver FAQ at http://www.
>> osronline.com/article.cfm?id=256
>>
>> You are currently subscribed to ntdev as: xxxxx@3dlabs.com
>> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>> ForwardSourceID:NT0000F9FA
>
>

5

I didn’t expect a 500MHz Pentium/K6 machine to have that high interrupt
latency. On a 486 with some decent interrupt handling code (note: Not
Windows) you get interrupt latency in the range of 1-5 milliseconds.
However, on a windows machine, you have to contend with things like paging
and such. If the old machine doesn’t have a DMA-based IDE controller you’d
also see interrupt/DPC latency that is pretty horrendous because the IDE
driver has to do the read-out from the hard-disk (or write-on to) in PIO
mode, which will, if I remember correctly, be done in either the interrupt
handler or a DPC.

The above is mostly hardware interrupt based. Of course, the other thing to
consider is “worst case DPC”. If there is a DPC in there that takes 10
milliseconds to finish, on a single processor machine, your DPC isn’t going
to get to run until the current DPC is finished.

DPC’s are used to do things that take more time than what’s allowed in an
interrupt. They still shouldn’t do really lengthy things, but they are
allowed to do things that take a bit longer than in an interrupt. Every
programmer has their own idea of what “long” is, but from memory, Microsoft
proposes a maximum of 10-25 microseconds in interrupts and 50 in DPC’s,
with no more than 100 microseconds per millisecond at any time. That will
guarantee that audio/video playback works correctly, they said at 2003
Driver Developers Conference. But many drivers break these limits today,
and your biggest culprit is probably DPC’s rather than interrupt latency,
but either could be bad with the wrong driver. I remember some creative
labs sound card drivers that couldn’t cope with 1GHz+ Athlon processors,
they would “interrupt storm”, which caused the whole system to run at an
apparent “half-speed” (including CPU-speed calculation software, which is
how AMD’s tech-support ended up with the question from customers. Replacing
the driver would solve that one…). These sort of crappy drivers could
well cause problems for your project, so careful selection on the list of
“Compatible software and hardware” is probably an important factor here.

As you say, if the equipment is sensitive to loss of data, you’re much
better of putting extra big buffers on there that, most of the time keeping
those buffers almost empty (on the read-side, almost full on the write
side), but sometimes using up more of the buffer if a bad spell happens.
1MB of RAM shouldn’t cost many dollars, and I suspect the cost of other
bits of the equipment (or the consequences of data loss) is MUCH more
expensive.

–
Mats

xxxxx@lists.osr.com wrote on 03/29/2005 06:54:16 PM:

Yes, I forgot to mention about the interrupt latency also that I also saw
in
the range of 8-10ms in an old machine of 500MHz without any extra
peripeheral on ISA, PCI buses except ours. That is another thing I need
to
tell them. The device can not underrun or overrun at any moment and it
will
be used to communicate with delicate equipments where 100% data accuracy
is
needed, that’s why we are looking at big figures, putting 1MB RAM is no
problem.

Thanks,
Hakim

“Mats PETERSSON” wrote in message
> news:xxxxx@ntdev…
> >
> >
> >
> >
> >
> > We just had a machine show a 17ms interrupt latency (it also showed
some
> > problems, which is why we started looking at it in the first place).
Don’t
> > ask me how or why… It seemed to cure itself at some later point…
> >
> > It’s reasonable to assume that ISR’s aren’t going to take much more
than a
> > few hundred milliseconds on 99% of all machines and configurations.
> > However, some drivers do REALLY stupid things in their interrupt
handler,
> > and may well cause all sorts of bad latency. Microsoft are on the
warpath
> > on this, as more than a few hundred milliseconds of interrupt latency
will
> > disrupt real-time audio and real-time video, unless you put up big
> > buffers,
> > which isn’t a particularly good solution.
> >
> > So what am I saying: Well, it depends on whether you want belt, braces
and
> > a couple of safety-pins in there, or just something that will work in
MOST
> > cases. What happens if you overflow one of your buffers? Does a nuclear
> > disaster happen, airplane falls out of the sky, some person calling on
a
> > mobile gets cut off, or just a little bit of missing data from a
> > weather-station?
> >
> > Is your hardware going to be installed on all sorts of different
machines,
> > or can you give “selected models” that you recommend and have
thoroughly
> > tested for compatibility, and refer the customer to this list if they
are
> > having problems?
> >
> > 400KB * 8 * 0.05 => 160KB. So that’s a big ram if it’s built into the
> > device, but rather a small one if it’s an external RAM-device (DRAM or
> > SRAM). Maybe a solution is to use a 256KB or 512KB external SRAM?
Should
> > give you plenty of space.
> >
> > –
> > Mats
> >
> > xxxxx@lists.osr.com wrote on 03/29/2005 06:26:02 PM:
> >
> >> Hello,
> >>
> >> I’ve to provide some information on DPC latency to hardware dept. for
> >> Windows2000\XP PCI device deisgn. I know this depends on many factors
but
> > I
> >> need to make some realistic assumption (wide enough) so that hardware
> >> engineers can decide how much RAM they will need to put into the
device.
> > The
> >> device will do Tx\Rx at 400 KBytes/sec without interruption by DMA per
> >> channel, there will be eight such channels in one system. I’ve
measured
> > in
> >> the past that DPC happens from 5us to 5ms after leaving ISR routine
by
> >> interrupt driven driver. In this device there won’t be many interrupts
as
> >
> >> DMA will be used. Any idea what should I quote- 20ms, 30ms (The larger
> > the
> >> bettter)? Please share your experience.
> >>
> >> Thanks,
> >> Hakim
> >>
> >>
> >>
> >> —
> >> Questions? First check the Kernel Driver FAQ at http://www.
> >> osronline.com/article.cfm?id=256
> >>
> >> You are currently subscribed to ntdev as: xxxxx@3dlabs.com
> >> To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >> ForwardSourceID:NT0000F9FA
> >
> >
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at http://www.
> osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@3dlabs.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

> ForwardSourceID:NT0000FA1E

6

>

I didn’t expect a 500MHz Pentium/K6 machine to have that high interrupt
latency. On a 486 with some decent interrupt handling code (note: Not
Windows) you get interrupt latency in the range of 1-5 milliseconds.
That’s supposed to say 1-5 microseconds, not milliseconds. I just arrived
in Alabama from England where I normally am, and it was a long flight with
lots of delays. That’s my excuse and I’m sticking to it… :wink:

–
Mats

7

xxxxx@attotech.com wrote:

I’ve recently run across a system (dual Itanium) where the average DPC
latency was very low (<1 ms), but occassionally (once a second or so)
would be 15-20 ms. I assumed some other device on the system was
taking a long time in an ISR or DPC and causing the delay.

The RealTek audio driver on the Intel 915 motherboard I have as a test
machine had a nasty habit of running KeStallExecutionProcessor with a
parameter of 10,000. Those of you keeping score at home will recognize
that this does a tight kernel-mode loop for 10 milliseconds.

I couldn’t figure out why one of the CPUs on my Hyperthreaded P4 2.8GHz
was always 35% loaded. That was why.

–

8

Then there is the Intel Ethernet driver, that on startup sits in a DPC for
over 1 second!

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

“Tim Roberts” wrote in message news:xxxxx@ntdev…
> xxxxx@attotech.com wrote:
>
>> I’ve recently run across a system (dual Itanium) where the average DPC
>> latency was very low (<1 ms), but occassionally (once a second or so)
>> would be 15-20 ms. I assumed some other device on the system was taking a
>> long time in an ISR or DPC and causing the delay.
>>
>
> The RealTek audio driver on the Intel 915 motherboard I have as a test
> machine had a nasty habit of running KeStallExecutionProcessor with a
> parameter of 10,000. Those of you keeping score at home will recognize
> that this does a tight kernel-mode loop for 10 milliseconds.
>
> I couldn’t figure out why one of the CPUs on my Hyperthreaded P4 2.8GHz
> was always 35% loaded. That was why.
>
> –
> - Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>

9

There are lots of devices that on startup perform “one time” initialization
stuff that can hog cpus etc. network and storage devices are notorious for
this stuff. A system hosting devices with real time ‘no data loss’
requirements ought to 1) wait for the system to finish configuration
(whatever that means) before starting data acquisition, 2) avoid deployment
on platforms that support hot-plug network and storage adapters.

=====================
Mark Roddy

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn
Sent: Tuesday, March 29, 2005 2:54 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] DPC latency

Then there is the Intel Ethernet driver, that on startup sits in a DPC for
over 1 second!

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

“Tim Roberts” wrote in message news:xxxxx@ntdev…
> xxxxx@attotech.com wrote:
>
>> I’ve recently run across a system (dual Itanium) where the average DPC
>> latency was very low (<1 ms), but occassionally (once a second or so)
>> would be 15-20 ms. I assumed some other device on the system was taking a

>> long time in an ISR or DPC and causing the delay.
>>
>
> The RealTek audio driver on the Intel 915 motherboard I have as a test
> machine had a nasty habit of running KeStallExecutionProcessor with a
> parameter of 10,000. Those of you keeping score at home will recognize
> that this does a tight kernel-mode loop for 10 milliseconds.
>
> I couldn’t figure out why one of the CPUs on my Hyperthreaded P4 2.8GHz
> was always 35% loaded. That was why.
>
> –
> - Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

10

> also see interrupt/DPC latency that is pretty horrendous because the IDE

driver has to do the read-out from the hard-disk (or write-on to) in PIO
mode, which will, if I remember correctly, be done in either the interrupt
handler or a DPC.

In the ISR.

BTW - DPCs are not fired exactly on IRQL drop to < DISPATCH - this is true only
on an idle CPU. Otherwise, they are sometimes delayed till the next timer tick,
and the kernel parameter IdealDpcRate in the registry governs this. Try playing
with it.

So, DPC latency can really be a timer tick even with a well-written driver.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

11

Another option is to have small FIFOs on your card and leverage system
memory for your buffers. The key is to design the DMA controller so it can
pick up the next buffer’s addresses without CPU intervention. The DMA
controller still interrupts the CPU at the end of each buffer. But since
the DMA is already running on the next buffer, the CPU has plenty of time
(the duration of the buffer) to provide the next next buffer’s addresses.

-Dan

----- Original Message -----

Subject: Re: DPC latency
From: Mats PETERSSON
> Date: Tue, 29 Mar 2005 18:44:16 +0100
>
> We just had a machine show a 17ms interrupt latency (it also showed some
> problems, which is why we started looking at it in the first place). Don’t
> ask me how or why… It seemed to cure itself at some later point…
>
> It’s reasonable to assume that ISR’s aren’t going to take much more than a
> few hundred milliseconds on 99% of all machines and configurations.
> However, some drivers do REALLY stupid things in their interrupt handler,
> and may well cause all sorts of bad latency. Microsoft are on the warpath
> on this, as more than a few hundred milliseconds of interrupt latency will
> disrupt real-time audio and real-time video, unless you put up big
> buffers,
> which isn’t a particularly good solution.
>
> So what am I saying: Well, it depends on whether you want belt, braces and
> a couple of safety-pins in there, or just something that will work in MOST
> cases. What happens if you overflow one of your buffers? Does a nuclear
> disaster happen, airplane falls out of the sky, some person calling on a
> mobile gets cut off, or just a little bit of missing data from a
> weather-station?
>
> Is your hardware going to be installed on all sorts of different machines,
> or can you give “selected models” that you recommend and have thoroughly
> tested for compatibility, and refer the customer to this list if they are
> having problems?
>
> 400KB * 8 * 0.05 => 160KB. So that’s a big ram if it’s built into the
> device, but rather a small one if it’s an external RAM-device (DRAM or
> SRAM). Maybe a solution is to use a 256KB or 512KB external SRAM? Should
> give you plenty of space.
>
> –
> Mats

12

We are thinking to provide array of buffers on the card that the card will
move from one buffer to next and throwing interrupt after each array filled.
With your design idea will 8 channels be able to get 400Kbytes/sec
simultaneously, I mean is there any chance of starvation by any channel?
Anyway, I like your design idea, can you please send me some literature or
tech. notes on this matter off the list?

Thanks,
Hakim

“Daniel E. Germann” wrote in message
news:xxxxx@ntdev…
> Another option is to have small FIFOs on your card and leverage system
> memory for your buffers. The key is to design the DMA controller so it
> can pick up the next buffer’s addresses without CPU intervention. The DMA
> controller still interrupts the CPU at the end of each buffer. But since
> the DMA is already running on the next buffer, the CPU has plenty of time
> (the duration of the buffer) to provide the next next buffer’s addresses.
>
> -Dan
>
> ----- Original Message -----
>> Subject: Re: DPC latency
>> From: Mats PETERSSON
>> Date: Tue, 29 Mar 2005 18:44:16 +0100
>>
>> We just had a machine show a 17ms interrupt latency (it also showed some
>> problems, which is why we started looking at it in the first place).
>> Don’t
>> ask me how or why… It seemed to cure itself at some later point…
>>
>> It’s reasonable to assume that ISR’s aren’t going to take much more than
>> a
>> few hundred milliseconds on 99% of all machines and configurations.
>> However, some drivers do REALLY stupid things in their interrupt handler,
>> and may well cause all sorts of bad latency. Microsoft are on the warpath
>> on this, as more than a few hundred milliseconds of interrupt latency
>> will
>> disrupt real-time audio and real-time video, unless you put up big
>> buffers,
>> which isn’t a particularly good solution.
>>
>> So what am I saying: Well, it depends on whether you want belt, braces
>> and
>> a couple of safety-pins in there, or just something that will work in
>> MOST
>> cases. What happens if you overflow one of your buffers? Does a nuclear
>> disaster happen, airplane falls out of the sky, some person calling on a
>> mobile gets cut off, or just a little bit of missing data from a
>> weather-station?
>>
>> Is your hardware going to be installed on all sorts of different
>> machines,
>> or can you give “selected models” that you recommend and have thoroughly
>> tested for compatibility, and refer the customer to this list if they are
>> having problems?
>>
>> 400KB * 8 * 0.05 => 160KB. So that’s a big ram if it’s built into the
>> device, but rather a small one if it’s an external RAM-device (DRAM or
>> SRAM). Maybe a solution is to use a 256KB or 512KB external SRAM? Should
>> give you plenty of space.
>>
>> –
>> Mats
>
>

13

Hey Max, this is an excellenet piece of info I ever had about DPC latency.

Hakim

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
> BTW - DPCs are not fired exactly on IRQL drop to < DISPATCH - this is true
> only
> on an idle CPU. Otherwise, they are sometimes delayed till the next timer
> tick,
> and the kernel parameter IdealDpcRate in the registry governs this. Try
> playing
> with it.
>
> So, DPC latency can really be a timer tick even with a well-written
> driver.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>

14

Maxim S. Shatskih wrote:

BTW - DPCs are not fired exactly on IRQL drop to < DISPATCH - this is true only
on an idle CPU. Otherwise, they are sometimes delayed till the next timer tick,
and the kernel parameter IdealDpcRate in the registry governs this. Try playing
with it.

It’s even more complicated than that, right? Consider the issue of
whether or not the DPC is to a local or remote processor, the DPC
Importance, etc, etc. It’d take a long article to describe how DPCs
really work.

Hmmmm… That’s a good idea for an article in The NT Insider. Assuming
we can get somebody from Microsoft to agree the information isn’t
confidential :slight_smile:

Now, having said all that, it’s been my experience that all the esoteric
DPC tuning parameters are FAR overshadowed by the behavior of the
drivers in the system. One misbehaving driver can screw-up ANY
assumptions that you want to make.

Remember, the problem you’re trying to quantify here isn’t average
ISR-DPC latency, it’s the WORST CASE ISR-DPC latency. As Don Burn said,
there are drivers that spend 1000ms in thier DPCs. I’ve personally
written a driver that spent 300ms in its DPCs (don’t try this at home,
but this was a special case… the device in question was being
benchmarked by a publication – The actual SYSTEM it ran on didn’t have
to be useable).

I’d suggest that unless you put some specific bounds on the problem,
there’s no reasonable answer that’ll be correct. Mark Roddy’s
suggestions are a good starting point for specifying these bounds. If
you can make some reasonable assumptions, allowing some number of tens
of milliseconds will typically make your device happy.

Of course, you also must consider the consequence of a failure, right?
If you’ve gotta be right or the nuclear reactor melts down, you’re
probably going to be more conservative in your specifications.

Peter
OSR

15

Thanks Peter.

I’d suggest that unless you put some specific bounds on the problem,
there’s no reasonable answer that’ll be correct. Mark Roddy’s suggestions
are a good starting point for specifying these bounds. If you can make
some reasonable assumptions, allowing some number of tens of milliseconds
will typically make your device happy.

I have got a very good understanding about the specification of the driver
that I’ll put forward.

Hmmmm… That’s a good idea for an article in The NT Insider. Assuming we
can get somebody from Microsoft to agree the information isn’t
confidential :slight_smile:

We really need to understand DPC latency in depth since hardware engineers
usually don’t count for the worst case scenario, also we are getting lot of
hardware design request recently that used be for real time OS. I hope
Microsoft will let you publish the article in The NT Insider.

Hakim

“Peter Viscarola (OSR)” wrote in message
news:xxxxx@ntdev…
> Maxim S. Shatskih wrote:
>>
>> BTW - DPCs are not fired exactly on IRQL drop to < DISPATCH - this is
>> true only
>> on an idle CPU. Otherwise, they are sometimes delayed till the next timer
>> tick,
>> and the kernel parameter IdealDpcRate in the registry governs this. Try
>> playing
>> with it.
>>
>
> It’s even more complicated than that, right? Consider the issue of
> whether or not the DPC is to a local or remote processor, the DPC
> Importance, etc, etc. It’d take a long article to describe how DPCs
> really work.
>
> Hmmmm… That’s a good idea for an article in The NT Insider. Assuming we
> can get somebody from Microsoft to agree the information isn’t
> confidential :slight_smile:
>
> Now, having said all that, it’s been my experience that all the esoteric
> DPC tuning parameters are FAR overshadowed by the behavior of the drivers
> in the system. One misbehaving driver can screw-up ANY assumptions that
> you want to make.
>
> Remember, the problem you’re trying to quantify here isn’t average ISR-DPC
> latency, it’s the WORST CASE ISR-DPC latency. As Don Burn said, there are
> drivers that spend 1000ms in thier DPCs. I’ve personally written a driver
> that spent 300ms in its DPCs (don’t try this at home, but this was a
> special case… the device in question was being benchmarked by a
> publication – The actual SYSTEM it ran on didn’t have to be useable).
>
> I’d suggest that unless you put some specific bounds on the problem,
> there’s no reasonable answer that’ll be correct. Mark Roddy’s suggestions
> are a good starting point for specifying these bounds. If you can make
> some reasonable assumptions, allowing some number of tens of milliseconds
> will typically make your device happy.
>
> Of course, you also must consider the consequence of a failure, right? If
> you’ve gotta be right or the nuclear reactor melts down, you’re probably
> going to be more conservative in your specifications.
>
> Peter
> OSR
>

16

> We really need to understand DPC latency in depth since hardware
engineers

usually don’t count for the worst case scenario, also we are getting lot
of
hardware design request recently that used be for real time OS. I hope
Microsoft will let you publish the article in The NT Insider.

I think you’ll have to REALLY consider what you mean by “DPC latency” in
this case. The WORST case with a selected set of drivers can be empirically
tested, but if anyone can install any driver after your hardware has been
installed, you’re looking at an “infinite” worst case (ok, so if someone
starts doing 2000 millisecond delay-loops in DPC, some users may start to
think about getting another driver or change the computer or something, but
it’s not impossible for someone to accomplish this inside a DPC, and there
certainly aint anything inside windows that prevents it from happening).
Naturally, your system can not be expected to work with “infinite” DPC
latency, so you must somewhat limit the scope at which your hardware can be
expected to work with “any system”.

I think there’s several ways to solve this, but ultimately, you will have
to rely on other drivers being “reasonable” in the use of DPC’s. Otherwise,
you will have to have something like 400KB * 8 * 20 seconds => 64000 KB of
memory on your device (and maybe that won’t be enough). This obviously
starts becomming rather silly (unless you’re building a graphics card, in
which case I’d say that 64MB is the very minimum you should consider :wink:

The approach of having large host memory buffers (i.e using system ram) is
a good one. It shouldn’t be too difficult to design a chip that uses PCI
(or PCI Express) to access the host memory and store/load data from that
path. I’m pretty sure there are plenty of documentation available on how to
do that, although I only have access to our internal documentation, which
would get me into serious trouble if I hand out. By using host memory
rather than adding memory to the board itself gives you freedom to expand
the memory size at will, without any big effort at all. Just change some
#define in the driver code, and you suddenly have 2, 4, 8, 13 or 42 times
more memory available… Even better, use multiple buffers, so you can fill
one buffer while another one is being used. If each channel has 3 or more
buffers, you can have one that is currently used, one “in queue” and one
being filled/emptied. This isn’t TRIVIAL, but it’s not rocket science
either. All graphics cards, ethernet, USB, IEEE-1394, (modern)IDE and SCSI
controlers already have some form of implementation of this method, with
slight variations depending on exactly what is required for that version of
card.

Doing 3200KB/s should be absolutely no problem. A PCI bus handles something
like 20-30 MB/s, without taking too much of a drastic measuer…

–
Mats

Hakim

“Peter Viscarola (OSR)” wrote in message
> news:xxxxx@ntdev…
> > Maxim S. Shatskih wrote:
> >>
> >> BTW - DPCs are not fired exactly on IRQL drop to < DISPATCH - this is
> >> true only
> >> on an idle CPU. Otherwise, they are sometimes delayed till the next
timer
> >> tick,
> >> and the kernel parameter IdealDpcRate in the registry governs this.
Try
> >> playing
> >> with it.
> >>
> >
> > It’s even more complicated than that, right? Consider the issue of
> > whether or not the DPC is to a local or remote processor, the DPC
> > Importance, etc, etc. It’d take a long article to describe how DPCs
> > really work.
> >
> > Hmmmm… That’s a good idea for an article in The NT Insider. Assuming
we
> > can get somebody from Microsoft to agree the information isn’t
> > confidential :slight_smile:
> >
> > Now, having said all that, it’s been my experience that all the
esoteric
> > DPC tuning parameters are FAR overshadowed by the behavior of the
drivers
> > in the system. One misbehaving driver can screw-up ANY assumptions
that
> > you want to make.
> >
> > Remember, the problem you’re trying to quantify here isn’t average
ISR-DPC
> > latency, it’s the WORST CASE ISR-DPC latency. As Don Burn said, there
are
> > drivers that spend 1000ms in thier DPCs. I’ve personally written a
driver
> > that spent 300ms in its DPCs (don’t try this at home, but this was a
> > special case… the device in question was being benchmarked by a
> > publication – The actual SYSTEM it ran on didn’t have to be useable).
> >
> > I’d suggest that unless you put some specific bounds on the problem,
> > there’s no reasonable answer that’ll be correct. Mark Roddy’s
suggestions
> > are a good starting point for specifying these bounds. If you can make
> > some reasonable assumptions, allowing some number of tens of
milliseconds
> > will typically make your device happy.
> >
> > Of course, you also must consider the consequence of a failure, right?
If
> > you’ve gotta be right or the nuclear reactor melts down, you’re
probably
> > going to be more conservative in your specifications.
> >
> > Peter
> > OSR
> >
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at http://www.
> osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@3dlabs.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

> ForwardSourceID:NT0000FC9E

17

Mats, thanks for your suggestion.
Harwdare engineers use to think DPC in the range of microseconds but it can
easily cross one clock interrupt which is usually 10ms. I’m trying to find
out OS mechanism that pushes the latecy so high at times as said by Max.
I’ve already passed the driver limitation info to hardware dept. that can be
imposed by other drivers.

Hakim

“Mats PETERSSON” wrote in message
news:xxxxx@ntdev…
>
>
>
>
>
>> We really need to understand DPC latency in depth since hardware
> engineers
>> usually don’t count for the worst case scenario, also we are getting lot
> of
>> hardware design request recently that used be for real time OS. I hope
>> Microsoft will let you publish the article in The NT Insider.
>
> I think you’ll have to REALLY consider what you mean by “DPC latency” in
> this case. The WORST case with a selected set of drivers can be
> empirically
> tested, but if anyone can install any driver after your hardware has been
> installed, you’re looking at an “infinite” worst case (ok, so if someone
> starts doing 2000 millisecond delay-loops in DPC, some users may start to
> think about getting another driver or change the computer or something,
> but
> it’s not impossible for someone to accomplish this inside a DPC, and there
> certainly aint anything inside windows that prevents it from happening).
> Naturally, your system can not be expected to work with “infinite” DPC
> latency, so you must somewhat limit the scope at which your hardware can
> be
> expected to work with “any system”.
>
> I think there’s several ways to solve this, but ultimately, you will have
> to rely on other drivers being “reasonable” in the use of DPC’s.
> Otherwise,
> you will have to have something like 400KB * 8 * 20 seconds => 64000 KB of
> memory on your device (and maybe that won’t be enough). This obviously
> starts becomming rather silly (unless you’re building a graphics card, in
> which case I’d say that 64MB is the very minimum you should consider :wink:
>
> The approach of having large host memory buffers (i.e using system ram) is
> a good one. It shouldn’t be too difficult to design a chip that uses PCI
> (or PCI Express) to access the host memory and store/load data from that
> path. I’m pretty sure there are plenty of documentation available on how
> to
> do that, although I only have access to our internal documentation, which
> would get me into serious trouble if I hand out. By using host memory
> rather than adding memory to the board itself gives you freedom to expand
> the memory size at will, without any big effort at all. Just change some
> #define in the driver code, and you suddenly have 2, 4, 8, 13 or 42 times
> more memory available… Even better, use multiple buffers, so you can
> fill
> one buffer while another one is being used. If each channel has 3 or more
> buffers, you can have one that is currently used, one “in queue” and one
> being filled/emptied. This isn’t TRIVIAL, but it’s not rocket science
> either. All graphics cards, ethernet, USB, IEEE-1394, (modern)IDE and SCSI
> controlers already have some form of implementation of this method, with
> slight variations depending on exactly what is required for that version
> of
> card.
>
> Doing 3200KB/s should be absolutely no problem. A PCI bus handles
> something
> like 20-30 MB/s, without taking too much of a drastic measuer…
>
> –
> Mats
>>
>> Hakim
>>
>>
>> “Peter Viscarola (OSR)” wrote in message
>> news:xxxxx@ntdev…
>> > Maxim S. Shatskih wrote:
>> >>
>> >> BTW - DPCs are not fired exactly on IRQL drop to < DISPATCH - this is
>> >> true only
>> >> on an idle CPU. Otherwise, they are sometimes delayed till the next
> timer
>> >> tick,
>> >> and the kernel parameter IdealDpcRate in the registry governs this.
> Try
>> >> playing
>> >> with it.
>> >>
>> >
>> > It’s even more complicated than that, right? Consider the issue of
>> > whether or not the DPC is to a local or remote processor, the DPC
>> > Importance, etc, etc. It’d take a long article to describe how DPCs
>> > really work.
>> >
>> > Hmmmm… That’s a good idea for an article in The NT Insider. Assuming
> we
>> > can get somebody from Microsoft to agree the information isn’t
>> > confidential :slight_smile:
>> >
>> > Now, having said all that, it’s been my experience that all the
> esoteric
>> > DPC tuning parameters are FAR overshadowed by the behavior of the
> drivers
>> > in the system. One misbehaving driver can screw-up ANY assumptions
> that
>> > you want to make.
>> >
>> > Remember, the problem you’re trying to quantify here isn’t average
> ISR-DPC
>> > latency, it’s the WORST CASE ISR-DPC latency. As Don Burn said, there
> are
>> > drivers that spend 1000ms in thier DPCs. I’ve personally written a
> driver
>> > that spent 300ms in its DPCs (don’t try this at home, but this was a
>> > special case… the device in question was being benchmarked by a
>> > publication – The actual SYSTEM it ran on didn’t have to be useable).
>> >
>> > I’d suggest that unless you put some specific bounds on the problem,
>> > there’s no reasonable answer that’ll be correct. Mark Roddy’s
> suggestions
>> > are a good starting point for specifying these bounds. If you can make
>> > some reasonable assumptions, allowing some number of tens of
> milliseconds
>> > will typically make your device happy.
>> >
>> > Of course, you also must consider the consequence of a failure, right?
> If
>> > you’ve gotta be right or the nuclear reactor melts down, you’re
> probably
>> > going to be more conservative in your specifications.
>> >
>> > Peter
>> > OSR
>> >
>>
>>
>>
>> —
>> Questions? First check the Kernel Driver FAQ at http://www.
>> osronline.com/article.cfm?id=256
>>
>> You are currently subscribed to ntdev as: xxxxx@3dlabs.com
>> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>> ForwardSourceID:NT0000FC9E
>
>