Windows Task Manager CPU usage

OSR_Community_User · May 22, 2007, 4:22pm

Dear all,

I am facing a fairly curious problem.

We are developing a USB driver for a data acquisition system. Our hardware delivers data at a selectable rate of 16, 32,…, 1024, 2048 Hz. The driver works absolutely fine, but its “CPU usage” drives me insane. In regular intervals the CPU usage slowly begins to rise until it reaches 100%. After some time it starts declining again and remains at zero. I spent several days looking for a hardware or driver failure but I think there is only one way to explain it:

The windows task manager outputs completely nonsense under certain conditions. I can substantiate this with the following observations:

Even though the task manager shows 100% CPU usage, the system doesn’t behave anything like it. I can happily use all kind of other applications without any performance loss.
If I select a data rate of 1024 Hz in our hardware, I get a completely symmetrical CPU “waveform”. The peaks (100%) and troughs (0%) have the same duration. This process just keeps on going and the CPU manager (I am actually using perfmon) gives me a nice looking square wave. The waveform has a period of about 4 minutes (observation).
To further validate this, one could theoretically show that the waveform period is around 4 minutes, any ideas?

I think the following happens:

The waveform symmetry occurs at 1024 Hz, which would suggest that the task manager polls at 1ms. This gives a slight mismatch of 24 Hz which causes the slow rising and falling of the CPU usage. The task manager doesn’t seem to take the average of CPU usage but some kind of snapshot. If our process kicked in just at the right time, the task manager would assume we used that much CPU usage all the time.

I am sorry about the long text. Its just pretty difficult to describe.

Do you have any similar experience?

Kind regards,

Johannes

OSR_Community_User · May 22, 2007, 4:36pm

I haven’t checked the Task Manager sources, but I think your guess about the behavior of Task Manager seems reasonable. Have you tried using Performance Monitor instead? It may sample at a different frequency, although I don’t see any way on the current (Vista) version to change the sample frequency. (Sigh.) You could also directly call the performance monitor APIs. .Net provides a very easy API for accessing them; if you care, look up the PerformanceCounter class. I’m not saying it’s the perfect or best way to do it, but if you want to spend 5 minutes getting it working, rather than the 5 hours it takes with the Win32 access to perf counters, it’s the way to go.

You could also use kernrate.

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Spinneken, Johannes
Sent: Tuesday, May 22, 2007 4:22 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Windows Task Manager CPU usage

Dear all,

I am facing a fairly curious problem.

We are developing a USB driver for a data acquisition system. Our hardware delivers data at a selectable rate of 16, 32,…, 1024, 2048 Hz. The driver works absolutely fine, but its “CPU usage” drives me insane. In regular intervals the CPU usage slowly begins to rise until it reaches 100%. After some time it starts declining again and remains at zero. I spent several days looking for a hardware or driver failure but I think there is only one way to explain it:

The windows task manager outputs completely nonsense under certain conditions. I can substantiate this with the following observations:

Even though the task manager shows 100% CPU usage, the system doesn’t behave anything like it. I can happily use all kind of other applications without any performance loss.
If I select a data rate of 1024 Hz in our hardware, I get a completely symmetrical CPU “waveform”. The peaks (100%) and troughs (0%) have the same duration. This process just keeps on going and the CPU manager (I am actually using perfmon) gives me a nice looking square wave. The waveform has a period of about 4 minutes (observation).
To further validate this, one could theoretically show that the waveform period is around 4 minutes, any ideas?

I think the following happens:

The waveform symmetry occurs at 1024 Hz, which would suggest that the task manager polls at 1ms. This gives a slight mismatch of 24 Hz which causes the slow rising and falling of the CPU usage. The task manager doesn’t seem to take the average of CPU usage but some kind of snapshot. If our process kicked in just at the right time, the task manager would assume we used that much CPU usage all the time.

I am sorry about the long text. Its just pretty difficult to describe.

Do you have any similar experience?

Kind regards,

Johannes

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Maxim_S_Shatskih · May 22, 2007, 4:44pm

>- Even though the task manager shows 100% CPU usage, the system doesn’t

behave anything like it. I can happily use all kind of other applications
without any
performance loss.

This only means that the CPU consumption occurs at low priority, by a worker
thread, for instance.

Is this the kernel time or user time?

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

OSR_Community_User · May 22, 2007, 6:23pm

> the waveform period is around 4 minutes

a slight mismatch of 24 Hz
The explanation looks just fine, here’s something I don’t understand.

The least common denominator (LCD) of 1024 = 2^10 and 1000 = 2^3 * 5^3 is 1024 * 125, so the period should be about 125 secs, which is approx. 2 minutes, not 4.

I’m loosing a factor of 2 (250 secs would be it), but where?

-------------- Original message --------------
From: “Spinneken, Johannes”

Dear all,

I am facing a fairly curious problem.

We are developing a USB driver for a data acquisition system. Our hardware delivers data at a selectable rate of 16, 32,…, 1024, 2048 Hz. The driver works absolutely fine, but its “CPU usage” drives me insane. In regular intervals the CPU usage slowly begins to rise until it reaches 100%. After some time it starts declining again and remains at zero. I spent several days looking for a hardware or driver failure but I think there is only one way to explain it:

The windows task manager outputs completely nonsense under certain conditions. I can substantiate this with the following observations:

- Even though the task manager shows 100% CPU usage, the system doesn’t behave anything like it. I can happily use all kind of other applications without any performance loss.
- If I select a data rate of 1024 Hz in our hardware, I get a completely symmetrical CPU “waveform”. The peaks (100%) and troughs (0%) have the same duration. This process just keeps on going and the CPU manager (I am actually using perfmon) gives me a nice looking square wave. The waveform has a period of about 4 minutes (observation).
- To further validate this, one could theoretically show that the waveform period is around 4 minutes, any ideas?

I think the following happens:

The waveform symmetry occurs at 1024 Hz, which would suggest that the task manager polls at 1ms. This gives a slight mismatch of 24 Hz which causes the slow rising and falling of the CPU usage. The task manager doesn’t seem to take the average of CPU usage but some kind of snapshot. If our process kicked in just at the right time, the task manager would assume we used that much CPU usage all the time.

I am sorry about the long text. Its just pretty difficult to describe.

Do you have any similar experience?

Kind regards,

Johannes

—
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

OSR_Community_User · May 23, 2007, 12:08am

Your explanation is most likely correct. Task manager sampling
is synchronized with the local machine’s timer. If workload is also
synchronized with the same timer, you can get permanently
skewed results, such as constant 100% usage. If workload
is synchronized with some other timer (a remote system, etc)
and the two timers are slowly drifting apart, you get alternating
periods of low and high usage.

–
This posting is provided “AS IS” with no warranties, and confers no
rights.

“Spinneken, Johannes” wrote:

I think the following happens:

The waveform symmetry occurs at 1024 Hz, which would
suggest that the task manager polls at 1ms. This gives a slight
mismatch of 24 Hz which causes the slow rising and falling
of the CPU usage. The task manager doesn’t seem to take
the average of CPU usage but some kind of snapshot. If our
process kicked in just at the right time, the task manager
would assume we used that much CPU usage all the time.

OSR_Community_User · May 23, 2007, 4:52am

>Even though the task manager shows 100% CPU usage, the system doesn’t

behave anything like it. I can happily use all kind of other applications
without any
performance loss.

This only means that the CPU consumption occurs at low priority, by a worker
thread, for instance.

Is this the kernel time or user time?

…

There is no straight answer to your question, I am afraid.

If I am running at 1024 Hz it?s mainly (99%) kernel time. You would expect your system to go nuts but it actually sits there quite happily.

Running at lower rates (say 128 Hz) gives an even more obscure behaviour. It?s first the kernel ramping up its time. It reaches 100% and stays there for some time. It then ?hands over? to user time (user time grows, kernel time declines, sum stays at 100%). I would explain this in the following way:

Our kernel mode driver gets blocks of data at regular intervals. The user mode software performs some actions when it receives the data from the driver. If you imagine the clock out of sink model I suggested earlier, this would mean the snapshot is first taken when the kernel mode driver performs its talks. The clocks keep on drifting apart, so the snapshot will happen more and more in the in time slice where the user mode performs its actions.

Johannes

Daniel_Terhell · May 23, 2007, 5:33am

Windows Task Manager CPU usageTask manager also displays the amount of kernel and user time for a process, these are not momentarily samples but accumulated time spent. If your process would consume 100% all the time you would be able to tell if this is corrent from the CPU time spent.

/Daniel
“Spinneken, Johannes” wrote in message news:xxxxx@ntdev…
Dear all,

I am facing a fairly curious problem.

We are developing a USB driver for a data acquisition system. Our hardware delivers data at a selectable rate of 16, 32,…, 1024, 2048 Hz. The driver works absolutely fine, but its “CPU usage” drives me insane. In regular intervals the CPU usage slowly begins to rise until it reaches 100%. After some time it starts declining again and remains at zero. I spent several days looking for a hardware or driver failure but I think there is only one way to explain it:

The windows task manager outputs completely nonsense under certain conditions. I can substantiate this with the following observations:

- Even though the task manager shows 100% CPU usage, the system doesn’t behave anything like it. I can happily use all kind of other applications without any performance loss.
- If I select a data rate of 1024 Hz in our hardware, I get a completely symmetrical CPU “waveform”. The peaks (100%) and troughs (0%) have the same duration. This process just keeps on going and the CPU manager (I am actually using perfmon) gives me a nice looking square wave. The waveform has a period of about 4 minutes (observation).
- To further validate this, one could theoretically show that the waveform period is around 4 minutes, any ideas?

I think the following happens:

The waveform symmetry occurs at 1024 Hz, which would suggest that the task manager polls at 1ms. This gives a slight mismatch of 24 Hz which causes the slow rising and falling of the CPU usage. The task manager doesn’t seem to take the average of CPU usage but some kind of snapshot. If our process kicked in just at the right time, the task manager would assume we used that much CPU usage all the time.

I am sorry about the long text. Its just pretty difficult to describe.

Do you have any similar experience?

Kind regards,

Johannes

OSR_Community_User · May 23, 2007, 6:24am

This is exactly what I am questioning here. Is the time actually accumulated time (as one would expect) or is there a possibility for the task manager to get it wrong. Fact is, if your system consumes 100% kernel time for about 100 secs you would expect to notice this (other than looking at the task managers CPU window).

I was running some tests with kernrate now. They bare absolutely no relationship to the CPU time shown in the task manager. I’ll keep on testing and hope to get some final results that prove my assumption.

I watched the timing of the process again. At 1024 Hz (our system) the repeat period of the “CPU square wave” is exactly 3:23 min (203 secs). Odd number!

Johannes

Task manager also displays the amount of kernel and user time for a process, these are not momentarily samples but accumulated time spent. If your process would consume 100% all the time you would be able to tell if this is corrent from the CPU time spent.

/Daniel

“Spinneken, Johannes” wrote in message news:xxxxx@ntdev…

Dear all,

I am facing a fairly curious problem.

We are developing a USB driver for a data acquisition system. Our hardware delivers data at a selectable rate of 16, 32,…, 1024, 2048 Hz. The driver works absolutely fine, but its “CPU usage” drives me insane. In regular intervals the CPU usage slowly begins to rise until it reaches 100%. After some time it starts declining again and remains at zero. I spent several days looking for a hardware or driver failure but I think there is only one way to explain it:

The windows task manager outputs completely nonsense under certain conditions. I can substantiate this with the following observations:

- Even though the task manager shows 100% CPU usage, the system doesn’t behave anything like it. I can happily use all kind of other applications without any performance loss.
- If I select a data rate of 1024 Hz in our hardware, I get a completely symmetrical CPU “waveform”. The peaks (100%) and troughs (0%) have the same duration. This process just keeps on going and the CPU manager (I am actually using perfmon) gives me a nice looking square wave. The waveform has a period of about 4 minutes (observation).
- To further validate this, one could theoretically show that the waveform period is around 4 minutes, any ideas?

I think the following happens:

The waveform symmetry occurs at 1024 Hz, which would suggest that the task manager polls at 1ms. This gives a slight mismatch of 24 Hz which causes the slow rising and falling of the CPU usage. The task manager doesn’t seem to take the average of CPU usage but some kind of snapshot. If our process kicked in just at the right time, the task manager would assume we used that much CPU usage all the time.

I am sorry about the long text. Its just pretty difficult to describe.

Do you have any similar experience?

Kind regards,

Johannes

—
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

OSR_Community_User · May 23, 2007, 8:11am

CPU time reported by task manager is updated at timer interrupts
so it can be inaccurate for perodic/timer synchronized workloads.

QueryThreadCycleTime/QueryProcessCycleTime/QueryIdleProcessorCycleTime
APIs in Vista use a counter which is updated when threads are
scheduled. I think they should produce more accurate results
in your case.

–
This posting is provided “AS IS” with no warranties, and confers no
rights.

“Spinneken, Johannes” wrote:

This is exactly what I am questioning here. Is the time actually
accumulated time (as one would expect) or is there a possibility
for the task manager to get it wrong.

Tim_Roberts · May 23, 2007, 1:58pm

Pavel Lebedinsky wrote:

Your explanation is most likely correct. Task manager sampling
is synchronized with the local machine’s timer.

I dispute this. I don’t believe that Task Manager uses a sampling
technique at all. It’s using the time accumulators in the process
tables, which get updated on task switch. So:

for each process:
subtract current accumulated CPU time from last CPU time for this
process
divide by real time since last check
display as percentage
remember current CPU time as last CPU time

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Daniel_Terhell · May 23, 2007, 2:57pm

Re:[ntdev] Windows Task Manager CPU usageThe problem is not so complicated, if you use the API function GetProcessTimes (kernel version also available) for a process then the kernel time and the user time returned by this function exactly add up to the CPU time displayed by task manager. This is administered by the system probably at every task switch as Tim says and is not administered by task manager. Task manager can report you these values the first time you run it. Observing this value alone should tell you whether your process is really consuming 100% or only at the intervals at which task manager is polling.

/Daniel

“Spinneken, Johannes” wrote in message news:xxxxx@ntdev…
This is exactly what I am questioning here. Is the time actually accumulated time (as one would expect) or is there a possibility for the task manager to get it wrong. Fact is, if your system consumes 100% kernel time for about 100 secs you would expect to notice this (other than looking at the task managers CPU window).

I was running some tests with kernrate now. They bare absolutely no relationship to the CPU time shown in the task manager. I’ll keep on testing and hope to get some final results that prove my assumption.

I watched the timing of the process again. At 1024 Hz (our system) the repeat period of the “CPU square wave” is exactly 3:23 min (203 secs). Odd number!

Johannes

Task manager also displays the amount of kernel and user time for a process, these are not momentarily samples but accumulated time spent. If your process would consume 100% all the time you would be able to tell if this is corrent from the CPU time spent.

/Daniel

“Spinneken, Johannes” wrote in message news:xxxxx@ntdev…

Dear all,

I am facing a fairly curious problem.

We are developing a USB driver for a data acquisition system. Our hardware delivers data at a selectable rate of 16, 32,…, 1024, 2048 Hz. The driver works absolutely fine, but its “CPU usage” drives me insane. In regular intervals the CPU usage slowly begins to rise until it reaches 100%. After some time it starts declining again and remains at zero. I spent several days looking for a hardware or driver failure but I think there is only one way to explain it:

The windows task manager outputs completely nonsense under certain conditions. I can substantiate this with the following observations:

- Even though the task manager shows 100% CPU usage, the system doesn’t behave anything like it. I can happily use all kind of other applications without any performance loss.
- If I select a data rate of 1024 Hz in our hardware, I get a completely symmetrical CPU “waveform”. The peaks (100%) and troughs (0%) have the same duration. This process just keeps on going and the CPU manager (I am actually using perfmon) gives me a nice looking square wave. The waveform has a period of about 4 minutes (observation).
- To further validate this, one could theoretically show that the waveform period is around 4 minutes, any ideas?

I think the following happens:

The waveform symmetry occurs at 1024 Hz, which would suggest that the task manager polls at 1ms. This gives a slight mismatch of 24 Hz which causes the slow rising and falling of the CPU usage. The task manager doesn’t seem to take the average of CPU usage but some kind of snapshot. If our process kicked in just at the right time, the task manager would assume we used that much CPU usage all the time.

I am sorry about the long text. Its just pretty difficult to describe.

Do you have any similar experience?

Kind regards,

Johannes

—
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

OSR_Community_User · May 24, 2007, 1:34am

“Tim Roberts” wrote:

> Your explanation is most likely correct. Task manager sampling
> is synchronized with the local machine’s timer.

I dispute this. I don’t believe that Task Manager uses a sampling
technique at all. It’s using the time accumulators in the process
tables, which get updated on task switch.

GetThreadTimes/GetProcessTimes and all CPU usage counters
in task manager are based on internal counters maintained by the
kernel for each thread, process and CPU (things like KTHREAD::UserTime,
KPRCB::UserTime etc).

Thread and PRCB counters are updated not on task switch
but in response to timer interrupts (KeUpdateRunTime is the function
that does this). Process counters are aggregates of corresponding
thread counters.

So technically you’re right that sampling does not occur in task
manager, but it still occurs in the kernel. This is why GetThreadTimes
or task manager can show unexpected results for workloads
that are synchronized with the timer. Many people have noticed this:

http://blog.kalmbachnet.de/?postid=28

In addition to timer based counters, Vista kernel maintains rdtsc
based cycle counters that track actual thread execution time,
excluding interrupts and DPCs:

http://www.microsoft.com/technet/technetmag/issues/2007/02/VistaKernel/

These counters are exposed to user apps through APIs like
QueryThreadCycleTime, and can be used in cases where
timer based APIs doesn’t work well (taking into account
all the usual caveats about using rdtsc).

–
This posting is provided “AS IS” with no warranties, and confers no
rights.

OSR_Community_User · May 24, 2007, 5:03am

Using GetProcessTimes() was the first way I tried to tackle the problem. It doesn?t give you any different results for the reasons Pavel explained.

http://blog.kalmbachnet.de/?postid=28

The problem is not so complicated, if you use the API function GetProcessTimes (kernel >version also available) for a process then the kernel time and the user time returned by this >function exactly add up to the CPU time displayed by task manager. This is administered by >the system probably at every task switch as Tim says and is not administered by task >manager. Task manager can report you these values the first time you run it. Observing this >value alone should tell you whether your process is really consuming 100% or only at the >ntervals at which task manager is polling.

And yes, the problem is pretty complicated!

OSR_Community_User · May 24, 2007, 5:15am

Just to convince the last ones still in doubt.

The following table shows the result of running kernrate for about half an hour on our process (I configured our hardware timing to 128 Hz, the time between 0% and 100% CPU usage is much longer than for 1024 Hz). During the entire time the task manager displayed zero percent CPU usage.

PrivilegedHitPct CPU
ModuleName T
NTKernel 62.774%
HAL 18.722%
Usbport 6.115%
Wdf01000 4.751%
Usbencore 3.792%
Win32k 1.745%
Usbehci 1.093%
Sisgrv 0.465%
Usbhub 0.262%

What do we learn?

For externally synchronised workload, the calls to GetSystemTimes() can result in absolutely erroneous results!

Daniel_Terhell · May 24, 2007, 6:23am

“Pavel Lebedinsky” wrote in message news:xxxxx@ntdev…
> GetThreadTimes/GetProcessTimes and all CPU usage counters
> in task manager are based on internal counters maintained by the
> kernel for each thread, process and CPU (things like KTHREAD::UserTime,
> KPRCB::UserTime etc).
>
> Thread and PRCB counters are updated not on task switch
> but in response to timer interrupts (KeUpdateRunTime is the function
> that does this). Process counters are aggregates of corresponding
> thread counters.
>

I am not understanding much of this or the article. A thread switch can
occur because of a timer interrupt or because of another reason. Do you or
the article say that if a thread switch occurs for other reason than the
timer interrupt KeUpdateRunTime is not called or the counters are not
updated ? Or are these counters updated in a fashion completely indendent
from the dispatcher, this would be really crap. I cannot imagine much of
this, if I call a wait function then a thread swich occurs and the counters
are not further increased. It would be nice if some real expert can explain
this well.

I must admit that since NT4 if I would create a thread with an infinite loop
doing nothing or something silly, the process in taskman would display to
consume 100% while the system behaves as if the thread is idle and consumes
nothing. I was assuming some entity realized this thread was doing nothing
and some kind of optimization was taking place but where or how, no idea.
And even then why it still displays 100%, I think it’s a question many
people have asked but were not asked publicly asked because it is could be
something they ought to understand. I realize it is better never to become
an expert so you got nothing to lose and keep your right to ask silly
questions like these. In any case the discussion has moved from the
assumption that taskman is behaving in a silly way to thread counters not
being updated by the dispatcher on every task switch.

/Daniel

Mike_Kemp · May 24, 2007, 6:40am

To add a question for any expert consideration, (this is not a driver
question, but hopefully relevant).

I have an app where I need to wait many short periods (sub mS), so I cannot
use Sleep(n) with n>0 as this always results in 10 - 20mS loss of control,
and although it is not critical to my app on any particular occasion, I
can’t wait this long every time or it runs too slowly.

So to be cooperative I loop with Sleep(0) until the high precision timer has
elapsed enough uS - this means I can yield to the system if it wants, but
maintain a reasonable average response time in my app.

Although this works well, it results in my app using 100% of the CPU as
reported by Task Manager. It does not slow down the system in any way, but
users do not accept this.

If anyone knows any better tricks to do short pauses while yielding
cooperatively, without TM reporting 100% usage, I’d be very interested…

M
----- Original Message -----
From: Daniel Terhell
Newsgroups: ntdev
To: Windows System Software Devs Interest List
Sent: Thursday, May 24, 2007 11:22 AM
Subject: Re:[ntdev] Windows Task Manager CPU usage

“Pavel Lebedinsky” wrote in message news:xxxxx@ntdev…
> GetThreadTimes/GetProcessTimes and all CPU usage counters
> in task manager are based on internal counters maintained by the
> kernel for each thread, process and CPU (things like KTHREAD::UserTime,
> KPRCB::UserTime etc).
>
> Thread and PRCB counters are updated not on task switch
> but in response to timer interrupts (KeUpdateRunTime is the function
> that does this). Process counters are aggregates of corresponding
> thread counters.
>

I am not understanding much of this or the article. A thread switch can
occur because of a timer interrupt or because of another reason. Do you or
the article say that if a thread switch occurs for other reason than the
timer interrupt KeUpdateRunTime is not called or the counters are not
updated ? Or are these counters updated in a fashion completely indendent
from the dispatcher, this would be really crap. I cannot imagine much of
this, if I call a wait function then a thread swich occurs and the counters
are not further increased. It would be nice if some real expert can explain
this well.

I must admit that since NT4 if I would create a thread with an infinite loop
doing nothing or something silly, the process in taskman would display to
consume 100% while the system behaves as if the thread is idle and consumes
nothing. I was assuming some entity realized this thread was doing nothing
and some kind of optimization was taking place but where or how, no idea.
And even then why it still displays 100%, I think it’s a question many
people have asked but were not asked publicly asked because it is could be
something they ought to understand. I realize it is better never to become
an expert so you got nothing to lose and keep your right to ask silly
questions like these. In any case the discussion has moved from the
assumption that taskman is behaving in a silly way to thread counters not
being updated by the dispatcher on every task switch.

/Daniel

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

OSR_Community_User · May 24, 2007, 6:55am

If this is really important to you, try and do the following:
Create some driver that will have a Sleep function that will call
KeDelayExecutionThread.

This should allow you to sleep for shorter time.

Thanks
Tzachi

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Mike Kemp
Sent: Thursday, May 24, 2007 1:40 PM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] Windows Task Manager CPU usage

To add a question for any expert consideration, (this is not
a driver question, but hopefully relevant).

I have an app where I need to wait many short periods (sub
mS), so I cannot use Sleep(n) with n>0 as this always results
in 10 - 20mS loss of control, and although it is not critical
to my app on any particular occasion, I can’t wait this long
every time or it runs too slowly.

So to be cooperative I loop with Sleep(0) until the high
precision timer has elapsed enough uS - this means I can
yield to the system if it wants, but maintain a reasonable
average response time in my app.

Although this works well, it results in my app using 100% of
the CPU as reported by Task Manager. It does not slow down
the system in any way, but users do not accept this.

If anyone knows any better tricks to do short pauses while
yielding cooperatively, without TM reporting 100% usage, I’d
be very interested…

M
----- Original Message -----
From: Daniel Terhell
Newsgroups: ntdev
To: Windows System Software Devs Interest List
Sent: Thursday, May 24, 2007 11:22 AM
Subject: Re:[ntdev] Windows Task Manager CPU usage

“Pavel Lebedinsky” wrote in message
> news:xxxxx@ntdev…
> > GetThreadTimes/GetProcessTimes and all CPU usage counters
> > in task manager are based on internal counters maintained by the
> > kernel for each thread, process and CPU (things like
> KTHREAD::UserTime,
> > KPRCB::UserTime etc).
> >
> > Thread and PRCB counters are updated not on task switch
> > but in response to timer interrupts (KeUpdateRunTime is the function
> > that does this). Process counters are aggregates of corresponding
> > thread counters.
> >
>
> I am not understanding much of this or the article. A thread
> switch can
> occur because of a timer interrupt or because of another
> reason. Do you or
> the article say that if a thread switch occurs for other
> reason than the
> timer interrupt KeUpdateRunTime is not called or the counters are not
> updated ? Or are these counters updated in a fashion
> completely indendent
> from the dispatcher, this would be really crap. I cannot
> imagine much of
> this, if I call a wait function then a thread swich occurs
> and the counters
> are not further increased. It would be nice if some real
> expert can explain
> this well.
>
> I must admit that since NT4 if I would create a thread with
> an infinite loop
> doing nothing or something silly, the process in taskman
> would display to
> consume 100% while the system behaves as if the thread is
> idle and consumes
> nothing. I was assuming some entity realized this thread was
> doing nothing
> and some kind of optimization was taking place but where or
> how, no idea.
> And even then why it still displays 100%, I think it’s a question many
> people have asked but were not asked publicly asked because
> it is could be
> something they ought to understand. I realize it is better
> never to become
> an expert so you got nothing to lose and keep your right to ask silly
> questions like these. In any case the discussion has moved from the
> assumption that taskman is behaving in a silly way to thread
> counters not
> being updated by the dispatcher on every task switch.
>
> /Daniel
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online
> at http://www.osronline.com/page.cfm?name=ListServer
>

OSR_Community_User · May 24, 2007, 9:35am

> this always results in 10 - 20mS loss of control
… due to thread scheduling. Until the round-robin will come back to
you, your thread is on hold anyway. Essentially, Sleep(0) means “yield”, that’s
how these 10-20 ms come up.

Having realized that, I fought this problem once by upping my thread (process?)
priority. IIRC it was not much of a success, but I did not try really hard.

You may want to look in this priority-related direction, appending something else
(timer? event?) to the machinery, of course.
Kinda: get rid of scheduler influence by prioritizing, then apply your controls.

[I would re-read appropriate chapter of Mark’s bible before proceeding though.]

-------------- Original message --------------
From: “Mike Kemp”

> To add a question for any expert consideration, (this is not a driver
> question, but hopefully relevant).
>
> I have an app where I need to wait many short periods (sub mS), so I cannot
> use Sleep(n) with n>0 as this always results in 10 - 20mS loss of control,
> and although it is not critical to my app on any particular occasion, I
> can’t wait this long every time or it runs too slowly.
>
> So to be cooperative I loop with Sleep(0) until the high precision timer has
> elapsed enough uS - this means I can yield to the system if it wants, but
> maintain a reasonable average response time in my app.
>
> Although this works well, it results in my app using 100% of the CPU as
> reported by Task Manager. It does not slow down the system in any way, but
> users do not accept this.
>
> If anyone knows any better tricks to do short pauses while yielding
> cooperatively, without TM reporting 100% usage, I’d be very interested…
>
> M
> ----- Original Message -----
> From: Daniel Terhell
> Newsgroups: ntdev
> To: Windows System Software Devs Interest List
> Sent: Thursday, May 24, 2007 11:22 AM
> Subject: Re:[ntdev] Windows Task Manager CPU usage
>
>
> “Pavel Lebedinsky” wrote in message news:xxxxx@ntdev…
> > GetThreadTimes/GetProcessTimes and all CPU usage counters
> > in task manager are based on internal counters maintained by the
> > kernel for each thread, process and CPU (things like KTHREAD::UserTime,
> > KPRCB::UserTime etc).
> >
> > Thread and PRCB counters are updated not on task switch
> > but in response to timer interrupts (KeUpdateRunTime is the function
> > that does this). Process counters are aggregates of corresponding
> > thread counters.
> >
>
> I am not understanding much of this or the article. A thread switch can
> occur because of a timer interrupt or because of another reason. Do you or
> the article say that if a thread switch occurs for other reason than the
> timer interrupt KeUpdateRunTime is not called or the counters are not
> updated ? Or are these counters updated in a fashion completely indendent
> from the dispatcher, this would be really crap. I cannot imagine much of
> this, if I call a wait function then a thread swich occurs and the counters
> are not further increased. It would be nice if some real expert can explain
> this well.
>
> I must admit that since NT4 if I would create a thread with an infinite loop
> doing nothing or something silly, the process in taskman would display to
> consume 100% while the system behaves as if the thread is idle and consumes
> nothing. I was assuming some entity realized this thread was doing nothing
> and some kind of optimization was taking place but where or how, no idea.
> And even then why it still displays 100%, I think it’s a question many
> people have asked but were not asked publicly asked because it is could be
> something they ought to understand. I realize it is better never to become
> an expert so you got nothing to lose and keep your right to ask silly
> questions like these. In any case the discussion has moved from the
> assumption that taskman is behaving in a silly way to thread counters not
> being updated by the dispatcher on every task switch.
>
> /Daniel
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

Gianluca_Varenni · May 24, 2007, 11:36am

Uhm, I might be wrong, but I think that will not help much. I see the
KeDelayExecutionThread as the kernel counterpart of Sleep(). So again you’ll
end up with the 10-15 ms minimum delay (the scheduling quantum).

If you want to sleep with a shorter time, I would instead play with
ExSetTimerResolution. USE IT CAREFULLY. FWIW, this function is used by the
Multimedia timers (user level) to obtain 1ms timers.

Hope it helps
GV

----- Original Message -----
From: “Tzachi Dar”
To: “Windows System Software Devs Interest List”
Sent: Thursday, May 24, 2007 3:51 AM
Subject: RE: Re:[ntdev] Windows Task Manager CPU usage

If this is really important to you, try and do the following:
Create some driver that will have a Sleep function that will call
KeDelayExecutionThread.

This should allow you to sleep for shorter time.

Thanks
Tzachi

> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Mike Kemp
> Sent: Thursday, May 24, 2007 1:40 PM
> To: Windows System Software Devs Interest List
> Subject: Re: Re:[ntdev] Windows Task Manager CPU usage
>
> To add a question for any expert consideration, (this is not
> a driver question, but hopefully relevant).
>
> I have an app where I need to wait many short periods (sub
> mS), so I cannot use Sleep(n) with n>0 as this always results
> in 10 - 20mS loss of control, and although it is not critical
> to my app on any particular occasion, I can’t wait this long
> every time or it runs too slowly.
>
> So to be cooperative I loop with Sleep(0) until the high
> precision timer has elapsed enough uS - this means I can
> yield to the system if it wants, but maintain a reasonable
> average response time in my app.
>
> Although this works well, it results in my app using 100% of
> the CPU as reported by Task Manager. It does not slow down
> the system in any way, but users do not accept this.
>
> If anyone knows any better tricks to do short pauses while
> yielding cooperatively, without TM reporting 100% usage, I’d
> be very interested…
>
> M
> ----- Original Message -----
> From: Daniel Terhell
> Newsgroups: ntdev
> To: Windows System Software Devs Interest List
> Sent: Thursday, May 24, 2007 11:22 AM
> Subject: Re:[ntdev] Windows Task Manager CPU usage
>
>
> “Pavel Lebedinsky” wrote in message
> news:xxxxx@ntdev…
> > GetThreadTimes/GetProcessTimes and all CPU usage counters
> > in task manager are based on internal counters maintained by the
> > kernel for each thread, process and CPU (things like
> KTHREAD::UserTime,
> > KPRCB::UserTime etc).
> >
> > Thread and PRCB counters are updated not on task switch
> > but in response to timer interrupts (KeUpdateRunTime is the function
> > that does this). Process counters are aggregates of corresponding
> > thread counters.
> >
>
> I am not understanding much of this or the article. A thread
> switch can
> occur because of a timer interrupt or because of another
> reason. Do you or
> the article say that if a thread switch occurs for other
> reason than the
> timer interrupt KeUpdateRunTime is not called or the counters are not
> updated ? Or are these counters updated in a fashion
> completely indendent
> from the dispatcher, this would be really crap. I cannot
> imagine much of
> this, if I call a wait function then a thread swich occurs
> and the counters
> are not further increased. It would be nice if some real
> expert can explain
> this well.
>
> I must admit that since NT4 if I would create a thread with
> an infinite loop
> doing nothing or something silly, the process in taskman
> would display to
> consume 100% while the system behaves as if the thread is
> idle and consumes
> nothing. I was assuming some entity realized this thread was
> doing nothing
> and some kind of optimization was taking place but where or
> how, no idea.
> And even then why it still displays 100%, I think it’s a question many
> people have asked but were not asked publicly asked because
> it is could be
> something they ought to understand. I realize it is better
> never to become
> an expert so you got nothing to lose and keep your right to ask silly
> questions like these. In any case the discussion has moved from the
> assumption that taskman is behaving in a silly way to thread
> counters not
> being updated by the dispatcher on every task switch.
>
> /Daniel
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online
> at http://www.osronline.com/page.cfm?name=ListServer
>

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Gianluca_Varenni · May 24, 2007, 11:41am

----- Original Message -----
From: “Mike Kemp”
To: “Windows System Software Devs Interest List”
Sent: Thursday, May 24, 2007 3:39 AM
Subject: Re: Re:[ntdev] Windows Task Manager CPU usage

> To add a question for any expert consideration, (this is not a driver
> question, but hopefully relevant).
>
> I have an app where I need to wait many short periods (sub mS), so I
> cannot use Sleep(n) with n>0 as this always results in 10 - 20mS loss of
> control, and although it is not critical to my app on any particular
> occasion, I can’t wait this long every time or it runs too slowly.
>
> So to be cooperative I loop with Sleep(0) until the high precision timer
> has elapsed enough uS - this means I can yield to the system if it wants,
> but maintain a reasonable average response time in my app.
>
> Although this works well, it results in my app using 100% of the CPU as
> reported by Task Manager. It does not slow down the system in any way, but
> users do not accept this.
>
> If anyone knows any better tricks to do short pauses while yielding
> cooperatively, without TM reporting 100% usage, I’d be very interested…

As I said in my other mail, if a sleep of about 1ms is enough for you, I
would go with ExsetTimerResolution (or using multimedia timers). If you need
to go lower than that, and polling is not an option (that’s what you are
basically doing when the CPU bumps to 100%), then you are in trouble.
Windows is not a real-time OS, and the best you can obtain is 1ms. In one of
the projects I worked on, we obtained shorter sleeps (well, something
similar) by means of the physical card we were working on: the card was
generating interrupts at a sub-millisecond rate and we used them as our own
“clock ticks”.

Hope it helps
GV

>
> M
> ----- Original Message -----
> From: Daniel Terhell
> Newsgroups: ntdev
> To: Windows System Software Devs Interest List
> Sent: Thursday, May 24, 2007 11:22 AM
> Subject: Re:[ntdev] Windows Task Manager CPU usage
>
>
> “Pavel Lebedinsky” wrote in message
> news:xxxxx@ntdev…
>> GetThreadTimes/GetProcessTimes and all CPU usage counters
>> in task manager are based on internal counters maintained by the
>> kernel for each thread, process and CPU (things like KTHREAD::UserTime,
>> KPRCB::UserTime etc).
>>
>> Thread and PRCB counters are updated not on task switch
>> but in response to timer interrupts (KeUpdateRunTime is the function
>> that does this). Process counters are aggregates of corresponding
>> thread counters.
>>
>
> I am not understanding much of this or the article. A thread switch can
> occur because of a timer interrupt or because of another reason. Do you
> or
> the article say that if a thread switch occurs for other reason than the
> timer interrupt KeUpdateRunTime is not called or the counters are not
> updated ? Or are these counters updated in a fashion completely indendent
> from the dispatcher, this would be really crap. I cannot imagine much of
> this, if I call a wait function then a thread swich occurs and the
> counters
> are not further increased. It would be nice if some real expert can
> explain
> this well.
>
> I must admit that since NT4 if I would create a thread with an infinite
> loop
> doing nothing or something silly, the process in taskman would display to
> consume 100% while the system behaves as if the thread is idle and
> consumes
> nothing. I was assuming some entity realized this thread was doing nothing
> and some kind of optimization was taking place but where or how, no idea.
> And even then why it still displays 100%, I think it’s a question many
> people have asked but were not asked publicly asked because it is could be
> something they ought to understand. I realize it is better never to become
> an expert so you got nothing to lose and keep your right to ask silly
> questions like these. In any case the discussion has moved from the
> assumption that taskman is behaving in a silly way to thread counters not
> being updated by the dispatcher on every task switch.
>
> /Daniel
>
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer