Hi all,
can you suggest different ways of performing a long term disk-accessing
task (like defragmentation, AV scanning, disk encryption) ?
Currently in my driver the latter is done in a dedicated system thread. In
some cases it seems not to be low-prio enough, as people
say the “User-experience” gets disturbed.
I’d rather play around with lower priorities, but won’t miss the
opportunity to ask “Is there anything else I can do ?”.
It is to run under W2K,XP,Vista.
Would using CPU-affinity (in mp-systems) help ?
Would use of IoPriorityHint (in Vista) help ?
Can I somehow regulate (KeDelayExecutionThread / boosting priority) fitting
to users busy/idle ?
Regards
Else
On Jan 16, 2008 10:27 AM, Else Kluger wrote:
> can you suggest different ways of performing a long term disk-accessing
> task (like defragmentation, AV scanning, disk encryption) ?
A pet peeve of mine, as a user, is various unknown disk I/O taking
place without my consent. After installing Vista, I had to disable
Defender, scheduled defrag and other suspicious services that I have
no use for. And I still get intermittent high volume disk accesses
from time to time (System process PID 4 writing to sys vol information
IIRC-- perhaps system restore?) that bugs me like a horse on potato
salad.
As I see it, you have already said your thread runs at a low priority,
including low IO priority (in Vista).
The next step might be to sleep() a bit between accessing the drive,
but… In that case your disk I/O will go on for a longer duration of
time, which translates into less battery for us road warriors!
(perhaps you should consider halting IO while the system is running
off battery?)
Another option is allowing the user to pause the operation and have
some UI somewhere that he can easily get at. Some AV vendors do this.
(I dislike AV products btw – never used them myself)
(sorry, not much help from me, I just wanted to flag my pet peeve 
)
–
Rune
It all depends on what you want to do.
We typically do exactly what Rune mentioned: Sleep a bit between drive accesses, and use a lower priority work thread. This is very effective in decreasing user impact.
If your goal is to only activate something when the user isn’t present, you could have an application detect system idle (such as detecting when the screensaver runs) and then have that app signal your driver code.
Peter
OSR
> can you suggest different ways of performing a long term disk-accessing
task (like defragmentation, AV scanning, disk encryption) ?
Currently in my driver the latter is done in a dedicated system thread. In
some cases it seems not to be low-prio enough, as people
say the “User-experience” gets disturbed.
Heavy IO must run in high priority thread, otherwise, it will starve a lot
before sending the next disk IRP.
To reduce the load in such a case, you need to add disk throttling code like
calling Sleep/KeDelayExecutionThread before each new disk operation for a
calibrated time. The thread priority must still be high.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
> Heavy IO must run in high priority thread,
otherwise, it will starve a lot before sending the next disk IRP.
Nonsense. All you get by raising the read priority is more deterministic behavior. Lowering the read priority simply makes the wait the MINUMUM wait time between I/O requests.
There’s particular no way it “must” be coded in this case and “starving” doesn’t apply. When you use a lower priority work thread with the wait, the thread is blocked by user compute-bound activity. It’s precisely this user activity you want to facilitate… not pre-empt.
When you use a high priority thread for your I/O, you can still see user impact. Tuning the wait-between-I/Os value becomes much harder.
Lower priority thread: Less deterministic, easier to tune, won’t impact user, but runs longer.
Higher priority thread: More deterministic, can impact user, takes less time.
Peter
OSR
>apply. When you use a lower priority work thread with the wait, the thread is
blocked by user compute-bound activity. It’s precisely this user activity you
want to
facilitate… not pre-empt.
Imagine the IO bound thread which runs for 1us submitting one more IRP, then
waits for this IRP to complete for, say, 5ms.
In this case, if it is low-priority, it can be preempted during the 1us, and
starve (sitting in ready queue till the high-priority thread will release the
CPU) for, say, same 5ms. More so, its awaken can be delayed for the same time,
and it will get its 1us much later.
This means 50% drop of the IO performance.
Making this thread high-priority solves the issue, and does not occupies the
whole machine - the thread cannot occupy more the 1/5000 of CPU time due to its
very essense.
Lower priority thread: Less deterministic, easier to tune, won’t impact user,
but
runs longer.
Good for CPU-bound tasks like video encoding or so.
Higher priority thread: More deterministic, can impact user, takes less time.
Good for IO bound tasks.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Maxim,
time.>
My post was because
=> It must not impact user.
So I like Peters suggestion:
This is very effective in decreasing user impact.
If your goal is to only activate something when the user isn’t present, you
could have an application detect system idle (such as detecting when the
screensaver runs) and then have that app signal your driver code.>
Thanks for thinking
Else
|---------±-------------------------------->
| | “Maxim S. Shatskih” |
| | | | m> |
| | Sent by: |
| | bounce-311725-16691@li|
| | sts.osr.com |
| | |
| | |
| | 01/17/2008 12:19 AM |
| | Please respond to |
| | “Windows System |
| | Software Devs Interest|
| | List” |
|---------±-------------------------------->
>-----------------------------------------------------------------------------------------------------------|
| |
| To: “Windows System Software Devs Interest List” |
| cc: |
| Subject: Re:[ntdev] performance issue - control over thread |
>-----------------------------------------------------------------------------------------------------------|
>apply. When you use a lower priority work thread with the wait, the thread
is
>blocked by user compute-bound activity. It’s precisely this user activity
you
want to
>facilitate… not pre-empt.
Imagine the IO bound thread which runs for 1us submitting one more IRP,
then
waits for this IRP to complete for, say, 5ms.
In this case, if it is low-priority, it can be preempted during the 1us,
and
starve (sitting in ready queue till the high-priority thread will release
the
CPU) for, say, same 5ms. More so, its awaken can be delayed for the same
time,
and it will get its 1us much later.
This means 50% drop of the IO performance.
Making this thread high-priority solves the issue, and does not occupies
the
whole machine - the thread cannot occupy more the 1/5000 of CPU time due to
its
very essense.
> Lower priority thread: Less deterministic, easier to tune, won’t impact
user,
but
>runs longer.
Good for CPU-bound tasks like video encoding or so.
> Higher priority thread: More deterministic, can impact user, takes less
time.
Good for IO bound tasks.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
“Maxim S. Shatskih” wrote in message news:xxxxx@ntdev…
…
>> Lower priority thread: Less deterministic, easier to tune, won’t impact user,
> but
>>runs longer.
>
> Good for CPU-bound tasks like video encoding or so.
>
>> Higher priority thread: More deterministic, can impact user, takes less time.
>
> Good for IO bound tasks.
The Linux kernel book by Robert Love has discussion on the heuristic
they use to determine, is a process i/o or cpu bound.
It’s interesting. The “ease of use” principle demands that OS is self-tuning
and just does the right thing without much developer’s effort.
In NT, the priority boost arg in IPR completion could be used to tune up
low-priority tasks… but IMHO it’s hard to use “as is”, therefore often ignored.
Regards,
–PA
> The Linux kernel book by Robert Love has discussion on the heuristic they use to determine,
Actually, “Understanding Linux kernel” gives you much more detailed vision of the whole thing -
“Linux kernel development” is meant to be just an introduction without too much details.
In any case, it does not really apply in the Windows world. Linux kernel itself has no means of telling interactive process from the batch one - it has to guess it. Therefore, it calculates process priority based upon its average sleep time - it assumes that interactive process spends most of its time in inactive state. However, once Windows does all window management in the kernel mode, the kernel does not have to guess whether a given thread is interactive or not - all it has to do is to look at ETHREAD in order to find out whether a given thread is GUI one.Furthermore, Windows kernel also knows which particular thread has a kbd focus, so that it can give it a priority boost…
In NT, the priority boost arg in IPR completion could be used to tune up low-priority tasks
Hardly…
Please note that priority boost gets applied to the *basic* priority and not to the current one.
Therefore, if thread priority is, say, 4, disk IO is not getting to elevate it above 5, no matter how long threads spends on it. Therefore, the thread in question will still have a priority below the normal…
Anton Bassov.