Hi All,
I have a thread in my driver which does some book keeping periodically. It does lot of IOs and consumes CPU to make system unresponsive for some time. So I want to set THREAD_MODE_BACKGROUND_BEGIN for my thread. Can I do so using KeSetPriorityThread API? BTW I am running Windows 7 and above.
Thanks & Regards,
Amit Kulkarni.
Why do you think that you need it? Imagine a code like the one below:
/* thread A of higher priority */
while(1)
{
/* specify waiting period of, say, few seconds */
KeDelayExecutionThread(KernelMode,FALSE, period);
/*signal synchronization event */
KeSetEvent(&event,0,0);
}
/* thread B of low priority */
while(1)
{
KeWaitXXX(&event,…);
do_the_actual_processing();
}
Why don’t you want to try something like that? You have 2 kernel threads. The one of high priority wakes up once in a VERY while, sets synchronization event that the one of the low priority may be waiting on, and immediately goes to sleep again. The one of the lower priority does the process - wait cycle in a loop.
If you do it this way you will be able to do the background processing without disturbing anyone, i.e. at the time when there are no urgent tasks around…
Anton Bassov
That’s a nice workaround, but still my question is can we use THREAD_MODE_BACKGROUND_BEGIN?
Why does your thread consume CPU? Does it do a lot of calculation or memory shuffling?
A waitable timer will do the same thing without consuming an extra kernel thread stack.
-p
Sent from my Windows Phone
From: xxxxx@hotmail.commailto:xxxxx
Sent: ?7/?15/?2014 5:05 AM
To: Windows System Software Devs Interest Listmailto:xxxxx
Subject: RE:[ntdev] Will THREAD_MODE_BACKGROUND_BEGIN work for kernel mode thread?
Why do you think that you need it? Imagine a code like the one below:
/* thread A of higher priority /
while(1)
{
/ specify waiting period of, say, few seconds */
KeDelayExecutionThread(KernelMode,FALSE, period);
/*signal synchronization event /
KeSetEvent(&event,0,0);
}
/ thread B of low priority */
while(1)
{
KeWaitXXX(&event,…);
do_the_actual_processing();
}
Why don’t you want to try something like that? You have 2 kernel threads. The one of high priority wakes up once in a VERY while, sets synchronization event that the one of the low priority may be waiting on, and immediately goes to sleep again. The one of the lower priority does the process - wait cycle in a loop.
If you do it this way you will be able to do the background processing without disturbing anyone, i.e. at the time when there are no urgent tasks around…
Anton Bassov
—
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>
> A waitable timer will do the same thing without consuming an extra kernel
thread stack.
-p
Sent from my Windows Phone
From: xxxxx@hotmail.commailto:xxxxx
> Sent: ý7/ý15/ý2014 5:05 AM
> To: Windows System Software Devs Interest Listmailto:xxxxx
> Subject: RE:[ntdev] Will THREAD_MODE_BACKGROUND_BEGIN work for kernel mode
> thread?
>
> Why do you think that you need it? Imagine a code like the one below:
>
>
> /* thread A of higher priority /
>
> while(1)
>
> {
>
> / specify waiting period of, say, few seconds */
>
> KeDelayExecutionThread(KernelMode,FALSE, period);
>
> /*signal synchronization event /
> KeSetEvent(&event,0,0);
>
> }
>
>
> / thread B of low priority */
>
> while(1)
>
> {
>
>
> KeWaitXXX(&event,…);
>
>
> do_the_actual_processing();
>
> }
>
This code cannot work correctly. First, you have to understand that
thread priority is irrelevant, and has absolutely NOTHING to do with
concurrency. There are silly people out there that think a low-priority
thread won’t run until the high-priority thread blocks. Of course,
thinking about this for more than a millisecond makes it painfully obvious
this is nonsense; there may be only two runnable threads on your multicore
machine, the low-priority thread and the high-priority thread. They will
run concurrently on different cores. In the presence of the Balance Set
Manager, even a single-core assumption (including setting thread
affinities) fails.
What is not shown here is where the event is set. If there is a race
condition such that the thread that sets the event does the set a
microsecond before the other thread does the clear, you have a deadlock.
In this case, you should be using a semaphore, which guarantees atomicity
across such conditions. In any case, it is not clear why you would have a
thread waiting on a timer in this case.
NEVER, EVER think priority can be used to guarantee correctness of
concurrent code.
>
>
>
> Why don’t you want to try something like that? You have 2 kernel threads.
> The one of high priority wakes up once in a VERY while, sets
> synchronization event that the one of the low priority may be waiting on,
> and immediately goes to sleep again. The one of the lower priority does
> the process - wait cycle in a loop.
> If you do it this way you will be able to do the background processing
> without disturbing anyone, i.e. at the time when there are no urgent tasks
> around…
This is nonsense. There is no way to tell what an “urgent” task is, or
what priority it is running at. And it doesn’t address the issue of
priority inversion if the “low priority” thread shares some resource with
an “urgent task”.
Consuming CPU cycles always disturbs something.
joe
>
>
>
> Anton Bassov
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>
> Hi All,
I have a thread in my driver which does some book keeping periodically. It
does lot of IOs and consumes CPU to make system unresponsive for some
time. So I want to set THREAD_MODE_BACKGROUND_BEGIN for my thread. Can I
do so using KeSetPriorityThread API? BTW I am running Windows 7 and above.Thanks & Regards,
Amit Kulkarni.
Never, ever mess with priorities. No matter what you do, the chances you
will make a bad situation worse are very high.
That said, VERY CAREFUL manipulation of priorities is sometimes both
useful and necessary. Emphasis on the “very careful” part. You can
trivially create a thread that locks up the entire system, or which
becomes indefinitely starved, neither of which are productive. Note that
the Balance Set Manager does not apply to threads at levels 16-31, meaning
starvation can be a real problem, whether you are starving some other
thread, or some other thread is starving you.
joe
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>This code cannot work correctly.
Of course it can…
What is not shown here is where the event is set. If there is a race condition such that the thread
that sets the event does the set a microsecond before the other thread does the clear, you
have a deadlock.
Apparently, what you are saying is that it is not shown where this event gets reset, right?
This is why I made it clear that we are speaking about synchronization event and not a notification one.
Unlike notification event, synchronization one does not need to be reset explicitly - it happens upon the release of the waiting thread…
In this case, you should be using a semaphore, which guarantees atomicity across such conditions.
Syncronization event is, for all practical purposes, just a semaphore that has maxcount of 1 (i.e a binary semaphore). Windows implementation of semaphores in just broken by design. This is why synch event is normally used under Windows in situations when other OSes use binary semaphores…
NEVER, EVER think priority can be used to guarantee correctness of concurrent code.
Sure…
However, the code shown does NOT rely upon the priorities for this purpose. The only reason why the priorities are mentioned here is because this event has to get signaled by someone. If a thread that wakes up once in a VERY while also happens to be of low priority as well…well, I am afraid the waiting thread would have a chance to perform its actual task too infrequently to be observed…
This is nonsense.
Do you mean your own participation on this thread? Indeed, up to the point it seems to be non-sensical - you seem to be arguing against yourself, because you are arguing against the statements and assumptions that no one on this thread made…
There is no way to tell what an “urgent” task is,
Well, as long as you mean priorities of a dynamic class, on my books an “urgent” task is the one that currently interacts with the user
or what priority it is running at.
… and this part gets addressed by assigning the priorities of either RT class (in case of high-priority tasks),
or, instead, of an IDLE class for something that is supposed to run only when there is nothing else to be done at the moment…
In any case, you cannot say what the “urgent” task is unless you compare its"urgency" to the one of other tasks…
And it doesn’t address the issue of priority inversion if the “low priority” thread shares some
resource with an “urgent task”.
Bu the way, once we are at it, this problem has been solved in the UNIX world ages ago. In the early UNIX this problem did not exist at all because all kernel code was non-preemptible by anyone, apart from interrupt handlers. When Solaris introduced kernel pre-emption it also ensured that all code that runs in the kernel has a priority above the one in the userland (RT threads are an exception which, IMHO, simply don’t belong in the world of GPOSes, in the first place), effectively introducing a kind of priority ceiling protocol…
Anton Bassov
>When Solaris introduced kernel pre-emption it also ensured that all code that runs in the kernel has a priority above the one in the userland
Default kernel thread priority in Windows is above any userland thread (except for realtime).
> Default kernel thread priority in Windows is above any userland thread (except for realtime).
Yes, but, IIRC, priority of a user thread does not get adjusted when it enters the kernel mode, right - instead, it runs at its original priority, which makes priority inversion a very real scenario if this threads uses shared resources…
Anton Bassov
>> Default kernel thread priority in Windows is above any userland thread
> (except for realtime).
Yes, but, IIRC, priority of a user thread does not get adjusted when it
enters the kernel mode, right - instead, it runs at its original priority,
which makes priority inversion a very real scenario if this threads uses
shared resources…
That is my understanding also. A thread running in the kernel in the user
thread context is subject to timeslicing, preemption, and all the other
slings and arrows of outrageous scheduling. However, you can force it to
move up to DISPATCH_LEVEL or DIRQL_LEVEL(s) by an assortment of methods,
and this is sometimes done specifically do prevent preemption. Messing
with IRQL levels is nearly as dangerous as messing with thread priorities
in most cases, and far more likely to be destructive in others. And that
covers 100% of all cases (all special case exceptions are lost in the
statistical roundoff–this originall referred to “90% of the world’s code
is written in COBOL, 10% in FORTRAN, and all the rest in other languages”.
That was true when I started, but the shift to C/C++, scripting
languages, PERL, Python, JavaScript, and even Java have messed up that
simple equation).
Playing with priorities can be hazardous (not to mention creating
opportunities for deadlock, or the appearance of deadlock).
And that default priority can get you into serious trouble, in that the
Balance Set Manager only looks at thread priorities 0-15 (no, I don’t know
what happens with the multimedia scheduling is being used), meaning that a
runaway or compute-intensive 16-31 thread can have a serious negative
impact on the entire system, and thread starvation of all threads whose
priority is lower is a potentially genuine problem. Even the “idle”
thread in the “time-sensitive” class (16-31) can starve all user threads,
and the Balance Set Manager can, at best, raise the priority of a starved
thread to 15. Setting priorities higher than the file system threads is
extremely risky, because you can starve the file system updating threads.
Not A Pretty Sight. Note also that when a mutex is acquired, the thread
that acquires it (IIRC) is boosted to 31 to avoid priority inversion
issues, but if it goes off and does something compute-intensive at that
point, you are well-and-truly screwed.
The SetThreadPriority calls in all their guises should have a “here there
be dragons” warning prominently displayed.
joe
Anton Bassov
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
> Note also that when a mutex is acquired, the thread that acquires it (IIRC) is boosted to 31
to avoid priority inversion issues,
Since when is priority ceiling protocol is implemented under Windows??? You must be confusing it with RTOSes…
Anton Bassov
>>This code cannot work correctly.
Of course it can…
> What is not shown here is where the event is set. If there is a race
> condition such that the thread
> that sets the event does the set a microsecond before the other thread
> does the clear, you
> have a deadlock.Apparently, what you are saying is that it is not shown where this event
gets reset, right?This is why I made it clear that we are speaking about synchronization
event and not a notification one.
Unlike notification event, synchronization one does not need to be reset
explicitly - it happens upon the release of the waiting thread…
Synchronization events have their own problems, which is why they are not
used very much…in fact, I remember someone from Microsoft rolling his
eyes and saying “Don’t DO that!”
> In this case, you should be using a semaphore, which guarantees
> atomicity across such conditions.Syncronization event is, for all practical purposes, just a semaphore
that has maxcount of 1 (i.e a binary semaphore). Windows implementation
of semaphores in just broken by design. This is why synch event is
normally used under Windows in situations when other OSes use binary
semaphores…
How is the Windows implementation of semaphores broken?
> NEVER, EVER think priority can be used to guarantee correctness of
> concurrent code.Sure…
However, the code shown does NOT rely upon the priorities for this
purpose. The only reason why the priorities are mentioned here is because
this event has to get signaled by someone. If a thread that wakes up once
in a VERY while also happens to be of low priority as well…well, I
am afraid the waiting thread would have a chance to perform its actual
task too infrequently to be observed…
Thread starvation is a very real problem.
> This is nonsense.
Do you mean your own participation on this thread? Indeed, up to the point
it seems to be non-sensical - you seem to be arguing against yourself,
because you are arguing against the statements and assumptions that no one
on this thread made…
> There is no way to tell what an “urgent” task is,
Well, as long as you mean priorities of a dynamic class, on my books an
“urgent” task is the one that currently interacts with the user.
That is not an urgent class. In fact, it is running at priority around 8
(it has changed a few times, but I believe it is in that range). This
puts seven more levels of “urgent” above it, and also means that there are
seven (not counting 0) “non-urgent” levels below it. Note that background
threads run two priority levels lower than the foreground thread (defined
as “that thread that owns the window that has the focus”). I have played
some very weird games with thread priorities to get video processing to
run at priority 15 while not killing the UI thread. This trick falls
under the VERY CAREFULLY rubric.
> or what priority it is running at.
… and this part gets addressed by assigning the priorities of either RT
class (in case of high-priority tasks),
or, instead, of an IDLE class for something that is supposed to run only
when there is nothing else to be done at the moment…
Well, not quite. Remember there is an IDLE class in the “realtime” (or
The Class Formerly Known As Realtime, because realtime people would make
faces and laugh at Microsoft people when they used this term), which is
Priority 16, and it preempts ALL of the normal user thread classes (1-15).
So if you put an “idle loop” in this “idle” thread, you are screwed.
In any case, you cannot say what the “urgent” task is unless you compare
its"urgency" to the one of other tasks…
I agree.
> And it doesn’t address the issue of priority inversion if the “low
> priority” thread shares some
> resource with an “urgent task”.Bu the way, once we are at it, this problem has been solved in the UNIX
world ages ago. In the early UNIX this problem did not exist at all
because all kernel code was non-preemptible by anyone, apart from
interrupt handlers. When Solaris introduced kernel pre-emption it also
ensured that all code that runs in the kernel has a priority above the one
in the userland (RT threads are an exception which, IMHO, simply don’t
belong in the world of GPOSes, in the first place), effectively
introducing a kind of priority ceiling protocol…
Well, as it turns out, that doesn’t “solve” the priority inversion
problem, it just suggests that it is harder for a user-priority thread to
induce it. There is a huge body of literature on solutions to the
priority-inversion problem, dating back to at least the early 1960s;
MULTICS knew about it; RTOS people have created working solutions (there
is a famous documented example of fixing the priority inversion problem on
one of the robotic space missions by uploaded a patch that implemented the
priority-inversion-avoidance algorithm, within the last couple years).
General purpose operating systems rarely have any mechanism for dealing
with priority inversion in the general case, just hacks (like the Balance
Set Manager and the mutex-to-31 hack). These hacks heuristically solve
the problem in the most common cases, but they don’t solve it for special
cases.
joe
Anton Bassov
NTDEV is sponsored by OSR
Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
OSR is HIRING!! See http://www.osr.com/careers
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
I think we have diverted from the original question which is…
Will THREAD_MODE_BACKGROUND_BEGIN work for kernel mode thread?
Yes, we should take care while altering the priorities and I will do so. BTW I am not only has concerns about CPU but also for IO, memory and others. And THREAD_MODE_BACKGROUND_BEGIN manages it in better way(Please go through MSDN). Also THREAD_MODE_BACKGROUND_BEGIN doesn’t seems to be a true priority 0-31. It just tells windows if there are foreground processes consuming the resources then this thread should not starve them.
>Note also that when a mutex is acquired, the thread that acquires it (IIRC) is boosted to 31 to avoid priority inversion issues
I don’t think it does. Windows scheduler knows when a higher priority thread is waiting for a mutex and this lets it bump the owning thread to the waiter’s priority. That’s the only thing that’s needed.
>
I don’t think it does. Windows scheduler knows when a higher priority
thread is waiting for a mutex and this lets it bump the owning thread to
the waiter’s priority. That’s the only thing that’s needed.
I don’t think that’s the case. I just tried it out (Windows 8.1) and there
is no priority boost for the owning thread.
Here’s a simple repro. I inspected the dynamic priorities with Process
Explorer. The main thread has priority 8 and the waiting thread has
priority 10.
#include <windows.h>
HANDLE mutex;
int main()
{
mutex = CreateMutex(nullptr, TRUE /* initial owner */, nullptr);
auto highPriThread = CreateThread(nullptr, 0, {
WaitForSingleObject(mutex, INFINITE);
return DWORD{ 0 };
}, nullptr, 0, nullptr);
SetThreadPriority(highPriThread, THREAD_PRIORITY_HIGHEST);
while (true) Yield();
return 0;
}</windows.h>
@Sasha Goldstein:
I don’t think the out of line priority change will get propagated from the waiting thread after it’s put to wait state. You need to change the thread priority within the thread function before calling WaitForSingleObject.
On Wed, Jul 16, 2014 at 8:05 PM, wrote:
> @Sasha Goldstein:
>
> I don’t think the out of line priority change will get propagated from the
> waiting thread after it’s put to wait state. You need to change the thread
> priority within the thread function before calling WaitForSingleObject.
>
>
Nope, that doesn’t change anything. Do you have any reference for your
claim that the scheduler propagates priority through a mutex? There is no
mention of it in “Windows Internals” or on MSDN as far as I can tell. The
only kind of starvation avoidance Windows applies, to the best of my
knowledge, is when the balance set manager boosts the priority of ready
threads that haven’t run for a long amount of time.
#include <windows.h>
HANDLE mutex;
int main()
{
mutex = CreateMutex(nullptr, TRUE /* initial owner */, nullptr);
CloseHandle(CreateThread(nullptr, 0, {
SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_HIGHEST);
WaitForSingleObject(mutex, INFINITE);
return DWORD{ 0 };
}, nullptr, 0, nullptr));
while (true) Yield();
return 0;
}</windows.h>
> I don’t think it does. Windows scheduler knows when a higher priority thread is waiting for
a mutex and this lets it bump the owning thread to the waiter’s priority. That’s the only thing that’s needed.
Well, you seem to go further than even Dr.Joe himself - according to him, Windows just boosts the priority of mutex owner to 31, which is a priority ceiling protocol (not a “hack” as Dr.Joe thinks but, instead, a fully workable solution from the world of hard-realtime design). You seem to go even further than that - you are speaking about boosting owner’s priority to that of higher-priority waiter, which is already a priority-inheritance protocol. Which OS are you speaking about, guys??? Look - both priority ceiling and priority-inheritance protocols firmly belong in the world of RTOSes and are virtually unheard of in the world of GPOSes, simply because they have very specific use and are subjected to the requirements that GPOSes normally don’t meet. In order to make the above protocols work for you you have to ensure that mutex owner never ever enters unbounded wait (i.e a wait on some construct other than RT mutex) - you cannot avoid priority inheritance if you rely upon semaphores, completions, RW locks and other dispatcher constructs that are widely used by drivers in GPOSes …
Anton Bassov
No, this isn’t something you can set from kernel mode with the Ke APIs. This
is handled in user mode and affects more than just scheduling priority, for
example is may also lower page and I/O priority.
-scott
OSR
@OSRDrivers
wrote in message news:xxxxx@ntdev…
I think we have diverted from the original question which is…
Will THREAD_MODE_BACKGROUND_BEGIN work for kernel mode thread?
Yes, we should take care while altering the priorities and I will do so. BTW
I am not only has concerns about CPU but also for IO, memory and others. And
THREAD_MODE_BACKGROUND_BEGIN manages it in better way(Please go through
MSDN). Also THREAD_MODE_BACKGROUND_BEGIN doesn’t seems to be a true priority
0-31. It just tells windows if there are foreground processes consuming the
resources then this thread should not starve them.