Using KeSetPriorityThread() on user threads

I have some code that needs to run at PASSIVE_LEVEL (or APC_LEVEL), but it is indirectly delaying a DPC - i.e. while this code runs, a DPC may not be able to finish a task and will reschedule itself later.

This doesn’t seem to be a big deal usually, but sometimes when the CPU load is high, my passive thread gets preempted, and now my DPC is at the mercy of the scheduler. That’s… less than ideal.

In the typical case, I’d use KeSetPriorityThread() on the passive thread to make sure it doesn’t get indefinitely preempted by other passive threads. However, per MSDN:

KPRIORITY KeSetPriorityThread(
Inout PKTHREAD Thread,
In KPRIORITY Priority
);
Thread [in, out] Pointer to the driver-created thread.

“driver-created thread”
Does this mean calling that on a user thread that entered kernelmode through e.g. DeviceIoControl() is not valid? What about system workitem thread pool?

If that’s the case, is there some universal way to prevent passive threads from being scheduled out in favor of other passive threads, except when explicitly making a blocking call, such as accessing paged memory? Raising the IRQL isn’t really an option here.

Thanks!
Milo

Look at ZwSetInformationThread to do this in general. But you really
should be looking at your algorithm and why such a delay is such a problem
for you.

Don Burn
Windows Driver Consulting
Website: http://www.windrvr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Tuesday, March 07, 2017 9:59 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Using KeSetPriorityThread() on user threads

I have some code that needs to run at PASSIVE_LEVEL (or APC_LEVEL), but it
is indirectly delaying a DPC - i.e. while this code runs, a DPC may not be
able to finish a task and will reschedule itself later.

This doesn’t seem to be a big deal usually, but sometimes when the CPU load
is high, my passive thread gets preempted, and now my DPC is at the mercy of
the scheduler. That’s… less than ideal.

In the typical case, I’d use KeSetPriorityThread() on the passive thread to
make sure it doesn’t get indefinitely preempted by other passive threads.
However, per MSDN:

KPRIORITY KeSetPriorityThread(
Inout PKTHREAD Thread,
In KPRIORITY Priority
);
Thread [in, out] Pointer to the driver-created thread.

“driver-created thread”
Does this mean calling that on a user thread that entered kernelmode through
e.g. DeviceIoControl() is not valid? What about system workitem thread pool?

If that’s the case, is there some universal way to prevent passive threads
from being scheduled out in favor of other passive threads, except when
explicitly making a blocking call, such as accessing paged memory? Raising
the IRQL isn’t really an option here.

Thanks!
Milo

—
NTDEV is sponsored by OSR

Visit the list online at:
http:

MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
drivers!
Details at http:

To unsubscribe, visit the List Server section of OSR Online at
http:</http:></http:></http:>

Thanks!

Ideally, I’d isolate the part that the DPC depends on and just put a spinlock there, but that might be a huge change. Hopefully ZwSetInformationThread will work in the meantime.

That sounds like a bad design.

Your design is broken from the very beginning. Let’s look at your requirements more closely…

when the CPU load is high, my passive thread gets preempted, and now my DPC is at
the mercy of the scheduler. That’s… less than ideal.

…

Raising the IRQL isn’t really an option here.

If raising IRQL is not an option the only conclusion that one may make is that you do something that may result in a blocking call (i.e.call KeXXX() directly or access pageable memory). At this point you are, indeed, “at the mercy of the scheduler”, and raising thread priority is not going to help you in any possible way. Consider a classical scenario of priority inversion when a thread of a lower priority has to signal the event that your top-priority thread is waiting on. There is nothing that you can do about it under GPOS like Windows. RTOSes may handle this problem by means of priority inheritance or priority ceiling protocol, but even under RTOS it may work only with mutexes - when it comes to semaphores, events and any other constructs that haven’t got an exclusive owner and may be signaled by anyone this problem cannot have a solution.

Anton Bassov

The OP has a legitimate question that actually reflects a pretty common problem: He has work upon which his DPC depends, but that work needs done at IRQL PASSIVE_LEVEL.

How, the OP is also less than super-clear about what he’s doing in his DPC.

Assuming he queues the work item, and exits from his DPC… does the work in his work item… and then re-queues his DPC, there is absolutely nothing at all wrong with this design pattern. So, cut him some slack, guys.

Now, let’s answer his very basic question:

No, it’s just gratuitous, and in this case misleading, guidance on the part of the WDK documentation. The docs went through a phase, long ago, when this sort of “help” was common. KeSetPriorityThread is precisely the function you want to call.

NOW… it’s bad form to fool with the priorities of the work queue threads. I would recommend that you simply create your own worker thread precisely for this purpose and set its priority to what you want/need. Hey… and in this case, you’ll even be using KeSetPriorityThread the way it’s documented.

Amazing how it works out that way, huh?

Peter
OSR
@OSRDrivers

Thank you, Peter.

NOW… it’s bad form to fool with the priorities of the work queue threads. I
would recommend that you simply create your own worker thread precisely for this
purpose and set its priority to what you want/need.

Understood, will do that. Purely out of curiosity, are these in any way special, or is it just a matter of them being shared by various drivers that may make assumptions this breaks?

If raising IRQL is not an option the only conclusion that one may make is that
you do something that may result in a blocking call (i.e.call KeXXX() directly
or access pageable memory). At this point you are, indeed, “at the mercy of the
scheduler”

I am, and that is fine. Being blocked and scheduled out is generally not a big deal. That’s why there’s the re-queuing code for the DPC in the first place. It’s just that delaying the DPC too much affects performance.

and raising thread priority is not going to help you in any possible way.

I disagree. Hinting to the scheduler to prioritize my thread over other passive threads will reduce the average delay of my DPC.

As a simple example, assume N CPU-bound threads (all same priority) and no external IRQs/DPCs.
When my thread, X, makes a blocking call, the CPU will be given to another thread. On average, it will take (N/num_cores)*quantum before my thread X is scheduled again. That is, in addition to the time it takes to service the blocking call (e.g. page fault).

If my thread X has a higher priority than the other threads, it will only take 1*quantum extra time (page fault servicing still there). Under high load, N is high and the difference is noticeable.

If it were a matter of correctness (i.e. if being blocked too long caused a crash or corruption), I’d agree that the design is fundamentally broken. In this case, it is just performance. So I’ll concede that it’s poorly structured, but not of the “burn everything and start over” variety.

Thanks for the critique though.

They’re not REALLY special in any way. So, if you wanted to bump one and then later return it to its previous priority, I guess you COULD do that. You wouldn’t be the first one, I’m sure.

It’s just that, well, they’re set at their priority by the OS… it’s safer (and cleaner… and the code reads better) IMHO in the long run to just do this with your own little private thread.

Peter
OSR
@OSRDrivers

> If it were a matter of correctness (i.e. if being blocked too long caused a crash or corruption),

I’d agree that the design is fundamentally broken.

It depends on how you look at it. For example,consider an interactive video game that misses its deadlines on a more or less regular basis. Although technically there is no error that is caused by a delay, user experience may be less than satisfactory, which means a given product, despite being technically error-free, is not really usable,at least as far as a customer is concerned. I thought that this is what you were complaining about…

Anton Bassov

You may also be a victim of the “new and improved” lock-free dispatcher of Longhorn+. Which queues your thread for execution of some chosen processor, and it will only run when it’s its turn on that processor…not when any other processor is available.

Yes, it is a known problem, high priority code depends on low priority code to run, but the scheduler wont let it.

You have to break the code down and try to pull out the critical bits. Perhaps some of the thread code can be run at a higher irql if you copy the paged data it is using into non paged first?

As Peter V. advised:

NOW… it’s bad form to fool with the priorities of the work queue threads. I
would recommend that you simply create your own worker thread precisely for this
purpose and set its priority to what you want/need.

A dedicated thread is a step in the right direction.
This lets you both assign the tread to certain CPU (you’ll need to find the CPU) and assign the priority above even the system worker threads. When you have 4 CPUs or 8 or more, you can *hope* to be able to hijack one. But again, there’s no 100% warranty that nothing will interfere - remember the IPCs… temperature throttling… and avoid dependency on anything else (such as paging).

– pa