Getting along without so many DPCs

My driver uses a lot of DPCs to do the work of dispatch calls for the reason that the device operates on a specific processor’s internal registers. Setting the target processor for the DPC ensures that it runs on that processor.
To avoid details, I’ll just say that I’m having some synchronization problems between the IRP dispatch, the DPCs, and some work items that are necessary to do some IRP processing back at PASSIVE_LEVEL after the DPC has finished.
As an alternative, I’ve thought of having a worker thread, pinned to the processor, do all the work at PASSIVE_LEVEL, using some sort of work queue to communicate with the IRP dispatch methods. And if the IRP is being called by a user thread that is also pinned to the same processor (which in my driver’s case would be a typical situation), then it could just do the work itself and avoid context switches.
There is a complication, though. The behavior of the hardware involves interrupts and an ISR. Could the ISR simply signal some event and a worker thread could be waiting for it? I believe that KeSetEvent is OK at DIRQL as long as Wait = FALSE. Is this so??
Or is there some reason why an ISR should use a DpcForIsr?
In short, I have three kinds of operations the driver will perform (other than the ISR), and these can be serialized easily enough:

  1. Handling an IRP in the same thread that called it, when that thread is pinned to the same processor.
  2. Handling an IRP in a worker thread on behalf of the thread that called the IRP.
  3. Servicing an interrupt in the hardware, in a worker thread.

Could the ISR simply signal some event and a worker thread could be waiting for it?

A short answer is “No” - you cannot call ANY dispatcher functions at DIRQL. The maximum that you are allowed to do at DIRQL is to queue a DPC that mayalready call KeSetEvent()…

Anton Bassov

With all due respect, it sounds to me like you need a complete review of your driver’s design and a well thought out implementation strategy.

That’s not likely to be something you can get one message at a time in a peer support forum. Nobody is likely to take the time here to think through your issues end-to-end and discuss potential positive and negative trade-offs vs your desired goals. I know I’m not able to do that here.

My point is that there’s a limit to what we can do here, and while almost every answer you’re likely to get will be a good one, taken in isolation, you are unlikely to come away will a well engineer SYSTEM this way.

Peter

My point is that there’s a limit to what we can do here, and while almost every answer you’re likely to get will be a good one,
taken in isolation, you are unlikely to come away will a well engineer SYSTEM this way.

Actually, the whole thing starts, for some reason, reminding me of VinayKP and his NDIS driver…

Anton Bassov

Thanks, Peter. I know I’m pretty much on my own here.
I’ll try the new approach, and if it doesn’t work I’ll scrap it and think of something else.

I want to add that my driver will still need a DPC in one place, since I use a TIMER to monitor the hardware. Actually, this is in lieu of responding to an interrupt, which is still questionable since I have to find an available interrupt vector to use (that’s another story). The TIMER calls a DPC when it times out. If I want the timer handler code to be serialized with IRP requests, I can always have the timer DPC put something in my work queue.
My ISR doesn’t have to invoke a DPC, it can just put something in the work queue directly.

@anton_bassov said:

Could the ISR simply signal some event and a worker thread could be waiting for it?

A short answer is “No” - you cannot call ANY dispatcher functions at DIRQL. The maximum that you are allowed to do at DIRQL is to queue a DPC that mayalready call KeSetEvent()…

Anton Bassov

Sorry for not seeing this before making my prior comment. So that makes two places where the driver will need a DPC (service a timer or service an interrupt).
Thanks, Anton.

Actually, the whole thing starts, for some reason, reminding me of VinayKP and his NDIS driver…
Out of curiosity, could you point me to this?

Running the driver functions using a private thread pool has been working fine so far. The IRP dispatch methods, and the timer and interrupt DPCs put work items on a list, which the threads in the pool will see and execute. (Actually there’s only one thread, so that the items are performed sequentially).
However, there was an issue of my work items not running as soon as they were put on the list when the worker had nothing to do. This was because the worker thread was running at a normal thread priority, whereas the user code on the same processor was running at priority 15.
I solved this by calling KeSetPriorityThread (KeGetCurrentThread (), LOW_REALTIME_PRIORITY); in the worker thread, setting the priority to 16. I didn’t think that was legal, but it worked. The thread’s priority was indeed set to 16, and the work was getting done promptly.

Michael_Rolle wrote:

However, there was an issue of my work items not running as soon as they were put on the list when the worker had nothing to do. This was because the worker thread was running at a normal thread priority, whereas the user code on the same processor was running at priority 15.

That’s not your problem to solve.  If that user believes his user-mode
code is more important than your driver’s work, that’s his prerogative,
and it’s not your job to override him.  If his system isn’t operating
correctly, then the user needs to adjust HIS priorities until the system
does operate correctly.

Playing games with priorities is always a losing proposition. Always.

I solved this by calling KeSetPriorityThread (KeGetCurrentThread (), LOW_REALTIME_PRIORITY); in the worker thread, setting the priority to 16. I didn’t think that was legal, but it worked. The thread’s priority was indeed set to 16, and the work was getting done promptly.

It’s “legal” in the sense that it will do what it says, but it’s not
good practice.  Do you have a private thread pool, or are you borrowing
a system worker thread?  If you’re borrowing  a system thread, then you
are screwing up the balance of power in the kernel.

That’s not your problem to solve. If that user believes his user-mode code is more important than your driver’s work,
that’s his prerogative, and it’s not your job to override him.

True, but don’t forget that the whole thing is sort of a “hobby project” for the OP - if I got his posts right, he is not planing to distribute his software.

In any case, messing around with threads priorities is just innocuous and funny, compared to some other things that the OP does. Check his posts, and you will see it yourself…

Anton Bassov

Thanks, Tim. I agree with what you say. But there are particular circumstances of this project of mine that apply here.
My driver and the user code that calls it are part of a joint design to achieve a common purpose. The purpose is to use special CPU-specific hardware to profile a piece of user code that runs in a loop. The hardware provides sampling of the user code, which will tell me about the precise timing of each instruction in the loop. In order to get the most samples over a period of time, and in order to keep the timing from being confounded by being interrupted, the user code pins itself to, say, CPU 2 and sets its thread priority to 15.
That’s why my worker thread currently sets its priority to 16. And to answer your question, it’s a private thread and a private pool. The thread is pinned also to CPU 2 because the samples are located on CPU 2. I am dedicating all of CPU 2 to the measurement task. So having a worker thread at priority 16 only steals a small bit of time on CPU 2 and still leaves my other 3 CPUs available for the rest of the system.

@anton_bassov said:

That’s not your problem to solve. If that user believes his user-mode code is more important than your driver’s work,
that’s his prerogative, and it’s not your job to override him.

True, but don’t forget that the whole thing is sort of a “hobby project” for the OP - if I got his posts right, he is not planing to distribute his software.

In any case, messing around with threads priorities is just innocuous and funny, compared to some other things that the OP does. Check his posts, and you will see it yourself…

Anton Bassov

Thanks for the moral support.
Regarding the other things that I, the OP, have done, I’ve been experimenting with various approaches to accomplish the basic task of collecting sample data from the hardware.
The hardware can be programmed to signal an interrupt when there’s a sample available, but to date I haven’t found a way that I am confident about to connect an ISR to this interrupt, without the benefit of Plug and Play.
For now, I’m using a Timer to trigger the driver every 0.5 milliseconds to read another sample. The hardware is capable of sampling about once every 10 microseconds, so an interrupt-triggered sample reading would give me a 50-fold increase in the sampling rate. So ultimately I’d like have an interrupt-driven solution.

In that regard, I’d still like to hear from someone about how to integrate my driver with Plug and Play. See my post about that. And Anton in particular, where is this “VinayKP and his NDIS driver” you mentioned above?

Playing games with priorities is always a losing proposition. Always.

As it usually happens with sweeping blanket statements that allow no compromise, the above statement is wrong. Although it, indeed, does apply in most cases, there are some some scenarios (admittedly rare ones) when it does not.

For example, consider the scenario when you driver creates threads A and B. The former one is an IO-bound thread that blocks most of the time ( i.e. is a kind of thread that you normally encounter in drivers), while the latter one is meant to run some CPU-intense task, which means it is more than likely to use up its quantum every time it is scheduled to run on the CPU. In such case it makes a perfect sense to assign the latter thread a priority that is slightly below the one of the former…

Anton Bassov