KMDF Driver Queue Design Discussion/Question

Hi Guys,
I have written many Windows (WDM, mini ports, avstreams, filters) drivers but have been out of windows driver development for about 8 years. Now I am back at it and found that WDF has made life a lot easier :).

I have to make a design decision for the queues that the driver will have. So a little background about the hardware that I am writing the function driver for. It is a hardware which is capable of handling many commands at the same time. For discussion sake lets say that 2k IOS can be fired to the controller at a time. At any point in time the driver knows the number of commands that are fired to the controller and if the controller is accepting any new commands or not.

After going through the WDF queues and the options that I have available, I believe that I can really write a driver without any INTERNAL driver queues in it. I can setup a WDF Queue which is power managed and parallel dispatch. When the driver receives a new WDF request and it cannot fire the command to the controller, it will put the command BACK in to the WDF queue which the driver has setup and stop the queues. When the driver detects that the controller can accept new commands, the driver will then start the queues again.

The driver will have to support cancellation by registering EvtWdfRequestCancel function and completion and Cancel will need to make sure that they do not step on each other.

What this basically means is that driver only owns the request that it can fire to the hardware and there is no other request in the driver any time.
Am I missing something here? My only doubt is that if the framework will deliver the commands fast enough to me so that the driver will be able to keep the controller busy.

Any opinions (for or against) are highly appreciated.

Thanks,
-AJ

Very very happy to be back in this space. :smiley:

You can’t do this with a single queue. If you put the request back in to your incoming parallel queue, the framework will immediately dispatch it right back at you.

I’m not sure why you are trying to minimize your use of queues. KMDF queues are a fabulous and lightweight resource. They add enormous design flexibility with very little overhead. If it makes your work simpler to have 32 manual queues, then for gosh sakes create 32 manual queues.

I agree with Tim that Queues are one of the best things in KMDF, and there’s no reason to try to minimize the number of them. OTOH, it sounds to ME like one Queue could be made to work.

If the number of I/Os that your hardware can handle is known in advance (and directly correlates to the received Requests), you could set the max count on the Queue and let KMDF handle this for you.

If not, you can stop the Queue and then put the Request you can’t handle back on the Queye. You’ll get it back when the Queue is started.

I’d keep the in-progress Requests on a Queue as well… very nice handling for Cancel,that way. Waaaaay easier in most cases than Registering a per Request cancel callback.

Peter

1 Like

Hi Peter,
I do believe that using stop and start will do the trick for me. Also having a active queue of all the commands that the driver has posted to hardware makes sense. Power stop is one place where we will have to cancel all the active commands.

Is there a “CancelAll” type call back that we can register? Any other place where we will have to cancel all the Requests?

-AJ

There are more elegant (and more “KMDF-like”) approaches that you could use instead of starting and stopping a parallel Queue. Consider the simplicity of a pair of Queues: a Sequential Queue at your top edge, and a Manual Queue that holds your pending Requests. As Requests arrive on your Sequential Queue, you check to see if there’s capacity on your hardware. If there is, you start the Request (and put it on your in-progress Queue that holds in-progress Requests). If not, you put the Request on the Manual Queue.

When a Request completes, in your DPCForISR, you remove as many Requests from the Manual Queue as can be started. If there are none, then there’s no pending work.

In terms of “cancel all” you can just walk through the in-progress Queue…

Peter

How will this work Peter? Isn’t sequential dispatch sends only one command to the driver at the time and will not dispatch another till the previous command is complete or cancelled? I will not be able to keep my hardware busy all the time. I think I do need a parallel dispatch queue.

  • AJ

It’ll work either way, depending on how you write your code. Remember that the Sequential Queue will dispatch a new Request when you forward the current Request to another Queue.

Hmmmm… if you do it exactly the way I outlined, I think the dispatching would need to be parallel… yes. Of course, you could simply always process through the Manual Queue, which would avoid this problem.

What I’m trying to avoid in my approach is the Queue stop/start, which is cumbersome and somewhat unexpected as a design element in KMDF. Better to substitute the Manual Queue, which really gives you exact.y what you want.

Peter

1 Like

Peter,

Just for my understanding, What is the problem with Stoping and restarting the queues? When you say its “cumbersome” , can you elaborate a bit on that? Does it acquire some lock or something to move the queue state to the requested one?

Will try and avoid the stop and start. In which case I will have to have a hold queue (manual dispatch) in my driver which will accept the request from the parallel KMDF dispatch queue. This would be the queue which the commands are held in the driver. The driver will then pick a request from the hold queue when it detects that the hardware can accept new command. When the command is fired to the hardware, the command moves from the Hold queue to Active queue(another Manual queue to keep track of fired command).

This will give driver all the control that it needs. The Cancellation will have to look at both the queues.

Thanks
-AJ

Just for my understanding, What is the problem with Stoping and restarting the queues

The stop/start state of a Queue is primarily designed to manage dispatching according to the D-State of the associated device. That’s why we have callbacks like EvtIoStop, that are called with each Request that is active from a Queue when the Queue’s state goes to stopped. Stopping/starting a Queue simply to control when you want one or two more Requests might initially sound like a good idea, but it confuses your code that would otherwise be able to use these built-in mechanisms. Not to mention that any dev who looks at your driver will almost certainly be “surprised” to see you manipulating a Queue the way you’ve described.

Peter

P.S. DO NOT indent your paragraphs, please. In Markdown, a line that starts with multiple spaces is considered code. That’s why you have strange highlighting on some of your paragraphs. I’ve edited your posts to remove that strange highlighting.

Thank you Peter. Really great information like always.