How to keep list of active requests in KMDF

NTDEV
1

I’m porting an existing WDM driver to KMDF. The driver completes most
IOCTLs immediately; however, there is one that it stashes away and
returns STATUS_PENDING. The pending IRPs are kept in a simple list, and
they get completed in some indeterminate order after some indeterminate
time. What KMDF idiom should I use to implement this?

Currently my default queue is sequential. Should I create a manual
queue and move those requests to it? Or can this be done by making the
default queue parallel, and retrieving the pending requests from it
later when it’s time to complete (and how is that done)? Or should I
just stick with the current solution and stash them away in a list (and
if so, is it safe to store the WDFREQUEST handle in my list)?

Cheers,
– mkj

//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//_______________________________________________

2

Michael Jones wrote:

I’m porting an existing WDM driver to KMDF. The driver
completes most IOCTLs immediately; however, there is
one that it stashes away and returns STATUS_PENDING.
The pending IRPs are kept in a simple list, and they get
completed in some indeterminate order after some
indeterminate time. What KMDF idiom should I use
to implement this?

You are right, manual queue is what you want. Then you get cancellation for free, too.

3

Interesting that this question should come up just as I fixed a bug with my
manual queue (see my EvtIoCancelledOnQueue thread).

I agree with Chris that a manual queue is the right way to deal with this.

I don’t agree that you get cancellation for free. It costs 2 cents (as
opposed to 100 dollars if you choose to do your own queue). You must
declare an EvtIoCancelledOnQueue routine, and complete the request in that
routine.

  • Dan.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Tuesday, July 17, 2007 7:03 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to keep list of active requests in KMDF

Michael Jones wrote:

I’m porting an existing WDM driver to KMDF. The driver
completes most IOCTLs immediately; however, there is
one that it stashes away and returns STATUS_PENDING.
The pending IRPs are kept in a simple list, and they get
completed in some indeterminate order after some
indeterminate time. What KMDF idiom should I use
to implement this?

You are right, manual queue is what you want. Then you get cancellation for
free, too.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

4

Dan Kyler wrote:

I don’t agree that you get cancellation for free. It costs
2 cents (as opposed to 100 dollars if you choose to do
your own queue). You must declare an
EvtIoCancelledOnQueue routine, and complete the request
in that routine.

What? I don’t think so. You don’t have to declare the routine, check the documentation…

5

EvtIoCancelledOnQueue is not required nor do you have to complete the
canceled request in this routine, you can pend the completion to a later
point in time. Why do you think this must be implemented?

d

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Tuesday, July 17, 2007 9:50 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to keep list of active requests in KMDF

Dan Kyler wrote:

I don’t agree that you get cancellation for free. It costs
2 cents (as opposed to 100 dollars if you choose to do
your own queue). You must declare an
EvtIoCancelledOnQueue routine, and complete the request
in that routine.

What? I don’t think so. You don’t have to declare the routine, check
the documentation…

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

6

It is my observation that the framework does not complete requests that have
been forwarded to a manual queue when they are cancelled. This differs from
the behavior for queues with requests that have never been delivered to the
driver. Am I wrong?

Assuming that is correct, and if not, I’ve apparently found a horrible bug
in Kmdf, then the driver must complete the canceled request. The way for
the driver to contemporaneously know that the request is being canceled is
to declare an EvtIoCancelledOnQueue routine. At that point I’m not sure why
one would repend things and do the completion later. Possibly something to
do with hardware–I wouldn’t know anything about that.

The way I use manual queues, and the way I’ve mostly seen the use of manual
queues being discussed on this list, is for long pended operations which
will complete on some event. The dispatch routine places the request in the
manual queue. When the event occurs, all the requests in the queue are
removed from the queue and completed. In my case this is an ioctl with no
arguments. The dispatch routine does no processing on the request.

Saying that a manual queue gives you cancellation for free (and I believe
I’ve used those very words on this list) implies that you don’t have to do
anything if the request is cancelled while on the queue. The request isn’t
really cancelled until it’s completed, so it’s not free, it’s cheap.

  • Dan.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Doron Holan
Sent: Wednesday, July 18, 2007 12:51 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] How to keep list of active requests in KMDF

EvtIoCancelledOnQueue is not required nor do you have to complete the
canceled request in this routine, you can pend the completion to a later
point in time. Why do you think this must be implemented?

d

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Tuesday, July 17, 2007 9:50 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to keep list of active requests in KMDF

Dan Kyler wrote:

I don’t agree that you get cancellation for free. It costs
2 cents (as opposed to 100 dollars if you choose to do
your own queue). You must declare an
EvtIoCancelledOnQueue routine, and complete the request
in that routine.

What? I don’t think so. You don’t have to declare the routine, check the
documentation…

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

7

>>
It is my observation that the framework does not complete requests that have
been forwarded to a manual queue when they are cancelled. This differs from
the behavior for queues with requests that have never been delivered to the
driver. Am I wrong?
<<

EvtIoCanceledOnQueue is called if it exists and the request has been presented to the driver.

In all other cases (no callback, never presented) the request is completed with STATUS_CANCELLED.

>
Assuming that is correct, and if not, I’ve apparently found a horrible bug
in Kmdf,
<<

I would most certainly like to know the details of this.

>
Saying that a manual queue gives you cancellation for free (and I believe I’ve used those very words on this list) implies that you don’t have to do anything if the request is cancelled while on the queue.
<<

You don’t have to do anything to complete the request when it is cancelled while on the queue unless you want to. You do have to consider in your rationale for using the queue that the request can be cancelled while on the queue and thus “disappear” on you. Some people don’t consider that “free”.

8

We are discussing a manual queue which has been populated with
WdfRequestForwardToIoQueue. The requests have always been previously
presented to the driver.

So, in this scenario, if there is an EvtIoCanceledOnQueue, it is called, the
driver owns the request, and the driver must complete the request.

That is consistent with what I have seen (I had a callback, for the purpose
of debug logging).

What is the behavior in this case if there is no EvtIoCanceledOnQueue
callback? Since the request has been presented, will it be completed, or
will it sit on the manual queue waiting for someone to do something?

Thanks,

  • Dan.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bob Kjelgaard
Sent: Wednesday, July 18, 2007 8:20 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] How to keep list of active requests in KMDF

>
It is my observation that the framework does not complete requests that have
been forwarded to a manual queue when they are cancelled. This differs from
the behavior for queues with requests that have never been delivered to the
driver. Am I wrong? <<

EvtIoCanceledOnQueue is called if it exists and the request has been
presented to the driver.

In all other cases (no callback, never presented) the request is completed
with STATUS_CANCELLED.

>
Assuming that is correct, and if not, I’ve apparently found a horrible bug
in Kmdf, <<

I would most certainly like to know the details of this.

>
Saying that a manual queue gives you cancellation for free (and I believe
I’ve used those very words on this list) implies that you don’t have to do
anything if the request is cancelled while on the queue. <<

You don’t have to do anything to complete the request when it is cancelled
while on the queue unless you want to. You do have to consider in your
rationale for using the queue that the request can be cancelled while on the
queue and thus “disappear” on you. Some people don’t consider that “free”.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

9

Okay, I think I’ve finally got this through my thick skull.

Given a manual queue populated with WdfRequestForwardToIoQueue, if there is
an EvtIoCanceledOnQueue callback, it is called, the driver owns the request,
and the driver must complete the request. If there is no callback the
request is cancelled and completed by the framework (verified
experimentally).

My problem was that I had a callback which did not complete the request. It
was not clear to me from the documentation that EvtIoCanceledOnQueue is not
just notification–its presence changes the cancellation behavior of the
framework.

I also note that in the Wdf doc section on cancellation, the term “delivered
to the driver” has 2 meanings: ever delivered in the past, and delivered but
not given back. Cancellation has 3 meanings: requested to be cancelled,
marked cancelled by the framework, and marked cancelled and completed by the
framework.

Of course my confusion would have been much less with framework source.

Thanks for clearing things up,

  • Dan.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Dan Kyler
Sent: Wednesday, July 18, 2007 8:44 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] How to keep list of active requests in KMDF

We are discussing a manual queue which has been populated with
WdfRequestForwardToIoQueue. The requests have always been previously
presented to the driver.

So, in this scenario, if there is an EvtIoCanceledOnQueue, it is called, the
driver owns the request, and the driver must complete the request.

That is consistent with what I have seen (I had a callback, for the purpose
of debug logging).

What is the behavior in this case if there is no EvtIoCanceledOnQueue
callback? Since the request has been presented, will it be completed, or
will it sit on the manual queue waiting for someone to do something?

Thanks,

  • Dan.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bob Kjelgaard
Sent: Wednesday, July 18, 2007 8:20 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] How to keep list of active requests in KMDF

>
It is my observation that the framework does not complete requests that have
been forwarded to a manual queue when they are cancelled. This differs from
the behavior for queues with requests that have never been delivered to the
driver. Am I wrong? <<

EvtIoCanceledOnQueue is called if it exists and the request has been
presented to the driver.

In all other cases (no callback, never presented) the request is completed
with STATUS_CANCELLED.

>
Assuming that is correct, and if not, I’ve apparently found a horrible bug
in Kmdf, <<

I would most certainly like to know the details of this.

>
Saying that a manual queue gives you cancellation for free (and I believe
I’ve used those very words on this list) implies that you don’t have to do
anything if the request is cancelled while on the queue. <<

You don’t have to do anything to complete the request when it is cancelled
while on the queue unless you want to. You do have to consider in your
rationale for using the queue that the request can be cancelled while on the
queue and thus “disappear” on you. Some people don’t consider that “free”.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

10

>>
We are discussing a manual queue which has been populated with
WdfRequestForwardToIoQueue. The requests have always been previously
presented to the driver.
<<

I understood your scenario, my point is that the cancellation logic applies even if there is no previous forwarding involved. Requests cancelled while on a queue are either completed by the framework or your callback is invoked. If they’re in any other state, I’m certain we want to know about it.

>
What is the behavior in this case if there is no EvtIoCanceledOnQueue
callback? Since the request has been presented, will it be completed, or
will it sit on the manual queue waiting for someone to do something?
<<

If a request sitting in a queue [hence owned by the framework and not the driver] is cancelled and there is no EvtIoCanceledOnQueue callback, the request is removed from the queue and completed STATUS_CANCELLED.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bob Kjelgaard
Sent: Wednesday, July 18, 2007 8:20 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] How to keep list of active requests in KMDF

>
It is my observation that the framework does not complete requests that have
been forwarded to a manual queue when they are cancelled. This differs from
the behavior for queues with requests that have never been delivered to the
driver. Am I wrong? <<

EvtIoCanceledOnQueue is called if it exists and the request has been
presented to the driver.

In all other cases (no callback, never presented) the request is completed
with STATUS_CANCELLED.

>
Assuming that is correct, and if not, I’ve apparently found a horrible bug
in Kmdf, <<

I would most certainly like to know the details of this.

>
Saying that a manual queue gives you cancellation for free (and I believe
I’ve used those very words on this list) implies that you don’t have to do
anything if the request is cancelled while on the queue. <<

You don’t have to do anything to complete the request when it is cancelled
while on the queue unless you want to. You do have to consider in your
rationale for using the queue that the request can be cancelled while on the
queue and thus “disappear” on you. Some people don’t consider that “free”.

11

Thanks to all for answering. Thanks also to Dan and Bob for the
discussion about EvtIoCanceledOnQueue; it certainly is not obvious from
the documentation that you have to complete the request if you provide
this callback; to wit:

“A driver’s EvtIoCanceledOnQueue event callback function informs the
driver that an I/O request was canceled while it was in an I/O queue.”

The Serial sample does complete the request in it’s
EvtIoCancelledOnQueue callback, so that would seem to confirm the
requirement.

Cheers,

–mkj

Michael Jones wrote:

I’m porting an existing WDM driver to KMDF. The driver completes most
IOCTLs immediately; however, there is one that it stashes away and
returns STATUS_PENDING. The pending IRPs are kept in a simple list, and
they get completed in some indeterminate order after some indeterminate
time. What KMDF idiom should I use to implement this?

Currently my default queue is sequential. Should I create a manual
queue and move those requests to it? Or can this be done by making the
default queue parallel, and retrieving the pending requests from it
later when it’s time to complete (and how is that done)? Or should I
just stick with the current solution and stash them away in a list (and
if so, is it safe to store the WDFREQUEST handle in my list)?

Cheers,
– mkj

//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//_______________________________________________

12

.

I find it helps to think in terms of “ownership”, rather than “delivery”. When ANY of the callbacks in the WDF_IO_QUEUE_CONFIG are called, they present a request which has been removed from the queue, and which the driver now “owns”. To give ownership back to the framework, you have to place the request in a queue (WdfRequestForwardToIoQueue, WdfRequestRequeue, or WdfRequestStopAcknowledge [hope I didn’t miss one]).

Your driver is expected to handle requests it owns- the framework will handle them otherwise [of course, that “handling” is often giving ownership back to you through one of the callbacks]. At any rate, it will never complete a request it doesn’t “own”.

wrt source access- I’d hate to count the number of times lately that having framework source hasn’t helped my confusion one iota [but that’s QA for you]…

One last thing [more for any other reader than specifically for Dan]- when I say I want to know about things you think are bugs, I’m not trying to imply “put up or shut up”. One of my main purposes for subscribing to this list is to see if I can pick up problems encountered in WDF that would otherwise not get reported [reference an um… “interesting” recent thread]. I’d like to avoid an MmGetSystemRoutineAddress type situation in WDF if I can [can’t say it won’t happen, life is far too unpredictable for that, but I can try until the time required overwhelms…]. Since the problems are usually well-thought through before they’re raised, it’s at least feasible…

13

Bob Kjelgaard wrote:

wrt source access- I’d hate to count the number of times lately that having
framework source hasn’t helped my confusion one iota [but that’s QA for you]…

Why I have the feeling that the KMDF team (and please forgive me Bob, Doron and others if I’m wrong) is not really *very interested* in releasing the sources?

Some time ago I thought the situation about KMDF sources was: Team and devs consider important to release the source. Management doesn’t agree and lawyers don’t give green light.

Now it seems to me that the team is not really convinced. I’m not saying they don’t want or they are the ones that deny source availability. Only that I have the *impression* that the team is not convinced that releasing KMDF source is really important.

And I’m not actually “blaming” them, they might have their reasons. But if the team is not convinced, then we have no chance of seeing the sources.

14

A bit OT, but:

Don’t read too much into my comment. It was a bit of venting- we recently spent many hours across several days root causing a hang in one of our tests at dispatch level that was highly but not perfectly reproducible in the lab, and both myself and one of the KMDF devs managed to misinterpret the cause several times before another dev finally nailed it. Source access just wasn’t enough, because sometimes the knowledge needed isn’t in the source, or is too hard to extract from it [or perhaps I was just too stupid to figure it out- I’m sure that happens, too].

I doubt I can do anything substantial about external source code licensing [I’d be surprised if anyone seriously asked me for my opinion on the subject], and don’t mean to imply otherwise. I also wouldn’t report the “interest” if I knew it, anyway [not my role]…

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@rahul.net
Sent: Wednesday, July 18, 2007 10:54 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] How to keep list of active requests in KMDF

Bob Kjelgaard wrote:

wrt source access- I’d hate to count the number of times lately that having
framework source hasn’t helped my confusion one iota [but that’s QA for you]…

Why I have the feeling that the KMDF team (and please forgive me Bob, Doron and others if I’m wrong) is not really *very interested* in releasing the sources?

Some time ago I thought the situation about KMDF sources was: Team and devs consider important to release the source. Management doesn’t agree and lawyers don’t give green light.

Now it seems to me that the team is not really convinced. I’m not saying they don’t want or they are the ones that deny source availability. Only that I have the *impression* that the team is not convinced that releasing KMDF source is really important.

And I’m not actually “blaming” them, they might have their reasons. But if the team is not convinced, then we have no chance of seeing the sources.