You can use device or queue level synchronization. The documentation page you would go to if you clicked the “automatic synchronization” link in the original of the text you pasted explains both options and how to specify them.
This concerns me a bit. Do you mean you are trying to manage the cancelable state of an IRP you have given to a lower level driver? As I understand it, you don’t own and shouldn’t attempt to manage an IRP once you’ve sent it to a target. The target driver will complete or cancel the IRP as it sees fit. In both cases, you will get your completion routine called and can check the cancelled state by looking at its completion status. For that matter, unless you have work to do when the request completes [your code snippet shows none, but I realize it may not accurately reflect your real situation], you can use the “Send and forget” option.
You should only use a cancel routine if the IRP is sitting in YOUR driver for a potentially long time with nothing going on [and you can safely cancel the operation, of course].
Also (regarding the code snippet)- I think your questions at the end reflect exactly what I said just now. Your cancel routine has an unregulated race condition with the driver(s) you sent the request to. You can’t complete it in the cancel routine if you no longer own it. You shouldn’t even say it can be canceled- you don’t and can’t control it anymore.
No, I am not sending IRP down to a lower level driver.
Basically these are IRPs that driver sends whenever an event occurs.
It will wait for that event to occur. When driver knows of an event, it
fills the data buffer in IRP and completes the IRP.
While the driver is about to complete the IRP, app might cancel the IRP.
Its this race condition b/w the two events that I was talking about.
This event can occur in the form of an asynchrous event like device
disconnect etc…
Does this still raise concerns?
“Bob Kjelgaard” wrote in message
news:xxxxx@ntdev…
This concerns me a bit. Do you mean you are trying to manage the cancelable
state of an IRP you have given to a lower level driver? As I understand it,
you don’t own and shouldn’t attempt to manage an IRP once you’ve sent it to
a target. The target driver will complete or cancel the IRP as it sees fit.
In both cases, you will get your completion routine called and can check the
cancelled state by looking at its completion status. For that matter,
unless you have work to do when the request completes [your code snippet
shows none, but I realize it may not accurately reflect your real
situation], you can use the “Send and forget” option.
You should only use a cancel routine if the IRP is sitting in YOUR driver
for a potentially long time with nothing going on [and you can safely cancel
the operation, of course].
>>No, I am not sending IRP down to a lower level driver.
Basically these are IRPs that driver sends whenever an event occurs.
It will wait for that event to occur. When driver knows of an event, it
fills the data buffer in IRP and completes the IRP.
While the driver is about to complete the IRP, app might cancel the IRP.
Its this race condition b/w the two events that I was talking about.
This event can occur in the form of an asynchrous event like device
disconnect etc…
Does this still raise concerns?<<
Then this is analogous to the stated race between the DpcForIsr and the cancel routine, if both can complete an IRP.
Pseudo code is probably not the best medium for this discussion- you can sketch an idea and it may looks safe, but then change something that looks insignificant and the whole thing breaks.
One thing to remember is that once you call WdfRequestComplete, the request handle itself is invalid (UNLESS you have referenced it with WdfObjectReference somewhere, in which case the original IRP is still gone, so you have to rely on your context info for everything). In your code snippet, you don’t show where the handle came from you are manipulating in the “completion event”. If you’re keeping it in something like a WDFCOLLECTION, for instance, the collection keeps such a reference, and you should be OK [if you do all the operations in the correct order]. But if not, then I’d reference it before making it cancelable to be sure it was still valid when the completion event arose (and dereference it afterward).
Other than that, I think the illustration may work (but I’d prefer to get Doron’s or Jake’s opinion on that- properly handling the synchronization issues surrounding cancellation is HARD, and I don’t feel adequate as judge in this matter- I prefer to let KMDF handle as much of that burden as possible when I’m using it).
Also, no matter what opinion gets rendered on a sketched implementation plan I’d also run SDV on the real code [I think you can do that, but I don’t remember if those changes for SDV-KMDF made it into Vista or will end up in SP1/Server- in the latter case, you might not be able to do this, yet], if I were in this position.
To clarify, an object reference (wdfObjectReference(request)) will not
prevent the underlying PIRP from being completed. The underlying PIRP
is required to be present when you call WdfRequestUnmarkCancelable, so
while the ref will keep the handle valid, you need to also maintain
state on the request itself.
d
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bob Kjelgaard
Sent: Tuesday, November 28, 2006 12:56 PM
To: Windows System Software Devs Interest List
Subject: RE: Re:[ntdev] Re:How is synchronization b/w completion/cancel
routine taken care of in kmdf?
>No, I am not sending IRP down to a lower level driver.
Basically these are IRPs that driver sends whenever an event occurs.
It will wait for that event to occur. When driver knows of an event, it
fills the data buffer in IRP and completes the IRP.
While the driver is about to complete the IRP, app might cancel the IRP.
Its this race condition b/w the two events that I was talking about.
This event can occur in the form of an asynchrous event like device
disconnect etc…
Does this still raise concerns?<<
Then this is analogous to the stated race between the DpcForIsr and the
cancel routine, if both can complete an IRP.
Pseudo code is probably not the best medium for this discussion- you can
sketch an idea and it may looks safe, but then change something that
looks insignificant and the whole thing breaks.
One thing to remember is that once you call WdfRequestComplete, the
request handle itself is invalid (UNLESS you have referenced it with
WdfObjectReference somewhere, in which case the original IRP is still
gone, so you have to rely on your context info for everything). In your
code snippet, you don’t show where the handle came from you are
manipulating in the “completion event”. If you’re keeping it in
something like a WDFCOLLECTION, for instance, the collection keeps such
a reference, and you should be OK [if you do all the operations in the
correct order]. But if not, then I’d reference it before making it
cancelable to be sure it was still valid when the completion event arose
(and dereference it afterward).
Other than that, I think the illustration may work (but I’d prefer to
get Doron’s or Jake’s opinion on that- properly handling the
synchronization issues surrounding cancellation is HARD, and I don’t
feel adequate as judge in this matter- I prefer to let KMDF handle as
much of that burden as possible when I’m using it).
Also, no matter what opinion gets rendered on a sketched implementation
plan I’d also run SDV on the real code [I think you can do that, but I
don’t remember if those changes for SDV-KMDF made it into Vista or will
end up in SP1/Server- in the latter case, you might not be able to do
this, yet], if I were in this position.
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
If you store your pending requests on a manual KMDF queue, then they will be
cancellable without you having to do anything. The framework handles
everything for you.
- Dan.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Praveen Kumar
Amritaluru
Sent: Tuesday, November 28, 2006 1:05 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Re:How is synchronization b/w completion/cancel routine
taken care of in kmdf?
No, I am not sending IRP down to a lower level driver. Basically these are
IRPs that driver sends whenever an event occurs. It will wait for that event
to occur. When driver knows of an event, it
fills the data buffer in IRP and completes the IRP.
While the driver is about to complete the IRP, app might cancel the IRP. Its
this race condition b/w the two events that I was talking about.
This event can occur in the form of an asynchrous event like device
disconnect etc…
Does this still raise concerns?
“Bob Kjelgaard” wrote in message
news:xxxxx@ntdev…
This concerns me a bit. Do you mean you are trying to manage the cancelable
state of an IRP you have given to a lower level driver? As I understand it,
you don’t own and shouldn’t attempt to manage an IRP once you’ve sent it to
a target. The target driver will complete or cancel the IRP as it sees fit.
In both cases, you will get your completion routine called and can check the
cancelled state by looking at its completion status. For that matter,
unless you have work to do when the request completes [your code snippet
shows none, but I realize it may not accurately reflect your real
situation], you can use the “Send and forget” option.
You should only use a cancel routine if the IRP is sitting in YOUR driver
for a potentially long time with nothing going on [and you can safely cancel
the operation, of course].
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
Your comment can apply to many different situations. This is, in fact,
exactly what I do in my drivers when I have a request that hasn’t had any
processing yet done on it. If, however, there has been some processing,
cancellation gets really complicated if the request is put back into a
manual queue. I prefer to avoid that, managing cancellation manually using
the rules I outlined in a separate post today.
- Jake Oshins
Windows Kernel Team
“Dan Kyler” wrote in message news:xxxxx@ntdev…
> If you store your pending requests on a manual KMDF queue, then they will
> be
> cancellable without you having to do anything. The framework handles
> everything for you.
>
> - Dan.
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Praveen Kumar
> Amritaluru
> Sent: Tuesday, November 28, 2006 1:05 PM
> To: Windows System Software Devs Interest List
> Subject: Re:[ntdev] Re:How is synchronization b/w completion/cancel
> routine
> taken care of in kmdf?
>
>
> No, I am not sending IRP down to a lower level driver. Basically these are
> IRPs that driver sends whenever an event occurs. It will wait for that
> event
> to occur. When driver knows of an event, it
> fills the data buffer in IRP and completes the IRP.
>
> While the driver is about to complete the IRP, app might cancel the IRP.
> Its
> this race condition b/w the two events that I was talking about.
>
> This event can occur in the form of an asynchrous event like device
> disconnect etc…
>
> Does this still raise concerns?
>
> “Bob Kjelgaard” wrote in message
> news:xxxxx@ntdev…
> This concerns me a bit. Do you mean you are trying to manage the
> cancelable
>
> state of an IRP you have given to a lower level driver? As I understand
> it,
>
> you don’t own and shouldn’t attempt to manage an IRP once you’ve sent it
> to
> a target. The target driver will complete or cancel the IRP as it sees
> fit.
>
> In both cases, you will get your completion routine called and can check
> the
>
> cancelled state by looking at its completion status. For that matter,
> unless you have work to do when the request completes [your code snippet
> shows none, but I realize it may not accurately reflect your real
> situation], you can use the “Send and forget” option.
>
> You should only use a cancel routine if the IRP is sitting in YOUR driver
> for a potentially long time with nothing going on [and you can safely
> cancel
>
> the operation, of course].
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
Agreed, if there’s been something done. But from the OP’s sketchy
description, this sounds like a case of an ioctl that completes as
notification of some event. I have implemented that scenario by immediately
putting the request on a manual queue (no pre-processing), and on the event,
just grab all the requests off the queue and complete them. No muss, no
fuss, no EvtCancelledOnQueue callback, and KMDF takes care of all the
cancellation issues.
- Dan.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Tuesday, November 28, 2006 5:16 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Re:How is synchronization b/w completion/cancel routine
taken care of in kmdf?
Your comment can apply to many different situations. This is, in fact,
exactly what I do in my drivers when I have a request that hasn’t had any
processing yet done on it. If, however, there has been some processing,
cancellation gets really complicated if the request is put back into a
manual queue. I prefer to avoid that, managing cancellation manually using
the rules I outlined in a separate post today.
- Jake Oshins
Windows Kernel Team
“Dan Kyler” wrote in message news:xxxxx@ntdev…
> If you store your pending requests on a manual KMDF queue, then they
> will
> be
> cancellable without you having to do anything. The framework handles
> everything for you.
>
> - Dan.
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Praveen Kumar
> Amritaluru
> Sent: Tuesday, November 28, 2006 1:05 PM
> To: Windows System Software Devs Interest List
> Subject: Re:[ntdev] Re:How is synchronization b/w completion/cancel
> routine
> taken care of in kmdf?
>
>
> No, I am not sending IRP down to a lower level driver. Basically these
> are IRPs that driver sends whenever an event occurs. It will wait for
> that event to occur. When driver knows of an event, it
> fills the data buffer in IRP and completes the IRP.
>
> While the driver is about to complete the IRP, app might cancel the
> IRP.
> Its
> this race condition b/w the two events that I was talking about.
>
> This event can occur in the form of an asynchrous event like device
> disconnect etc…
>
> Does this still raise concerns?
>
> “Bob Kjelgaard” wrote in message
> news:xxxxx@ntdev… This concerns me a bit. Do you mean you are
> trying to manage the cancelable
>
> state of an IRP you have given to a lower level driver? As I
> understand
> it,
>
> you don’t own and shouldn’t attempt to manage an IRP once you’ve sent
> it
> to
> a target. The target driver will complete or cancel the IRP as it sees
> fit.
>
> In both cases, you will get your completion routine called and can
> check
> the
>
> cancelled state by looking at its completion status. For that matter,
> unless you have work to do when the request completes [your code
> snippet shows none, but I realize it may not accurately reflect your
> real situation], you can use the “Send and forget” option.
>
> You should only use a cancel routine if the IRP is sitting in YOUR
> driver for a potentially long time with nothing going on [and you can
> safely cancel
>
> the operation, of course].
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
I’m sorry if yesterday’s posts from me have confused the issue. So for a brief background:
I’m responsible for testing of KMDF IO targets (among other things). As part of this, I have a trio of software device drivers- one is essentially a read/write memory block device, the second device “forwards” or sectors read/write requests to the first, and the third is a bus device that can dynamically spawn and remove instances of the first two. The first two drivers manage devices that are direct, buffered, or neither, and use KMDF synchronsized / unsynchronized dispatch / passive sequential / parallel queues [I do all that at AddDevice time, based upon the bus slot address, so it’s not programmable on the fly]. I use DPCs and work items in various programmable combinations to simulate DMA and hardware setup delays and to force completion at various IRQLs in the first driver, while the second driver’s behavior with respect to sectoring or “forwarding” requests to the first driver, synchronous or asynchronous I/O, start/stop state of its target and “cancel-after” delays is also highly programmable. Both drivers work with power-managed and non-power managed queues, and use idle timers with simulated power up and down delays. The upper driver can be instructed to send to particular target, or to pick one at random from the existing set after completing some number of I/Os. There’s probably more to it than that, but I’m not certain even all of that is relevant, other than to convey complexity of their design.
On top of that sit a pair of test programs- one just exhaustively runs all the reasonably achievable combinations of the programmable and fixed behaviors (so, for instance, a passive-level synchronized parallel queue sectors a request, sends the resulting multiple requests asynchronously to the lower driver, which uses an unsynchronized serial queue and completes the requests at dispatch level is just one of literally tens of thousands of scenarios it runs)- the second is a multithreaded stress app that spans and surprise removes instances of both drivers while simultaneously reprogramming their programmable behaviors and sending I/O to them (it also occasionally kills one of its threads, of course).
In stress, just about any of the programmable behaviors use a common mechanism where they can be given a range of responses (delays, error codes, etc) and weights to use in randomly distributing behavior, so I can attempt to poke harder at behaviors I think might need it. This stress setup runs weekly (along with many others) on all of the OS and computer platforms we support.
My point (other than, perhaps, “really, we do try to test stuff before we ship it”) being that I had to handle cancellation concerns for all of that, so I’ve got some understanding of what happens when you manage it yourself outside the framework (and sometimes I had to, so I certainly realize it has to be done). Since I wasn’t specifically trying to catch cancel issues, I’m certain there are cancellation solution scenarios I left out by design, though, so I hesitate to say I can give someone certain and ironclad advice based on a sketchy sample without more understanding of the environment it’s presented in. On the other hand, I did think I could or should point out a couple of things that might be problems…
Which is why I deferred to Jake and Doron, who I think delivered pithy and concise summations of the rules. In my head, after that exercise the KMDF issues are still tied up with issues of managing the consequences of other design choices. I’m also not entirely convinced I fully solved all of the issues I faced. I am, however, quite confident that if you understand and follow their advice, you are protected against cancellation problems in the framework itself. I also recommend SDV as a faster way to find anything that might otherwise have slipped past you, as opposed to trying to devise your own ad-hoc tests for it.
Hopefully I’m not still leaving a muddled impression after all of that.
PS- I used “forwards” in quotes because all targets are remote, hence reside in a separate stack. I never pass the original request itself on. So the upper driver always has work going on involving driver-created requests to consider when the original request gets cancelled, while the lower driver has to handle issues where the work on a cancelled request is all going on within the driver.