Need some wisdom on (maybe) yet another redesign of our WDFQUEUE usage.
We have identified an issue with our design that uses pre-allocated SGL
buffers for overlapped/queued DMA transfers. As you might recall, we have
a parallel default IO queue, where we build the SGL into the pre-allocated
buffer from the default queue’s callback. The issue is that there is no
way to guarantee that the app won’t exhaust the pre-allocated resources (we
could make the app keep track, but that’s obnoxious, and we don’t want to
require only one instance of the app, and all the other issues that arise
when you make an app responsible for not exceeding a hardware limit), and
once that happens, we have no way to put back any IOs delivered out of the
queue that we can’t service before we can stop the queue.
My first thought is to make the default IO queue a manual queue, instead of
parallel. I can’t find any samples that do that. Is that configuration
supported?
My second thought is to put anything we can’t service it on a manual
holding queue that we only use when we can’t service something. Seems like
the logic to get the requests on that queue back in play while more are
arriving from the app is messy. The other concern is that it seems like
the extra queue adds (potentially) avoidable overhead. Perhaps that belief
in the avoidability of said overhead may be misplaced. And perhaps the
concern about that overhead is also misplaced.
My third thought is to just dump all the requests into the manual queue
directly from the default queue’s callback. The logic is simpler, but the
same concern about avoidable overhead applies.
Any thoughts?
Phil
Philip D. Barila
Seagate Technology LLC
(720) 684-1842
As you later asked, a manual default queue is perhaps the easiest way to address this. That works fine. It’s not demonstrated because it requires a lot more work on the coder’s part, and in a sample (so far, anyway) we use the simplest mechanism that works for that sample.
I have had a little experience [one driver’s worth] with requeuing requests to a manual queue if they require additional servicing. The primary problem I’ve encountered in requeuing requests comes when some work has been done in between. You have to make sure you can undo or recover /ignore (depends upon circumstances) whatever that work was if the request is cancelled while on the latter queue. So, for instance, if you allocate a portion of your resources to a request before it goes on the second queue, you have to return them to your set of available resources if that request gets cancelled before you get the rest of the resources it needed. That’s not unique to KMDF, though- the same problem exists in WDM.
My impression (from reading some of the code, among other things) is that the queue overhead isn’t particularly large other than that it’s correct (so it covers edge and corner cases that may never occur in your own case, but could in someone else’s). But I’ve never attempted a through characterization. Thanks for mentioning it, though- it occurs to me that we ought to start adding this sort of performance measurement into our automated test suites (or at least think about doing so).
Thanks for the confirmation. Although the overhead of having a parallel
default queue with a default callback that simply forwards it to a manual
queue is likely to be small, it doesn’t improve the clarity of the design
or of the code to have it that way, so we’ll just go ahead and make the
default queue manual, call WdfIoQueueReadyNotify to set our callback, and
do most of the work in that callback.
And yes, we already handle returning the resources to the pools of them,
but it’s a good reminder to make sure that gets handled right.
Thanks again,
Phil
Philip D. Barila
Seagate Technology LLC
(720) 684-1842
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bob Kjelgaard
Sent: Tuesday, October 24, 2006 9:43 AM
To: “Windows System Software Devs Interest List”
Subject: RE: [ntdev] KMDF design for concurrent IO handling with minimum
overhead
As you later asked, a manual default queue is perhaps the easiest way to
address this. That works fine. It’s not demonstrated because it requires a
lot more work on the coder’s part, and in a sample (so far, anyway) we use
the simplest mechanism that works for that sample.
I have had a little experience [one driver’s worth] with requeuing requests
to a manual queue if they require additional servicing. The primary problem
I’ve encountered in requeuing requests comes when some work has been done
in between. You have to make sure you can undo or recover /ignore (depends
upon circumstances) whatever that work was if the request is cancelled
while on the latter queue. So, for instance, if you allocate a portion of
your resources to a request before it goes on the second queue, you have to
return them to your set of available resources if that request gets
cancelled before you get the rest of the resources it needed. That’s not
unique to KMDF, though- the same problem exists in WDM.
My impression (from reading some of the code, among other things) is that
the queue overhead isn’t particularly large other than that it’s correct
(so it covers edge and corner cases that may never occur in your own case,
but could in someone else’s). But I’ve never attempted a through
characterization. Thanks for mentioning it, though- it occurs to me that we
ought to start adding this sort of performance measurement into our
automated test suites (or at least think about doing so).