Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results
The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.
Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/
Upcoming OSR Seminars | ||
---|---|---|
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead! | ||
Kernel Debugging | 9-13 Sept 2024 | Live, Online |
Developing Minifilters | 15-19 July 2024 | Live, Online |
Internals & Software Drivers | 11-15 Mar 2024 | Live, Online |
Writing WDF Drivers | 20-24 May 2024 | Live, Online |
Comments
>
> I have a controldevice in my driver with which I communicate to user mode.
> Now I want to create a queue in the driver and my devices should put there information asynchronously. An service on the user mode should now look periodically in the driver if information are in the queue.
> Can anybody say which example in the wdk example shows me the path to such an solution?
That's the wrong model. You should have your user-mode app submit an ioctl that gets pended in the driver. When new information is available, the driver completes that ioctl. This is called "inverted call". In fact, you can save one step by just returning the data when you complete the ioctl.
—
Tim Roberts, [email protected]
Providenza & Boekelheide, Inc.
Tim Roberts, [email protected]
Software Wizard Emeritus
d
-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of [email protected]
Sent: Sunday, September 9, 2018 4:37 PM
To: Windows System Software Devs Interest List <[email protected]>
Subject: Re: [ntdev] asynchronous queue
On Sep 9, 2018, at 4:25 AM, [email protected] <[email protected]> wrote:
>
> I have a controldevice in my driver with which I communicate to user mode.
> Now I want to create a queue in the driver and my devices should put there information asynchronously. An service on the user mode should now look periodically in the driver if information are in the queue.
> Can anybody say which example in the wdk example shows me the path to such an solution?
That's the wrong model. You should have your user-mode app submit an ioctl that gets pended in the driver. When new information is available, the driver completes that ioctl. This is called "inverted call". In fact, you can save one step by just returning the data when you complete the ioctl.
—
Tim Roberts, [email protected]
Providenza & Boekelheide, Inc.
---
NTDEV is sponsored by OSR
Visit the list online at: <https://na01.safelinks.protection.outlook.com/?url=http://www.osronline.com/showlists.cfm?list=ntdev&data=02|01|[email protected]|d57abe28f3eb4008371208d616ad2154|72f988bf86f141af91ab2d7cd011db47|1|0|636721330177632027&sdata=E1T348szFbaT/5FuR6pBM5OIim+dqyBJkR9FUguLdq8=&reserved=0>
MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at <https://na01.safelinks.protection.outlook.com/?url=http://www.osr.com/seminars&data=02|01|[email protected]|d57abe28f3eb4008371208d616ad2154|72f988bf86f141af91ab2d7cd011db47|1|0|636721330177632027&sdata=9P8iOMGf1lG4ksYAcgb0oM5j0GG3CTyxjCLKif4JrY0=&reserved=0>
To unsubscribe, visit the List Server section of OSR Online at <https://na01.safelinks.protection.outlook.com/?url=http://www.osronline.com/page.cfm?name=ListServer&data=02|01|[email protected]|d57abe28f3eb4008371208d616ad2154|72f988bf86f141af91ab2d7cd011db47|1|0|636721330177632027&sdata=JHorx2IrZtVMQWkKkPgzA42+wHpOlubGq6aGNlWgLN0=&reserved=0>
to be notified that there is data unless you poll which in today's systems
is a pretty poor approach. Also, you have the challenges of sharing memory
from kernel to user space, which can be many.
Inverted call is the model that is well proven.
Don Burn
Windows Driver Consulting
Website: http://www.windrvr.com
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of
[email protected]
Sent: Monday, September 10, 2018 10:58 AM
To: Windows System Software Devs Interest List <[email protected]>
Subject: RE:[ntdev] asynchronous queue
What speaks against the solution that I put the data in a ring buffer or
list in the driver and the user app gets the data directly from that buffer
without generating an request and this completion things?
---
NTDEV is sponsored by OSR
Visit the list online at:
<http://www.osronline.com/showlists.cfm?list=ntdev>
MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
drivers!
Details at <http://www.osr.com/seminars>
To unsubscribe, visit the List Server section of OSR Online at
<http://www.osronline.com/page.cfm?name=ListServer>
https://www.osr.com/nt-insider/2013-issue1/inverted-call-model-kmdf/
Don Burn
Windows Driver Consulting
Website: http://www.windrvr.com
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of
[email protected]
Sent: Monday, September 10, 2018 11:41 AM
To: Windows System Software Devs Interest List <[email protected]>
Subject: RE:[ntdev] asynchronous queue
Yes, this polling isn't a nice solution. In my case I have very less data
but I will look how this inverted call must be implemented. Which example
from the sdk shows this inverted call approach?
---
NTDEV is sponsored by OSR
Visit the list online at:
<http://www.osronline.com/showlists.cfm?list=ntdev>
MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
drivers!
Details at <http://www.osr.com/seminars>
To unsubscribe, visit the List Server section of OSR Online at
<http://www.osronline.com/page.cfm?name=ListServer>
> What speaks against the solution that I put the data in a ring buffer or list in the driver and the user app gets the data directly from that buffer without generating an request and this completion things?
How would your user app know that data was available? You'd either have
to poll periodically, which is wasteful and not timely, or you'd need an
event, which is exactly what having a pending ioctl does.
Plus, when you're caching in the driver, you need to figure out how to
deal with buffer overflows. You can't cache an infinite amount of data.
--
Tim Roberts, [email protected]
Providenza & Boekelheide, Inc.
Tim Roberts, [email protected]
Software Wizard Emeritus
If I understood the concept correctly then it is possible that if data are in my list or ring buffer on the driver I can immediately get that data back. If this is not the case I make a pending request and wait till something is completed.
My requirement says that I should as fast as possible get the data from the driver queue. In this case I let always a pending request in the driver open. Is this right?
complete the request. Basically have the number of requests that would
handle most flows, and then have a ring buffer or other mechanism for
exceptional circumstances.
Don Burn
Windows Driver Consulting
Website: http://www.windrvr.com
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of
[email protected]
Sent: Monday, September 10, 2018 2:35 PM
To: Windows System Software Devs Interest List <[email protected]>
Subject: RE:[ntdev] asynchronous queue
Ah, very nice that model. I have one question.
If I understood the concept correctly then it is possible that if data are
in my list or ring buffer on the driver I can immediately get that data
back. If this is not the case I make a pending request and wait till
something is completed.
My requirement says that I should as fast as possible get the data from the
driver queue. In this case I let always a pending request in the driver
open. Is this right?
---
NTDEV is sponsored by OSR
Visit the list online at:
<http://www.osronline.com/showlists.cfm?list=ntdev>
MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
drivers!
Details at <http://www.osr.com/seminars>
To unsubscribe, visit the List Server section of OSR Online at
<http://www.osronline.com/page.cfm?name=ListServer>
> Ah, very nice that model. I have one question.
> If I understood the concept correctly then it is possible that if data are in my list or ring buffer on the driver I can immediately get that data back. If this is not the case I make a pending request and wait till something is completed.
Well, the application just submits a read request (or an equivalent
ioctl). If the driver has data waiting, it copies the data and
completes the request immediately. Otherwise, it returns, leaving the
request pending. Later, when data arrives, it pops the next waiting
request and completes it.
> My requirement says that I should as fast as possible get the data from the driver queue. In this case I let always a pending request in the driver open. Is this right?
Right. Or even submit multiple requests, so there's always one waiting
if data comes in quickly. That's how streaming audio/video drivers
handle it. In addition, if you submit your read requests before
triggering the device, you may be able to eliminate the ring buffer
completely. That simplifies the processing and eliminates strange
corner cases.
--
Tim Roberts, [email protected]
Providenza & Boekelheide, Inc.
Tim Roberts, [email protected]
Software Wizard Emeritus
What speaks against the solution that I put the data in a ring buffer or list in the driver and the user app gets the data directly from that buffer without generating an request and this completion things?
</quote>
The whole thing can be described with a single word "poll", which is generally taken as yet another 4-letter word by the kernel-mode developers...
There are some situations when using this approach may be reasonable. For example, consider the scenario when the device produces large amounts of data in large bursts that occur from time to time, while being generally inactive most of the time. In sch case you may, indeed, want to wait a bit after having been notified by a driver just in order to make sure that you read as much data as possible, rather than reading data from the buffer straight away upon receiving notification (or,
in other words, to implement something that is known as select()/poll() semantics on UNIX-like systems).
However, even with this approach you still need asynch notifications from the driver.
If you want to implement polling in its classical sense (i.e. to check the target buffer periodically), I can assure you that this approach is just profoundly dumb....
Anton Bassov
I think your use case is nicely shown in the avscan sample where the driver
simply sends "scan requests" to the user-mode service and the user-mode
service worker threads process these async.
The sample is quite big but looking at the user-mode userscan.c should tell
you enough whether this is a viable option for you.
PS: You do not need to be (mini)filtering anything in your driver in order
to use the communication ports. Just register with the filter manager just
like the nullFilter and then just use the ports in your driver.
Cheers,
Gabriel
On Tue, Sep 11, 2018 at 1:34 AM [email protected] <
[email protected]> wrote:
>
>
>
> What speaks against the solution that I put the data in a ring buffer or
> list in the driver and the user app gets the data directly from that buffer
> without generating an request and this completion things?
>
>
>
> The whole thing can be described with a single word "poll", which is
> generally taken as yet another 4-letter word by the kernel-mode
> developers...
>
> There are some situations when using this approach may be reasonable. For
> example, consider the scenario when the device produces large amounts of
> data in large bursts that occur from time to time, while being generally
> inactive most of the time. In sch case you may, indeed, want to wait a bit
> after having been notified by a driver just in order to make sure that you
> read as much data as possible, rather than reading data from the buffer
> straight away upon receiving notification (or,
> in other words, to implement something that is known as select()/poll()
> semantics on UNIX-like systems).
>
>
>
> However, even with this approach you still need asynch notifications from
> the driver.
> If you want to implement polling in its classical sense (i.e. to check the
> target buffer periodically), I can assure you that this approach is just
> profoundly dumb....
>
>
>
> Anton Bassov
>
> ---
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
>
--
Bercea. G.
Cheers,
Gabriel
not-empty/not-full generally satisfies the objectives.
Mark Roddy
On Mon, Sep 10, 2018 at 7:33 PM [email protected] <
[email protected]> wrote:
>
>
>
> What speaks against the solution that I put the data in a ring buffer or
> list in the driver and the user app gets the data directly from that buffer
> without generating an request and this completion things?
>
>
>
> The whole thing can be described with a single word "poll", which is
> generally taken as yet another 4-letter word by the kernel-mode
> developers...
>
> There are some situations when using this approach may be reasonable. For
> example, consider the scenario when the device produces large amounts of
> data in large bursts that occur from time to time, while being generally
> inactive most of the time. In sch case you may, indeed, want to wait a bit
> after having been notified by a driver just in order to make sure that you
> read as much data as possible, rather than reading data from the buffer
> straight away upon receiving notification (or,
> in other words, to implement something that is known as select()/poll()
> semantics on UNIX-like systems).
>
>
>
> However, even with this approach you still need asynch notifications from
> the driver.
> If you want to implement polling in its classical sense (i.e. to check the
> target buffer periodically), I can assure you that this approach is just
> profoundly dumb....
>
>
>
> Anton Bassov
>
> ---
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
>
would also consider using the minifilter's communication ports for this.
</quote>
While that WOULD work, Minifilter Communication Ports are really just a specific implementation of Inverted Call. Unless you're actually writing a Minifilter, I'd suggest you NOT use Minifilter Communication Ports... Nobody who maintains your driver after you, except perhaps Mr. Bercea, is going to expect to see Minifilter Communication Ports used in a device driver. When they DO see them, its going to cause them to stop and wonder what they're missing... "Is this driver a Minifilter?"
Think more of the people who come AFTER you, who need to maintain your code, than merely what's expedient for you today. Please.
Peter
OSR
@OSRDrivers
Peter Viscarola
OSR
@OSRDrivers
The same efficiency can be applied in both Km and UM code. I know I have in my own private library of code that I have been using in both for about two decades now
Sent from Mail for Windows 10
________________________________
From: [email protected] on behalf of [email protected]
Sent: Monday, September 10, 2018 7:33:55 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] asynchronous queue
What speaks against the solution that I put the data in a ring buffer or list in the driver and the user app gets the data directly from that buffer without generating an request and this completion things?
The whole thing can be described with a single word "poll", which is generally taken as yet another 4-letter word by the kernel-mode developers...
There are some situations when using this approach may be reasonable. For example, consider the scenario when the device produces large amounts of data in large bursts that occur from time to time, while being generally inactive most of the time. In sch case you may, indeed, want to wait a bit after having been notified by a driver just in order to make sure that you read as much data as possible, rather than reading data from the buffer straight away upon receiving notification (or,
in other words, to implement something that is known as select()/poll() semantics on UNIX-like systems).
However, even with this approach you still need asynch notifications from the driver.
If you want to implement polling in its classical sense (i.e. to check the target buffer periodically), I can assure you that this approach is just profoundly dumb....
Anton Bassov
---
NTDEV is sponsored by OSR
Visit the list online at:
MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at
To unsubscribe, visit the List Server section of OSR Online at
> very high. Most modern NIC / HBA drivers will switch between the two modes depending on traffic
Well, this is not about "poll vs wait", but about "poll vs interrupt" (i.e "poll vs asynch notification)".....
As I had said in my previous post, a combination of these two may be quite useful when the device produces large amounts of data in large bursts that occur from time to time, while being generally inactive most of the time. High-performing NICs and HBAs fall into this class of devices, so that dealing with them may be based upon this approach. Linux NAPI ("new" network API) is the very first example that gets into my head.
However, please note that this approach is not necessarily going to suit everyone.
The fact that it is beneficial for a high-performing device that produces dozens of gigabytes of data per second does not necessarily imply that it may be suitable for, say, a kbd, a mouse,or any other low-datarate device. In fact, most devices would be better off if they apply "process upon interrupt
occurence" principle, rather than combining interrupt mode with polling...
Anton Bassov