Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

asynchronous queue

makrurisan_makkelnmakrurisan_makkeln Member - All Emails Posts: 99
I have a controldevice in my driver with which I communicate to user mode.
Now I want to create a queue in the driver and my devices should put there information asynchronously. An service on the user mode should now look periodically in the driver if information are in the queue.
Can anybody say which example in the wdk example shows me the path to such an solution?

Comments

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,720
    On Sep 9, 2018, at 4:25 AM, xxxxx@x-publisher.com <xxxxx@lists.osr.com> wrote:
    >
    > I have a controldevice in my driver with which I communicate to user mode.
    > Now I want to create a queue in the driver and my devices should put there information asynchronously. An service on the user mode should now look periodically in the driver if information are in the queue.
    > Can anybody say which example in the wdk example shows me the path to such an solution?

    That's the wrong model. You should have your user-mode app submit an ioctl that gets pended in the driver. When new information is available, the driver completes that ioctl. This is called "inverted call". In fact, you can save one step by just returning the data when you complete the ioctl.

    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Doron_HolanDoron_Holan Member - All Emails Posts: 10,381
    Even when using the inverted call model, the driver should have a ring buffer of data in case there is no pended IOCTL at the time the data is generated

    d

    -----Original Message-----
    From: xxxxx@lists.osr.com <xxxxx@lists.osr.com> On Behalf Of xxxxx@probo.com
    Sent: Sunday, September 9, 2018 4:37 PM
    To: Windows System Software Devs Interest List <xxxxx@lists.osr.com>
    Subject: Re: [ntdev] asynchronous queue

    On Sep 9, 2018, at 4:25 AM, xxxxx@x-publisher.com <xxxxx@lists.osr.com> wrote:
    >
    > I have a controldevice in my driver with which I communicate to user mode.
    > Now I want to create a queue in the driver and my devices should put there information asynchronously. An service on the user mode should now look periodically in the driver if information are in the queue.
    > Can anybody say which example in the wdk example shows me the path to such an solution?

    That's the wrong model. You should have your user-mode app submit an ioctl that gets pended in the driver. When new information is available, the driver completes that ioctl. This is called "inverted call". In fact, you can save one step by just returning the data when you complete the ioctl.

    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.


    ---
    NTDEV is sponsored by OSR

    Visit the list online at: <https://na01.safelinks.protection.outlook.com/?url=http://www.osronline.com/showlists.cfm?list=ntdev&amp;data=02|01|[email protected]|d57abe28f3eb4008371208d616ad2154|72f988bf86f141af91ab2d7cd011db47|1|0|636721330177632027&amp;sdata=E1T348szFbaT/5FuR6pBM5OIim+dqyBJkR9FUguLdq8=&amp;reserved=0&gt;

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at <https://na01.safelinks.protection.outlook.com/?url=http://www.osr.com/seminars&amp;data=02|01|[email protected]|d57abe28f3eb4008371208d616ad2154|72f988bf86f141af91ab2d7cd011db47|1|0|636721330177632027&amp;sdata=9P8iOMGf1lG4ksYAcgb0oM5j0GG3CTyxjCLKif4JrY0=&amp;reserved=0&gt;

    To unsubscribe, visit the List Server section of OSR Online at <https://na01.safelinks.protection.outlook.com/?url=http://www.osronline.com/page.cfm?name=ListServer&amp;data=02|01|[email protected]|d57abe28f3eb4008371208d616ad2154|72f988bf86f141af91ab2d7cd011db47|1|0|636721330177632027&amp;sdata=JHorx2IrZtVMQWkKkPgzA42+wHpOlubGq6aGNlWgLN0=&amp;reserved=0&gt;
  • makrurisan_makkelnmakrurisan_makkeln Member - All Emails Posts: 99
    What speaks against the solution that I put the data in a ring buffer or list in the driver and the user app gets the data directly from that buffer without generating an request and this completion things?
  • Don_BurnDon_Burn Member - All Emails Posts: 1,634
    The problems with this are you are still calling the kernel, you need a call
    to be notified that there is data unless you poll which in today's systems
    is a pretty poor approach. Also, you have the challenges of sharing memory
    from kernel to user space, which can be many.

    Inverted call is the model that is well proven.


    Don Burn
    Windows Driver Consulting
    Website: http://www.windrvr.com



    -----Original Message-----
    From: xxxxx@lists.osr.com
    [mailto:xxxxx@lists.osr.com] On Behalf Of
    xxxxx@x-publisher.com
    Sent: Monday, September 10, 2018 10:58 AM
    To: Windows System Software Devs Interest List <xxxxx@lists.osr.com>
    Subject: RE:[ntdev] asynchronous queue

    What speaks against the solution that I put the data in a ring buffer or
    list in the driver and the user app gets the data directly from that buffer
    without generating an request and this completion things?



    ---
    NTDEV is sponsored by OSR

    Visit the list online at:
    <http://www.osronline.com/showlists.cfm?list=ntdev>;

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
    drivers!
    Details at <http://www.osr.com/seminars>;

    To unsubscribe, visit the List Server section of OSR Online at
    <http://www.osronline.com/page.cfm?name=ListServer>;
  • makrurisan_makkelnmakrurisan_makkeln Member - All Emails Posts: 99
    Yes, this polling isn't a nice solution. In my case I have very less data but I will look how this inverted call must be implemented. Which example from the sdk shows this inverted call approach?
  • Don_BurnDon_Burn Member - All Emails Posts: 1,634
    Take a look at
    https://www.osr.com/nt-insider/2013-issue1/inverted-call-model-kmdf/


    Don Burn
    Windows Driver Consulting
    Website: http://www.windrvr.com




    -----Original Message-----
    From: xxxxx@lists.osr.com
    [mailto:xxxxx@lists.osr.com] On Behalf Of
    xxxxx@x-publisher.com
    Sent: Monday, September 10, 2018 11:41 AM
    To: Windows System Software Devs Interest List <xxxxx@lists.osr.com>
    Subject: RE:[ntdev] asynchronous queue

    Yes, this polling isn't a nice solution. In my case I have very less data
    but I will look how this inverted call must be implemented. Which example
    from the sdk shows this inverted call approach?

    ---
    NTDEV is sponsored by OSR

    Visit the list online at:
    <http://www.osronline.com/showlists.cfm?list=ntdev>;

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
    drivers!
    Details at <http://www.osr.com/seminars>;

    To unsubscribe, visit the List Server section of OSR Online at
    <http://www.osronline.com/page.cfm?name=ListServer>;
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,720
    xxxxx@x-publisher.com wrote:
    > What speaks against the solution that I put the data in a ring buffer or list in the driver and the user app gets the data directly from that buffer without generating an request and this completion things?

    How would your user app know that data was available?  You'd either have
    to poll periodically, which is wasteful and not timely, or you'd need an
    event, which is exactly what having a pending ioctl does.

    Plus, when you're caching in the driver, you need to figure out how to
    deal with buffer overflows.  You can't cache an infinite amount of data.

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • makrurisan_makkelnmakrurisan_makkeln Member - All Emails Posts: 99
    Ah, very nice that model. I have one question.
    If I understood the concept correctly then it is possible that if data are in my list or ring buffer on the driver I can immediately get that data back. If this is not the case I make a pending request and wait till something is completed.
    My requirement says that I should as fast as possible get the data from the driver queue. In this case I let always a pending request in the driver open. Is this right?
  • Don_BurnDon_Burn Member - All Emails Posts: 1,634
    If you want the fastest have more than one request queued, then you can
    complete the request. Basically have the number of requests that would
    handle most flows, and then have a ring buffer or other mechanism for
    exceptional circumstances.


    Don Burn
    Windows Driver Consulting
    Website: http://www.windrvr.com




    -----Original Message-----
    From: xxxxx@lists.osr.com
    [mailto:xxxxx@lists.osr.com] On Behalf Of
    xxxxx@x-publisher.com
    Sent: Monday, September 10, 2018 2:35 PM
    To: Windows System Software Devs Interest List <xxxxx@lists.osr.com>
    Subject: RE:[ntdev] asynchronous queue

    Ah, very nice that model. I have one question.
    If I understood the concept correctly then it is possible that if data are
    in my list or ring buffer on the driver I can immediately get that data
    back. If this is not the case I make a pending request and wait till
    something is completed.
    My requirement says that I should as fast as possible get the data from the
    driver queue. In this case I let always a pending request in the driver
    open. Is this right?

    ---
    NTDEV is sponsored by OSR

    Visit the list online at:
    <http://www.osronline.com/showlists.cfm?list=ntdev>;

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software
    drivers!
    Details at <http://www.osr.com/seminars>;

    To unsubscribe, visit the List Server section of OSR Online at
    <http://www.osronline.com/page.cfm?name=ListServer>;
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,720
    xxxxx@x-publisher.com wrote:
    > Ah, very nice that model. I have one question.
    > If I understood the concept correctly then it is possible that if data are in my list or ring buffer on the driver I can immediately get that data back. If this is not the case I make a pending request and wait till something is completed.

    Well, the application just submits a read request (or an equivalent
    ioctl).  If the driver has data waiting, it copies the data and
    completes the request immediately.  Otherwise, it returns, leaving the
    request pending.  Later, when data arrives, it pops the next waiting
    request and completes it.


    > My requirement says that I should as fast as possible get the data from the driver queue. In this case I let always a pending request in the driver open. Is this right?

    Right.  Or even submit multiple requests, so there's always one waiting
    if data comes in quickly.  That's how streaming audio/video drivers
    handle it.  In addition, if you submit your read requests before
    triggering the device, you may be able to eliminate the ring buffer
    completely.  That simplifies the processing and eliminates strange
    corner cases.
    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • anton_bassovanton_bassov Member Posts: 4,828
    <quote>

    What speaks against the solution that I put the data in a ring buffer or list in the driver and the user app gets the data directly from that buffer without generating an request and this completion things?

    </quote>

    The whole thing can be described with a single word "poll", which is generally taken as yet another 4-letter word by the kernel-mode developers...

    There are some situations when using this approach may be reasonable. For example, consider the scenario when the device produces large amounts of data in large bursts that occur from time to time, while being generally inactive most of the time. In sch case you may, indeed, want to wait a bit after having been notified by a driver just in order to make sure that you read as much data as possible, rather than reading data from the buffer straight away upon receiving notification (or,
    in other words, to implement something that is known as select()/poll() semantics on UNIX-like systems).



    However, even with this approach you still need asynch notifications from the driver.
    If you want to implement polling in its classical sense (i.e. to check the target buffer periodically), I can assure you that this approach is just profoundly dumb....



    Anton Bassov
  • Gabriel_BerceaGabriel_Bercea Member - All Emails Posts: 464
    I would also consider using the minifilter's communication ports for this.
    I think your use case is nicely shown in the avscan sample where the driver
    simply sends "scan requests" to the user-mode service and the user-mode
    service worker threads process these async.
    The sample is quite big but looking at the user-mode userscan.c should tell
    you enough whether this is a viable option for you.

    PS: You do not need to be (mini)filtering anything in your driver in order
    to use the communication ports. Just register with the filter manager just
    like the nullFilter and then just use the ports in your driver.

    Cheers,
    Gabriel


    On Tue, Sep 11, 2018 at 1:34 AM xxxxx@hotmail.com <
    xxxxx@lists.osr.com> wrote:

    >
    >
    >
    > What speaks against the solution that I put the data in a ring buffer or
    > list in the driver and the user app gets the data directly from that buffer
    > without generating an request and this completion things?
    >
    >
    >
    > The whole thing can be described with a single word "poll", which is
    > generally taken as yet another 4-letter word by the kernel-mode
    > developers...
    >
    > There are some situations when using this approach may be reasonable. For
    > example, consider the scenario when the device produces large amounts of
    > data in large bursts that occur from time to time, while being generally
    > inactive most of the time. In sch case you may, indeed, want to wait a bit
    > after having been notified by a driver just in order to make sure that you
    > read as much data as possible, rather than reading data from the buffer
    > straight away upon receiving notification (or,
    > in other words, to implement something that is known as select()/poll()
    > semantics on UNIX-like systems).
    >
    >
    >
    > However, even with this approach you still need asynch notifications from
    > the driver.
    > If you want to implement polling in its classical sense (i.e. to check the
    > target buffer periodically), I can assure you that this approach is just
    > profoundly dumb....
    >
    >
    >
    > Anton Bassov
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: <
    > http://www.osronline.com/showlists.cfm?list=ntdev>;
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >


    --
    Bercea. G.

    Cheers,
    Gabriel

  • Mark_RoddyMark_Roddy Member - All Emails Posts: 4,280
    a "large enough" ringbuffer with signalling on transition to
    not-empty/not-full generally satisfies the objectives.
    Mark Roddy


    On Mon, Sep 10, 2018 at 7:33 PM xxxxx@hotmail.com <
    xxxxx@lists.osr.com> wrote:

    >
    >
    >
    > What speaks against the solution that I put the data in a ring buffer or
    > list in the driver and the user app gets the data directly from that buffer
    > without generating an request and this completion things?
    >
    >
    >
    > The whole thing can be described with a single word "poll", which is
    > generally taken as yet another 4-letter word by the kernel-mode
    > developers...
    >
    > There are some situations when using this approach may be reasonable. For
    > example, consider the scenario when the device produces large amounts of
    > data in large bursts that occur from time to time, while being generally
    > inactive most of the time. In sch case you may, indeed, want to wait a bit
    > after having been notified by a driver just in order to make sure that you
    > read as much data as possible, rather than reading data from the buffer
    > straight away upon receiving notification (or,
    > in other words, to implement something that is known as select()/poll()
    > semantics on UNIX-like systems).
    >
    >
    >
    > However, even with this approach you still need asynch notifications from
    > the driver.
    > If you want to implement polling in its classical sense (i.e. to check the
    > target buffer periodically), I can assure you that this approach is just
    > profoundly dumb....
    >
    >
    >
    > Anton Bassov
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > Visit the list online at: <
    > http://www.osronline.com/showlists.cfm?list=ntdev>;
    >
    > MONTHLY seminars on crash dump analysis, WDF, Windows internals and
    > software drivers!
    > Details at
    >
    > To unsubscribe, visit the List Server section of OSR Online at <
    > http://www.osronline.com/page.cfm?name=ListServer>;
    >
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 6,844
    <quote>
    would also consider using the minifilter's communication ports for this.
    </quote>

    While that WOULD work, Minifilter Communication Ports are really just a specific implementation of Inverted Call. Unless you're actually writing a Minifilter, I'd suggest you NOT use Minifilter Communication Ports... Nobody who maintains your driver after you, except perhaps Mr. Bercea, is going to expect to see Minifilter Communication Ports used in a device driver. When they DO see them, its going to cause them to stop and wonder what they're missing... "Is this driver a Minifilter?"

    Think more of the people who come AFTER you, who need to maintain your code, than merely what's expedient for you today. Please.

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • MBondMBond Member - All Emails Posts: 846
    Note that poll ? wait is proven to be more effective when the frequency of ?events? is very high. Most modern NIC / HBA drivers will switch between the two modes depending on traffic



    The same efficiency can be applied in both Km and UM code. I know I have in my own private library of code that I have been using in both for about two decades now







    Sent from Mail for Windows 10



    ________________________________
    From: xxxxx@lists.osr.com on behalf of xxxxx@hotmail.com
    Sent: Monday, September 10, 2018 7:33:55 PM
    To: Windows System Software Devs Interest List
    Subject: RE:[ntdev] asynchronous queue




    What speaks against the solution that I put the data in a ring buffer or list in the driver and the user app gets the data directly from that buffer without generating an request and this completion things?



    The whole thing can be described with a single word "poll", which is generally taken as yet another 4-letter word by the kernel-mode developers...

    There are some situations when using this approach may be reasonable. For example, consider the scenario when the device produces large amounts of data in large bursts that occur from time to time, while being generally inactive most of the time. In sch case you may, indeed, want to wait a bit after having been notified by a driver just in order to make sure that you read as much data as possible, rather than reading data from the buffer straight away upon receiving notification (or,
    in other words, to implement something that is known as select()/poll() semantics on UNIX-like systems).



    However, even with this approach you still need asynch notifications from the driver.
    If you want to implement polling in its classical sense (i.e. to check the target buffer periodically), I can assure you that this approach is just profoundly dumb....



    Anton Bassov

    ---
    NTDEV is sponsored by OSR

    Visit the list online at:

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at

    To unsubscribe, visit the List Server section of OSR Online at
  • anton_bassovanton_bassov Member Posts: 4,828
    > Note that poll ? wait is proven to be more effective when the frequency of ?events? is
    > very high. Most modern NIC / HBA drivers will switch between the two modes depending on traffic


    Well, this is not about "poll vs wait", but about "poll vs interrupt" (i.e "poll vs asynch notification)".....


    As I had said in my previous post, a combination of these two may be quite useful when the device produces large amounts of data in large bursts that occur from time to time, while being generally inactive most of the time. High-performing NICs and HBAs fall into this class of devices, so that dealing with them may be based upon this approach. Linux NAPI ("new" network API) is the very first example that gets into my head.


    However, please note that this approach is not necessarily going to suit everyone.
    The fact that it is beneficial for a high-performing device that produces dozens of gigabytes of data per second does not necessarily imply that it may be suitable for, say, a kbd, a mouse,or any other low-datarate device. In fact, most devices would be better off if they apply "process upon interrupt
    occurence" principle, rather than combining interrupt mode with polling...


    Anton Bassov
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!