Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.

DeviceIoControl with direct I/O and two buffers

Michael_RolleMichael_Rolle Member - All Emails Posts: 74
Just curious, in case I decide to use this in the future, or for other people to see on this list.

If the user calls DeviceIoControl with both an input and an output buffer, how is the driver to know which is which?

The IRP should have an MDL for the user buffer. If the caller actually has only an input or an output buffer, the MDL would point to that one buffer. If there are two buffers, then I expect a chain of two MDLs. Is this correct, and which MDL is the input and which MDL is the output?

I looked for an answer in Windows Internals, both 6th and 7th editions. If the answer is actually in there, please let me know.

Comments

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,714
    xxxxx@rolle.name wrote:
    > Just curious, in case I decide to use this in the future, or for other people to see on this list.
    >
    > If the user calls DeviceIoControl with both an input and an output buffer, how is the driver to know which is which?
    >
    > The IRP should have an MDL for the user buffer. If the caller actually has only an input or an output buffer, the MDL would point to that one buffer.

    Not so.  It's a little complicated.


    > If there are two buffers, then I expect a chain of two MDLs. Is this correct, and which MDL is the input and which MDL is the output?

    The Microsoft documentation describes this:
       
    https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/buffer-descriptions-for-i-o-control-codes

    The key point to remember is that buffers are not symmetrical.  The
    "input buffer" is always copied into kernel memory (except for
    METHOD_NEITHER, which I will ignore).  If you are using direct I/O, the
    MDL points to the output buffer.  The terms "input buffer" and "output
    buffer" are a little bit misleading; I tend to think of them as "buffer
    #1" and "buffer #2".

    Method             Input       Output

    METHOD_BUFFERED    Copied      Copied
    METHOD_IN_DIRECT   Copied      Mappedw/MDL
    METHOD_OUT_DIRECT  Copied      Mapped w/MDL
    METHOD_NEITHER     Nothing     Nothing

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • >If the user calls DeviceIoControl with both an input and an output buffer, how
    is the driver to know which is which?
    wow...i just know this
    PIO_STACK_LOCATION irp = IoGetCurrentIrpStackLocation(pIrp);//get stack
    ULONG Io_Control_Code = irp->Parameters.DeviceIoControl.IoControlCode;//this is u ControlCode
    ULONG Input_Lenght = irp->Parameters.DeviceIoControl.InputBufferLength;//input buffer lenght
    ULONG Output_Lenght = irp->Parameters.DeviceIoControl.OutputBufferLength;//output buffer lenght
    pIrp->AssociatedIrp.SystemBuffer//input buffer in here
    u dont need know which is which
    if u need user mode input buffer.copy from AssociatedIrp.SystemBuffer
    if u need output to usermode .copy to AssociatedIrp.SystemBuffer
    that is
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,714
    On Feb 28, 2018, at 7:00 PM, processhacker@163.com <xxxxx@lists.osr.com> wrote:
    >>
    > wow...i just know this
    > PIO_STACK_LOCATION irp = IoGetCurrentIrpStackLocation(pIrp);//get stack
    > ULONG Io_Control_Code = irp->Parameters.DeviceIoControl.IoControlCode;//this is u ControlCode
    > ULONG Input_Lenght = irp->Parameters.DeviceIoControl.InputBufferLength;//input buffer lenght
    > ULONG Output_Lenght = irp->Parameters.DeviceIoControl.OutputBufferLength;//output buffer lenght
    > pIrp->AssociatedIrp.SystemBuffer//input buffer in here
    > u dont need know which is which
    > if u need user mode input buffer.copy from AssociatedIrp.SystemBuffer
    > if u need output to usermode .copy to AssociatedIrp.SystemBuffer
    > that is

    That's only true for METHOD_BUFFERED. For METHOD_IN_DIRECT and METHOD_OUT_DIRECT, you absolutely do need to know which is which.

    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Michael_RolleMichael_Rolle Member - All Emails Posts: 74
    Tim, your answer still leaves me confused.

    The Microsoft page you gave says...
    METHOD_IN_DIRECT or METHOD_OUT_DIRECT
    For these transfer types, IRPs supply a pointer to a buffer at Irp->AssociatedIrp.SystemBuffer. This represents the input buffer that is specified in calls to DeviceIoControl and IoBuildDeviceIoControlRequest. The buffer size is specified by Parameters.DeviceIoControl.InputBufferLength in the driver's IO_STACK_LOCATION structure.

    For these transfer types, IRPs also supply a pointer to an MDL at Irp->MdlAddress. This represents the output buffer that is specified in calls to DeviceIoControl and IoBuildDeviceIoControlRequest. However, this buffer can actually be used as either an input buffer or an output buffer, as follows:

    METHOD_IN_DIRECT is specified if the driver that handles the IRP receives data in the buffer when it is called. The MDL describes an input buffer, and specifying METHOD_IN_DIRECT ensures that the executing thread has read-access to the buffer.

    METHOD_OUT_DIRECT is specified if the driver that handles the IRP will write data into the buffer before completing the IRP. The MDL describes an output buffer, and specifying METHOD_OUT_DIRECT ensures that the executing thread has write-access to the buffer.

    For both of these transfer types, Parameters.DeviceIoControl.OutputBufferLength specifies the size of the buffer that is described by the MDL.

    ------------------------

    On the other hand, The Windows 2000 Device Driver Book says that IN is for moving data from the device into user space, and OUT is for the opposite direction. I assume that this is wrong. But I want to confirm that.

    CodeProject has a page about METHOD_XXX and it agrees with you and with the microsoft doc. It stresses the fact that the 'lpInBuffer' buffer is always copied to the driver as though using METHOD_BUFFERED, and the 'lpOutBuffer' is always mapped via the MDL. The only difference between IN and OUT is in having read vs. write access to the lpOutBuffer.

    This is really what you wrote, but I'm putting it into my own words somewhat more verbosely.

    If I've got anything wrong, please post a correction. Otherwise I'm happy with the answer.
  • GUGU Member - All Emails Posts: 8
    @Tim,


    > The key point to remember is that buffers are not symmetrical.
    > The "input buffer" is always copied into kernel memory (except for METHOD_NEITHER, which I will ignore).
    >If you are using direct I/O, the MDL points to the output buffer.? The terms "input buffer" and "output buffer" are a little bit misleading;

    Is your above comment a rule or there could be some exception?
    I am working on handling of IOCTL_SCSI_PASS_THROUGH_DIRECT

    MS doc below seems are saying output buffer are the same as input buffer (Irp->AssociatedIrp.SystemBuffer)
    https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntddscsi/ni-ntddscsi-ioctl_scsi_pass_through_direct

    ======
    Input Buffer Length

    Parameters.DeviceIoControl.InputBufferLength indicates the size, in bytes, of the buffer at Irp->AssociatedIrp.SystemBuffer,


    ....

    Output Buffer

    The port driver returns any request-sense data and the SCSI_PASS_THROUGH_DIRECT structure to the buffer at Irp->AssociatedIrp.SystemBuffer.
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 6,833
    This is more confusing than it is complicated. And this is yet ANOTHER reason why you should be writing your drivers using WDF, which abstracts all this stupidity.

    It takes 30 minutes to explain this in driver class. And lots of people remain confused.

    It might help to understand that the original intent of the IN buffer was for passing parameters and small data structures into the driver. The original intent of the OUT buffer was to be the PAYLOAD buffer, for large data transfers, IN EITHER DIRECTION.

    Now... read this: *IGNORING METHOD_NEITHER in everything below*

    First: IN and OUT for IOCTLs... refer to the direction FROM THE DRIVER'S PERSPECTIVE. A OUT buffer is one to which the driver writes (conventionally, a user read operation). An IN buffer is one from which the driver reads (conventionally, a user's write operation).

    Second: Not all IOCTLs need to use both buffers, or either buffer.

    Third: If there is an IN buffer specified, and its length is non-zero, it uses Buffered I/O. Which means the data is automagically copied by the I/O Manager between the user's buffer within his/her process address space and a system buffer in non-paged pool. This system buffer is automatically allocated by the I/O Manager when the I/O Request is issued, and freed by the I/O Manager when the I/O Request is completed.

    The pointer (kernel virtual address, in the high half of KV address space) to this system buffer is passed to your driver in your IRP at
    Irp->AssociatedIrp.SystemBuffer, and the length of the buffer (the length the user specified on his/her DeviceIoControl call) is at Parameters.DeviceIoControl.InputBufferLength.

    Fourth: If you specify METHOD_BUFFERED for your IOCTL, and if the call to DeviceIoControl specifies an OUT buffer and its length is non-zero, the out data is automagically copied by the I/O Manager between the user's buffer within his/her process address space and a system buffer in non-paged pool. This system buffer is automatically allocated by the I/O Manager when the I/O Request is issued, and freed by the I/O Manager when the I/O Request is completed. The length of this buffer (the length the user specified on his/her DeviceIoControl call) is passed to your driver at Parameters.DeviceIoControl.OutputBufferLength. The pointer to the buffer (again, KVA in high half of the address space) is passed to your driver at Irp->AssociatedIrp.SystemBuffer.

    Continuing for METHOD_BUFFERED... If the user who calls DeviceIoControl specifies BOTH an IN buffer and an OUT buffer (of non-zero length) the I/O Manager allocates a buffer that is the size of the larger of the two (in other words max(InBufferLength, OutBufferLength). This buffer is used for BOTH to pass data to your driver from the user's IN buffer, and for your driver to return data to the user's OUT buffer (this was considered clever and memory efficient in 1989 when this was designed). Thus, your driver must consume any data from the system buffer that was passed to your from the IN buffer, before you start writing data to (that exact same) system buffer that you want returned in the OUT buffer.

    Ugly, confusing, silly by 2017 standards... but "it is what it is"... celebrate the fact that we routinely have systems with more than 16MB of memory (which was the minimum officially supported when NT was released).

    Fifth: METHOD_xxxx_DIRECT only has to deal with the OUT buffer, and when selected, causes the IN buffer to be described using an MDL. The address of this MDL is at Irp->MdlAddress. You can use it to call MmGetSystemAddressForMdlSafe, to map the buffer described by the MDL into (the high half of) Kernel Virtual Address space. The length of the buffer (the length the user specified on his/her DeviceIoControl call) will be at Parameters.DeviceIoControl.OutputBufferLength.

    Now... here's a cute thing: When you specify METHOD_OUT_DIRECT the get WRITE access to the OUT buffer, and you can (in fact) use the OUT buffer in either (or both) directions (read and/or write). When you specify METHOD_IN_DIRECT you get READ access to the OUT buffer... this treating the IN buffer as read-only.

    This 15 year old article might help: http://www.osronline.com/article.cfm?id=92

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,714
    xxxxx@rolle.name wrote:
    > On the other hand, The Windows 2000 Device Driver Book says that IN is for moving data from the device into user space, and OUT is for the opposite direction. I assume that this is wrong. But I want to confirm that.

    Yes, that is wrong.

     
    > CodeProject has a page about METHOD_XXX and it agrees with you and with the microsoft doc. It stresses the fact that the 'lpInBuffer' buffer is always copied to the driver as though using METHOD_BUFFERED, and the 'lpOutBuffer' is always mapped via the MDL. The only difference between IN and OUT is in having read vs. write access to the lpOutBuffer.
    >
    > This is really what you wrote, but I'm putting it into my own words somewhat more verbosely.
    >
    > If I've got anything wrong, please post a correction. Otherwise I'm happy with the answer.

    Nope, I think you have it.

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,714
    xxxxx@gmail.com wrote:
    >
    > Is your above comment a rule or there could be some exception?
    > I am working on handling of IOCTL_SCSI_PASS_THROUGH_DIRECT
    >
    > MS doc below seems are saying output buffer are the same as input buffer (Irp->AssociatedIrp.SystemBuffer)

    Right.  That's not an exception.  IOCTL_SCSI_PASS_THROUGH_DIRECT is
    0x4D014, so based on the low-order 2 bits, it is METHOD_BUFFERED.  Copy
    in, copy out; no mapping.

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 6,833
    (forgive the intrusion)

    <quote>
    Right. That's not an exception...
    </quote>

    Tim... are you posting via email or via NNTP? All about the network here today :-)

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,714
    xxxxx@osr.com wrote:
    > (forgive the intrusion)
    >
    > <quote>
    > Right. That's not an exception...
    > </quote>
    >
    > Tim... are you posting via email or via NNTP? All about the network here today :-)

    I go in by email.  That always seems to work...

    I do rue the demise of NNTP, because a web forum is such a dreadful way
    to do information exchange.

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 6,833
    >I go in by email

    Thank you.

    >I do rue the demise of NNTP

    We still support it... but I don't know how long we're going to be able to continue to do so.

    Sorry for the thread hijack,

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • processhacker@163.com[email protected] Member Posts: 5
    all right all right...
    u said is method, sorry...i never not use METHOD_IN_DIRECT or METHOD_OUT_DIRECT
    but!...just see this
    https://github.com/markjandrews/wrk-v1.2/blob/daeeb598c7bf091b1370e8875626ad0cb3cd1ec0/base/ntos/io/iomgr/internal.c
    in 9038 line:
    //
    // Based on the method that the buffer are being passed, either allocate
    // buffers or build MDLs. Note that in some cases no probing has taken
    // place so the exception handler must catch access violations.
    //

    irp->MdlAddress = (PMDL) NULL;
    irp->AssociatedIrp.SystemBuffer = (PVOID) NULL;

    switch ( method ) {

    case METHOD_BUFFERED:
    .....
    case METHOD_IN_DIRECT:
    case METHOD_OUT_DIRECT:


    and u can flow these function
    R3

    => kernel32!DeviceIoControl

    => ntdll!ZwDeviceIoControlFile

    => ntdll!KiFastSystemCall ///< mov eax,42h

    => sysenter ///< ??R0



    ???R0????DeviceIoControl?????, ?????????sysenter??R0??????



    R0 => ntdll!KiFastSystemCallRet

    => nt!KiFastCallEntry

    => nt!NtDeviceIoControlFile

    => nt!IopXxxControlFile

    => nt!IopSynchronousServiceTail

    => nt!IopfCallDriver

    => LsNtDrv!DisPatchDeviceControl ///< ???????
    so. that is u need?
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,714
    On Mar 1, 2018, at 11:23 PM, processhacker@163.com <xxxxx@lists.osr.com> wrote:
    >
    > all right all right...
    > u said is method, sorry...i never not use METHOD_IN_DIRECT or METHOD_OUT_DIRECT
    > but!...just see this
    > https://github.com/markjandrews/wrk-v1.2/blob/daeeb598c7bf091b1370e8875626ad0cb3cd1ec0/base/ntos/io/iomgr/internal.c
    > in 9038 line:
    ...
    > so. that is u need?

    I have no idea what you're asking.

    And, by the way, it's illegal for that code to be posted in public. The Windows Research Kernel license certainly does not include republication.

    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Alex_GrigAlex_Grig Member Posts: 3,238
    >It might help to understand that the original intent of the IN buffer was for
    passing parameters and small data structures into the driver. The original
    intent of the OUT buffer was to be the PAYLOAD buffer, for large data transfers,
    IN EITHER DIRECTION.

    I believe the original NT 3 IOCTL didn't have a concept of DIRECT or NEITHER. It was only BUFFERED.
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,714
    xxxxx@broadcom.com wrote:
    > It might help to understand that the original intent of the IN buffer was for
    > passing parameters and small data structures into the driver. The original
    > intent of the OUT buffer was to be the PAYLOAD buffer, for large data transfers,
    > IN EITHER DIRECTION.
    >
    > I believe the original NT 3 IOCTL didn't have a concept of DIRECT or NEITHER. It was only BUFFERED.

    Color me dubious.  The <devioctl.h> file from the NT 3.51 DDK is dated
    1992 (which was the original release of NT 3.1), and it includes all
    four methods.

    (I never throw anything away.)

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

  • Don_BurnDon_Burn Member - All Emails Posts: 1,630
    Tim,

    I have the NT 3.5 includes and the methods are in there. If I really want to go crazy I have DVD's of the original driver presentations at Microsoft before NT came out, I suspect they are in there.


    Don Burn
    Windows Driver Consulting
    Website: http://www.windrvr.com



    -----Original Message-----
    From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@probo.com
    Sent: Friday, March 02, 2018 2:29 PM
    To: Windows System Software Devs Interest List <xxxxx@lists.osr.com>
    Subject: Re: [ntdev] DeviceIoControl with direct I/O and two buffers

    xxxxx@broadcom.com wrote:
    > It might help to understand that the original intent of the IN buffer
    > was for passing parameters and small data structures into the driver.
    > The original intent of the OUT buffer was to be the PAYLOAD buffer,
    > for large data transfers, IN EITHER DIRECTION.
    >
    > I believe the original NT 3 IOCTL didn't have a concept of DIRECT or NEITHER. It was only BUFFERED.

    Color me dubious. The <devioctl.h> file from the NT 3.51 DDK is dated
    1992 (which was the original release of NT 3.1), and it includes all four methods.

    (I never throw anything away.)

    --
    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.


    ---
    NTDEV is sponsored by OSR

    Visit the list online at: <http://www.osronline.com/showlists.cfm?list=ntdev>;

    MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
    Details at <http://www.osr.com/seminars>;

    To unsubscribe, visit the List Server section of OSR Online at <http://www.osronline.com/page.cfm?name=ListServer>;
  • Michael_RolleMichael_Rolle Member - All Emails Posts: 74
    Boy, did I stir up a hornet's nest. I think it's good to have this conversation.

    The point I got from all of this was about using WDF. I originally started with some wdm driver code that someone else wrote, and I didn't want to go through the learning curve and the necessary rewrites to move it to WDF, since this driver of mine is just an ancillary tool for myself to do some performance profiling. I'll certainly consider WDF for future work if any. Thanks.

    Oh, and Tim, what's with these ? characters in some of your posts. I just wanted to alert you that they are showing up, at least on my Firefox browser, but not on Thunderbird email.
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 6,833
    <quote>
    what's with these ? characters in some of your posts
    </quote>

    You may thank the "ultra sophisticated" (read: ancient and annoying) forum front-end to the Lyris Manager for that.

    No, we're not going to fix it. We're going to throw it out and get something new. Someday.

    Peter
    OSR
    @OSRDrivers

    Peter Viscarola
    OSR
    @OSRDrivers

  • Tim_RobertsTim_Roberts Member - All Emails Posts: 12,714
    On Mar 2, 2018, at 1:39 PM, xxxxx@rolle.name <xxxxx@lists.osr.com> wrote:
    >
    > Oh, and Tim, what's with these  characters in some of your posts. I just wanted to alert you that they are showing up, at least on my Firefox browser, but not on Thunderbird email.

    That sounds like a personal problem to me.

    Tim Roberts, xxxxx@probo.com
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!