Direct buffereing ...

When using METHOD_IN/OUT_DIRECT, is the Input buffer copied back into the
user’s buffer, or is data returned to the user only in the MDL and only if
OUT is used?


Gary G. Little
Seagate Technologies, LLC

I don’t think there is any copying at all with these methods. The MmXYZ
functions are used to probe/lock/map the user-mode memory to the kernel. The
driver then has its way with it as if it was any other page-locked memory.

There may be games that you can play, but basiclly the input buffer should
be considered to be read-only by the driver. Converse for output buffer.

When using direct I/O you should probably take extra steps in verifying the
user’s buffers. There was a very good talk about this at DevCon under the
topic of driver security. I think the presentation slides are available on
the WInHEC site.

Thomas F. Divine

“Gary G. Little” wrote in message
news:xxxxx@ntdev…
> When using METHOD_IN/OUT_DIRECT, is the Input buffer copied back into the
> user’s buffer, or is data returned to the user only in the MDL and only if
> OUT is used?
>
> –
> Gary G. Little
> Seagate Technologies, LLC
>
>
>

> -----Original Message-----

From: Thomas F. Divine [mailto:xxxxx@hotmail.com]
Sent: Wednesday, April 21, 2004 3:11 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Direct buffereing …

I don’t think there is any copying at all with these methods.
The MmXYZ functions are used to probe/lock/map the user-mode
memory to the kernel. The driver then has its way with it as
if it was any other page-locked memory.

There may be games that you can play, but basiclly the input
buffer should be considered to be read-only by the driver.
Converse for output buffer.

Well sort of conversely for the output buffer. For METHOD_IN_DIRECT the
‘output buffer’ is an additional ‘input buffer’, for METHOD_OUT_DIRECT it is
indeed an ‘output buffer’.

=====================
Mark Roddy

When using direct I/O you should probably take extra steps in
verifying the user’s buffers. There was a very good talk
about this at DevCon under the topic of driver security. I
think the presentation slides are available on the WInHEC site.

Thomas F. Divine

“Gary G. Little” wrote in message
> news:xxxxx@ntdev…
> > When using METHOD_IN/OUT_DIRECT, is the Input buffer copied
> back into
> > the user’s buffer, or is data returned to the user only in
> the MDL and
> > only if OUT is used?
> >
> > –
> > Gary G. Little
> > Seagate Technologies, LLC
> >
> >
> >
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as:
> xxxxx@stratus.com To unsubscribe send a blank email to
> xxxxx@lists.osr.com
>

> When using METHOD_IN/OUT_DIRECT, is the Input buffer copied back into the

user’s buffer, or is data returned to the user only in the MDL and only if
OUT is used?


Gary G. Little
Seagate Technologies, LLC

As I understand it with METHOD_DIRECT the Input buffer is available
for input but is not copied back. Output (from the driver) is only
available through the Output buffer and only with METHOD_IN_DIRECT
(they were named from the point of view of the callee (application)
not the driver. METHOD_OUT_DIRECT is used to send a buffer “in” to the
driver and may still be used in conjunction with the Input parameter.
This makes sense as you may want to input some control information
along with the input/output buffer.

Robert Newton

Gary G. Little wrote:

When using METHOD_IN/OUT_DIRECT, is the Input buffer copied back into the
user’s buffer, or is data returned to the user only in the MDL and only if
OUT is used?

Can’t you WAIT until WDF is released, so we won’t have to deal with this?

Anyhow, the answers were more complicated than the question.

Let me try to answer this the simple way – Unfortunately, it’ll take
three or four paragraphs. In WDM, that’s simple :slight_smile:

The InBuffer is described as for Buffered I/O. So, the user’s input
buffer, if there is one and it’s length is non-zero, is copied into a
system buffer in non-paged pool. The pointer to this input data is
supplied at Irp->AssociatedIrp.SystemBuffer (again, if the InBuffer is
used, and it’s length is non-zero). The user must have READ access to
this data.

If the OutBuffer is provided by the user and it’s length is non-zero,
the OutBuffer is mapped using an MDL, and the MDL pointer is provided at
Irp->MdlAddress.

The xxx in METHOD_xxx_DIRECT identifies which probe and lock function
the I/O Manager users to check the user’s data buffer, and thus the
access the user must have to the memory. For METHOD_IN_DIRECT the user
must have READ access (read FROM the memory, TO the driver). For
METHOD_OUT_DIRECT the user must have WRITE access (write TO the memory,
FROM the driver). That means the METHOD_OUT_DIRECT can be used for both
input OR output (because Windows doesn’t DO write only memory).

One I/O Completion, the special kernel APC for I/O completion *does not*
copy the contents of the system buffer (which was for input only) back
to the user’s InBuffer. Note that this is an inviolable rule for the
InBuffer (even in METHOD_BUFFERED, when the contents of the SystemBuffer
is copied back to the user’s OUT Buffer – unless, of course, his in and
out buffer are the same).

Hope that helps,

Peter
OSR

Thanks to all for confirming what Phil and I were seeing.


Gary G. Little
Seagate Technologies, LLC

“Thomas F. Divine” wrote in message news:xxxxx@ntdev…
> I don’t think there is any copying at all with these methods. The MmXYZ
> functions are used to probe/lock/map the user-mode memory to the kernel.
The
> driver then has its way with it as if it was any other page-locked memory.
>
> There may be games that you can play, but basiclly the input buffer should
> be considered to be read-only by the driver. Converse for output buffer.
>
> When using direct I/O you should probably take extra steps in verifying
the
> user’s buffers. There was a very good talk about this at DevCon under the
> topic of driver security. I think the presentation slides are available on
> the WInHEC site.
>
> Thomas F. Divine
>
> “Gary G. Little” wrote in message
> news:xxxxx@ntdev…
> > When using METHOD_IN/OUT_DIRECT, is the Input buffer copied back into
the
> > user’s buffer, or is data returned to the user only in the MDL and only
if
> > OUT is used?
> >
> > –
> > Gary G. Little
> > Seagate Technologies, LLC
> >
> >
> >
>
>
>

Well yeah I could wait until WDF, but then I’d be sending you a resume
begging for a paycheck. :)) This is deep in the critical path of a project
and no it can’t wait for WDF.

The solution I took, was to use METHOD_NEITHER, since it was the only
solution other than IN/OUT_DIRECT that provided two separate buffer pointers
and which would permit the return of explicit results in a response buffer
provided by the calling application. The only extra work required was to pin
the buffers to memory while in the kernel and then to unlock them before
completing the IRP.

I must admit that DriverWorks made that a fairly simple task to do (just
don’t ask me about DriverWorks and Power management).


Gary G. Little
Seagate Technologies, LLC

“PeterGV” wrote in message news:xxxxx@ntdev…
> Gary G. Little wrote:
>
> > When using METHOD_IN/OUT_DIRECT, is the Input buffer copied back into
the
> > user’s buffer, or is data returned to the user only in the MDL and only
if
> > OUT is used?
> >
>
> Can’t you WAIT until WDF is released, so we won’t have to deal with this?
>
> Anyhow, the answers were more complicated than the question.
>
> Let me try to answer this the simple way – Unfortunately, it’ll take
> three or four paragraphs. In WDM, that’s simple :slight_smile:
>
> The InBuffer is described as for Buffered I/O. So, the user’s input
> buffer, if there is one and it’s length is non-zero, is copied into a
> system buffer in non-paged pool. The pointer to this input data is
> supplied at Irp->AssociatedIrp.SystemBuffer (again, if the InBuffer is
> used, and it’s length is non-zero). The user must have READ access to
> this data.
>
> If the OutBuffer is provided by the user and it’s length is non-zero,
> the OutBuffer is mapped using an MDL, and the MDL pointer is provided at
> Irp->MdlAddress.
>
> The xxx in METHOD_xxx_DIRECT identifies which probe and lock function
> the I/O Manager users to check the user’s data buffer, and thus the
> access the user must have to the memory. For METHOD_IN_DIRECT the user
> must have READ access (read FROM the memory, TO the driver). For
> METHOD_OUT_DIRECT the user must have WRITE access (write TO the memory,
> FROM the driver). That means the METHOD_OUT_DIRECT can be used for both
> input OR output (because Windows doesn’t DO write only memory).
>
> One I/O Completion, the special kernel APC for I/O completion does not
> copy the contents of the system buffer (which was for input only) back
> to the user’s InBuffer. Note that this is an inviolable rule for the
> InBuffer (even in METHOD_BUFFERED, when the contents of the SystemBuffer
> is copied back to the user’s OUT Buffer – unless, of course, his in and
> out buffer are the same).
>
> Hope that helps,
>
> Peter
> OSR
>

Wouldn’t METHOD_OUT_DIRECT have allowed the “return of explicit results
in a response buffer provided by the calling application”? I take it
that you needed three buffers - control in, big-block-of-data and
response out?

You could put the address of the response buffer as a parameter in the
input buffer. Let the I/O system handle your control & data buffers and
just probe-and-lock the response buffer yourself.

Not that I advise anyone to pass pointers between client & driver. it
opens you up to many more attacks and bugs. But if you’re going pass
pointers around, why pass around more of them than you have to?

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, April 22, 2004 9:19 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Direct buffereing …

Well yeah I could wait until WDF, but then I’d be sending you a resume
begging for a paycheck. :)) This is deep in the critical path of a
project and no it can’t wait for WDF.

The solution I took, was to use METHOD_NEITHER, since it was the only
solution other than IN/OUT_DIRECT that provided two separate buffer
pointers and which would permit the return of explicit results in a
response buffer provided by the calling application. The only extra work
required was to pin the buffers to memory while in the kernel and then
to unlock them before completing the IRP.

I must admit that DriverWorks made that a fairly simple task to do (just
don’t ask me about DriverWorks and Power management).


Gary G. Little
Seagate Technologies, LLC

“PeterGV” wrote in message news:xxxxx@ntdev…
> Gary G. Little wrote:
>
> > When using METHOD_IN/OUT_DIRECT, is the Input buffer copied back
> > into
the
> > user’s buffer, or is data returned to the user only in the MDL and
> > only
if
> > OUT is used?
> >
>
> Can’t you WAIT until WDF is released, so we won’t have to deal with
this?
>
> Anyhow, the answers were more complicated than the question.
>
> Let me try to answer this the simple way – Unfortunately, it’ll take
> three or four paragraphs. In WDM, that’s simple :slight_smile:
>
> The InBuffer is described as for Buffered I/O. So, the user’s input
> buffer, if there is one and it’s length is non-zero, is copied into a
> system buffer in non-paged pool. The pointer to this input data is
> supplied at Irp->AssociatedIrp.SystemBuffer (again, if the InBuffer is

> used, and it’s length is non-zero). The user must have READ access to

> this data.
>
> If the OutBuffer is provided by the user and it’s length is non-zero,
> the OutBuffer is mapped using an MDL, and the MDL pointer is provided
> at
> Irp->MdlAddress.
>
> The xxx in METHOD_xxx_DIRECT identifies which probe and lock function
> the I/O Manager users to check the user’s data buffer, and thus the
> access the user must have to the memory. For METHOD_IN_DIRECT the
> user must have READ access (read FROM the memory, TO the driver). For

> METHOD_OUT_DIRECT the user must have WRITE access (write TO the
> memory, FROM the driver). That means the METHOD_OUT_DIRECT can be
> used for both input OR output (because Windows doesn’t DO write only
memory).
>
> One I/O Completion, the special kernel APC for I/O completion does
not

> copy the contents of the system buffer (which was for input only)
> back to the user’s InBuffer. Note that this is an inviolable rule for

> the InBuffer (even in METHOD_BUFFERED, when the contents of the
> SystemBuffer is copied back to the user’s OUT Buffer – unless, of
> course, his in and out buffer are the same).
>
> Hope that helps,
>
> Peter
> OSR
>


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Peter,

We needed two buffers,. one in and one out. However, either one could
potentially be very large, though the base case was that they were both
small, and both large would never occur.

If METHOD_IN/OUT_DIRECT were implemented in a logical, intuitive fashion
(One buffer locked for access in one direction, the other buffered in the
other direction), we could have simply chosen whichever one resulted in the
larger buffer being locked for transfer in the direction we wanted, and
moved on. We even tried that, without a careful reading of the IOCTL
definition documentation, which does, in fact, say that it doesn’t work that
way.

From my reading and poking through the SP1 source, it appears that there is
no IOCTL definition that results in a direct output buffer coupled with a
buffered input buffer. In the case of METHOD_IN_DIRECT, which is the method
to use for sending a large buffer to the driver from UM, the “other” buffer,
passed in the lpInBuffer parameter, is also an input buffer to the driver,
and anything the driver puts in it is not copied back to the UM buffer. I
think we are too far down the road to change it, but is there any way we
could have accomplished this?

So since, as far as I could tell, we would have to use METHOD_NEITHER for
that case or pay the copy overhead for using METHOD_BUFFERED, the very
simple answer was to use METHOD_NEITHER and lock them both all the time. It
dramtically simplified the code, as there are no runtime decisions to make
in the app or the driver.

Phil

Philip D. Barila Windows DDK MVP
Seagate Technology, LLC
(720) 684-1842
As if I need to say it: Not speaking for Seagate.

“Peter Wieland” wrote in message
news:xxxxx@ntdev…
Wouldn’t METHOD_OUT_DIRECT have allowed the “return of explicit results
in a response buffer provided by the calling application”? I take it
that you needed three buffers - control in, big-block-of-data and
response out?

You could put the address of the response buffer as a parameter in the
input buffer. Let the I/O system handle your control & data buffers and
just probe-and-lock the response buffer yourself.

Not that I advise anyone to pass pointers between client & driver. it
opens you up to many more attacks and bugs. But if you’re going pass
pointers around, why pass around more of them than you have to?

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, April 22, 2004 9:19 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Direct buffereing …

Well yeah I could wait until WDF, but then I’d be sending you a resume
begging for a paycheck. :)) This is deep in the critical path of a
project and no it can’t wait for WDF.

The solution I took, was to use METHOD_NEITHER, since it was the only
solution other than IN/OUT_DIRECT that provided two separate buffer
pointers and which would permit the return of explicit results in a
response buffer provided by the calling application. The only extra work
required was to pin the buffers to memory while in the kernel and then
to unlock them before completing the IRP.

I must admit that DriverWorks made that a fairly simple task to do (just
don’t ask me about DriverWorks and Power management).


Gary G. Little
Seagate Technologies, LLC

“PeterGV” wrote in message news:xxxxx@ntdev…
> Gary G. Little wrote:
>
> > When using METHOD_IN/OUT_DIRECT, is the Input buffer copied back
> > into
the
> > user’s buffer, or is data returned to the user only in the MDL and
> > only
if
> > OUT is used?
> >
>
> Can’t you WAIT until WDF is released, so we won’t have to deal with
this?
>
> Anyhow, the answers were more complicated than the question.
>
> Let me try to answer this the simple way – Unfortunately, it’ll take
> three or four paragraphs. In WDM, that’s simple :slight_smile:
>
> The InBuffer is described as for Buffered I/O. So, the user’s input
> buffer, if there is one and it’s length is non-zero, is copied into a
> system buffer in non-paged pool. The pointer to this input data is
> supplied at Irp->AssociatedIrp.SystemBuffer (again, if the InBuffer is

> used, and it’s length is non-zero). The user must have READ access to

> this data.
>
> If the OutBuffer is provided by the user and it’s length is non-zero,
> the OutBuffer is mapped using an MDL, and the MDL pointer is provided
> at
> Irp->MdlAddress.
>
> The xxx in METHOD_xxx_DIRECT identifies which probe and lock function
> the I/O Manager users to check the user’s data buffer, and thus the
> access the user must have to the memory. For METHOD_IN_DIRECT the
> user must have READ access (read FROM the memory, TO the driver). For

> METHOD_OUT_DIRECT the user must have WRITE access (write TO the
> memory, FROM the driver). That means the METHOD_OUT_DIRECT can be
> used for both input OR output (because Windows doesn’t DO write only
memory).
>
> One I/O Completion, the special kernel APC for I/O completion does
not

> copy the contents of the system buffer (which was for input only)
> back to the user’s InBuffer. Note that this is an inviolable rule for

> the InBuffer (even in METHOD_BUFFERED, when the contents of the
> SystemBuffer is copied back to the user’s OUT Buffer – unless, of
> course, his in and out buffer are the same).
>
> Hope that helps,
>
> Peter
> OSR
>


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com