Is It Efficient?

Amit_Kulkarni-2 · July 19, 2007, 2:34am

Hi All,

I have a user mode buffer of 10 MB. I want to send it to kernel mode through IOCTL, where I make a copy of it. I am freeing that kernel mode copy after some time.

Is this operation efficient, if I use Buffered or Direct method?

Or is it efficient if I break the buffer in smaller peaces?

Thanks & Regards,
Amit.

Gianluca_Varenni · July 19, 2007, 2:55am

How often do you need to pass this 10MB buffer to the kernel driver? In this
case I would probably opt for DIRECT_IO, but it heavily depends on how often
you need to copy the buffer.

Have a nice day
GV

–
Gianluca Varenni, Windows DDK MVP

CACE Technologies
http://www.cacetech.com

----- Original Message -----
From:
To: “Windows System Software Devs Interest List”
Sent: Wednesday, July 18, 2007 11:35 PM
Subject: [ntdev] Is It Efficient?

> Hi All,
>
> I have a user mode buffer of 10 MB. I want to send it to kernel mode
> through IOCTL, where I make a copy of it. I am freeing that kernel mode
> copy after some time.
>
> Is this operation efficient, if I use Buffered or Direct method?
>
> Or is it efficient if I break the buffer in smaller peaces?
>
> Thanks & Regards,
> Amit.
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

Amit_Kulkarni-2 · July 19, 2007, 4:56am

I have found that there is no any difference between Buffered and Direct method for input buffer as Io manager allocates a system buffer and copy input to it and send the address of system buffer with IRP.

Then why Direct method is prefferable?

zhao_changkun · July 19, 2007, 8:35am

when you use IOCTL, you can choose the
METHOD_BUFFERED,METHOD_IN_DIRECT,METHOD_OUT_DIRECT,and METHOD_NEITHER.
with both the METHOD_IN_DIRECT and METHOD_OUT_DIRECT , the IO manager
provides the copy buffer (irp->SystemBuffer) for the input data and mdl for
the output data buffer.so for you
, METHOD_BUFFERED,METHOD_IN_DIRECT,METHOD_OUT_DIRECT are same.
you can choose METHOD_NEITHER for efficiency.

you confuse the ReadFile ,WriteFile with IOCTL. if you use ReadFile or
WriteFile,you can choose DO_DIRECT_IO,DO_BUFFERED_IO. if you choose
DO_DIRECT_IO,when use WriteFile,the IO manage provide for the mdl for user
buffer. it is efficient than DO_BUFFERED_IO.
danny

Gary_Little-2 · July 19, 2007, 10:43am

I would label it as inefficient. Yes, BUFFERED_DIRECT_IO does do a copy on
the input buffer, but you should be putting the behemoth buffer in the
output side where it is converted to an MDL before your driver gets it. Use
the Input buffer for command/control of the entire operation. Jamie Hanrahan
once described BUFFERED_DIRECT as a command buffer in the input buffer, and
the output buffer as your data buffer. Keep your command buffer short, much
over 256 bytes and your doing more preaching than programming.

–
The personal opinion of
Gary G. Little

wrote in message news:xxxxx@ntdev…
> Hi All,
>
> I have a user mode buffer of 10 MB. I want to send it to kernel mode
> through IOCTL, where I make a copy of it. I am freeing that kernel mode
> copy after some time.
>
> Is this operation efficient, if I use Buffered or Direct method?
>
> Or is it efficient if I break the buffer in smaller peaces?
>
> Thanks & Regards,
> Amit.
>
>

David_R_Cattley · July 19, 2007, 10:56am

To take advantage of DIRECT_IO for IOCTL, the buffer you wish to be ‘direct’
must be specified as the OutputBuffer parameter. The difference between
the two DIRECT methods is in what validation is done.

You are correct in observing that the InputBuffer is always buffered in this
case. The suggestion (probably) assumed you were aware that the
OutputBuffer can be used as an Input, Output, or Both when using DIRECT IO.

-dave

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@yahoo.com
Sent: Thursday, July 19, 2007 4:59 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Is It Efficient?

I have found that there is no any difference between Buffered and Direct
method for input buffer as Io manager allocates a system buffer and copy
input to it and send the address of system buffer with IRP.

Then why Direct method is prefferable?

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

anton_bassov · July 19, 2007, 12:13pm

> I have found that there is no any difference between Buffered and Direct method

for input buffer as Io manager allocates a system buffer and copy input to it
and send the address of system buffer with IRP.

Then why Direct method is prefferable?

I know this is a bit confusing…

Basically, there are 2 different things:

IO model for IRP_MJ_READ and IMP_MJ_WRITE requests, i.e. respectively ReadFile() and WriteFile() calls. On these calls IO method is defined by device->Flags. If DO_DIRECT_IO is specified, irp->MdlAddress describes a user buffer to read from/write to. If DO_BUFFERED_IO is specified, irp->AssociatedIrp.SystemBuffer describes a user buffer to read from/write to
IO model for IRP_MJ_DEVICE_CONTROL and IRP_MJ_INTERNAL_DEVICE_CONTROL. For these requests IO method is defined by IOCTL code. If METHOD_BUFFERED is specified, irp->AssociatedIrp.SystemBuffer is used for both input and output - the system copies data to it from the user input buffer before calling a Dispatch() routine, and, after Dispatch() routine returns, it copies data from it to the user output buffer. If METHOD_NEITHER is specified, StackLocation-> Parameters.DeviceIoControl.Type3InputBuffer and Irp->UserBuffer represent virtual addresses (as known in the caller’s address space) of respectively input and output buffers that are specified in calls to DeviceIoControl() and IoBuildDeviceIoControlRequest().

If METHOD_IN_DIRECT or METHOD_OUT_DIRECT is specified, things get a bit more complicated.
Irp->MdlAddress represents an output buffer that is specified in calls to DeviceIoControl() and IoBuildDeviceIoControlRequest(). It means that, in practical terms , an output buffer that is specified in DeviceIoControl() and IoBuildDeviceIoControlRequest() calls may be used for both input and output by a driver. Irp ->AssociatedIrp.SystemBuffer represents an input buffer that is specified in calls to DeviceIoControl() and IoBuildDeviceIoControlRequest() - the system copies data to it from the user input buffer before calling a Dispatch() routine, i.e. acts the same way it does when METHOD_BUFFERED is specified. However, unlike METHOD_BUFFERED, it does not copy data from it after a Dispatch() routine returns. Therefore, Irp ->AssociatedIrp.SystemBuffer can be used by a driver only for input and not for output if METHOD_IN_DIRECT or METHOD_OUT_DIRECT is specified.

In other words, from the system’s perspective there is no difference between METHOD_IN_DIRECT or METHOD_OUT_DIRECT - this distinction may be meaningfull only for a driver itself.

Anton Bassov

Gianluca_Varenni · July 19, 2007, 12:33pm

I actually suggested DIRECT_IO because it was midnight and I wasn’t paying
too much attention, sorry.

In any case, my question still holds: you can use DIRECT_IO and use the
output buffer as input buffer as well. It all depends how often you need to
write data to the driver.

Have a nice day
GV

----- Original Message -----
From: “David R. Cattley”
To: “Windows System Software Devs Interest List”
Sent: Thursday, July 19, 2007 7:55 AM
Subject: RE: [ntdev] Is It Efficient?

> To take advantage of DIRECT_IO for IOCTL, the buffer you wish to be
> ‘direct’
> must be specified as the OutputBuffer parameter. The difference between
> the two DIRECT methods is in what validation is done.
>
> You are correct in observing that the InputBuffer is always buffered in
> this
> case. The suggestion (probably) assumed you were aware that the
> OutputBuffer can be used as an Input, Output, or Both when using DIRECT
> IO.
>
> -dave
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of
> xxxxx@yahoo.com
> Sent: Thursday, July 19, 2007 4:59 AM
> To: Windows System Software Devs Interest List
> Subject: RE:[ntdev] Is It Efficient?
>
> I have found that there is no any difference between Buffered and Direct
> method for input buffer as Io manager allocates a system buffer and copy
> input to it and send the address of system buffer with IRP.
>
> Then why Direct method is prefferable?
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

Gianluca_Varenni · July 19, 2007, 12:36pm

Unless there is a very specific need for that (and it’s mostly extreme cases), I would definitely avoid METHOD_NEITHER. Having all the necessary checks needed to validate the pointers passed from user level is not trivial, if you use BUFFERED or DIRECT the I/O manager does the checks for you.

Have a nice day
GV

–
Gianluca Varenni, Windows DDK MVP

CACE Technologies
http://www.cacetech.com
----- Original Message -----
From: danny zhao
To: Windows System Software Devs Interest List
Sent: Thursday, July 19, 2007 5:35 AM
Subject: Re: [ntdev] Is It Efficient?

when you use IOCTL, you can choose the METHOD_BUFFERED,METHOD_IN_DIRECT,METHOD_OUT_DIRECT,and METHOD_NEITHER.
with both the METHOD_IN_DIRECT and METHOD_OUT_DIRECT , the IO manager provides the copy buffer (irp->SystemBuffer) for the input data and mdl for the output data buffer.so for you , METHOD_BUFFERED,METHOD_IN_DIRECT,METHOD_OUT_DIRECT are same.
you can choose METHOD_NEITHER for efficiency.

you confuse the ReadFile ,WriteFile with IOCTL. if you use ReadFile or WriteFile,you can choose DO_DIRECT_IO,DO_BUFFERED_IO. if you choose DO_DIRECT_IO,when use WriteFile,the IO manage provide for the mdl for user buffer. it is efficient than DO_BUFFERED_IO.
danny

— Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256 To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Maxim_S_Shatskih · July 22, 2007, 4:05pm

Direct method does no copies. Instead, it locks the user app’s pages in
memory and creates an object called MDL which describes them.

With a MDL, you can:
a) run DMA over the buffer described by the MDL
b) create a sub-MDL which will describe the part of the buffer
c) map the buffer to the kernel addresses, so the same buffer will be
mapped for both the driver and the user app.

After doing c), you can do your own copy, but this is not mandatory.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

wrote in message news:xxxxx@ntdev…
> I have found that there is no any difference between Buffered and Direct
method for input buffer as Io manager allocates a system buffer and copy input
to it and send the address of system buffer with IRP.
>
> Then why Direct method is prefferable?
>

Maxim_S_Shatskih · July 22, 2007, 4:09pm

> In other words, from the system’s perspective there is no difference between

METHOD_IN_DIRECT or METHOD_OUT_DIRECT - this distinction may be
meaningfull only for a driver itself.

With one of them, the driver cannot update the data at ->MdlAddress, it can
only read it. With another (forgot what) - the driver can update this data.

With the first one, the IOCTL is “long write with short additional info”. Yes,
2 input buffers - output buffer is actually also input one.

With the second one, the IOCTL is “get long direct response for short buffered
command”.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com