which buffer to use?

OSR_Community_User · March 13, 2002, 2:54am

Which buffer should I use for IRP_MJ_QUERY_INFORMATION? Should I use
Irp->MdlAddress, Irp->UserBuffer, or Irp->AssociatedIrp.SystemBuffer? My
FSD uses Direct I/O.

I’ve asked this question before, but nobody seems to want to answer it.
Is this because it’s such a stupid question? It doesn’t seem stupid to
me…

Currently I’m trying to use Irp->MdlAddress, if there is one. If not, I
use Irp->UserBuffer. In some rare cases I take an exception (access
violation) when I try to touch the buffer.

WHere is some doc on this? I see in the CDFS example that they use
Irp->AssociatedIrp.SystemBuffer. How do I know when to use which buffer?

Eric_Chan · March 13, 2002, 3:21am

I got the following in the Filemon filter driver.

irp->AssociatedIrp.SystemBuffer = FileQueryBuffer;
…
…
ioStackLocation->MajorFunction = IRP_MJ_QUERY_INFORMATION;

OSR_Community_User · March 13, 2002, 3:29am

Eric,

Thanks! - I’ll check out the filemon source - I changed my code to use
associatedirp.systembuffer and I’ll try it when I get to the office. I
might have finally figured it out…!

Regards,
Greg

OSR_Community_User · March 13, 2002, 10:54am

Hello All,

I tried this, but the Irp->AssociatedIrp.SystemBuffer is NULL on
IRP_MJ_QUERY_DIRECTORY. I use direct i/o, and in Nagar’s book, (page 147)
it says “System buffers are allocated by the IO Manager when a kernel mode
driver requests buffered I/O”.

Any other ideas?

Regards, Greg

OSR_Community_User · March 13, 2002, 11:28am

Also in Nagar’s book, there is a discussion of Handling user-space buffer
pointers. I’m doing what he says on page 184 for direct i/o, but once in
a while, I still get access violation (currently only in
IRP_MJ_DIRECTORY_CONTROL).

My code is like this:

If MDLaddress is null and requestor mode is not kernel mode, then I alloc
an MDL using Irp->UserBuffer as input parameter 1 and associate the MDL
with the current Irp. Next, I probe and lock the pages of the new mdl
with Irp->requestor mode (which is always usermode), and IoWriteAccess.
Once in a while MmProbeAndLockPages gets the access violation. The input
to MmProbeAndLockPages looks good to me at the time of failure.
MmProbeAndLockPages is wrapped in a try/except where I return
STATUS_INVALID_USER_BUFFER when this happens, instead of taking the blue
screen

Is this the right way to do this for Direct i/o?

I’m confused even further because the CDFS example (which I think is
DIRECT_IO), uses Irp->AssociatedIrp.Systembuffer for the pointer in this
routine.

Any help is always appreciated…

Greg

OSR_Community_User · March 13, 2002, 1:03pm

The Flags field of the device object describes the correct transfer method
for: IRP_MJ_READ, IRP_MJ_WRITE,
IRP_MJ_DIRECTORY_CONTROL/IRP_MN_QUERY_DIRECTORY, IRP_MJ_QUERY_SECURITY,
IRP_MJ_SET_SECURITY, IRP_MJ_QUERY_EA, IRP_MJ_SET_EA (those operations that
transfer large data elements). For IRP_MJ_DEVICE_CONTROL,
IRP_MJ_INTERNAL_DEVICE_CONTROL and IRP_MJ_FILE_SYSTEM_CONTROL it is a
function of the control code.

All other IRP_MJ_* values use buffered I/O
(Irp->AssociatedIrp.SystemBuffer).

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Greg Pearce [mailto:xxxxx@filetek.com]
Sent: Wednesday, March 13, 2002 2:53 AM
To: File Systems Developers
Subject: [ntfsd] which buffer to use?

Which buffer should I use for IRP_MJ_QUERY_INFORMATION? Should I use
Irp->MdlAddress, Irp->UserBuffer, or Irp->AssociatedIrp.SystemBuffer? My
FSD uses Direct I/O.

I’ve asked this question before, but nobody seems to want to answer it.
Is this because it’s such a stupid question? It doesn’t seem stupid to
me…

Currently I’m trying to use Irp->MdlAddress, if there is one. If not, I
use Irp->UserBuffer. In some rare cases I take an exception (access
violation) when I try to touch the buffer.

WHere is some doc on this? I see in the CDFS example that they use
Irp->AssociatedIrp.SystemBuffer. How do I know when to use which buffer?

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%

OSR_Community_User · March 13, 2002, 2:04pm

> Which buffer should I use for IRP_MJ_QUERY_INFORMATION? Should I use

Irp->MdlAddress, Irp->UserBuffer, or Irp->AssociatedIrp.SystemBuffer? My
FSD uses Direct I/O.

Irp->AssociatedIrp.SystemBuffer.
See the FASTFAT source.

Max

OSR_Community_User · March 13, 2002, 4:02pm

Hi Max (and everyone),

I looked in fastfat and CDFS and they do infact use
Irp->AssociatedIrp.SystemBuffer, and i tried that. I found that this
pointer is always null in my driver!

I specify in my driver device object to use DO_DIRECT_IO. I guess this
doesn’t mean ALWAYS use direct I/O, but just sometimes, for some IRPs?

I thought I read somewhere (Nagar’s book) that
Irp->AssociatedIrp.SystemBuffer is only used for buffered I/O.

I still don’t get it.

Thanks - Greg

OSR_Community_User · March 13, 2002, 4:15pm

Hi Tony,

Thanks for the info - I am setting DO_DIRECT_IO in the DeviceObject when I
create it, so I should be able to use the MDL address for this Irp, right?

Sometimes the MDL address is NULL (Is this because the Irp is from my FSP
queue or from a filter driver above mine in the stack? I really see this
access violation frequently when DFS is running (this doesn’t happen in
win2k pro).

I found a tip in your book on page 325 that states that it is possible to
get a zero length buffer. Is the user buffer length found in
IrpSp->Parameters.QueryDirectory.Length, and if so, is it valid for
whichever buffer I get (either MDLAddress buffer or Irp->UserBuffer)?

I was not checking for this possibility before, but now I am and maybe my
test will survive the night!

Thanks and regards,

Greg

OSR_Community_User · March 13, 2002, 5:50pm

Greg,

Of course you would have a problem with DFS - I bet it has never been tested
against anything OTHER than a NEITHER I/O file system! The I/O Manager does
handle such a case.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Greg Pearce [mailto:xxxxx@filetek.com]
Sent: Wednesday, March 13, 2002 4:14 PM
To: File Systems Developers
Subject: [ntfsd] Re: which buffer to use?

Hi Tony,

Thanks for the info - I am setting DO_DIRECT_IO in the DeviceObject when I
create it, so I should be able to use the MDL address for this Irp, right?

Sometimes the MDL address is NULL (Is this because the Irp is from my FSP
queue or from a filter driver above mine in the stack? I really see this
access violation frequently when DFS is running (this doesn’t happen in
win2k pro).

I found a tip in your book on page 325 that states that it is possible to
get a zero length buffer. Is the user buffer length found in
IrpSp->Parameters.QueryDirectory.Length, and if so, is it valid for
whichever buffer I get (either MDLAddress buffer or Irp->UserBuffer)?

I was not checking for this possibility before, but now I am and maybe my
test will survive the night!

Thanks and regards,

Greg

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%

OSR_Community_User · March 14, 2002, 12:00pm

While what you say may or may not be true regarding DFS, Greg’s problem
is different.
QUERY_INFORMATION & SET_INFORMATION are always buffered by NT i/o
manager regardless of what kind of i/o you support - the device object
flags don’t matter.
Use AssociatedIrp.SystemBuffer always.

I/O manager does initialize the Irp->UserBuffer field to the original
user buffer. The MdlAddress can be NULL because i/o doesn’t allocate MDL
for it - since this operation is buffered always.
If you want the MDL for some reason you can always allocate it.

Ravi

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, March 13, 2002 2:47 PM
To: File Systems Developers
Subject: [ntfsd] Re: which buffer to use?

Greg,

Of course you would have a problem with DFS - I bet it has never been
tested against anything OTHER than a NEITHER I/O file system! The I/O
Manager does handle such a case.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Greg Pearce [mailto:xxxxx@filetek.com]
Sent: Wednesday, March 13, 2002 4:14 PM
To: File Systems Developers
Subject: [ntfsd] Re: which buffer to use?

Hi Tony,

Thanks for the info - I am setting DO_DIRECT_IO in the DeviceObject when
I create it, so I should be able to use the MDL address for this Irp,
right?

Sometimes the MDL address is NULL (Is this because the Irp is from my
FSP queue or from a filter driver above mine in the stack? I really see
this access violation frequently when DFS is running (this doesn’t
happen in win2k pro).

I found a tip in your book on page 325 that states that it is possible
to get a zero length buffer. Is the user buffer length found in
IrpSp->Parameters.QueryDirectory.Length, and if so, is it valid for
whichever buffer I get (either MDLAddress buffer or Irp->UserBuffer)?

I was not checking for this possibility before, but now I am and maybe
my test will survive the night!

Thanks and regards,

Greg

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

OSR_Community_User · March 14, 2002, 6:21pm

Ravi,

I think the confusion (on my part) is that he refers to
Parameters.QueryDirectory.Length, which immediately made me think he was
using an IRP_MJ_DIRECTORY_CONTROL/IRP_MN_QUERY_DIRECTORY.

Upon further reading (and a quick peek at DFS) I would note that I think
this is just some very fundamental level of confusion. DFS does not handle
IRP_MJ_DIRECTORY_CONTROL in any case, as you note.

Greg - can you tell us what operation you are performing? If you are trying
IRP_MJ_QUERY_INFORMATION or IRP_MJ_SET_INFORMATION, you cannot use the
MdlAddress, no matter that you are setting the DO_DIRECT_IO option.

The DeviceObject->Flags field is only used for a subset of operations
(notably, those that transfer large data elements between user/kernel
memory) and not for ALL operations. I listed these operations earlier, but
I’ll list them again.

IRP_MJ_READ
IRP_MJ_WRITE
IRP_MJ_QUERY_SECURITY
IRP_MJ_SET_SECURITY
IRP_MJ_QUERY_EA
IRP_MJ_SET_EA
IRP_MJ_DIRECTORY_CONTROL

And, on a per control-code basis:

IRP_MJ_DEVICE_CONTROL
IRP_MJ_INTERNAL_DEVICE_CONTROL
IRP_MJ_FILE_SYSTEM_CONTROL

ALL other operations are done via buffered I/O, no matter what it says in
the DeviceObject->Flags field.

Look at Irp->AssociatedIrp.SystemBuffer for these other cases.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Ravisankar Pudipeddi [mailto:xxxxx@windows.microsoft.com]
Sent: Thursday, March 14, 2002 11:54 AM
To: File Systems Developers
Subject: [ntfsd] Re: which buffer to use?

While what you say may or may not be true regarding DFS, Greg’s problem
is different.
QUERY_INFORMATION & SET_INFORMATION are always buffered by NT i/o
manager regardless of what kind of i/o you support - the device object
flags don’t matter.
Use AssociatedIrp.SystemBuffer always.

I/O manager does initialize the Irp->UserBuffer field to the original
user buffer. The MdlAddress can be NULL because i/o doesn’t allocate MDL
for it - since this operation is buffered always.
If you want the MDL for some reason you can always allocate it.

Ravi

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, March 13, 2002 2:47 PM
To: File Systems Developers
Subject: [ntfsd] Re: which buffer to use?

Greg,

Of course you would have a problem with DFS - I bet it has never been
tested against anything OTHER than a NEITHER I/O file system! The I/O
Manager does handle such a case.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Greg Pearce [mailto:xxxxx@filetek.com]
Sent: Wednesday, March 13, 2002 4:14 PM
To: File Systems Developers
Subject: [ntfsd] Re: which buffer to use?

Hi Tony,

Thanks for the info - I am setting DO_DIRECT_IO in the DeviceObject when
I create it, so I should be able to use the MDL address for this Irp,
right?

Sometimes the MDL address is NULL (Is this because the Irp is from my
FSP queue or from a filter driver above mine in the stack? I really see
this access violation frequently when DFS is running (this doesn’t
happen in win2k pro).

I found a tip in your book on page 325 that states that it is possible
to get a zero length buffer. Is the user buffer length found in
IrpSp->Parameters.QueryDirectory.Length, and if so, is it valid for
whichever buffer I get (either MDLAddress buffer or Irp->UserBuffer)?

I was not checking for this possibility before, but now I am and maybe
my test will survive the night!

Thanks and regards,

Greg

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%

OSR_Community_User · March 14, 2002, 6:57pm

> Is this the right way to do this for Direct i/o?

With DO_DIRECT_IO, all of this is already done by the IO manager, you must only use Irp->MdlAddress which is a locked MDL.

Max

OSR_Community_User · March 15, 2002, 9:07am

I’m trying to process IRP_MJ_DIRECTORY_CONTROL with IRP_MN_QUERY_DIRECTORY
minor function.

I swear that Irp->AssociatedIrp.SystemBuffer is NULL!

Thanks - Greg

OSR_Community_User · March 15, 2002, 12:13pm

That’s to be expected. I wish you read Tony’s list which he posted twice
already to the mailing list within
the span of one day.
I/O rolls the IRP based on the device for this IRP. Use device object
flags as the hint
(for DO_BUFFERED_IO it’s SystemBuffer, for DO_DIRECT_IO its MdlAddress,
for neither it’s UserBuffer)

(note to Tony: sounds like your list should be in the FAQ if it isn’t
already …)

Ravi

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Greg Pearce [mailto:xxxxx@filetek.com]
Sent: Friday, March 15, 2002 6:06 AM
To: File Systems Developers
Subject: [ntfsd] Re: which buffer to use?

I’m trying to process IRP_MJ_DIRECTORY_CONTROL with
IRP_MN_QUERY_DIRECTORY minor function.

I swear that Irp->AssociatedIrp.SystemBuffer is NULL!

Thanks - Greg

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

OSR_Community_User · March 16, 2002, 6:35am

Ravi,

An excellent suggestion - I’ll work on getting this information added to the
FAQ.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----

(note to Tony: sounds like your list should be in the FAQ if it isn’t
already …)

Ravi

This posting is provided “AS IS” with no warranties, and confers no
rights.

OSR_Community_User · March 16, 2002, 2:36pm

Ravi,

I DID read it both times; I was just reposting the details of what I was
trying to do because Tony asked me to clarify it once and for all in a
previous post. Before I knew all this, I just asked which buffer to use,
because I didn’t understand the relationship between direct_io and
buffered_io and which buffers to use.

I am the one that sets DO_DIRECT_IO when I create my device object, so I
assumed all I/O to my FSD would be DIRECT I/O. Now I’ve learned that for
some IRPs it’s always buffered I/O and for some it is direct I/O.

I am still a newbie at this; I inherited this ancient FSD and still have
to develop other products, so I am easily confused because of other
distractions! It’s tough juggling all this stuff with the amount of time
that I have, so I really do appreciate the help you guys have given me.

Thanks again,
Greg