* Associated IRPs - split I/O requests

NTFSD
1

Hi all,
In my disk filter driver I am trying to split I/O read requests into
different pieces and send them in PARALLEL to underlying disk device
drivers.
At first step I try do that without using of associated IRPs.
Here is part of my current test :

…
//
// Build all partial IRPs
//
for (i=0; i
//
// Create notification event object for request completion.
//
KeInitializeEvent(&event[i], NotificationEvent, FALSE);
pEvent[i]=&event[i];

//
// Build Read
//
irp[i] = IoBuildSynchronousFsdRequest(IRP_MJ_READ,
HbaDeviceObject[i],
PartialBuffer[i],
PartialLength[i],
&PartialByteOffset[i],
&event[i],
&PartialIoStatus[i] );

if (!irp[i]) {
…
return STATUS_UNSUCCESSFUL;
}

}

//
// Send all partial IRPs
//
for (i=0; i
//
// Pass request to port driver and wait for request to complete.
//
status = IoCallDriver( HbaDeviceObject[i], irp[i] );
…
}

//
// Wait for all partial IRP completion
//
KeWaitForMultipleObjects(
HBANUM,
pEvent,
WaitAll,
Executive,
KernelMode,
FALSE,
NULL,
NULL
);
…

If I use this code for reads with (Length > 100K) approx. everything is
fine, but if I call this code for all (Lenght > SECTOR_SIZE) - by some
reasons KeWaitForMultipleObjects() never returns from wait state!?

(!) As I read in Rajeev Nagar’s book for my purpose the ideal choice is
using Associated IRPs but I coudn’t find any appropriate sample code…

Any help will be appreciated!

Thanks in advance,
Kristian

—
You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

2

Kristian,

You cannot use associated IRPs in a disk filter driver. The file systems
use associated IRPs and because of this, you cannot. The reason for this is
that an associated IRP cannot itself be divided into a set of associated
IRPs.

Unfortunately, the correct solution is probably to debug your existing code
and figure out why the I/O operations are never completing. For example,
one possibility is that you aren’t setting your code up to set the events in
your completion routine - if you rely upon the OS, and you are called at
APC_LEVEL, you will never see the events get set. This would happen, for
instance, with paging I/O, which would be < 100K.

The question would be - what is the state of your IRPs. Are there any
pending APCs awaiting delivery.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@atia.com [mailto:xxxxx@atia.com]
Sent: Tuesday, September 25, 2001 3:23 AM
To: File Systems Developers
Subject: [ntfsd] * Associated IRPs - split I/O requests

Hi all,
In my disk filter driver I am trying to split I/O read requests into
different pieces and send them in PARALLEL to underlying disk device
drivers.
At first step I try do that without using of associated IRPs.
Here is part of my current test :

…
//
// Build all partial IRPs
//
for (i=0; i
//
// Create notification event object for request completion.
//
KeInitializeEvent(&event[i], NotificationEvent, FALSE);
pEvent[i]=&event[i];

//
// Build Read
//
irp[i] = IoBuildSynchronousFsdRequest(IRP_MJ_READ,
HbaDeviceObject[i],
PartialBuffer[i],
PartialLength[i],
&PartialByteOffset[i],
&event[i],
&PartialIoStatus[i] );

if (!irp[i]) {
…
return STATUS_UNSUCCESSFUL;
}

}

//
// Send all partial IRPs
//
for (i=0; i
//
// Pass request to port driver and wait for request to complete.
//
status = IoCallDriver( HbaDeviceObject[i], irp[i] );
…
}

//
// Wait for all partial IRP completion
//
KeWaitForMultipleObjects(
HBANUM,
pEvent,
WaitAll,
Executive,
KernelMode,
FALSE,
NULL,
NULL
);
…

If I use this code for reads with (Length > 100K) approx. everything is
fine, but if I call this code for all (Lenght > SECTOR_SIZE) - by some
reasons KeWaitForMultipleObjects() never returns from wait state!?

(!) As I read in Rajeev Nagar’s book for my purpose the ideal choice is
using Associated IRPs but I coudn’t find any appropriate sample code…

Any help will be appreciated!

Thanks in advance,
Kristian

—
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

—
You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

3

> You cannot use associated IRPs in a disk filter driver. The file systems

use associated IRPs and because of this, you cannot. The reason for this
is
that an associated IRP cannot itself be divided into a set of associated
IRPs.

But we can invent our own mechanism of associated IRPs which will not use
the AssociatedIrp field, but will use some completion context structure
instead.

Is it a wrong way?

Max

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

4

Check out ClassSplitRequest in nt50ddk\src\storage\class\classpnp\class.c
It does it’s own version of associated irps…

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Tuesday, September 25, 2001 8:54 AM
To: File Systems Developers
Subject: [ntfsd] RE: * Associated IRPs - split I/O requests

You cannot use associated IRPs in a disk filter driver. The file
systems use associated IRPs and because of this, you cannot. The
reason for this
is
that an associated IRP cannot itself be divided into a set of
associated IRPs.

But we can invent our own mechanism of associated IRPs which will not use
the AssociatedIrp field, but will use some completion context structure
instead.

Is it a wrong way?

Max

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

5

Max,

There’s no problem with constructing your own mechanism (similar to what
Kristian described earlier.) In fact, my point is that if you are involved
in the storage stack, unless you are an FSD, you really should not be using
associated IRPs because the file systems are using them (see deviosup.c in
FastFat for examples.) Of course, FAT doesn’t use associated IRPs for every
I/O operation, only when necessary, so it is even possible that you might
not discover this until you were testing. Hence, my standard advice for
anyone working in the storage stack is that unless you are writing a file
system, you should not use associated IRPs.

As you point out, the driver must construct its own mechanism for achieving
comparable functionality.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Tuesday, September 25, 2001 8:54 AM
To: File Systems Developers
Subject: [ntfsd] RE: * Associated IRPs - split I/O requests

You cannot use associated IRPs in a disk filter driver. The file systems
use associated IRPs and because of this, you cannot. The reason for this
is
that an associated IRP cannot itself be divided into a set of associated
IRPs.

But we can invent our own mechanism of associated IRPs which will not use
the AssociatedIrp field, but will use some completion context structure
instead.

Is it a wrong way?

Max

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

6

Thank you for fast replies, guys!

Tony, you was right. The problem with never completed
KeWaitForMultipleObject() was for paging I/O requests only…
At the moment the filter splits all non paging I/O successfuly.

For paging I/O I have tried to set CompletionRoutine and there I do:

  • set the completion Event
  • free allocated IRP
  • return STATUS_MORE_PROCESSING_REQUIRE.

When I try reading / usualy by playing some AVI from MediaPlayer / after
few successful reads I have BSOD!

I build my IRPs using IoBuildSynchronousFsdRequest() - may be this is the
problem?

Thanks,
Kristian

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

7

> in the storage stack, unless you are an FSD, you really should not be
using

associated IRPs because the file systems are using them (see deviosup.c in
FastFat for examples.)

Yes, Tony, surely it is so.

Max

You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com