Reusing IRPs when transfer is incomplete

NTDEV
1

Hi,
Is it suggested to reuse an IRP by calling IoReuseIrp(…) inside a completion routine even when there is a data transfer pending on the same irp?

The code that is in question looks like this:

if(NT_SUCCESS(Status))
{
RequestContext->Length -= RequestContext->Urb->UrbBulkOrInterruptTransfer.TransferBufferLength;

if(RequestContext->Length)
{
//
// We have at least another stage transfer to do
//
// reinitialise the Urb
//
NextStack = IoGetNextIrpStackLocation(Irp);

// IoReuseIrp() required???

NextStack->MajorFunction = IRP_MJ_INTERNAL_DEVICE_CONTROL;
NextStack->Parameters.Others.Argument1 = RequestContext->Urb;
NextStack->Parameters.DeviceIoControl.IoControlCode = IOCTL_INTERNAL_USB_SUBMIT_URB;

IoSetCompletionRoutine(Irp, RequestCompleteRoutine, RequestContext, TRUE, TRUE, TRUE);

IoCallDriver(RequestContext->DevExt->NextLowerDriver, Irp);
…
…
}
}

2

How will the application get a notification of completion of data transfer,
if you have re-used it for another I/O.
You need to complete the I/O before re-using it.
Moreover it can be used only if you have allocated irp using IoAllocateIrp
or as raw memory.

Regards
Deepak

On Tue, Jan 4, 2011 at 2:53 PM, wrote:

> Hi,
> Is it suggested to reuse an IRP by calling IoReuseIrp(…) inside a
> completion routine even when there is a data transfer pending on the same
> irp?
>
> The code that is in question looks like this:
>
> if(NT_SUCCESS(Status))
> {
> RequestContext->Length -=
> RequestContext->Urb->UrbBulkOrInterruptTransfer.TransferBufferLength;
>
> if(RequestContext->Length)
> {
> //
> // We have at least another stage transfer to do
> //
> // reinitialise the Urb
> //
> NextStack = IoGetNextIrpStackLocation(Irp);
>
> // IoReuseIrp() required???
>
> NextStack->MajorFunction = IRP_MJ_INTERNAL_DEVICE_CONTROL;
> NextStack->Parameters.Others.Argument1 = RequestContext->Urb;
> NextStack->Parameters.DeviceIoControl.IoControlCode =
> IOCTL_INTERNAL_USB_SUBMIT_URB;
>
> IoSetCompletionRoutine(Irp, RequestCompleteRoutine,
> RequestContext, TRUE, TRUE, TRUE);
>
> IoCallDriver(RequestContext->DevExt->NextLowerDriver, Irp);
> …
> …
> }
> }
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

3

Yes the irp is allocated through IoAllocateIrp. The code that I have posted is from the completion routine and the case is for only when IoAllocateIrp is used for the irp and there is more data pending transfer on that irp.

4

Still, since the Irp is marked pending, you can’t re-use it.
You need to complete it

Regards
deepak

On Tue, Jan 4, 2011 at 5:06 PM, wrote:

> Yes the irp is allocated through IoAllocateIrp. The code that I have posted
> is from the completion routine and the case is for only when IoAllocateIrp
> is used for the irp and there is more data pending transfer on that irp.
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

5

>Is it suggested to reuse an IRP by calling IoReuseIrp(…) inside a

completion routine even when there is a data transfer pending on >the same
irp?

It’s probably not strictly necessary in your case (so if you’re having some
problem that you’re hoping this will fix you’re probably out of luck),
though it would be cleaner to do so. It’s basically just going to put your
IRP back into the exact state that it was when you first allocated it, so it
will clear any fields that might have been modified by the lower layers
(e.g. Cancel).

Also note that it is ONLY correct to call IoReuseIrp on IRPs that you have
allocated yourself with IoAllocateIrp. It is an error to call it on other
IRPs (Call Usage Verifier used to catch this, not sure if Driver Verifier
does now).

-scott

–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

Hope to see you at the next OSR kernel debugging class February 14th in
Columbia, MD!

6

>“Deepak Gupta” wrote in message
>news:xxxxx@ntdev…
>How will the application get a notification of completion of data transfer,
>if you have re-used it for another I/O.

The OP is staging a single user transfer across multiple transfers to the
lower drivers. Think of it as servicing a user request for a disk I/O to a
stripe set where you use a single IRP to read from each disk in the stripe
set (sort of a bad example because that wouldn’t be the greatest design, but
the point is there).

-scott

–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

Hope to see you at the next OSR kernel debugging class February 14th in
Columbia, MD!

7

Deepak Gupta wrote:

How will the application get a notification of completion of data
transfer, if you have re-used it for another I/O.
You need to complete the I/O before re-using it.

I’ve started three different replies contradicting this, but I think I
finally understand what Deepak is trying to say here. He is right,
although I would have worded it differently.

> wrote:
> Hi,
> Is it suggested to reuse an IRP by calling IoReuseIrp(…) inside a
> completion routine even when there is a data transfer pending on the
> same irp?

We are all assuming here that you have received an IRP from above, which
you need to resubmit to a lower driver in smaller pieces.

The problem with IoReuseIrp is that it will erase any connection to the
driver (or application) above you. If you IoReuseIrp and resubmit, by
the time it gets back to your completion routine, you can no longer
complete it, because the upper level information is gone. You can only
use IoReuseIrp on an IRP that YOU allocated, not on an IRP that you
received.

You CAN resubmit the IRP in your completion routine to start the next
stage of your I/O, but you do so by allocating and filling in a stack
location, NOT by wiping out the whole IRP. The I/O stack location is
yours to modify. The rest of the IRP is not. You need the rest of the
I/O stack to remain intact.

The comment about being “marked pending” also threw me off. The issue
is that the IRP came to you from elsewhere, with information that needs
to be retained.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

8

“Tim Roberts” wrote in message news:xxxxx@ntdev…

You can only use IoReuseIrp on an IRP that YOU allocated, not on an IRP
that you received.

And the OP already indicated that the IRP in question is one that was
allocated by their driver:

“Yes the irp is allocated through IoAllocateIrp.”

-scott

–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

Hope to see you at the next OSR kernel debugging class February 14th in
Columbia, MD!

9

> The problem with IoReuseIrp

I would say IoReuseIrp is only the replacement of IoFreeIrp+IoAllocateIrp, with the benefit of avoiding the costs of allocate/free.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

10

I posted it earlier. Was not logged in hence I assume it has not appeared. Posting a new one. Excuse me if it becomes a double post.
Ok let us ignore the case of Iomanager’s Irps. The whole context is for my own Irps allocated using IoAllocateIrp.
Now I see that I can choose to do a IoReuseIrp inside a completion routine before sending the Irp down for next stage of transfer after setting up the next stack. However, I was thinking, why is it ok to wipe out all flags using IoReuseIrp (assuming IoReuseIrp clears everything) on my own Irp when it is not ok to do the same on IoManager’s Irp. I understand that no one else cares for my irp unlike that of Iomanager’s Irp. Hence this should be alright. But is it ok to alter the state of the Irp in the middle of transfer in a completion routine before doing next stage of transfer? Is it known to cause any issues?
On the face of it looks ok as long as the lower driver has not kept any info that it needs in the half transfered Irp.

11

On Wed, Jan 5, 2011 at 10:52 AM, wrote:

> I understand that no one else cares for my irp unlike that of Iomanager’s
> Irp. Hence this should be alright. But is it ok to alter the state of the
> Irp in the middle of transfer in a completion routine before doing next
> stage of transfer? Is it known to cause any issues?
> On the face of it looks ok as long as the lower driver has not kept any
> info that it needs in the half transfered Irp.
>

I/O Manager also would not care for completion of IRP’s if it had not to
notify it’s clients (Applications/other kernel components) of the
completion.
If we think on a broader scale, you are developing your driver to provide
service to a usable application (it can be your own application or lot of
third party apps).
That means you are generating IRP’s from your driver on some request of any
application.

Now you would like to inform the application of the operations you have done
in it’s entirety (either asynchronously or synchronously)

Now if you are re-using the IRPs for staging and transfer is still happening
then you have to maintain that information in some private piece of memory
and keep track of it.
Once all you are convinced that all staging thing has been done, you would
eventually like to complete main request from application.
Now here you would need that information that you have kept in private piece
of memory of all the transfers and depending on it, you will need to
complete the main request from application correctly.

If above is the case then I believe, you can do it.

Regards
Deepak

12

>why is it ok to wipe out all flags using IoReuseIrp (assuming IoReuseIrp

clears everything) on my own Irp when it is not ok to do the same >on
IoManager’s Irp.

The main reason is that the I/O manager queues the IRP to the thread that
issued it, making what’s called a “threaded IRP”. By calling IoReuseIrp on a
threaded IRP you will mess up the queue and lead to horrible things
(IoReuseIrp will actually zero the IRP structure).

IoAllocateIrp does not create threaded IRPs, thus you do not have this
problem. Note however that there ARE other I/O manager APIs that will create
threaded IRPs for you and would be incompatible with IoReuseIrp. Those APIs
can be found in the MSDN docs about reusing IRPs:

http://msdn.microsoft.com/en-us/library/ff561107(v=vs.85).aspx

On the face of it looks ok as long as the lower driver has not kept any
info that it needs in the half transfered Irp.

Your completion routine will not be called until the lower driver calls
IoCompleteRequest on the IRP. Once a driver calls IoCompleteRequest they no
longer “own” the IRP and are prohibited from doing anything with the IRP
structure or its contents.

-scott

–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

Hope to see you at the next OSR kernel debugging class February 14th in
Columbia, MD!

13

xxxxx@gmail.com wrote:

Ok let us ignore the case of Iomanager’s Irps. The whole context is for my own Irps allocated using IoAllocateIrp.
Now I see that I can choose to do a IoReuseIrp inside a completion routine before sending the Irp down for next stage of transfer after setting up the next stack. However, I was thinking, why is it ok to wipe out all flags using IoReuseIrp (assuming IoReuseIrp clears everything) on my own Irp when it is not ok to do the same on IoManager’s Irp. I understand that no one else cares for my irp unlike that of Iomanager’s Irp. Hence this should be alright. But is it ok to alter the state of the Irp in the middle of transfer in a completion routine before doing next stage of transfer?

Your last sentence does not describe the reality. For an IRP that you
created, when it gets back to your completion routine, its life is
over. It has served its purpose. It has nowhere else to go. It’s not
in “the middle of a transfer” – that’s merely the way you are USING the
IRP. The IRP itself has expired. You are free to do whatever you want
with it.

That’s not the case with an I/O manager IRP. When such an IRP reaches
your completion routine, it still has things on its bucket list that
must be accomplished before it can be allowed to die. You must complete
such an IRP, so it can rise up to the next level and fulfill its promises.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

14

One of the odd problems about receiving your own IRP back is that you need
to complete with STATUS_MORE_PROCESSING_REQUIRED, which is a really bad name
for what should have been
STATUS_PLEASE_DO_NOT_TRY_TO_COMPLETE_THIS_IRP_AT_A_HIGHER_LEVEL, because
there isn’t a higher level. I find a number of people confused by this
name, saying things like “But that doesn’t make any sense. I’ve just freed
the IRP, and therefore there is no more processing required”. So they
complete with STATUS_SUCCESS and wonder why they bluescreen. It is easy to
see how completing with this code suggests that the IRP is somehow in the
“middle of a transfer”, but as you point out, there is no transfer it could
possibly be in the middle of; it’s dead and gone. Some *other* IRP was in
the middle of a transfer, and that’s the IRP you now need to complete in
your completion routine. The confusion is not lessened by the fact that if
you reuse your IRP for another transfer (the classic USB Bulk Transfer
source example we had for decades in the old DDK), you *also* respond with
STATUS_MORE_PROCESSING_REQUIRED. So the same code says “I’m sending the IRP
down again” and “the IRP is dead and gone”. [But Microsoft has not been
noted for selecting sensible names (or even names that are spelled
correctly, e.g, PAGABLE, meaning “something that can be pagged”. And who
would name an API “CreateFile” whose real job is “OpenAnIoHandle”, since it
opens handles to named pipes, mailslots, devices, and, oh yes, files. When
I was first learning Win32 20 years ago (Time Files When You’re Having Fun)
I went looking for a corresponding OpenFile API, thinking that this was
reflecting the old _creat/_open model. Of course, it wasn’t.) Bottom line:
it is confusing when the concepts of “I really am in the middle of a
transfer” and “I have completed using my IRP” are indicated by the same
return value, which has a misleading name.

I haven’t written a driver in years, but I get to read code of non-working
drivers all the time, trying to explain to the client why what they have
[usually outsourced overseas] could not possibly ever work under any
imaginable conditions and can only be salvaged by a total rewrite. And I
often turn them down if they ask me if I could rewrite it, and frequently
point them at OSR. I cannot emulate what’s in Tony’s head [yes, many of
these are file system drivers, and even *I* can see that they don’t work.
If I’m uncertain, I use Driver Verifier to demonstrate that they are wrong,
usually in PnP or Power but often in fundamental ways that are easy to see
and hard to fix].
joe

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, January 05, 2011 2:41 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Reusing IRPs when transfer is incomplete

xxxxx@gmail.com wrote:

Ok let us ignore the case of Iomanager’s Irps. The whole context is for my
own Irps allocated using IoAllocateIrp.
Now I see that I can choose to do a IoReuseIrp inside a completion routine
before sending the Irp down for next stage of transfer after setting up the
next stack. However, I was thinking, why is it ok to wipe out all flags
using IoReuseIrp (assuming IoReuseIrp clears everything) on my own Irp when
it is not ok to do the same on IoManager’s Irp. I understand that no one
else cares for my irp unlike that of Iomanager’s Irp. Hence this should be
alright. But is it ok to alter the state of the Irp in the middle of
transfer in a completion routine before doing next stage of transfer?

Your last sentence does not describe the reality. For an IRP that you
created, when it gets back to your completion routine, its life is over. It
has served its purpose. It has nowhere else to go. It’s not in “the middle
of a transfer” – that’s merely the way you are USING the IRP. The IRP
itself has expired. You are free to do whatever you want with it.

That’s not the case with an I/O manager IRP. When such an IRP reaches your
completion routine, it still has things on its bucket list that must be
accomplished before it can be allowed to die. You must complete such an
IRP, so it can rise up to the next level and fulfill its promises.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

–
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

15

>>Your last sentence does not describe the reality. For an IRP that you
created, when it gets back to your completion routine, its life is
over. It has served its purpose. It has nowhere else to go. It’s not
in “the middle of a transfer” – that’s merely the way you are USING the
IRP. The IRP itself has expired. You are free to do whatever you want
with it.

That’s not the case with an I/O manager IRP. When such an IRP reaches
your completion routine, it still has things on its bucket list that
must be accomplished before it can be allowed to die. You must complete
such an IRP, so it can rise up to the next level and fulfill its promises.

That explains it.

16

>wipe out all flags using IoReuseIrp (assuming IoReuseIrp clears everything) on my own Irp when it is

not ok to do the same on IoManager’s Irp

Same way you cannot IoFreeIrp the IO manager’s IRP.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com