I would like to issue a separate write operation to a different target using a new IRP that I will build in my IoCompletion routine. I would like to do this after the original write IRP is complete on the original target device hence my implementation will be in IoCompletion routine that I will hook during IRP_MJ_WRITE dispatch routine for original IRP. My implementation is going to be in volume upper filter. Is it safe to block the original IRP from completing till my new write IRP has completed writing to the different target device ? If its not safe then what would be a good solution to implement this ?
Do you *really* want this to be synchronous? Why not send the write request off to a work item, and avoid the issues of (a) large stack usage, (b) elevated IRQL dispatching, (c) the potential for deadlocks in the paging page that are inherent in sending requests directly from within your completion routine to random places in the storage stack.
I would like to issue a separate write operation to a different target
using a new IRP that I will build in my IoCompletion routine. I would like
to do this after the original write IRP is complete on the original target
device hence my implementation will be in IoCompletion routine that I will
hook during IRP_MJ_WRITE dispatch routine for original IRP. My
implementation is going to be in volume upper filter. Is it safe to block
the original IRP from completing till my new write IRP has completed
writing to the different target device ? If its not safe then what would
be a good solution to implement this ?
This is a common paradigm; you hold onto an incoming IRP, create a new
IRP’, send IRP’ down with a completion routine set, and in the completion
routine, you can either send IRP’ down again or delete it and finally call
IoCompleteRequest on IRP. Note that in some cases, if you send the SAME
IRP down, then to “complete” it you simply return from your completion
routine with STATUS_SUCCESS, and to “hold” it you simply return with
STATUS_MORE_PROCESSING_REQUIRED.
joe
It is a commonly *misused and misunderstood* paradigm.
As I said in my reply to the OP, sending a request directly from within a completion routine is SOMETIMES a good idea… but it’s almost ALWAYS idea to post the action of sending the request to a work item.
Sending the request from a work item avoids problems with excessive stack usage (consider a request which is processed synchronously throughout a branch in the device tree, is completed, and then a completion routine within that branch sends an IRP… we’re talking the potential for LOTS of stack usage). Posting also avoids the problem of sending a request at IRQL DISPATCH_LEVEL to a target that excepts its dispatch entry points to be called at IRQL PASSIVE_LEVEL.
The problem with workitems vs paging path is that the workitems are allowed to mess with paged data and code, and thus are allowed to get blocked by paging IO. There is possibility that at some time all worker threads are blocked by paging, and then your driver needs a workitem to proceed with some paging request. That’s a deadlock.
And then there is that evil problem of the past OSes when the worker threads were waiting with UserMode, so their stacks could have been paged out.
IF you’re not sending the request synchronously with respect to the paging I/O or IF you’re sending the request to a branch of the device tree where there’s no paging presently allowed, then you’re good to go.
Hence… “it depends” – I’m not saying you’re wrong. I’m agreeing with you. But, again as I said earlier, this whole issue is often misused and misunderstood.
Please excuse me for staying away from this thread for so long. I failed to realize that most of the posts in this thread would show up under different subject line in my outlook.
Thanks for clarifying on why I need to stay away from issuing an IRP in IoCompletion routine more often than not. “(b) elevated IRQL dispatching,” as mentioned by Peter, was in my mind when I asked the question. But Peter has pointed out other issues with issuing I/O in IoCompletion routine.
However, I am still unclear about the fact that what do I do with the original IRP. I would delegate the new IRP through workitem approach to avoid the issues mentioned by you. But, I am kind of implementing a mirroring type of solution here. So, I would like to hold back the original IRP. Once my work item is completed with success, I can signal an event that would let the original IRP be propagated to the upper driver (I intend to hold back original IRP in IoCompletion routine). Is it safe to hold back the original IRP which could be running synchronously or through a Dpc routine ?
Regarding paging I/O, now I am wondering if I can skip the paging I/O mirroring.
Oh, no… please don’t do that. That would be very bad.
What you want to do is save the pointer to the IRP, and return STATUS_MORE_PROCESSING_REQUIRED from your completion routine. Then, after your OTHER IRP finishes, simply call IoCompleteRequest on the original IRP. How simple this is depends on how you originally sent the IRP from your dispatch entry point, by the way. If your dispatch entry point returns STATUS_PENDING, you’ll be OK.
Thanks for clarifications on how to process the original IRP. So, in my scenario, can I always return STATUS_PENDING from my dispatch routine except for paging I/O ? I will be mirroring all other blocks of the volume. Does the following algorithm look good ?
if( Irp->PendingReturned )
{
IoMarkIrpPending(Irp);
return STATUS_SUCCESS;
}
else
{
Save original (completed) IRP pointer
Create a new IRP and queue the work item
return STATUS_MORE_PROCESSING_REQUIRED;
}
System thread will complete the Original IRP synchronously by sending it to target device and will then complete the original IRP by setting
So, as Mr. Grig referenced earlier… paging I/O is not the same thing as I/O requests being sent to the paging file.
I’m sorry to say that I think you’re more than just a little off track here, OP.
Like I said earlier… this depends on what you want to do: If you *must* handle the original request synchronously, then WHERE you’re sending the request matters.
If you’re sending your new request down a branch of the storage stack, follow Mr. Grig’s and Dr. Newcomer’s original advice and just IoCallDriver the request from your completion routine. Just be aware of the issues I raised.
This is where “been there, done that” is the cache phrase comes…
I saw the some readyboost execution path does exactly this, issues IRPs from completion path, though it checks for stack space availability ( hard coded to, I think, 1024 bytes ). If okay, then it goes, so basically if someone has a bad design in the stack, it will catch, and bugcheck.
On the other side, in error handling I once tried to issue IRP from the completion, and the retry mechanism can go couple round - and I saw in debugger that DPC timeout warning fires off, quite frequently… So pushed it out as workitem ( basically landing on another stack )…
It is a commonly *misused and misunderstood* paradigm.
As I said in my reply to the OP, sending a request directly from within a completion routine is SOMETIMES a good idea… but it’s almost ALWAYS idea to post the action of sending the request to a work item.
Sending the request from a work item avoids problems with excessive stack usage (consider a request which is processed synchronously throughout a branch in the device tree, is completed, and then a completion routine within that branch sends an IRP… we’re talking the potential for LOTS of stack usage). Posting also avoids the problem of sending a request at IRQL DISPATCH_LEVEL to a target that excepts its dispatch entry points to be called at IRQL PASSIVE_LEVEL.
>Paging I/O in windows does not necessarily mean “a write to the paging file.”
So this means it’s not allowed to cheat and skip mirror write is the IO is marked as paging? otherwise other that pagefile.sys files (created by MM IO) would not be mirrored.
Having said that, you CAN check explicitly to see if the I/O is against the paging file’s FILE_OBJECT. There’s a call FsRtlIsPagingFile that does this.
Thanks for clarifications on how to process the original IRP. So, in my scenario, can I always return STATUS_PENDING from my dispatch routine except for paging I/O ? I will be mirroring all other blocks of the volume. Does the following algorithm look good ?
if( Irp->PendingReturned )
{
IoMarkIrpPending(Irp);
return STATUS_SUCCESS;
}
else
{
Save original (completed) IRP pointer
Create a new IRP and queue the work item
return STATUS_MORE_PROCESSING_REQUIRED;
}
System thread will complete the Original IRP synchronously by sending it to target device and will then complete the original IRP by setting