Building Irp using IoBuildAsynchronousFsdRequest

OSR_Community_User · December 20, 2007, 10:47am

Please tell me if this aproach is good:

DispatchFunctionWrite( …PIRP Irp… )
{
PIRP MyIrp
KEVENT Event;

Init event
Build irp using IoBuildAsynchronousFsdRequest ( MyIrp )
Set completion for MyIrp ( pass event as context )
Send MyIrp to the next device
if STATUS_PENDING return then KeWaitForSingleObject( Event )
Set IOSB for Irp ( original one ) to STATUS_SUCCESS
Call IoCompleteRequest( Irp )
}

IoCompletion( …Irp, Context … )
{

if irp pending then setEvent ( PKEVENT(Context)… )
FreeIrp ( Irp );
return status_more_processing_required;
}

Thanks,
Radu Vornicu

OSR_Community_User · December 20, 2007, 10:59am

No, it’s not.
irp_mj_write may be called at dpc_level - and you can’t use
KeWaitForSingleObject

it’d be better to modify the current irp, set up a completion routine and
call the lower fsd
then your completion routine will be called with the IOSB.

-pk

wrote in message news:xxxxx@ntdev…
> Please tell me if this aproach is good:
>
> DispatchFunctionWrite( …PIRP Irp… )
> {
> PIRP MyIrp
> KEVENT Event;
>
> - Init event
> - Build irp using IoBuildAsynchronousFsdRequest ( MyIrp )
> - Set completion for MyIrp ( pass event as context )
> - Send MyIrp to the next device
> - if STATUS_PENDING return then KeWaitForSingleObject( Event )
>
> - Set IOSB for Irp ( original one ) to STATUS_SUCCESS
> - Call IoCompleteRequest( Irp )
> }
>
> IoCompletion( …Irp, Context … )
> {
> - if irp pending then setEvent ( PKEVENT(Context)… )
> - FreeIrp ( Irp );
> - return status_more_processing_required;
> }
>
> Thanks,
> Radu Vornicu
>

Don_Burn_1 · December 20, 2007, 11:01am

Why are you creating the new IRP? For most situations you are presenting
just forwarding the existing IRP (perhaps with changes in parameters) make
much more sense. Additionally, why are you waiting for completion of this
new IRP before returning from the dispatch routine, you should return from
the dispatch without the wait pending the first IRP and returning
STATUS_PENDING, then have the IoCompletion routine complete the original
IRP.

Basically why do you think you need this pattern?

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply

wrote in message news:xxxxx@ntdev…
> Please tell me if this aproach is good:
>
> DispatchFunctionWrite( …PIRP Irp… )
> {
> PIRP MyIrp
> KEVENT Event;
>
> - Init event
> - Build irp using IoBuildAsynchronousFsdRequest ( MyIrp )
> - Set completion for MyIrp ( pass event as context )
> - Send MyIrp to the next device
> - if STATUS_PENDING return then KeWaitForSingleObject( Event )
>
> - Set IOSB for Irp ( original one ) to STATUS_SUCCESS
> - Call IoCompleteRequest( Irp )
> }
>
> IoCompletion( …Irp, Context … )
> {
> - if irp pending then setEvent ( PKEVENT(Context)… )
> - FreeIrp ( Irp );
> - return status_more_processing_required;
> }
>
> Thanks,
> Radu Vornicu
>

OSR_Community_User · December 20, 2007, 12:09pm

Thanks for the replays,

I build a new irp, because i need to change the buffer from the original IRP. ( i’ve done the method make mdl, replace mdl, Iocomplete put original mdl back and now i try to learn how to make this work to.)
The problem is that i may need to split the buffer into smaller ones, and build irps and send them down. I was thinking to do something link this:

Spit original buffer into small buffers
while more buffers to send
{

build irp
send irp
wait for completion
}

Thanks,
Radu Vornicu

Mark_Roddy · December 20, 2007, 12:45pm

Because you are going to potentially be at raised IRQL you need to do this
asynchronously, maintaining state across multiple operations that use
IoAllocateIrp, IoAllocateMdl, and IoBuildPartialMdl to create each new
request and that use IoSetCompletionRoutine and a completion handler to
continue and/or complete the entire series of requests. This technique is
illustrated somewhat crptucally in the classpnp sample (see
xferpkt.cSubmitTransferPacket() for instance.)

On Dec 20, 2007 12:11 PM, wrote:

> Thanks for the replays,
>
> - I build a new irp, because i need to change the buffer from the
> original IRP. ( i’ve done the method make mdl, replace mdl, Iocomplete put
> original mdl back and now i try to learn how to make this work to.)
> - The problem is that i may need to split the buffer into smaller ones,
> and build irps and send them down. I was thinking to do something link this:
>
> Spit original buffer into small buffers
> while more buffers to send
> {
> - build irp
> - send irp
> - wait for completion
> }
>
>
> Thanks,
> Radu Vornicu
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

–
Mark Roddy

Mark_Roddy · December 20, 2007, 12:46pm

crptucally - that ought to have been cryptically

On Dec 20, 2007 12:44 PM, Mark Roddy wrote:

> Because you are going to potentially be at raised IRQL you need to do this
> asynchronously, maintaining state across multiple operations that use
> IoAllocateIrp, IoAllocateMdl, and IoBuildPartialMdl to create each new
> request and that use IoSetCompletionRoutine and a completion handler to
> continue and/or complete the entire series of requests. This technique is
> illustrated somewhat crptucally in the classpnp sample (see xferpkt.cSubmitTransferPacket() for instance.)
>
>
> On Dec 20, 2007 12:11 PM, wrote:
>
> > Thanks for the replays,
> >
> > - I build a new irp, because i need to change the buffer from the
> > original IRP. ( i’ve done the method make mdl, replace mdl, Iocomplete put
> > original mdl back and now i try to learn how to make this work to.)
> > - The problem is that i may need to split the buffer into smaller ones,
> > and build irps and send them down. I was thinking to do something link this:
> >
> > Spit original buffer into small buffers
> > while more buffers to send
> > {
> > - build irp
> > - send irp
> > - wait for completion
> > }
> >
> >
> > Thanks,
> > Radu Vornicu
> >
> >
> >
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
> >
> >
>
>
>
> –
> Mark Roddy

–
Mark Roddy

David_R_Cattley · December 20, 2007, 4:07pm

> crptucally - that ought to have been cryptically

Ah! But it was

Happy Holidays

-dave

OSR_Community_User · December 21, 2007, 4:10am

Thanks for the replays,

Currently i’m only trying to send one irp, and when that is done, i’ll try to split the buffer.
I’ve done what Don suggested and all is working great, until IoBuildAsynchronousFsdRequest returns NULL. IoFreeIrp is called in my completion function and is freeing m irp ( used windbg to test that ), but i don’t know if i need to free the MDL my self, or IoFreeIrp is doing that to.
Am I dealing with a memory leak, or is there something else that i’m missing?
In PoolMon, what memory tags should i watch for? ( Irp?, Mdl? )

Thanks and HappyHolidays,
Radu Vornicu

OSR_Community_User · December 21, 2007, 9:10am

Update:
After reading http://support.microsoft.com/kb/326315, i found that i need to free the MDL my self. I’ve done that using the code below and now i don’t get why after IoFreeMdl, Irp->MdlAddress isn’t NULL

if (Irp->MdlAddress != NULL) {
for (mdl = Irp->MdlAddress; mdl != NULL; mdl = nextMdl) {
nextMdl = mdl->Next;
MmUnlockPages( mdl ); IoFreeMdl( mdl ); // This function will also unmap pages.
}
if( Irp->MdlAddress != NULL )
DbgBreakPoint();
}

Doug_Swallow · December 21, 2007, 9:46am

Vornicu, look at the sample code on http://support.microsoft.com/kb/326315 again; you’ll see that they’re explicitly setting Irp->MdlAddress to NULL after freeing the MDL. Nothing will set it for you; just like heap allocations, a pointer is just a number that can be valid or invalid – allocated or not.

Maxim_S_Shatskih · December 21, 2007, 2:28pm

Why not reuse the IRP instead of allocating a second IRP? if the IRP’s
stack count is OK - you can reuse it.

Why not do the things in an async way, remove the event and the wait, and
do the postprocessing (if any) in the completion routine instead of “after
IoCallDriver and the wait”?

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

wrote in message news:xxxxx@ntdev…
> Please tell me if this aproach is good:
>
> DispatchFunctionWrite( …PIRP Irp… )
> {
> PIRP MyIrp
> KEVENT Event;
>
> - Init event
> - Build irp using IoBuildAsynchronousFsdRequest ( MyIrp )
> - Set completion for MyIrp ( pass event as context )
> - Send MyIrp to the next device
> - if STATUS_PENDING return then KeWaitForSingleObject( Event )
>
> - Set IOSB for Irp ( original one ) to STATUS_SUCCESS
> - Call IoCompleteRequest( Irp )
> }
>
> IoCompletion( …Irp, Context … )
> {
> - if irp pending then setEvent ( PKEVENT(Context)… )
> - FreeIrp ( Irp );
> - return status_more_processing_required;
> }
>
> Thanks,
> Radu Vornicu
>

OSR_Community_User · January 8, 2008, 8:05am

Update:
New approach - new problems

The approach:

start a thread
Irp Write Dispach inserts all the irps into my list that is parsed by the thread function and then sets the event that the thread function is waiting on
Thread function:
get irp out of the list ( head of the list )
get buffer using MmGetSystemAddressForMdlSafe
build my buffer
( if there is no space to allocate my buffer, then insert the irp back in the list and start again )
copy old buffer into my buffer
build irp using IoBuildAsynchronousFsdRequest
save my buffer into MyIrp->UserBuffer ( my buffer is define in the dispach function )
send my irp
CompletionFunction:
complete original irp
delete my buffer
delete MyIrp->MdlAddress ( using the method from here http://support.microsoft.com/kb/326315 )
Free MyIrp

This is working until MmGetSystemAddressForMdlSafe is returning NULL.

According to the help: “If the pages are not already mapped to system space and the attempt to map them fails, NULL is returned.”. If this is the case, how do i access the buffer?
After reading the forum archives i found out that this may be also caused by an memory leak or b being out of system PTEs.

How can i find out what is the problem?

the error occurred when i was copying a file
!memusage returned:
Compiling memory usage data (99% Complete).
Zeroed: 0 ( 0 kb)
Free: 0 ( 0 kb)
Standby: 96325 (385300 kb)
Modified: 3090 ( 12360 kb)
ModifiedNoWrite: 0 ( 0 kb)
Active/Valid: 30638 (122552 kb)
Transition: 904 ( 3616 kb)
Bad: 0 ( 0 kb)
Unknown: 0 ( 0 kb)
TOTAL: 130957 (523828 kb)
Building kernel map
Finished building kernel map
Invalid PTE framebase - (09% complete)
PFN 00002F06 at address 818524A8
flink 00000000 blink / share count 00000001 pteaddress 00017831
reference count 0001 Cached color 0
restore pte 00000080 containing page FFEDCB Active RW
ReadInProgress WriteInProgress
PFN 00FFEDCB at address 9D7E0234
flink 00000000 blink / share count 00000000 pteaddress 00000000
reference count 0000 NonCached color 0
restore pte 00000000 containing page 000000 Zeroed
… other 14 Invalid PTE framebase
Usage Summary (in Kb):
Control Valid Standby Dirty Shared Locked PageTables name
82b0e270 480 385044 10624 0 64 0 mapped_file( test.dat )

Mark_Roddy · January 8, 2008, 9:56am

I’d start with !sysptes.

If you are literally just copying old buffer to new buffer you can avoid
MmGetSystemAddressForMdlSafe entirely by using IoBuildPartialMdl. You would
only need the system virtual address if you are modifying the data.

On Jan 8, 2008 8:02 AM, wrote:

> Update:
> New approach - new problems
>
> The approach:
> - start a thread
> - Irp Write Dispach inserts all the irps into my list that is parsed by
> the thread function and then sets the event that the thread function is
> waiting on
>
> - Thread function:
> - get irp out of the list ( head of the list )
> - get buffer using MmGetSystemAddressForMdlSafe
> - build my buffer
> ( if there is no space to allocate my buffer, then insert the irp back in
> the list and start again )
> - copy old buffer into my buffer
> - build irp using IoBuildAsynchronousFsdRequest
> - save my buffer into MyIrp->UserBuffer ( my buffer is define in the
> dispach function )
> - send my irp
>
> - CompletionFunction:
> - complete original irp
> - delete my buffer
> - delete MyIrp->MdlAddress ( using the method from here
> http://support.microsoft.com/kb/326315 )
> - Free MyIrp
>
> This is working until MmGetSystemAddressForMdlSafe is returning NULL.
> - According to the help: “If the pages are not already mapped to system
> space and the attempt to map them fails, NULL is returned.”. If this is the
> case, how do i access the buffer?
> - After reading the forum archives i found out that this may be also
> caused by an memory leak or b being out of system PTEs.
>
> How can i find out what is the problem?
>
> - the error occurred when i was copying a file
> - !memusage returned:
> Compiling memory usage data (99% Complete).
> Zeroed: 0 ( 0 kb)
> Free: 0 ( 0 kb)
> Standby: 96325 (385300 kb)
> Modified: 3090 ( 12360 kb)
> ModifiedNoWrite: 0 ( 0 kb)
> Active/Valid: 30638 (122552 kb)
> Transition: 904 ( 3616 kb)
> Bad: 0 ( 0 kb)
> Unknown: 0 ( 0 kb)
> TOTAL: 130957 (523828 kb)
> Building kernel map
> Finished building kernel map
> Invalid PTE framebase - (09% complete)
> PFN 00002F06 at address 818524A8
> flink 00000000 blink / share count 00000001 pteaddress 00017831
> reference count 0001 Cached color 0
> restore pte 00000080 containing page FFEDCB Active RW
> ReadInProgress WriteInProgress
> PFN 00FFEDCB at address 9D7E0234
> flink 00000000 blink / share count 00000000 pteaddress 00000000
> reference count 0000 NonCached color 0
> restore pte 00000000 containing page 000000 Zeroed
> … other 14 Invalid PTE framebase
> Usage Summary (in Kb):
> Control Valid Standby Dirty Shared Locked PageTables name
> 82b0e270 480 385044 10624 0 64 0 mapped_file( test.dat )
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

–
Mark Roddy

OSR_Community_User · January 8, 2008, 1:04pm

Thanks for the replay,

currently i'm doing this:
...
pBufferCrypt = (LPBYTE)mem_alloc( bufferLength );
pBufferOrig = MmGetSystemAddressForMdlSafe( Irp->MdlAddress, NormalPagePriority )
mem_copy( pBufferCrypt, pBufferOrig, bufferLength );
// Encrypt( pBufferCrypt); ( this will be done after i get this working )
irp = IoBuildAsynchronousFsdRequest(... pBufferCrypt, ... );
...

i think that if i switch to IoBuildPartialMdl i'll need to use IoAllocateIrp.
while searching the forum i found another idea: "allocate some reserve mappings during initialization using MmAllocateMappingAddress"

Off-topic:
what books on Driver Development / Debug do you recommend?

Building Irp using IoBuildAsynchronousFsdRequest

Thanks, Radu Vornicu

Thanks,
Radu Vornicu