Calling FltReadFile in post read operation callback routine.

OSR_Community_User · November 5, 2009, 1:18am

Hello!

Can you provide me a guideline on how to call FltReadFile in post read operation callback routine?

My driver, in the post read operation callback routine, needs to read a certain amount of data from the same file but at a different offset . My questions are,

Can I use the intercetped read operation’s FltObject and Instance as the parameters to call FltReadFile? They are stored as Data->Iopb->TargetFileObject and Data->Iopb->TargetInstance. In this way, I don’t need to call FltCreateFile and FltClose in order to get FileObject.
How to set the “Flags” input parameters in order to avoid deadlocks or other undesirable situations? Is FLTFL_IO_OPERATION_NON_CACHED|FLTFL_IO_OPERATION_DO_NOT_UPDATE_BYTE_OFFSET good enough?
FltReadFile requires IRQL passiv_level. How to accomodate this requirement but not to fail the functionality? I tried to modify the WDK swapbuffer example, and use FltDoCompletionProcessingWhenSafe to let a worker thread to read data at a safe IRQL level. However, sometimes the post failed because the intercepted read IRP is paging IO. For memory mapping file, I need to handle paging io.
It seems that FltDoCompletionProcessingWhenSafe is not the solution for my case. Then, if I pend the read IO at its post operation, should I use FltQueueDeferredIoWorkItem or cancel safe queue? (Forgive me if I am wrong about these two.)

I also tried calling FltReadFile, without pending the io, in post read operation callback routinue. For a small file, it had been successful. However, when read a big file, the FltReadFile in the callback failed with error 0xc0000128.

Please advise. Thank you in advance.

Heidi

OSR_Community_User · November 5, 2009, 9:14pm

Hello! All,

I decided to try the way that pend the post read operation, queue it using cancel safe queue, and let a worker thread to read the data and then complete the intercepted operation.

So sad .... I got a BSOD. The result of "analyze -v" is at the end of the file. The cause seems be that I use Mutex to protect the queue and it reqires IRQL<=APC_LEVEL in order to call FltCbdqInsertIo.

Should I use spin lock instead to fix this problem? However, many posts here seem not recommend spin lock.

If I still use the mutex, that means I have to give up this read operation if IRQL level is not <=APC_LEVEL, which can cause my driver functionlity wrong.

If I will not pend the post read operation, I still can't carry on my driver's functionality because FltReadFile requires PASSIVE_LEVEL.

Can any one here tell me what I should do? Looking forward to hearing from you soon! I am quite stuck now.
Thank you.
Heidi

=============================================================================================
kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

ATTEMPTED_SWITCH_FROM_DPC (b8)
A wait operation, attach process, or yield was attempted from a DPC routine.
This is an illegal operation and the stack track will lead to the offending
code and original DPC routine.
Arguments:
Arg1: 00000000, Original thread which is the cause of the failure
Arg2: 00000000, New thread
Arg3: 00000000, Stack address of the original thread
Arg4: 00000000

Debugging Details:

DEFAULT_BUCKET_ID: DRIVER_FAULT

BUGCHECK_STR: 0xB8

PROCESS_NAME: System

LAST_CONTROL_TRANSFER: from 804f7bad to 80527bec

STACK_TEXT:
f78b2750 804f7bad 00000003 f78b2aac 00000000 nt!RtlpBreakWithStatusInstruction
f78b279c 804f879a 00000003 862019f8 863b2020 nt!KiBugCheckDebugBreak+0x19
f78b2b7c 804f8ca0 000000b8 00000000 00000000 nt!KeBugCheck2+0x574
f78b2b9c 80541a47 000000b8 f78b2cd0 00000202 nt!KeBugCheck+0x14
f78b2bac 805417bf f78b2bd0 86200858 00000000 nt!SwapContext+0x157
f78b2bc0 804fb4a7 8055b17c 00000000 f78b2be4 nt!KiUnlockDispatcherDatabase+0x77
f78b2bd0 80534a7b 8055b17c 86013528 86200858 nt!KeInsertQueue+0x25
f78b2be4 bae324c9 86013528 00000001 86158b8c nt!ExQueueWorkItem+0x1b
f78b2bf8 f7887faa 86013528 86200858 f7887e56 fltMgr!FltQueueGenericWorkItem+0x49
f78b2c20 bae3222d 86013528 861bbe64 00000000 myMini!MyMiniCsqInsertIo+0x5a
f78b2c38 bae32c61 86158b4c 867e8e48 00000000 fltMgr!FltpInsertIrp+0x1d
f78b2c5c bae3231a 86158b4c 867e8e48 861ee2c0 fltMgr!FltpIoCsqInsertIrpEx+0x49
f78b2c7c f78876ea 86158b4c 861bbe64 861ee2c0 fltMgr!FltCbdqInsertIo+0x2c
f78b2ca0 f7887c5b f78b2d04 861ccc40 00000000 myMini!PendingReadIOForEncryption+0x7a
f78b2ce0 bae2bef3 861bbe64 f78b2d04 861ccc40 myMini!MyMiniPostReadBuffers+0x37
f78b2d48 bae2e338 001bbe08 867e8e48 861bbe08 fltMgr!FltpPerformPostCallbacks+0x1c5
f78b2d5c bae2e867 861bbe08 f78b2df4 f78b2d90 fltMgr!FltpProcessIoCompletion+0x10
f78b2d6c 8064d8b0 8607f020 867e8e48 861bbe08 fltMgr!FltpPassThroughCompletion+0x89
f78b2d90 804f06ae 8607f020 867e8e48 f78b2df4 nt!IovpLocalCompletionRoutine+0xb4
f78b2dc0 8064dd38 861df260 8637cb70 00000000 nt!IopfCompleteRequest+0xa2
f78b2e2c f74c7bf4 f78b2e5c f74c7ed4 8637cab8 nt!IovCompleteRequest+0x9a
f78b2e34 f74c7ed4 8637cab8 867e8e48 00000001 CLASSPNP!ClassCompleteRequest+0x11
f78b2e5c 8064d8b0 00000000 86d50f48 861df260 CLASSPNP!TransferPktComplete+0x180
f78b2e80 804f06ae 00000000 86d50f48 f78b2ee4 nt!IovpLocalCompletionRoutine+0xb4
f78b2eb0 8064dd38 86160008 86160008 861df30c nt!IopfCompleteRequest+0xa2
f78b2f1c bae4f8f8 863bcae8 86d50f48 f78b2f60 nt!IovCompleteRequest+0x9a
f78b2f2c bae4f436 86160008 00000001 00000000 SCSIPORT!SpCompleteRequest+0x5e
f78b2f60 bae4f6f7 863bcae8 86160008 f78b2fcf SCSIPORT!SpProcessCompletedRequest+0x632
f78b2fd0 80541b9d 863bcaa4 863bca30 00000000 SCSIPORT!ScsiPortCompletionDpc+0x2b5
f78b2ff4 8054186a f59a4934 00000000 00000000 nt!KiRetireDpcList+0x46
f78b2ff8 f59a4934 00000000 00000000 00000000 nt!KiDispatchInterrupt+0x2a
WARNING: Frame IP not in any known module. Following frames may be wrong.
8054186a 00000000 00000009 bb835675 00000128 0xf59a4934

STACK_COMMAND: kb

FOLLOWUP_IP:
myMini!myMiniCsqInsertIo+5a
f7887faa 8bd8 mov ebx,eax

FAULTING_SOURCE_CODE:
1060: Status = FltQueueGenericWorkItem( WorkItem,
1061: InstCtx->Instance,
1062: MyMiniPendingIOWorkItemRoutine,
1063: DelayedWorkQueue,

1064: InstCtx->Instance );
1065:
1066: if (!NT_SUCCESS(Status)) {
1067: LOG_PRINT(LOGFL_ERRORS,
1068: ("[MYMINI]: Failed to queue the work item (Status = 0x%x)\n",
1069: Status) );

SYMBOL_STACK_INDEX: 9

SYMBOL_NAME: myMini!MyMiniCsqInsertIo+5a

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: myMini

IMAGE_NAME: myMini.sys

DEBUG_FLR_IMAGE_TIMESTAMP: 4af36a9b

FAILURE_BUCKET_ID: 0xB8_VRF_myMini!MyMiniCsqInsertIo+5a

BUCKET_ID: 0xB8_VRF_myMini!MyMiniCsqInsertIo+5a

Followup: MachineOwner

OSR_Community_User · November 5, 2009, 9:27pm

The answer is that you cannot use a mutex if you are protecting
something that can be accessed in a DPC routine. Use a spin lock.

Tony
OSR

OSR_Community_User · November 6, 2009, 12:43pm

can I have more than one cancel safe queue in one driver instance? In my case, one for pre-create, one for pre-write and one for post-read. For the first two, still use mutex since I didn’t run into BSOD for them; for the latter, use spin lock. In this way, I can gain better performance because of using small lock. Does this make sense?

BTW, I read again about FltQueueDeferredIoWorItem on msdn (http://msdn.microsoft.com/en-us/library/aa488733(loband).aspx). It seem not a solution for my case because msdn says “FltQueueDeferredIoWorkItem cannot post a paging I/O operation to a worker thread.” and “Caution: To avoid deadlocks, a minifilter must not post an I/O operation to a system work queue in the post-operation callback for any I/O operations that a driver can complete directly in the storage stack, such as the following:”. For some reason, the sentence got cut off but I guess IRP_MJ_READ is one of them. Please advise if I am wrong.

OSR_Community_User · November 6, 2009, 2:03pm

> can I have more than one cancel safe queue in one driver instance?

Yes.

for the latter, use spin lock.

Any CSQ must use a spin lock, since cancellation can arrive at == DISPATCH

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

David_J_Craig · November 6, 2009, 2:27pm

Read the documentation for the callbacks. If they ‘can’ run at an IRQL of
DISPATCH_LEVEL or higher, you cannot use a mutex. I believe that pre-create
will be passive level, but all reads and writes may not have such a
restriction. Paging files come to mind as a problem.

If the documentation does not list the specific IRQL limits, then click on
the ‘feedback’ link and tell the doc writers about the omission.

wrote in message news:xxxxx@ntfsd…
> can I have more than one cancel safe queue in one driver instance? In my
> case, one for pre-create, one for pre-write and one for post-read. For the
> first two, still use mutex since I didn’t run into BSOD for them; for the
> latter, use spin lock. In this way, I can gain better performance because
> of using small lock. Does this make sense?
>
> BTW, I read again about FltQueueDeferredIoWorItem on msdn
> (http://msdn.microsoft.com/en-us/library/aa488733(loband).aspx). It
> seem not a solution for my case because msdn says
> “FltQueueDeferredIoWorkItem cannot post a paging I/O operation to a worker
> thread.” and “Caution: To avoid deadlocks, a minifilter must not post an
> I/O operation to a system work queue in the post-operation callback for
> any I/O operations that a driver can complete directly in the storage
> stack, such as the following:”. For some reason, the sentence got cut off
> but I guess IRP_MJ_READ is one of them. Please advise if I am wrong.
>

OSR_Community_User · November 6, 2009, 4:05pm

Thank all very much.

However, I think that I am back to squre one after reading http://download.microsoft.com/download/f/0/5/f05a42ce-575b-4c60-82d6-208d3754b2d6/ManagingIo.ppt. It says “Used only when pending preOperation callbacks” for cancel safe queuing. Is it ture? (The document was back to 2004.)

Then I am stuck since I can’t pend paging-IO post read operation in order to get better throughput. Neither deferred IO queuing nor cancel safe queuing support. If I don’t pend the post read operation, I still can’t carry on my driver’s functionality because FltReadFile requires PASSIVE_LEVEL.

Please advise me!

OSR_Community_User · November 6, 2009, 6:31pm

Hello Heidi,

I have no idea why the document says that. I’ve looked at the code and I think it should work. I’ve checked with Neal who wrote the document and he couldn’t remember why that line is there either… So I would say you should try this but make sure to test it thoroughly.

Regards,
Alex.
This posting is provided “AS IS” with no warranties, and confers no rights.

OSR_Community_User · November 6, 2009, 8:08pm

Thank all OSROnlines for the help.

With Alex’s clarification, I tried with cancel safe queue and queued spin lock. Unfortunately, I got another BSOD. It seems that the cause is the function FltGetInstanceContext requires IRQL <= APC_LEVEL. The reason that I call this function is to get the cancel safe queue stored in instance context.

Sigh …

In this case, what I did,

If the IRQL level is greater then APC_LEVEL, I am going to pend this i/o using FltDoCompletionProcessingWhenSafe.
If the IRQL level is <= APC_LEVEL, I am going to pend this i/o using cancel safe queue.

Does this make sense?

I tried in this way. However, sometime I saw the error that FltDoCompletionProcessingWhenSafe failed to post the operation ALTHOUGH the user mode application (e.g. notepad) can open the file.

Based on your experience, how should i handle pending post read operation correctly?

Thank you for any help and advice.

Heidi