latency issues of FltWriteFile and FltReadFile in swapbuffers PreWrite operations

NTFSD
1

Hello,

I modifiled swapbuffer example to do on-the-fly compression/decompression project recently. My PreWrite operation is on nocache and no FastIO mode as follows:

// only handle NoCache operation
if ( !FlagOn(iopb->IrpFlags, IRP_NOCACHE) ) leave;
// if it’s fast IO operation, return disallow and it will come with normal IRP commands later
if ( FLT_IS_FASTIO_OPERATION(Cbd) ) { retValue = FLT_PREOP_DISALLOW_FASTIO; leave; }

Has anyone experienced in latency issues of FltWriteFile and FltReadFile in swapbuffers PreWrite operation()?

In PreWrite operation() from swapbuffers examples, we are able to intercept, change write length and data context on-the-fly for each file write. Therefore, I can do on-the-fly compression/decompression, I chop the original write length to several small segments and compress each segment and write it to disk using FltWriteFile separately.
Sicne the data has been compressed, total length to be written will be shorter than original write length.

However, as long as I don’t write the enough same length to cover full range from original file write offset. The minfilter driver will have lantecy on disk benchmark throughput tested result. (Tested with ATTO disk benchmark tool)
Do we need to notify any layer about the size has been shorted?

Here is a easy way to duplicate the issues I mentioned, assume the data length has been compressed to 1/2. Just add following code to DDK swapbuffers example, you will see the lantecy using ATTO disk benchmark tool. My HDD throughput will drop from 60MB/sec to 48MB/sec.

> iopb->Parameters.Write.Length = (iopb->Parameters.Write.Length/2);
> FltSetCallbackDataDirty( Cbd );

And, I also found FltReadFile also will have huge lantecy in PreWrite operation() if I’m trying to read my compression header tag from same file.

Thanks a lot,

Sammy

2

>Here is a easy way to duplicate the issues I mentioned, assume the data

length has been compressed to 1/2. Just add >following code to DDK
swapbuffers example, you will see the lantecy using ATTO disk benchmark
tool. My HDD throughput >will drop from 60MB/sec to 48MB/sec.

> iopb->Parameters.Write.Length = (iopb->Parameters.Write.Length/2);
> FltSetCallbackDataDirty( Cbd );

What exactly are you measuring…Are you getting 60MB/s with a stock
swapbuffers and 48MB/s with the change? Or are you getting 60MB/s with *no*
swapbuffers and 48MB/s with the modified swapbuffers?

Also, is this the free build or the checked build? Is Driver Verifier
running? Is this real hardware or virtual? Performance analysis is hard (my
free tip of the day :)) and you have to be very explicit about what it is
you’re measuring.

And, I also found FltReadFile also will have huge lantecy in PreWrite
operation() if I’m trying to read my compression header >tag from same
file.

Any time you add I/O there is overhead involved. What does, “huge latency”
mean? There could be a lot going on here, so I wouldn’t expect a simple
answer even with a lot more data.

-scott
OSR

wrote in message news:xxxxx@ntfsd…

Hello,

I modifiled swapbuffer example to do on-the-fly compression/decompression
project recently. My PreWrite operation is on nocache and no FastIO mode as
follows:

// only handle NoCache operation
if ( !FlagOn(iopb->IrpFlags, IRP_NOCACHE) ) leave;
// if it’s fast IO operation, return disallow and it will come with normal
IRP commands later
if ( FLT_IS_FASTIO_OPERATION(Cbd) ) { retValue = FLT_PREOP_DISALLOW_FASTIO;
leave; }

Has anyone experienced in latency issues of FltWriteFile and FltReadFile in
swapbuffers PreWrite operation()?

In PreWrite operation() from swapbuffers examples, we are able to intercept,
change write length and data context on-the-fly for each file write.
Therefore, I can do on-the-fly compression/decompression, I chop the
original write length to several small segments and compress each segment
and write it to disk using FltWriteFile separately.
Sicne the data has been compressed, total length to be written will be
shorter than original write length.

However, as long as I don’t write the enough same length to cover full range
from original file write offset. The minfilter driver will have lantecy on
disk benchmark throughput tested result. (Tested with ATTO disk benchmark
tool)
Do we need to notify any layer about the size has been shorted?

Here is a easy way to duplicate the issues I mentioned, assume the data
length has been compressed to 1/2. Just add following code to DDK
swapbuffers example, you will see the lantecy using ATTO disk benchmark
tool. My HDD throughput will drop from 60MB/sec to 48MB/sec.

> iopb->Parameters.Write.Length = (iopb->Parameters.Write.Length/2);
> FltSetCallbackDataDirty( Cbd );

And, I also found FltReadFile also will have huge lantecy in PreWrite
operation() if I’m trying to read my compression header tag from same file.

Thanks a lot,

Sammy