Question on IRP_MJ_FLUSH_BUFFERS behavior

NTFSD
1

The FlushFileBuffers() will generate IRP_MJ_FLUSH_BUFFERS.
Also, I believe the FlushFileBuffers() API is blocking and will not return until all dirty pages are flushed.

I have a question about IRP_MJ_FLUSH_BUFFERS from minifilter perspective.

Will the PostOperation of IRP_MJ_FLUSH_BUFFERS will not be called until all pages are flushed?

I observed that in my minifilter PostOp gets called (i.e the IRP is completed by NTFS) but no writes coming in before this.

I believe I should see paging writes after PreOperation of IRP_MJ_FLUSH_BUFFERS. But I do not see such writes. Most of the times the writes come on closing the file.

ProcMon also shows similar behavior.
Looked at FASTFAT, which calls CcFlushCache() for the file, so the IRP should be blocking and paging writes should come after PreOp.

To make my test more clear, I mmapped a entire file (>10MB) and updated all mmaped buffer and then did FlushFileBuffer().

So the questions are:

  • FlushFileBuffers() is blocked until all dirty pages are flushed to disk?
  • Is IRP_MJ_FLUSH_BUFFERS blocking? and the PostOperation of IRP_MJ_FLUSH_BUFFERS will not be called until all pages are flushed?
  • Does NTFS behaves differently?

Thanks
Rohan

2

I think each file system will implement this MJ as it wants so I think you
should not make many assumption about it. The thing is that the
documentation says what a filter should do and FSs usually do but you
cannot know for sure what NTFS does and what other file system will do when
this is received.
I would interpret the documentation as it is. So in you scenario when you
do a map and afterwards you flush, you see no writes (yet) but pbably your
next read, will generate the writes behind and then the read, or another FS
will do things differently so when you see the MJ just consider it flushed
if it succeeds.

On Thu, May 14, 2015 at 2:17 PM, wrote:

> The FlushFileBuffers() will generate IRP_MJ_FLUSH_BUFFERS.
> Also, I believe the FlushFileBuffers() API is blocking and will not return
> until all dirty pages are flushed.
>
> I have a question about IRP_MJ_FLUSH_BUFFERS from minifilter perspective.
>
> Will the PostOperation of IRP_MJ_FLUSH_BUFFERS will not be called until
> all pages are flushed?
>
> I observed that in my minifilter PostOp gets called (i.e the IRP is
> completed by NTFS) but no writes coming in before this.
>
> I believe I should see paging writes after PreOperation of
> IRP_MJ_FLUSH_BUFFERS. But I do not see such writes. Most of the times the
> writes come on closing the file.
>
> ProcMon also shows similar behavior.
> Looked at FASTFAT, which calls CcFlushCache() for the file, so the IRP
> should be blocking and paging writes should come after PreOp.
>
> To make my test more clear, I mmapped a entire file (>10MB) and updated
> all mmaped buffer and then did FlushFileBuffer().
>
> So the questions are:
> - FlushFileBuffers() is blocked until all dirty pages are flushed to disk?
> - Is IRP_MJ_FLUSH_BUFFERS blocking? and the PostOperation of
> IRP_MJ_FLUSH_BUFFERS will not be called until all pages are flushed?
> - Does NTFS behaves differently?
>
> Thanks
> Rohan
>
> —
> NTFSD is sponsored by OSR
>
> OSR is hiring!! Info at http://www.osr.com/careers
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

–
Bercea. G.

3

On some older Windows, RAW have not implement this, so, FlushFileBuffers on unformatted volume was an error.
“Gabriel Bercea” wrote in message news:xxxxx@ntfsd…
I think each file system will implement this MJ as it wants so I think you should not make many assumption about it. The thing is that the documentation says what a filter should do and FSs usually do but you cannot know for sure what NTFS does and what other file system will do when this is received.

I would interpret the documentation as it is. So in you scenario when you do a map and afterwards you flush, you see no writes (yet) but pbably your next read, will generate the writes behind and then the read, or another FS will do things differently so when you see the MJ just consider it flushed if it succeeds.

On Thu, May 14, 2015 at 2:17 PM, wrote:

The FlushFileBuffers() will generate IRP_MJ_FLUSH_BUFFERS.
Also, I believe the FlushFileBuffers() API is blocking and will not return until all dirty pages are flushed.

I have a question about IRP_MJ_FLUSH_BUFFERS from minifilter perspective.

Will the PostOperation of IRP_MJ_FLUSH_BUFFERS will not be called until all pages are flushed?

I observed that in my minifilter PostOp gets called (i.e the IRP is completed by NTFS) but no writes coming in before this.

I believe I should see paging writes after PreOperation of IRP_MJ_FLUSH_BUFFERS. But I do not see such writes. Most of the times the writes come on closing the file.

ProcMon also shows similar behavior.
Looked at FASTFAT, which calls CcFlushCache() for the file, so the IRP should be blocking and paging writes should come after PreOp.

To make my test more clear, I mmapped a entire file (>10MB) and updated all mmaped buffer and then did FlushFileBuffer().

So the questions are:
- FlushFileBuffers() is blocked until all dirty pages are flushed to disk?
- Is IRP_MJ_FLUSH_BUFFERS blocking? and the PostOperation of IRP_MJ_FLUSH_BUFFERS will not be called until all pages are flushed?
- Does NTFS behaves differently?

Thanks
Rohan

—
NTFSD is sponsored by OSR

OSR is hiring!! Info at http://www.osr.com/careers

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

–

Bercea. G.

4

Thanks Gabriel and Maxim.

From application perspective after it calls FlushFileBuffers(), should it assume all necessary data is flushed to the disk? The MSDN docs suggests so.

Also, do you mean to say that blocking nature of FlushFileBuffers() is implemented in that API and not in underline FS? Which I think is unlikely.

So minifilter would get PreOp, pass it down to FS, FS will do necessary writes and then complete the IRP. So minifilter would see PostOp of the IRP.

But I don’t see any writes before PostOp. I also tried to normal ReadFile() just to give hint to the cache manager to init the caching.

Am I missing anything?