FlushFileBuffers() failing on NTFS partition.

NTFSD
1

Gday All,

We are facing a problem of data inconsistency on using
FlushFileBuffers() on NTFS partitions. We need to run “chkdsk” utility
to bring the data
in Sync.

I have two applications both of whom access the drive through RAW mode
(using CreateFile() API with the parameter \\.\g:.) One application
writes to the drive (writer app) and the other application reads from
the drive.(reader app)

The problem arises when I try to see the contents of the drive through
Windows Explorer. It does not show me the writes done by the writer
application. I tried to issue a FlushFileBuffers() on the file handle
of the writer app before checking the contents through Windows
Explorer but was not successful. If I try the same set of operations
on a FAT FS it succeeds and data can be seen in Explorer.

Interestingly if I lock the drive using
(DeviceIoControl(hVolume, FSCTL_LOCK_VOLUME, NULL, 0, NULL, 0,
&dwBytesReturned, NULL))
and then unlock it using
DeviceIoControl(hVolume, FSCTL_UNLOCK_VOLUME, NULL, 0, NULL, 0,
&dwBytesReturned, NULL))
where hVolume = File Descriptor of writer app.
data is found to be consistent even on NTFS partitions. ( I am not
reading or writing data when obtaining the lock)

Could anyone please let me know why FlushFileBuffers() is failing and
how does data comes in sync on issuing “chkdsk” for NTFS partition ?

Thanks,

  • Saurabh
2

While are you writing to a raw volume? what can be the purpose of doing
this?

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “saurabh agrawal”
To: “Windows File Systems Devs Interest List”
Sent: Wednesday, March 30, 2005 5:32 PM
Subject: [ntfsd] FlushFileBuffers() failing on NTFS partition.

> Gday All,
>
> We are facing a problem of data inconsistency on using
> FlushFileBuffers() on NTFS partitions. We need to run “chkdsk” utility
> to bring the data
> in Sync.
>
> I have two applications both of whom access the drive through RAW mode
> (using CreateFile() API with the parameter \\.\g:.) One application
> writes to the drive (writer app) and the other application reads from
> the drive.(reader app)
>
> The problem arises when I try to see the contents of the drive through
> Windows Explorer. It does not show me the writes done by the writer
> application. I tried to issue a FlushFileBuffers() on the file handle
> of the writer app before checking the contents through Windows
> Explorer but was not successful. If I try the same set of operations
> on a FAT FS it succeeds and data can be seen in Explorer.
>
> Interestingly if I lock the drive using
> (DeviceIoControl(hVolume, FSCTL_LOCK_VOLUME, NULL, 0, NULL, 0,
> &dwBytesReturned, NULL))
> and then unlock it using
> DeviceIoControl(hVolume, FSCTL_UNLOCK_VOLUME, NULL, 0, NULL, 0,
> &dwBytesReturned, NULL))
> where hVolume = File Descriptor of writer app.
> data is found to be consistent even on NTFS partitions. ( I am not
> reading or writing data when obtaining the lock)
>
>
> Could anyone please let me know why FlushFileBuffers() is failing and
> how does data comes in sync on issuing “chkdsk” for NTFS partition ?
>
> Thanks,
> - Saurabh
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

3

The data (and/or meta-data) is cached. Locking and unlocking the volume
causes NTFS to purge its cache. Since the cache is purged and reloaded
you see correct behavior. Flush buffers does not PURGE the cache, it
merely forces data that needs to be written (from memory) is written.

Of course locking a volume will fail if anything has a file open on the
volume. This is EXACTLY what chkdsk does when it tries to check the
volume: it tries to lock and if that fails schedules a chkdisk for the
next reboot.

As Max noted, however, this isn’t necessarily the best way to achieve
functionality, so if you give us more context for what you are doing we
might be able to suggest alternatives…

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Looking forward to seeing you at the Next OSR File Systems Class April
4, 2005 in Boston!

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of saurabh agrawal
Sent: Wednesday, March 30, 2005 8:32 AM
To: ntfsd redirect
Subject: [ntfsd] FlushFileBuffers() failing on NTFS partition.

Gday All,

We are facing a problem of data inconsistency on using
FlushFileBuffers() on NTFS partitions. We need to run “chkdsk” utility
to bring the data
in Sync.

I have two applications both of whom access the drive through RAW mode
(using CreateFile() API with the parameter \\.\g:.) One application
writes to the drive (writer app) and the other application reads from
the drive.(reader app)

The problem arises when I try to see the contents of the drive through
Windows Explorer. It does not show me the writes done by the writer
application. I tried to issue a FlushFileBuffers() on the file handle
of the writer app before checking the contents through Windows
Explorer but was not successful. If I try the same set of operations
on a FAT FS it succeeds and data can be seen in Explorer.

Interestingly if I lock the drive using
(DeviceIoControl(hVolume, FSCTL_LOCK_VOLUME, NULL, 0, NULL, 0,
&dwBytesReturned, NULL))
and then unlock it using
DeviceIoControl(hVolume, FSCTL_UNLOCK_VOLUME, NULL, 0, NULL, 0,
&dwBytesReturned, NULL))
where hVolume = File Descriptor of writer app.
data is found to be consistent even on NTFS partitions. ( I am not
reading or writing data when obtaining the lock)

Could anyone please let me know why FlushFileBuffers() is failing and
how does data comes in sync on issuing “chkdsk” for NTFS partition ?

Thanks,

  • Saurabh

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

4

Gday All,
Thanks a lot for the replies. I was mistaken that FlushFileBuffers()
also invalidates the cache. Here are the details of my system.

I have a disk mirroring application for which I need to open the
target drive in RAW mode. Data is written to the target partition in
this RAW mode.
Until now I was getting the handle to my target drive with an
exclusive lock so that no other application can access it. Whenever I
had to check the contents of my target in Explorer I would call
FSCTL_LOCK_VOLUME and FSCTL_UNLOCK_VOLUME and then release the handle.
This way the contents of my target drive were always valid/in Sync.

Now I want to extend the mirroring to 2 target partitions in linked
list way i.e. drive A to drive B and from drive B to drive C. Thus now
I would need both the writer app and reader app to indenpendently work
on B and at the same time. Thus I cannot obtain an exclusive lock on
B. This means that drive B can be viewed in the Explorer.

If I allow the user to view the contents of B in Windows Explorer
during the mirroring operation it would show that data is not in sync.
I want to somehow mask my drive from Windows Explorer and yet continue
to access it by reader app and writer app.

Thanks a lot,

  • Saurabh
    ( I had earlier assumed that giving a FlushFileBuffers() operation
    would sync its data with the explorer and so I can ask the user to run
    a script just before checking the contents in Windows explorer. The
    script would just containg FlushFileBuffers(). )