FILE_FLAG_NO_BUFFERING

RE: [ntfsd] RE: very stuck with a bug - very long and involvedHi,

It is my understanding that when you use FILE_FLAG_NO_BUFFERING to open a file in CreateFile,
there should be no buffering between the cache manager/file system driver and the storage driver.

However, when I write and read to a file created in this way, I do not see the reads being directed to
the storage device, it is as if the data is being cached and read from a buffer, which seems completely
contradictory to the documentation.

Is this expected behaviour ?
Is there a way I can specify a file to be non-cached in all respects ?
Is there a way I can purge the cache (from User level, I have tried FlushFileBuffers, that does not seem to
purge the cache)
Any other suggestions ?
Isn’t there a way to verify that data has been written from disk ?

I am using W2K and the above only happens on an NTFS partition, FAT32 is OK, I have also tried CreateFile with FILE_FLAG_WRITE_THROUGH as well…that didn’t work either,

regards,

John

For small files NTFS will store the data in the MFT, rather than
allocate a run of clusters. Perhaps this is causing the behavior you
are seeing? What happens if your test file is big (try 100KB or more)?

-----Original Message-----
From: John Tobin [mailto:xxxxx@paceap.com]
Sent: Tuesday, June 20, 2000 3:58 PM
To: File Systems Developers
Subject: [ntfsd] FILE_FLAG_NO_BUFFERING

Hi,

It is my understanding that when you use FILE_FLAG_NO_BUFFERING to open
a file in CreateFile,
there should be no buffering between the cache manager/file system
driver and the storage driver.

However, when I write and read to a file created in this way, I do not
see the reads being directed to
the storage device, it is as if the data is being cached and read from a
buffer, which seems completely
contradictory to the documentation.

Is this expected behaviour ?
Is there a way I can specify a file to be non-cached in all respects ?
Is there a way I can purge the cache (from User level, I have tried
FlushFileBuffers, that does not seem to
purge the cache)
Any other suggestions ?
Isn’t there a way to verify that data has been written from disk ?

I am using W2K and the above only happens on an NTFS partition, FAT32 is
OK, I have also tried CreateFile with FILE_FLAG_WRITE_THROUGH as
well…that didn’t work either,

regards,

John

RE: [ntfsd] RE: very stuck with a bug - very long and involvedMy file is in the order of 1600bytes, which does seem to match what you are saying.
Is there a method for retrieving a file size that would not be stored in the MFT, (instead of just blindly choosing a “big” size) ?

BTW, what does MFT mean ?

John (Still waiting for my IFS kit)

----- Original Message -----
From: Nathan Nesbit
To: File Systems Developers
Sent: Tuesday, June 20, 2000 5:03 PM
Subject: [ntfsd] RE: FILE_FLAG_NO_BUFFERING

For small files NTFS will store the data in the MFT, rather than allocate a run of clusters. Perhaps this is causing the behavior you are seeing? What happens if your test file is big (try 100KB or more)?
-----Original Message-----
From: John Tobin [mailto:xxxxx@paceap.com]
Sent: Tuesday, June 20, 2000 3:58 PM
To: File Systems Developers
Subject: [ntfsd] FILE_FLAG_NO_BUFFERING

Hi,

It is my understanding that when you use FILE_FLAG_NO_BUFFERING to open a file in CreateFile,
there should be no buffering between the cache manager/file system driver and the storage driver.

However, when I write and read to a file created in this way, I do not see the reads being directed to
the storage device, it is as if the data is being cached and read from a buffer, which seems completely
contradictory to the documentation.

Is this expected behaviour ?
Is there a way I can specify a file to be non-cached in all respects ?
Is there a way I can purge the cache (from User level, I have tried FlushFileBuffers, that does not seem to
purge the cache)
Any other suggestions ?
Isn’t there a way to verify that data has been written from disk ?

I am using W2K and the above only happens on an NTFS partition, FAT32 is OK, I have also tried CreateFile with FILE_FLAG_WRITE_THROUGH as well…that didn’t work either,

regards,

John

RE: [ntfsd] RE: very stuck with a bug - very long and involved>BTW, what
does MFT mean ?

Master File Table. Similar to inode table in UNIX’s filesystems - contains
file/directory descriptor structures.
The differences between the UNIX layout and MFT are:

  • UNIX uses a predefined (at volume creation time) sector numbers to store
    the inode table. MFT is a file - allocated by cluster runs as any other NTFS
    file.
    There are 2 (mirrored) MFT records which describe the MFT itself.
  • MFT records are of variable size - can have continuation records.
  • small attribute streams (file data & ACL are 2 default ones, one can
    define his/her own - this feature is used by SFM file server) are stored in
    a MFT record directly instead of allocating cluster runs for them.

There are numerous Web resources which describe the NTFS on-disk layout -
though this is unofficial information from third-parties (accurate enough to
write a read-only NTFS driver for OSes other than NT).

BTW - a question to MS:

  • is there any ways to shrink the MFT, purging the free space in it (NT4SP6)
    without volume reformatting? I remember NtBugTraq message about “MFT never
    shrinks” problem - creating a multitudes of tiny files can render the whole
    volume out of space till reformatting. Is it fixed?

Max