Atomicity in ZWWriteFile and ZWReadFile.

I am writing at the end of file in my filter driver and I am reading from user space.

Scenario is like whenever a rename request comes then just write the name into a log file I am opening file with FILE_APPEND_DATA. I always write at the end of file.

And at the same time there may be a read happening on the same file from user space like it reads line by line (the name of file).

Q1:: Is it possible that I may read partial data because read and write may happen at the same time. ?

Do I need synchronization between write’s? Like multiple rename may be happening, so is it possible the data I am writing may get intermingled for eg
First name of file : AAAAAA.txt
Second name of file : BBBBBB.txt

So is it possible that I get the name in file as AAAABBB(corrupted data)?

Q2:: Is it possible that file name gets intermingled?? or file system handles this?

I read about FILE_APPEND_DATA that it is atomic operation. “Write operation will not write over existing data” .And buffer in ZWWritefile will always be written to contiguous area of memory.

To test for Q2, I write an application :
Buffer size : 1MB
4 threads : Continously writing at the end of file.
1st Buffer : “AAA…” (string contains only characters ‘A’)
2nd Buffer : “BBB…” (string contains only characters ‘B’)
3rd Buffer : “CCC…” (string contains only characters ‘C’)
4th Buffer : “DDD…” (string contains only characters ‘D’)

then after all the writes are finished then read the data to check whether characters got intermingled or not. But I did not find this happening. Corruption did not happen.

To test Q1, I am writing in 1 process (1MB) with FILE_APPEND_DATA and reading from another process(1MB), I never saw the scenario where ReadFile reads less than 1MB or simply it never reads partial data. It just return from file that it reached end of file though Write is happening in another process.

So is it possible that Q1 or Q2 scenario may happen or I did the right testing?

At a high level, the answers to your questions are often “it depends upon the file system”. Having said that, I can make some observations about how things are usually done in Windows file systems.

Yes, this is possible. But it’s not something you should be fixing in your file system driver or filter. Applications that permit this and do not use synchronization are *already broken*. A fundamental precept I use in my own work is “do not focus on making broken applications work correctly”.

I find this question confusing. You are using I/O synchronization and then discussing namespace operations. The two are actually orthogonal. You will not see DATA getting mixed in files across rename operations. That’s because the control structure (FILE_OBJECT) doesn’t lose it’s association just because some external factor (the file name) changes. The WRITE is done against the control structure.

APPLICATIONS might get confused if they are opening and closing the file to perform I/O, but once again, you have a broken application then.

No, this isn’t going to happen. Rename is a discreet operation and I can’t envision a situation in which any real file system would merge in parts of two names.

No. File systems handle this sort of issue. The granularity of rename is the entire name, not sub-pieces of the name.

The key to this is that the file systems serialize the movement of the EOF marker.

Thread 1: Save oldEOF, set EOF = oldEOF + BytesToBeWriten. Write bytes.
Thread 2: Save oldEOF, set EOF = oldEOF + BytesToBeWritten, Write bytes.

The FSD will serialize (using a lock) the “save old EOF then move it out to the new location”. The write doesn’t need to be serialized.

If you want to end up with interesting patterns in your data, don’t use FILE_APPEND_DATA, use FILE_WRITE_DATA. Then randomly perform overlapping writes in the threads. Do it 100 times on different files. It is QUITE possible to end up with 100 different files, even though each one “did the same thing”. I’d call it the “Heisenberg File Systems Test”. Such a real world application is broken anway.

The file systems shouldn’t permit that to happen, so it sounds like they are doing their job.

Tony
OSR