CcFlushCache under SRV2

It’s not a bug. The reason of such behavior I described in my two previous
posts.

2 нояб. 2016 г. 12:30 AM пользователь
написал:

> A final note. My FSD exhibits the same behavior as NTFS:
>
> 1. FAILURE on Win8 when using \localhost.
>
> 2. SUCCESS on Win10 when using \localhost.
>
> 3. SUCCESS on Win10 to Win8 using a \win8dbg\users share.
>
> I think I can safely say that this is a bug in Win8 (version 6.2 Build
> 9200).
>
> Bill
>
>
> —
> NTFSD is sponsored by OSR
>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

About Win10, seems like SRV2 didn’t open files with FILE_FLAG_NO_BUFFERING
flag anymore.

2016-11-02 6:07 GMT+03:00 Anatoly Mikhailov :

> It’s not a bug. The reason of such behavior I described in my two previous
> posts.
>
> 2 нояб. 2016 г. 12:30 AM пользователь
> написал:
>
> A final note. My FSD exhibits the same behavior as NTFS:
>>
>> 1. FAILURE on Win8 when using \localhost.
>>
>> 2. SUCCESS on Win10 when using \localhost.
>>
>> 3. SUCCESS on Win10 to Win8 using a \win8dbg\users share.
>>
>> I think I can safely say that this is a bug in Win8 (version 6.2 Build
>> 9200).
>>
>> Bill
>>
>>
>> —
>> NTFSD is sponsored by OSR
>>
>>
>> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>> software drivers!
>> Details at http:
>>
>> To unsubscribe, visit the List Server section of OSR Online at <
>> http://www.osronline.com/page.cfm?name=ListServer&gt;
>>
></http:>

Anatoly wrote:

It’s not a bug. The reason of such behavior I described in my two previous
posts.

I am not sure which post you mean. If you mean the one about doing a WRITE with 0x20064 bytes I would disagree that this is the problem.

Recall that the test does not do ReadFile, WriteFile but only memory mapped accesses. Furthermore recall that the test passes when used again the file systems directly (i.e. the C: drive) rather than going through a redirector, even with the FILE_FLAG_NO_BUFFERING flag. Either the test should fail in all cases because of non-aligned WRITE’s or in none.

Furthermore I believe it is legal for a WRITE past the EndOfFile (or ValidDataLength) to not be aligned on a sector boundary. I cannot find the relevant docs right now, but FastFat certainly seems to allow it:

https://github.com/Microsoft/Windows-driver-samples/blob/master/filesys/fastfat/write.c#L2105

Notice that the if branch will be taken in two cases:

  • when StartingVbo is not correctly aligned or
  • ByteCount is not correctly aligned AND the WRITE does not extend the file

So a WRITE with an aligned buffer at an offset of zero and a non-aligned size of 0x20064 is actually legal.

Bill

> Recall that the test does not do ReadFile

paging IRP_MJ_READ always reads multiple pages, for example 4096 bytes

> Furthermore I believe it is legal for a WRITE past the EndOfFile (or
ValidDataLength) to not be aligned on a sector boundary.

legal for local file system, SRV2 is usual client of FSD, and FSD knows
nothing about real request

> Recall that the test does not do ReadFile

you could open file by yourself for example

Furthermore recall that the test passes when used again the file systems
directly (i.e. the C: drive) rather than going through a redirector, even
with the FILE_FLAG_NO_BUFFERING flag.

because PAGING_IO flag was set for IRP

> Furthermore I believe it is legal for a WRITE past the EndOfFile (or
ValidDataLength) to not be aligned on a sector boundary.

I forgot to mention: it’s legal, for PAGING_IO requests

So a WRITE with an aligned buffer at an offset of zero and a non-aligned
size of 0x20064 is actually legal.

For paging read/write

https://github.com/Microsoft/Windows-driver-samples/blob/
master/filesys/fastfat/write.c#L2105

See again carefully.

You can test it out by yourself, do ReadFile on file opened with
FILE_FLAG_NO_BUFFERING flag. ReadFile will fail.

You can do simple test via redirector:

  1. OpenFile without FILE_FLAG_NO_BUFFERING flag.
  2. ReadFile
  3. OpenFile with FILE_FLAG_NO_BUFFERING flag.
  4. Map file from step 3
  5. Fill buffer

This test must succeeded.

One more thing :slight_smile:

Furthermore I believe it is legal for a WRITE past the EndOfFile (or
ValidDataLength) to not be aligned on a sector boundary.

It’s legal for cached read/write

“When opening or creating a file with the *CreateFile*
https:
function, the FILE_FLAG_NO_BUFFERING flag can be specified to disable
system caching of data being read from or written to the file. Although
this gives complete and direct control over data I/O buffering, in the case
of files and similar devices there are data alignment requirements that
must be considered.”

as documented here:
https://msdn.microsoft.com/en-us/library/windows/desktop/cc644950(v=vs.85).aspx</https:>

Previous post was wrotten just once i woke up, so i think i wasn’t clear :slight_smile:

To summ up:

First of all when you open file with FILE_FLAG_NO_BUFFERING flag you must
consider data alignment, i.e. offset is on sector boundary and size is
multiple of sectors. Violation to this rule leads to read/write failings.

From point of view of FSD there can be serveral IRP_MJ_READ/IRP_MJ_WRITE
types. Cached, NonCached, Paging.

Cached comes from usual user request, like ReadFile/WriteFile and use cache
manager services to process request. NonCached comes from usual user
request also but it directs FSD to read/write data directly to/from disk
without any caching(hence, alignment requirement starts here because disk
let read or write only sectors). Paging is special request from memory
manager. When memory manager wants to flush mapped file to disk it sends
paging read request. When you access mapped view which is not in memory yet
memory manager sends paging read request to read data from disk. Cache
manager works via memory manager services. Therefore paging read/write can
be from cache manager implicitly.

Now when you open file(instance 1) and then read it or map it, on first
sending read/write request memory manager will reference FILE_OBJECT which
was passed to it. And then memory manager will use only this referenced
FILE_OBJECT to do paging read/write. Then you open file again(instance 2)
and read or write such data which forces memory manager to read/write data
to/from disk, it will be using FILE_OBJECT from previous open, i.e.
instance 1 in our example. Also, FSD doesn’t check alignment like I/O
manager do. I/O manager will fail if offset is not on sector boundary or
size is not multiple sectors. I.e. ReadFile and WriteFile will fail. I
stress that checking is done for files which was opened with
FILE_FLAG_NO_BUFFERING flag.

Network redirector checks file alignment and fails request if there is a
violation.

So, when you opened file with FILE_FLAG_NO_BUFFERING flag and then mapped
it, on first access to view memory manager sends paging IRP_MJ_READ with
offset aligned on sector boudary and size of multiple memory pages. I.e. no
violation. Therefore network redirector sent this request. And request was
processed by your FSD. Then, when your test filled buffer memory manager
will flush modified data. Therefore it sends paging IRP_MJ_WRITE which has
aligned offset but size is not multiple pages because file size is less
than mapped view. Therefore network redirector fails this request. The only
reason is FILE_FLAG_NO_BUFFERING flag was set on file opening. That forced
network redirector do these checks. That’s my theory. I didn’t prove it yet
because i have no Windows 8 under my hand, but will check this soon. It’s
rather easy to prove it. Considering memory manager behavior i’ll do next:

First case:

  1. Open file without FILE_FLAG_NO_BUFFERING flag
  2. Read data to force memory manager to reference FILE_OBJECT from step 1
  3. Open file with FILE_FLAG_NO_BUFFERING flag
  4. Map file and fill buffer
  5. Close both files to see if IRP_MJ_WRITE request will come

Test must succeed because memory manager will send request on FILE_OBJECT
from step 1 because it was previous referenced. Therefore network
redirector will not do checking on file alignment and forward request to
SRV2.

Second case:

  1. Open file with FILE_FLAG_NO_BUFFERING flag(file size must be exactly
    4096 bytes)
  2. Map file and fill buffer
  3. Close file to see if IRP_MJ_WRITE request will come

Test must succeed because when memory manager will send write request,
network redirector will do checking on file alignment and it’ll be ok.
Becase file size is exactly 4096 bytes.

About local FSD and FILE_FLAG_NO_BUFFERING. As i mentioned it before, FSD
will see that IRP_MJ_WRITE is from memory manager(i.e. this is a paging
request) and do special processing. Thus there will not be any checkings of
file alignment from I/O manager and so on. FSD will not care about what
FILE_OBJECT it got unlike network redirector. First of all it’s because of
FSD is local one and presents local files unlike network redirector. As for
network redirector, its FILE_OBJECT’s are associated with opened ones on
SRV2 side. That’s why it can do this check on paging requests unlike local
FSD. Because it knows in advance that request will fail for FILE_OBJECT
that was opened with FILE_FLAG_NO_BUFFERING flag.

As for Windows 10 i think SRV2 doesn’t open file with
FILE_FLAG_NO_BUFFERING flag anymore.