CcFlushCache under SRV2

Then there is the issue with your FSD. At least for case 3. For some reason
network redirector purge the data instead of flushing. I’ll take a look at
memory manager soon.

2016-10-31 23:54 GMT+03:00 :

> > Will i be right if i say the case 3 doesn’t wotk with your FSD?
>
> You are right, yes.
>
> Cases 1, 2 work with my FSD. Cases 3 and 4 do not.
>
>
> —
> NTFSD is sponsored by OSR
>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

I’m not sure there wasn’t attempts to write modified data by redirector.
Possibly the write request wasn’t forward to your FSD and just failed
somewhere before. To find out it, you can attach filter to network
redirector to see if there were paging IRP_MJ_WRITE and if it was so, see
where it fails. Once failed IRP_MJ_WRITE forces memory manager to fail
further I/O on mapped file. Thus, after unmapping all filled buffer data is
lost.

2016-10-31 23:58 GMT+03:00 Anatoly Mikhailov :

> Then there is the issue with your FSD. At least for case 3. For some
> reason network redirector purge the data instead of flushing. I’ll take a
> look at memory manager soon.
>
> 2016-10-31 23:54 GMT+03:00 :
>
>> > Will i be right if i say the case 3 doesn’t wotk with your FSD?
>>
>> You are right, yes.
>>
>> Cases 1, 2 work with my FSD. Cases 3 and 4 do not.
>>
>>
>> —
>> NTFSD is sponsored by OSR
>>
>>
>> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>> software drivers!
>> Details at http:
>>
>> To unsubscribe, visit the List Server section of OSR Online at <
>> http://www.osronline.com/page.cfm?name=ListServer&gt;
>>
>
></http:>

That doesn’t mesh with my experience – in my experience, Mm will retry the writes specifically on the off chance it was a transient (recoverable) error. Last I looked, Mm knows if the file backing the section is on the network, or not, and if it is on the network it will eventually give up and discard the data.

Tony
OSR

> I’m not sure there wasn’t attempts to write modified data by redirector.

Possibly the write request wasn’t forward to your FSD and just failed
somewhere before. To find out it, you can attach filter to network
redirector to see if there were paging IRP_MJ_WRITE and if it was so, see
where it fails.

Anatoly, I think you are onto something. Here is an excerpt from a FileSpy log:
<<
43 winfsp-tests-x64.exe IRP_MJ_CREATE 40000884 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_SUCCESS FILE_OVERWRITE_IF CreOpts: 00000068 Access: 0012019F Share: 00000003 Attrib: 00000080 Result: FILE_CREATED
35 System IRP_MJ_CREATE 40000884 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS FILE_OVERWRITE_IF CreOpts: 00000148 Access: 0012019B Share: 00000003 Attrib: 00000080 Result: FILE_CREATED
36 System FASTIO_QUERY_NETWORK_OPEN_INFO FFFFF98001416E00 Z:\winfsp-tests\file0 FAILURE
37 System IRP_MJ_QUERY_INFORMATION 40060874 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS FileNetworkOpenInformation CreateTime: 01D12188-9016B21F LastAccTime: 01D12188-9016B21F LastWrtTime: 01D12188-9016B21F ChangeTime: 01D12188-9016B21F AllocationSize: 00000000-00000000 EndOfFile: 00000000-00000000 Attrib: 00000020
38 System IRP_MJ_QUERY_SECURITY 40000004 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS
39 System IRP_MJ_QUERY_INFORMATION 40060874 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS FileInternalInformation IndexNumber: 00000000-00000005
40 System IRP_MJ_QUERY_INFORMATION 40060874 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS FileStandardInformation AllocationSize: 00000000-00000000 EndOfFile: 00000000-00000000 NumberOfLinks: 1 DeletePending: FALSE
41 System IRP_MJ_QUERY_INFORMATION 40060874 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS FileBasicInformation CreateTime: 01D12188-9016B21F LastAccTime: 01D12188-9016B21F LastWrtTime: 01D12188-9016B21F ChangeTime: 01D12188-9016B21F Attrib: 00000020
42 System IRP_MJ_QUERY_INFORMATION 40060874 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS FileStandardInformation AllocationSize: 00000000-00000000 EndOfFile: 00000000-00000000 NumberOfLinks: 1 DeletePending: FALSE
44 winfsp-tests-x64.exe FASTIO_QUERY_STANDARD_INFO FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 FAILURE
46 winfsp-tests-x64.exe IRP_MJ_QUERY_INFORMATION 40000042 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_SUCCESS FileStandardInformation AllocationSize: 00000000-00000000 EndOfFile: 00000000-00000000 NumberOfLinks: 1 DeletePending: FALSE
45 System IRP_MJ_QUERY_INFORMATION 40060874 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS FileStandardInformation AllocationSize: 00000000-00000000 EndOfFile: 00000000-00000000 NumberOfLinks: 1 DeletePending: FALSE
48 winfsp-tests-x64.exe IRP_MJ_SET_INFORMATION 40000042 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_SUCCESS FileEndOfFileInformation EndOfFile: 00000000-00020064
47 System IRP_MJ_SET_INFORMATION 40060834 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS FileEndOfFileInformation EndOfFile: 00000000-00020064
50 winfsp-tests-x64.exe IRP_MJ_READ 40060043 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_SUCCESS Offset: 00000000-00000000 ToRead: 8000 Read: 8000
49 winfsp-tests-x64.exe IRP_MJ_READ 40000001 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS Offset: 00000000-00000000 ToRead: 8000 Read: 8000
52 winfsp-tests-x64.exe IRP_MJ_READ 40060043 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_SUCCESS Offset: 00000000-00008000 ToRead: 8000 Read: 8000
51 winfsp-tests-x64.exe IRP_MJ_READ 40000001 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS Offset: 00000000-00008000 ToRead: 8000 Read: 8000
54 winfsp-tests-x64.exe IRP_MJ_READ 40060043 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_SUCCESS Offset: 00000000-00010000 ToRead: 10000 Read: 10000
53 winfsp-tests-x64.exe IRP_MJ_READ 40000001 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS Offset: 00000000-00010000 ToRead: 10000 Read: 10000
56 winfsp-tests-x64.exe IRP_MJ_READ 40060043 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_SUCCESS Offset: 00000000-00020000 ToRead: 1000 Read: 64
55 winfsp-tests-x64.exe IRP_MJ_READ 40000001 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS Offset: 00000000-00020000 ToRead: 1000 Read: 64
60 winfsp-tests-x64.exe IRP_MJ_CLEANUP 40000404 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_SUCCESS
57 System IRP_MJ_QUERY_INFORMATION 40060874 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS FileBasicInformation CreateTime: 01D12188-9016B21F LastAccTime: 01D12188-9016B21F LastWrtTime: 01D12188-9016B21F ChangeTime: 01D12188-9016B21F Attrib: 00000020
58 winfsp-tests-x64.exe IRP_MJ_WRITE 40060043 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_INVALID_PARAMETER Offset: 00000000-00000000 ToWrite: 20064 Written: 0
59 winfsp-tests-x64.exe IRP_MJ_WRITE 40060043 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_INVALID_PARAMETER Offset: 00000000-00000000 ToWrite: 20064 Written: 0
70 winfsp-tests-x64.exe IRP_MJ_CLOSE 40000404 FFFFF8A00243C010 \Device\Mup\localhost\winfsp-tests-share\winfsp-tests\file0 STATUS_SUCCESS
69 System IRP_MJ_CLEANUP 40000404 FFFFF98001416E00 Z:\winfsp-tests\file0 STATUS_SUCCESS

>

Notice that after \Device\Mup receives CLEANUP (line 60) there are 2 WRITE IRP’s sent to \Device\Mup that are failed with STATUS_INVALID_PARAMETER. My FSD never sees these IRP’s and cannot act on them. [Actually the line numbers are: 58,59 on the WRITE IRP’s and 60 on the CLEANUP IRP. Perhaps this means that the WRITE IRP’s are attempted from within the redirector CLEANUP handler.]

There is also an IRP_MJ_QUERY_INFORMATION but it succeeds and it looks reasonable. The FileAttrib means FILE_ATTRIBUTE_ARCHIVE and the date/times are reasonable: Tuesday, November 17, 2015 10:37:39pm (when this VM was snapshotted).

So I have no clue why the WRITE IRP’s would fail with STATUS_INVALID_PARAMETER. There must something about my FSD they don’t like in this case, but I am not wiser as to what.

Bill

And I apologize for the badly formatted log, but I do not how to make it format properly when posting through the web interface.

Ok, I am at a loss. The test now fails under NTFS with FILE_FLAG_NO_BUFFERING under all circumstances.

I am going to call a timeout on this thread, because I think I may be wasting everyone’s time. Please allow me a couple of days to examine this with a clear head and report back.

Thanks everyone for your help so far.

Bill

> Last I looked, Mm knows if the file backing the section is on the
network, or not, and if it is on the network it will eventually give up and
discard the data.

That’s right. Mm consider if the file is on the network. If so then Mm
marks it to itself to fail all I/O if there was at least one write error.
Except for STATUS_FILE_LOCK_CONFLICT. For this status Mm will try to flush
data on remote file again later. If Mm gets STATUS_FILE_INVALID or
STATUS_MEDIA_WRITE_PROTECTED statuses on write, the Mm will fail all
further I/O on file whether it’s remote or not.

2016-11-01 2:40 GMT+03:00 :

> Ok, I am at a loss. The test now fails under NTFS with
> FILE_FLAG_NO_BUFFERING under all circumstances.
>
> I am going to call a timeout on this thread, because I think I may be
> wasting everyone’s time. Please allow me a couple of days to examine this
> with a clear head and report back.
>
> Thanks everyone for your help so far.
>
> Bill
>
>
> —
> NTFSD is sponsored by OSR
>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

> So I have no clue why the WRITE IRP’s would fail with
STATUS_INVALID_PARAMETER.

You opened file with FILE_FLAG_NO_BUFFERING. That means you can read or
write file only on sectors boundary. Also reading or writing size must be
of multiple sectors. Otherwise you’ll get STATUS_INVALID_PARAMETER error.
The log says there was attempt to write 20064 bytes. It’s not multiple of
512 byte sectors. It’s not multiple of 4096 byte sectors also.

Try to map file which has exactly 4096 byte length to see what changes. I
think it’ll be written ok.

By the way. Since you have seen test 2 succeeded against NTFS then don’t be
confused. Likely remote file was opened with no FILE_FLAG_NO_BUFFERING set
before opening the file with the flag. And if it’s so, there could be file
reading and memory manager referenced the opened file instance with no the
flag. Therefore when you opened the file with the flag set and did tests,
writing was ok because memory manager was using to I/O not your opened file
instance but previous one.

Here is an update. What I say below applies only to the discussed test when run with FILE_FLAG_NO_BUFFERING against NTFS via network share. My FSD is *not* involved at all.

  1. On Win8 using \localhost: in all test runs this morning the test failed. Here is a typical run:

C:\NtfsTest>\Users\billziss\Downloads\winfsp\winfsp-tests-x64.exe --ntfs --resilient --share=winfsp-tests-share=C:\ rdwr_mmap_test
rdwr_mmap_test… KO
ASSERT(*Bgn == (rand() & 0xff)) failed at rdwr-test.c:585:rdwr_mmap_dotest

  1. On Win10 using \localhost: in all test runs this morning the test succeeded. Here is a typical run:

C:\NtfsTest>\Users\billziss\Projects\winfsp\build\VStudio\build\Debug\winfsp-tests-x64.exe --ntfs --resilient --share=winfsp-tests-share=C:\ rdwr_mmap_test
rdwr_mmap_test… OK 1s
— COMPLETE —

  1. From Win10 to Win8 using a \win8dbg\users share: in all test runs this morning the test succeeded. Here is a typical run:

C:\>\Users\billziss\Projects\winfsp\build\VStudio\build\Debug\winfsp-tests-x64.exe --ntfs --resilient --share=\win8dbg\users\billziss\downloads\winfsp rdwr_mmap_test
rdwr_mmap_test… OK 2s
— COMPLETE —

Full FileSpy logs for all cases are available upon request.

This suggests to me that there is indeed an issue with the Win8 OS when using the \localhost share. This issue does not have anything to do with my FSD, and can be exhibited against NTFS.

OS version: 6.2 Build 9200

Bill

A final note. My FSD exhibits the same behavior as NTFS:

  1. FAILURE on Win8 when using \localhost.

  2. SUCCESS on Win10 when using \localhost.

  3. SUCCESS on Win10 to Win8 using a \win8dbg\users share.

I think I can safely say that this is a bug in Win8 (version 6.2 Build 9200).

Bill

It’s not a bug. The reason of such behavior I described in my two previous
posts.

2 нояб. 2016 г. 12:30 AM пользователь
написал:

> A final note. My FSD exhibits the same behavior as NTFS:
>
> 1. FAILURE on Win8 when using \localhost.
>
> 2. SUCCESS on Win10 when using \localhost.
>
> 3. SUCCESS on Win10 to Win8 using a \win8dbg\users share.
>
> I think I can safely say that this is a bug in Win8 (version 6.2 Build
> 9200).
>
> Bill
>
>
> —
> NTFSD is sponsored by OSR
>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer&gt;
></http:>

About Win10, seems like SRV2 didn’t open files with FILE_FLAG_NO_BUFFERING
flag anymore.

2016-11-02 6:07 GMT+03:00 Anatoly Mikhailov :

> It’s not a bug. The reason of such behavior I described in my two previous
> posts.
>
> 2 нояб. 2016 г. 12:30 AM пользователь
> написал:
>
> A final note. My FSD exhibits the same behavior as NTFS:
>>
>> 1. FAILURE on Win8 when using \localhost.
>>
>> 2. SUCCESS on Win10 when using \localhost.
>>
>> 3. SUCCESS on Win10 to Win8 using a \win8dbg\users share.
>>
>> I think I can safely say that this is a bug in Win8 (version 6.2 Build
>> 9200).
>>
>> Bill
>>
>>
>> —
>> NTFSD is sponsored by OSR
>>
>>
>> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>> software drivers!
>> Details at http:
>>
>> To unsubscribe, visit the List Server section of OSR Online at <
>> http://www.osronline.com/page.cfm?name=ListServer&gt;
>>
></http:>

Anatoly wrote:

It’s not a bug. The reason of such behavior I described in my two previous
posts.

I am not sure which post you mean. If you mean the one about doing a WRITE with 0x20064 bytes I would disagree that this is the problem.

Recall that the test does not do ReadFile, WriteFile but only memory mapped accesses. Furthermore recall that the test passes when used again the file systems directly (i.e. the C: drive) rather than going through a redirector, even with the FILE_FLAG_NO_BUFFERING flag. Either the test should fail in all cases because of non-aligned WRITE’s or in none.

Furthermore I believe it is legal for a WRITE past the EndOfFile (or ValidDataLength) to not be aligned on a sector boundary. I cannot find the relevant docs right now, but FastFat certainly seems to allow it:

https://github.com/Microsoft/Windows-driver-samples/blob/master/filesys/fastfat/write.c#L2105

Notice that the if branch will be taken in two cases:

  • when StartingVbo is not correctly aligned or
  • ByteCount is not correctly aligned AND the WRITE does not extend the file

So a WRITE with an aligned buffer at an offset of zero and a non-aligned size of 0x20064 is actually legal.

Bill

> Recall that the test does not do ReadFile

paging IRP_MJ_READ always reads multiple pages, for example 4096 bytes

> Furthermore I believe it is legal for a WRITE past the EndOfFile (or
ValidDataLength) to not be aligned on a sector boundary.

legal for local file system, SRV2 is usual client of FSD, and FSD knows
nothing about real request

> Recall that the test does not do ReadFile

you could open file by yourself for example

Furthermore recall that the test passes when used again the file systems
directly (i.e. the C: drive) rather than going through a redirector, even
with the FILE_FLAG_NO_BUFFERING flag.

because PAGING_IO flag was set for IRP

> Furthermore I believe it is legal for a WRITE past the EndOfFile (or
ValidDataLength) to not be aligned on a sector boundary.

I forgot to mention: it’s legal, for PAGING_IO requests

So a WRITE with an aligned buffer at an offset of zero and a non-aligned
size of 0x20064 is actually legal.

For paging read/write

https://github.com/Microsoft/Windows-driver-samples/blob/
master/filesys/fastfat/write.c#L2105

See again carefully.

You can test it out by yourself, do ReadFile on file opened with
FILE_FLAG_NO_BUFFERING flag. ReadFile will fail.

You can do simple test via redirector:

  1. OpenFile without FILE_FLAG_NO_BUFFERING flag.
  2. ReadFile
  3. OpenFile with FILE_FLAG_NO_BUFFERING flag.
  4. Map file from step 3
  5. Fill buffer

This test must succeeded.

One more thing :slight_smile:

Furthermore I believe it is legal for a WRITE past the EndOfFile (or
ValidDataLength) to not be aligned on a sector boundary.

It’s legal for cached read/write

“When opening or creating a file with the *CreateFile*
https:
function, the FILE_FLAG_NO_BUFFERING flag can be specified to disable
system caching of data being read from or written to the file. Although
this gives complete and direct control over data I/O buffering, in the case
of files and similar devices there are data alignment requirements that
must be considered.”

as documented here:
https://msdn.microsoft.com/en-us/library/windows/desktop/cc644950(v=vs.85).aspx</https:>

Previous post was wrotten just once i woke up, so i think i wasn’t clear :slight_smile:

To summ up:

First of all when you open file with FILE_FLAG_NO_BUFFERING flag you must
consider data alignment, i.e. offset is on sector boundary and size is
multiple of sectors. Violation to this rule leads to read/write failings.

From point of view of FSD there can be serveral IRP_MJ_READ/IRP_MJ_WRITE
types. Cached, NonCached, Paging.

Cached comes from usual user request, like ReadFile/WriteFile and use cache
manager services to process request. NonCached comes from usual user
request also but it directs FSD to read/write data directly to/from disk
without any caching(hence, alignment requirement starts here because disk
let read or write only sectors). Paging is special request from memory
manager. When memory manager wants to flush mapped file to disk it sends
paging read request. When you access mapped view which is not in memory yet
memory manager sends paging read request to read data from disk. Cache
manager works via memory manager services. Therefore paging read/write can
be from cache manager implicitly.

Now when you open file(instance 1) and then read it or map it, on first
sending read/write request memory manager will reference FILE_OBJECT which
was passed to it. And then memory manager will use only this referenced
FILE_OBJECT to do paging read/write. Then you open file again(instance 2)
and read or write such data which forces memory manager to read/write data
to/from disk, it will be using FILE_OBJECT from previous open, i.e.
instance 1 in our example. Also, FSD doesn’t check alignment like I/O
manager do. I/O manager will fail if offset is not on sector boundary or
size is not multiple sectors. I.e. ReadFile and WriteFile will fail. I
stress that checking is done for files which was opened with
FILE_FLAG_NO_BUFFERING flag.

Network redirector checks file alignment and fails request if there is a
violation.

So, when you opened file with FILE_FLAG_NO_BUFFERING flag and then mapped
it, on first access to view memory manager sends paging IRP_MJ_READ with
offset aligned on sector boudary and size of multiple memory pages. I.e. no
violation. Therefore network redirector sent this request. And request was
processed by your FSD. Then, when your test filled buffer memory manager
will flush modified data. Therefore it sends paging IRP_MJ_WRITE which has
aligned offset but size is not multiple pages because file size is less
than mapped view. Therefore network redirector fails this request. The only
reason is FILE_FLAG_NO_BUFFERING flag was set on file opening. That forced
network redirector do these checks. That’s my theory. I didn’t prove it yet
because i have no Windows 8 under my hand, but will check this soon. It’s
rather easy to prove it. Considering memory manager behavior i’ll do next:

First case:

  1. Open file without FILE_FLAG_NO_BUFFERING flag
  2. Read data to force memory manager to reference FILE_OBJECT from step 1
  3. Open file with FILE_FLAG_NO_BUFFERING flag
  4. Map file and fill buffer
  5. Close both files to see if IRP_MJ_WRITE request will come

Test must succeed because memory manager will send request on FILE_OBJECT
from step 1 because it was previous referenced. Therefore network
redirector will not do checking on file alignment and forward request to
SRV2.

Second case:

  1. Open file with FILE_FLAG_NO_BUFFERING flag(file size must be exactly
    4096 bytes)
  2. Map file and fill buffer
  3. Close file to see if IRP_MJ_WRITE request will come

Test must succeed because when memory manager will send write request,
network redirector will do checking on file alignment and it’ll be ok.
Becase file size is exactly 4096 bytes.

About local FSD and FILE_FLAG_NO_BUFFERING. As i mentioned it before, FSD
will see that IRP_MJ_WRITE is from memory manager(i.e. this is a paging
request) and do special processing. Thus there will not be any checkings of
file alignment from I/O manager and so on. FSD will not care about what
FILE_OBJECT it got unlike network redirector. First of all it’s because of
FSD is local one and presents local files unlike network redirector. As for
network redirector, its FILE_OBJECT’s are associated with opened ones on
SRV2 side. That’s why it can do this check on paging requests unlike local
FSD. Because it knows in advance that request will fail for FILE_OBJECT
that was opened with FILE_FLAG_NO_BUFFERING flag.

As for Windows 10 i think SRV2 doesn’t open file with
FILE_FLAG_NO_BUFFERING flag anymore.