Non-cached write does not work on 2003

NTFSD
1

Hi, all,

I have problem with non-cached writes when testing our file system
filter under Windows 2003 server.

Let’s test a simple situation - I copy an example file (Winnt.h)
from E:\Test\SubDir1\Winnt.h to E:\Winnt.h.

When the cache manager writes the target file to disk,
our filter catches non-cached IRP_MJ_WRITE for the purpose
of data encryption. In this case, the filter doesn’t
modify the data.

The first write if at offset 0, length 0x10000 (65535 bytes).
It is a MDL write with system buffer at 0xEE189000:

:db EE189000
0010:EE189000 2F 2A 2B 2B 20 42 55 49-4C 44 20 56 65 72 73 69 /*++ BUILD
Versi
0010:EE189010 6F 6E 3A 20 30 30 38 32-20 20 20 20 20 49 6E 63 on: 0082
Inc
0010:EE189020 72 65 6D 65 6E 74 20 74-68 69 73 20 69 66 20 61 rement this
if a

Our filter copies the stack location to next with changing
the file object. After calling lower driver (NTFS.sys),
STATUS_SUCCESS is returned, (the same value is in Irp->IoStatus),
but Irp->IoStatus.Information contains zero.
The cached write is really not completed, which
can be assured after system restart (the file is filled by zeros).

The goal is, that this approach works well on NT4.0, 2000, XP SP1,
but not on Windows 2003. I examined the listings for XPSP1 and 2003,
they were the same (except for pointers, of course).
The bug occurs on NTFS only, (On FASTFAT or redirectors works
well)

Can anyone from Microsoft give me a hint what is wrong ?

Here comes the IRP listing from Soft-ICE. First is the Irp, then the
original file object, then the changed file object.

*** Irp ***

MdlAddress * : F9CA8B80
Flags : 00000043 IRP_NOCACHE | IRP_PAGING_IO |
IRP_MOUNT_COMPLETION | IRP_INPUT_OPERATION |
IRP_SYNCHRONOUS_PAGING_IO
AssociatedIrp : 00000000
&ThreadListEntry : FF4E2448
IoStatus.Status : 00000000
IoStatus.Information : 00000000
RequestorMode : 00
PendingReturned : False
StackCount : 09
CurrentLocation : 09
Cancel : False
CancelIrql : 00
ApcEnvironment : 00
UserIosb * : F9CA8D50
UserEvent * : F9CA8BF0
Overlay : 00000000 00000000
CancelRoutine * : 00000000
UserBuffer * : 00000000
Tail.Overlay
&DeviceQueueEntry : FF4E2478
Thread * : 823CB610
AuxiliaryBuffer * : 00000000
&ListEntry : FF4E2490
CurrentStackLoc * : FF4E25C8
OrigFileObject * : FF7E1680
Tail.Apc * : FF4E2478
Tail.ComplKey : 00000000

CurrentStackLocation at FF4E25C8:
MajorFunction : 04 IRP_MJ_WRITE
MinorFunction : 00
Control : 00
Flags : 00
Length : 00010000
Key : 00000000
ByteOffset : 00
DeviceObject * : 822E93C8
FileObject * : FF7E1680
CompletionRout * : 00000000
Context * : 00000000

*** Original file object ***

:fobj FF7E1680
DeviceObject * : 82356040
Vpb * : 82399770
FsContext * : E1E2F008
FsContext2 * : E185C510
SecObjPointer * : FF49B728
PrivateCacheMap * : 00000000
FinalStatus : 00000000
RelatedFileObj * : 00000000
LockOperation : False
DeletePending : False
ReadAccess : True
WriteAccess : True
DeleteAccess : False
SharedRead : True
SharedWrite : True
SharedDelete : False
Flags : 00044062 FO_SYNCHRONOUS_IO | FO_SEQUENTIAL_ONLY |
FO_CACHE_SUPPORTED | FO_CLEANUP_COMPLETE |
FO_HANDLE_CREATED
FileName : \WinNT.h
CurrentByteOffset : 049383
Waiters : 00000000
Busy : 00000000
LastLock* : 00000000
&Lock : FF7E16CC
&Event : FF7E16DC
ComplContext* : 00000000

*** Changed file object ***

:fobj FF7E19F0
DeviceObject * : 82356040
Vpb * : 82399770
FsContext * : E1D45D90
FsContext2 * : E1D45EE8
SecObjPointer * : FF4CAA54
PrivateCacheMap * : 00000000
FinalStatus : 00000000
RelatedFileObj * : 00000000
LockOperation : False
DeletePending : False
ReadAccess : True
WriteAccess : True
DeleteAccess : False
SharedRead : True
SharedWrite : True
SharedDelete : False
Flags : 00044028 FO_NO_INTERMEDIATE_BUFFERING |
FO_SEQUENTIAL_ONLY | FO_CLEANUP_COMPLETE | FO_HANDLE_CREATED
FileName : \WinNT.h
CurrentByteOffset : 00
Waiters : 00000000
Busy : 00000000
LastLock* : 00000000
&Lock : FF7E1A3C
&Event : FF7E1A4C
ComplContext* : 00000000

L.

2

Totally off topic to your question but WHY do you change the FO?

A possible issue: file size is not changed prior to the paging I/O - paging I/O
will not change file size… maybe it’s not Windows 2K3, but rather some other bug…
or some specific case you happened to notice on W2K3.

–
Kind regards, Dejan M. MVP for DDK
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.

3

> Totally off topic to your question but WHY do you change the FO?

This is at the most from historic reasons. The creator
of the driver has written it that way because of cache-related
things (It’s a long story to tell all reasons).
I know it is not 100% correct behavior, but currently
I don’t have time to change it, it would require to
rewrite the complete filter.

A possible issue: file size is not changed prior
to the paging I/O - paging I/O will not change file size

But the result file size is correct. As test, I copy a file to
another folder (the file did not exist before). After the
copying, the file size is good, but the content is zeroed.
But I’ll rather check that once again.

maybe it’s not Windows 2K3, but rather some other bug…
or some specific case you happened to notice on W2K3.

Yes, I didn’t say it’s problem of W2K3. This is certainly
a bug in our filter.

L.

4

How do you get the replacement file object?
How come if it refers to the same file the FsContext is not the same?

Ladislav Zezula wrote:

> Totally off topic to your question but WHY do you change the FO?

This is at the most from historic reasons. The creator of the driver has written it
that way because of cache-related things (It’s a long story to tell all reasons). I
know it is not 100% correct behavior, but currently
I don’t have time to change it, it would require to rewrite the complete filter.

–
Kind regards, Dejan M. MVP for DDK
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.

5

> How do you get the replacement file object?

How come if it refers to the same file the FsContext is not the same?

When the file is created/opened. Our filter creates a duplicate
file object, which is then used. Both FsContexts are managed
by the filter (In fact, the filter is a bit like intermediate file system,
not as a filter).
But as I alreay wrote, this feature is not good and I am going
to rewrite the complete filter, when I have enough time and
a bit more knowledge (I am not the author).
But we cannot stop supporting it now and go to rewrite it completely
(it will not be complete within a month :-)).

The problem with non-cached writes is here and now; I have to fix it.

L.

6

I’m not asking this to waste time but to see if it may be the source of the
problem.

> How do you get the replacement file object?
> How come if it refers to the same file the FsContext is not the same?

The problem with non-cached writes is here and now; I have to fix it.

–
Kind regards, Dejan M. MVP for DDK
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.

7

> I’m not asking this to waste time but to see if it may be the source of
the

problem.

Yes, sure. The mechanism for replacing file objects
is a bit dirty, but it worked until 2003. The underlying file
system receives ONLY the duplicate, not the original one.
The FCB uniqueness is hold too. The file objects are
never mixed and the FCB for the same file is always the same.

L.

8

From your fobj output it doesn’t seem FsContext is the same…
So how do you duplicate the object? ZwCreateFile and ObReferenceObjectByHandle
during create?

Yes, sure. The mechanism for replacing file objects
is a bit dirty, but it worked until 2003. The underlying file
system receives ONLY the duplicate, not the original one.
The FCB uniqueness is hold too. The file objects are
never mixed and the FCB for the same file is always the same.

–
Kind regards, Dejan M. MVP for DDK
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.

9

> From your fobj output it doesn’t seem FsContext is the same…

Eh, I wrote it wrong. The FsContexts on the source file object
is ours, the FsContexts on the duplicate file object is original
(set by FSD).
By “the same FsContexts” I meant that the FsContexts which
upper filters (or I/O Manager) receives, are of the same value
for the same file (but not the same like the underlying Fs
returned).

So how do you duplicate the object?
ZwCreateFile and ObReferenceObjectByHandle
during create?

No.
Our filter uses ObCreateObject + RtlCopyMemory + ObjInsertObject
After RtlCopyMemory, we clear some flags in the copy,
like FO_STREAM_FILE and FO_FILE_HAS_EXTENSION.
Maybe here is a problem, because the underlying file system never
receives a file object with that flags or with some 2003 extensions
(if any).

L.

10

Hi, all

After some digging in Ntfs.sys code (namely NtfsCommonWrite),
I found one possible reason of the problem.

NtfsCommonWritewill do nothing if the “ValidDataLength”
field in the file object’s FCB is zero. I hope this must be set using
IRP_MJ_SET_INFORMATION with FileValidDataLength-
Information.

It seems that this is not a problem for Windows XP, but
Windows 2003 will write nothing.

Is this call supported in WinXP+ file systems only ? The
“FILE_VALID_DATA_LENGTH_INFORMATION”
structure and “FileValidDataLengthInformation”
constant is defined only in IFS Kit for Windows
XP and 2003.

L.

11

The ability to explicitly set the valid data length was added in XP.

Can you tell me what your filter/file system thinks is the FileSize and
ValidDataLength for the stream and what Ntfs thinks is the FileSize and
ValidDataLength for this stream?

Also, can you tell who initiated this non-cached write? I’m
particularly interested in knowing if it is either the lazy writer
(CcWriteBehind should be near the beginning of the stack) or mapped page
writer (MiMappedPageWriter should be near the beginning of the stack).

Thanks,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, April 21, 2004 5:10 AM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Non-cached write does not work on 2003

Hi, all

After some digging in Ntfs.sys code (namely NtfsCommonWrite), I found
one possible reason of the problem.

NtfsCommonWritewill do nothing if the “ValidDataLength”
field in the file object’s FCB is zero. I hope this must be set using
IRP_MJ_SET_INFORMATION with FileValidDataLength- Information.

It seems that this is not a problem for Windows XP, but Windows 2003
will write nothing.

Is this call supported in WinXP+ file systems only ? The
“FILE_VALID_DATA_LENGTH_INFORMATION”
structure and “FileValidDataLengthInformation”
constant is defined only in IFS Kit for Windows XP and 2003.

L.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

12

Does your filter handle cached write requests itself without calling
underlying FS at all? If it is the case then valid data length will never be
set by the underlying file system. I believe that Microsoft implemented
optimization in W2003 NTFS so any data beyond valid data length is ignored
when request comes from the lazy writer. Creating your own IRP_MJ_WRITE and
sending it to NTFS is not an option because this will extend logical size of
the file.
A possible work around is to change top level IRP to
FSRTL_MOD_WRITE_TOP_LEVEL_IRP before sending IRP to NTFS and restore it back
after return from IoCallDriver. By doing this you will get your data written
out but I am not sure about possible side-effects.

Alexei.

“Ladislav Zezula” wrote in message news:xxxxx@ntfsd…
> Hi, all
>
> After some digging in Ntfs.sys code (namely NtfsCommonWrite),
> I found one possible reason of the problem.
>
> NtfsCommonWritewill do nothing if the “ValidDataLength”
> field in the file object’s FCB is zero. I hope this must be set using
> IRP_MJ_SET_INFORMATION with FileValidDataLength-
> Information.
>
> It seems that this is not a problem for Windows XP, but
> Windows 2003 will write nothing.
>
> Is this call supported in WinXP+ file systems only ? The
> “FILE_VALID_DATA_LENGTH_INFORMATION”
> structure and “FileValidDataLengthInformation”
> constant is defined only in IFS Kit for Windows
> XP and 2003.
>
> L.
>
>

13

Hi,

thank you very much for the response.

Can you tell me what your filter/file system thinks is the FileSize and
ValidDataLength for the stream and what Ntfs thinks is the FileSize and
ValidDataLength for this stream?

Yes, of course :-))
First, I have to briefly explain our logic. Let’s imagine simple
example of CreateFile(new_file)-WriteFile(0x10000 bytes)-CloseHandle.
When the write comes (cached first), our filter increases
the file size to number of bytes to be written (thus, 0x10000
bytes) using IRP_MJ_SET_INFORMATION(FileEndOfFileInfo)
using our duplicated file object, so the lower FS knows that the file
size changed.
Then we call CcCopyWrite on the file object coming from I/O manager.
After a while (while means when CcWorkerThread - CcWriteBehind
awakes), the non-cached write comes to our filter (0x10000 bytes).
In this moment, the AllocationSize and FileSize in the FS FCB
is set to 0x10000, but the ValidDataLength is set to zero.
(Our filter thinks it is 0x10000 0x10000 0x10000)

Then our filter calls the underlying FS to write the data
(non-cached). NtfsFsdWrite returns STATUS_SUCCESS
and the IoStatus.Information contains zero.

Well the problem is not in NTFS, but between my chair
and keyboard (as usually :-))). I probably have to set
the ValidDataLength too. But one thing worries me -
IFS documentation says that when I call IRP_MJ_SET_INFORMATION
(FileValidDataLengthInfo), the ValidDataLength must be
greater than the previous ValidDataLength (which is no problem)
BUT less than the current file size (which means it cannot be equal).

I’m convinced that I cannot simply change the ValidDataLength
in the FCB header of the FS FileObject (I didn’t even tried, because I
do not want to see the buch of bugchecks because of such dirty
hack).

L.

14

First, I’ll answer the easy part of the question –

The FILE_VALID_DATA_LENGTH_INFORMATION docs are incorrect (sorry we
missed that in review). ValidDataLength must be less than or *equal to*
file size. It cannot be greater than file size. I’ll talk to Diane
about getting the docs corrected.

Now the more complicated part –

From your previous email, you showed that the IRP you send down to the
file system when forwarding the lazy writer’s write has the PAGING_IO
flag set. NTFS does not expect PAGING_IO to extend the valid data
length of the file. In this scenario, NTFS expects to have seen the
cached write and NTFS would have extended the valid data length at that
time. Therefore, when the lazy writer’s write comes along, it looks
like PAGING_IO and NTFS ignores such writes which extend valid data
length (returns STATUS_SUCCESS, length = 0 as you are seeing).

To set the valid data length of a file through IRP_MJ_SET_INFORMATION,
the user needs to have SE_MANAGE_VOLUME_PRIVILEGE and this is enforced
by NTFS. The SDK docs for SetFileValidData describes the intended use
for this operation and what you are trying to do is not listed there ;).

In any case, extending the valid data length will cause NTFS to 0 this
range of the file. That’s unnecessary since you’ve go the real data
which you want stored in that range of the file.

I’d suggest clearing the PAGING_IO flag when you send the non-cached
write IRP down to NTFS. Then this write will just look like a regular
non-cached, write which extends the file and NTFS will do the right
thing to update it’s internal file sizes.

One caveat here: I’m assuming that since your filter is taking over the
caching on the data stream, your filter is also taking responsibility
for the locking on the file. NTFS won’t be getting the internal Cc
callbacks (the ones registered with Cc when CcInitializeCacheMap is
called), so it would be up to your filter do the appropriate
synchronization.

Regards,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, April 21, 2004 10:42 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Non-cached write does not work on 2003

Hi,

thank you very much for the response.

Can you tell me what your filter/file system thinks is the FileSize
and
ValidDataLength for the stream and what Ntfs thinks is the FileSize
and
ValidDataLength for this stream?

Yes, of course :-))
First, I have to briefly explain our logic. Let’s imagine simple
example of CreateFile(new_file)-WriteFile(0x10000 bytes)-CloseHandle.
When the write comes (cached first), our filter increases
the file size to number of bytes to be written (thus, 0x10000
bytes) using IRP_MJ_SET_INFORMATION(FileEndOfFileInfo)
using our duplicated file object, so the lower FS knows that the file
size changed.
Then we call CcCopyWrite on the file object coming from I/O manager.
After a while (while means when CcWorkerThread - CcWriteBehind
awakes), the non-cached write comes to our filter (0x10000 bytes).
In this moment, the AllocationSize and FileSize in the FS FCB
is set to 0x10000, but the ValidDataLength is set to zero.
(Our filter thinks it is 0x10000 0x10000 0x10000)

Then our filter calls the underlying FS to write the data
(non-cached). NtfsFsdWrite returns STATUS_SUCCESS
and the IoStatus.Information contains zero.

Well the problem is not in NTFS, but between my chair
and keyboard (as usually :-))). I probably have to set
the ValidDataLength too. But one thing worries me -
IFS documentation says that when I call IRP_MJ_SET_INFORMATION
(FileValidDataLengthInfo), the ValidDataLength must be
greater than the previous ValidDataLength (which is no problem)
BUT less than the current file size (which means it cannot be equal).

I’m convinced that I cannot simply change the ValidDataLength
in the FCB header of the FS FileObject (I didn’t even tried, because I
do not want to see the buch of bugchecks because of such dirty
hack).

L.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

15

But the size would have to be adjusted - since paging I/O would come in page_size
boundaries - while the offset would be OK, the length (most of the time) won’t be
what’s required.
You’d need to call FileSetEndOfFileInformation right AFTER this non-cached
non-paging write.

I’d suggest clearing the PAGING_IO flag when you send the non-cached write IRP down
to NTFS. Then this write will just look like a regular non-cached, write which
extends the file and NTFS will do the right thing to update it’s internal file
sizes.

–
Kind regards, Dejan M. MVP for DDK
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.

16

Molly,

You are saying that PAGING_IO doesn’t extend valid data length.
Correct me if I am wrong, but I believe that PAGING_IO originated from the
modified page writer does extend valid data length and doesn’t change EOF.
So by changing TopLevelIrp you can convince NTFS that IRP_MJ_WRITE is
originated by modified page writer and thus need to be written to the disk.
I wonder what problems may cause changing TopLevelIrp just before calling
NTFS and restoring it back immediately after?
It is probably safer then removing paging IO flag IRP.

Alexei.

“Molly Brown” wrote in message
news:xxxxx@ntfsd…
First, I’ll answer the easy part of the question –

The FILE_VALID_DATA_LENGTH_INFORMATION docs are incorrect (sorry we
missed that in review). ValidDataLength must be less than or equal to
file size. It cannot be greater than file size. I’ll talk to Diane
about getting the docs corrected.

Now the more complicated part –

From your previous email, you showed that the IRP you send down to the
file system when forwarding the lazy writer’s write has the PAGING_IO
flag set. NTFS does not expect PAGING_IO to extend the valid data
length of the file. In this scenario, NTFS expects to have seen the
cached write and NTFS would have extended the valid data length at that
time. Therefore, when the lazy writer’s write comes along, it looks
like PAGING_IO and NTFS ignores such writes which extend valid data
length (returns STATUS_SUCCESS, length = 0 as you are seeing).

To set the valid data length of a file through IRP_MJ_SET_INFORMATION,
the user needs to have SE_MANAGE_VOLUME_PRIVILEGE and this is enforced
by NTFS. The SDK docs for SetFileValidData describes the intended use
for this operation and what you are trying to do is not listed there ;).

In any case, extending the valid data length will cause NTFS to 0 this
range of the file. That’s unnecessary since you’ve go the real data
which you want stored in that range of the file.

I’d suggest clearing the PAGING_IO flag when you send the non-cached
write IRP down to NTFS. Then this write will just look like a regular
non-cached, write which extends the file and NTFS will do the right
thing to update it’s internal file sizes.

One caveat here: I’m assuming that since your filter is taking over the
caching on the data stream, your filter is also taking responsibility
for the locking on the file. NTFS won’t be getting the internal Cc
callbacks (the ones registered with Cc when CcInitializeCacheMap is
called), so it would be up to your filter do the appropriate
synchronization.

Regards,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, April 21, 2004 10:42 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Non-cached write does not work on 2003

Hi,

thank you very much for the response.

> Can you tell me what your filter/file system thinks is the FileSize
and
> ValidDataLength for the stream and what Ntfs thinks is the FileSize
and
> ValidDataLength for this stream?

Yes, of course :-))
First, I have to briefly explain our logic. Let’s imagine simple
example of CreateFile(new_file)-WriteFile(0x10000 bytes)-CloseHandle.
When the write comes (cached first), our filter increases
the file size to number of bytes to be written (thus, 0x10000
bytes) using IRP_MJ_SET_INFORMATION(FileEndOfFileInfo)
using our duplicated file object, so the lower FS knows that the file
size changed.
Then we call CcCopyWrite on the file object coming from I/O manager.
After a while (while means when CcWorkerThread - CcWriteBehind
awakes), the non-cached write comes to our filter (0x10000 bytes).
In this moment, the AllocationSize and FileSize in the FS FCB
is set to 0x10000, but the ValidDataLength is set to zero.
(Our filter thinks it is 0x10000 0x10000 0x10000)

Then our filter calls the underlying FS to write the data
(non-cached). NtfsFsdWrite returns STATUS_SUCCESS
and the IoStatus.Information contains zero.

Well the problem is not in NTFS, but between my chair
and keyboard (as usually :-))). I probably have to set
the ValidDataLength too. But one thing worries me -
IFS documentation says that when I call IRP_MJ_SET_INFORMATION
(FileValidDataLengthInfo), the ValidDataLength must be
greater than the previous ValidDataLength (which is no problem)
BUT less than the current file size (which means it cannot be equal).

I’m convinced that I cannot simply change the ValidDataLength
in the FCB header of the FS FileObject (I didn’t even tried, because I
do not want to see the buch of bugchecks because of such dirty
hack).

L.

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

17

Molly and Dejan,

first of all, than you both for the responses.

One caveat here: I’m assuming that since your filter is taking over the
caching on the data stream, your filter is also taking responsibility
for the locking on the file.
so it would be up to your filter do the appropriate synchronization.

Yes, the filter does this (among other things). But after some experience
with such model (I mean using two file objects in filter and translaing
almost
all IRP calls), I strongly don’t recommend to other developers
(planning to write an encryption filter and asking here how to do it)
to use that approach, otherwise they will see what “programmer’s hell”
is.

You’d need to call FileSetEndOfFileInformation right AFTER this non-cached
non-paging write.

I don’t think so. This is OK because the cache manager does it
(In my problem, the file size is OK but the content is wrong).
The filter works that way some time. Maybe this will change after
I’ll try Molly’s suggestion

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Wednesday, April 21, 2004 10:42 PM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Non-cached write does not work on 2003

Hi,

thank you very much for the response.

Can you tell me what your filter/file system thinks is the FileSize
and
ValidDataLength for the stream and what Ntfs thinks is the FileSize
and
ValidDataLength for this stream?

Yes, of course :-))
First, I have to briefly explain our logic. Let’s imagine simple
example of CreateFile(new_file)-WriteFile(0x10000 bytes)-CloseHandle.
When the write comes (cached first), our filter increases
the file size to number of bytes to be written (thus, 0x10000
bytes) using IRP_MJ_SET_INFORMATION(FileEndOfFileInfo)
using our duplicated file object, so the lower FS knows that the file
size changed.
Then we call CcCopyWrite on the file object coming from I/O manager.
After a while (while means when CcWorkerThread - CcWriteBehind
awakes), the non-cached write comes to our filter (0x10000 bytes).
In this moment, the AllocationSize and FileSize in the FS FCB
is set to 0x10000, but the ValidDataLength is set to zero.
(Our filter thinks it is 0x10000 0x10000 0x10000)

Then our filter calls the underlying FS to write the data
(non-cached). NtfsFsdWrite returns STATUS_SUCCESS
and the IoStatus.Information contains zero.

Well the problem is not in NTFS, but between my chair
and keyboard (as usually :-))). I probably have to set
the ValidDataLength too. But one thing worries me -
IFS documentation says that when I call IRP_MJ_SET_INFORMATION
(FileValidDataLengthInfo), the ValidDataLength must be
greater than the previous ValidDataLength (which is no problem)
BUT less than the current file size (which means it cannot be equal).

I’m convinced that I cannot simply change the ValidDataLength
in the FCB header of the FS FileObject (I didn’t even tried, because I
do not want to see the buch of bugchecks because of such dirty
hack).

L.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@volny.cz
To unsubscribe send a blank email to xxxxx@lists.osr.com

18

> I wonder what problems may cause changing TopLevelIrp just before calling

NTFS and restoring it back immediately after?
It is probably safer then removing paging IO flag IRP.

I’ll better try both approaches, maybe I’ll find another problem
I think before removing paging IO flag, I have to test if the
write really comes from the cache manager. If the write
comes from VMM, I think it’s better to keep the flag there :-))

Once again, thank you all for the suggestions.

L.

19

If you clear PAGING_IO that changes your filter by far. Depending on what calls
you let through and which you do yourself you might need to call it. I think you
will, but only you can tell:-)

I don’t think so. This is OK because the cache manager does it
(In my problem, the file size is OK but the content is wrong).
The filter works that way some time. Maybe this will change after
I’ll try Molly’s suggestion

–
Kind regards, Dejan M. MVP for DDK
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.

20

> If you clear PAGING_IO that changes your filter by far. Depending on what
calls

you let through and which you do yourself you might need to call it. I
think you
will, but only you can tell:-)

For now, I have set the Top Level Irp to simulate
modified page writer. This seems that it works, but I have to
play with it at least some days to be sure.

L.