Filesize in paging I/O paths?

NTFSD
1

Hi all,
For those of you out there who have done or are doing data
modifying/encrypting filters, I was wondering if anyone has come up with
a reliable and/or good method for determining the file size in the
paging I/O paths. An example is in order:

-during a paging I/O write, I see a 4k write for example (offset 0,
length 4096), which could possibly contain the last byte of the file
-in the dispatch path for the write, I don’t know what the EOF actually
is, only the filesystem knows
-how can I determine this so that I can perform operations involving the
file data that may depend on this information?

In addition, I have seen people on this list say that tracking file size
in a filter-owned variable is an option, but it seems like keeping this
atomically synchronized with the filesystem’s file size would be very
complicated, possibly causing performance and/or synch problems, because
filter specific locks would surely be involved. I would like to send
down a QUERY_INFO, but then the FCB resources may be acquired already,
and this could cause a deadlock. Simply plucking the size out of the
FCB seems unreasonable, considering that the locks in the FCB are there
to synch access to this information in the first place, and NTFS uses 64
bit file sizes, which can’t be atomically read/written anyway.

A while ago on this list on a thread relating to this question, Nick
Ryan said:

“Maintaining file size in your own private variable works but does not
buy you anything you can use during paging I/O. Since it would be
protected by your own lock and not the filesystem’s, it won’t change
atomically with the FCB filesize. What if the added portion of a newly
extended file is immediately flushed by the filesystem after having
updated the FCB filesize? You may mistreat that I/O since your private
variable will not be updated yet.
It appears that we must resign ourselves to being completely file-size
unaware during our paging I/O handlers, whatever changes to our
algorithms that entails.”

So, should I resign myself to only using an algorithm that is
independent of the file size in the paging I/O path (as Nick says), or
has anyone done this successfully? Is there a definitive answer for
this issue?

Thanks,
Matt

2

Matthew N. White wrote:



> So, should I resign myself to only using an algorithm that is
> independent of the file size in the paging I/O path (as Nick says), or
> has anyone done this successfully? Is there a definitive answer for
> this issue?
>
> Thanks,
> Matt

Basically, you must resign yourself. You can be sure that in today’s
filesystems the file size will not DECREASE during paging I/O activity,
as evidenced by this comment in FastFat:

//
// This call is unconditional, because MM always wants to know.
// Also serialize here with paging io since we are truncating
// the file size.
//

But you can’t be sure it won’t INCREASE. So you can lower-bound it, but
not upper-bound it.

–
Nick Ryan (MVP for DDK)

3

Hmmm, strange. I am confused now. At the OSR filesystems seminar, Tony
said that paging I/O writes will not extend file sizes, and that this
rule allows filesystem block allocator code to be paged (which made
sense to me at the time). But you are saying that the file size can’t
be increased in this situation. But that comment does make sense as
well…

Matt

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Nick Ryan
Sent: Wednesday, September 24, 2003 2:44 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Re: Filesize in paging I/O paths?

Matthew N. White wrote:



> So, should I resign myself to only using an algorithm that is
> independent of the file size in the paging I/O path (as Nick says), or

> has anyone done this successfully? Is there a definitive answer for
> this issue?
>
> Thanks,
> Matt

Basically, you must resign yourself. You can be sure that in today’s
filesystems the file size will not DECREASE during paging I/O activity,
as evidenced by this comment in FastFat:

//
// This call is unconditional, because MM always wants to know. //
Also serialize here with paging io since we are truncating // the file
size. //

But you can’t be sure it won’t INCREASE. So you can lower-bound it, but
not upper-bound it.

–
Nick Ryan (MVP for DDK)

—
You are currently subscribed to ntfsd as: xxxxx@bitarmor.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

4

Paging I/O itself cannot case file size to increase, but other threads
may initiate a size increase, and they WON’T be synchronized with paging
I/O activity. File size decreases initiated by other threads WILL be
synchronized with paging I/O, however.

I think the fundamental reason is simply because that the fact that
paging I/O is going on means that Mm is transferring file data into or
out of a file-mapped range, and the system would go down pretty hard if
a portion of this mapped range suddenly was made invalid because of a
file truncation while this memory is being used by the transfer.

Matthew N. White wrote:

Hmmm, strange. I am confused now. At the OSR filesystems seminar, Tony
said that paging I/O writes will not extend file sizes, and that this
rule allows filesystem block allocator code to be paged (which made
sense to me at the time). But you are saying that the file size can’t
be increased in this situation. But that comment does make sense as
well…

Matt

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Nick Ryan
Sent: Wednesday, September 24, 2003 2:44 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Re: Filesize in paging I/O paths?

Matthew N. White wrote:


>
>>So, should I resign myself to only using an algorithm that is
>>independent of the file size in the paging I/O path (as Nick says), or
>
>
>>has anyone done this successfully? Is there a definitive answer for
>>this issue?
>>
>>Thanks,
>>Matt
>
>
> Basically, you must resign yourself. You can be sure that in today’s
> filesystems the file size will not DECREASE during paging I/O activity,
> as evidenced by this comment in FastFat:
>
> //
> // This call is unconditional, because MM always wants to know. //
> Also serialize here with paging io since we are truncating // the file
> size. //
>
> But you can’t be sure it won’t INCREASE. So you can lower-bound it, but
> not upper-bound it.
>

–
Nick Ryan (MVP for DDK)

5

There is one more complication related to paging IO. NTFS supports
persistent Valid Data Length. It allows NTFS to implement some optimization
and truncate Paging IO write based on current Valid data length instead of
EOF. So if you encrypt data between valid data length and EOF (which is all
0’s), subsequent read will return 0’s instead of encrypted 0’s.
Paging IO may or may not extend Valid data length depending on TopLevelIrp,
this optimization is applied only to requests that do not extend Valid Data
Length.

Alexei.

“Matthew N. White” wrote in message news:xxxxx@ntfsd…
>
> Hmmm, strange. I am confused now. At the OSR filesystems seminar, Tony
> said that paging I/O writes will not extend file sizes, and that this
> rule allows filesystem block allocator code to be paged (which made
> sense to me at the time). But you are saying that the file size can’t
> be increased in this situation. But that comment does make sense as
> well…
>
> Matt
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Nick Ryan
> Sent: Wednesday, September 24, 2003 2:44 PM
> To: Windows File Systems Devs Interest List
> Subject: [ntfsd] Re: Filesize in paging I/O paths?
>
>
> Matthew N. White wrote:
>
>
>
> > So, should I resign myself to only using an algorithm that is
> > independent of the file size in the paging I/O path (as Nick says), or
>
> > has anyone done this successfully? Is there a definitive answer for
> > this issue?
> >
> > Thanks,
> > Matt
>
> Basically, you must resign yourself. You can be sure that in today’s
> filesystems the file size will not DECREASE during paging I/O activity,
> as evidenced by this comment in FastFat:
>
> //
> // This call is unconditional, because MM always wants to know. //
> Also serialize here with paging io since we are truncating // the file
> size. //
>
> But you can’t be sure it won’t INCREASE. So you can lower-bound it, but
> not upper-bound it.
>
> –
> Nick Ryan (MVP for DDK)
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@bitarmor.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>

6

Ah, ok, this is starting to make sense. I checked that piece of code in
the fastfat fileinfo.c file, and it takes the paging I/O resource
exclusive. So, basically you are saying that in the case where the file
EOF may be extended during a paging I/O operation, this would occur
because the filesystem would synchronize the file EOF changes with the
mainresource in the FCB header, and this synchronization is oblivious to
the pagingioresource in the FCB header, which is used to synchronize the
paging I/O operations. Is this correct?

As a side note, if you have Nagar’s book, on pg. 268, 2/3 of the way
down, he says, “A second important point to note is that changes in the
filesize are generally not synchronized with paging I/O read or write
requests.” This statement obviously concurs with the above logic.
Before that, however, on the same page, 3rd paragraph, he says, “Both
the mainresource and the pagingioresource must be acquired exclusively
before changing either of the file size values.” in reference to the
AllocationSize and FileSize values in the FCB header. I don’t
understand how both of these statements can be completely true, since
this whole discussion implies that the FileSize may be increased w/o
having the paging I/O resource acquired. Do you agree with this? Am I
going insane?

Thanks,
Matt

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Nick Ryan
Sent: Wednesday, September 24, 2003 7:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Re: Filesize in paging I/O paths?

Paging I/O itself cannot case file size to increase, but other threads
may initiate a size increase, and they WON’T be synchronized with paging

I/O activity. File size decreases initiated by other threads WILL be
synchronized with paging I/O, however.

I think the fundamental reason is simply because that the fact that
paging I/O is going on means that Mm is transferring file data into or
out of a file-mapped range, and the system would go down pretty hard if
a portion of this mapped range suddenly was made invalid because of a
file truncation while this memory is being used by the transfer.

Matthew N. White wrote:

Hmmm, strange. I am confused now. At the OSR filesystems seminar,
Tony said that paging I/O writes will not extend file sizes, and that
this rule allows filesystem block allocator code to be paged (which
made sense to me at the time). But you are saying that the file size
can’t be increased in this situation. But that comment does make
sense as well…

Matt

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Nick Ryan
Sent: Wednesday, September 24, 2003 2:44 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Re: Filesize in paging I/O paths?

Matthew N. White wrote:


>
>>So, should I resign myself to only using an algorithm that is
>>independent of the file size in the paging I/O path (as Nick says), or
>
>
>>has anyone done this successfully? Is there a definitive answer for
>>this issue?
>>
>>Thanks,
>>Matt
>
>
> Basically, you must resign yourself. You can be sure that in today’s
> filesystems the file size will not DECREASE during paging I/O
activity,
> as evidenced by this comment in FastFat:
>
> //
> // This call is unconditional, because MM always wants to know. //
> Also serialize here with paging io since we are truncating // the
> file size. //
>
> But you can’t be sure it won’t INCREASE. So you can lower-bound it,
> but
> not upper-bound it.
>

–
Nick Ryan (MVP for DDK)

—
You are currently subscribed to ntfsd as: xxxxx@bitarmor.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

7

Thanks for the info. Is there anything you can do about this, or a
workaround of some kind?

Matt

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alexei Jelvis
Sent: Wednesday, September 24, 2003 9:42 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Re: Filesize in paging I/O paths?

There is one more complication related to paging IO. NTFS supports
persistent Valid Data Length. It allows NTFS to implement some
optimization and truncate Paging IO write based on current Valid data
length instead of EOF. So if you encrypt data between valid data length
and EOF (which is all 0’s), subsequent read will return 0’s instead of
encrypted 0’s. Paging IO may or may not extend Valid data length
depending on TopLevelIrp, this optimization is applied only to requests
that do not extend Valid Data Length.

Alexei.

“Matthew N. White” wrote in message
news:xxxxx@ntfsd…
>
> Hmmm, strange. I am confused now. At the OSR filesystems seminar,
> Tony said that paging I/O writes will not extend file sizes, and that
> this rule allows filesystem block allocator code to be paged (which
> made sense to me at the time). But you are saying that the file size
> can’t be increased in this situation. But that comment does make
> sense as well…
>
> Matt
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Nick Ryan
> Sent: Wednesday, September 24, 2003 2:44 PM
> To: Windows File Systems Devs Interest List
> Subject: [ntfsd] Re: Filesize in paging I/O paths?
>
>
> Matthew N. White wrote:
>
>
>
> > So, should I resign myself to only using an algorithm that is
> > independent of the file size in the paging I/O path (as Nick says),
> > or
>
> > has anyone done this successfully? Is there a definitive answer for

> > this issue?
> >
> > Thanks,
> > Matt
>
> Basically, you must resign yourself. You can be sure that in today’s
> filesystems the file size will not DECREASE during paging I/O
> activity, as evidenced by this comment in FastFat:
>
> //
> // This call is unconditional, because MM always wants to know. //
> Also serialize here with paging io since we are truncating // the
> file size. //
>
> But you can’t be sure it won’t INCREASE. So you can lower-bound it,
> but not upper-bound it.
>
> –
> Nick Ryan (MVP for DDK)
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@bitarmor.com To
> unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>

—
You are currently subscribed to ntfsd as: xxxxx@bitarmor.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

8

Matthew N. White wrote:

Ah, ok, this is starting to make sense. I checked that piece of code in
the fastfat fileinfo.c file, and it takes the paging I/O resource
exclusive. So, basically you are saying that in the case where the file
EOF may be extended during a paging I/O operation, this would occur
because the filesystem would synchronize the file EOF changes with the
mainresource in the FCB header, and this synchronization is oblivious to
the pagingioresource in the FCB header, which is used to synchronize the
paging I/O operations. Is this correct?

Yes.

As a side note, if you have Nagar’s book, on pg. 268, 2/3 of the way
down, he says, “A second important point to note is that changes in the
filesize are generally not synchronized with paging I/O read or write
requests.” This statement obviously concurs with the above logic.
Before that, however, on the same page, 3rd paragraph, he says, “Both
the mainresource and the pagingioresource must be acquired exclusively
before changing either of the file size values.” in reference to the
AllocationSize and FileSize values in the FCB header. I don’t
understand how both of these statements can be completely true, since
this whole discussion implies that the FileSize may be increased w/o
having the paging I/O resource acquired. Do you agree with this? Am I
going insane?

Thanks,
Matt

I don’t have the book in front of me so I can’t see the context, but in
general the second statement is just not true. There are obvious cases
in fastfat that we’ve just seen where Fcb->Header.FileSize.LowPart is
set with just main held. I think that somebody outside the FSD wanting
to LOOK at this value can safely do so only after taking both resources,
however.

–
Nick Ryan (MVP for DDK)

9

> I think the fundamental reason is simply because that the fact that

paging I/O is going on means that Mm is transferring file data into or
out of a file-mapped range, and the system would go down pretty hard if
a portion of this mapped range suddenly was made invalid because of a
file truncation while this memory is being used by the transfer.

No. The cause - as Daniel Lovinger from MS said - is that the disk blocks can
be reallocated for other file just after the truncation, so, truncation with
pending writes (pending in the SCSI tagged queue, for instance) can lead to the
pending writes overwriting the data in the new file.

Thus truncate is mutually exclusive with any kinds of IO on the file (both FCB
locks are grabbed).

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

10

> exclusive. So, basically you are saying that in the case where the file

EOF may be extended during a paging I/O operation, this would occur
because the filesystem would synchronize the file EOF changes with the
mainresource in the FCB header, and this synchronization is oblivious to
the pagingioresource in the FCB header, which is used to synchronize the
paging I/O operations. Is this correct?

More exact description:

Truncate is mutually exclusive with any IO on the file, be it paging or
non-paging. So, PagingIoResource is taken exclusively on truncate. This (and
also AcquireForCreateSection which guards the SECTION_OBJECT_POINTERS content)
is the only place where PagingIoResource is taken exclusively.

On the other hand, file extend can proceed in parallel with the paging IO, so,
PagingIoResource is not taken exclusively. Only the main resource is taken
exclusively to guard against a race between several extend operations.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

11

Maxim S. Shatskih wrote:

>I think the fundamental reason is simply because that the fact that
>paging I/O is going on means that Mm is transferring file data into or
>out of a file-mapped range, and the system would go down pretty hard if
>a portion of this mapped range suddenly was made invalid because of a
>file truncation while this memory is being used by the transfer.

No. The cause - as Daniel Lovinger from MS said - is that the disk blocks can
be reallocated for other file just after the truncation, so, truncation with
pending writes (pending in the SCSI tagged queue, for instance) can lead to the
pending writes overwriting the data in the new file.

Thus truncate is mutually exclusive with any kinds of IO on the file (both FCB
locks are grabbed).

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

That’s not a ‘no’, that is an ‘in addition to’. :slight_smile: Certainly if Mm is
currently flushing dirty pages to disk from a Cc file view that spans an
area of the file about to be truncated, Cc is not going to unmap the
view in response to being told about the truncate via CcSetFileSizes
while the flush is going on.

–
Nick Ryan (MVP for DDK)

12

Nick Ryan wrote:

Maxim S. Shatskih wrote:

>> I think the fundamental reason is simply because that the fact that
>> paging I/O is going on means that Mm is transferring file data into or
>> out of a file-mapped range, and the system would go down pretty hard if
>> a portion of this mapped range suddenly was made invalid because of a
>> file truncation while this memory is being used by the transfer.
>
>
>
> No. The cause - as Daniel Lovinger from MS said - is that the disk
> blocks can
> be reallocated for other file just after the truncation, so,
> truncation with
> pending writes (pending in the SCSI tagged queue, for instance) can
> lead to the
> pending writes overwriting the data in the new file.
>
> Thus truncate is mutually exclusive with any kinds of IO on the file
> (both FCB
> locks are grabbed).
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com

That’s not a ‘no’, that is an ‘in addition to’. :slight_smile: Certainly if Mm is
currently flushing dirty pages to disk from a Cc file view that spans an
area of the file about to be truncated, Cc is not going to unmap the
view in response to being told about the truncate via CcSetFileSizes
while the flush is going on.

Let me rephrase that as ‘it would be bad if Cc unmapped the view…’,
since it’s apparent that the onus is on the FSD to prevent this
situation from happening in this case by serializing itself with paging
I/O activity by taking the appropriate locks.

–
Nick Ryan (MVP for DDK)