RE: SetFile.AdvanceOnly

I am trying to understand the persistent length of a cached file. By this
I mean the length that would be seen after the machine suffered a power
cycle (no shutdown). Imagine some file is opened for cached i/o, and
writes to the file extend the file. Here is what I see in the filter
driver (basically its filemon) …

  1. IRP_MJ_CREATE
  2. IRP_MJ_WRITE
  3. IRP_MJ_WRITE irp page bits set
  4. IRP_MJ_SET_INFORMATION irp page bits set, advance only true,
    EndOfFileInformation
  5. IRP_MJ_CLEANUP
  6. IRP_MJ_CLOSE

The IRP_MJ_WRITE at 2. is not page i/o.

The IRP_MJ_SET_INFORMATION at 4 has EndOfFile equal to the expected size
of the extended file; using FsRtlGetFileSize in completion gives the file
size equal to the EndOfFile in the dispatch. (This is true unless the new
size of less than 4K, in which case EndOfFile is 4K in dispatch, and
FsRtlGetFileSize in completion is the proper size.)

I dont see any IRP_MJ_SET_INFORMATION before that at 4. If I bluescreen
the system in dispatch of 4, then on reboot the file size has not been
changed. i.e it it was 33 before I ran the program and bluescreened here,
its still 33 on reboot. On the other hand if let 4 succeed then the file
size i extended. So it looks like 4 is where the persisted length can be
seen.

I get the impression that this not quite what I expect to see on the basis
of Rob’s message here. Can anyone suggest whether this is what I should
expect to see? Or a cleaner way to get at the persisted file size?

Thanks in advance - Lyndon

Specifically, AdvanceOnly is used by the cache manager to tell the FSD to
update the file’s ValidDataLength, not to allocate storage. The storage was
already allocated by a previous IRP_MJ_SET_INFORMATION. Accordingly, NTFS
uses the AdvanceOnly flag to increase the ValidDataLength, and does not
allocate additional storage. Since FASTFAT has no concept of
ValidDataLength, it uses the AdvanceOnly flag to update the file size stored
in the directory entry, but does not allocate storage.

> -----Original Message-----
> From: Tony Mason [mailto:xxxxx@osr.com]
> Sent: Monday, May 08, 2000 1:52 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: SetFile.AdvanceOnly
>
>
> Advance only is set when the Cache Manager is trying to
> ensure the file is
> of sufficient size. In general, the VM system does not do
> paging I/O writes
> that extend the size of the file.
>
> Regards,
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com http:
> >
> >
> > -----Original Message-----
> > From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
> > Sent: Monday, May 08, 2000 2:51 PM
> > To: File Systems Developers
> > Subject: [ntfsd] SetFile.AdvanceOnly
> >
> >
> >
> >
> > Can anyone tell me exactly where when and how the
> > ‘AvanceOnly’ flag
> > of the SetFile parameter is used?
> > Thanks
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nsisw.com
> > To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> ></http:>

I’m not sure how you’re reading my prior message. This sentence in particular suggests the behavior you are observing:

“Since FASTFAT has no concept of ValidDataLength, it uses the AdvanceOnly flag to update the file size stored in the directory entry, but does not allocate storage.”

Meaning that FASTFAT updates its on disk structures with the new end of file at this stage. I suspect NTFS does as well.

-----Original Message-----
From: Lyndon J. Clarke [mailto:xxxxx@gcplc.com]
Sent: Thursday, October 10, 2002 10:00 AM
To: File Systems Developers
Subject: [ntfsd] RE: SetFile.AdvanceOnly

I am trying to understand the persistent length of a cached file. By this
I mean the length that would be seen after the machine suffered a power
cycle (no shutdown). Imagine some file is opened for cached i/o, and
writes to the file extend the file. Here is what I see in the filter
driver (basically its filemon) …

  1. IRP_MJ_CREATE
  2. IRP_MJ_WRITE
  3. IRP_MJ_WRITE irp page bits set
  4. IRP_MJ_SET_INFORMATION irp page bits set, advance only true,
    EndOfFileInformation
  5. IRP_MJ_CLEANUP
  6. IRP_MJ_CLOSE

The IRP_MJ_WRITE at 2. is not page i/o.

The IRP_MJ_SET_INFORMATION at 4 has EndOfFile equal to the expected size
of the extended file; using FsRtlGetFileSize in completion gives the file
size equal to the EndOfFile in the dispatch. (This is true unless the new
size of less than 4K, in which case EndOfFile is 4K in dispatch, and
FsRtlGetFileSize in completion is the proper size.)

I dont see any IRP_MJ_SET_INFORMATION before that at 4. If I bluescreen
the system in dispatch of 4, then on reboot the file size has not been
changed. i.e it it was 33 before I ran the program and bluescreened here,
its still 33 on reboot. On the other hand if let 4 succeed then the file
size i extended. So it looks like 4 is where the persisted length can be
seen.

I get the impression that this not quite what I expect to see on the basis
of Rob’s message here. Can anyone suggest whether this is what I should
expect to see? Or a cleaner way to get at the persisted file size?

Thanks in advance - Lyndon

Specifically, AdvanceOnly is used by the cache manager to tell the FSD to
update the file’s ValidDataLength, not to allocate storage. The storage was
already allocated by a previous IRP_MJ_SET_INFORMATION. Accordingly, NTFS
uses the AdvanceOnly flag to increase the ValidDataLength, and does not
allocate additional storage. Since FASTFAT has no concept of
ValidDataLength, it uses the AdvanceOnly flag to update the file size stored
in the directory entry, but does not allocate storage.

> -----Original Message-----
> From: Tony Mason [mailto:xxxxx@osr.com]
> Sent: Monday, May 08, 2000 1:52 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: SetFile.AdvanceOnly
>
>
> Advance only is set when the Cache Manager is trying to
> ensure the file is
> of sufficient size. In general, the VM system does not do
> paging I/O writes
> that extend the size of the file.
>
> Regards,
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com http:
> >
> >
> > -----Original Message-----
> > From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
> > Sent: Monday, May 08, 2000 2:51 PM
> > To: File Systems Developers
> > Subject: [ntfsd] SetFile.AdvanceOnly
> >
> >
> >
> >
> > Can anyone tell me exactly where when and how the
> > ‘AvanceOnly’ flag
> > of the SetFile parameter is used?
> > Thanks
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nsisw.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >


You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%</http:>

In case it wasn’t clear, that corresponds to your case #4. Hence, that is when the on-disk size is updated.

-----Original Message-----
From: Fuller, Rob
Sent: Thursday, October 10, 2002 11:03 AM
To: ‘File Systems Developers’
Subject: RE: [ntfsd] RE: SetFile.AdvanceOnly

I’m not sure how you’re reading my prior message. This sentence in particular suggests the behavior you are observing:

“Since FASTFAT has no concept of ValidDataLength, it uses the AdvanceOnly flag to update the file size stored in the directory entry, but does not allocate storage.”

Meaning that FASTFAT updates its on disk structures with the new end of file at this stage. I suspect NTFS does as well.

-----Original Message-----
From: Lyndon J. Clarke [mailto:xxxxx@gcplc.com]
Sent: Thursday, October 10, 2002 10:00 AM
To: File Systems Developers
Subject: [ntfsd] RE: SetFile.AdvanceOnly

I am trying to understand the persistent length of a cached file. By this
I mean the length that would be seen after the machine suffered a power
cycle (no shutdown). Imagine some file is opened for cached i/o, and
writes to the file extend the file. Here is what I see in the filter
driver (basically its filemon) …

  1. IRP_MJ_CREATE
  2. IRP_MJ_WRITE
  3. IRP_MJ_WRITE irp page bits set
  4. IRP_MJ_SET_INFORMATION irp page bits set, advance only true,
    EndOfFileInformation
  5. IRP_MJ_CLEANUP
  6. IRP_MJ_CLOSE

The IRP_MJ_WRITE at 2. is not page i/o.

The IRP_MJ_SET_INFORMATION at 4 has EndOfFile equal to the expected size
of the extended file; using FsRtlGetFileSize in completion gives the file
size equal to the EndOfFile in the dispatch. (This is true unless the new
size of less than 4K, in which case EndOfFile is 4K in dispatch, and
FsRtlGetFileSize in completion is the proper size.)

I dont see any IRP_MJ_SET_INFORMATION before that at 4. If I bluescreen
the system in dispatch of 4, then on reboot the file size has not been
changed. i.e it it was 33 before I ran the program and bluescreened here,
its still 33 on reboot. On the other hand if let 4 succeed then the file
size i extended. So it looks like 4 is where the persisted length can be
seen.

I get the impression that this not quite what I expect to see on the basis
of Rob’s message here. Can anyone suggest whether this is what I should
expect to see? Or a cleaner way to get at the persisted file size?

Thanks in advance - Lyndon

Specifically, AdvanceOnly is used by the cache manager to tell the FSD to
update the file’s ValidDataLength, not to allocate storage. The storage was
already allocated by a previous IRP_MJ_SET_INFORMATION. Accordingly, NTFS
uses the AdvanceOnly flag to increase the ValidDataLength, and does not
allocate additional storage. Since FASTFAT has no concept of
ValidDataLength, it uses the AdvanceOnly flag to update the file size stored
in the directory entry, but does not allocate storage.

> -----Original Message-----
> From: Tony Mason [mailto:xxxxx@osr.com]
> Sent: Monday, May 08, 2000 1:52 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: SetFile.AdvanceOnly
>
>
> Advance only is set when the Cache Manager is trying to
> ensure the file is
> of sufficient size. In general, the VM system does not do
> paging I/O writes
> that extend the size of the file.
>
> Regards,
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com http:
> >
> >
> > -----Original Message-----
> > From: Smith, Joel [mailto:xxxxx@ntpsoftware.com]
> > Sent: Monday, May 08, 2000 2:51 PM
> > To: File Systems Developers
> > Subject: [ntfsd] SetFile.AdvanceOnly
> >
> >
> >
> >
> > Can anyone tell me exactly where when and how the
> > ‘AvanceOnly’ flag
> > of the SetFile parameter is used?
> > Thanks
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nsisw.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >


You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%</http:>

thanks for your helpful responses. i was not thinking about the sentence
about FASTFAT since i am just atop ntfs here. on basis of your message i
had thought i should have seen a prior IRP_MJ_SET_INFORMATION.