FileObject CurrentByteOffset "wrong"

NTFSD
1

I was under the impression that the file offset (CurrentByteOffset) of an
open FileObject was maintained by the lower level FSD. My prototype
active file filter (decrypter) does neither compress nor expand the data
read by the FSD, so it seems logical that updates to the CurrentByteOffset
made by the FSD would remain valid.

Why, then, do I see some IRP_MJ_READ requests with an indicted offset of
0, yet the data read by the FSD,upon inspection, is clearly not from file
offset 0? Is there something I am not properly propagating?

TIA,
Bill

2

Ever wondered why stack location for read/write has CurrentByteOffset
field?

Bill wrote:

I was under the impression that the file offset (CurrentByteOffset) of an
open FileObject was maintained by the lower level FSD. My prototype
active file filter (decrypter) does neither compress nor expand the data
read by the FSD, so it seems logical that updates to the CurrentByteOffset
made by the FSD would remain valid.

Why, then, do I see some IRP_MJ_READ requests with an indicted offset of
0, yet the data read by the FSD,upon inspection, is clearly not from file
offset 0? Is there something I am not properly propagating?

TIA,
Bill

You are currently subscribed to ntfsd as: xxxxx@alfasp.com
To unsubscribe send a blank email to %%email.unsub%%

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32 developers.

3

Of course I have, but I have yet to discover a “rule” for when to pay
attention to which field, or how to combine them. The field I’ve seen
that makes most sense" is the one in the FileObject. In fact, my code
works against most files when paying attention to it; in the cases where
it fails, I am getting a zero in the Current ByteOffset of the FileObject.

Since you seem to know, would you care to enlighten me?

4

The rule is: don’t look at it.

The fileobject current byte offset is used by the IO manager to fill in
the IRP if it is a synchronous operation that didn’t pass an explicit
byte offset into NtRead/WriteFile. The IRP stack field is always where
the IO will occur, regardless of whether it is a synchronous or
asynchronous operation.

You’re just getting a lot of synchronous IO for which, of course, the
fileobject cbo happens to be the same as what is in the IRP. This breaks
down when async IO goes by, which is what I assume is happening here.

–
This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Bill [mailto:xxxxx@endwell.net]
Sent: Monday, November 25, 2002 10:01 AM
To: File Systems Developers
Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”

Of course I have, but I have yet to discover a “rule” for when to pay
attention to which field, or how to combine them. The field I’ve seen
that makes most sense" is the one in the FileObject. In fact, my code
works against most files when paying attention to it; in the cases where
it fails, I am getting a zero in the Current ByteOffset of the
FileObject.

Since you seem to know, would you care to enlighten me?

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

5

A filter just needs to pay attention to the ByteOffset field in the
stack location (pIrpSp->Parameters.Read.ByteOffset or
pIrpSp->Parameters.Write.ByteOffset), EXCEPT when HighPart is -1 and
LowPart is one of the two constants FILE_WRITE_TO_END_OF_FILE or
FILE_USE_FILE_POINTER_POSITION.

In the case of FILE_WRITE_TO_END_OF_FILE, then you need to interpret the
write as happening at the current EOF (whatever this means in the
context of your filter).

In the case of FILE_USE_FILE_POINTER_POSITION, you pull the intended
offset from the file object.

  • Nicholas Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bill
Sent: Monday, November 25, 2002 10:01 AM
To: File Systems Developers
Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”

Of course I have, but I have yet to discover a “rule” for
when to pay attention to which field, or how to combine them.
The field I’ve seen that makes most sense" is the one in the
FileObject. In fact, my code works against most files when
paying attention to it; in the cases where it fails, I am
getting a zero in the Current ByteOffset of the FileObject.

Since you seem to know, would you care to enlighten me?

You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to %%email.unsub%%

6

So a filter should never have to handle FILE_USE_FILE_POINTER_POSITION?
I thought this MIGHT be the case because FastFat does not check for this
flag. However, the flag is listed right next to
FILE_WRITE_TO_END_OF_FILE in ntifs.h, which FastFat DOES check for. I
decided to play it safe and handle both.

  • Nicholas Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Daniel Lovinger
Sent: Monday, November 25, 2002 11:27 AM
To: File Systems Developers
Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”

The rule is: don’t look at it.

The fileobject current byte offset is used by the IO manager
to fill in the IRP if it is a synchronous operation that
didn’t pass an explicit byte offset into NtRead/WriteFile.
The IRP stack field is always where the IO will occur,
regardless of whether it is a synchronous or asynchronous operation.

You’re just getting a lot of synchronous IO for which, of
course, the fileobject cbo happens to be the same as what is
in the IRP. This breaks down when async IO goes by, which is
what I assume is happening here.

–
This posting is provided “AS IS” with no warranties, and
confers no rights.

-----Original Message-----
From: Bill [mailto:xxxxx@endwell.net]
Sent: Monday, November 25, 2002 10:01 AM
To: File Systems Developers
Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”

Of course I have, but I have yet to discover a “rule” for
when to pay attention to which field, or how to combine them.
The field I’ve seen that makes most sense" is the one in the
FileObject. In fact, my code works against most files when
paying attention to it; in the cases where it fails, I am
getting a zero in the Current ByteOffset of the FileObject.

Since you seem to know, would you care to enlighten me?

You are currently subscribed to ntfsd as:
xxxxx@windows.microsoft.com To unsubscribe send a blank email
to %%email.unsub%%

You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to %%email.unsub%%

7

Thanks, Nick. Clear, and to the point. Saved me a bunch of DbgPrint
statements trying to discover this heuristically.

8

That flag is never seen in the driver stack. You pass that into
NtRead/WriteFile (with HighPart == -1 as well) when you want IO to use
the file object cbo as the IO offset. It is equivalent to passing in a
NULL ByteOffset pointer to those APIs.

–
This posting is provided “AS IS” with no warranties, and
confers no rights.

-----Original Message-----
From: Nicholas Ryan [mailto:xxxxx@nryan.com]
Sent: Monday, November 25, 2002 11:55 AM
To: File Systems Developers
Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”

So a filter should never have to handle FILE_USE_FILE_POINTER_POSITION?
I thought this MIGHT be the case because FastFat does not check for this
flag. However, the flag is listed right next to
FILE_WRITE_TO_END_OF_FILE in ntifs.h, which FastFat DOES check for. I
decided to play it safe and handle both.

  • Nicholas Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Daniel Lovinger
Sent: Monday, November 25, 2002 11:27 AM
To: File Systems Developers
Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”

The rule is: don’t look at it.

The fileobject current byte offset is used by the IO manager
to fill in the IRP if it is a synchronous operation that
didn’t pass an explicit byte offset into NtRead/WriteFile.
The IRP stack field is always where the IO will occur,
regardless of whether it is a synchronous or asynchronous operation.

You’re just getting a lot of synchronous IO for which, of
course, the fileobject cbo happens to be the same as what is
in the IRP. This breaks down when async IO goes by, which is
what I assume is happening here.

–
This posting is provided “AS IS” with no warranties, and
confers no rights.

-----Original Message-----
From: Bill [mailto:xxxxx@endwell.net]
Sent: Monday, November 25, 2002 10:01 AM
To: File Systems Developers
Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”

Of course I have, but I have yet to discover a “rule” for
when to pay attention to which field, or how to combine them.
The field I’ve seen that makes most sense" is the one in the
FileObject. In fact, my code works against most files when
paying attention to it; in the cases where it fails, I am
getting a zero in the Current ByteOffset of the FileObject.

Since you seem to know, would you care to enlighten me?

You are currently subscribed to ntfsd as:
xxxxx@windows.microsoft.com To unsubscribe send a blank email
to %%email.unsub%%

You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to %%email.unsub%%

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

9

NT4 FASTFAT uses FILE_WRITE_TO_END_OF_FILE but does not use
FILE_USE_FILE_POINTER_POSITION.

Max

----- Original Message -----
From: “Nicholas Ryan”
To: “File Systems Developers”
Sent: Monday, November 25, 2002 10:27 PM
Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”

> A filter just needs to pay attention to the ByteOffset field in the
> stack location (pIrpSp->Parameters.Read.ByteOffset or
> pIrpSp->Parameters.Write.ByteOffset), EXCEPT when HighPart is -1 and
> LowPart is one of the two constants FILE_WRITE_TO_END_OF_FILE or
> FILE_USE_FILE_POINTER_POSITION.
>
> In the case of FILE_WRITE_TO_END_OF_FILE, then you need to interpret
the
> write as happening at the current EOF (whatever this means in the
> context of your filter).
>
> In the case of FILE_USE_FILE_POINTER_POSITION, you pull the intended
> offset from the file object.
>
> - Nicholas Ryan
>
>
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Bill
> > Sent: Monday, November 25, 2002 10:01 AM
> > To: File Systems Developers
> > Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”
> >
> >
> > Of course I have, but I have yet to discover a “rule” for
> > when to pay attention to which field, or how to combine them.
> > The field I’ve seen that makes most sense" is the one in the
> > FileObject. In fact, my code works against most files when
> > paying attention to it; in the cases where it fails, I am
> > getting a zero in the Current ByteOffset of the FileObject.
> >
> > Since you seem to know, would you care to enlighten me?
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to %%email.unsub%%
>

10

Is there a change that these will appear if IRP_NOCACHE bit is set?

Nicholas Ryan wrote:

A filter just needs to pay attention to the ByteOffset field in the
stack location (pIrpSp->Parameters.Read.ByteOffset or
pIrpSp->Parameters.Write.ByteOffset), EXCEPT when HighPart is -1 and
LowPart is one of the two constants FILE_WRITE_TO_END_OF_FILE or
FILE_USE_FILE_POINTER_POSITION.

In the case of FILE_WRITE_TO_END_OF_FILE, then you need to interpret the
write as happening at the current EOF (whatever this means in the
context of your filter).

In the case of FILE_USE_FILE_POINTER_POSITION, you pull the intended
offset from the file object.

  • Nicholas Ryan

> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Bill
> Sent: Monday, November 25, 2002 10:01 AM
> To: File Systems Developers
> Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”
>
>
> Of course I have, but I have yet to discover a “rule” for
> when to pay attention to which field, or how to combine them.
> The field I’ve seen that makes most sense" is the one in the
> FileObject. In fact, my code works against most files when
> paying attention to it; in the cases where it fails, I am
> getting a zero in the Current ByteOffset of the FileObject.
>
> Since you seem to know, would you care to enlighten me?
>
> —
> You are currently subscribed to ntfsd as: xxxxx@nryan.com
> To unsubscribe send a blank email to %%email.unsub%%
>

You are currently subscribed to ntfsd as: xxxxx@alfasp.com
To unsubscribe send a blank email to %%email.unsub%%

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32
developers.

11

Yes, FILE_WRITE_TO_END_OF_FILE is compatible with IRP_NOCACHE (but not
IRP_PAGING_IO).

  • Nicholas Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Dejan Maksimovic
Sent: Monday, December 02, 2002 2:53 AM
To: File Systems Developers
Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”

Is there a change that these will appear if IRP_NOCACHE
bit is set?

Nicholas Ryan wrote:

> A filter just needs to pay attention to the ByteOffset field in the
> stack location (pIrpSp->Parameters.Read.ByteOffset or
> pIrpSp->Parameters.Write.ByteOffset), EXCEPT when HighPart is -1 and
> LowPart is one of the two constants FILE_WRITE_TO_END_OF_FILE or
> FILE_USE_FILE_POINTER_POSITION.
>
> In the case of FILE_WRITE_TO_END_OF_FILE, then you need to
interpret
> the write as happening at the current EOF (whatever this
means in the
> context of your filter).
>
> In the case of FILE_USE_FILE_POINTER_POSITION, you pull the
intended
> offset from the file object.
>
> - Nicholas Ryan
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Bill
> > Sent: Monday, November 25, 2002 10:01 AM
> > To: File Systems Developers
> > Subject: [ntfsd] Re: FileObject CurrentByteOffset “wrong”
> >
> >
> > Of course I have, but I have yet to discover a “rule” for when to
> > pay attention to which field, or how to combine them. The field
> > I’ve seen that makes most sense" is the one in the
FileObject. In
> > fact, my code works against most files when paying
attention to it;
> > in the cases where it fails, I am getting a zero in the Current
> > ByteOffset of the FileObject.
> >
> > Since you seem to know, would you care to enlighten me?
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > unsubscribe send a blank email to %%email.unsub%%
> >
>
> —
> You are currently subscribed to ntfsd as: xxxxx@alfasp.com
> To unsubscribe send a blank email to %%email.unsub%%

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32
developers. Alfa File Protector - File protection and hiding
library for Win32 developers.

You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to %%email.unsub%%

12

Works for me! Thanks.

Nicholas Ryan wrote:

Yes, FILE_WRITE_TO_END_OF_FILE is compatible with IRP_NOCACHE (but not
IRP_PAGING_IO).

–
Kind regards, Dejan M. www.alfasp.com
E-mail: xxxxx@alfasp.com ICQ#: 56570367
Alfa File Monitor - File monitoring library for Win32 developers.
Alfa File Protector - File protection and hiding library for Win32
developers.