Purge files

NTFSD
1

Hi,

I’m writing a xp file system filter driver and trying to purge a file
using
deviceioctrl. But, this purge will cause the driver to set the file object
to null instead.

This does not make sense since the same purge routine can be used to purge
the file during create, cleanup and close. (Thanks to pointers from this
mailing list)

I’ve noticed that mostly people recommend purging the files during cleanup
but not during deviceiocontrol, is there a contrain? Has anybody tried it
successfully or unsuccessfully? Can we achieve the same result if we purge
the whole volume instead?

Thanks,
John

2

Theoretically, you can purge that sucker at anytime, although I’ve
encountered problems purging on create or close. Doing at cleanup or at
the request of an IOCTL should both be fine, however. What are you
trying to accomplish by purging the file?

  • Nicholas Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of john
Sent: Sunday, October 06, 2002 11:41 PM
To: File Systems Developers
Subject: [ntfsd] Purge files

Hi,

I’m writing a xp file system filter driver and trying to
purge a file using deviceioctrl. But, this purge will cause
the driver to set the file object to null instead.

This does not make sense since the same purge routine can be
used to purge the file during create, cleanup and close.
(Thanks to pointers from this mailing list)

I’ve noticed that mostly people recommend purging the files
during cleanup but not during deviceiocontrol, is there a
contrain? Has anybody tried it successfully or
unsuccessfully? Can we achieve the same result if we purge
the whole volume instead?

Thanks,
John

You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to %%email.unsub%%

3

It was actually your instructions that I followed and they worked really
fine. Thank you very much for that! Purging in create and close did not
give me additional problem so far.

Whenever a user logs out from Windows, the cache of encrypted files that
he opened is still in memory and I’m trying to purge ALL encrypted files
after receiving an IOCTL from the event – this is what’s causing problem
for me. Or alternatively do it in cleanup and close but this would lose
some performance. I also purge in create for case when the file has been
opened before the user activates encryption.

Thanks,
John

Theoretically, you can purge that sucker at anytime, although I’ve
encountered problems purging on create or close. Doing at cleanup or at
the request of an IOCTL should both be fine, however. What are you
trying to accomplish by purging the file?

  • Nicholas Ryan

> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of john
> Sent: Sunday, October 06, 2002 11:41 PM
> To: File Systems Developers
> Subject: [ntfsd] Purge files
>
>
> Hi,
>
> I’m writing a xp file system filter driver and trying to
> purge a file using deviceioctrl. But, this purge will cause
> the driver to set the file object to null instead.
>
> This does not make sense since the same purge routine can be
> used to purge the file during create, cleanup and close.
> (Thanks to pointers from this mailing list)
>
> I’ve noticed that mostly people recommend purging the files
> during cleanup but not during deviceiocontrol, is there a
> contrain? Has anybody tried it successfully or
> unsuccessfully? Can we achieve the same result if we purge
> the whole volume instead?
>
> Thanks,
> John
>
> —
> You are currently subscribed to ntfsd as: xxxxx@nryan.com
> To unsubscribe send a blank email to %%email.unsub%%
>

4

Hmm… In that case I think you can just purge on cleanup and be done
with it. All of the user’s processes and thus all his open files will
automatically be closed when he logs out, so you shouldn’t have to worry
about invoking the purges yourself. Waiting for the cleanups should be
enough.

In general, purging the cache of an open file (that hasn’t been
cleaned-up yet) won’t get you much security-wise. Since the application
still has a user handle open to the file, it probably has data from the
file scattered in its own heap anyways. (And if it’s mapped the file,
the purge will just fail).

  • Nicholas Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of john
Sent: Monday, October 07, 2002 7:09 PM
To: File Systems Developers
Subject: [ntfsd] RE: Purge files

It was actually your instructions that I followed and they
worked really fine. Thank you very much for that! Purging in
create and close did not give me additional problem so far.

Whenever a user logs out from Windows, the cache of encrypted
files that he opened is still in memory and I’m trying to
purge ALL encrypted files after receiving an IOCTL from the
event – this is what’s causing problem for me. Or
alternatively do it in cleanup and close but this would lose
some performance. I also purge in create for case when the
file has been opened before the user activates encryption.

Thanks,
John

> Theoretically, you can purge that sucker at anytime, although I’ve
> encountered problems purging on create or close. Doing at
cleanup or
> at the request of an IOCTL should both be fine, however.
What are you
> trying to accomplish by purging the file?
>
> - Nicholas Ryan
>
>
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of john
> > Sent: Sunday, October 06, 2002 11:41 PM
> > To: File Systems Developers
> > Subject: [ntfsd] Purge files
> >
> >
> > Hi,
> >
> > I’m writing a xp file system filter driver and trying to
> > purge a file using deviceioctrl. But, this purge will cause
> > the driver to set the file object to null instead.
> >
> > This does not make sense since the same purge routine can be
> > used to purge the file during create, cleanup and close.
> > (Thanks to pointers from this mailing list)
> >
> > I’ve noticed that mostly people recommend purging the files
> > during cleanup but not during deviceiocontrol, is there a
> > contrain? Has anybody tried it successfully or
> > unsuccessfully? Can we achieve the same result if we purge
> > the whole volume instead?
> >
> > Thanks,
> > John
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > unsubscribe send a blank email to %%email.unsub%%
> >

You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to %%email.unsub%%

5

Hi,

I’m back again with the same issue. Was out of town for last 2 weeks due
to
another matter. Thanks Ryan for replies.

As described earlier, after receiving a IOCTL, I’ll search through all my
file contexts and purge all the files. The symptom I’m seeing while
stepping
through with a debugger is that the moment I step over the
CcPurgeCacheSection, it will reset my own file context values to zeroes
and
mangle the filename in the file object pointer.

Is there any locks or thread context or pageble memory that would cause
such
a scenario?

Thanks,
John

Hmm… In that case I think you can just purge on cleanup and be done
with it. All of the user’s processes and thus all his open files will
automatically be closed when he logs out, so you shouldn’t have to worry
about invoking the purges yourself. Waiting for the cleanups should be
enough.

In general, purging the cache of an open file (that hasn’t been
cleaned-up yet) won’t get you much security-wise. Since the application
still has a user handle open to the file, it probably has data from the
file scattered in its own heap anyways. (And if it’s mapped the file,
the purge will just fail).

  • Nicholas Ryan

> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of john
> Sent: Monday, October 07, 2002 7:09 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: Purge files
>
>
> It was actually your instructions that I followed and they
> worked really fine. Thank you very much for that! Purging in
> create and close did not give me additional problem so far.
>
> Whenever a user logs out from Windows, the cache of encrypted
> files that he opened is still in memory and I’m trying to
> purge ALL encrypted files after receiving an IOCTL from the
> event – this is what’s causing problem for me. Or
> alternatively do it in cleanup and close but this would lose
> some performance. I also purge in create for case when the
> file has been opened before the user activates encryption.
>
> Thanks,
> John
>
> > Theoretically, you can purge that sucker at anytime, although I’ve
> > encountered problems purging on create or close. Doing at
> cleanup or
> > at the request of an IOCTL should both be fine, however.
> What are you
> > trying to accomplish by purging the file?
> >
> > - Nicholas Ryan
> >
> >
> >
> > > -----Original Message-----
> > > From: xxxxx@lists.osr.com
> > > [mailto:xxxxx@lists.osr.com] On Behalf Of john
> > > Sent: Sunday, October 06, 2002 11:41 PM
> > > To: File Systems Developers
> > > Subject: [ntfsd] Purge files
> > >
> > >
> > > Hi,
> > >
> > > I’m writing a xp file system filter driver and trying to
> > > purge a file using deviceioctrl. But, this purge will cause
> > > the driver to set the file object to null instead.
> > >
> > > This does not make sense since the same purge routine can be
> > > used to purge the file during create, cleanup and close.
> > > (Thanks to pointers from this mailing list)
> > >
> > > I’ve noticed that mostly people recommend purging the files
> > > during cleanup but not during deviceiocontrol, is there a
> > > contrain? Has anybody tried it successfully or
> > > unsuccessfully? Can we achieve the same result if we purge
> > > the whole volume instead?
> > >
> > > Thanks,
> > > John
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > > unsubscribe send a blank email to %%email.unsub%%
> > >
>
> —
> You are currently subscribed to ntfsd as: xxxxx@nryan.com
> To unsubscribe send a blank email to %%email.unsub%%
>

6

Well, as far as FileObject->FileName goes, in general you can only
assume this is valid between the point you get an IRP_MJ_CREATE request
and the time you pass down that request. Even then, it may not be
meaningful in certain cases (file is opened by ID, etc). Best way is to
ask the file itself what its pathname is after you pass down the create
(see FileSpy in XP IFS kit).

There’s no way that purging the file can cause the FsContext values in
the FileObject to change, however. I’ve heard rumors that for some
network files the FsContext value can change over time, but I’ve never
encountered this myself. This will never happen for local files.

  • Nicholas Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of John Lee
Sent: Tuesday, October 22, 2002 3:20 AM
To: File Systems Developers
Subject: [ntfsd] RE: Purge files

Hi,

I’m back again with the same issue. Was out of town for last
2 weeks due to another matter. Thanks Ryan for replies.

As described earlier, after receiving a IOCTL, I’ll search
through all my file contexts and purge all the files. The
symptom I’m seeing while stepping through with a debugger is
that the moment I step over the CcPurgeCacheSection, it will
reset my own file context values to zeroes and mangle the
filename in the file object pointer.

Is there any locks or thread context or pageble memory that
would cause such a scenario?

Thanks,
John

> Hmm… In that case I think you can just purge on cleanup
and be done
> with it. All of the user’s processes and thus all his open
files will
> automatically be closed when he logs out, so you shouldn’t have to
> worry about invoking the purges yourself. Waiting for the cleanups
> should be enough.
>
> In general, purging the cache of an open file (that hasn’t been
> cleaned-up yet) won’t get you much security-wise. Since the
> application still has a user handle open to the file, it
probably has
> data from the file scattered in its own heap anyways. (And if it’s
> mapped the file, the purge will just fail).
>
> - Nicholas Ryan
>
>
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of john
> > Sent: Monday, October 07, 2002 7:09 PM
> > To: File Systems Developers
> > Subject: [ntfsd] RE: Purge files
> >
> >
> > It was actually your instructions that I followed and they
> > worked really fine. Thank you very much for that! Purging in
> > create and close did not give me additional problem so far.
> >
> > Whenever a user logs out from Windows, the cache of encrypted
> > files that he opened is still in memory and I’m trying to
> > purge ALL encrypted files after receiving an IOCTL from the
> > event – this is what’s causing problem for me. Or
> > alternatively do it in cleanup and close but this would lose
> > some performance. I also purge in create for case when the
> > file has been opened before the user activates encryption.
> >
> > Thanks,
> > John
> >
> > > Theoretically, you can purge that sucker at anytime,
although I’ve
> > > encountered problems purging on create or close. Doing at
> > cleanup or
> > > at the request of an IOCTL should both be fine, however.
> > What are you
> > > trying to accomplish by purging the file?
> > >
> > > - Nicholas Ryan
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: xxxxx@lists.osr.com
> > > > [mailto:xxxxx@lists.osr.com] On Behalf Of john
> > > > Sent: Sunday, October 06, 2002 11:41 PM
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] Purge files
> > > >
> > > >
> > > > Hi,
> > > >
> > > > I’m writing a xp file system filter driver and trying
to purge a
> > > > file using deviceioctrl. But, this purge will cause
the driver
> > > > to set the file object to null instead.
> > > >
> > > > This does not make sense since the same purge routine can be
> > > > used to purge the file during create, cleanup and
close. (Thanks
> > > > to pointers from this mailing list)
> > > >
> > > > I’ve noticed that mostly people recommend purging the files
> > > > during cleanup but not during deviceiocontrol, is there a
> > > > contrain? Has anybody tried it successfully or
unsuccessfully?
> > > > Can we achieve the same result if we purge the whole volume
> > > > instead?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > > > unsubscribe send a blank email to %%email.unsub%%
> > > >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > unsubscribe send a blank email to %%email.unsub%%
> >

You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to %%email.unsub%%

7

Found the problem. The CcPurgeCacheSection can cause the I/O Manager to
issue a close on the file. This causes my driver to remove my own file
context (which becomes zeroes) in close before IOCTL was done with it.
I’ve fixed the problem by adding a reference to the file. Also,
dereference can only be done after I have finished removing my file
context in the IOCTL because the dereference will close the file as well.

In addition, other options I was considering is to send a file open with
FILE_NO_INTERMEDIATE_BUFFERING flag set as recommended in this mailing
list.

Thanks,
John Lee

Well, as far as FileObject->FileName goes, in general you can only
assume this is valid between the point you get an IRP_MJ_CREATE request
and the time you pass down that request. Even then, it may not be
meaningful in certain cases (file is opened by ID, etc). Best way is to
ask the file itself what its pathname is after you pass down the create
(see FileSpy in XP IFS kit).

There’s no way that purging the file can cause the FsContext values in
the FileObject to change, however. I’ve heard rumors that for some
network files the FsContext value can change over time, but I’ve never
encountered this myself. This will never happen for local files.

  • Nicholas Ryan

> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of John Lee
> Sent: Tuesday, October 22, 2002 3:20 AM
> To: File Systems Developers
> Subject: [ntfsd] RE: Purge files
>
>
> Hi,
>
> I’m back again with the same issue. Was out of town for last
> 2 weeks due to another matter. Thanks Ryan for replies.
>
> As described earlier, after receiving a IOCTL, I’ll search
> through all my file contexts and purge all the files. The
> symptom I’m seeing while stepping through with a debugger is
> that the moment I step over the CcPurgeCacheSection, it will
> reset my own file context values to zeroes and mangle the
> filename in the file object pointer.
>
> Is there any locks or thread context or pageble memory that
> would cause such a scenario?
>
> Thanks,
> John
>
> > Hmm… In that case I think you can just purge on cleanup
> and be done
> > with it. All of the user’s processes and thus all his open
> files will
> > automatically be closed when he logs out, so you shouldn’t have to
> > worry about invoking the purges yourself. Waiting for the cleanups
> > should be enough.
> >
> > In general, purging the cache of an open file (that hasn’t been
> > cleaned-up yet) won’t get you much security-wise. Since the
> > application still has a user handle open to the file, it
> probably has
> > data from the file scattered in its own heap anyways. (And if it’s
> > mapped the file, the purge will just fail).
> >
> > - Nicholas Ryan
> >
> >
> >
> > > -----Original Message-----
> > > From: xxxxx@lists.osr.com
> > > [mailto:xxxxx@lists.osr.com] On Behalf Of john
> > > Sent: Monday, October 07, 2002 7:09 PM
> > > To: File Systems Developers
> > > Subject: [ntfsd] RE: Purge files
> > >
> > >
> > > It was actually your instructions that I followed and they
> > > worked really fine. Thank you very much for that! Purging in
> > > create and close did not give me additional problem so far.
> > >
> > > Whenever a user logs out from Windows, the cache of encrypted
> > > files that he opened is still in memory and I’m trying to
> > > purge ALL encrypted files after receiving an IOCTL from the
> > > event – this is what’s causing problem for me. Or
> > > alternatively do it in cleanup and close but this would lose
> > > some performance. I also purge in create for case when the
> > > file has been opened before the user activates encryption.
> > >
> > > Thanks,
> > > John
> > >
> > > > Theoretically, you can purge that sucker at anytime,
> although I’ve
> > > > encountered problems purging on create or close. Doing at
> > > cleanup or
> > > > at the request of an IOCTL should both be fine, however.
> > > What are you
> > > > trying to accomplish by purging the file?
> > > >
> > > > - Nicholas Ryan
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: xxxxx@lists.osr.com
> > > > > [mailto:xxxxx@lists.osr.com] On Behalf Of john
> > > > > Sent: Sunday, October 06, 2002 11:41 PM
> > > > > To: File Systems Developers
> > > > > Subject: [ntfsd] Purge files
> > > > >
> > > > >
> > > > > Hi,
> > > > >
> > > > > I’m writing a xp file system filter driver and trying
> to purge a
> > > > > file using deviceioctrl. But, this purge will cause
> the driver
> > > > > to set the file object to null instead.
> > > > >
> > > > > This does not make sense since the same purge routine can be
> > > > > used to purge the file during create, cleanup and
> close. (Thanks
> > > > > to pointers from this mailing list)
> > > > >
> > > > > I’ve noticed that mostly people recommend purging the files
> > > > > during cleanup but not during deviceiocontrol, is there a
> > > > > contrain? Has anybody tried it successfully or
> unsuccessfully?
> > > > > Can we achieve the same result if we purge the whole volume
> > > > > instead?
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > > > > unsubscribe send a blank email to %%email.unsub%%
> > > > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > > unsubscribe send a blank email to %%email.unsub%%
> > >
>
> —
> You are currently subscribed to ntfsd as: xxxxx@nryan.com
> To unsubscribe send a blank email to %%email.unsub%%
>