File contexts

Hello,

I would like to remove file contexts when the file is going to be deleted, but I don’t know when exactly is the right time. The file can be marked as delete pending by calling NtSetInformationFile(FileDisposition) or when it’s opened with FILE_DELETE_ON_CLOSE. This flag is stored in the file contexts, which are created for the files from the monitored folders selected by a user. The files may be already opened when the user will decide to monitor such folders, that’s why I may not receive all IRP_MJ_CREATE requests. On the contrary, I’ll receive all IRP_MJ_CLEANUP requests and I know, the right time to remove file contexts is the last IRP_MJ_CLEANUP before IRP_MJ_CLOSE.

Is it possible to recognize the last IRP_MJ_CLEANUP?

Thanks for sober advices,
Petr

Hello Peter!

Have you seen “Tracking State and Context - Reference Counting for File System Filter Drivers” (http://www.osronline.com/article.cfm?id=102) by the OSR?

Regards,
Dmitry Fleytman

P.S. Happy new year!

Since contexts existed before Filter Manager I don’t know if you’re referring to a legacy or a mini. So, assuming it’s a mini, the answer is easy - just register a cleanup callback. Note that you may not see the callback for some time after the file is closed under certain circumstances but when it does get called, you can safely release your context resources. Note that you don’t have to delete your context because the internal FM context is being torn down.

That’s the simple case and relates to streams. If you’re talking about files which can have multiple streams, you may have multiple deletes, one for each stream being deleted. If you’re wanting to remove a file context then you’ll have to wait for all stream cleanups but again, you can use a cleanup callback for a file or a stream context rather than waiting for the close. (Or at least for the streams that you know has their disposition set.)

Now, one thing you need to be aware of is that if you’re not seeing all the creates on streams, you won’t be able to identify the streams that were opened delete-on-close. That’s a real pain with DOC because if you query the disposition and the stream was opened DOC, the disposition will not be set. I worked on a driver that had to know if a stream was marked for deletion and the driver had to monitor all opens to watch for DOC. Also keep in mind that the disposition could be cleared later before the stream is closed.

“Petr Kurtin” wrote in message news:xxxxx@ntfsd…
Hello,

I would like to remove file contexts when the file is going to be deleted, but I don’t know when exactly is the right time. The file can be marked as delete pending by calling NtSetInformationFile(FileDisposition) or when it’s opened with FILE_DELETE_ON_CLOSE. This flag is stored in the file contexts, which are created for the files from the monitored folders selected by a user. The files may be already opened when the user will decide to monitor such folders, that’s why I may not receive all IRP_MJ_CREATE requests. On the contrary, I’ll receive all IRP_MJ_CLEANUP requests and I know, the right time to remove file contexts is the last IRP_MJ_CLEANUP before IRP_MJ_CLOSE.

Is it possible to recognize the last IRP_MJ_CLEANUP?

Thanks for sober advices,
Petr

AFAIK the DOC problem is not soluble. It is possible that some filter
(legacy) below uses shadow device method (or worse) and the IRP_MJ_CREATE
which has DOC is not observable even though the later IRP_MJ_CLEANUP is
observable.

“Jerry Kelley” wrote in message news:xxxxx@ntfsd…
Since contexts existed before Filter Manager I don’t know if you’re
referring to a legacy or a mini. So, assuming it’s a mini, the answer is
easy - just register a cleanup callback. Note that you may not see the
callback for some time after the file is closed under certain circumstances
but when it does get called, you can safely release your context resources.
Note that you don’t have to delete your context because the internal FM
context is being torn down.

That’s the simple case and relates to streams. If you’re talking about files
which can have multiple streams, you may have multiple deletes, one for each
stream being deleted. If you’re wanting to remove a file context then you’ll
have to wait for all stream cleanups but again, you can use a cleanup
callback for a file or a stream context rather than waiting for the close.
(Or at least for the streams that you know has their disposition set.)

Now, one thing you need to be aware of is that if you’re not seeing all the
creates on streams, you won’t be able to identify the streams that were
opened delete-on-close. That’s a real pain with DOC because if you query the
disposition and the stream was opened DOC, the disposition will not be set.
I worked on a driver that had to know if a stream was marked for deletion
and the driver had to monitor all opens to watch for DOC. Also keep in mind
that the disposition could be cleared later before the stream is closed.

“Petr Kurtin” wrote in message news:xxxxx@ntfsd…
Hello,

I would like to remove file contexts when the file is going to be deleted,
but I don’t know when exactly is the right time. The file can be marked as
delete pending by calling NtSetInformationFile(FileDisposition) or when it’s
opened with FILE_DELETE_ON_CLOSE. This flag is stored in the file contexts,
which are created for the files from the monitored folders selected by a
user. The files may be already opened when the user will decide to monitor
such folders, that’s why I may not receive all IRP_MJ_CREATE requests. On
the contrary, I’ll receive all IRP_MJ_CLEANUP requests and I know, the right
time to remove file contexts is the last IRP_MJ_CLEANUP before IRP_MJ_CLOSE.

Is it possible to recognize the last IRP_MJ_CLEANUP?

Thanks for sober advices,
Petr

Hello Petr

I assume (2007) you speak of a mini-filter driver. I am wondering what it is
about the filter manager context cleanup model which is not sufficient for
your purposes.

Thanks
Lyndon

“Petr Kurtin” wrote in message news:xxxxx@ntfsd…
Hello,

I would like to remove file contexts when the file is going to be deleted,
but I don’t know when exactly is the right time. The file can be marked as
delete pending by calling NtSetInformationFile(FileDisposition) or when it’s
opened with FILE_DELETE_ON_CLOSE. This flag is stored in the file contexts,
which are created for the files from the monitored folders selected by a
user. The files may be already opened when the user will decide to monitor
such folders, that’s why I may not receive all IRP_MJ_CREATE requests. On
the contrary, I’ll receive all IRP_MJ_CLEANUP requests and I know, the right
time to remove file contexts is the last IRP_MJ_CLEANUP before IRP_MJ_CLOSE.

Is it possible to recognize the last IRP_MJ_CLEANUP?

Thanks for sober advices,
Petr

Whether it’s soluble or not, I know from a lot of experience that you can
only detect DOC on create as an option and you have to respect the create
that you get. That is, unless you work on the kernel team and know a way to
get the true disposition when DOC is involved.

The bottom line is that DOC is just like setting the disposition except
querying the disposition doesn’t come back as being set if DOC is active.
However, it can be overridden by clearing the disposition. This I know for a
fact as of a couple of months ago. If something in the architecture has
changed since then, I stand corrected.

Now, if some filter above is hijacking the create and changing options, well
then, what can you do? I’m just providing information on how DOC works when
you see the create in it’s “pure” form. Of course, that’s not to say a lower
filter could change the create options as well. The create should behave as
your filter saw it but anyone who’s worked in this area for some time knows
that, in reality, all bets are off and you do what you can.

That said, assuming that DOC is a function that just can’t be handled is not
going to fly. If you see DOC you have to assume the stream is marked for
deletion unless you see the disposition get cleared. Of course, it could be
set again later but that’s just the fun of file deletion. Almost as fun as
renames.

From my experience, when my filter saw DOC it handled it. If a shadow device
or some other filter is playing games well you can blame the results on
them. I’ve encountered quite a few poorly written third-party filters that
play games and break other filters or, even worse, just bring the system
down due to some (illegal) shortcut, assumption, or inexperience.

jerry

“Lyndon J. Clarke” wrote in message
news:xxxxx@ntfsd…
> AFAIK the DOC problem is not soluble. It is possible that some filter
> (legacy) below uses shadow device method (or worse) and the IRP_MJ_CREATE
> which has DOC is not observable even though the later IRP_MJ_CLEANUP is
> observable.
>
> “Jerry Kelley” wrote in message news:xxxxx@ntfsd…
> Since contexts existed before Filter Manager I don’t know if you’re
> referring to a legacy or a mini. So, assuming it’s a mini, the answer is
> easy - just register a cleanup callback. Note that you may not see the
> callback for some time after the file is closed under certain
> circumstances but when it does get called, you can safely release your
> context resources. Note that you don’t have to delete your context because
> the internal FM context is being torn down.
>
> That’s the simple case and relates to streams. If you’re talking about
> files which can have multiple streams, you may have multiple deletes, one
> for each stream being deleted. If you’re wanting to remove a file context
> then you’ll have to wait for all stream cleanups but again, you can use a
> cleanup callback for a file or a stream context rather than waiting for
> the close. (Or at least for the streams that you know has their
> disposition set.)
>
> Now, one thing you need to be aware of is that if you’re not seeing all
> the creates on streams, you won’t be able to identify the streams that
> were opened delete-on-close. That’s a real pain with DOC because if you
> query the disposition and the stream was opened DOC, the disposition will
> not be set. I worked on a driver that had to know if a stream was marked
> for deletion and the driver had to monitor all opens to watch for DOC.
> Also keep in mind that the disposition could be cleared later before the
> stream is closed.
>
> “Petr Kurtin” wrote in message news:xxxxx@ntfsd…
> Hello,
>
> I would like to remove file contexts when the file is going to be
> deleted, but I don’t know when exactly is the right time. The file can be
> marked as delete pending by calling NtSetInformationFile(FileDisposition)
> or when it’s opened with FILE_DELETE_ON_CLOSE. This flag is stored in the
> file contexts, which are created for the files from the monitored folders
> selected by a user. The files may be already opened when the user will
> decide to monitor such folders, that’s why I may not receive all
> IRP_MJ_CREATE requests. On the contrary, I’ll receive all IRP_MJ_CLEANUP
> requests and I know, the right time to remove file contexts is the last
> IRP_MJ_CLEANUP before IRP_MJ_CLOSE.
>
> Is it possible to recognize the last IRP_MJ_CLEANUP?
>
> Thanks for sober advices,
> Petr
>
>

> Whether it’s soluble or not, I know from a lot of experience that you can

No argumrnt I had same experience :slight_smile:

The bottom line is that DOC is just like setting the disposition except
querying the disposition doesn’t come back as being set if DOC is active.

Yeah, so, its kinda like setting the disposition, but just not quite exactly
the same …

However, it can be overridden by clearing the disposition. This I know for
a

Hmm, this is sending FileDispositionInformation on the FO which has the DOC
before IRP_MJ_CLEANUP on the FO which has the DOC? Curiouser and curiouser

That said, assuming that DOC is a function that just can’t be handled …
as fun as renames.

No argument I had same experience :slight_smile:

“Jerry Kelley” wrote in message news:xxxxx@ntfsd…
> Whether it’s soluble or not, I know from a lot of experience that you can
> only detect DOC on create as an option and you have to respect the create
> that you get. That is, unless you work on the kernel team and know a way
> to get the true disposition when DOC is involved.
>
> The bottom line is that DOC is just like setting the disposition except
> querying the disposition doesn’t come back as being set if DOC is active.
> However, it can be overridden by clearing the disposition. This I know for
> a fact as of a couple of months ago. If something in the architecture has
> changed since then, I stand corrected.
>
> Now, if some filter above is hijacking the create and changing options,
> well then, what can you do? I’m just providing information on how DOC
> works when you see the create in it’s “pure” form. Of course, that’s not
> to say a lower filter could change the create options as well. The create
> should behave as your filter saw it but anyone who’s worked in this area
> for some time knows that, in reality, all bets are off and you do what you
> can.
>
> That said, assuming that DOC is a function that just can’t be handled is
> not going to fly. If you see DOC you have to assume the stream is marked
> for deletion unless you see the disposition get cleared. Of course, it
> could be set again later but that’s just the fun of file deletion. Almost
> as fun as renames.
>
> From my experience, when my filter saw DOC it handled it. If a shadow
> device or some other filter is playing games well you can blame the
> results on them. I’ve encountered quite a few poorly written third-party
> filters that play games and break other filters or, even worse, just bring
> the system down due to some (illegal) shortcut, assumption, or
> inexperience.
>
> jerry
>
>
> “Lyndon J. Clarke” wrote in message
> news:xxxxx@ntfsd…
>> AFAIK the DOC problem is not soluble. It is possible that some filter
>> (legacy) below uses shadow device method (or worse) and the IRP_MJ_CREATE
>> which has DOC is not observable even though the later IRP_MJ_CLEANUP is
>> observable.
>>
>> “Jerry Kelley” wrote in message
>> news:xxxxx@ntfsd…
>> Since contexts existed before Filter Manager I don’t know if you’re
>> referring to a legacy or a mini. So, assuming it’s a mini, the answer is
>> easy - just register a cleanup callback. Note that you may not see the
>> callback for some time after the file is closed under certain
>> circumstances but when it does get called, you can safely release your
>> context resources. Note that you don’t have to delete your context
>> because the internal FM context is being torn down.
>>
>> That’s the simple case and relates to streams. If you’re talking about
>> files which can have multiple streams, you may have multiple deletes, one
>> for each stream being deleted. If you’re wanting to remove a file context
>> then you’ll have to wait for all stream cleanups but again, you can use a
>> cleanup callback for a file or a stream context rather than waiting for
>> the close. (Or at least for the streams that you know has their
>> disposition set.)
>>
>> Now, one thing you need to be aware of is that if you’re not seeing all
>> the creates on streams, you won’t be able to identify the streams that
>> were opened delete-on-close. That’s a real pain with DOC because if you
>> query the disposition and the stream was opened DOC, the disposition will
>> not be set. I worked on a driver that had to know if a stream was marked
>> for deletion and the driver had to monitor all opens to watch for DOC.
>> Also keep in mind that the disposition could be cleared later before the
>> stream is closed.
>>
>> “Petr Kurtin” wrote in message news:xxxxx@ntfsd…
>> Hello,
>>
>> I would like to remove file contexts when the file is going to be
>> deleted, but I don’t know when exactly is the right time. The file can be
>> marked as delete pending by calling NtSetInformationFile(FileDisposition)
>> or when it’s opened with FILE_DELETE_ON_CLOSE. This flag is stored in the
>> file contexts, which are created for the files from the monitored folders
>> selected by a user. The files may be already opened when the user will
>> decide to monitor such folders, that’s why I may not receive all
>> IRP_MJ_CREATE requests. On the contrary, I’ll receive all IRP_MJ_CLEANUP
>> requests and I know, the right time to remove file contexts is the last
>> IRP_MJ_CLEANUP before IRP_MJ_CLOSE.
>>
>> Is it possible to recognize the last IRP_MJ_CLEANUP?
>>
>> Thanks for sober advices,
>> Petr
>>
>>
>
>
>

Yes, the disposition can be cleared for the DOC case just like it was
specifically set for a standard delete. Probably a rare occasion but we have
to be prepared for anything. If we could just query the disposition and get
an accurate value for it in the DOC scenario I think tracking the
pending-delete state would be simplified.

jerry

“Lyndon J. Clarke” wrote in message
news:xxxxx@ntfsd…
>> Whether it’s soluble or not, I know from a lot of experience that you can
>> …
>
> No argumrnt I had same experience :slight_smile:
>
>> The bottom line is that DOC is just like setting the disposition except
>> querying the disposition doesn’t come back as being set if DOC is active.
>
> Yeah, so, its kinda like setting the disposition, but just not quite
> exactly the same …
>
>> However, it can be overridden by clearing the disposition. This I know
>> for a
>
> Hmm, this is sending FileDispositionInformation on the FO which has the
> DOC before IRP_MJ_CLEANUP on the FO which has the DOC? Curiouser and
> curiouser …
>
>> That said, assuming that DOC is a function that just can’t be handled …
>> as fun as renames.
>
> No argument I had same experience :slight_smile:
>
> “Jerry Kelley” wrote in message news:xxxxx@ntfsd…
>> Whether it’s soluble or not, I know from a lot of experience that you can
>> only detect DOC on create as an option and you have to respect the create
>> that you get. That is, unless you work on the kernel team and know a way
>> to get the true disposition when DOC is involved.
>>
>> The bottom line is that DOC is just like setting the disposition except
>> querying the disposition doesn’t come back as being set if DOC is active.
>> However, it can be overridden by clearing the disposition. This I know
>> for a fact as of a couple of months ago. If something in the architecture
>> has changed since then, I stand corrected.
>>
>> Now, if some filter above is hijacking the create and changing options,
>> well then, what can you do? I’m just providing information on how DOC
>> works when you see the create in it’s “pure” form. Of course, that’s not
>> to say a lower filter could change the create options as well. The create
>> should behave as your filter saw it but anyone who’s worked in this area
>> for some time knows that, in reality, all bets are off and you do what
>> you can.
>>
>> That said, assuming that DOC is a function that just can’t be handled is
>> not going to fly. If you see DOC you have to assume the stream is marked
>> for deletion unless you see the disposition get cleared. Of course, it
>> could be set again later but that’s just the fun of file deletion. Almost
>> as fun as renames.
>>
>> From my experience, when my filter saw DOC it handled it. If a shadow
>> device or some other filter is playing games well you can blame the
>> results on them. I’ve encountered quite a few poorly written third-party
>> filters that play games and break other filters or, even worse, just
>> bring the system down due to some (illegal) shortcut, assumption, or
>> inexperience.
>>
>> jerry
>>
>>
>> “Lyndon J. Clarke” wrote in message
>> news:xxxxx@ntfsd…
>>> AFAIK the DOC problem is not soluble. It is possible that some filter
>>> (legacy) below uses shadow device method (or worse) and the
>>> IRP_MJ_CREATE which has DOC is not observable even though the later
>>> IRP_MJ_CLEANUP is observable.
>>>
>>> “Jerry Kelley” wrote in message
>>> news:xxxxx@ntfsd…
>>> Since contexts existed before Filter Manager I don’t know if you’re
>>> referring to a legacy or a mini. So, assuming it’s a mini, the answer is
>>> easy - just register a cleanup callback. Note that you may not see the
>>> callback for some time after the file is closed under certain
>>> circumstances but when it does get called, you can safely release your
>>> context resources. Note that you don’t have to delete your context
>>> because the internal FM context is being torn down.
>>>
>>> That’s the simple case and relates to streams. If you’re talking about
>>> files which can have multiple streams, you may have multiple deletes,
>>> one for each stream being deleted. If you’re wanting to remove a file
>>> context then you’ll have to wait for all stream cleanups but again, you
>>> can use a cleanup callback for a file or a stream context rather than
>>> waiting for the close. (Or at least for the streams that you know has
>>> their disposition set.)
>>>
>>> Now, one thing you need to be aware of is that if you’re not seeing all
>>> the creates on streams, you won’t be able to identify the streams that
>>> were opened delete-on-close. That’s a real pain with DOC because if you
>>> query the disposition and the stream was opened DOC, the disposition
>>> will not be set. I worked on a driver that had to know if a stream was
>>> marked for deletion and the driver had to monitor all opens to watch for
>>> DOC. Also keep in mind that the disposition could be cleared later
>>> before the stream is closed.
>>>
>>> “Petr Kurtin” wrote in message news:xxxxx@ntfsd…
>>> Hello,
>>>
>>> I would like to remove file contexts when the file is going to be
>>> deleted, but I don’t know when exactly is the right time. The file can
>>> be marked as delete pending by calling
>>> NtSetInformationFile(FileDisposition) or when it’s opened with
>>> FILE_DELETE_ON_CLOSE. This flag is stored in the file contexts, which
>>> are created for the files from the monitored folders selected by a user.
>>> The files may be already opened when the user will decide to monitor
>>> such folders, that’s why I may not receive all IRP_MJ_CREATE requests.
>>> On the contrary, I’ll receive all IRP_MJ_CLEANUP requests and I know,
>>> the right time to remove file contexts is the last IRP_MJ_CLEANUP before
>>> IRP_MJ_CLOSE.
>>>
>>> Is it possible to recognize the last IRP_MJ_CLEANUP?
>>>
>>> Thanks for sober advices,
>>> Petr
>>>
>>>
>>
>>
>>
>
>
>

> If we could just query the disposition and get an accurate value for it in

the DOC scenario I think tracking the pending-delete state would be
simplified.

Second that !!

“Jerry Kelley” wrote in message news:xxxxx@ntfsd…
> Yes, the disposition can be cleared for the DOC case just like it was
> specifically set for a standard delete. Probably a rare occasion but we
> have to be prepared for anything. If we could just query the disposition
> and get an accurate value for it in the DOC scenario I think tracking the
> pending-delete state would be simplified.
>
> jerry
>
> “Lyndon J. Clarke” wrote in message
> news:xxxxx@ntfsd…
>>> Whether it’s soluble or not, I know from a lot of experience that you
>>> can …
>>
>> No argumrnt I had same experience :slight_smile:
>>
>>> The bottom line is that DOC is just like setting the disposition except
>>> querying the disposition doesn’t come back as being set if DOC is
>>> active.
>>
>> Yeah, so, its kinda like setting the disposition, but just not quite
>> exactly the same …
>>
>>> However, it can be overridden by clearing the disposition. This I know
>>> for a
>>
>> Hmm, this is sending FileDispositionInformation on the FO which has the
>> DOC before IRP_MJ_CLEANUP on the FO which has the DOC? Curiouser and
>> curiouser …
>>
>>> That said, assuming that DOC is a function that just can’t be handled
>>> … as fun as renames.
>>
>> No argument I had same experience :slight_smile:
>>
>> “Jerry Kelley” wrote in message
>> news:xxxxx@ntfsd…
>>> Whether it’s soluble or not, I know from a lot of experience that you
>>> can only detect DOC on create as an option and you have to respect the
>>> create that you get. That is, unless you work on the kernel team and
>>> know a way to get the true disposition when DOC is involved.
>>>
>>> The bottom line is that DOC is just like setting the disposition except
>>> querying the disposition doesn’t come back as being set if DOC is
>>> active. However, it can be overridden by clearing the disposition. This
>>> I know for a fact as of a couple of months ago. If something in the
>>> architecture has changed since then, I stand corrected.
>>>
>>> Now, if some filter above is hijacking the create and changing options,
>>> well then, what can you do? I’m just providing information on how DOC
>>> works when you see the create in it’s “pure” form. Of course, that’s not
>>> to say a lower filter could change the create options as well. The
>>> create should behave as your filter saw it but anyone who’s worked in
>>> this area for some time knows that, in reality, all bets are off and you
>>> do what you can.
>>>
>>> That said, assuming that DOC is a function that just can’t be handled is
>>> not going to fly. If you see DOC you have to assume the stream is marked
>>> for deletion unless you see the disposition get cleared. Of course, it
>>> could be set again later but that’s just the fun of file deletion.
>>> Almost as fun as renames.
>>>
>>> From my experience, when my filter saw DOC it handled it. If a shadow
>>> device or some other filter is playing games well you can blame the
>>> results on them. I’ve encountered quite a few poorly written third-party
>>> filters that play games and break other filters or, even worse, just
>>> bring the system down due to some (illegal) shortcut, assumption, or
>>> inexperience.
>>>
>>> jerry
>>>
>>>
>>> “Lyndon J. Clarke” wrote in message
>>> news:xxxxx@ntfsd…
>>>> AFAIK the DOC problem is not soluble. It is possible that some filter
>>>> (legacy) below uses shadow device method (or worse) and the
>>>> IRP_MJ_CREATE which has DOC is not observable even though the later
>>>> IRP_MJ_CLEANUP is observable.
>>>>
>>>> “Jerry Kelley” wrote in message
>>>> news:xxxxx@ntfsd…
>>>> Since contexts existed before Filter Manager I don’t know if you’re
>>>> referring to a legacy or a mini. So, assuming it’s a mini, the answer
>>>> is easy - just register a cleanup callback. Note that you may not see
>>>> the callback for some time after the file is closed under certain
>>>> circumstances but when it does get called, you can safely release your
>>>> context resources. Note that you don’t have to delete your context
>>>> because the internal FM context is being torn down.
>>>>
>>>> That’s the simple case and relates to streams. If you’re talking about
>>>> files which can have multiple streams, you may have multiple deletes,
>>>> one for each stream being deleted. If you’re wanting to remove a file
>>>> context then you’ll have to wait for all stream cleanups but again, you
>>>> can use a cleanup callback for a file or a stream context rather than
>>>> waiting for the close. (Or at least for the streams that you know has
>>>> their disposition set.)
>>>>
>>>> Now, one thing you need to be aware of is that if you’re not seeing all
>>>> the creates on streams, you won’t be able to identify the streams that
>>>> were opened delete-on-close. That’s a real pain with DOC because if you
>>>> query the disposition and the stream was opened DOC, the disposition
>>>> will not be set. I worked on a driver that had to know if a stream was
>>>> marked for deletion and the driver had to monitor all opens to watch
>>>> for DOC. Also keep in mind that the disposition could be cleared later
>>>> before the stream is closed.
>>>>
>>>> “Petr Kurtin” wrote in message news:xxxxx@ntfsd…
>>>> Hello,
>>>>
>>>> I would like to remove file contexts when the file is going to be
>>>> deleted, but I don’t know when exactly is the right time. The file can
>>>> be marked as delete pending by calling
>>>> NtSetInformationFile(FileDisposition) or when it’s opened with
>>>> FILE_DELETE_ON_CLOSE. This flag is stored in the file contexts, which
>>>> are created for the files from the monitored folders selected by a
>>>> user. The files may be already opened when the user will decide to
>>>> monitor such folders, that’s why I may not receive all IRP_MJ_CREATE
>>>> requests. On the contrary, I’ll receive all IRP_MJ_CLEANUP requests and
>>>> I know, the right time to remove file contexts is the last
>>>> IRP_MJ_CLEANUP before IRP_MJ_CLOSE.
>>>>
>>>> Is it possible to recognize the last IRP_MJ_CLEANUP?
>>>>
>>>> Thanks for sober advices,
>>>> Petr
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>

Note that due to the peculiarities of implementation, delete-on-close
cannot be cleared for FAT, but it can be for NTFS, although I did log a
bug against this and thus it might have been fixed for Vista.

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Curious … here is a simple test code I cobbled together to look at the
behaviour … it is supposed to be doing …

NtCreateFile Blah -> Handle1
NtCreateFile Blah [DeleteOnClose] -> Handle2
NtQueryInformationFile FileStandardInformaiton Handle1
NtQueryInformationFile FileStandardInformaiton Handle2
NtSetInformationFile FileDispositionInformation [FALSE] Handle2
NtClose Handle2
NtQueryInformationFile FileStandardInformaiton Handle1
NtClose Handle1

The first two NtQueryInformationFile return DeletePending FALSE and the
third returns DeletePending TRUE. The file Blah is delete after the second
NtClose. This is for NTFS and FAT on W2K3 SP1 and XP SP2.

I wonder have I goofed the code or just misunderstood the method to clear
the DeleteOnClose?

By the way, if replace the “NtSetInformationFile Handle2” *before* “NtClose
Handle2” with a NtSetInformatyionFile Handle1 *after* NtCloseHandle Handle2
then all three NtQueryInformationFile return DeletePending FALSE and the
file is not delete after the second NtClose.

int Bladder(int argc, WCHAR * argv)
{
WCHAR FileName[1024];
NTSTATUS NtStatus;
UNICODE_STRING UnicodeString;
OBJECT_ATTRIBUTES ObjectAttributes;
HANDLE FileHandle1, FileHandle2;
IO_STATUS_BLOCK IoStatusBlock;
FILE_DISPOSITION_INFORMATION FileDispositionInformationData;
FILE_STANDARD_INFORMATION FileStandardInformationData;

if (argc != (1 + 1))
{
fwprintf(stderr, L"Usage: %s filename\n", argv[0]);
exit(1);
}

wcscpy(FileName, L"\??\“);
wcscat(FileName, argv[1]);
fwprintf(stdout, L"FileName %s\n”, FileName);

fwprintf(stdout, L"Opening file (1) …\n");

RtlInitUnicodeString(&UnicodeString, FileName);

InitializeObjectAttributes(&ObjectAttributes,
&UnicodeString,
OBJ_CASE_INSENSITIVE | OBJ_KERNEL_HANDLE,
NULL,
NULL);

NtStatus = NtCreateFile(&FileHandle1,
SYNCHRONIZE | DELETE,
&ObjectAttributes,
&IoStatusBlock,
0,
FILE_ATTRIBUTE_NORMAL,
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
FILE_OPEN_IF,
FILE_SYNCHRONOUS_IO_NONALERT,
NULL,
0);

if (!NT_SUCCESS(NtStatus))
{
fwprintf(stderr, L"NtCreateFile returns %X\n", NtStatus);
exit(1);
}

if (! NT_SUCCESS(IoStatusBlock.Status))
{
fwprintf(stderr, L"NtCreateFile IoStatusBlock.Status %X\n",
IoStatusBlock.Status);
exit(1);
}

fwprintf(stdout, L"Opening file (2) with delete on close …\n");

RtlInitUnicodeString(&UnicodeString, FileName);

InitializeObjectAttributes(&ObjectAttributes,
&UnicodeString,
OBJ_CASE_INSENSITIVE | OBJ_KERNEL_HANDLE,
NULL,
NULL);

NtStatus = NtCreateFile(&FileHandle2,
SYNCHRONIZE | DELETE,
&ObjectAttributes,
&IoStatusBlock,
0,
FILE_ATTRIBUTE_NORMAL,
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
FILE_OPEN,
FILE_SYNCHRONOUS_IO_NONALERT | FILE_DELETE_ON_CLOSE,
NULL,
0);

if (!NT_SUCCESS(NtStatus))
{
fwprintf(stderr, L"NtCreateFile returns %X\n", NtStatus);
exit(1);
}

if (! NT_SUCCESS(IoStatusBlock.Status))
{
fwprintf(stderr, L"NtCreateFile IoStatusBlock.Status %X\n",
IoStatusBlock.Status);
exit(1);
}

fwprintf(stdout, L"Querying disposition (1) …\n");

NtStatus = NtQueryInformationFile(FileHandle1,
&IoStatusBlock,
&FileStandardInformationData,
sizeof(FILE_STANDARD_INFORMATION),
FileStandardInformation);

if (!NT_SUCCESS(NtStatus))
{
fwprintf(stderr, L"NtQueryInformationFile returns %X\n", NtStatus);
exit(1);
}

if (! NT_SUCCESS(IoStatusBlock.Status))
{
fwprintf(stderr, L"NtQueryInformationFile IoStatusBlock.Status %X\n",
IoStatusBlock.Status);
exit(1);
}

fwprintf(stdout, L"Disposition (1) is %d\n",
FileStandardInformationData.DeletePending);

fwprintf(stdout, L"Querying disposition (2) …\n");

NtStatus = NtQueryInformationFile(FileHandle2,
&IoStatusBlock,
&FileStandardInformationData,
sizeof(FILE_STANDARD_INFORMATION),
FileStandardInformation);

if (!NT_SUCCESS(NtStatus))
{
fwprintf(stderr, L"NtQueryInformationFile returns %X\n", NtStatus);
exit(1);
}

if (! NT_SUCCESS(IoStatusBlock.Status))
{
fwprintf(stderr, L"NtQueryInformationFile IoStatusBlock.Status %X\n",
IoStatusBlock.Status);
exit(1);
}

fwprintf(stdout, L"Disposition (2) is %d\n",
FileStandardInformationData.DeletePending);

fwprintf(stdout, L"Setting file (2) delete file false …\n");

FileDispositionInformationData.DeleteFile = FALSE;

NtStatus = NtSetInformationFile(FileHandle2,
&IoStatusBlock,
&FileDispositionInformationData,
sizeof(FILE_DISPOSITION_INFORMATION),
FileDispositionInformation);

if (!NT_SUCCESS(NtStatus))
{
fwprintf(stderr, L"NtSetInformationFile returns %X\n", NtStatus);
exit(1);
}

if (! NT_SUCCESS(IoStatusBlock.Status))
{
fwprintf(stderr, L"NtSetInformationFile IoStatusBlock.Status %X\n",
IoStatusBlock.Status);
exit(1);
}

fwprintf(stdout, L"Closing file (2) …\n");

NtStatus = NtClose(FileHandle2);

if (!NT_SUCCESS(NtStatus))
{
fwprintf(stderr, L"NtCloseFile returns %X\n", NtStatus);
exit(1);
}

if (! NT_SUCCESS(IoStatusBlock.Status))
{
fwprintf(stderr, L"NtCloseFile IoStatusBlock.Status %X\n",
IoStatusBlock.Status);
exit(1);
}

fwprintf(stdout, L"Querying disposition (1) …\n");

NtStatus = NtQueryInformationFile(FileHandle1,
&IoStatusBlock,
&FileStandardInformationData,
sizeof(FILE_STANDARD_INFORMATION),
FileStandardInformation);

if (!NT_SUCCESS(NtStatus))
{
fwprintf(stderr, L"NtQueryInformationFile returns %X\n", NtStatus);
exit(1);
}

if (! NT_SUCCESS(IoStatusBlock.Status))
{
fwprintf(stderr, L"NtQueryInformationFile IoStatusBlock.Status %X\n",
IoStatusBlock.Status);
exit(1);
}

fwprintf(stdout, L"Disposition (1) is %d\n",
FileStandardInformationData.DeletePending);

fwprintf(stdout, L"Closing file (1) …\n");

NtStatus = NtClose(FileHandle1);

if (!NT_SUCCESS(NtStatus))
{
fwprintf(stderr, L"NtCloseFile returns %X\n", NtStatus);
exit(1);
}

if (! NT_SUCCESS(IoStatusBlock.Status))
{
fwprintf(stderr, L"NtCloseFile IoStatusBlock.Status %X\n",
IoStatusBlock.Status);
exit(1);
}

return 0;
}

“Tony Mason” wrote in message news:xxxxx@ntfsd…
Note that due to the peculiarities of implementation, delete-on-close
cannot be cleared for FAT, but it can be for NTFS, although I did log a
bug against this and thus it might have been fixed for Vista.

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Lyndon,

You are seeing an artifact of the implementation. Delete pending is
tracked in two places (using NTFS terminology here, switching to FAT
terminology below when walking through the FAT code):

The SCB (which represents the specific link) and the CCB (which
represents the specific open instance.) You clear/set the pending in
the CCB; it gets propagated to the SCB when the file is closed (CLEANUP
time). The query is satisfied from the SCB, NOT the CCB.

The “bug” in FAT was that when you did a “delete on close” it set it in
one place. Here’s the code in create.c (FatOpenExistingFcb):

//
// Mark the DeleteOnClose bit if the operation was successful.
//
if ( DeleteOnClose ) {
SetFlag( Ccb->Flags, CCB_FLAG_DELETE_ON_CLOSE );
}

But when you clear it in fileinfo.c (FatSetDispositionInfo):

//
// The user doesn’t want to delete the file so clear
// the delete on close bit
//

DebugTrace(0, Dbg, “User want to not delete file\n”, 0);

ClearFlag( Fcb->FcbState, FCB_STATE_DELETE_ON_CLOSE );
FileObject->DeletePending = FALSE;

So it clears it in the FCB, not the CCB. In cleanup.c
(FatCommonCleanup):

if (FlagOn(Ccb->Flags, CCB_FLAG_DELETE_ON_CLOSE)) {

ASSERT( NodeType(Fcb) != FAT_NTC_ROOT_DCB );

SetFlag(Fcb->FcbState, FCB_STATE_DELETE_ON_CLOSE);

//

So voila! A file you said “don’t delete” gets deleted anyway. As for
what it returns, if you look back in fileinfo.c (FatQueryStandardInfo)
you can see why the results vary:

Buffer->DeletePending = BooleanFlagOn( Fcb->FcbState,
FCB_STATE_DELETE_ON_CLOSE );

It’s based entirely on the data in the FCB, not the CCB. I believe you
see the same basic behavior in NTFS, with the difference that they will
clear the delete pending bit in the SCB as well as the CCB.

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Hi Tony

Thanks. You’ve confirmed my reading of the fastfat code, which I had started
to think was unreliable, and the difference between the CCB and FCB bits.

The point of my post was, that I seem to see *identical* behaviour, in both
NTFS and FAT, at least in for example W2K3 SP1, which seems to be au
contraire with other posts in this thread. So unless I have goofed in my
test code, or unless I have misunderstood the approach to clear the
DeleteOnClose, then I’d have to conclude that neither does NTFS clear the
CCB bit when the FCB[/SCB] bit is being cleared. Hence I posted the test
prog code.

Cheers
Lyndon

“Tony Mason” wrote in message news:xxxxx@ntfsd…
Lyndon,

You are seeing an artifact of the implementation. Delete pending is
tracked in two places (using NTFS terminology here, switching to FAT
terminology below when walking through the FAT code):

The SCB (which represents the specific link) and the CCB (which
represents the specific open instance.) You clear/set the pending in
the CCB; it gets propagated to the SCB when the file is closed (CLEANUP
time). The query is satisfied from the SCB, NOT the CCB.

The “bug” in FAT was that when you did a “delete on close” it set it in
one place. Here’s the code in create.c (FatOpenExistingFcb):

//
// Mark the DeleteOnClose bit if the operation was successful.
//
if ( DeleteOnClose ) {
SetFlag( Ccb->Flags, CCB_FLAG_DELETE_ON_CLOSE );
}

But when you clear it in fileinfo.c (FatSetDispositionInfo):

//
// The user doesn’t want to delete the file so clear
// the delete on close bit
//

DebugTrace(0, Dbg, “User want to not delete file\n”, 0);

ClearFlag( Fcb->FcbState, FCB_STATE_DELETE_ON_CLOSE );
FileObject->DeletePending = FALSE;

So it clears it in the FCB, not the CCB. In cleanup.c
(FatCommonCleanup):

if (FlagOn(Ccb->Flags, CCB_FLAG_DELETE_ON_CLOSE)) {

ASSERT( NodeType(Fcb) != FAT_NTC_ROOT_DCB );

SetFlag(Fcb->FcbState, FCB_STATE_DELETE_ON_CLOSE);

//

So voila! A file you said “don’t delete” gets deleted anyway. As for
what it returns, if you look back in fileinfo.c (FatQueryStandardInfo)
you can see why the results vary:

Buffer->DeletePending = BooleanFlagOn( Fcb->FcbState,
FCB_STATE_DELETE_ON_CLOSE );

It’s based entirely on the data in the FCB, not the CCB. I believe you
see the same basic behavior in NTFS, with the difference that they will
clear the delete pending bit in the SCB as well as the CCB.

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Petr,

I am going to assume you are talking about “File Contexts” as the new type of context available in filter manager for Vista. They are a context (like stream and stream handle contests) that is unique across all streams of a given file. If this is not the case you can ignore the rest of my comments.

Is there a specific reason why you feel you need to delete the file context when the file is being deleted? File contexts are automatically cleaned up by filter manager as the internal NTFS FCB is cleaned up. In other words the file context will go away when all internal references to all streams of the file go away.

Neal Christiansen
Microsoft NTFS Development Lead
This posting is provided “AS IS” with no warranties, and confers no Rights

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Petr Kurtin
Sent: Sunday, December 31, 2006 9:48 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] File contexts

Hello,
?
? I would like to remove file contexts when the file is going to be deleted, but I don’t know when exactly is the right time. The file can be marked as delete pending by calling NtSetInformationFile(FileDisposition) or when it’s opened with FILE_DELETE_ON_CLOSE. This flag is stored in the file contexts, which are created for the files from the monitored folders selected by a user. The files may be already opened when the user will decide to monitor such folders, that’s why I may not receive all IRP_MJ_CREATE requests. On the contrary, I’ll receive all IRP_MJ_CLEANUP requests and I know, the right time to remove file contexts is the last IRP_MJ_CLEANUP before IRP_MJ_CLOSE.
?
? Is it possible to recognize the last IRP_MJ_CLEANUP?
?
Thanks for sober advices,
Petr
?


Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com