not receiving IRP_MJ_CLOSE on filter driver

Hi!

In my filter driver (based on Win2003 IFS SFilter), I need to control
all instances of files in a specific directory, and modify their
contents. To control the files, I’m using a list indexed by FsContext,
with a sublist with all FILE_OBJECTs for this FsContext. Here is the
list algorithm:

  • In IRP_MJ_CREATE, I check if the FsContext is in my list. If it
    isn’t, I put it in the list, change the ref count to 1 and link the
    FILE_OBJECT with the FsContext. If it is in the list, I just link the
    FILE_OBJECT with the FsContext and do a RefCount++.
  • In IRP_MJ_CLOSE, I do a RefCount-- and delete the FILE_OBJECT from
    the sublist associated with this FsContext. If the RefCount is 0 and
    there’s no more FILE_OBJECT linked with the FsContext, I delete this
    entry from my list.
  • In IRP_MJ_WRITE, IRP_MJ_READ, QUERY_INFORMATION, SET_INFORMATION, I
    check if the current FsContext is in my list. If it is, but the
    FILE_OBJECT isn’t in the sublist, I’ll put it and RefCount++.

I have some questions and some problems:

  • Do I really need the RefCount? Can I just check the FILE_OBJECT list?
  • Sometimes, when the some app is doing a lot of I/O, It looks like the
    system is reusing the memory pointed by some FsContext in my list with
    another file. To overcome this, I’m checking the file name (on
    IRP_MJ_CREATE) to see if the list is consistent. If the file name
    doesn’t match, I remove the original entry and insert the new one. But I
    think the question is: Why I’m not receiving the final close for this files?
  • Sometimes, while reading or write, I check that the current file name
    (extracted from the Irp’s FILE_OBJECT) doesn’t match with the one in my
    list (and I’m getting the files corrupted). I think this problem is
    caused by the missed close. And I don’t think checking the name on every
    read/write is a good approach. This is happening even if I check the
    file name in the IRJ_MJ_CREATE.

I did read the OSR articles about tracking files, but they don’t say
nothing about missing closes (Do I need to read again? :-)).

Thanks,

Strauss

I think you have the right idea here, but there are a couple of things you
may need consider that could be causing what you think of as a “lost” close.
In your description, I am assuming you do all this processing in completion
routines. If so, here is what to look out for:

  • It is possible to see the CLOSE IRP for a FO before your CREATE completion
    runs. This is true if a lower filter (or FSD) forces an otherwise successful
    CREATE closed before your completion routine gets invoked and the CLOSE
    comes down the entire device stack.

  • The FsContext pointer is not valid in CLOSE completion (and is most likely
    NULL) since the FSD could have “freed” it during CLOSE processing. And yes,
    I have seen it re-used before your CLOSE processing completes! I find it
    easiest to remove the FsContext reference pre-CLOSE and free the FO
    reference post CLOSE. This may mean you’ll have to pass the FO tracking data
    to your post-CLOSE completion routine since you can not use the FsContext
    any longer.

  • Filename tracking should only have to be done during create & rename
    processing. You cannot count on the FileObject having a valid buffer at any
    time other than pre-CREATE and it is possible to determine it in both
    post-create and post-rename processing by querying the file system for the
    object name.

Hope this helps, /ted

-----Original Message-----
From: Rodrigo Strauss [mailto:xxxxx@scua.com.br]
Sent: Wednesday, November 19, 2003 7:23 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] not receiving IRP_MJ_CLOSE on filter driver

Hi!

In my filter driver (based on Win2003 IFS SFilter), I need to
control
all instances of files in a specific directory, and modify their
contents. To control the files, I’m using a list indexed by FsContext,
with a sublist with all FILE_OBJECTs for this FsContext. Here is the
list algorithm:

  • In IRP_MJ_CREATE, I check if the FsContext is in my list. If it
    isn’t, I put it in the list, change the ref count to 1 and link the
    FILE_OBJECT with the FsContext. If it is in the list, I just link the
    FILE_OBJECT with the FsContext and do a RefCount++.
  • In IRP_MJ_CLOSE, I do a RefCount-- and delete the FILE_OBJECT from

the sublist associated with this FsContext. If the RefCount is 0 and
there’s no more FILE_OBJECT linked with the FsContext, I delete this
entry from my list.

  • In IRP_MJ_WRITE, IRP_MJ_READ, QUERY_INFORMATION, SET_INFORMATION,
    I
    check if the current FsContext is in my list. If it is, but the
    FILE_OBJECT isn’t in the sublist, I’ll put it and RefCount++.

I have some questions and some problems:

  • Do I really need the RefCount? Can I just check the FILE_OBJECT
    list?
  • Sometimes, when the some app is doing a lot of I/O, It looks like
    the
    system is reusing the memory pointed by some FsContext in my list with
    another file. To overcome this, I’m checking the file name (on
    IRP_MJ_CREATE) to see if the list is consistent. If the file name
    doesn’t match, I remove the original entry and insert the new one. But I
    think the question is: Why I’m not receiving the final close for this files?
  • Sometimes, while reading or write, I check that the current file
    name
    (extracted from the Irp’s FILE_OBJECT) doesn’t match with the one in my
    list (and I’m getting the files corrupted). I think this problem is
    caused by the missed close. And I don’t think checking the name on every
    read/write is a good approach. This is happening even if I check the
    file name in the IRJ_MJ_CREATE.

I did read the OSR articles about tracking files, but they don’t say

nothing about missing closes (Do I need to read again? :-)).

Thanks,

Strauss


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To unsubscribe
send a blank email to xxxxx@lists.osr.com

Hi Ted:

I think you have the right idea here, but there are a couple of things you
may need consider that could be causing what you think of as a “lost” close.
In your description, I am assuming you do all this processing in completion
routines. If so, here is what to look out for:

In IRP_MJ_CREATE, I’m using the same approach SFilter uses. I pass a
KEVENT as context to the Completion, and wait it. As I need to do some
file name check and UNICODE/ANSI conversion, I’ve got some
IRQL_NOT_LESS_OR_EQUAL (and ASSERTs in Win2k dbg build) dealing with
these string manipulation in completion. It’s working well with the
event approach.
In IRP_MJ_CLOSE I’m not using completion, since FSD can’t return a
error to a close operation (I can assume this, can’t I?). And I check
the FsContext BEFORE the IoCallDriver.

  • It is possible to see the CLOSE IRP for a FO before your CREATE completion
    runs. This is true if a lower filter (or FSD) forces an otherwise successful
    CREATE closed before your completion routine gets invoked and the CLOSE
    comes down the entire device stack.

I didn’t make tests with another filter yet. But I’ll do.

  • The FsContext pointer is not valid in CLOSE completion (and is most likely
    NULL) since the FSD could have “freed” it during CLOSE processing. And yes,
    I have seen it re-used before your CLOSE processing completes! I find it
    easiest to remove the FsContext reference pre-CLOSE and free the FO
    reference post CLOSE. This may mean you’ll have to pass the FO tracking data
    to your post-CLOSE completion routine since you can not use the FsContext
    any longer.

Do you mean keep this FILE_OBJECT “tracked” for I/O operations until
the close completion?

  • Filename tracking should only have to be done during create & rename
    processing. You cannot count on the FileObject having a valid buffer at any
    time other than pre-CREATE and it is possible to determine it in both
    post-create and post-rename processing by querying the file system for the
    object name.

The FileName can be NULL, but it can’t be different, right?

Hope this helps, /ted

Thanks a lot!

Strauss

-----Original Message-----
From: Rodrigo Strauss [mailto:xxxxx@scua.com.br]
Sent: Wednesday, November 19, 2003 7:23 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] not receiving IRP_MJ_CLOSE on filter driver

Hi!

In my filter driver (based on Win2003 IFS SFilter), I need to
control
all instances of files in a specific directory, and modify their
contents. To control the files, I’m using a list indexed by FsContext,
with a sublist with all FILE_OBJECTs for this FsContext. Here is the
list algorithm:

  • In IRP_MJ_CREATE, I check if the FsContext is in my list. If it
    isn’t, I put it in the list, change the ref count to 1 and link the
    FILE_OBJECT with the FsContext. If it is in the list, I just link the
    FILE_OBJECT with the FsContext and do a RefCount++.
  • In IRP_MJ_CLOSE, I do a RefCount-- and delete the FILE_OBJECT from

the sublist associated with this FsContext. If the RefCount is 0 and
there’s no more FILE_OBJECT linked with the FsContext, I delete this
entry from my list.

  • In IRP_MJ_WRITE, IRP_MJ_READ, QUERY_INFORMATION, SET_INFORMATION,
    I
    check if the current FsContext is in my list. If it is, but the
    FILE_OBJECT isn’t in the sublist, I’ll put it and RefCount++.

I have some questions and some problems:

  • Do I really need the RefCount? Can I just check the FILE_OBJECT
    list?
  • Sometimes, when the some app is doing a lot of I/O, It looks like
    the
    system is reusing the memory pointed by some FsContext in my list with
    another file. To overcome this, I’m checking the file name (on
    IRP_MJ_CREATE) to see if the list is consistent. If the file name
    doesn’t match, I remove the original entry and insert the new one. But I
    think the question is: Why I’m not receiving the final close for this files?
  • Sometimes, while reading or write, I check that the current file
    name
    (extracted from the Irp’s FILE_OBJECT) doesn’t match with the one in my
    list (and I’m getting the files corrupted). I think this problem is
    caused by the missed close. And I don’t think checking the name on every
    read/write is a good approach. This is happening even if I check the
    file name in the IRJ_MJ_CREATE.

I did read the OSR articles about tracking files, but they don’t say

nothing about missing closes (Do I need to read again? :-)).