When using the per stream context support in XP and Windows 2003, is it
possible that the callback to release the per stream context can be called
after a IRP_MJ_CLOSE has been sent down to the File System but before it has
synchronized back to the mainline thread in the filter above?
Thanks,
Ken
Yes, this is possible because the file system has no notion of when the
CLOSE operation finishes progressing through the entire file system and
filter stack. If the file system determines that it is time to free its
FsContext during its CLOSE processing, then as part of the cleanup of
the FsContext, the file system will call FsRtlTeardownPerStreamContexts
(which calls the FreeCallback for each context attached to this stream).
Note that some file systems lazily teardown the FsContext for a stream
in hopes that this same stream will be reopened again soon and then a
good bit of the initialization work can be avoided. This can make it
*look* like the file system will always wait until the entire stack has
completed CLOSE operation processing before tearing down the per-stream
contexts, but there is no guarantee of that behavior.
Thanks,
Molly Brown
Microsoft Corporation
This posting is provided “AS IS” with no warranties and confers no
rights.
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Thursday, September 16, 2004 2:07 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Per Stream Context callback during Close
When using the per stream context support in XP and Windows 2003, is it
possible that the callback to release the per stream context can be
called after a IRP_MJ_CLOSE has been sent down to the File System but
before it has synchronized back to the mainline thread in the filter
above?
Thanks,
Ken
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
After look again at my code I don’t reference anything after the call to the
driver below.
Can it be freed prior to what is presumably the last close (section object
pointers are NULL). I got the context during the start of the close but
before I call down the context has been freed.
Thanks,
Ken
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Molly Brown
Sent: Thursday, September 16, 2004 12:08 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Yes, this is possible because the file system has no notion of when the
CLOSE operation finishes progressing through the entire file system and
filter stack. If the file system determines that it is time to free its
FsContext during its CLOSE processing, then as part of the cleanup of
the FsContext, the file system will call FsRtlTeardownPerStreamContexts
(which calls the FreeCallback for each context attached to this stream).
Note that some file systems lazily teardown the FsContext for a stream
in hopes that this same stream will be reopened again soon and then a
good bit of the initialization work can be avoided. This can make it
*look* like the file system will always wait until the entire stack has
completed CLOSE operation processing before tearing down the per-stream
contexts, but there is no guarantee of that behavior.
Thanks,
Molly Brown
Microsoft Corporation
This posting is provided “AS IS” with no warranties and confers no
rights.
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Thursday, September 16, 2004 2:07 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Per Stream Context callback during Close
When using the per stream context support in XP and Windows 2003, is it
possible that the callback to release the per stream context can be
called after a IRP_MJ_CLOSE has been sent down to the File System but
before it has synchronized back to the mainline thread in the filter
above?
Thanks,
Ken
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
Do you ever call FsRtlRemovePerStreamContext()? If so you probably have a race between your filter doing an explicit delete and the system deleting it. We saw this issue when writing system restore. You need to do the appropriate synchronization to make this work properly.
Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided "AS IS" with no warranties, and confers no rights
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Thursday, September 16, 2004 10:51 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
After look again at my code I don't reference anything after the call to the driver below.
Can it be freed prior to what is presumably the last close (section object pointers are NULL). I got the context during the start of the close but before I call down the context has been freed.
Thanks,
Ken
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Molly Brown
Sent: Thursday, September 16, 2004 12:08 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Yes, this is possible because the file system has no notion of when the
CLOSE operation finishes progressing through the entire file system and
filter stack.? If the file system determines that it is time to free its
FsContext during its CLOSE processing, then as part of the cleanup of
the FsContext, the file system will call FsRtlTeardownPerStreamContexts
(which calls the FreeCallback for each context attached to this stream).
Note that some file systems lazily teardown the FsContext for a stream
in hopes that this same stream will be reopened again soon and then a
good bit of the initialization work can be avoided.? This can make it
*look* like the file system will always wait until the entire stack has
completed CLOSE operation processing before tearing down the per-stream
contexts, but there is no guarantee of that behavior.
Thanks,
Molly Brown
Microsoft Corporation
?
This posting is provided "AS IS" with no warranties and confers no
rights.
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Thursday, September 16, 2004 2:07 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Per Stream Context callback during Close
Questions? First check the IFS FAQ at
You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
''
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at The NT Insider:Windows NT Virtual Memory (Part I)
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ''
To unsubscribe send a blank email to xxxxx@lists.osr.com
I don't call FsRtlRemovePerStreamContext in this case. It seems like the
call back to delete the PSC is happening in the pre create of MJ IRP. Its
there at the beginning of the call but not just prior to the I/O call driver
to the driver below.
Ken
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Neal Christiansen
Sent: Friday, September 17, 2004 1:50 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Do you ever call FsRtlRemovePerStreamContext()? If so you probably have a
race between your filter doing an explicit delete and the system deleting
it. We saw this issue when writing system restore. You need to do the
appropriate synchronization to make this work properly.
Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided "AS IS" with no warranties, and confers no rights
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Thursday, September 16, 2004 10:51 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
After look again at my code I don't reference anything after the call to the
driver below.
Can it be freed prior to what is presumably the last close (section object
pointers are NULL). I got the context during the start of the close but
before I call down the context has been freed.
Thanks,
Ken
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Molly Brown
Sent: Thursday, September 16, 2004 12:08 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Yes, this is possible because the file system has no notion of when the
CLOSE operation finishes progressing through the entire file system and
filter stack.? If the file system determines that it is time to free its
FsContext during its CLOSE processing, then as part of the cleanup of
the FsContext, the file system will call FsRtlTeardownPerStreamContexts
(which calls the FreeCallback for each context attached to this stream).
Note that some file systems lazily teardown the FsContext for a stream
in hopes that this same stream will be reopened again soon and then a
good bit of the initialization work can be avoided.? This can make it
*look* like the file system will always wait until the entire stack has
completed CLOSE operation processing before tearing down the per-stream
contexts, but there is no guarantee of that behavior.
Thanks,
Molly Brown
Microsoft Corporation
?
This posting is provided "AS IS" with no warranties and confers no
rights.
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Thursday, September 16, 2004 2:07 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Per Stream Context callback during Close
Questions? First check the IFS FAQ at
You are currently subscribed to ntfsd as: unknown lmsubst tag argument:
''
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ''
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ''
To unsubscribe send a blank email to xxxxx@lists.osr.com
Ken,
You mentioned below that you were getting a context cleanup callback while processing a create IRP? I believe that is possible because internally NTFS keeps a list of recently used SCBs around in case the files are quickly reopened. I believe it is possible when a new file is opened for it to remove an entry of this list.
My question is why does it matter? The system will call the context cleanup when the file system deletes its internal SCB/FCB. You should simply process it when it occurs.
At this point I am not sure the exact problem you are having. Can you please describe it again?
Note that we use PerStreamContexts extensively internally and have not seen issues with how they work.
Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided "AS IS" with no warranties, and confers no rights
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Saturday, September 18, 2004 9:05 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
I don't call FsRtlRemovePerStreamContext in this case. It seems like the call back to delete the PSC is happening in the pre create of MJ IRP. Its there at the beginning of the call but not just prior to the I/O call driver to the driver below.
Ken
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Neal Christiansen
Sent: Friday, September 17, 2004 1:50 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Do you ever call FsRtlRemovePerStreamContext()?? If so you probably have a race between your filter doing an explicit delete and the system deleting it.? We saw this issue when writing system restore.? You need to do the appropriate synchronization to make this work properly.
Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided "AS IS" with no warranties, and confers no rights
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Thursday, September 16, 2004 10:51 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
After look again at my code I don't reference anything after the call to the driver below.
Can it be freed prior to what is presumably the last close (section object pointers are NULL). I got the context during the start of the close but before I call down the context has been freed.
Thanks,
Ken
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Molly Brown
Sent: Thursday, September 16, 2004 12:08 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Yes, this is possible because the file system has no notion of when the
CLOSE operation finishes progressing through the entire file system and
filter stack.? If the file system determines that it is time to free its
FsContext during its CLOSE processing, then as part of the cleanup of
the FsContext, the file system will call FsRtlTeardownPerStreamContexts
(which calls the FreeCallback for each context attached to this stream).
Note that some file systems lazily teardown the FsContext for a stream
in hopes that this same stream will be reopened again soon and then a
good bit of the initialization work can be avoided.? This can make it
*look* like the file system will always wait until the entire stack has
completed CLOSE operation processing before tearing down the per-stream
contexts, but there is no guarantee of that behavior.
Thanks,
Molly Brown
Microsoft Corporation
?
This posting is provided "AS IS" with no warranties and confers no
rights.
Questions? First check the IFS FAQ at The NT Insider:Windows NT Virtual Memory (Part I)
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ''
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at The NT Insider:Windows NT Virtual Memory (Part I)
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ''
To unsubscribe send a blank email to xxxxx@lists.osr.com
Hi Neal,
If I said CREATE I meant pre CLOSE. I also have seen this happen during and
OPLOCK FS control.
Why does it matter? Because I am getting a call back to delete my context
after I have gotten it from the GetPerStreamCOntext call but before I pass
the IRP to the driver below. In other words it seems like the FSContext and
my Context are being freed while I am still using them. I can understand it
happening after I send the CLOSE IRP to the driver below but before.
In the case of the oplock, it was after the oplock call had completed.
Thanks,
Ken
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Neal Christiansen
Sent: Saturday, September 25, 2004 1:25 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Ken,
You mentioned below that you were getting a context cleanup callback while
processing a create IRP? I believe that is possible because internally NTFS
keeps a list of recently used SCBs around in case the files are quickly
reopened. I believe it is possible when a new file is opened for it to
remove an entry of this list.
My question is why does it matter? The system will call the context cleanup
when the file system deletes its internal SCB/FCB. You should simply
process it when it occurs.
At this point I am not sure the exact problem you are having. Can you
please describe it again?
Note that we use PerStreamContexts extensively internally and have not seen
issues with how they work.
Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided "AS IS" with no warranties, and confers no rights
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Saturday, September 18, 2004 9:05 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
I don't call FsRtlRemovePerStreamContext in this case. It seems like the
call back to delete the PSC is happening in the pre create of MJ IRP. Its
there at the beginning of the call but not just prior to the I/O call driver
to the driver below.
Ken
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Neal Christiansen
Sent: Friday, September 17, 2004 1:50 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Do you ever call FsRtlRemovePerStreamContext()?? If so you probably have a
race between your filter doing an explicit delete and the system deleting
it.? We saw this issue when writing system restore.? You need to do the
appropriate synchronization to make this work properly.
Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided "AS IS" with no warranties, and confers no rights
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Thursday, September 16, 2004 10:51 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
After look again at my code I don't reference anything after the call to the
driver below.
Can it be freed prior to what is presumably the last close (section object
pointers are NULL). I got the context during the start of the close but
before I call down the context has been freed.
Thanks,
Ken
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Molly Brown
Sent: Thursday, September 16, 2004 12:08 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Yes, this is possible because the file system has no notion of when the
CLOSE operation finishes progressing through the entire file system and
filter stack.? If the file system determines that it is time to free its
FsContext during its CLOSE processing, then as part of the cleanup of
the FsContext, the file system will call FsRtlTeardownPerStreamContexts
(which calls the FreeCallback for each context attached to this stream).
Note that some file systems lazily teardown the FsContext for a stream
in hopes that this same stream will be reopened again soon and then a
good bit of the initialization work can be avoided.? This can make it
*look* like the file system will always wait until the entire stack has
completed CLOSE operation processing before tearing down the per-stream
contexts, but there is no guarantee of that behavior.
Thanks,
Molly Brown
Microsoft Corporation
?
This posting is provided "AS IS" with no warranties and confers no
rights.
Questions? First check the IFS FAQ at
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ''
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ''
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ''
To unsubscribe send a blank email to xxxxx@lists.osr.com
Ken,
I don't know what to tell you. We use per-stream contexts extensively and we have no problems with them. To be honest I have never looked at all the situations where the file system releases them.
The simple answer is this. All of the MS file systems issue the context cleanup callback just before they delete the SCB. If you get the callback it is because the SCB is going away immediately. There is no exception to this rule. In our usage we use a proper reference counting model so it doesn't matter if we happen to be using it right when they decide to get rid of it. I would recommend you do the same type thing.
Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided "AS IS" with no warranties, and confers no rights
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Tuesday, September 28, 2004 8:25 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Hi Neal,
If I said CREATE I meant pre CLOSE. I also have seen this happen during and OPLOCK FS control.
Why does it matter? Because I am getting a call back to delete my context after I have gotten it from the GetPerStreamCOntext call but before I pass the IRP to the driver below. In other words it seems like the FSContext and my Context are being freed while I am still using them. I can understand it happening after I send the CLOSE IRP to the driver below but before.
In the case of the oplock, it was after the oplock call had completed.
Thanks,
Ken
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Neal Christiansen
Sent: Saturday, September 25, 2004 1:25 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Ken,
You mentioned below that you were getting a context cleanup callback while processing a create IRP?? I believe that is possible because internally NTFS keeps a list of recently used SCBs around in case the files are quickly reopened.? I believe it is possible when a new file is opened for it to remove an entry of this list.?
My question is why does it matter?? The system will call the context cleanup when the file system deletes its internal SCB/FCB.? You should simply process it when it occurs.
At this point I am not sure the exact problem you are having.? Can you please describe it again?
Note that we use PerStreamContexts extensively internally and have not seen issues with how they work.
Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided "AS IS" with no warranties, and confers no rights
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Saturday, September 18, 2004 9:05 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
I don't call FsRtlRemovePerStreamContext in this case. It seems like the call back to delete the PSC is happening in the pre create of MJ IRP. Its there at the beginning of the call but not just prior to the I/O call driver to the driver below.
Ken
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Neal Christiansen
Sent: Friday, September 17, 2004 1:50 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Do you ever call FsRtlRemovePerStreamContext()?? If so you probably have a race between your filter doing an explicit delete and the system deleting it.? We saw this issue when writing system restore.? You need to do the appropriate synchronization to make this work properly.
Neal Christiansen
Microsoft File System Filter Group Lead
This posting is provided "AS IS" with no warranties, and confers no rights
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ken Galipeau
Sent: Thursday, September 16, 2004 10:51 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
After look again at my code I don't reference anything after the call to the driver below.
Can it be freed prior to what is presumably the last close (section object pointers are NULL). I got the context during the start of the close but before I call down the context has been freed.
Thanks,
Ken
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Molly Brown
Sent: Thursday, September 16, 2004 12:08 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Per Stream Context callback during Close
Yes, this is possible because the file system has no notion of when the
CLOSE operation finishes progressing through the entire file system and
filter stack.? If the file system determines that it is time to free its
FsContext during its CLOSE processing, then as part of the cleanup of
the FsContext, the file system will call FsRtlTeardownPerStreamContexts
(which calls the FreeCallback for each context attached to this stream).
Note that some file systems lazily teardown the FsContext for a stream
in hopes that this same stream will be reopened again soon and then a
good bit of the initialization work can be avoided.? This can make it
*look* like the file system will always wait until the entire stack has
completed CLOSE operation processing before tearing down the per-stream
contexts, but there is no guarantee of that behavior.
Thanks,
Molly Brown
Microsoft Corporation
?
This posting is provided "AS IS" with no warranties and confers no
rights.
Questions? First check the IFS FAQ at The NT Insider:Windows NT Virtual Memory (Part I)
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ''
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at The NT Insider:Windows NT Virtual Memory (Part I)
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ''
To unsubscribe send a blank email to xxxxx@lists.osr.com