PerStreamContext leaking?

Hi:

I’m using the FileSpy sample from IFS for W2003 and it looks like it’s
leaking PerStreamContexts.

I’m using the FileSpy sample exactly as is, without any modification, except
for the line that DbgPrint the gRecordsAllocated variable that stores the
number of PerStreamContexts allocated.

I found that as the FileSpy filter driver is used (using the windows
explorer, running programs, etc.), the value stored in gRecordsAllocated is
increased until eventually (after several days) it finds the
gMaxRecordsToAllocate value and stops allocating the perStreamContext
information.

I wrote a filter driver using the FileSpy sample as a starting point and, of
course, it has the same problem.

Has anybody an idea on how to fix this situation?

Best regards.

Pablo Frank

To the best of my knowledge FileSpy does not leak PerStreamContexts but
it has been a while since I reviewed the code.

NTFS does not immediately delete its SCB (stream control block, which
you see as the FsContext in the file object) when it receives the last
IRP_MJ_CLOSE for a file. It keeps an LRU list of recently used SCBs
around so if they are quickly reopened, the information is already in
memory.

The PerStream context will not be deleted until the actual SCB is
deleted.

You can force the entries on this list to be freed by doing something
that opens a lot of files; try copying a large tree from one location to
another.

Neal Christiansen
Microsoft File System Filter Group Lead

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Pablo G. Frank
Sent: Friday, February 27, 2004 10:01 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] PerStreamContext leaking?

Hi:

I’m using the FileSpy sample from IFS for W2003 and it looks like it’s
leaking PerStreamContexts.

I’m using the FileSpy sample exactly as is, without any modification,
except
for the line that DbgPrint the gRecordsAllocated variable that stores
the
number of PerStreamContexts allocated.

I found that as the FileSpy filter driver is used (using the windows
explorer, running programs, etc.), the value stored in gRecordsAllocated
is
increased until eventually (after several days) it finds the
gMaxRecordsToAllocate value and stops allocating the perStreamContext
information.

I wrote a filter driver using the FileSpy sample as a starting point
and, of
course, it has the same problem.

Has anybody an idea on how to fix this situation?

Best regards.

Pablo Frank


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Thanks for your answer, Neal.

Well the problem I’m trying to solve is that FileSpy is continuously
increasing the value of gRecordsAllocated, that is the counter for allocated
context records. If the driver is running for enough time (in my case,
around a week), the maximum allowed is reached (500, by default) so no more
records are allocated. Of course, if I set gMaxRecordsToAllocate to 50, the
maximum is reached faster.

I tested the FileSpy sample code without modifications, so it’s not any code
added by me (I just added the DbgPrint for the gRecordsAllocated value).

You load the driver and you can see the gRecordsAllocated increasing and
decreasing as you open Windows Explorer, open folders, run programs, etc…
Without any known reason, the number decrease only to 2. After a while,
always without a known reason and without any known method to repeat it, the
number decrease only to 3, then 4, 5, 10, 20, etc.

Any ideas?

Best regards.

Pablo Frank

“Neal Christiansen” escribió en el mensaje
news:xxxxx@ntfsd…
To the best of my knowledge FileSpy does not leak PerStreamContexts but
it has been a while since I reviewed the code.

NTFS does not immediately delete its SCB (stream control block, which
you see as the FsContext in the file object) when it receives the last
IRP_MJ_CLOSE for a file. It keeps an LRU list of recently used SCBs
around so if they are quickly reopened, the information is already in
memory.

The PerStream context will not be deleted until the actual SCB is
deleted.

You can force the entries on this list to be freed by doing something
that opens a lot of files; try copying a large tree from one location to
another.

Neal Christiansen
Microsoft File System Filter Group Lead

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Pablo G. Frank
Sent: Friday, February 27, 2004 10:01 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] PerStreamContext leaking?

Hi:

I’m using the FileSpy sample from IFS for W2003 and it looks like it’s
leaking PerStreamContexts.

I’m using the FileSpy sample exactly as is, without any modification,
except
for the line that DbgPrint the gRecordsAllocated variable that stores
the
number of PerStreamContexts allocated.

I found that as the FileSpy filter driver is used (using the windows
explorer, running programs, etc.), the value stored in gRecordsAllocated
is
increased until eventually (after several days) it finds the
gMaxRecordsToAllocate value and stops allocating the perStreamContext
information.

I wrote a filter driver using the FileSpy sample as a starting point
and, of
course, it has the same problem.

Has anybody an idea on how to fix this situation?

Best regards.

Pablo Frank


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Interesting, I will take a look and see if I can find anything.

Neal Christiansen
Microsoft File System Filter Group Lead

This posting is provided “AS IS” with no warranties, and confers no rights.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Pablo G. Frank
Sent: Monday, March 01, 2004 7:12 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] PerStreamContext leaking?

Thanks for your answer, Neal.

Well the problem I’m trying to solve is that FileSpy is continuously
increasing the value of gRecordsAllocated, that is the counter for allocated
context records. If the driver is running for enough time (in my case,
around a week), the maximum allowed is reached (500, by default) so no more
records are allocated. Of course, if I set gMaxRecordsToAllocate to 50, the
maximum is reached faster.

I tested the FileSpy sample code without modifications, so it’s not any code
added by me (I just added the DbgPrint for the gRecordsAllocated value).

You load the driver and you can see the gRecordsAllocated increasing and
decreasing as you open Windows Explorer, open folders, run programs, etc…
Without any known reason, the number decrease only to 2. After a while,
always without a known reason and without any known method to repeat it, the
number decrease only to 3, then 4, 5, 10, 20, etc.

Any ideas?

Best regards.

Pablo Frank

“Neal Christiansen” escribi? en el mensaje
news:xxxxx@ntfsd…
To the best of my knowledge FileSpy does not leak PerStreamContexts but
it has been a while since I reviewed the code.

NTFS does not immediately delete its SCB (stream control block, which
you see as the FsContext in the file object) when it receives the last
IRP_MJ_CLOSE for a file. It keeps an LRU list of recently used SCBs
around so if they are quickly reopened, the information is already in
memory.

The PerStream context will not be deleted until the actual SCB is
deleted.

You can force the entries on this list to be freed by doing something
that opens a lot of files; try copying a large tree from one location to
another.

Neal Christiansen
Microsoft File System Filter Group Lead

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Pablo G. Frank
Sent: Friday, February 27, 2004 10:01 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] PerStreamContext leaking?

Hi:

I’m using the FileSpy sample from IFS for W2003 and it looks like it’s
leaking PerStreamContexts.

I’m using the FileSpy sample exactly as is, without any modification,
except
for the line that DbgPrint the gRecordsAllocated variable that stores
the
number of PerStreamContexts allocated.

I found that as the FileSpy filter driver is used (using the windows
explorer, running programs, etc.), the value stored in gRecordsAllocated
is
increased until eventually (after several days) it finds the
gMaxRecordsToAllocate value and stops allocating the perStreamContext
information.

I wrote a filter driver using the FileSpy sample as a starting point
and, of
course, it has the same problem.

Has anybody an idea on how to fix this situation?

Best regards.

Pablo Frank


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Well I have a problem with this situation. I made a driver for a client and
I have this problem with stream contexts. But if I’m right the problem is in
the IFS samples or somewhere else but not in the code I wrote. So I don’t
know how to solve this problem.

Where can I ask for help on this issue?. Any ideas?

Best regards.

Pablo Frank

“Neal Christiansen” escribió en el mensaje
news:xxxxx@ntfsd…
To the best of my knowledge FileSpy does not leak PerStreamContexts but
it has been a while since I reviewed the code.

NTFS does not immediately delete its SCB (stream control block, which
you see as the FsContext in the file object) when it receives the last
IRP_MJ_CLOSE for a file. It keeps an LRU list of recently used SCBs
around so if they are quickly reopened, the information is already in
memory.

The PerStream context will not be deleted until the actual SCB is
deleted.

You can force the entries on this list to be freed by doing something
that opens a lot of files; try copying a large tree from one location to
another.

Neal Christiansen
Microsoft File System Filter Group Lead

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Pablo G. Frank
Sent: Friday, February 27, 2004 10:01 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] PerStreamContext leaking?

Hi:

I’m using the FileSpy sample from IFS for W2003 and it looks like it’s
leaking PerStreamContexts.

I’m using the FileSpy sample exactly as is, without any modification,
except
for the line that DbgPrint the gRecordsAllocated variable that stores
the
number of PerStreamContexts allocated.

I found that as the FileSpy filter driver is used (using the windows
explorer, running programs, etc.), the value stored in gRecordsAllocated
is
increased until eventually (after several days) it finds the
gMaxRecordsToAllocate value and stops allocating the perStreamContext
information.

I wrote a filter driver using the FileSpy sample as a starting point
and, of
course, it has the same problem.

Has anybody an idea on how to fix this situation?

Best regards.

Pablo Frank


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

You are talking with the right people who can help you with this. I am the one that wrote this code.

I have never run filespy for an extended period of time which is probably why we have never noticed this before. It sounds like there is a rare case where we might not be cleaning up properly.

I will try and take a look at this but I do have higher priority tasks right now.

Neal Christiansen

Microsoft File System Filter Group Lead
This posting is provided “AS IS” with no warranties, and confers no rights.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Pablo G. Frank
Sent: Friday, March 05, 2004 7:16 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] PerStreamContext leaking?

Well I have a problem with this situation. I made a driver for a client and
I have this problem with stream contexts. But if I’m right the problem is in
the IFS samples or somewhere else but not in the code I wrote. So I don’t
know how to solve this problem.

Where can I ask for help on this issue?. Any ideas?

Best regards.

Pablo Frank

“Neal Christiansen” escribi? en el mensaje
news:xxxxx@ntfsd…
To the best of my knowledge FileSpy does not leak PerStreamContexts but
it has been a while since I reviewed the code.

NTFS does not immediately delete its SCB (stream control block, which
you see as the FsContext in the file object) when it receives the last
IRP_MJ_CLOSE for a file. It keeps an LRU list of recently used SCBs
around so if they are quickly reopened, the information is already in
memory.

The PerStream context will not be deleted until the actual SCB is
deleted.

You can force the entries on this list to be freed by doing something
that opens a lot of files; try copying a large tree from one location to
another.

Neal Christiansen
Microsoft File System Filter Group Lead

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Pablo G. Frank
Sent: Friday, February 27, 2004 10:01 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] PerStreamContext leaking?

Hi:

I’m using the FileSpy sample from IFS for W2003 and it looks like it’s
leaking PerStreamContexts.

I’m using the FileSpy sample exactly as is, without any modification,
except
for the line that DbgPrint the gRecordsAllocated variable that stores
the
number of PerStreamContexts allocated.

I found that as the FileSpy filter driver is used (using the windows
explorer, running programs, etc.), the value stored in gRecordsAllocated
is
increased until eventually (after several days) it finds the
gMaxRecordsToAllocate value and stops allocating the perStreamContext
information.

I wrote a filter driver using the FileSpy sample as a starting point
and, of
course, it has the same problem.

Has anybody an idea on how to fix this situation?

Best regards.

Pablo Frank


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com