A customer of ours is slowly running out of memory. Poolmon shows the tag
that we use for stream handle contexts keeps increasing. We have seen the
same thing in one of our test systems. The contexts include data that we
have allocated and release in our context cleanup code. Our test system had
verifier running on our file system minifilter. We were able to stop the
driver without a verifier memory leak crash. That leads into the question of
when does the fltmgr release the contexts?
Bill Wandel
The lifetime of a stream context is the lifetime of the file object.
There are two sources of reference on a file object: handles (actual “open instances” of the file object) and references (someone in kernel mode keeping a pointer around.) It sounds like the “leak” is just due to high utilization since you can unload the drivers (and filter manager won’t allow you to unload with dangling contexts outstanding.)
If you want to see outstanding handle references, you can use “!handle” in the debugger - it will display the handles for a given process.
As for OS references, well, pretty much anything can keep those around (like the memory manager does for memory mapped files, for example. Or the cache manager does for cache maps.) Unless you enable object tracking (via the global flags) there’s no easy way to find all those objects (and tracking them will slow the system down quite a bit.) You might find it easier to simply add your context structures to a big global list in your driver and then walk through that list to see where all of them are being used.
Tony
OSR
Hope to see everyone September 19 for the next Developing File Systems for Windows seminar (http://www.osr.com/fsd.html)
Thanks Tony. I have seen stream contexts hanging around for ever but these
are stream handle contexts. I had thought that a stream handle context would
be released when the handle was closed.
It looks likes NFS might have something to do with what I am seeing.
Bill Wandel
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, August 19, 2011 4:38 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Lifetime of stream handle contexts
The lifetime of a stream context is the lifetime of the file object.
There are two sources of reference on a file object: handles (actual “open
instances” of the file object) and references (someone in kernel mode
keeping a pointer around.) It sounds like the “leak” is just due to high
utilization since you can unload the drivers (and filter manager won’t allow
you to unload with dangling contexts outstanding.)
If you want to see outstanding handle references, you can use “!handle” in
the debugger - it will display the handles for a given process.
As for OS references, well, pretty much anything can keep those around (like
the memory manager does for memory mapped files, for example. Or the cache
manager does for cache maps.) Unless you enable object tracking (via the
global flags) there’s no easy way to find all those objects (and tracking
them will slow the system down quite a bit.) You might find it easier to
simply add your context structures to a big global list in your driver and
then walk through that list to see where all of them are being used.
Tony
OSR
Hope to see everyone September 19 for the next Developing File Systems for
Windows seminar (http://www.osr.com/fsd.html)
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
Stream Handle Context = lifetime of file object
Stream Context = lifetime of FCB (so lifetime of ALL file objects for a given file stream.)
NFS would make sense, as it would likely keep things open “for a while”.
Tony
OSR
Hope to see everyone September 19 for the next Developing File Systems for Windows seminar (http://www.osr.com/fsd.html)
Thanks again.
Bill
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Friday, August 19, 2011 5:23 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Lifetime of stream handle contexts
Stream Handle Context = lifetime of file object
Stream Context = lifetime of FCB (so lifetime of ALL file objects for a
given file stream.)
NFS would make sense, as it would likely keep things open “for a while”.
Tony
OSR
Hope to see everyone September 19 for the next Developing File Systems for
Windows seminar (http://www.osr.com/fsd.html)
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
The StreamHandleContext goes away some time after the minifilter returns
from the preClose callback but before the IRP_MJ_CLOSE reaches the file
system.
Thanks,
Alex.