StreamContexts and reference counts

NTFSD
1

I went through the MSDN documentation and also some blog posts on this topic. However, I am still having some issues in my PostCreate() code with the reference counts.

To make sure I am understanding it right,

I maintain a ref_count=0 intiialized in PostCreate().

If FltGetStreamContext() passes, increment ref_count.
Else go to cleanup:

If FltSetStreamContext() passes, increment ref_count.
Else go to cleanup:

If FltAllocateContext() passes, increment ref_count.
else go to cleanup:

Cleanup:

for (i=0; i<ref_count i> if (context != NULL)
FltReleaseContext(Context);</ref_count>

2

You don’t need to keep track of the reference count yourself. FltMgr
manages the reference counting of the context internally. Your
responsibility is to ensure that when you reference the context, you
subsequently de-reference it. There have been many threads on here
about how to do this properly. The most confusing aspect of this is
understanding when you need to release the context when you allocate and
set it. Within the last month or two, somebody posted a nice
explanation of how that might work internally, and Alex confirmed that
yes, you may think of it that way but the particulars are subject to
change.

Pragmatically speaking, you probably don’t need to keep a count to
handle this properly. Maintaining a pointer should be sufficient unless
you’re doing something weird.

~Eric

A couple points on your code follow:

I maintain a ref_count=0 intiialized in PostCreate().

If FltGetStreamContext() passes, increment ref_count.

conceptually correct, if you call FltGetXxxContext, you need to release
the context afterwards

Else go to cleanup:

If FltSetStreamContext() passes, increment ref_count.

What context would you be setting here? You haven’t allocated a new one
yet. The only context you have is the one you just got.

Else go to cleanup:

If FltAllocateContext() passes, increment ref_count.

If you allocate context, you do need to release it. If you then try to
set a newly allocated context, you need to release it whether the call
to FltSetXxxContext succeeds or fails. If the Set call succeeds, FltMgr
(in essence) adds its own reference to the context which it releases
when the object is disposed of. If it fails, FltMgr does not. In
either case, you still hold a reference to the context that you need to
release.

else go to cleanup:

Cleanup:

for (i=0; i<ref_count i>> if (context != NULL)
> FltReleaseContext(Context);
></ref_count>

3

Eric, I did read that post by Alex- I guess I missed the part that I dont have to explicitly maintain a reference count. Although it made things so much more clearer, I still have some doubts.

In my postcreate(), I have GetContext, AllocateContext and a bunch of SetStreamContexts.

Hence as long as I have a FltReleaseContext() before I exit PostCreate, the ref count on this context should be automatically decremented to zero?

For some reason, when I do the above, I am unable to unload the minifilter(it hangs) - which points to the fact that the reference count on the context is not zero?

4

Maybe I wasn’t as clear as I could have been. If you do several things
that reference a context without releasing it, you may need to maintain
a count. I’m only pointing out that if you’re only dealing with one
context, you probably don’t need to maintain a ref count.

I was able to order the operations such that I only have at most one
reference to any given context, which spares me the hassle of
maintaining my own count of how many references I have. Also, if I only
ever have one, that means I can’t screw up the counting (except possibly
by not NULLing a pointer after I FltReleaseContext it, which is a rare
issue because FltReleaseContext is usually the last thing to happen in a
function).

~Eric

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@yahoo.com
Sent: Tuesday, December 08, 2009 11:25 AM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] StreamContexts and reference counts

Eric, I did read that post by Alex- I guess I missed the
part that I dont have to explicitly maintain a reference
count. Although it made things so much more clearer, I still
have some doubts.

In my postcreate(), I have GetContext, AllocateContext and a
bunch of SetStreamContexts.

Hence as long as I have a FltReleaseContext() before I exit
PostCreate, the ref count on this context should be
automatically decremented to zero?

For some reason, when I do the above, I am unable to unload
the minifilter(it hangs) - which points to the fact that the
reference count on the context is not zero?

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online
at http://www.osronline.com/page.cfm?name=ListServer

5

In the simplest case, the fact that your filter has outstanding contexts on some objects should not prevent it from unloading (i.e. you don’t have to somehow cleanup all your contexts before unloading). So at unload time your minifilter might successfully unload leaving some contexts on some streams (for example) behind. This should be fine, filter manager will simply clean them up when the stream gets torn down and that’s that.

However, if you add references on things in your contexts (for example, if each stream context has a reference to the instance for some reason) then the things you have referenced will not be released until all the contexts go away, which might take a while. It is also possible to get into a loop of references, meaning you will never release all the references.

If you enable driver verifier for filter manager and your filter you might be able to get a lot more information on why the filter isn’t unloading.

Regards,
Alex.
This posting is provided “AS IS” with no warranties, and confers no rights.

6

Alex, I enabled driver verifier for fltmgr.sys with all options enabled. How do I attached my filter? Also I am able to see the global counters, but where would I see the information about contexts?

7

Well, just add the name of the file for filter (myfilter.sys) to the list of drivers to be verified. Then just load the filter and run your tests and stuff and then when you try to unload it should tell you if it finds any leaks.

Regards,
Alex.
This posting is provided “AS IS” with no warranties, and confers no rights.

8

Alex, I added my scanner.sys to the verifier list. When I load the filter, run the tests and unload the filter, nothing happens - I dont see any bug checks. The filter unload still hangs.

9

I commented out the FltSetStreamContext and the filter unload works fine.

Is there something I am missing in the use of SetContextStream?

PostCreate()
{
//Get FileName

status = FltGetStreamContext();
if (!NT_SUCCESS(status))
{
if (status == STATUS_NOT_FOUND)
{
status = FltAllocateContext();
if (!NT_SUCCESS(status))
{
//Reference count is zero now
FltReleaseFileNameInformation();
return POSTOP_FINISHED_PROCESSING;
}
}
else
{
//Reference count is zero now
FltReleaseFileNameInformation();
return POSTOP_FINISHED_PROCESSING;
}
}

if(Context->AlreadyScanned == FALSE)
{
ScannerpScanFileInUserMode();
if (!Scanned)
{
context->AlreadyScanned = FALSE;
status = FltSetStreamContext(FltObjects->Instance, FltObjects->FileObject,
FLT_SET_CONTEXT_KEEP_IF_EXISTS, context, NULL);
if ( !NT_SUCCESS(status ))
{
//reference count is 1 at this point
FltReleaseContext();
FltReleaseFileNameInformation();
return;
}
}
//Reference count 1 at this point
FltReleaseContext();
FltReleaseFileNameInformation();
return;

}

}

10

Well, please look at the CTX sample. From this pseudocode it seems to me that if you already find a context (i.e. FltGetStreamContext() finds a context and returns it) then you will scan the file and set the same context again…

Regards,
Alex.
This posting is provided “AS IS” with no warranties, and confers no rights.

11

The sample shows use of streamHandle and File contexts in the PostCreate operation. All I am trying to do is prevent multiple scans of a file. So just using flags in a streamContext should be sufficient?

12

As you pointed on FltSetStreamContext…,

u are suppose to call release even if it fails, which ur code is not dong currently.

from wdk

“After calling FltSetStreamContext, the caller must call FltReleaseContext to decrement the reference count on NewContext, even if the call to FltSetStreamContext was unsuccessful.” It could be a possible reason.