I saw a post several months ago from someone from Microsoft (possibly Neal) that said that Filter Manager might provide some help in Vista/2008 for identifying what objects are being leaked that are preventing a mini-filter from unloading.
Anyone know if that has been implemented and if so how to take advantage of it?
In my experience verifier and your mini-filter do a fine job of identifying
what is leaked on xp and later (I just don’t test w2k.). ‘Why’ is usually
more difficult than ‘what’.
On Fri, Apr 11, 2008 at 4:45 PM, wrote:
> I saw a post several months ago from someone from Microsoft (possibly
> Neal) that said that Filter Manager might provide some help in Vista/2008
> for identifying what objects are being leaked that are preventing a
> mini-filter from unloading.
>
> Anyone know if that has been implemented and if so how to take advantage
> of it?
>
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> You are currently subscribed to ntfsd as: xxxxx@hollistech.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
–
Mark Roddy
I wish I could agree with you that figuring out “what” is the easy part. It would be nice if the Filter Manager would tell me what the issue is, since it clearly knows something isn’t right. An address of an object or even just the type of object would probably be plenty.
You turned verifier on?
On Fri, Apr 11, 2008 at 6:08 PM, wrote:
> I wish I could agree with you that figuring out “what” is the easy part.
> It would be nice if the Filter Manager would tell me what the issue is,
> since it clearly knows something isn’t right. An address of an object or
> even just the type of object would probably be plenty.
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> You are currently subscribed to ntfsd as: xxxxx@hollistech.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
–
Mark Roddy
Yes, !verifier 3 <driver.sys> says I have no outstanding allocations and I get no BSODs.
Here is how it goes down:
1. Mini-filter starts
2. User-mode service starts
3. Some time later user-mode service crashes or is intentionally killed.
4. User-mode service starts up again
5. Driver will no longer unload.
If #3 does not happen, everything shuts down fine. If #3 does not happen, my context cleanup routine is called. If #3 does happen, it is not called at all.
I had some memory allocations in my InstanceSetup that were not getting freed b/c of this, so I just removed all the code that needed those allocations and
it stills won’t shutdown.
The driver and service communicate with each other via overlapped FltSendMessage calls and non-overlapped FilterSendMessage calls.
Is there some other debugger command that might help here?</driver.sys>
Yes, you are trying to display data, but have you turned on verifier for
that driver? The answer is probably no, or else the data you are looking
for would have been there.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply
wrote in message news:xxxxx@ntfsd…
> Yes, !verifier 3 <driver.sys> says I have no outstanding allocations and I
> get no BSODs.
>
> Here is how it goes down:
>
> 1. Mini-filter starts
> 2. User-mode service starts
> 3. Some time later user-mode service crashes or is intentionally killed.
> 4. User-mode service starts up again
> 5. Driver will no longer unload.
>
> If #3 does not happen, everything shuts down fine. If #3 does not happen,
> my context cleanup routine is called. If #3 does happen, it is not called
> at all.
> I had some memory allocations in my InstanceSetup that were not getting
> freed b/c of this, so I just removed all the code that needed those
> allocations and
> it stills won’t shutdown.
>
> The driver and service communicate with each other via overlapped
> FltSendMessage calls and non-overlapped FilterSendMessage calls.
>
> Is there some other debugger command that might help here?
></driver.sys>
Yes, of course it is turned on for that driver. That’s how I discovered that the context cleanup callback was not being invoked. !verifier 3 showed me outstanding allocations with my pool tag.
If this were any easy one, I wouldn’t be asking for help.
Oh, and a small correction on the above post…
If the user-mode service does not restart after it crashes (which is #4, not #3), I can stop the driver just fine. But once the user-mode service starts up again, it will no longer unload.
Here is a stack trace, maybe this will be helpful.
THREAD 8221b3f0 Cid 0004.0044 Teb: 00000000 Win32Thread: 00000000 WAIT: (Unknown) KernelMode Non-Alertable
f88b2b20 SynchronizationEvent
Not impersonating
DeviceMap e10018b8
Owning Process 8221e7a8 Image: System
Wait Start TickCount 22782 Ticks: 1032 (0:00:00:16.125)
Context Switch Count 2035
UserTime 00:00:00.000
KernelTime 00:00:05.968
Start Address nt!ExpWorkerThread (0x8087f844)
Stack Init f88b3000 Current f88b2a6c Base f88b3000 Limit f88b0000 Call 0
Priority 12 BasePriority 12 PriorityDecrement 0
ChildEBP RetAddr Args to Child
f88b2a84 80832f7a 8221b468 8221b3f0 8221b498 nt!KiSwapContext+0x26 (FPO: [Uses EBP] [0,0,4])
f88b2ab0 8082925c 820777e8 820777e0 00000002 nt!KiSwapThread+0x284 (FPO: [Non-Fpo])
f88b2af8 f82167c8 f88b2b20 00000000 00000000 nt!KeWaitForSingleObject+0x346 (FPO: [Non-Fpo])
f88b2b38 f8214b60 f88b2b70 f820e854 820777e0 fltMgr!FltpExWaitForRundownProtectionRelease+0x72 (FPO: [Non-Fpo])
f88b2b40 f820e854 820777e0 8087bf24 82326498 fltMgr!FltpObjectRundownWait+0x10 (FPO: [Non-Fpo])
f88b2b70 f8215538 81884008 00000004 81c0df38 fltMgr!FltpFreeInstance+0xa4 (FPO: [Non-Fpo])
f88b2b94 f70c2595 82077814 00000000 00000000 fltMgr!FltUnregisterFilter+0x96 (FPO: [Non-Fpo])
f88b2ba8 f8210bec 00000001 8087bf24 8187d004 Driver!Driver_Unload+0x315 (FPO: [Non-Fpo]) (CONV: stdcall) [Path to driver.c]
f88b2d40 f8210d9f 82326498 00000001 00000001 fltMgr!FltpDoUnloadFilter+0xf8 (FPO: [Non-Fpo])
f88b2d64 808f6213 00000000 8221b3f0 808ae5fc fltMgr!FltpMiniFilterDriverUnload+0xab (FPO: [Non-Fpo])
f88b2d80 8087f92f f7459b84 00000000 8221b3f0 nt!IopLoadUnloadDriver+0x19 (FPO: [Non-Fpo])
f88b2dac 80948bd0 f7459b84 00000000 00000000 nt!ExpWorkerThread+0xeb (FPO: [Non-Fpo])
f88b2ddc 8088d4e2 8087f844 00000001 00000000 nt!PspSystemThreadStartup+0x2e (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16
xxxxx@charter.net wrote:
I have to ask, you did setup a disconnectnotifycallback when you created
the communication port, right?
Within this callback, are you calling FltCloseClientPort?
Given the scenario your describing, perhaps your usermode service
crashes before calling CloseHandle. If your
DisconnectNotifyCallback is not cleaning-up after that by closing the
clientport, then your leaking that handle (port object,
whatever you wana call it).
Next time you start and close it (between steps 4 - 5) the filter is
refusing to unload due to the first leaked handle.
Just guessing…
Good Luck,
Matt
Yes, !verifier 3 <driver.sys> says I have no outstanding allocations and I get no BSODs.
>
> Here is how it goes down:
>
> 1. Mini-filter starts
> 2. User-mode service starts
> 3. Some time later user-mode service crashes or is intentionally killed.
> 4. User-mode service starts up again
> 5. Driver will no longer unload.
>
> If #3 does not happen, everything shuts down fine. If #3 does not happen, my context cleanup routine is called. If #3 does happen, it is not called at all.
> I had some memory allocations in my InstanceSetup that were not getting freed b/c of this, so I just removed all the code that needed those allocations and
> it stills won’t shutdown.
>
> The driver and service communicate with each other via overlapped FltSendMessage calls and non-overlapped FilterSendMessage calls.
>
> Is there some other debugger command that might help here?
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> You are currently subscribed to ntfsd as: matt-martin@tx.rr.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
></driver.sys>
Matt wrote:
I have to ask, you did setup a disconnectnotifycallback when you
created the communication port, right?
Of course you did, just remembered that’s required
I use a counter for each of my ports that gets incremented in Connect and decremented in Disconnect. The counters are 0. I also have logging in those callbacks to ensure they are being invoked, and they are even when the service crashes or is manually terminated in Task Manager.
I use four different ports and for only two of them do I track the client port in the driver. I don’t for the other two b/c I don’t have a specific need for the handles in the driver and there can be multiple clients of those ports.
Could this be an issue? Do I have to explicitely track and close all of the client ports? This doesn’t seem to be an issue when the service and driver are stopped normally.
xxxxx@charter.net wrote:
I would hope you would review this particular code. It sounds as if your
problem could lay here.
Do I have to explicitely track and close all of the client ports?
Those that were opened, yes - as far as I know they must be closed
before unloading.
This doesn’t seem to be an issue when the service and driver are stopped normally.
And you have code in an unload routine to wrap-up these lose ends (user
mode)? Killing an app is quite different…
Perhaps a simple test would be in order, what happens if you close all
ports in the driver’s disconnectnotify routine?
I honestly don’t know what is wrong, but for some reason my thoughts
gravitated towards port creation/destruction.
Good Luck.
I would make sure you’re turning on the I/O Verification in the Driver Verifier and therefore enabling the Filter Manager Verifier. Filter Manager team recommended the 0x7b value last time I checked. Filter Manager then spews into the debugger trace at the filter unload time everything it is unhappy about. That helped me fix my unload problems on multiple occasions. You obviously need to have the system debug trace enabled if running on Vista or later.
Regards,
Ilya Faenson
I have verifier on at 0xffb for my driver, which is everything but Low Resource Simulation, and the Debug Print Filter for DEFAULT and FLTMGR both set to 0xF. I rebooted after setting verifier and the Debug Print Filter.
I am still getting no such spewing as Ilya has described.
Have you tried !process 0 7 ?
May be there are unacknowledged FltSendMessage-calls wating because of
killing your user-mode service.
wrote news:xxxxx@ntfsd…
>I have verifier on at 0xffb for my driver, which is everything but Low
>Resource Simulation, and the Debug Print Filter for DEFAULT and FLTMGR both
>set to 0xF. I rebooted after setting verifier and the Debug Print Filter.
>
> I am still getting no such spewing as Ilya has described.
>
Thanks Frank, but I did that already and it shows nothing useful. There are two threads I can find that have anything to do with my product: the one calling ControlService in my user-mode service to stop the driver and the one that called my unload routine (which you can see a stack trace for above). All of the other user-mode threads have shutdown and I see no others with my driver on them.
Hey guys, thanks for the help. Two days later I finally found root cause. Data passed to the driver when the service starts is used to help obtain the instance pointer via a call to FltGetVolumeInstanceFromName. I was not checking the instance pointer before making that call and it was being leaked. Verifier does not help if the leak is caused by non-pool memory or handles as far as I can tell.
Sure would be nice if Verifier could help with that stuff.
In Vista and beyond, Fltmgr verifier will track following objects when used by a minifilter:
On minifilter unload, Verifier detects if any of the above objects have been leaked
Verifier also detects over-deref of the objects listed above.
As was correctly pointed out in this thread, you can enable fltmgr verifier by turning on the I/O Verification in the Driver Verifier for your driver.
Regards,
Sarosh.
This posting is provided “AS IS” with no warranties, and confers no rights
