Hi all,
I’m working on a HSM product, and am having some problems with an AV solution.
In fact the problem seems to be when I am opening a reparse point and sending the IRP back down again.
Here is what I do (in pseudocode)
IrpCreate{
…
sendirpdownwithcompletion (iocopycurrent + Iosetcompl + IoCall(attachedtodevobj)
waitforcompletion (completion routine signals an event)
if status is reparse, and this is my GUID
modify create options -> |= FILE_OPEN_REPARSE_POINT
sendirpdownwithcompletion
waitforcompletion
completeIrp
return status
In completion routine, if this is my GUID, return STATUS_MORE_PROCESSING_REQUIRED
In my tests, I have a simple test that opens and closes a known reparse point, and does this over and over and …
When I run this standalone, (no AV), it works perfectly, in fact with some AV vendors this works fine, but with one of them, I start geting leaks of memory after a while.
Is there anything I am doing wrong that anyone can comment on?
Thanks!
Did you look at the tag for the memory being leaked? Is it *your* tag? If not, it is likely an issue in the AV product in question and you will need to work with them to get it resolved.
Filter-to-filter interaction issues are a constant problem. We recently found that Clearcase also has issues when we return STATUS_REPARSE (apparently it causes them to walk an unusual path in their code and leads to a double free of memory under some circumstances. We were asked - indirectly, via our customer - to change the architecture of our product to not use STATUS_REPARSE to work around this issue. Since it’s fundamental to the way that we work, we declined and suggested they fix their code instead. We’ll see how things work out.)
Anyone that builds file systems and/or file system filter drivers should be prepared for an endless support burden associated with interop issues. That is why I encourage everyone to attend Plugfest, since it is often easier to get such issues resolved there, with actual developers present, than to work through the normal support channels (which, for most companies appear to be more geared towards annoying customers enough so they go away, not to expeditiously resolving such issues.)
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Hi John, I work for a security software company and have recently fixed just such a bug. We can discuss further offline if you’d like.
Thanks,
Ron
xxxxx@hotmail.com
Tony,
I have a list here of the pooltags that are using more than 500KB of memory,
AfdC 4706 752960 0 0 Afd connection structure , Binary: afd.sys
CM16 0 0 100 421888 Internal Configuration manager allocations , Binary: nt!cm
CM25 0 0 207 974848 Internal Configuration manager allocations , Binary: nt!cm
CM35 0 0 223 4800512 Internal Configuration manager allocations , Binary: nt!cm
CMAl 0 0 186 761856 internal registry memory allocator pool tag , Binary: nt!cm
File 8539 1300216 0 0 File objects
Gh05 0 0 42 647880 UNKNOWN pooltag ‘Gh05’, please update pooltag.txt
Gh25 0 0 174 1114800 UNKNOWN pooltag ‘Gh25’, please update pooltag.txt
Gla1 3 336 320 660480 Gdi handle manager specific object types allocated from lookaside memory: defined in w32\ntgdi\inc\ntgdistr.h , Binary: win32k.sys
LSwi 1 2576384 0 0 initial work context
MmCm 4735 12773896 0 0 Calls made to MmAllocateContiguousMemory , Binary: nt!mm
MmSt 0 0 2160 8694920 Mm section object prototype ptes , Binary: nt!mm
NtFC 1678685 201445240 10 648 Create.c , Binary: ntfs.sys
NtFU 0 0 46854 5622704 usnsup.c , Binary: ntfs.sys
NtFs 48953 1959944 49423 2040536 StrucSup.c , Binary: ntfs.sys
Ntff 14 2912 48431 39519696 FCB_DATA , Binary: ntfs.sys
Ntfn 48947 1960248 0 0 SCB_NONPAGED , Binary: ntfs.sys
Ntfr 97356 6231752 0 0 ERESOURCE , Binary: ntfs.sys
Obtb 0 0 254 626736 object tables via EX handle.c , Binary: nt!ob
R100 21 9585384 44 4412328 UNKNOWN pooltag ‘R100’, please update pooltag.txt
SACC 1 8192 250 1008968 UNKNOWN pooltag ‘SACC’, please update pooltag.txt
SavE 3 168 672 29176240 UNKNOWN pooltag ‘SavE’, please update pooltag.txt
TCPt 32 1422960 0 0 TCP/IP network protocol , Binary: TCP
TPLA 128 524288 0 0 UNKNOWN pooltag ‘TPLA’, please update pooltag.txt
TSdd 0 0 45 713680 RDPDD - Hydra Display Driver , Binary: rdpdd.sys
TSwd 5 312 18 839920 RDPWD - Hydra Winstation Driver , Binary: rdpwd.sys
Thre 1091 680784 0 0 Thread objects , Binary: nt!ps
Toke 0 0 314 522520 Token objects , Binary: nt!se
Ttfd 0 0 1130 991592 TrueType Font driver
UlHT 0 0 1 4198400 Hash Table , Binary: http.sys
Wdm 1132 765664 1 56 WDM
Wmit 8 401408 8 393872 Wmi Trace
TOTAL 1947924 250349088 210887 117891152
My own driver is using none of these tags.
the biggest user is NtFC (201,445,240), which makes me think that another driver is not releasing some ntfs data when the IRP goes back down again.
Any guesses/suggestions?
Thanks!
John
It sure looks like someone is leaving files around for far too long. Note that the NTFS FCB tag is also very large (non-paged pool) and the File tag (FILE_OBJECT) is also fairly substantial. 200MB of paged pool is pretty big.
I’d suggest touching bases with Ron and see if the issue he knows about is the issue you are seeing (in which case you have your fix!) If not, you’re stuck trying to figure out why the files aren’t going away (I hope you are very good in the debugger, since you’ll be looking at someone ELSE’s binary to figure this out.)
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com