In a test scenario I’m running on an amd64 box I observe a leak of
around 220MB/day of non-paged pool for the tag Vepp (so it would show up
in code as ‘ppeV’.) If there is some kind reader out there who might
have knowledge of this tag, please let me know (privately or in the
forum). If I don’t hear from anyone, rest assured that when I do find
it, I’ll publish my findings.
Indeed, this might make a good article for The NT Insider - “how to find
a pool leak”. Fortunately (for me) my own tags are now all down in the
noise. I need to find this leak because it limits the amount of time I
can run my tests (I hit around 24 hours and then get some nice “low
memory usage” simulations. Last time I crashed it was after 600+
non-paged pool allocations had failed… 
)
Here is the machine’s loaded module list:
start end module name
fffff80000800000 fffff8000085e000 hal (deferred)
fffff80001000000 fffff8000146e000 nt (pdb symbols)
fffff97fff000000 fffff97fff45d000 win32k (deferred)
fffff97fff45d000 fffff97fff484000 dxg (deferred)
fffff97fff484000 fffff97fffa0a400 nv4_disp (deferred)
fffffadfc46b6000 fffffadfc4780000 srv (deferred)
fffffadfc4852000 fffffadfc48e2000 HTTP (deferred)
fffffadfc4928000 fffffadfc4950000 vmx86 (deferred)
fffffadfc4950000 fffffadfc4999000 mrxdav (deferred)
fffffadfc4c73000 fffffadfc4cab000 kmixer (deferred)
fffffadfc4cab000 fffffadfc4cc5000 swmidi (deferred)
fffffadfc4cc5000 fffffadfc4cfa000 aec (deferred)
fffffadfc4cfa000 fffffadfc4d25000 sysaudio (deferred)
fffffadfc4d25000 fffffadfc4d59000 wdmaud (deferred)
fffffadfc61a1000 fffffadfc61ad000 hcmon (deferred)
fffffadfc6cd8000 fffffadfc6d02000 dump_nvata64 (deferred)
fffffadfc6d02000 fffffadfc6d23000 Cdfs (deferred)
fffffadfc6d69000 fffffadfc6e35000 mrxsmb (deferred)
fffffadfc6e35000 fffffadfc6e86000 rdbss (deferred)
fffffadfc6e86000 fffffadfc6ed4000 afd (deferred)
fffffadfc6ed4000 fffffadfc6f37000 netbt (deferred)
fffffadfc6f37000 fffffadfc6f6f000 ipnat (deferred)
fffffadfc6f6f000 fffffadfc7030000 tcpip (deferred)
fffffadfc7030000 fffffadfc705b000 ipsec (deferred)
fffffadfc705b000 fffffadfc706f000 Npfs (deferred)
fffffadfc708f000 fffffadfc7098000 dump_WMILIB (deferred)
fffffadfc728d000 fffffadfc72a3a00 nvarm64 (deferred)
fffffadfc72a4000 fffffadfc73b8500 nvmcp64 (deferred)
fffffadfc73b9000 fffffadfc73f9000 portcls (deferred)
fffffadfc73f9000 fffffadfc748c400 nvapu64 (deferred)
fffffadfc748d000 fffffadfc74ad000 usbhub (deferred)
fffffadfc74f3000 fffffadfc7507000 NDProxy (deferred)
fffffadfc7507000 fffffadfc751d000 termdd (deferred)
fffffadfc751d000 fffffadfc7573000 rdpdr (deferred)
fffffadfc7573000 fffffadfc7589000 msgpc (deferred)
fffffadfc7589000 fffffadfc75a9000 psched (deferred)
fffffadfc7619000 fffffadfc7624000 ndisuio (deferred)
fffffadfc7629000 fffffadfc7637000 vmnetbridge (deferred)
fffffadfc7649000 fffffadfc766c000 raspptp (deferred)
fffffadfc766c000 fffffadfc7680000 raspppoe (deferred)
fffffadfc7680000 fffffadfc76ac000 ndiswan (deferred)
fffffadfc76ac000 fffffadfc76d2000 rasl2tp (deferred)
fffffadfc76d2000 fffffadfc76ef000 i8042prt (deferred)
fffffadfc76ef000 fffffadfc7712000 serial (deferred)
fffffadfc7712000 fffffadfc7735000 VIDEOPRT (deferred)
fffffadfc7735000 fffffadfc7ba1f00 nv4_mini (deferred)
fffffadfc7ba2000 fffffadfc7bec000 NVSNPU (deferred)
fffffadfc7bec000 fffffadfc7c48b80 NVNRM (deferred)
fffffadfc7c49000 fffffadfc7c8f000 yk51x64 (deferred)
fffffadfc7cd5000 fffffadfc7ceb000 redbook (deferred)
fffffadfc7ceb000 fffffadfc7d06000 cdrom (deferred)
fffffadfc7d06000 fffffadfc7d1d000 imapi (deferred)
fffffadfc7d1d000 fffffadfc7d66000 ks (deferred)
fffffadfc7d66000 fffffadfc7d7c180 nvax64 (deferred)
fffffadfc7d7d000 fffffadfc7db7000 USBPORT (deferred)
fffffadfc7e97000 fffffadfc7e9d000 AsIO (deferred)
fffffadfc8108000 fffffadfc813e000 Mup (deferred)
fffffadfc813e000 fffffadfc8199000 NDIS (deferred)
fffffadfc8199000 fffffadfc82b1000 Ntfs (deferred)
fffffadfc82b1000 fffffadfc82e4000 KSecDD (deferred)
fffffadfc82e4000 fffffadfc8307000 sr (deferred)
fffffadfc8307000 fffffadfc8344000 fltMgr (deferred)
fffffadfc8344000 fffffadfc8363000 CLASSPNP (deferred)
fffffadfc8363000 fffffadfc8378000 disk (deferred)
fffffadfc8378000 fffffadfc8393000 SI3114 (deferred)
fffffadfc8393000 fffffadfc83c4000 SCSIPORT (deferred)
fffffadfc83c4000 fffffadfc8414000 SI3114R5 (deferred)
fffffadfc8414000 fffffadfc843e000 nvata64 (deferred)
fffffadfc843e000 fffffadfc846b000 atapi (deferred)
fffffadfc846b000 fffffadfc84b6000 volsnap (deferred)
fffffadfc84b6000 fffffadfc84fc000 dmio (deferred)
fffffadfc84fc000 fffffadfc853c000 ftdisk (deferred)
fffffadfc853c000 fffffadfc8552000 MountMgr (deferred)
fffffadfc8552000 fffffadfc8565a00 1394BUS (deferred)
fffffadfc8566000 fffffadfc857cc80 ohci1394 (deferred)
fffffadfc857d000 fffffadfc859e000 pci (deferred)
fffffadfc859e000 fffffadfc85f2000 ACPI (deferred)
fffffadfc86f3000 fffffadfc8704000 processr (deferred)
fffffadfc8706000 fffffadfc8718000 wanarp (deferred)
fffffadfc8719000 fffffadfc872b000 netbios (deferred)
fffffadfc872c000 fffffadfc873e000 Fips (deferred)
fffffadfc89fb000 fffffadfc8a04000 kdcom (deferred)
fffffadfc8a0b000 fffffadfc8a14000 BOOTVID (deferred)
fffffadfc8a1b000 fffffadfc8a24000 WMILIB (deferred)
fffffadfc8a2b000 fffffadfc8a34000 isapnp (deferred)
fffffadfc8a3b000 fffffadfc8a4b000 PCIIDEX (deferred)
fffffadfc8a4b000 fffffadfc8a5b000 PartMgr (deferred)
fffffadfc8a5b000 fffffadfc8a65000 SiWinAcc (deferred)
fffffadfc8a6b000 fffffadfc8a76000 crcdisk (deferred)
fffffadfc8a8b000 fffffadfc8a98000 mouclass (deferred)
fffffadfc8abb000 fffffadfc8ac9000 kbdclass (deferred)
fffffadfc8acb000 fffffadfc8ad5000 ndistapi (deferred)
fffffadfc8adb000 fffffadfc8aea000 TDI (deferred)
fffffadfc8afb000 fffffadfc8b08000 ptilink (deferred)
fffffadfc8b0b000 fffffadfc8b16000 raspti (deferred)
fffffadfc8b1b000 fffffadfc8b2b000 update (deferred)
fffffadfc8b2b000 fffffadfc8b35000 rasacd (deferred)
fffffadfc8b3b000 fffffadfc8b45000 VMNET (deferred)
fffffadfc8b4b000 fffffadfc8b57000 Dxapi (deferred)
fffffadfc8b5b000 fffffadfc8b68000 mssmbios (deferred)
fffffadfc8b6b000 fffffadfc8b75980 usbehci (deferred)
fffffadfc8b8b000 fffffadfc8b97a00 NVENETFD (deferred)
fffffadfc8b9b000 fffffadfc8ba7000 flpydisk (deferred)
fffffadfc8bab000 fffffadfc8bb9000 vga (deferred)
fffffadfc8bbb000 fffffadfc8bc5000 mnmdd (deferred)
fffffadfc8bcb000 fffffadfc8bd5000 RDPCDD (deferred)
fffffadfc8bfb000 fffffadfc8c06000 nvnetbus (deferred)
fffffadfc8c0b000 fffffadfc8c15000 Fs_Rec (deferred)
fffffadfc8c2b000 fffffadfc8c34000 watchdog (deferred)
fffffadfc8c3b000 fffffadfc8c48000 Msfs (deferred)
fffffadfc8c4b000 fffffadfc8c59000 fdc (deferred)
fffffadfc8c5b000 fffffadfc8c67000 serenum (deferred)
fffffadfc8cab000 fffffadfc8cb3000 ASACPI (deferred)
fffffadfc8cc3000 fffffadfc8ccb000 audstub (deferred)
fffffadfc8cdb000 fffffadfc8ce3000 CdaC15BA (deferred)
fffffadfc8cfb000 fffffadfc8d03000 CdaD10BA (deferred)
fffffadfc8d7b000 fffffadfc8d83000 vmnetadapter (deferred)
fffffadfc8deb000 fffffadfc8df3000 secdrv (deferred)
fffffadfc8df3000 fffffadfc8dfb000 Null (deferred)
fffffadfc8dfb000 fffffadfc8e02000 pciide (deferred)
fffffadfc8e02000 fffffadfc8e09000 dmload (deferred)
fffffadfc8f13000 fffffadfc8f19700 usbohci (deferred)
fffffadfc8f28000 fffffadfc8f2de80 ksthunk (deferred)
fffffadfc9063000 fffffadfc906a000 Beep (deferred)
fffffadfc9193000 fffffadfc9195800 splitter (deferred)
fffffadfc91f9000 fffffadfc91fa400 swenum (deferred)
fffffadfc9205000 fffffadfc9206d80 USBD (deferred)
This is a base 64-bit Windows system (asus motherboard, based on an
NVIDIA chipset.) I’ve installed VMWare on this machine as well. 2MB
installed (maybe if I install another 2MB I’ll get an additional 24
hours out of the box.)
Verifier is enabled for ALL drivers on this system - but the tag value
does not show up in verifier (“!verifier 3”) but does show up in the
pool tag list (already 1MB in use) - suggesting that whoever owns this
driver is bypassing verifier’s version of the memory allocation routine.
A quick string search of the binaries (\windows\system32\drivers) does
not show the tag in any of them, either.
My own driver is a totally virtual in-memory file system - except for my
ERESOURCES my structures come from paged pool. The test scenario is a
continuous rename/delete multi-threaded test (where we’re trying to
delete the files and rename them simultaneously, looking for complex
race conditions down these paths.)
So, if anyone has information to save me time, please let me know.
Otherwise, you know what I’ll be doing for the next day or two.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.