BAD_POOL_CALLER

Hi all…

I’ve been looking at a memory dump with the following bugcheck
information from a Windows 2000 server, 4 processor machine:

BAD_POOL_CALLER (c2)

The current thread is making a bad pool request. Typically this is at a
bad IRQL level or double freeing the same allocation, etc.

Arguments:

Arg1: 00000007, Attempt to free pool which was already freed

Arg2: 00000b8a, (reserved)

Arg3: e2d1f780, Memory contents of the pool block

Arg4: e2d1f788, Address of the block of pool being deallocated

According to the top of the stack trace, this appears to come from a
call to free off a memory block at e2d1f788 as follows:-

f61a2ce0 8046c84d e2d1f788 00000000 f5d26cc0 nt!ExFreePoolWithTag+0x19b

f61a2cec f5d26cc0 e2d1f788 e1d7f4d0 e1430fb8 nt!ExFreePool+0xb

From the documentation, it appears that this error is an attempt to free
off a memory block that has already been freed. I suspect that the
problem that I am trying to solve is caused by an incorrectly
decremented reference count so this is entirely possible. However, it
appears that at the time of freeing, this memory address is still valid
for the following reasons:

Firstly, immediately prior to freeing the memory, I set the first four
bytes to “FREE” to mark it as invalid. I don’t free this memory from
anywhere else and, if this had already been freed, I would have expected
this to generate an access violation. Moreover, the address does appear
to be valid: I have verified this by using the !pte command in WINDBG
which yields the following result:-

!pte e2d1f788

E2D1F788 - PDE at C0300E2C PTE at C038B47C

contains 2C38E963 contains 10295963

pfn 2c38e G-DA–KWV pfn 10295 G-DA-KWV

I can also dereference this address and see the signature that I wrote
immediately prior to the call. Finally, I have converted the virtual
address to a physical address and have dumped this using the !dc command
and compared it with the dump of the virtual address to verify that it
is correctly backed by physical memory.

I am allocating memory using the ExAllocateMemoryWithTag function and
the bugcheck occurs when I attempt to free it with ExFreePool function.

One clue may be the output of the !pool command on this address which is
as follows:-

e2d1f000 size: 80 previous size: 0 (Allocated) Lfs

e2d1f080 size: 80 previous size: 80 (Free) …

e2d1f100 size: 80 previous size: 80 (Allocated) Lfs

e2d1f180 size: 80 previous size: 80 (Allocated) Lfs

e2d1f200 size: 80 previous size: 80 (Allocated) Lfs

e2d1f280 size: 20 previous size: 80 (Free) FSMm

e2d1f2a0 size: 80 previous size: 20 (Allocated) Lfs

e2d1f320 size: 80 previous size: 80 (Allocated) Lfs

e2d1f3a0 size: 80 previous size: 80 (Allocated) Lfs

e2d1f420 size: 20 previous size: 80 (Free) FSMm

e2d1f440 size: 80 previous size: 20 (Allocated) Lfs

e2d1f4c0 size: 80 previous size: 80 (Allocated) Lfs

e2d1f540 size: 80 previous size: 80 (Allocated) Lfs

e2d1f5c0 size: 40 previous size: 80 (Lookaside) Tag0

e2d1f600 size: 80 previous size: 40 (Lookaside) Lfs

e2d1f680 size: 100 previous size: 80 (Free) Lfs

*e2d1f780 size: 40 previous size: 100 (Lookaside) *Tag0

e2d1f7c0 size: 1a0 previous size: 40 (Free) Lfs

e2d1f960 size: 80 previous size: 1a0 (Lookaside) Lfs

e2d1f9e0 size: 100 previous size: 80 (Free) Lfs

e2d1fae0 size: 80 previous size: 100 (Lookaside) Lfs

e2d1fb60 size: 80 previous size: 80 (Free) Lfs

e2d1fbe0 size: 80 previous size: 80 (Lookaside) Lfs

e2d1fc60 size: 120 previous size: 80 (Free) Lfs

e2d1fd80 size: 80 previous size: 120 (Lookaside) Lfs

e2d1fe00 size: 200 previous size: 80 (Free) Lfs

The memory block in question at e2d1f780 is marked as Lookaside instead
of Free. If it had been pushed to a lookaside list when freed it would
explain most of what I am seeing. However, I am not using lookaside
lists.

Also, does anybody know what the size and previous size values refer to
?

Regards

Mark Cook
Computer Associates
Development Leader
tel: +44 (0)1823 624411
fax: +44 (0)1823 624401
mark.cook@ca.com

Mark,

I’m not sure I will be much help here, but I’ll try to share my experience
in debugging strange memory behaviour…

I’m pretty sure the OS is right in saying that you’re trying to free
something that is already freed. The OS is USUALLY right. :wink:

The fact that the memory is “available” is not an indication that it hasn’t
been freed. It’s just an indication that the whole page hasn’t been freed.
The OS will have more than one block of memory in a page, and unless all of
the memory is freed at any given time, the page will still be valid, and
you’ll be able to access the memory that your pointer pointed/points to.
Only memory that is more than one page will automatically be unmapped when
it’s freeed. [1]

It is also very likely that it’s saying that you tried to free the page
twice, whilst it’s actually been re-used for something else. At least, that
how I’ve seen it before… So the fact that it’s Lookaside memory now,
doesn’t mean that it was when you freed it the first time.

Have you tried running with Driver Verifier? If you do, you will find that
all allocations are done into separate pages (obvious causes more memory
waste), so it’s easier to detect badly managed memory in many ways. There’s
also a bunch of extra checks that are made when you allocate and de-allocate
memory to see that you do things right. You’ll probably get a slightly
different error message, but it will come up as “trying to free twice”.

We have built our own tracking mechanisms for memory management, simply
because it causes lots of problems when allocating and freeing memory. This
has detection for double-free’s and leakage, so that if you have allocates
without free’s it will be listed at appropriate times. It’s fairly trivial
to do this, all you need to do is add a small structure at the beginning (or
end) of the allocation with some admin data. We keep a linked list of all
allocations, and add new allocations to the list, then remove them when
freeing them up. We also have an option for “keeping” the freed items around
for a while after being deleted, in which case they are just marked as
deleted and filled with a pattern to recognise if they have been overwritten
before they are actually freed some time later [in our case, we keep X
amount of memory around in the “has been deleted”, and delete the oldest
ones until it’s below the watermark whenever it’s too much].
We use a macro to replace the memory allocation routine with the ‘tracking’
one, something like this:

#if WANT_MEM_TRACKING
#define EngAllocMem(size, flags) EngAllocMemTracked(size, flags, FILE,
LINE)
#define EngFreeMem(ptr) EngFreeMemTracked(ptr, FILE,
LINE)
#endif

The FILE and LINE from EngAllocMem are stored as part of the extra
data that is stored with the memory allocation. This means that you can find
out which piece of code is being responsible for the memory that was
allocated when leaking. The Free call also has FILE and LINE being
passed, so that we can discover who’s been trying to free something twice.

I understand that adding and debugging this sort of tracking mechanism may
not be suitable solution to finding your current problem, but it’s really
worth having when you’re coming up to this sort of problem next time. And
with all likelyhood, you will come up to a similar problem at some time…
Or a memory leak…

My first approach, however, would be to enable Driver Verifier, and make
sure you enable:
Special Pool
Pool tracking

I hope this helps. And I know what a pain it can be to track these things
down, because it’s usually not the obvious candidates.


Mats

-----Original Message-----
From: Cook, Mark [mailto:Mark.Cook@ca.com]
Sent: Friday, April 30, 2004 10:11 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] BAD_POOL_CALLER

Hi all…

I’ve been looking at a memory dump with the following bugcheck information
from a Windows 2000 server, 4 processor machine:

BAD_POOL_CALLER (c2)

The current thread is making a bad pool request. Typically this is at a bad
IRQL level or double freeing the same allocation, etc.

Arguments:

Arg1: 00000007, Attempt to free pool which was already freed

Arg2: 00000b8a, (reserved)

Arg3: e2d1f780, Memory contents of the pool block

Arg4: e2d1f788, Address of the block of pool being deallocated

According to the top of the stack trace, this appears to come from a call to
free off a memory block at e2d1f788 as follows:-

f61a2ce0 8046c84d e2d1f788 00000000 f5d26cc0 nt!ExFreePoolWithTag+0x19b

f61a2cec f5d26cc0 e2d1f788 e1d7f4d0 e1430fb8 nt!ExFreePool+0xb

From the documentation, it appears that this error is an attempt to free off
a memory block that has already been freed. I suspect that the problem that
I am trying to solve is caused by an incorrectly decremented reference count
so this is entirely possible. However, it appears that at the time of
freeing, this memory address is still valid for the following reasons:

Firstly, immediately prior to freeing the memory, I set the first four bytes
to “FREE” to mark it as invalid. I don’t free this memory from anywhere
else and, if this had already been freed, I would have expected this to
generate an access violation. Moreover, the address does appear to be valid:
I have verified this by using the !pte command in WINDBG which yields the
following result:-

!pte e2d1f788

E2D1F788 - PDE at C0300E2C PTE at C038B47C

contains 2C38E963 contains 10295963

pfn 2c38e G-DA–KWV pfn 10295 G-DA-KWV

I can also dereference this address and see the signature that I wrote
immediately prior to the call. Finally, I have converted the virtual address
to a physical address and have dumped this using the !dc command and
compared it with the dump of the virtual address to verify that it is
correctly backed by physical memory.

I am allocating memory using the ExAllocateMemoryWithTag function and the
bugcheck occurs when I attempt to free it with ExFreePool function.

One clue may be the output of the !pool command on this address which is as
follows:-

e2d1f000 size: 80 previous size: 0 (Allocated) Lfs

e2d1f080 size: 80 previous size: 80 (Free) …

e2d1f100 size: 80 previous size: 80 (Allocated) Lfs

e2d1f180 size: 80 previous size: 80 (Allocated) Lfs

e2d1f200 size: 80 previous size: 80 (Allocated) Lfs

e2d1f280 size: 20 previous size: 80 (Free) FSMm

e2d1f2a0 size: 80 previous size: 20 (Allocated) Lfs

e2d1f320 size: 80 previous size: 80 (Allocated) Lfs

e2d1f3a0 size: 80 previous size: 80 (Allocated) Lfs

e2d1f420 size: 20 previous size: 80 (Free) FSMm

e2d1f440 size: 80 previous size: 20 (Allocated) Lfs

e2d1f4c0 size: 80 previous size: 80 (Allocated) Lfs

e2d1f540 size: 80 previous size: 80 (Allocated) Lfs

e2d1f5c0 size: 40 previous size: 80 (Lookaside) Tag0

e2d1f600 size: 80 previous size: 40 (Lookaside) Lfs

e2d1f680 size: 100 previous size: 80 (Free) Lfs

*e2d1f780 size: 40 previous size: 100 (Lookaside) *Tag0

e2d1f7c0 size: 1a0 previous size: 40 (Free) Lfs

e2d1f960 size: 80 previous size: 1a0 (Lookaside) Lfs

e2d1f9e0 size: 100 previous size: 80 (Free) Lfs

e2d1fae0 size: 80 previous size: 100 (Lookaside) Lfs

e2d1fb60 size: 80 previous size: 80 (Free) Lfs

e2d1fbe0 size: 80 previous size: 80 (Lookaside) Lfs

e2d1fc60 size: 120 previous size: 80 (Free) Lfs

e2d1fd80 size: 80 previous size: 120 (Lookaside) Lfs

e2d1fe00 size: 200 previous size: 80 (Free) Lfs

The memory block in question at e2d1f780 is marked as Lookaside instead of
Free. If it had been pushed to a lookaside list when freed it would explain
most of what I am seeing. However, I am not using lookaside lists.

Also, does anybody know what the size and previous size values refer to ?

Regards

Mark Cook
Computer Associates
Development Leader
tel: +44 (0)1823 624411
fax: +44 (0)1823 624401
mark.cook@ca.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@3dlabs.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Hi Mats

Doh! Thanks for the info. It seems obvious now that you’ve explained it
but isn’t that always the case !!

Regards

Mark

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@3Dlabs.com
Sent: 30 April 2004 11:11
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] BAD_POOL_CALLER

Mark,

I’m not sure I will be much help here, but I’ll try to share my
experience in debugging strange memory behaviour…

I’m pretty sure the OS is right in saying that you’re trying to free
something that is already freed. The OS is USUALLY right. :wink:

The fact that the memory is “available” is not an indication that it
hasn’t been freed. It’s just an indication that the whole page hasn’t
been freed. The OS will have more than one block of memory in a page,
and unless all of the memory is freed at any given time, the page will
still be valid, and you’ll be able to access the memory that your
pointer pointed/points to. Only memory that is more than one page will
automatically be unmapped when it’s freeed. [1]

It is also very likely that it’s saying that you tried to free the page
twice, whilst it’s actually been re-used for something else. At least,
that how I’ve seen it before… So the fact that it’s Lookaside memory
now, doesn’t mean that it was when you freed it the first time.

Have you tried running with Driver Verifier? If you do, you will find
that all allocations are done into separate pages (obvious causes more
memory waste), so it’s easier to detect badly managed memory in many
ways. There’s also a bunch of extra checks that are made when you
allocate and de-allocate memory to see that you do things right. You’ll
probably get a slightly different error message, but it will come up as
“trying to free twice”.

We have built our own tracking mechanisms for memory management, simply
because it causes lots of problems when allocating and freeing memory.
This has detection for double-free’s and leakage, so that if you have
allocates without free’s it will be listed at appropriate times. It’s
fairly trivial to do this, all you need to do is add a small structure
at the beginning (or end) of the allocation with some admin data. We
keep a linked list of all allocations, and add new allocations to the
list, then remove them when freeing them up. We also have an option for
“keeping” the freed items around for a while after being deleted, in
which case they are just marked as deleted and filled with a pattern to
recognise if they have been overwritten before they are actually freed
some time later [in our case, we keep X amount of memory around in the
“has been deleted”, and delete the oldest ones until it’s below the
watermark whenever it’s too much].

We use a macro to replace the memory allocation routine with the
‘tracking’ one, something like this:

#if WANT_MEM_TRACKING

#define EngAllocMem(size, flags) EngAllocMemTracked(size, flags,
FILE, LINE)

#define EngFreeMem(ptr) EngFreeMemTracked(ptr, FILE,
LINE)

#endif

The FILE and LINE from EngAllocMem are stored as part of the
extra data that is stored with the memory allocation. This means that
you can find out which piece of code is being responsible for the memory
that was allocated when leaking. The Free call also has FILE and
LINE being passed, so that we can discover who’s been trying to free
something twice.

I understand that adding and debugging this sort of tracking mechanism
may not be suitable solution to finding your current problem, but it’s
really worth having when you’re coming up to this sort of problem next
time. And with all likelyhood, you will come up to a similar problem at
some time… Or a memory leak…

My first approach, however, would be to enable Driver Verifier, and make
sure you enable:

Special Pool

Pool tracking

I hope this helps. And I know what a pain it can be to track these
things down, because it’s usually not the obvious candidates.

Mats

-----Original Message-----
From: Cook, Mark [mailto:Mark.Cook@ca.com]
Sent: Friday, April 30, 2004 10:11 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] BAD_POOL_CALLER

Hi all…

I’ve been looking at a memory dump with the following bugcheck
information from a Windows 2000 server, 4 processor machine:

BAD_POOL_CALLER (c2)

The current thread is making a bad pool request. Typically this is at a
bad IRQL level or double freeing the same allocation, etc.

Arguments:

Arg1: 00000007, Attempt to free pool which was already freed

Arg2: 00000b8a, (reserved)

Arg3: e2d1f780, Memory contents of the pool block

Arg4: e2d1f788, Address of the block of pool being deallocated

According to the top of the stack trace, this appears to come from a
call to free off a memory block at e2d1f788 as follows:-

f61a2ce0 8046c84d e2d1f788 00000000 f5d26cc0 nt!ExFreePoolWithTag+0x19b

f61a2cec f5d26cc0 e2d1f788 e1d7f4d0 e1430fb8 nt!ExFreePool+0xb

From the documentation, it appears that this error is an attempt to free
off a memory block that has already been freed. I suspect that the
problem that I am trying to solve is caused by an incorrectly
decremented reference count so this is entirely possible. However, it
appears that at the time of freeing, this memory address is still valid
for the following reasons:

Firstly, immediately prior to freeing the memory, I set the first four
bytes to “FREE” to mark it as invalid. I don’t free this memory from
anywhere else and, if this had already been freed, I would have expected
this to generate an access violation. Moreover, the address does appear
to be valid: I have verified this by using the !pte command in WINDBG
which yields the following result:-

!pte e2d1f788

E2D1F788 - PDE at C0300E2C PTE at C038B47C

contains 2C38E963 contains 10295963

pfn 2c38e G-DA–KWV pfn 10295 G-DA-KWV

I can also dereference this address and see the signature that I wrote
immediately prior to the call. Finally, I have converted the virtual
address to a physical address and have dumped this using the !dc command
and compared it with the dump of the virtual address to verify that it
is correctly backed by physical memory.

I am allocating memory using the ExAllocateMemoryWithTag function and
the bugcheck occurs when I attempt to free it with ExFreePool function.

One clue may be the output of the !pool command on this address which is
as follows:-

e2d1f000 size: 80 previous size: 0 (Allocated) Lfs

e2d1f080 size: 80 previous size: 80 (Free) …

e2d1f100 size: 80 previous size: 80 (Allocated) Lfs

e2d1f180 size: 80 previous size: 80 (Allocated) Lfs

e2d1f200 size: 80 previous size: 80 (Allocated) Lfs

e2d1f280 size: 20 previous size: 80 (Free) FSMm

e2d1f2a0 size: 80 previous size: 20 (Allocated) Lfs

e2d1f320 size: 80 previous size: 80 (Allocated) Lfs

e2d1f3a0 size: 80 previous size: 80 (Allocated) Lfs

e2d1f420 size: 20 previous size: 80 (Free) FSMm

e2d1f440 size: 80 previous size: 20 (Allocated) Lfs

e2d1f4c0 size: 80 previous size: 80 (Allocated) Lfs

e2d1f540 size: 80 previous size: 80 (Allocated) Lfs

e2d1f5c0 size: 40 previous size: 80 (Lookaside) Tag0

e2d1f600 size: 80 previous size: 40 (Lookaside) Lfs

e2d1f680 size: 100 previous size: 80 (Free) Lfs

*e2d1f780 size: 40 previous size: 100 (Lookaside) *Tag0

e2d1f7c0 size: 1a0 previous size: 40 (Free) Lfs

e2d1f960 size: 80 previous size: 1a0 (Lookaside) Lfs

e2d1f9e0 size: 100 previous size: 80 (Free) Lfs

e2d1fae0 size: 80 previous size: 100 (Lookaside) Lfs

e2d1fb60 size: 80 previous size: 80 (Free) Lfs

e2d1fbe0 size: 80 previous size: 80 (Lookaside) Lfs

e2d1fc60 size: 120 previous size: 80 (Free) Lfs

e2d1fd80 size: 80 previous size: 120 (Lookaside) Lfs

e2d1fe00 size: 200 previous size: 80 (Free) Lfs

The memory block in question at e2d1f780 is marked as Lookaside instead
of Free. If it had been pushed to a lookaside list when freed it would
explain most of what I am seeing. However, I am not using lookaside
lists.

Also, does anybody know what the size and previous size values refer to
?

Regards

Mark Cook
Computer Associates
Development Leader
tel: +44 (0)1823 624411
fax: +44 (0)1823 624401
mark.cook@ca.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@3dlabs.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: Mark.Cook@ca.com
To unsubscribe send a blank email to xxxxx@lists.osr.com