I’m trying to maintain an old driver, and saw some odd coding, which makes we wonder about these behaviors:
If a driver allocates memory using ExAllocatePoolWithTag(), and then uses ExFreePool() to release it, are there any undesired effects from such behavior?
The driver I’m looking at is doing this extensively, but as far as I can tell, it works fine.
This one is just to satisfy my curiosity: ?If a driver allocates memory using ExAllocatePoolWithTag(), and then uses ExFreePoolWithTag() to release it, specifying the wrong tag, are there any undesired effects from such behavior?
In general the tags are associated with the memory but not checked. So
allocating with a tag, then using a different tag or no tag to free
memory is fine. You can change this behavior by or’ing PROTECTED_POOL
to the tag, then the tag is checked at memory freeing to ensure they are
the same. Using PROTECTED_POOL can be a good diagnostic.
“Aspiring Programmer” wrote in message news:xxxxx@ntdev:
> I’m trying to maintain an old driver, and saw some odd coding, which makes we wonder about these behaviors: > > 1. If a driver allocates memory using ExAllocatePoolWithTag(), and then uses ExFreePool() to release it, are there any undesired effects from such behavior? > > The driver I’m looking at is doing this extensively, but as far as I can tell, it works fine. > > 2. This one is just to satisfy my curiosity: If a driver allocates memory using ExAllocatePoolWithTag(), and then uses ExFreePoolWithTag() to release it, specifying the wrong tag, are there any undesired effects from such behavior?
In my opinion ExFreePoolWithTag is mostly useless, unlike
ExAllocatePoolWithTag, which is hugely useful but needs a bigger tag field.
There are many cases where the de-allocator has no clue what the tag might
have been.
Mark Roddy
On Fri, Dec 21, 2012 at 6:31 AM, Aspiring Programmer < xxxxx@yahoo.com> wrote:
I’m trying to maintain an old driver, and saw some odd coding, which makes
we wonder about these behaviors:
If a driver allocates memory using ExAllocatePoolWithTag(), and then
uses ExFreePool() to release it, are there any undesired effects from such
behavior?
The driver I’m looking at is doing this extensively, but as far as I can
tell, it works fine.
This one is just to satisfy my curiosity: If a driver allocates
memory using ExAllocatePoolWithTag(), and then uses ExFreePoolWithTag() to
release it, specifying the wrong tag, are there any undesired effects from
such behavior?
In my opinion ExFreePoolWithTag is mostly useless, unlike
ExAllocatePoolWithTag, which is hugely useful but needs a bigger tag
field. There are many cases where the de-allocator has no clue what the
tag might have been.
You can use ExFreePool in those cases. ExFreePoolWithTag is useful when
you are deallocating a buffer which you allocated (and therefore know
the tag).
Keeping track of 64 different tags (which the code currently uses) would be very tedious.
I now appreciate the original developer’s decision to use ExFreePool() much more.
It looks like there’s not much lost, because the tags are still completely available in Poolmon when checking for memory leaks.
From: George M. Garner Jr. To: Windows System Software Devs Interest List Sent: Friday, December 21, 2012 8:59 AM Subject: Re:[ntdev] ExFreePoolWithTag etc.
On 12/21/2012 11:46 AM, Mark Roddy wrote: > In my opinion ExFreePoolWithTag is mostly useless, unlike > ExAllocatePoolWithTag, which is hugely useful but needs a bigger tag > field. There are many cases where the de-allocator has no clue what the > tag might have been. >
You can use ExFreePool in those cases.? ExFreePoolWithTag is useful when you are deallocating a buffer which you allocated (and therefore know the tag).
If you can’t keep track of the tags, how do you knowcwhat you are freeing?
Sounds like the original developer thought writing correct code was a Bad
Idea. You should not fall into the same trap.
joe
Keeping track of 64 different tags (which the code currently uses) would
be very tedious.
I now appreciate the original developer’s decision to use ExFreePool()
much more.
It looks like there’s not much lost, because the tags are still completely
available in Poolmon when checking for memory leaks.
From: George M. Garner Jr. > To: Windows System Software Devs Interest List > Sent: Friday, December 21, 2012 8:59 AM > Subject: Re:[ntdev] ExFreePoolWithTag etc. > > On 12/21/2012 11:46 AM, Mark Roddy wrote: >> In my opinion ExFreePoolWithTag is mostly useless, unlike >> ExAllocatePoolWithTag, which is hugely useful but needs a bigger tag >> field. There are many cases where the de-allocator has no clue what the >> tag might have been. >> > > You can use ExFreePool in those cases. ExFreePoolWithTag is useful when > you are deallocating a buffer which you allocated (and therefore know the > tag). > > > — > NTDEV is sponsored by OSR > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at > http://www.osronline.com/page.cfm?name=ListServer > — > NTDEV is sponsored by OSR > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at > http://www.osronline.com/page.cfm?name=ListServer
Several times when I received a driver (either Windows or Linux to base
a port on), I have put in the PROTECT_POOL on all the allocations I
could and converted things to ExFreePoolWithTag. In almost every case,
I get the bug check and once I fix it the vendor say “Oh yeah we had a
random crash once in a while”. It is amazing how many drivers don’t do
a good job of tracking pointers and free a pointer to something other
than what they think they are.
> If you can’t keep track of the tags, how do you knowcwhat you are freeing? > > Sounds like the original developer thought writing correct code was a Bad > Idea. You should not fall into the same trap. > joe > > > Keeping track of 64 different tags (which the code currently uses) would > > be very tedious. > > > > I now appreciate the original developer’s decision to use ExFreePool() > > much more. > > > > It looks like there’s not much lost, because the tags are still completely > > available in Poolmon when checking for memory leaks. > > > > > > ________________________________ > > From: George M. Garner Jr. > > To: Windows System Software Devs Interest List > > Sent: Friday, December 21, 2012 8:59 AM > > Subject: Re:[ntdev] ExFreePoolWithTag etc. > > > > On 12/21/2012 11:46 AM, Mark Roddy wrote: > >> In my opinion ExFreePoolWithTag is mostly useless, unlike > >> ExAllocatePoolWithTag, which is hugely useful but needs a bigger tag > >> field. There are many cases where the de-allocator has no clue what the > >> tag might have been. > >> > > > > You can use ExFreePool in those cases. ExFreePoolWithTag is useful when > > you are deallocating a buffer which you allocated (and therefore know the > > tag). > > > > > > — > > NTDEV is sponsored by OSR > > > > For our schedule of WDF, WDM, debugging and other seminars visit: > > http://www.osr.com/seminars > > > > To unsubscribe, visit the List Server section of OSR Online at > > http://www.osronline.com/page.cfm?name=ListServer > > — > > NTDEV is sponsored by OSR > > > > For our schedule of WDF, WDM, debugging and other seminars visit: > > http://www.osr.com/seminars > > > > To unsubscribe, visit the List Server section of OSR Online at > > http://www.osronline.com/page.cfm?name=ListServer
XP was the last Windows release where PROTECTED_POOL actually did anything. All subsequent releases ignore this bit, and in the Win8 WDK it is officially deprecated:
//
// PROTECTED_POOL is deprecated, do not use.
//
#define PROTECTED_POOL 0x0
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn
Sent: Friday, December 21, 2012 2:32 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] ExFreePoolWithTag etc.
Several times when I received a driver (either Windows or Linux to base a port on), I have put in the PROTECT_POOL on all the allocations I could and converted things to ExFreePoolWithTag. In almost every case, I get the bug check and once I fix it the vendor say “Oh yeah we had a random crash once in a while”. It is amazing how many drivers don’t do a good job of tracking pointers and free a pointer to something other than what they think they are.
> If you can’t keep track of the tags, how do you knowcwhat you are freeing? > > Sounds like the original developer thought writing correct code was a > Bad Idea. You should not fall into the same trap. > joe > > > Keeping track of 64 different tags (which the code currently uses) > > would be very tedious. > > > > I now appreciate the original developer’s decision to use > > ExFreePool() much more. > > > > It looks like there’s not much lost, because the tags are still > > completely available in Poolmon when checking for memory leaks. > > > > > > ________________________________ > > From: George M. Garner Jr. > > To: Windows System Software Devs Interest List > > Sent: Friday, December 21, 2012 8:59 AM > > Subject: Re:[ntdev] ExFreePoolWithTag etc. > > > > On 12/21/2012 11:46 AM, Mark Roddy wrote: > >> In my opinion ExFreePoolWithTag is mostly useless, unlike > >> ExAllocatePoolWithTag, which is hugely useful but needs a bigger > >> tag field. There are many cases where the de-allocator has no clue > >> what the tag might have been. > >> > > > > You can use ExFreePool in those cases.? ExFreePoolWithTag is useful > > when you are deallocating a buffer which you allocated (and > > therefore know the tag). > > > > > > — > > NTDEV is sponsored by OSR > > > > For our schedule of WDF, WDM, debugging and other seminars visit: > > http://www.osr.com/seminars > > > > To unsubscribe, visit the List Server section of OSR Online at > > http://www.osronline.com/page.cfm?name=ListServer > > — > > NTDEV is sponsored by OSR > > > > For our schedule of WDF, WDM, debugging and other seminars visit: > > http://www.osr.com/seminars > > > > To unsubscribe, visit the List Server section of OSR Online at > > http://www.osronline.com/page.cfm?name=ListServer
Since the bit was first documented in Windows Server 2003 are you
sure XP was the last OS? Sorry to see it go, but then like a useful
checked build, and CUV Microsoft keeps throwing out things that help
produce good code in drivers. Maybe reversing the trend can be a
resolution for 2013.
“Pavel Lebedynskiy” wrote in message news:xxxxx@ntdev:
> XP was the last Windows release where PROTECTED_POOL actually did anything. All subsequent releases ignore this bit, and in the Win8 WDK it is officially deprecated: > > // > // PROTECTED_POOL is deprecated, do not use. > // > > #define PROTECTED_POOL 0x0 > > -----Original Message----- > From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn > Sent: Friday, December 21, 2012 2:32 PM > To: Windows System Software Devs Interest List > Subject: Re:[ntdev] ExFreePoolWithTag etc. > > Several times when I received a driver (either Windows or Linux to base a port on), I have put in the PROTECT_POOL on all the allocations I could and converted things to ExFreePoolWithTag. In almost every case, I get the bug check and once I fix it the vendor say “Oh yeah we had a random crash once in a while”. It is amazing how many drivers don’t do a good job of tracking pointers and free a pointer to something other than what they think they are. > > > Don Burn > Windows Filesystem and Driver Consulting > Website: http://www.windrvr.com > Blog: http://msmvps.com/blogs/WinDrvr > > > > > “xxxxx@flounder.com” wrote in message > news:xxxxx@ntdev: > > > If you can’t keep track of the tags, how do you knowcwhat you are freeing? > > > > Sounds like the original developer thought writing correct code was a > > Bad Idea. You should not fall into the same trap. > > joe > > > > > Keeping track of 64 different tags (which the code currently uses) > > > would be very tedious. > > > > > > I now appreciate the original developer’s decision to use > > > ExFreePool() much more. > > > > > > It looks like there’s not much lost, because the tags are still > > > completely available in Poolmon when checking for memory leaks. > > > > > > > > > ________________________________ > > > From: George M. Garner Jr. > > > To: Windows System Software Devs Interest List > > > Sent: Friday, December 21, 2012 8:59 AM > > > Subject: Re:[ntdev] ExFreePoolWithTag etc. > > > > > > On 12/21/2012 11:46 AM, Mark Roddy wrote: > > >> In my opinion ExFreePoolWithTag is mostly useless, unlike > > >> ExAllocatePoolWithTag, which is hugely useful but needs a bigger > > >> tag field. There are many cases where the de-allocator has no clue > > >> what the tag might have been. > > >> > > > > > > You can use ExFreePool in those cases. ExFreePoolWithTag is useful > > > when you are deallocating a buffer which you allocated (and > > > therefore know the tag). > > > > > > > > > — > > > NTDEV is sponsored by OSR > > > > > > For our schedule of WDF, WDM, debugging and other seminars visit: > > > http://www.osr.com/seminars > > > > > > To unsubscribe, visit the List Server section of OSR Online at > > > http://www.osronline.com/page.cfm?name=ListServer > > > — > > > NTDEV is sponsored by OSR > > > > > > For our schedule of WDF, WDM, debugging and other seminars visit: > > > http://www.osr.com/seminars > > > > > > To unsubscribe, visit the List Server section of OSR Online at > > > http://www.osronline.com/page.cfm?name=ListServer > > > — > NTDEV is sponsored by OSR > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
For a while PROTECTED_POOL was used in various OS components but the number of actionable bugs it found for us was either zero or a number very close to that. At the same time, verifier/special pool found thousands of bugs in 3rd party drivers as well as the OS itself, all without having to modify any existing code. If you look at it from this point of view it may not seem very surprising that PROTECTED_POOL didn’t survive while verifier is still around and continues to evolve.
What’s the problem with the checked build by the way? In my experience checked win8 builds are at least as usable as checked win7/vista/ws03/xp, and definitely more usable than pre-XP releases.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn
Sent: Friday, December 21, 2012 3:05 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Re:ExFreePoolWithTag etc.
Pavel,
Since the bit was first documented in Windows Server 2003 are you sure XP was the last OS? Sorry to see it go, but then like a useful checked build, and CUV Microsoft keeps throwing out things that help
produce good code in drivers. Maybe reversing the trend can be a
resolution for 2013.
Something is wrong here since I definitely have gotten PROTECTED_POOL
triggers on Win2003. In fact there was a discussion about
PROTECTED_POOL during the WDF development where a number of folks from
Microsoft were surprised how many people outside of Redmond used it.
Checked build has been broken since Vista. Before that the logging and
the ASSERTS were useful tools for developers. With Vista we lost all of
that, including the ASSERT messages in many cases, and while a little
came back thanks to people complaining at the last DDC. The statement
by Mr Sinofsky after Microsoft realized they had destroyed the
usefulness of the Checked build for everyone who had the stripped PDB’s
was “We just found out about this, we will fix the basics ASAP, and then
work on restoring the rest in future revisions”. Only problem is the
fix is the basics (and pretty basic at that) and nothing else ever came
back.
Basically as far as runtime checking capabilities, things peaked around
release of Server 2003 and has slowly gotten worse since then. Yes, WDF
has improved some things, but it is disturbing that tools that helped
improve driver quality are being removed even if new one for other areas
are being added. This is one of the reasons I do not build with the
Win8 WDK normally, since I do use the Checked Build for XP/2003, I do
use PROTECTED_POOL, and I use some runtime checking (with hand crafted
runtimes) that Microsoft does not enable for the kernel.
“Pavel Lebedynskiy” wrote in message news:xxxxx@ntdev:
> Yes, PROTECTED_POOL is ignored on WS03. > > For a while PROTECTED_POOL was used in various OS components but the number of actionable bugs it found for us was either zero or a number very close to that. At the same time, verifier/special pool found thousands of bugs in 3rd party drivers as well as the OS itself, all without having to modify any existing code. If you look at it from this point of view it may not seem very surprising that PROTECTED_POOL didn’t survive while verifier is still around and continues to evolve. > > What’s the problem with the checked build by the way? In my experience checked win8 builds are at least as usable as checked win7/vista/ws03/xp, and definitely more usable than pre-XP releases. > > -----Original Message----- > From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn > Sent: Friday, December 21, 2012 3:05 PM > To: Windows System Software Devs Interest List > Subject: RE:[ntdev] Re:ExFreePoolWithTag etc. > > Pavel, > > Since the bit was first documented in Windows Server 2003 are you sure XP was the last OS? Sorry to see it go, but then like a useful checked build, and CUV Microsoft keeps throwing out things that help > produce good code in drivers. Maybe reversing the trend can be a > resolution for 2013.
The 0x80000000 is exactly the PROTECTED_POOL, just discussed…
– pa
“Maxim S. Shatskih” wrote in message news:xxxxx@ntdev… >>1. If a driver allocates memory using ExAllocatePoolWithTag(), and then >>uses ExFreePool() to >>release it, are there any undesired effects from such behavior? > > This is the normal and usual way. > >>and then uses ExFreePoolWithTag() to release it, specifying the wrong tag, >>are there any undesired >>effects from such behavior? > > Should crash. > > If the tag has bit 0x8000000 set, then ExFreePoolWithTag must be used to > free it. Usual ExFreePool and wrong tag will cause a crash. > > – > Maxim S. Shatskih > Windows DDK MVP > xxxxx@storagecraft.com > http://www.storagecraft.com > >
The ASSERT messages were returned in Win7 (maybe Vista, who cares about Vista… that’s just a bad memory as far as I’m concerned). They were also missing, unintentionally, from the released builds of Win8 symbols on the public symbol server. SNoone (here at OSR) reported this issue to a good friend of ours at Microsoft, who was nice enough to try to get this fixed for Win8 DURING HIS VACATION. If it is not already fixed, the right person knows about it and it will be fixed soon.
Personally, MY only problem with the checked build (aside from the original issue with ASSERT messages) has been getting my clients and students to USE it. If I hear one more student/client say “The checked build sucks. I tried to run my driver on the checked build once, but it crashed!!”… I’m going to scream.
Having said that, Windows Driver Verifier just gets better and better with each release of Windows.
I used to get the same reaction about using the debug MFC and CRT: “Those
debug versions are full of bugs! Every time I run with it, I get ASSERT
errors. I never get any in the release build!” Some people are
unteachable.
Then there’s the people who say “I never use precompiled headers. I used
them once, and got compilation errors I didn’t understand…” and
complain that their 100,000-line build takes 40 minutes (500 source
files). I was very annoyed when my full-build time doubled with one of
the VS releases (132,000 lines of code in 600+ source files). It went
from 25 seconds to 55 seconds.
Fundamental misunderstandings of technology is not limited to kernel mode
programmers.
joe
The ASSERT messages were returned in Win7 (maybe Vista, who cares about
Vista… that’s just a bad memory as far as I’m concerned). They were
also missing, unintentionally, from the released builds of Win8 symbols on
the public symbol server. SNoone (here at OSR) reported this issue to a
good friend of ours at Microsoft, who was nice enough to try to get this
fixed for Win8 DURING HIS VACATION. If it is not already fixed, the right
person knows about it and it will be fixed soon.
Personally, MY only problem with the checked build (aside from the
original issue with ASSERT messages) has been getting my clients and
students to USE it. If I hear one more student/client say “The checked
build sucks. I tried to run my driver on the checked build once, but it
crashed!!”… I’m going to scream.
Having said that, Windows Driver Verifier just gets better and better with
each release of Windows.
I looked this up in source control logs and you’re right, support for PROTECTED_POOL was still there in WS03 RTM. It got removed in SP1 (which was more like a point release, with new features like x64 support). Apparently part of the reason for removing it was preserving compatibility with existing drivers that were using non-ASCII pool tags.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn
Sent: Saturday, December 22, 2012 4:19 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] RE:Re:ExFreePoolWithTag etc.
Something is wrong here since I definitely have gotten PROTECTED_POOL triggers on Win2003. In fact there was a discussion about PROTECTED_POOL during the WDF development where a number of folks from Microsoft were surprised how many people outside of Redmond used it.
>If I hear one more student/client say “The checked build sucks. I tried to run
my driver on the checked build once, but it crashed!!”… I’m going to scream.
On the other hand, it’s sometimes the inbox apps that crash or assert in the checked build. Especially if it’s pre-release.
Thanks for checking. I can certainly see the problem with Unicode
tags. I just wish Microsoft would have found a way to provide the
functionality of PROTECTED_POOL, since to me it is was a useful
debugging tool and we have lost too many of those over the years.
“Pavel Lebedynskiy” wrote in message news:xxxxx@ntdev:
> I looked this up in source control logs and you’re right, support for PROTECTED_POOL was still there in WS03 RTM. It got removed in SP1 (which was more like a point release, with new features like x64 support). Apparently part of the reason for removing it was preserving compatibility with existing drivers that were using non-ASCII pool tags. > > -----Original Message----- > From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn > Sent: Saturday, December 22, 2012 4:19 AM > To: Windows System Software Devs Interest List > Subject: RE:[ntdev] RE:Re:ExFreePoolWithTag etc. > > Something is wrong here since I definitely have gotten PROTECTED_POOL triggers on Win2003. In fact there was a discussion about PROTECTED_POOL during the WDF development where a number of folks from Microsoft were surprised how many people outside of Redmond used it.
Btw the issue wasn’t Unicode tags, it was tags that were not ascii. For instance you had drivers that declared tags like this
#define TAG “abcd”
Which means the tag was a pointer value, not 4 chars as a ulong. And as a kernel pointer, it had the high bit set
d
From: Don Burnmailto:xxxxx Sent: ?12/?25/?2012 4:30 AM To: Windows System Software Devs Interest Listmailto:xxxxx Subject: RE:[ntdev] RE:RE:Re:ExFreePoolWithTag etc.
Pavel,
Thanks for checking. I can certainly see the problem with Unicode tags. I just wish Microsoft would have found a way to provide the functionality of PROTECTED_POOL, since to me it is was a useful debugging tool and we have lost too many of those over the years.
“Pavel Lebedynskiy” wrote in message news:xxxxx@ntdev:
> I looked this up in source control logs and you’re right, support for PROTECTED_POOL was still there in WS03 RTM. It got removed in SP1 (which was more like a point release, with new features like x64 support). Apparently part of the reason for removing it was preserving compatibility with existing drivers that were using non-ASCII pool tags. > > -----Original Message----- > From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn > Sent: Saturday, December 22, 2012 4:19 AM > To: Windows System Software Devs Interest List > Subject: RE:[ntdev] RE:Re:ExFreePoolWithTag etc. > > Something is wrong here since I definitely have gotten PROTECTED_POOL triggers on Win2003. In fact there was a discussion about PROTECTED_POOL during the WDF development where a number of folks from Microsoft were surprised how many people outside of Redmond used it.
Pre-compiled headers are evil and can easily hide bugs that full compilation might uncover. That most people are unreachable gets no argument from me
wrote in message news:xxxxx@ntdev…
I used to get the same reaction about using the debug MFC and CRT: “Those
debug versions are full of bugs! Every time I run with it, I get ASSERT
errors. I never get any in the release build!” Some people are
unteachable.
Then there’s the people who say “I never use precompiled headers. I used
them once, and got compilation errors I didn’t understand…” and
complain that their 100,000-line build takes 40 minutes (500 source
files). I was very annoyed when my full-build time doubled with one of
the VS releases (132,000 lines of code in 600+ source files). It went
from 25 seconds to 55 seconds.
Fundamental misunderstandings of technology is not limited to kernel mode
programmers.
joe
The ASSERT messages were returned in Win7 (maybe Vista, who cares about
Vista… that’s just a bad memory as far as I’m concerned). They were
also missing, unintentionally, from the released builds of Win8 symbols on
the public symbol server. SNoone (here at OSR) reported this issue to a
good friend of ours at Microsoft, who was nice enough to try to get this
fixed for Win8 DURING HIS VACATION. If it is not already fixed, the right
person knows about it and it will be fixed soon.
Personally, MY only problem with the checked build (aside from the
original issue with ASSERT messages) has been getting my clients and
students to USE it. If I hear one more student/client say “The checked
build sucks. I tried to run my driver on the checked build once, but it
crashed!!”… I’m going to scream.
Having said that, Windows Driver Verifier just gets better and better with
each release of Windows.