For an alternate approach you can try adding the following class as a
base class for the objects that you want to allocate using new or delete:
template<pool_type _pool ulong _tag> > class NewAllocatorImpl > {…}; > > Place this code within the namespace of your choice.
This works, but you need to be very aware of the problems involved in using template classes in a kernel driver. If any part of your code marked “paged”, you have an issue, because the compiler doesn’t specify where it puts compiler-generated code like this.
There is an algorithm that works empirically, but it’s not stated in any Microsoft document.
– Tim Roberts, xxxxx@probo.com Providenza & Boekelheide, Inc.</pool_type>
All true, but in practice, as long as you do not specifically mark anything
as paged, you’ll be fine, imo.
There are a few ways (that I know of) to cause compiler generated code to
end up in a paged section, but none of them are possible without using
segment pragma’s.
Just do not page anything.
mm
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Thursday, May 19, 2011 2:25 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] C++ support lib for kernel?
Rossetoecioccolato wrote:
For an alternate approach you can try adding the following class as a
base class for the objects that you want to allocate using new or delete:
template<pool_type _pool ulong _tag> > class NewAllocatorImpl > {…}; > > Place this code within the namespace of your choice.
This works, but you need to be very aware of the problems involved in using template classes in a kernel driver. If any part of your code marked “paged”, you have an issue, because the compiler doesn’t specify where it puts compiler-generated code like this.
There is an algorithm that works empirically, but it’s not stated in any Microsoft document.
– Tim Roberts, xxxxx@probo.com Providenza & Boekelheide, Inc.
I get the linker error when I have the #pragma comments, but it links fine without them.
Actually, including htscpp.h without including the CPP files at all also works just fine. I am guessing since it
links without atexit and Co. that I don’t need anything more than new/delete?
Rossetoecioccolato wrote:
On 5/19/2011 2:20 PM, Dejan Maksimovic wrote:
>
> Why is this required, BTW?
>
>
There is an algorithm that works empirically, but it’s not stated in any
Microsoft document.
Thanks Tim for the warning which is important to understand. The
algorithm does work with current MS compilers. Rather than forgo
pageable sections my solution is to decompile released drivers and
manually verify the location of non-pageable functions. The approach
which you suggest may be better for the OP since the undocumented
approach requires substantial experience with C++ driver development
using Microsoft compilers.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Rossetoecioccolato
Sent: Thursday, May 19, 2011 2:52 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] C++ support lib for kernel?
Actually, truth is that the .pragma comments are needed for static
variable initialization support.
Why the devil are people zero’ing memory in new??? That’s stupid and non-standard behavior that wastes cycles a lot of the time and could be dangerous. If you need memory zeroed then why not do it the C++ way that documents it as such in the code:
DATA *x = new DATA; // uninitialized
DATA *y = new DATA(); // initialized to 0–compiler will insert a memset for you
Why the devil are people zero’ing memory in new??? That’s stupid and non-standard behavior that wastes cycles a lot of the time and could be dangerous.
I’ll grant that it’s not standard, but the time is trivial, and I
challenge you to concoct a case where it is dangerous.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
All it takes is when someone uses the code with a different version of new that doesn’t do that. It could be as innocent as someone cutting and pasting a snippet of code from a sample or a forum particularly since the Microsoft header file is stupidly zeroing everything that people will look at and say gee, i don’t need to do that anymore in my code since it has become redundant. Then someone else gets this snippet and their standard new doesn’t zero and so garbage is being passed around in their code rather than zeroed values. It might even work–sometimes.
Personally, for something like this, I think that defining your own is a
good idea. That way you know what the default pool type is, et. C.,you know
what the overloads are, et. C.
From that point of view, that’s what the kcom file is doing - it was
presumably never intended to be of ‘general’ use.
I hear what you’re saying about the official way of doing it, but that’s a
subtle, very easily overlooked syntactic difference, far more likely to
cause problems than the day arriving where somehow another kernel mode new
gets in your code without your knowing it, imo.
If you’re trying to use c++ in the kernel and you don’t specify everything
explicitly, you’re asking for trouble, IMO.
Mm
On May 19, 2011 9:12 PM, wrote: > All it takes is when someone uses the code with a different version of new that doesn’t do that. It could be as innocent as someone cutting and pasting a snippet of code from a sample or a forum particularly since the Microsoft header file is stupidly zeroing everything that people will look at and say gee, i don’t need to do that anymore in my code since it has become redundant. Then someone else gets this snippet and their standard new doesn’t zero and so garbage is being passed around in their code rather than zeroed values. It might even work–sometimes. > > > — > NTDEV is sponsored by OSR > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Maybe then it should set it all to 0xCC and force some errors.
Cheers,
Dave Cattley
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Thursday, May 19, 2011 9:12 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] C++ support lib for kernel?
All it takes is when someone uses the code with a different version of new
that doesn’t do that. It could be as innocent as someone cutting and pasting
a snippet of code from a sample or a forum particularly since the Microsoft
header file is stupidly zeroing everything that people will look at and say
gee, i don’t need to do that anymore in my code since it has become
redundant. Then someone else gets this snippet and their standard new
doesn’t zero and so garbage is being passed around in their code rather than
zeroed values. It might even work–sometimes.
But this only cause a BP instruction, if by chance it gets executed
IMO, the protocol should be either consistently set to zero or leave it alone, and is the concern of this note!
-pro
On May 19, 2011, at 7:28 PM, David R. Cattley wrote:
Maybe then it should set it all to 0xCC and force some errors.
Cheers,
Dave Cattley
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Thursday, May 19, 2011 9:12 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] C++ support lib for kernel?
All it takes is when someone uses the code with a different version of new
that doesn’t do that. It could be as innocent as someone cutting and pasting
a snippet of code from a sample or a forum particularly since the Microsoft
header file is stupidly zeroing everything that people will look at and say
gee, i don’t need to do that anymore in my code since it has become
redundant. Then someone else gets this snippet and their standard new
doesn’t zero and so garbage is being passed around in their code rather than
zeroed values. It might even work–sometimes.
Collectively, a couple news + deletes will run you like 20 lines. Write it
yourself, anyway you want.
Mm
On May 19, 2011 10:47 PM, “Prokash Sinha” wrote: > But this only cause a BP instruction, if by chance it gets executed > > IMO, the protocol should be either consistently set to zero or leave it alone, and is the concern of this note! > > -pro > > On May 19, 2011, at 7:28 PM, David R. Cattley wrote: > >> Maybe then it should set it all to 0xCC and force some errors. >> >> Cheers, >> Dave Cattley >> >> -----Original Message----- >> From: xxxxx@lists.osr.com >> [mailto:xxxxx@lists.osr.com] On Behalf Of >> xxxxx@gmail.com >> Sent: Thursday, May 19, 2011 9:12 PM >> To: Windows System Software Devs Interest List >> Subject: RE:[ntdev] C++ support lib for kernel? >> >> All it takes is when someone uses the code with a different version of new >> that doesn’t do that. It could be as innocent as someone cutting and pasting >> a snippet of code from a sample or a forum particularly since the Microsoft >> header file is stupidly zeroing everything that people will look at and say >> gee, i don’t need to do that anymore in my code since it has become >> redundant. Then someone else gets this snippet and their standard new >> doesn’t zero and so garbage is being passed around in their code rather than >> zeroed values. It might even work–sometimes. >> >> >> — >> NTDEV is sponsored by OSR >> >> For our schedule of WDF, WDM, debugging and other seminars visit: >> http://www.osr.com/seminars >> >> To unsubscribe, visit the List Server section of OSR Online at >> http://www.osronline.com/page.cfm?name=ListServer >> >> >> — >> NTDEV is sponsored by OSR >> >> For our schedule of WDF, WDM, debugging and other seminars visit: >> http://www.osr.com/seminars >> >> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer > > > — > NTDEV is sponsored by OSR > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
xxxxx@gmail.com wrote:
> Why the devil are people zero’ing memory in new??? That’s stupid and non-standard behavior that wastes cycles a lot of the time and could be dangerous.
I’ll grant that it’s not standard, but the time is trivial, and I
challenge you to concoct a case where it is dangerous.
We know that underlying allocator should not even try to initialize to anything. It is a traffic area!
We also know that if we craft our own new and delete, and localized to our module, it is fair enough to say that we can have consistency ( as you mentioned 20 or so lines ).
On the other hand, if we are succumbed to some header file that is bit out of the norm ( or for that matter not knowing what to expect …), then the problem comes.
-pro
On May 19, 2011, at 8:04 PM, Martin O’Brien wrote:
Collectively, a couple news + deletes will run you like 20 lines. Write it yourself, anyway you want.
Mm
On May 19, 2011 10:47 PM, “Prokash Sinha” wrote: > > But this only cause a BP instruction, if by chance it gets executed > > > > IMO, the protocol should be either consistently set to zero or leave it alone, and is the concern of this note! > > > > -pro > > > > On May 19, 2011, at 7:28 PM, David R. Cattley wrote: > > > >> Maybe then it should set it all to 0xCC and force some errors. > >> > >> Cheers, > >> Dave Cattley > >> > >> -----Original Message----- > >> From: xxxxx@lists.osr.com > >> [mailto:xxxxx@lists.osr.com] On Behalf Of > >> xxxxx@gmail.com > >> Sent: Thursday, May 19, 2011 9:12 PM > >> To: Windows System Software Devs Interest List > >> Subject: RE:[ntdev] C++ support lib for kernel? > >> > >> All it takes is when someone uses the code with a different version of new > >> that doesn’t do that. It could be as innocent as someone cutting and pasting > >> a snippet of code from a sample or a forum particularly since the Microsoft > >> header file is stupidly zeroing everything that people will look at and say > >> gee, i don’t need to do that anymore in my code since it has become > >> redundant. Then someone else gets this snippet and their standard new > >> doesn’t zero and so garbage is being passed around in their code rather than > >> zeroed values. It might even work–sometimes. > >> > >> > >> — > >> NTDEV is sponsored by OSR > >> > >> For our schedule of WDF, WDM, debugging and other seminars visit: > >> http://www.osr.com/seminars > >> > >> To unsubscribe, visit the List Server section of OSR Online at > >> http://www.osronline.com/page.cfm?name=ListServer > >> > >> > >> — > >> NTDEV is sponsored by OSR > >> > >> For our schedule of WDF, WDM, debugging and other seminars visit: > >> http://www.osr.com/seminars > >> > >> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer > > > > > > — > > NTDEV is sponsored by OSR > > > > For our schedule of WDF, WDM, debugging and other seminars visit: > > http://www.osr.com/seminars > > > > To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer > — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
> but that’s a subtle, very easily overlooked syntactic difference,
far more likely to cause problems
Well yes if Microsoft had their way then I presume it would be a long verbose keyword in camel case like AndZeroAllBytesOfTheDataWhileYouAreAtIt that no one would miss, but is totally unnecessary and stupid. The compact syntax DATA() isn’t really much different than in C where you can do this to zero data:
DATA x = {};
Saying it’s subtle and overlooked is a weak argument. I consider it short, sweet, clean and a terrific feature of C++. The syntax DATA() during allocation is to make it look like a function call is occurring so you know something extra is going on beyond allocation so it is nearly intuitive too. It’s certainly going to be less error prone and cleaner code than following allocations with a kludgey RtlZeroMemory specifying an address and length both of which are easy to get wrong and won’t show up as problems until later. I’ve sometimes get tasked to figure out why peoples drivers don’t work after they give up. Getting RtlZeroMemory wrong was one of those cases and I am sure many here have screwed it up one time or another.
One more time - write your own. No ambiguity. No surprises. Does just
what you want. In 20 lines.
If you want to go with new v. New(), fantastic. Great way to do it.
But the authors of that file apparently wanted to zero everything, and they
never intended it to be used otherwise.
Cleaner, yes. Less error prone, I’m not sure. You may consider it a weak
argument, but that’s kind of my point - as a generic feature, it will be
used by weak programmers too, and plenty of them won’t know the difference
between those two syntaxes. They will however know how to zero memory, and
if they wrote their own, they’ll know what they did.
As far as the errors nit showing up until later, that’s every bit as true if
your method, if not more so. Driver verifier can catch overruns due to
misusing RtlZeroMemory, but not cases of usingnon-zeroed memory that one
accidentally acquired by omitting the trailing ().
In the genetic case, the Kernel is not the place to leverage the more subtle
features of c++ - like, say, operator overloading- simply because one is not
expecting to see that sort of thing there. Take a look around user mode
code - that syntax is not especially common, imo. If it works for you,
fantastic, but insisting it’s better for everybody because it’s part of the
standard or elegant or because you think those that choose otherwise are
enablers of weak and stupid ideas, is done so at the expense of ignoring the
context in which it is used, the habits of those who might use it and that’s
what makes library design in general difficult to do well, and this one in
particular is all about lowest common denominator safety.
I forgot - there are plenty of excellent driver Devs who do not partake of
C++ normally.
Mm
On May 20, 2011 12:11 AM, “Martin O’Brien” < xxxxx@gmail.com> wrote:
One more time - write your own. No ambiguity. No surprises. Does just
what you want. In 20 lines.
If you want to go with new v. New(), fantastic. Great way to do it.
But the authors of that file apparently wanted to zero everything, and
they
never intended it to be used otherwise.
Cleaner, yes. Less error prone, I’m not sure. You may consider it a weak
argument, but that’s kind of my point - as a generic feature, it will be
used by weak programmers too, and plenty of them won’t know the difference
between those two syntaxes. They will however know how to zero memory, and
if they wrote their own, they’ll know what they did.
As far as the errors nit showing up until later, that’s every bit as true
if
your method, if not more so. Driver verifier can catch overruns due to
misusing RtlZeroMemory, but not cases of usingnon-zeroed memory that one
accidentally acquired by omitting the trailing ().
In the genetic case, the Kernel is not the place to leverage the more
subtle
features of c++ - like, say, operator overloading- simply because one is
not
expecting to see that sort of thing there. Take a look around user mode
code - that syntax is not especially common, imo. If it works for you,
fantastic, but insisting it’s better for everybody because it’s part of
the
standard or elegant or because you think those that choose otherwise are
enablers of weak and stupid ideas, is done so at the expense of ignoring
the
context in which it is used, the habits of those who might use it and
that’s
what makes library design in general difficult to do well, and this one in
particular is all about lowest common denominator safety.
> the Kernel is not the place to leverage the more subtle features of c++ - like, say, operator overloading
On the contrary, operator overloading is fantastic in the kernel and Microsoft has even gone to the trouble of defining some operators for us. Here is an example. I am eager to to hear how this is evil and the C way is superior:
GUID a, b;
// in C we are forced to scour the docs looking for this ridiculous function
if (IsEqualGUID(&a, &b))
// in C++ it just happens naturally; a round of applause for operator ==
if (a == b)
write your own. No ambiguity. No surprises. Does just what you want
No ambiguity? No surprises? Could not be further from the truth and it is pretty astonishing to see someone advocate new or delete should do what you want. So why aren’t we all overriding ExAllocatePool with our own version that zeros every single allocation out too? And then when Microsoft decides to include a basic new & delete in the next ntddk.h what are we to do with our special version, comment out Microsoft’s I guess? Also keep in mind there is more of a need now to base off or even share source code between apps, drivers, and other operating systems. Really there are countless reasons that writing your code to try to elevate undefined behavior to defined behavior and rely on it could be a time bomb that will cause a substantial architectural failure on something down the line.
Save for the C++ hecklers, I am pretty sure everyone else is capable of picking out DATA() init syntax very naturally. That’s grade school type stuff you could play “which one of these things is not like the other” on sesame street with. It’s just not that hard. And it absolutely cannot accidentally zero out the wrong area nor the wrong length which RtlZeroMemory can, has, and will continue to do from time to time because it on the other hand absolutely does lend itself to being used incorrectly. Geesh, I guarantee you if DATA x={}; syntax was a C++ construct to zero data rather than C then a number of people would be complaining about how bad and evil it. It is quite clear to me that DATA() initialization is one of those cases where C++ is more suitable than C for writing kernel code.
And then when Microsoft decides to include a basic new& delete in the next ntddk.h what are we to do with our special version, comment out Microsoft’s I guess?
That’s what the namespace is for. Sorry if I intruded on your religious
beliefs. I am not much into faith-based programming.
