C++ support lib for kernel?

[Daniel_Terhell]

I just might have something to say on the topic but I believe this belongs
on nttalk so there.

//Daniel

“Dejan Maksimovic” wrote in message news:xxxxx@ntdev…
>
> (quite) some time ago, I have used NuMega DriverWorks for a driver
> that required C++ support. The main reason I needed DW is support for
> new and delete.
> Is there any C++ library for kernel mode today for more than
> “super-C”, that works in both x86 and x64? I believe the driver won’t
> require anything fancier than new/delete over what the WDK provides, but
> since templates are used, I am not sure. The new/delete is the only
> thing preventing a successful link, and yes I know that doesn’t mean the
> code would run after.
>
> Remembering how the code handled a possible new failure will be a
> big task as there is no SEH support
>
> –
> Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
> http://www.alfasp.com
> File system audit, security and encryption kits.
>
>
>

[Aditya_Shrivastava]

“”
If you need memory zeroed then why not do it the C++ way that documents it as such in the code:

DATA *x = new DATA; // uninitialized
DATA *y = new DATA(); // initialized to 0–compiler will insert a memset for
“”

I have never experienced this in C++, all new should do after memory allocation is to call constructor, Now if constructor calls memset for variable it will set it to zero else not.

I still wrote some sample code in VS 2008 to verify this,

class test
{
int a[10];
public:
test()
{
printf(“hi” );
}
};

int _tmain(int argc, _TCHAR* argv)
{
test * a = new test;
test * b = new test();
return 0;
}

and its disassemblly is identical, with NO memset in between. As expected isn’t it.

Aditya

[Tim_Roberts]

xxxxx@gmail.com wrote:

“”
If you need memory zeroed then why not do it the C++ way that documents it as such in the code:

DATA *x = new DATA; // uninitialized
DATA *y = new DATA(); // initialized to 0–compiler will insert a memset for
“”

I have never experienced this in C++, all new should do after memory allocation is to call constructor, Now if constructor calls memset for variable it will set it to zero else not.
…
and its disassemblly is identical, with NO memset in between. As expected isn’t it.

That’s because you supplied a constructor. The behavior referred to by
the original poster is the behavior of the default constructor. Remove
your constructor, and you’ll see the memset.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

[Aditya_Shrivastava]

Thanks Tim for pointing that. You are indeed correct and this definitely is *NEW* to me. Still surprised actually as it creates more confusion than benefit.(But that is off topic for this thread)

got in more detail here, http://stackoverflow.com/questions/620137/do-the-parentheses-after-the-type-name-make-a-difference-with-new

Thanks again.

[Rossetoecioccolato]

The confusion which we see expressed in this thread is itself a
substantial reason not to rely solely on the compiler’s default memset
constructor, given the security problems which can result from not
initializing kernel memory. Even experienced developers sometimes make
mistakes and omitting the parenthesis on the constructor is an easy
mistake to make. The compiler will not complain about the “error” and
sadly neither will most code review. Also, most of the classes that I
use have constructors. Even the current OS has objects with
uninitialized data fields because the fields aren’t currently in use and
someone forgot to initialize them and code review missed it. In kernel
programming it is best not to trust anybody.

That’s because you supplied a constructor. The behavior referred to by
the original poster is the behavior of the default constructor. Remove
your constructor, and you’ll see the memset.

Wouldn’t this discussion be better over in nttalk?

[OSR_Community_User]

> omitting the parenthesis on the constructor is an easy mistake to make

Think about it this way. After you allocate a data structure, you will say to yourself “now the next step is to zero it”. So you put in a RtlZeroMemory, (), memset, = {}, or whatever is appropriate. So if you think about it, none of these is the slightest bit easier to forget than the other. The other thing you may do while reviewing the code is say “did it get zerod?”. With new you look for the () and say “yes, got it” in a millisecond. RtlZeroMemory requires careful inspection of its parameters looking closely at using &'s and *'s appropriately before you can confirm it so advantage new on readability.

Now say you did forget to zero for whatever reason. With new, the compiler is aware the data structure is uninitialized and will give warnings if any of it is read by your code so there is some safety net. You can only get that with C by running special tools in the DDK that understand ExAllocatePool. And with RtlZeroMemory you must be very careful not to accidentally zero the wrong window of data and there is nothing to help there. You can’t accidentally get the area wrong with using new. And let’s not forget new also guarantees the correct size is allocated. With C you have to manually come up with the size and add it as an additional parameter which is just another unnecessary way to make a mistake.

So new/delete carries a suite of advantages. If it was a C construct then I suspect everyone would just use it naturally rather than try to figure out some way to disregard its existence.

Wouldn’t this discussion be better over in nttalk?

No. This is one of the most popular threads on the list right now and we are covering “new” territory here. Pardon the pun

[Maxim_S_Shatskih]

> Amen, not just the first ones either. I have witnessed such examples. I even

hated anything C++ for years because the people using it said they were superior
people compared to C people. They claimed object orientation meant they would
write code twice as fast, they would have no bugs making QA irrelevant.

Ha-ha-ha. Abused C++ makes the potential for extremely hard to find bugs, which are much worse then most C bugs.

Also, such people (my bosses were such in early 1990ies, and so was I, we were all young, and yes, C++ was young too) often do not understand that C++ is a paradigm mix of OOP and ADT/metaprogramming, that some features belong to OO and some to ADT, and mixing OO+ADT styles in the same program produces even more mess then C’s spaghetti.

Sutter and Alexandrescu understand this well, but not all people write C++ code in conformance with their recommendations.

For instance: not all classes are suitable to be base classes (base class is an OO notion), especially STL containers do not (STL is an ADT thing).

Beware of inheritance, it produces the fragile base class bugs which are by far harder to fix then C-style spaghetti bugs.

And yes, people like my bosses and I in early 1990ies were writing copy constructors with updates to the original object, implementing the B-tree page split function as copy constructor. Yes, this is bad style.

Even here on this list somebody suggests to inherit from a template which defines operator new/delete. Evil. For the correct way, see how std::vector uses std::allocator.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

[OSR_Community_User]

I am so sorry I even asked for help on measly new/delete support after seeing the amount of offtopic discussion in this thread.

–
Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
http://www.alfasp.com
File system audit, security and encryption kits.

[Maxim_S_Shatskih]

> DATA *x = new DATA; // uninitialized

DATA *y = new DATA(); // initialized to 0–compiler will insert a memset for

Just plain nonsense.

The correct answer is:

• both constructs are synonyms
• both call the default constructor if provided or auto-generated.
• auto-generated default constructor just calls the default constructors of base classes and members of class types, members of non-class types are not initialized at all.
• only static objects are zero-filled, not allocations by new().

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

[Maxim_S_Shatskih]

>is the slightest bit easier to forget than the other. The other thing you may do while reviewing the code is say "did it

get zerod?".

“Did it get initialized?” is the correct question.

In C++, this is solved by providing a constructor.

In C, I usually review that all serious fields are assigned. There are also unserious fields, which are only used at some points of structure lifetime, and commented this way.

Zeroing gives you nothing - just NULL pointer fault instead of random pointer fault, and with integers it is even more useless.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

[Maxim_S_Shatskih]

>In the genetic case, the Kernel is not the place to leverage the more subtle features of c++ - like, say, operator

overloading

Overloading is ADT, and ADT also requires all other stuff to support by-value objects - STL, exceptions and so on.

ADT/meta programming in kernel is possible, but note: the most complexity in the kernel is - connecting your code to Windows in a proper way, you must obey lots of Windows-specific tiny details for this. And this makes most theoretized architectures (if not even the whole content of GoF book) next to useless - “What facade? NDIS facade? then say “NDIS”, and not “facade”, we need to know NDIS details and not GoF abstractions”.

ADT will hardly provide any good in solving this particular issue.

This is not to say the trivial possibilities of overloading abuse.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

[Peter_Hurley]

Max wrote:

> DATA *x = new DATA; // uninitialized
> DATA *y = new DATA(); // initialized to 0–compiler will insert a
memset for

Just plain nonsense.

The correct answer is:

• both constructs are synonyms
• both call the default constructor if provided or auto-generated.
• auto-generated default constructor just calls the default constructors
  of base classes and members of class types, members of non-class types
  are not initialized at all.
• only static objects are zero-filled, not allocations by new().

Not true.

From C++ 2003 (avail here: http://code.google.com/p/openassist/downloads/detail?name=C%2B%2B%20Standard%20-%20ANSI%20ISO%20IEC%2014882%202003.pdf):

“12.6 Initialization [class.init]
When no initializer is specified for an object of (possibly cv-qualified) class type (or array thereof), or the initializer has the form (), the object is initialized as specified in 8.5. The object is default-initialized if there is no initializer, or value-initialized if the initializer is ().”

and

"8.5 Initializers [dcl.init]
…
To value-initialize an object of type T means:

• if T is a class type (clause 9) with a user-declared constructor (12.1), then the default constructor for T is called (and the initialization is ill-formed if T has no accessible default constructor);
• if T is a non-union class type without a user-declared constructor, then every non-static data member and base-class component of T is value-initialized;
• if T is an array type, then each element is value-initialized;
• otherwise, the object is zero-initialized
  A program that calls for default-initialization or value-initialization of an entity of reference type is ill-formed. If T is a cv-qualified type, the cv-unqualified version of T is used for these definitions of zero-initialization, default-initialization, and value-initialization."

What this means is that an object initialized like this -> DATA() <- is recursively initialized, resolving every base, non-static member or array element with either default construction (or compiler diagnostic if the class has a user-defined constructor but not a default constructor) or zero-initialization.

The point of this semantic construction is really to support template authoring, where the template parameter type may not have a default constructor (eg. built-in types) but the template author still needs to guarantee initialized objects.

A simple example is zero-terminating a string of templated char type (which may or may not be a built-in type).

template <class _chart …>
class basic_string : private internal::_string_base<> {
…
private:
void terminate_string() { *this->end_of_string = _CharT(); }
…
}

Ironically, it’s really the C-compatibility of C++ that makes this construct less-well-known, because

DATA a();

is not a declaration of object ‘a’ of type DATA that is value-initialized
but rather a declaration of function ‘a’ returning type DATA!

Peter Hurley

[Maxim_S_Shatskih]

>From C++ 2003 (avail here: http://code.google.com/p/openassist/downloads/detail?name=C%2B%2B%

20Standard%20-%20ANSI%20ISO%20IEC%2014882%202003.pdf):

Then it have changed since the Stroustrup/Ellis book which I have cited above.

void terminate_string() { *this->end_of_string = _CharT(); }

With Stroustrup/Ellis, this constructor call is allowed and a no-op.

*but rather* a declaration of function ‘a’ returning type DATA!

Yes, but:

auto DATA a();

declares an object.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

[mm1]

> auto DATA a();

Yes, but who uses that?

Also, I believe that this usage of ‘auto’ is going the way of the Dodo in
C++0x.

mm
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Tuesday, May 24, 2011 11:11 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Re:C++ support lib for kernel?

From C++ 2003 (avail here:
http://code.google.com/p/openassist/downloads/detail?name=C%2B%2B%
20Standard%20-%20ANSI%20ISO%20IEC%2014882%202003.pdf):

Then it have changed since the Stroustrup/Ellis book which I have cited
above.

void terminate_string() { *this->end_of_string = _CharT(); }

With Stroustrup/Ellis, this constructor call is allowed and a no-op.

*but rather* a declaration of function ‘a’ returning type DATA!

Yes, but:

auto DATA a();

declares an object.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

---

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

[Peter_Hurley]

Rossetoecioccolato wrote:

On 5/20/2011 4:09 AM, xxxxx@gmail.com wrote:
> And then when Microsoft decides to include a basic new& delete in the
next ntddk.h what are we to do with our special version, comment out
Microsoft’s I guess?

That’s what the namespace is for.

new and delete can only be class members or global functions. Again, from C++ 2003;

“3.7.3.1 Allocation functions [basic.stc.dynamic.allocation]
An allocation function shall be a class member function or a global function; a program is ill-formed if an allocation function is declared in a namespace scope other than global scope or declared static in global scope.”

The MS C++ compiler won’t diagnostic if you *declare* new and delete within a namespace. However, if you attempt to reference that name, eg.

int* p = doesnt_work::new int;

the compiler will issue diagnostic C2834: ‘operator new’ must be globally qualified.

If you try to end-run around that by using new within the namespace, eg.

namespace doesnt_work {
void* operator new(size_t);
void operator delete(void*);

int* foo() {
return new int;
}
}

the MS compiler won’t diagnostic (it really should) but ignores the ‘doesnt_work::new’ declaration and calls ‘::new’ anyway.

Peter Hurley

[mm1]

Wow. I knew the first part, but I didn’t know that that’s what would happen
(with CL) if you tried to namespace it.

mm

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Peter Hurley
Sent: Tuesday, May 24, 2011 12:59 PM
To: Windows System Software Devs Interest List
Subject: RE: Re:[ntdev] C++ support lib for kernel?

Rossetoecioccolato wrote:

On 5/20/2011 4:09 AM, xxxxx@gmail.com wrote:
> And then when Microsoft decides to include a basic new& delete in the
next ntddk.h what are we to do with our special version, comment out
Microsoft’s I guess?

That’s what the namespace is for.

new and delete can only be class members or global functions. Again, from
C++ 2003;

“3.7.3.1 Allocation functions [basic.stc.dynamic.allocation]
An allocation function shall be a class member function or a global
function; a program is ill-formed if an allocation function is declared in a
namespace scope other than global scope or declared static in global scope.”

The MS C++ compiler won’t diagnostic if you *declare* new and delete within
a namespace. However, if you attempt to reference that name, eg.

int* p = doesnt_work::new int;

the compiler will issue diagnostic C2834: ‘operator new’ must be globally
qualified.

If you try to end-run around that by using new within the namespace, eg.

namespace doesnt_work {
void* operator new(size_t);
void operator delete(void*);

int* foo() {
return new int;
}
}

the MS compiler won’t diagnostic (it really should) but ignores the
‘doesnt_work::new’ declaration and calls ‘::new’ anyway.

Peter Hurley

---

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

[Mark_Roddy]

If a discussion goes on long enough on ntdev it becomes a flame war
either on the merits of C++ in the kernel or one on the merits of
linux vs windows.

Also note that flame wars on the merits of linux vs windows are
indistinguishable from flame wars on the merits of C++ in the kernel.

Mark Roddy

On Tue, May 24, 2011 at 9:55 AM, Dejan Maksimovic wrote:
>
> ? ?I am so sorry I even asked for help on measly new/delete support after seeing the amount of offtopic discussion in this thread.
>
> –
> Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
> http://www.alfasp.com
> File system audit, security and encryption kits.
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>

[Peter_Hurley]

Dejan Maksimovic wrote:

I am so sorry I even asked for help on measly new/delete support
after seeing the amount of offtopic discussion in this thread.

I don’t understand this statement.

*Every* post in this thread is valuable and on-topic. That there is disagreement among smart people about the ‘best’ or ‘right’ way to do a thing is typical when there is not a canonical implementation on the platform (and even then, there’s disagreement).

For the record: Sven and Max are right. Do not zero memory within operator new (or any other general-purpose allocator).

1. There is no security benefit.

If there was a security benefit to an allocator function zeroing memory then it would have already been incorporated into ExAllocatePoolWithTag. Notwithstanding, deliberately and specifically initialize any memory that will be made visible to user-mode - just not in operator new.

Security-wise, the implementation in kcom.h is wrong anyway. RtlZeroMemory is an inline function which the compiler will optimize away if it ‘thinks’ the memory is unreferenced - that is why RtlSecureZeroMemory was added.

If you have major security concerns, zero the memory in operator delete instead. (But if the code is a heavy, frequent user of the free-store, please consider implementing a pool allocator that only zeroes memory that is being returned to the free-store).

2. Explicitly establish class invariants in constructors (preferably in the initializer list).

Relying on implicitly zeroed memory is a false crutch that will fail *someone else*. One thing that MS has done fairly well at is keeping the core DDI stable for more than a decade. Be prepared that it will not be you building upon the foundation you’re laying.

3. It’s unnecessarily wasteful.

Your driver might not care about those CPU/memory cycles but my apps do. (For the same reason, do not use lists because that’s all that’s provided and then search them linearly.)

For language reference, I highly recommend the following reading:

1. The C++ Programming Language: Special Edition. Stroustrup.
   Covers Standard C++ 2003.
2. C++ Coding Standards. Sutter & Alexandrescu.
   In it’s 9th printing - 'nuff said.
3. Exceptional C++. Sutter.
   Excerpts of GotW. At the time, no one could do all of
   problems without looking ahead.
4. Effective C++. Meyers.
5. Effective STL. Meyers.

Hope that helps,
Peter Hurley

[Rossetoecioccolato]

> RtlSecureZeroMemory <

Thanks for bringing this function to my attention. What platforms is it
supported on?

[Peter_Hurley]

Rossetoecioccolato wrote:

> RtlSecureZeroMemory <

Thanks for bringing this function to my attention. What platforms is it
supported on?

All platforms.

At least since 2003 DDK, RtlSecureZeroMemory has been defined in either ntddk.h or wdm.h as a inline function which aliases the dest ptr with a volatile ptr and zeroes the memory through the volatile ptr.

This works because the compiler guarantees writes to volatile storage (although it may reorder accesses to other local, non-volatile storage).

It is (comparatively) slow however; no attempt is made to chop the memory accesses into aligned word-lengths. The latest WDK makes a wimpy effort to improve it by using the __stosb intrinsic for x64.

Peter Hurley