Re: how to ensure atomic code in a kernel mode driver

NTDEV
1

At 10:05 AM 12/3/2003 -0500, you wrote:

No, first using assembler period is a bad idea, and you should never use CLI
in a driver. The kernel provides a synchronization mechanism between your
interrupts and the rest of your driver, I’m sure DriverStudio provides a
wrapper for this mechanism.

DriverWorks provides KDevice::SynchronizeInterrupts

Russ Poffenberger
NPTest, Inc.
xxxxx@NPTest.com

2

No-no-no! “Designing” is a forbidden word in this game! Your objection
is overruled! :slight_smile:

-----Original Message-----
From: George Blat [mailto:xxxxx@ntrealtime.com]
Sent: Friday, December 05, 2003 3:18 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: how to ensure atomic code in a kernel mode driver

>(3) you can replace p->thisandthat
> by a simple thisandthat,

Which is obfuscating, and thus bad in terms of securiry.

This is the same group of gentlemen that think that calling the hardware
as
knowledgeable
practitioners is bad but calling a Microsoft API written by interns like
my
then 16 year old
daughter is not risky nor obfuscating given the incorrect or lacking
documentation.

C++ is not as great as the hype would present it, but it helps me reduce

the debugging time/
designing+coding time ratio from 2:1 to about 0.75:1. Your mileage will
vary.

Stop attacking a very important tool. It’s just a tool. If you like to
put
nails with your wrench feel
free to do it. Just don’t use my wrench. All my drivers were written in
C++
and they are as
reliable, efficient and predictable as any driver can be.

George Blat
NT with real time performance
8016 188th SW, Edmonds, WA 98026

phone: 425-775-7475
fax: 781-998-5940
mailto:xxxxx@ntrealtime.com

>and (4) you can make those fields private so that
> they’ll be opaque to the caller no matter what, the compiler won’t
even
> allow the caller to reference that field.

But a stray pointer will be able to do this anyway.

> in your call, you no longer have to use things such as
p->thisandthat
inside
> your code,

This decreases the code readability and clearness, and thus introduces
a
potential for hard-to-find bugs.

Don’t you know that nearly always the fields in C++ are named started
from
“m_”, to avoid this particular mess?

> packet *p = new packet(…);
>
> and your memory allocation is automatic and needs no size
management;

Now is it better then malloc()?

> on your own either way, but in C++ at least you can use references
instead
> of pointers. Reference counts are managed inside the object:

Not so. C++ references like int& do not apply any counting.

> handles them for you. And indeed, C++ can’t stop you from casting
pointers,
> but you don’t need to use pointers: you can use references instead,
and
> goodbye casting.

In any case, there is a need of using the arbitrary-sized buffers of
bytes, for
IO, for instance. C++ will not help with them.

> runtime. Anything that needs SMP synchronization could go through an
IL
> monitor construct, which gets compiled into spinlocks,

And why explicit spinlock calls are worse?

> at runtime ! No more synchronization or memory management of any
kind it’s
> all automatic or buried inside the OO model.

Any “buried inside” things are bad. All must be clear and at surface,
to
facilitate bugfixing.

> to write the high level logic, and any junior programmer could write
a
> driver !

…and will be halted in his work by the very first BSOD at strange
call
stack.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@ntrealtime.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

3

All right, wise guy. I’ll take that challenge. I consider “operator
new”, in C++, to be syntactic sugar because you can achieve the exact
same thing, down to the assembly level, using equivalent C.

Yes, there are many language constructs that can be thought of as
“macros”, in the sense that you can always achieve the same thing by
doing itself, albeit in a more “low level” fashion. C is a nice
front-end to assembly. C++ is a nice front-end to C. And so on and so
on.

Did I say these things are BAD? No, of course not. I’m not one of
these programmers that harkens back to some mythological golden age,
where hairy-chested Real Programmers banged on assembly all day. I
would MUCH rather use C++ than C (or assembly) – for most tasks, and
given that I’m using C++ “correctly”, and not just slathering classes
and overloaded operators all over everything in sight.

Again, I don’t think these things are bad at ALL. They make life
reasonably livable. However, C is beyond 30 years old, and it is simply
inadequate for many real-world apps today. C++ is just make-up on a
horse – it’s a slightly prettier horse, but it’s still a horse.

How is what I’m “claiming” obfuscation? I’m pointing out that there are
real weaknesses in C/C++ – and anyone who has ever worked on a project
beyond a few dozen KLOC has hit these same problems. Mismatched
headers. Heap corruption nightmares. Mismatched method/function
signatures. Totally broken encapsulation (unless you bend over backward
with use of public interfaces). The whole confusion between #define and
const. The confusion between “const means compile-time constant” and
“const means read-only”. The evil pass-by-reference & syntax.
Bolted-on post-facto templates.

C++ is a bastard’s mix of OOP and C. Most of us use it because we have
to. Some of us have a, well, provincial affection for C++, and mistake
its design flaws for features. I still work in C++ because I have to –
but most of my work, when possible, has transitioned to managed
languages (CLR). The productivity gains are real. C/C++ will
eventually fade into a niche (performance, machine-specific, etc.),
similar to what has happened with assembly. And me? I won’t shed a
tear when I don’t have to chase down another leaked pointer or spammed
heap block again.

– arlie

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of George Blat
Sent: Saturday, December 06, 2003 7:48 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: how to ensure atomic code in a kernel mode driver

In many areas simple programs are called macros. If you don’t define the
context within you are talking you can say anything that comes to your
mind. In the context of C++ (and C) macros are well “defined” and what
you’re claiming doesn’t do anything but obfuscation. You can define
anything that you don’t like as syntactic sugar, or saccharine or what
have you. I here define your use of English as syntactic sugar over
using your hands and grunts to try to communicate.

George Blat

At 10:33 PM 12/5/2003, you wrote:

The “new” keyword is a macro. Not in the sense of a #define. It’s a
macro in the sense that it is just syntactic sugar for a call to the
global “void * operator new (size_t len)” function, and then to a call
to a constructor of the class. This is a macro in the conceptual
sense, not a #define. Duh.

Nobody can access amunge? Bullshit. All I have to do is copy your
header, change “private:” to “public:”, and I’m done. That, or I can
generate a method signature that matches it and calls it.

You still don’t get it. I’m talking about REAL type-safety, as in CLR.

– arlie

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Moreira, Alberto
Sent: Friday, December 05, 2003 5:38 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: how to ensure atomic code in a kernel mode driver

I’ll only reply to the technical points. The sequence

CFoo * instance = (CFoo *) 0x12345678;
instance->method(); // boom

is C disguised as C++, and you get what you shop for. If you’re doing
it in
C++, you can use references instead of pointers: no cast, and no boom.
Indeed, no language can prevent you from acting like a jackass, but
some languages provide you with alternatives that make it less likely
you will.

The “this” pointer is, indeed, a hidden pointer, but obviously, you’re
passing the worry of maintaining it to the compiler where it belongs.
Heck, you are willing to delegate to the API at runtime, which is
inherently unsafe (even the runtime stack size cannot be guaranteed to
accomodate you
!) and yet you grumble when you’re allowed to delegate to the compiler,

which is eons safer ? One ounce of compile-time checking is worth a ton

of runtime shit. And that’s additional protection too, because if I
issue “datafield” without a pointer or a class:: in front of it, what
do you know, it’s an implied “this”: again, one more chore delegated to

the compiler. And that helps preventing us from acting like jackasses.

The “new” keyword is not a macro - it’s an OPERATOR. It can be
overridden through the standard C++ mechanisms.

As for “same level of protection”, try this. Assume I have the
following
program:

class b
{
private:
class a
{
public: int amunge(…);
};
a aobj;
public:
NTSTATUS bmunge(…)
{
aobj.amunge(…);
}
};

Nobody outside class b can access amunge(). Now, write me a C program
that doesn’t let anyone but objects of class b to invoke function
amunge. Once you manage it, try this one:

class a
{
protected:
int amunge() { … }
};
class b: private a
{
public:
int bmunge() { return amunge(); }
};

and again, write me a C program that doesn’t let anyone but objects of
class b, or derived from class a, to invoke function amunge. In this
last case, even the sequence

main()
{
a aobj;
aobj.amunge();
…
}

will be frigged by the compiler as a violation; if I want that to work,

I must open my protection and declare amunge as public instead of
protected.
Bottomline: I have a three-level C++ function call hierarchy here, duly

scoped and duly encapsulated, while C only gives me two. I could have
multiple levels if I wanted too; not in C.

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Arlie Davis
Sent: Friday, December 05, 2003 4:48 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: how to ensure atomic code in a kernel mode driver

Bullshit. C++ provides the exact same level of protection as C –
virtually none. I’m not throwing a tantrum, I’m insulted that you
think I don’t understand C or C++, when I’ve been earning my living
with them (and assembly, and years of k-mode development, blah blah
blah) for many, many years.

I never said that C++ doesn’t provide static type checking. It does –

but it does NOT prevent you from violating its own type safety. It
still provides ABSOLUTELY NO constraints to what developers can do.
And most large-scale applications REQUIRE you to violate type safety to

get any real work done. C++ is just syntactic sugar. In case you
didn’t know, the first C++ compilers were just front-ends – they just
transformed the C++ code into equivalent C code.

“this” management is hardly revoluationary, Alberto. It’s JUST A
POINTER. Just because you don’t see it in the argument list, doesn’t
mean it isn’t there. It isn’t any more protected than any other
pointer. It’s just the “favorite” pointer of the compiler, within a
given class.

The “new” keyword in C++ is just a macro over “malloc” (or whatever
malloc-equivalent you want to use). There is nothing sacred about it.
Again, STOP LECTURING – I’ve been doing this for DECADES and yes, I
understand ALL OF THIS. You aren’t contributing anything new. And I
can still construct an “object” by doing this:

CFoo * instance = (CFoo *) 0x12345678;
instance->method(); // boom

I’m not going to continue this bullshit argument any more. C/C++ are
fundamentally flawed. Don’t bother telling me all about the joys of
OOP
– I know all about it, I’ve been doing all of this for YEARS, and not
just as some leaf-node. I’ve managed teams of developers, on projects
that took years to develop and release. Apparently you don’t get it
that *I* get it, so this argument is just totally pointless.

– arlie

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Moreira, Alberto
Sent: Friday, December 05, 2003 4:22 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: how to ensure atomic code in a kernel mode driver

Arlie, you stated that C++ doesn’t protect. I just gave you a fair
amount of examples of how it does protect. But if you’re as senior as
you claim, you will know well enough the difference in level between C
and C++. If you don’t, so be it, but at least give me a few valid
technical points as to why, tantrums won’t do - remember, I have
pointed hair.

C++ gives you additional type checking and safety because member
C++ functions
are now part of the class. For example, if you have a function

NTSTATUS blahblah (packet *p, int packetLength, …) { . //
references to
p->thisandthat; … }

your have near zero protection for both the pointer and the length. If
now you say

NTSTATUS packet::blahblah(…references to thisandthat, no p-> needed
in this context ! …) { }

You’re more protected, because (1) no more direct references to pointer

p in your calls, actually, no need for that pointer at all, (2) no more

need for the programmer to handle packet lengths, (3) you can replace
p->thisandthat by a simple thisandthat, and (4) you can make those
fields private so that they’ll be opaque to the caller no matter what,
the compiler won’t even allow the caller to reference that field. You
also have two less parameters in your call, you no longer have to use
things such as p->thisandthat inside your code, because you KNOW that
“thisandthat” are fields inside the object: a shitload of pointer
dereferencing has been avoided and that’s protection; and if you use
references instead of pointers, you don’t even run into casting issues.

As for memory abuse, try

packet *p = new packet(…);

and your memory allocation is automatic and needs no size management;
delete your memory when you delete your object, presto, gone. Deleting
a deleted object can only happen if you’re using pointers, but then,
good
luck: you’re on your own either way, but in C++ at least you can use
references instead of pointers. Reference counts are managed inside the
object: it’s easy enough to extend the class with an interface that
manages the counter automatically for you, so that the caller doesn’t
know about it; and circularity is guaranteed not to happen because you
KNOW that the counters are incremented every way in and decremented
every way out, because that’s the way your object works ! In garbage
collected languages such as C# it’s even better: you don’t need to
worry
about those counters, the compiler handles them for you. And indeed,
C++
can’t stop you from casting pointers, but you don’t need to use
pointers: you can use references instead, and goodbye casting. Also,
precisely because there’s nothing in the DDK that prevents you from
doing harm, it’s a good idea to encapsulate those things with classes
that guarantee some sanity - provided of course you use them. Believe
me, the highest level language won’t prevent you from acting like a
jackass, but at least they’ll give you plenty of ways to choose that
are
as far from stupidity as it can gets.

Until we have a kernel side CLR - and I’m all for it, let’s hide all
that shit inside a C# managed code model and get the JIT compiler to
generate compiled code at DriverEntry time - we have to live with the
nonsense we have, and I find C++ well superior to C in that respect.
And hey, wouldn’t it be great ? The JIT Compiler could know whether
you’re running on a single processor or on a multiprocessor, or on a
Xeon, or on an Opteron, and compile your code differently according to
what it finds out about the runtime. Anything that needs SMP
synchronization could go through an IL monitor construct, which gets
compiled into spinlocks, barriers or what else at runtime ! No more
synchronization or memory management of any kind it’s all automatic or
buried inside the OO model. You could write a driver in C#, debug it
inside MSVC.NET in the user side, then just move the IL to kernel side,

done: all low level stuff buried inside the IL, all you have to do is
to write the high level logic, and any junior programmer could write a
driver !

Alberto.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com

To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only.
It contains information that may be confidential. Unless you are the
named addressee or an authorized designee, you may not copy or use it,
or disclose
it to anyone else. If you received it in error please notify us
immediately
and then destroy it.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@sublinear.org To
unsubscribe send a blank email to xxxxx@lists.osr.com

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@ntrealtime.com To
unsubscribe send a blank email to xxxxx@lists.osr.com

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@sublinear.org To
unsubscribe send a blank email to xxxxx@lists.osr.com

4

Alberto,

Take the MDL object as example and describe how can you make it better
with c++
And i mean make it better not replace IoAllocateMdl with new because there
is no
real advantage here. I’m not pretend to know c++, never write more than few
lines
in c++, but i read carefully all your holly war for last few years and i
never seen a
real life advantage in c++ side of war examples :). So here a real object,
implemented in
c, for my opinion clean object oriented one, just show me any advantage in
implementing
this object in c++.

Regards Ilya.

At 10:27 AM 8/12/2003 -0500, you wrote:

Let me give you guys a few suggestions.

You call it Driver “object”, no ? Device “object” ? Make it so ! Real
objects, I mean. Make the driver stack an object too, make the Irp have a
member function that returns a reference to the stack. Remove all need to
pass Driver Object or Device Object as a parameter by using the kinds of
techniques employed by MFC to make templates out of documents and views.
Bury memory allocation inside the objects, so that we never have to invoke
any allocator function directly and we never have to worry about which pool
we getting data from: if an object requires the nonpaged pool, for example,
that should be buried inside the object itself through overriding the “new”
operator. Make partially opaque data structures into objects and use
attributes such as “private” and “protected” to hide what you don’t want the
rest of the world to see. Use inheritance to derive Irp objects from other
Irp objects. Define all interfaces as abstract classes, COM style. Define
objects for all relevant hardware entities: buses, bridges, dma, agp,
processor, you name it, and make those objects behave like OS monitors,
multiprocessor safe: the serialization is inherent to the object and needs
not be negotiated by the programmer, it comes automatically. Give us a
coherent interface into the runtime, like for example what we have in STL.
Give us polymorphic versions of the API functions, so that we can omit
unnecessary parameters from the function calls and thus reduce the
possibility of error.

This and other things, would make writing drives a breeze.

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Peter Wieland
Sent: Friday, December 05, 2003 6:38 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: how to ensure atomic code in a kernel mode driver

Since this relates to code I was directly (and indirectly) responsible
for I feel compelled to say that C++ would have been of little help
here.

I guess we might have written an allocator class for certain structures
that could keep some back-pocket resources to ensure we can make
progress on one i/o request at a time. Instead we wrote a set of
functions to do it. Doing it with the class keyword would have added
little.

Overriding operator-new so we didn’t define new allocator functions
would also provide little - either we pass in enough new data at each
invocation of operator-new to tell it to follow the work-in-low-memory
semantics, or we check the value when operator-new comes back and do
something special. Encapsulating it in a set of functions was just as
good as operator-new. And if you stick to a good function naming scheme
you can get most of the benefits of methods anyway.

OOP design paradigms are what are valuable here. If using an “OOP”
language without the associated OOP paradigms is not helpful, then
perhaps it’s not the language that’s providing the benefits.

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Moreira, Alberto
Sent: Friday, December 05, 2003 1:26 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: how to ensure atomic code in a kernel mode driver

…

>Given the amount of labour MS did to the disk stack on XP (NO resources

>are allocated in critical IO paths) - I can hardly imagine C++ to be
>suitable
for
>such a task.

Let me put it this way: doing the XP stack in C++ would entail a lot
less work.

…

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@jungo.com
To unsubscribe send a blank email to xxxxx@lists.osr.com