How much does/should a driver programmer earn?

NTDEV
21

> If you want to learn about kernel mode programming in general, however, I’d suggest to

participate in the development of one of the many smaller open-source operating systems
like Haiku, Syllable or KolibriOS.

Well, this will just help you to understand what actually makes OSes tick. In a way, it may be helpful. For example, you will never question the fact that you cannot wait at elevated IRQL, because you will be able to understand the logic behind that - you will be able to understand why a piece of code may be running in atomic context (as well as to understand how the OS without a notion of atomic context can be designed, i.e. to give every execution unit , including interrupt handlers, its own stack where it can save its context, so that applying dispatcher policies to all execution units without any exception becomes possible) .

At the same time, this knowledge is going to be “of limited usefulness”, so to say, when it comes to actually writing drivers for some particular major OS, because different systems may have totally different architectures and IO models - even if underlying concepts that they rely upon are basically the same. Therefore, all your in-depth knowledge of the system X is pretty useless when it come to writing drivers for the system Y…

Anton Bassov

22

Pretty much everything you said is statistically very very significant,
hence almost surely tautology :).

-pro

Bill McKenzie wrote:

Not sure what tautology you speak of…but then again don’t really care.

:slight_smile:

Bill M.

23

From a monetary standpoint, no. It is most emphatically NOT worth it to learn kernel-mode programming. Not anymore.

This isn’t because “bone headed middle managers don’t get it.” There are two general factors that cause the salaries of kernel devs to exceed those of other dev:

  1. There’s insufficient demand for kernel-mode devs in most areas to create a specialist market. Back in the day when lots of operating systems were being developed, kernel-mode devs commanded a VERY significant premium. Now? Not so much. You wanna write OS code, where are you gonna go work?

  2. As somebody already pointed out, the VAST majority of kernel-mode devs are occasional kernel-mode developers: They work on some .NET application in the morning, some MSMQ-based communication system in the afternoon, and tomorrow they’ll spend a few hours fixing bugs in the company’s filter driver. Do they “deserve” – from a corporate compensation point of view – a significant pay premium for what might be less than 30% of their job? Hey, if they don’t wanna do the driver work, the next guy might be interested in learning something new.

In terms of consultants, nobody should get the idea that consulting is a goldmine for a kernel dev. What a consultant can charge a client is all about (a) the certainty that you’ll deliver a working solution, and (b) the value you bring to client and their products with that solution.

MANY hardware companies view a driver much like they view electrical safety and emissions testing (FCC, UL, CSA, etc): An annoying yet necessary step in bringing their product to market. The faster, easier, and cheaper it can be accomplished… the better. Thus, when they hire consultants to write drivers for these products, the consultants are paid commodity rates. I have seen entire drivers written for significantly less than what I charge for one week of my time.

As Mr. Terhell said, if low-level programming is what interests you and it you derive maximal satisfaction from it, then it’s worth it from a personal standpoint … of course.

If you wanna make “the big bucks”… Forget kernel-mode programming, go learn SQL or javascript or some such stuff that’s highly in demand. You’ll sleep better, and your bank account will be happier.

Peter
OSR

24

OK, maybe not the perfect injection point but this does, sort of, dove
tail into the latest Pontificates article. Can it be that you (Peter)
have crossed over to the dark side? Implementing garbage collection,
preventing buffer overruns, what’s next, allowing threads to trounce on
all of memory and the system fully recover or better yet crash and
continue running? Hmm, maybe the Blue Pill does have its application in
Windows.

Allowing this type of blind development; sloppy implementation, bad
design and Unix-like memory consumption which is rampant in almost all
C# applications, would lead us into a world where every Gingus Milktoast
would be implementing a driver. Wait, maybe we’re already there!

I’m not saying that there shouldn’t be added protection mechanisms to
minimize the times where a Java programmer writing their first
compression driver as a filter decides to allocate all buffers as
MustSucceed since their driver is crucial to MSFT making their next
dollar BUT how about the other end? How about making it easier to
diagnose problems when they do occur? Adding layers of crap onto an
already complex system is only going to make OCA useless at best and
make our job one of trying to decipher some MSFT geniuses idea of what
he thinks is a slick implementation of ‘new’ that offers everything
including … well, we won’t go there.

My point is, I can’t believe that you are offering up anything other
than going back to our roots, everything in C and the occasional few
lines of assembly when needed, variables at the top of a function as God
intended (thanks snoone) and un-mangled function names! What has our
world come to, I ask …

Sing#? C#? These types of layers are only required for developers that
don’t care about how they implement the design and want to get to an end
point without the satisfaction of writing 25K lines of C to ensure a
fully robust implementation of AES-in-a-filter. Once we are there, we
might as well just offer out complete solutions in the WDK for
everything from content control to encryption to compression, hell,
throw in a full data manipulation framework, all for free. Have these
samples wrapped in the latest, or better yet implemented in, Sing# just
to show that it can be done!

This will really inject into the kernel mode consulting industry the
needed opiate so we can further chase the next obscure problem down the
twisted path of futility. There, I tied it back to the current thread,
sort of …

Pete

Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

xxxxx@osr.com wrote:

From a monetary standpoint, no. It is most emphatically NOT worth it to learn kernel-mode programming. Not anymore.

This isn’t because “bone headed middle managers don’t get it.” There are two general factors that cause the salaries of kernel devs to exceed those of other dev:

  1. There’s insufficient demand for kernel-mode devs in most areas to create a specialist market. Back in the day when lots of operating systems were being developed, kernel-mode devs commanded a VERY significant premium. Now? Not so much. You wanna write OS code, where are you gonna go work?

  2. As somebody already pointed out, the VAST majority of kernel-mode devs are occasional kernel-mode developers: They work on some .NET application in the morning, some MSMQ-based communication system in the afternoon, and tomorrow they’ll spend a few hours fixing bugs in the company’s filter driver. Do they “deserve” – from a corporate compensation point of view – a significant pay premium for what might be less than 30% of their job? Hey, if they don’t wanna do the driver work, the next guy might be interested in learning something new.

In terms of consultants, nobody should get the idea that consulting is a goldmine for a kernel dev. What a consultant can charge a client is all about (a) the certainty that you’ll deliver a working solution, and (b) the value you bring to client and their products with that solution.

MANY hardware companies view a driver much like they view electrical safety and emissions testing (FCC, UL, CSA, etc): An annoying yet necessary step in bringing their product to market. The faster, easier, and cheaper it can be accomplished… the better. Thus, when they hire consultants to write drivers for these products, the consultants are paid commodity rates. I have seen entire drivers written for significantly less than what I charge for one week of my time.

As Mr. Terhell said, if low-level programming is what interests you and it you derive maximal satisfaction from it, then it’s worth it from a personal standpoint … of course.

If you wanna make “the big bucks”… Forget kernel-mode programming, go learn SQL or javascript or some such stuff that’s highly in demand. You’ll sleep better, and your bank account will be happier.

Peter
OSR

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

25

> This isn’t because “bone headed middle managers don’t get it.” There are

two general factors that cause the salaries of kernel devs to exceed those
of other dev:

  1. There’s insufficient demand for kernel-mode devs in most areas to
    create a specialist market. Back in the day when lots of operating
    systems were being developed, kernel-mode devs commanded a VERY
    significant premium. Now? Not so much. You wanna write OS code, where
    are you gonna go work?

I keep running into several companies that have a true need for significant
kernel development work, but the managers are completely clueless as to what
that involves. There is some argument there for *sufficient* demand
perhaps, but clueless managers is definitely a huge factor.

For instance I had a manager that I interviewed with once who needed a
virtual serial setup on top of a USB device along with about 10 other
drivers. He said, “Well, we figure that one will take about a week, I am
more worried about some of the other stuff.”

A week for a virtual serial port driver, I asked? “Sure, those have been
around forever, its a simple device and I am sure there is some simple
sample code somewhere”. If that had been the only clueless thing he said,
it wouldn’t be noteworthy. I remember that one in particular because I
literally and uncontrollably started laughing.

Bill M.

wrote in message news:xxxxx@ntdev…
>


>
> From a monetary standpoint, no. It is most emphatically NOT worth it to
> learn kernel-mode programming. Not anymore.
>
> This isn’t because “bone headed middle managers don’t get it.” There are
> two general factors that cause the salaries of kernel devs to exceed those
> of other dev:
>
> 1) There’s insufficient demand for kernel-mode devs in most areas to
> create a specialist market. Back in the day when lots of operating
> systems were being developed, kernel-mode devs commanded a VERY
> significant premium. Now? Not so much. You wanna write OS code, where
> are you gonna go work?
>
> 2) As somebody already pointed out, the VAST majority of kernel-mode devs
> are occasional kernel-mode developers: They work on some .NET application
> in the morning, some MSMQ-based communication system in the afternoon, and
> tomorrow they’ll spend a few hours fixing bugs in the company’s filter
> driver. Do they “deserve” – from a corporate compensation point of
> view – a significant pay premium for what might be less than 30% of their
> job? Hey, if they don’t wanna do the driver work, the next guy might be
> interested in learning something new.
>
> In terms of consultants, nobody should get the idea that consulting is a
> goldmine for a kernel dev. What a consultant can charge a client is all
> about (a) the certainty that you’ll deliver a working solution, and (b)
> the value you bring to client and their products with that solution.
>
> MANY hardware companies view a driver much like they view electrical
> safety and emissions testing (FCC, UL, CSA, etc): An annoying yet
> necessary step in bringing their product to market. The faster, easier,
> and cheaper it can be accomplished… the better. Thus, when they hire
> consultants to write drivers for these products, the consultants are paid
> commodity rates. I have seen entire drivers written for significantly
> less than what I charge for one week of my time.
>
> As Mr. Terhell said, if low-level programming is what interests you and it
> you derive maximal satisfaction from it, then it’s worth it from a
> personal standpoint … of course.
>
> If you wanna make “the big bucks”… Forget kernel-mode programming, go
> learn SQL or javascript or some such stuff that’s highly in demand.
> You’ll sleep better, and your bank account will be happier.
>
> Peter
> OSR
>
>
>

26

Shouldn’t it be the norm rather than exception ?

There is a theory behind it, it is called “Revenge of the C grader”. If I
had money I would have tried to make film about it…

C’mon Bill, what are you expecting from them?. They should tell you that a
driver should take about 3 mo. to be developed, and another 2 to 3 mo. to
deploy after cleaning corner cases …

-pro

> This isn’t because “bone headed middle managers don’t get it.” There
> are
> two general factors that cause the salaries of kernel devs to exceed
> those
> of other dev:
>
> 1) There’s insufficient demand for kernel-mode devs in most areas to
> create a specialist market. Back in the day when lots of operating
> systems were being developed, kernel-mode devs commanded a VERY
> significant premium. Now? Not so much. You wanna write OS code, where
> are you gonna go work?

I keep running into several companies that have a true need for
significant
kernel development work, but the managers are completely clueless as to
what
that involves. There is some argument there for *sufficient* demand
perhaps, but clueless managers is definitely a huge factor.

For instance I had a manager that I interviewed with once who needed a
virtual serial setup on top of a USB device along with about 10 other
drivers. He said, “Well, we figure that one will take about a week, I am
more worried about some of the other stuff.”

A week for a virtual serial port driver, I asked? “Sure, those have been
around forever, its a simple device and I am sure there is some simple
sample code somewhere”. If that had been the only clueless thing he said,
it wouldn’t be noteworthy. I remember that one in particular because I
literally and uncontrollably started laughing.

Bill M.

wrote in message news:xxxxx@ntdev…
>>


>>
>> From a monetary standpoint, no. It is most emphatically NOT worth it to
>> learn kernel-mode programming. Not anymore.
>>
>> This isn’t because “bone headed middle managers don’t get it.” There
>> are
>> two general factors that cause the salaries of kernel devs to exceed
>> those
>> of other dev:
>>
>> 1) There’s insufficient demand for kernel-mode devs in most areas to
>> create a specialist market. Back in the day when lots of operating
>> systems were being developed, kernel-mode devs commanded a VERY
>> significant premium. Now? Not so much. You wanna write OS code, where
>> are you gonna go work?
>>
>> 2) As somebody already pointed out, the VAST majority of kernel-mode
>> devs
>> are occasional kernel-mode developers: They work on some .NET
>> application
>> in the morning, some MSMQ-based communication system in the afternoon,
>> and
>> tomorrow they’ll spend a few hours fixing bugs in the company’s filter
>> driver. Do they “deserve” – from a corporate compensation point of
>> view – a significant pay premium for what might be less than 30% of
>> their
>> job? Hey, if they don’t wanna do the driver work, the next guy might be
>> interested in learning something new.
>>
>> In terms of consultants, nobody should get the idea that consulting is a
>> goldmine for a kernel dev. What a consultant can charge a client is all
>> about (a) the certainty that you’ll deliver a working solution, and (b)
>> the value you bring to client and their products with that solution.
>>
>> MANY hardware companies view a driver much like they view electrical
>> safety and emissions testing (FCC, UL, CSA, etc): An annoying yet
>> necessary step in bringing their product to market. The faster, easier,
>> and cheaper it can be accomplished… the better. Thus, when they hire
>> consultants to write drivers for these products, the consultants are
>> paid
>> commodity rates. I have seen entire drivers written for significantly
>> less than what I charge for one week of my time.
>>
>> As Mr. Terhell said, if low-level programming is what interests you and
>> it
>> you derive maximal satisfaction from it, then it’s worth it from a
>> personal standpoint … of course.
>>
>> If you wanna make “the big bucks”… Forget kernel-mode programming, go
>> learn SQL or javascript or some such stuff that’s highly in demand.
>> You’ll sleep better, and your bank account will be happier.
>>
>> Peter
>> OSR
>>
>>
>>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

27

Well, OF COURSE they’re completely clueless. I didn’t suggest that they weren’t. Just that their unabashed cluelessness isn’t the primary reason for driver dev jobs not paying as much as other dev jobs.

Pete Scott: I’m happy to have this debate BUT IN ANOTHER THREAD. Suffice it to say that there were once plenty of people who believed you shouldn’t drive a car unless you can fix it when it breaks down. Now, there are people who can’t find the oil fill to save their lives, and they seem to manageusing their cars daily without any big problems. SO START ANOTHER THREAD and I’ll be happy to point out the error of your ways :slight_smile:

[My Manifesto For Change: All drivers, except perhaps those needed to boot the system, should be in user mode… as should the vast majority of the operating system. Drivers should be restartable on error. All drivers, user-mode or not, should be “correct by construction” and written in high-level languages with only the very minimal-most support for direct pointer manipulation. As Darwin said: “Evolve or die!”]

Peter
OSR

28

xxxxx@osr.com wrote:

[My Manifesto For Change: All drivers, except perhaps those needed to boot the system, should be in user mode… as should the vast majority of the operating system. Drivers should be restartable on error. All drivers, user-mode or not, should be “correct by construction” and written in high-level languages with only the very minimal-most support for direct pointer manipulation. As Darwin said: “Evolve or die!”]

Amen to this. There will always be people who feel the need to “push
the envelope” (witness the folks a few months back who insisted on
talking directly to the hypervisor), but people who are getting real
work done would be much happier in the utopia you just described.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

29

> -----Original Message-----

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, September 03, 2008 6:57 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] How much does/should a driver programmer earn?

xxxxx@osr.com wrote:
> [My Manifesto For Change: All drivers, except perhaps those
needed to boot the system, should be in user mode… as
should the vast majority of the operating system. Drivers
should be restartable on error. All drivers, user-mode or
not, should be “correct by construction” and written in
high-level languages with only the very minimal-most support
for direct pointer manipulation. As Darwin said: “Evolve or die!”]
>

Amen to this. There will always be people who feel the need to “push
the envelope” (witness the folks a few months back who insisted on
talking directly to the hypervisor), but people who are getting real
work done would be much happier in the utopia you just described.

I’d agree. The problem is that this utopia needs perfectly flawless
operating system with design which presumes all possible needs. Even
beyond that; how many times had you to fix hardware problems in
software? I’m affraid there will be always a need to ‘go deeper’ and do
something non-standard what any framework can’t do and what can’t be
expressed by high level language syntax. We speak about encapsulation on
higher and higher levels. That’s right and saves from all these tedious
details but sometimes it becomes an obstacle. Gosh, I know it sounds
like Alberto demanding direct hardware access. I known the boundary
between hardware and software is very unclear. I just feel we can go too
far this way. Where is the boundary? I don’t know. Maybe between kernel
and user mode?

I’m still now quite persuaded that user mode drivers is a good idea.
There are many advantages and current disadvantages (as performance) can
be solved or will disappear in the future. However, there is one which
is hard to avoid. Drivers can include more and more functionality which
doesn’t pertain to a driver. Just because it is possible and easy. It is
user mode and most original limitations are gone. I’m just working on
such a project and there is no way back…

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]

30

It’s also reasonable to note that people that are good at low level programming tend to be BAD at business. Even if you happen to be ‘good at business’, the rest of us saying OK to lowball offers is gonna take away your leverage right out from underneath you. Sneaky sales-like recruiters who make more than most of us aren’t helping - nor are people who decide to write software for free just because they can.

Remember that what you do is HARD and most people aren’t even capable of doing it. To me, that means you deserve more for having put in the time and hard work to figure it all out. Although I’m fundamentally against unions, i’ve often thought that we are a people that could greatly benefit from one.