Hi,
has anyone of you experience with unit testing in kernel mode ?
I feel quite comfortable (and nearly secure) with Driver Verifier,
having extensive registry configurable debug logs
and running Prefast from time to time.
And I have no idea how to mock OS functionality in a way to make
it feel real for the driver.
best regards
Else
My experience has been that different people mean different things when they
speak of unit testing depending upon whether you are in comp.object or down
here:-)
I see no reason why you cant write unit tests and pass control to test them
using debug IOCTLS the nunit style. But as to using mock OS functionality
here is what I would do:
Don’t invoke OS functions directly in your driver rather route them through
the abstract factory pattern (gof). For the debug IOCTLS the rest of your
driver code will work normal just that they have a pointer to the fake OS
object which has methods that merely check bounds/parameter correctness
(also known as environment testing). A secondary advantage of this approach
is arriving at OS neutrality in your code.
Most importantly your mock object MUST NOT try to mimic OS functionality
just the interface. Check out the original paper on mock objects at
http://www.connextra.com/aboutUs/mockobjects.pdf
banks
“Else Kluger” wrote in message news:xxxxx@ntdev…
> Hi,
>
> has anyone of you experience with unit testing in kernel mode ?
> I feel quite comfortable (and nearly secure) with Driver Verifier,
> having extensive registry configurable debug logs
> and running Prefast from time to time.
> And I have no idea how to mock OS functionality in a way to make
> it feel real for the driver.
>
> best regards
> Else
>
>
>
Hi, Else,
I’m a low level programmer, and assembler is still my choice
computer language, so, you may not find this very helpful,
Still…
I find it important to use a debugger and to single-step at
least once through every nook and cranny of my driver. What I
normally do is, I write a few very simple C++ or C# test
programs that exercise the driver API and its functions, one at
a time, and then I use SoftICE or Windbg to step through the
whole thing. So, I might first write a program that does an Open
and nothing else, then I add a Close, then I add a Read, then I
replace it by a Write. If I’m doing Ioctls, I write one little
test for each Ioctl, and here again, my emphasis isn’t in seeing
the result but in being able to reach into the code. I only feel
happy when I have covered all my code, and only then I go for
results-oriented testing.
Also, if the peripheral has plug-and-play or power requirements,
I delay complex things like hot plugging and unplugging, power
irps, and so on, until I have single stepped enough through my
driver that I’m happy with the way things work. I pay particular
attention to tables, lists, queues, and so on. I also make sure
that every shared data structure has its own Mutex or Semaphore,
I make sure that every time someone uses that structure the
Mutex gets exercised properly, and so on. And I may be a bit
paranoid, but I need to single step through the code to see it,
nothing else does it. And I step through it at assembler level,
paying very good attention to what code the compiler generated
for me; I do not trust compilers!
Only after I have single stepped through the whole driver a
healthy number of times I will then resort to tools like
Verifier or Prefast, that is, when I use them at all. I have
another personal trait, I am a strict observer of Occam’s razor,
and I only use an API when I cannot possibly avoid it; hence, I
tend to be more self-relying than other driver writers, and my
drivers have less code that I do not directly control. I also
try to isolate those parts of the driver that depend on the API,
so that I can test the API independently of the driver: I often
stub API-independent routines when I’m testing the API, and I
replace my API modules with some hand-crafted code to test my
own code independently of the APIs. It’s one thing that gives me
the jitters, to have to trust someone else’s code and I cannot
control or freely modify it! Hence, I’m very careful with my own
code, and I carefully single-step through it, over and again,
until I’m happy with the way it looks and performs.
Hope this helps!
Alberto.
----- Original Message -----
From: “Else Kluger”
To: “Windows System Software Devs Interest List”
Sent: Friday, December 09, 2005 6:35 AM
Subject: [ntdev] Driver Unit Tests
> Hi,
>
> has anyone of you experience with unit testing in kernel mode
> ?
> I feel quite comfortable (and nearly secure) with Driver
> Verifier,
> having extensive registry configurable debug logs
> and running Prefast from time to time.
> And I have no idea how to mock OS functionality in a way to
> make
> it feel real for the driver.
>
> best regards
> Else
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@ieee.org
> To unsubscribe send a blank email to
> xxxxx@lists.osr.com
This is all well and good I suppose, and it reminds me the days when Ken
Thomson openly said " You all need to worry when you hire a programmer
like me". Well the context was for security/correctness etc of programs.
And I see your point here.
But how could you avoid all ( or as much as posssible) dependencies ?.
When you put out a software, I or other ( ie. the set of all programmers
except U ) will have the same feeling.
It is really really good to program in assembler, it is also very good to
get into microcode, it is also very good to know the working of the
circuits, and finally it is extreemly good to know how the semiconductor
physics is working ( A relative of mine taught me that everything is
carbon, boy ! ).
But from engineering sense, it would not make any sense to do everything
on your own.
But one thing I agree, what fits the bill ( or rather whatever makes
someone comfortable). After all the idea is to have good quality product
:).
-pro
----- Original Message -----
From: “Alberto Moreira”
To: “Windows System Software Devs Interest List”
Sent: Saturday, December 10, 2005 6:36 AM
Subject: Re: [ntdev] Driver Unit Tests
> Hi, Else,
>
> I’m a low level programmer, and assembler is still my choice
> computer language, so, you may not find this very helpful,
> Still…
>
> I find it important to use a debugger and to single-step at
> least once through every nook and cranny of my driver. What I
> normally do is, I write a few very simple C++ or C# test
> programs that exercise the driver API and its functions, one at
> a time, and then I use SoftICE or Windbg to step through the
> whole thing. So, I might first write a program that does an Open
> and nothing else, then I add a Close, then I add a Read, then I
> replace it by a Write. If I’m doing Ioctls, I write one little
> test for each Ioctl, and here again, my emphasis isn’t in seeing
> the result but in being able to reach into the code. I only feel
> happy when I have covered all my code, and only then I go for
> results-oriented testing.
>
> Also, if the peripheral has plug-and-play or power requirements,
> I delay complex things like hot plugging and unplugging, power
> irps, and so on, until I have single stepped enough through my
> driver that I’m happy with the way things work. I pay particular
> attention to tables, lists, queues, and so on. I also make sure
> that every shared data structure has its own Mutex or Semaphore,
> I make sure that every time someone uses that structure the
> Mutex gets exercised properly, and so on. And I may be a bit
> paranoid, but I need to single step through the code to see it,
> nothing else does it. And I step through it at assembler level,
> paying very good attention to what code the compiler generated
> for me; I do not trust compilers!
>
> Only after I have single stepped through the whole driver a
> healthy number of times I will then resort to tools like
> Verifier or Prefast, that is, when I use them at all. I have
> another personal trait, I am a strict observer of Occam’s razor,
> and I only use an API when I cannot possibly avoid it; hence, I
> tend to be more self-relying than other driver writers, and my
> drivers have less code that I do not directly control. I also
> try to isolate those parts of the driver that depend on the API,
> so that I can test the API independently of the driver: I often
> stub API-independent routines when I’m testing the API, and I
> replace my API modules with some hand-crafted code to test my
> own code independently of the APIs. It’s one thing that gives me
> the jitters, to have to trust someone else’s code and I cannot
> control or freely modify it! Hence, I’m very careful with my own
> code, and I carefully single-step through it, over and again,
> until I’m happy with the way it looks and performs.
>
> Hope this helps!
>
>
> Alberto.
>
>
>
> ----- Original Message -----
> From: “Else Kluger”
> To: “Windows System Software Devs Interest List”
>
> Sent: Friday, December 09, 2005 6:35 AM
> Subject: [ntdev] Driver Unit Tests
>
>
>> Hi,
>>
>> has anyone of you experience with unit testing in kernel mode
>> ?
>> I feel quite comfortable (and nearly secure) with Driver
>> Verifier,
>> having extensive registry configurable debug logs
>> and running Prefast from time to time.
>> And I have no idea how to mock OS functionality in a way to
>> make
>> it feel real for the driver.
>>
>> best regards
>> Else
>>
Alberto
I agree that sometimes stepping through the entire code and is just not
beatable. But as the size of the code grows that possibility diminishes. Yes
I am referring to regression. So you check in some piece of code and boom
that which was workin in the past does not do so now. Which is what brings
people to the need for automated unit tests, tests run overnight which will
scream at you and mail a threatening message to your manager and every other
member in your group of your ill checkin. Not hide there latently and show
up at a customer site. In short automated tests to do code coverage. But
yes I agree automating tests is easier said than done.
banks
“Alberto Moreira” wrote in message news:xxxxx@ntdev…
> Hi, Else,
>
> I’m a low level programmer, and assembler is still my choice computer
> language, so, you may not find this very helpful, Still…
>
> I find it important to use a debugger and to single-step at least once
> through every nook and cranny of my driver. What I normally do is, I write
> a few very simple C++ or C# test programs that exercise the driver API and
> its functions, one at a time, and then I use SoftICE or Windbg to step
> through the whole thing. So, I might first write a program that does an
> Open and nothing else, then I add a Close, then I add a Read, then I
> replace it by a Write. If I’m doing Ioctls, I write one little test for
> each Ioctl, and here again, my emphasis isn’t in seeing the result but in
> being able to reach into the code. I only feel happy when I have covered
> all my code, and only then I go for results-oriented testing.
>
> Also, if the peripheral has plug-and-play or power requirements, I delay
> complex things like hot plugging and unplugging, power irps, and so on,
> until I have single stepped enough through my driver that I’m happy with
> the way things work. I pay particular attention to tables, lists, queues,
> and so on. I also make sure that every shared data structure has its own
> Mutex or Semaphore, I make sure that every time someone uses that
> structure the Mutex gets exercised properly, and so on. And I may be a bit
> paranoid, but I need to single step through the code to see it, nothing
> else does it. And I step through it at assembler level, paying very good
> attention to what code the compiler generated for me; I do not trust
> compilers!
>
> Only after I have single stepped through the whole driver a healthy number
> of times I will then resort to tools like Verifier or Prefast, that is,
> when I use them at all. I have another personal trait, I am a strict
> observer of Occam’s razor, and I only use an API when I cannot possibly
> avoid it; hence, I tend to be more self-relying than other driver writers,
> and my drivers have less code that I do not directly control. I also try
> to isolate those parts of the driver that depend on the API, so that I can
> test the API independently of the driver: I often stub API-independent
> routines when I’m testing the API, and I replace my API modules with some
> hand-crafted code to test my own code independently of the APIs. It’s one
> thing that gives me the jitters, to have to trust someone else’s code and
> I cannot control or freely modify it! Hence, I’m very careful with my own
> code, and I carefully single-step through it, over and again, until I’m
> happy with the way it looks and performs.
>
> Hope this helps!
>
>
> Alberto.
>
>
>
> ----- Original Message -----
> From: “Else Kluger”
> To: “Windows System Software Devs Interest List”
> Sent: Friday, December 09, 2005 6:35 AM
> Subject: [ntdev] Driver Unit Tests
>
>
>> Hi,
>>
>> has anyone of you experience with unit testing in kernel mode ?
>> I feel quite comfortable (and nearly secure) with Driver Verifier,
>> having extensive registry configurable debug logs
>> and running Prefast from time to time.
>> And I have no idea how to mock OS functionality in a way to make
>> it feel real for the driver.
>>
>> best regards
>> Else
>>
>>
>>
>> —
>> Questions? First check the Kernel Driver FAQ at
>> http://www.osronline.com/article.cfm?id=256
>>
>> You are currently subscribed to ntdev as: xxxxx@ieee.org
>> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
Hi, Pro,
You seem to have put your finger right on my development
approach. Yes, I happen to know assembler well, and I do believe
that it is fundamentally important to know the machine
architecture well enough, which leads to knowing what’s going on
at electronics level - and yes, I’m an EE and I kind of
understand it all the way from Solid State Physics, although I’m
not an expert except on my own professional area.
It makes for having a machine-architecture-centric approach to
development, rather than an OS-centric or an API-centric. To me,
OS’s are by and large similar, see one see all, and the
differences are relatively minor. To me, APIs are pretty
irrelevant in the big picture: you learn them, you use them to
your best advantage, you discard them when they don’t suit your
best interest, and most of all, you isolate them from your
mainstream code so that you know exactly where you must rely on
an API, and then you can minimize those occasions. And if you
can avoid an API, you avoid it, because if you use my
development approach, you trust your hand better than you trust
the hands who wrote that API routine.
I believe that from an engineering point of view it makes sense
to have as much control over the code as possible. So, my
philosophy is, I’ll reuse anything I wrote; I’ll reuse a fair
amount of stuff that I have control over its source code; I’ll
be very, very reluctant about reusing anything I don’t have
direct source code control.
That’s why I like to see my code working, instruction by
instruction, before I go into automated testing. And I also like
to see the compiler’s code working at machine level too,
because, again, I’m not sure I trust compilers: I’ve seen enough
to feel the urge to vet their generated code.
I only trust my own teeth, and even then, occasionally they bite
back my tongue!
Alberto.
----- Original Message -----
From:
To: “Windows System Software Devs Interest List”
Sent: Saturday, December 10, 2005 11:37 AM
Subject: RE: [ntdev] Driver Unit Tests
This is all well and good I suppose, and it reminds me the days
when Ken
Thomson openly said " You all need to worry when you hire a
programmer
like me". Well the context was for security/correctness etc of
programs.
And I see your point here.
But how could you avoid all ( or as much as posssible)
dependencies ?.
When you put out a software, I or other ( ie. the set of all
programmers
except U ) will have the same feeling.
It is really really good to program in assembler, it is also
very good to
get into microcode, it is also very good to know the working of
the
circuits, and finally it is extreemly good to know how the
semiconductor
physics is working ( A relative of mine taught me that
everything is
carbon, boy ! ).
But from engineering sense, it would not make any sense to do
everything
on your own.
But one thing I agree, what fits the bill ( or rather whatever
makes
someone comfortable). After all the idea is to have good quality
product
:).
-pro
----- Original Message -----
From: “Alberto Moreira”
To: “Windows System Software Devs Interest List”
Sent: Saturday, December 10, 2005 6:36 AM
Subject: Re: [ntdev] Driver Unit Tests
> Hi, Else,
>
> I’m a low level programmer, and assembler is still my choice
> computer language, so, you may not find this very helpful,
> Still…
>
> I find it important to use a debugger and to single-step at
> least once through every nook and cranny of my driver. What I
> normally do is, I write a few very simple C++ or C# test
> programs that exercise the driver API and its functions, one
> at
> a time, and then I use SoftICE or Windbg to step through the
> whole thing. So, I might first write a program that does an
> Open
> and nothing else, then I add a Close, then I add a Read, then
> I
> replace it by a Write. If I’m doing Ioctls, I write one little
> test for each Ioctl, and here again, my emphasis isn’t in
> seeing
> the result but in being able to reach into the code. I only
> feel
> happy when I have covered all my code, and only then I go for
> results-oriented testing.
>
> Also, if the peripheral has plug-and-play or power
> requirements,
> I delay complex things like hot plugging and unplugging, power
> irps, and so on, until I have single stepped enough through my
> driver that I’m happy with the way things work. I pay
> particular
> attention to tables, lists, queues, and so on. I also make
> sure
> that every shared data structure has its own Mutex or
> Semaphore,
> I make sure that every time someone uses that structure the
> Mutex gets exercised properly, and so on. And I may be a bit
> paranoid, but I need to single step through the code to see
> it,
> nothing else does it. And I step through it at assembler
> level,
> paying very good attention to what code the compiler generated
> for me; I do not trust compilers!
>
> Only after I have single stepped through the whole driver a
> healthy number of times I will then resort to tools like
> Verifier or Prefast, that is, when I use them at all. I have
> another personal trait, I am a strict observer of Occam’s
> razor,
> and I only use an API when I cannot possibly avoid it; hence,
> I
> tend to be more self-relying than other driver writers, and my
> drivers have less code that I do not directly control. I also
> try to isolate those parts of the driver that depend on the
> API,
> so that I can test the API independently of the driver: I
> often
> stub API-independent routines when I’m testing the API, and I
> replace my API modules with some hand-crafted code to test my
> own code independently of the APIs. It’s one thing that gives
> me
> the jitters, to have to trust someone else’s code and I cannot
> control or freely modify it! Hence, I’m very careful with my
> own
> code, and I carefully single-step through it, over and again,
> until I’m happy with the way it looks and performs.
>
> Hope this helps!
>
>
> Alberto.
>
>
>
> ----- Original Message -----
> From: “Else Kluger”
> To: “Windows System Software Devs Interest List”
>
> Sent: Friday, December 09, 2005 6:35 AM
> Subject: [ntdev] Driver Unit Tests
>
>
>> Hi,
>>
>> has anyone of you experience with unit testing in kernel mode
>> ?
>> I feel quite comfortable (and nearly secure) with Driver
>> Verifier,
>> having extensive registry configurable debug logs
>> and running Prefast from time to time.
>> And I have no idea how to mock OS functionality in a way to
>> make
>> it feel real for the driver.
>>
>> best regards
>> Else
>>
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: unknown lmsubst tag
argument: ‘’
To unsubscribe send a blank email to
xxxxx@lists.osr.com
Hi Alberto,
No, not at all.
I really dont see anyway I can be independent of all apis, development
tools, debugger and other stuff. I have to depend on something to start
with. For commercial products, time/cost/etc matters most. BTW, I’d been
involved in multiplatform projects, and all the base synchronization APIs
were wrapped around to make them platform independent, but still baseline
apis were used. It made sense to me.
Alberto, you probably are a Sr. Scientist. But what you mentioned ( and if
I take limit tends to infinity) every one should either have source code
with them or dont use API or any infrastructure. This is what bothers me
most. It really fails my belief for abstraction(s). I dont want to build
my shoes, period. I would try to use it, and with caution, if it hurts,
return or throw, and never use. But I dont want to make shoes unless I see
there is/are no other choices. It should be applied to all aspects of
life, and that is my belief.
So there is the optimization and/or clear boundary that I’m trying to
emphasize. And again, your approach might not fit the bill :).
Finally, I know you are an EE, solid state physicist and what not :-). But
what I’m after is separation of concern for most designer/developer.
I agree that debugging at assembler level is important. Well sometimes,
not all the times.
I agree that trust anything with caution. Even breathin Air.
I dont agree to avoid most of the API while I’m on commercial programming.
I dont agree that I’ve to have the source before I use it.
-pro
Hi, Pro,
You seem to have put your finger right on my development
approach. Yes, I happen to know assembler well, and I do believe
that it is fundamentally important to know the machine
architecture well enough, which leads to knowing what’s going on
at electronics level - and yes, I’m an EE and I kind of
understand it all the way from Solid State Physics, although I’m
not an expert except on my own professional area.
It makes for having a machine-architecture-centric approach to
development, rather than an OS-centric or an API-centric. To me,
OS’s are by and large similar, see one see all, and the
differences are relatively minor. To me, APIs are pretty
irrelevant in the big picture: you learn them, you use them to
your best advantage, you discard them when they don’t suit your
best interest, and most of all, you isolate them from your
mainstream code so that you know exactly where you must rely on
an API, and then you can minimize those occasions. And if you
can avoid an API, you avoid it, because if you use my
development approach, you trust your hand better than you trust
the hands who wrote that API routine.
I believe that from an engineering point of view it makes sense
to have as much control over the code as possible. So, my
philosophy is, I’ll reuse anything I wrote; I’ll reuse a fair
amount of stuff that I have control over its source code; I’ll
be very, very reluctant about reusing anything I don’t have
direct source code control.
That’s why I like to see my code working, instruction by
instruction, before I go into automated testing. And I also like
to see the compiler’s code working at machine level too,
because, again, I’m not sure I trust compilers: I’ve seen enough
to feel the urge to vet their generated code.
I only trust my own teeth, and even then, occasionally they bite
back my tongue!
Alberto.
----- Original Message -----
From:
To: “Windows System Software Devs Interest List”
Sent: Saturday, December 10, 2005 11:37 AM
Subject: RE: [ntdev] Driver Unit Tests
This is all well and good I suppose, and it reminds me the days
when Ken
Thomson openly said " You all need to worry when you hire a
programmer
like me". Well the context was for security/correctness etc of
programs.
And I see your point here.
But how could you avoid all ( or as much as posssible)
dependencies ?.
When you put out a software, I or other ( ie. the set of all
programmers
except U ) will have the same feeling.
It is really really good to program in assembler, it is also
very good to
get into microcode, it is also very good to know the working of
the
circuits, and finally it is extreemly good to know how the
semiconductor
physics is working ( A relative of mine taught me that
everything is
carbon, boy ! ).
But from engineering sense, it would not make any sense to do
everything
on your own.
But one thing I agree, what fits the bill ( or rather whatever
makes
someone comfortable). After all the idea is to have good quality
product
:).
-pro
----- Original Message -----
From: “Alberto Moreira”
To: “Windows System Software Devs Interest List”
Sent: Saturday, December 10, 2005 6:36 AM
Subject: Re: [ntdev] Driver Unit Tests
> Hi, Else,
>
> I’m a low level programmer, and assembler is still my choice
> computer language, so, you may not find this very helpful,
> Still…
>
> I find it important to use a debugger and to single-step at
> least once through every nook and cranny of my driver. What I
> normally do is, I write a few very simple C++ or C# test
> programs that exercise the driver API and its functions, one
> at
> a time, and then I use SoftICE or Windbg to step through the
> whole thing. So, I might first write a program that does an
> Open
> and nothing else, then I add a Close, then I add a Read, then
> I
> replace it by a Write. If I’m doing Ioctls, I write one little
> test for each Ioctl, and here again, my emphasis isn’t in
> seeing
> the result but in being able to reach into the code. I only
> feel
> happy when I have covered all my code, and only then I go for
> results-oriented testing.
>
> Also, if the peripheral has plug-and-play or power
> requirements,
> I delay complex things like hot plugging and unplugging, power
> irps, and so on, until I have single stepped enough through my
> driver that I’m happy with the way things work. I pay
> particular
> attention to tables, lists, queues, and so on. I also make
> sure
> that every shared data structure has its own Mutex or
> Semaphore,
> I make sure that every time someone uses that structure the
> Mutex gets exercised properly, and so on. And I may be a bit
> paranoid, but I need to single step through the code to see
> it,
> nothing else does it. And I step through it at assembler
> level,
> paying very good attention to what code the compiler generated
> for me; I do not trust compilers!
>
> Only after I have single stepped through the whole driver a
> healthy number of times I will then resort to tools like
> Verifier or Prefast, that is, when I use them at all. I have
> another personal trait, I am a strict observer of Occam’s
> razor,
> and I only use an API when I cannot possibly avoid it; hence,
> I
> tend to be more self-relying than other driver writers, and my
> drivers have less code that I do not directly control. I also
> try to isolate those parts of the driver that depend on the
> API,
> so that I can test the API independently of the driver: I
> often
> stub API-independent routines when I’m testing the API, and I
> replace my API modules with some hand-crafted code to test my
> own code independently of the APIs. It’s one thing that gives
> me
> the jitters, to have to trust someone else’s code and I cannot
> control or freely modify it! Hence, I’m very careful with my
> own
> code, and I carefully single-step through it, over and again,
> until I’m happy with the way it looks and performs.
>
> Hope this helps!
>
>
> Alberto.
>
>
>
> ----- Original Message -----
> From: “Else Kluger”
> To: “Windows System Software Devs Interest List”
>
> Sent: Friday, December 09, 2005 6:35 AM
> Subject: [ntdev] Driver Unit Tests
>
>
>> Hi,
>>
>> has anyone of you experience with unit testing in kernel mode
>> ?
>> I feel quite comfortable (and nearly secure) with Driver
>> Verifier,
>> having extensive registry configurable debug logs
>> and running Prefast from time to time.
>> And I have no idea how to mock OS functionality in a way to
>> make
>> it feel real for the driver.
>>
>> best regards
>> Else
>>
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: unknown lmsubst tag
argument: ‘’
To unsubscribe send a blank email to
xxxxx@lists.osr.com
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
NOD32 1.1317 (20051209) Information
This message was checked by NOD32 antivirus system.
http://www.eset.com
Thank you Banks for the link and Alberto for the description of how you do
it.
I started programming in assembler 17 years ago, and do driver
development for 6 years now. (Not asm, but /FA is always on.)
My way of thinking is probably near to yours Alberto, but that’s not really
what I
wanted to discuss.
I’d like to know if there’s anybody out there who did write unit tests for
his or her
driver, found it useful AND could really improve the driver with affordable
effort ?
best regards
Else
|---------±-------------------------------->
| | Else Kluger |
| | | | e> |
| | Sent by: |
| | bounce-228975-16691@li|
| | sts.osr.com |
| | |
| | |
| | 12/09/2005 11:35 AM |
| | Please respond to |
| | “Windows System |
| | Software Devs Interest|
| | List” |
|---------±-------------------------------->
>-----------------------------------------------------------------------------------------------------------|
| |
| To: “Windows System Software Devs Interest List” |
| cc: |
| Subject: [ntdev] Driver Unit Tests (Unsigned Mail) |
>-----------------------------------------------------------------------------------------------------------|
Hi,
has anyone of you experience with unit testing in kernel mode ?
I feel quite comfortable (and nearly secure) with Driver Verifier,
having extensive registry configurable debug logs
and running Prefast from time to time.
And I have no idea how to mock OS functionality in a way to make
it feel real for the driver.
best regards
Else
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@utimaco.de
To unsubscribe send a blank email to xxxxx@lists.osr.com
“Else Kluger” wrote in message news:xxxxx@ntdev…
> I’d like to know if there’s anybody out there who did write unit tests for
> his or her
> driver, found it useful AND could really improve the driver with
> affordable
> effort ?
Well it depends on your definition of unit tests (this has recently with
certain programming methodologies changed meaning somewhat). In many of my
drivers, I create a test harness for various service layers, and have
detailed logging of the parameters to those layers. This can allow me to
easily reproduce a problem, and identify the layer it occurs in.
One example of this was I had a driver that had to store a large amount of
directory like data. At the lowest level I created an AVL tree module (the
tree had to be position independant so I could not use the kernel or off the
self code). I then created a test harness to drive just this module, and
wrote a number of input scripts to test this. Above this was a directory
services module, that used the AVL support, again I created a harness to
exerecise this code (with the AVL linked in). Finally, there was the
actual exported interface, that called the directory module, this had an
additional harness to test with. By using these three harnesses, I was
quickly able to isolate and find the bugs. All of this went to the customer
as part of the package, and it was used there.
A different test example was a bus driver that multiplexed between two
devices (seperate driver) below it with a complex memory based transactional
interface. I created a simulation that used DbgPrompt to handle a number of
the common events (including loss of one of the devices), then developed a
test harness above the bus driver to exercise it. This driver, the driver
below it and a simple test driver above had a three day integration period.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply
I’ve avoided this thread because the correct process seems sort of
obvious. What Don said. Build test points and hooks into your code and
build test applications to drive them. Verify that you have covered most
or all of the functions your driver performs. Use the available tools
like DV, SDV etc to cover the ordinary functionality and use your built
in hooks and test apps for the other functions, the ones that make this
driver unique.
If you have a complex abstract function - such as Don’s AVL tree module
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn
Sent: Monday, December 12, 2005 9:45 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Driver Unit Tests
“Else Kluger” wrote in message
news:xxxxx@ntdev…
> I’d like to know if there’s anybody out there who did write unit tests
for
> his or her
> driver, found it useful AND could really improve the driver with
> affordable
> effort ?
Well it depends on your definition of unit tests (this has recently with
certain programming methodologies changed meaning somewhat). In many of
my
drivers, I create a test harness for various service layers, and have
detailed logging of the parameters to those layers. This can allow me
to
easily reproduce a problem, and identify the layer it occurs in.
One example of this was I had a driver that had to store a large amount
of
directory like data. At the lowest level I created an AVL tree module
(the
tree had to be position independant so I could not use the kernel or off
the
self code). I then created a test harness to drive just this module,
and
wrote a number of input scripts to test this. Above this was a
directory
services module, that used the AVL support, again I created a harness to
exerecise this code (with the AVL linked in). Finally, there was the
actual exported interface, that called the directory module, this had an
additional harness to test with. By using these three harnesses, I was
quickly able to isolate and find the bugs. All of this went to the
customer
as part of the package, and it was used there.
A different test example was a bus driver that multiplexed between two
devices (seperate driver) below it with a complex memory based
transactional
interface. I created a simulation that used DbgPrompt to handle a
number of
the common events (including loss of one of the devices), then developed
a
test harness above the bus driver to exercise it. This driver, the
driver
below it and a simple test driver above had a three day integration
period.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
> yes I agree automating tests is easier said than done.
Why? BAT file with output redirected to a text file, which runs some tiny test
app built from 1 tiny C file, which calls the driver.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
> One example of this was I had a driver that had to store a large amount of
directory like data. At the lowest level I created an AVL tree module (the
tree had to be position independant so I could not use the kernel or off the
self code). I then created a test harness to drive just this module, and
wrote a number of input scripts to test this.
I also did exactly the same in one of my projects 
yes, with AVL tree too.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Hi,
it is indeed obvious that everybody including myself uses his / her own
way of testing, appropriate to the situation or ability we want to test and
our own preferences. This includes tools like DV, tiny test tools we write
ourselves, single stepping thru the code …
The purpose of unit testing as I understood is to provide a common
testframe that everybody (quality assurance for automated testing or
the person that starts an overall build process) is able to use.
Quality assurance might use our selfwritten tools or DV too (though I
could not yet convince them to do so), while the integration into the
overall build process requires things similar to what Banks in his answer
described:
< For the debug IOCTLS the rest of your driver code will work normal just
that they have a pointer to the fake OS object which has methods that
merely check bounds/parameter correctness
(also known as environment testing). >
And my question refers to the 2nd part …
Because I doubt that running checks for parameter correctness and this is
what unit tests are mainly doing will really help to improve complex code.
I’m checking parameters, sometimes I’m even paranoid. But then I have
trouble
with sync issues (forgetting a spinlock or so), or I still bring the OS
down
because of some issue I have never ever thought of or simply do not know
yet:
In earlier days of plug and play my driver sometimes failed because a USB
stick
said “well, I’m a removable disk” and when I wanted to know its geometry it
acted
like a washing machine. Or the system crashed because some driver below
mine
expected to be the highest in the stack.
This is getting less and less, and gradually I have workarounds for the
things I
know - but to know them I had to see them in a live system !
Not in my dreams I could foresee or avoid such things and catch them in
unit tests.
And exactly this last sentence of mine was the intention to create this
thread and
ask if your experience is any different.
You see, the idea was “maybe experts can convince me”…
regards
Else
|---------±-------------------------------->
| | “Roddy, Mark” |
| | | | m> |
| | Sent by: |
| | bounce-229300-16691@li|
| | sts.osr.com |
| | |
| | |
| | 12/12/2005 04:31 PM |
| | Please respond to |
| | “Windows System |
| | Software Devs Interest|
| | List” |
|---------±-------------------------------->
>-----------------------------------------------------------------------------------------------------------|
| |
| To: “Windows System Software Devs Interest List” |
| cc: |
| Subject: RE: [ntdev] Driver Unit Tests (Unsigned Mail) |
>-----------------------------------------------------------------------------------------------------------|
I’ve avoided this thread because the correct process seems sort of
bvious. What Don said. Build test points and hooks into your code and
build test applications to drive them. Verify that you have covered most
or all of the functions your driver performs. Use the available tools
like DV, SDV etc to cover the ordinary functionality and use your built
in hooks and test apps for the other functions, the ones that make this
driver unique.
If you have a complex abstract function - such as Don’s AVL tree module
- build that thing in user mode and debug it up there where life is a
whole lot easier and where there exists a vast array of tools, ranging
from free to amazingly expensive, that will be happy to inform you just
what sort of crappy code you write and how badly it behaves. Then when
it is actually verified as less than crappy code, move it into your
driver, and of course test it again.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn
Sent: Monday, December 12, 2005 9:45 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Driver Unit Tests
“Else Kluger” wrote in message
news:xxxxx@ntdev…
> I’d like to know if there’s anybody out there who did write unit tests
for
> his or her
> driver, found it useful AND could really improve the driver with
> affordable
> effort ?
Well it depends on your definition of unit tests (this has recently with
certain programming methodologies changed meaning somewhat). In many of
my
drivers, I create a test harness for various service layers, and have
detailed logging of the parameters to those layers. This can allow me
to
easily reproduce a problem, and identify the layer it occurs in.
One example of this was I had a driver that had to store a large amount
of
directory like data. At the lowest level I created an AVL tree module
(the
tree had to be position independant so I could not use the kernel or off
the
self code). I then created a test harness to drive just this module,
and
wrote a number of input scripts to test this. Above this was a
directory
services module, that used the AVL support, again I created a harness to
exerecise this code (with the AVL linked in). Finally, there was the
actual exported interface, that called the directory module, this had an
additional harness to test with. By using these three harnesses, I was
quickly able to isolate and find the bugs. All of this went to the
customer
as part of the package, and it was used there.
A different test example was a bus driver that multiplexed between two
devices (seperate driver) below it with a complex memory based
transactional
interface. I created a simulation that used DbgPrompt to handle a
number of
the common events (including loss of one of the devices), then developed
a
test harness above the bus driver to exercise it. This driver, the
driver
below it and a simple test driver above had a three day integration
period.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
“The purpose of unit testing as I understood is to provide a common
testframe that everybody (quality assurance for automated testing or
the person that starts an overall build process) is able to use.”
[Roddy, Mark]
Hmmm… I think of unit testing as the lowest level testing done.
Typically it is done by the developer in an ad hoc fashion (although
some common tools or methodologies may be used) and the intent is to
attempt to exercise to some level of completeness all new code. It is
‘white box’ testing that uses knowledge of the code to get at error
paths that might be very difficult to provide external tests for.
Typically test hooks and debuggers are used for fault insertion. I never
expect anyone else to use anything resembling a test framework that
might fall out of the unit testing I do, although it has happened. That
is just an unintended byproduct.
Generally I unit test as I code, adopting the model that there should
always be a working prototype and it should always be under test for the
entire development process.