“Tim Roberts” wrote in message news:xxxxx@ntdev…
> xxxxx@osr.com wrote:
>> So, your point is… Getting logo’ed is annoying and arbitrary? Well,
>> yeaaahhhh. It’s not like it’s EVER been about technical quality or
>> correctness.
>>
>> Bottom line: YOU want THEIR logo to help sell YOUR products… so YOU
>> have to do what THEY want.
>>
>> Sorry, but that’s all it really comes down to,
>>
>
> No, there really is one more important aspect to this, and that is the
> dreaded “CAUTION! This driver is unsigned; it was probably written by
> crazed paranoid crack addicts, and its installation will probably result
> in the end of Western Civilization” warning. The logo signature is the
> only way to eliminate that. It’s more having Microsoft help sell my
> products, it’s Microsoft actively disparaging my products unless I jump
> through the “annoying and arbitrary” hoops.
I had discussions at WinHEC about the lousy state of driver logo tests, and
driver testing in general. I pointed out that compliance testing for
languages and UNIX OS’es was an open and respected process. And that the
process is set up so that once a core of the item being tested works, you
can run the tests and get a pretty good idea of all the remaining problems
(unlike the logo tests that fail on the first bug).
I’ve run groups using the compliance tests, and normally I would schedule 2
days at the end of the work, to rerun the complete set of tests, and submit
the results. Compare this with a customer I was at yesterday that is
setting aside 4 months to take a running system and get DTM working and
then run the tests (and they have logo’d some of the drivers in the past!).
Now we all know driver testing is hard. Look at the NT Insider on testing
that came out a while back. There was very little on writing tests. So I
want to ask a question, what do people on this group do to test a driver,
and what would you like to do but not have enough time?
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply
I assume people from Microsoft read this. I further assume the hassle of
driver signing is mainly because of DRM, i.e. Microsoft formally being able
to revoke their signature under a third party at their will (more on this at
http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.html). Now may I
suggest the following:
- Those wanting the WHQL logo may stick to the current procedure with DTM or
whatever. - Those believing the logo isn’t worth the costs (especially those with
software drivers who are not eligible anyway) just get a WHQL signature
without the testing (except of maybe the drivers directly related to the DRM
stack).
Both scenarios have equal effect in terms of DRM, so Microsoft and whoever
else is involved should be satisfied. Of course, the stupid prompt should
not appear as long as there is a WHQL signature, whether tested or not.
Cheers
“Don Burn” wrote in message news:xxxxx@ntdev…
>
> “Tim Roberts” wrote in message news:xxxxx@ntdev…
>> xxxxx@osr.com wrote:
>>> So, your point is… Getting logo’ed is annoying and arbitrary? Well,
>>> yeaaahhhh. It’s not like it’s EVER been about technical quality or
>>> correctness.
>>>
>>> Bottom line: YOU want THEIR logo to help sell YOUR products… so YOU
>>> have to do what THEY want.
>>>
>>> Sorry, but that’s all it really comes down to,
>>>
>>
>> No, there really is one more important aspect to this, and that is the
>> dreaded “CAUTION! This driver is unsigned; it was probably written by
>> crazed paranoid crack addicts, and its installation will probably result
>> in the end of Western Civilization” warning. The logo signature is the
>> only way to eliminate that. It’s more having Microsoft help sell my
>> products, it’s Microsoft actively disparaging my products unless I jump
>> through the “annoying and arbitrary” hoops.
>
>
> I had discussions at WinHEC about the lousy state of driver logo tests,
> and driver testing in general. I pointed out that compliance testing for
> languages and UNIX OS’es was an open and respected process. And that the
> process is set up so that once a core of the item being tested works, you
> can run the tests and get a pretty good idea of all the remaining problems
> (unlike the logo tests that fail on the first bug).
>
> I’ve run groups using the compliance tests, and normally I would schedule
> 2 days at the end of the work, to rerun the complete set of tests, and
> submit the results. Compare this with a customer I was at yesterday that
> is setting aside 4 months to take a running system and get DTM working and
> then run the tests (and they have logo’d some of the drivers in the
> past!).
>
> Now we all know driver testing is hard. Look at the NT Insider on testing
> that came out a while back. There was very little on writing tests. So I
> want to ask a question, what do people on this group do to test a driver,
> and what would you like to do but not have enough time?
>
>
> –
> Don Burn (MVP, Windows DDK)
> Windows 2k/XP/2k3 Filesystem and Driver Consulting
> Website: http://www.windrvr.com
> Blog: http://msmvps.com/blogs/WinDrvr
> Remove StopSpam to reply
>
>
> -----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-289489-
xxxxx@lists.osr.com] On Behalf Of Don Burn
Sent: Thursday, June 07, 2007 4:31 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Driver testing: was MmGetSystemRoutineAddress
BugCheck?“Tim Roberts” wrote in message news:xxxxx@ntdev…
> > xxxxx@osr.com wrote:
> >> So, your point is… Getting logo’ed is annoying and arbitrary?
> Well,
> >> yeaaahhhh. It’s not like it’s EVER been about technical quality or
> >> correctness.
> >>
> >> Bottom line: YOU want THEIR logo to help sell YOUR products… so
> YOU
> >> have to do what THEY want.
> >>
> >> Sorry, but that’s all it really comes down to,
> >>
> >
> > No, there really is one more important aspect to this, and that is
> the
> > dreaded “CAUTION! This driver is unsigned; it was probably written by
> > crazed paranoid crack addicts, and its installation will probably
> result
> > in the end of Western Civilization” warning. The logo signature is
> the
> > only way to eliminate that. It’s more having Microsoft help sell my
> > products, it’s Microsoft actively disparaging my products unless I
> jump
> > through the “annoying and arbitrary” hoops.
>
>
> I had discussions at WinHEC about the lousy state of driver logo tests,
> and
> driver testing in general. I pointed out that compliance testing for
> languages and UNIX OS’es was an open and respected process. And that
> the
> process is set up so that once a core of the item being tested works,
> you
> can run the tests and get a pretty good idea of all the remaining
> problems
> (unlike the logo tests that fail on the first bug).
>
> I’ve run groups using the compliance tests, and normally I would
> schedule 2
> days at the end of the work, to rerun the complete set of tests, and
> submit
> the results. Compare this with a customer I was at yesterday that is
> setting aside 4 months to take a running system and get DTM working and
> then run the tests (and they have logo’d some of the drivers in the
> past!).
>
> Now we all know driver testing is hard. Look at the NT Insider on
> testing
> that came out a while back. There was very little on writing tests.
> So I
> want to ask a question, what do people on this group do to test a
> driver,
> and what would you like to do but not have enough time?
>
Don,
I’ve just spent about a man-month using DTM to get an unclassified Driver
Reliability signature for a RNDIS driver for Windows XP and Vista. This is
the simplest possible signature, and the only drivers involved are Microsoft
in-box RNDIS drivers.
Quite a chore. I cannot imagine what pain there must be for those who have
to sign more complex devices and drivers!
Of the many issues that I have with DTM one stands out:
It (currently) is not useful in the driver development process.
If it passes, then you can get a signature. If it doesn’t, then the DTM test
results may not give any indication of the cause of the failure and, in
fact, it is possible that the failure is covered by an Errata (that you
don’t know about…) that identifies the error as one to be ignored.
The biggest testing problems that I (currently) have concern the edge cases
such as standby/hibernation, device removal, etc. when I am actively
stimulating my device in the way it is actually intended to be used. Here I
would like to see tools that I can call or integrate with my own test
framework to test these edge conditions systematically.
I know there are tools in the DDK that can be used to initiate these
conditions, and possibly I just haven’t tried hard enough to glue them into
my own tests.
I need to run real data through my drivers and systematically test these
edge conditions as realistically as possible.
Just one of many thoughts. I think it’s a business opportunity, actually…
Thomas F. Divine