KMDF - Regarding WDFSPINLOCK and acquiring at DISPATCH_LEVEL

NTDEV
1

Out of curiosity, why does KMDF remove the equivalent of
KeAcquireXxxSpinLockAtDpc() and KeyReleaseXxxSpinLockAtDpc()? Am I missing
something obvious? (probably!)

NDIS6 goes to great pains (seemingly, mostly *our* pain) to provide
mechanisms to leverage design-time knowledege or IRQL yet an NDIS/KMDF
driver apparently needs to eschew WDFSPINLOCK for NDIS_SPIN_LOCK (or
KSPINLOCK) to leverage the ‘hints’.

Were exposing KMDF equivalents for XxxAtDpc deemed ‘too dangerous’ or
something or is all of the hullablaloo in NDIS6 trying to *not* ‘raise’ IRQL
from DISPATCH_LEVEL->DISPATCH_LEVEL simply not worth the performance gain?

Thanks,
-dave

2

> NDIS6 goes to great pains (seemingly, mostly *our* pain) to

provide mechanisms to leverage design-time knowledege or IRQL

Where is the “pain” here??? It just gives you a chance to avoid unnecessary IRQL-related calls by providing the flag (which, btw, you can just ignore if you want -NDIS does not force you to rely upon it, does it ) that indicates whether IRQL is elevated, so that you can avoid unnecessary calls to IRQL-related functions.

Concerning your actual question, I’ve got my own theory. I think KMDF adds not-so-thin extra layer, so that it just does not make sense to be bothered about few extra cycles under these circumstances - idiomatically, it does not really make sense to be bothered about wasting few secs while doing an operation that takes at least an hour to accomplish. This is just my theory - I may well be wrong here…

Anton Bassov

3

We did not say to hell with it and not add such an API b/c why save a few when we are burning many. That would be stupid. When people say that KMDF adds extra cycles to a driver, I always ask what they think we added compared to a WDM driver. Typically a WDM driver takes short cuts which can blow up in the edge cases (like how the wdm toaster sample tracks pnp state) , but if you wrote a fully functioning WDM driver with the same (I would say advanced) feature set, the cycles come out to about the same.

We actually debated these 2 DDIs for quite awhile, in the end it was decided that the DPC level versions were used incorrectly more often than not and confusing to the end user. They are also a bit harder to verify with static tools. In the end, if you need the At/FromDpcLevel versions, use a KSPIN_LOCK instead.

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Tuesday, January 15, 2008 5:49 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] KMDF - Regarding WDFSPINLOCK and acquiring at DISPATCH_LEVEL

NDIS6 goes to great pains (seemingly, mostly *our* pain) to
provide mechanisms to leverage design-time knowledege or IRQL

Where is the “pain” here??? It just gives you a chance to avoid unnecessary IRQL-related calls by providing the flag (which, btw, you can just ignore if you want -NDIS does not force you to rely upon it, does it ) that indicates whether IRQL is elevated, so that you can avoid unnecessary calls to IRQL-related functions.

Concerning your actual question, I’ve got my own theory. I think KMDF adds not-so-thin extra layer, so that it just does not make sense to be bothered about few extra cycles under these circumstances - idiomatically, it does not really make sense to be bothered about wasting few secs while doing an operation that takes at least an hour to accomplish. This is just my theory - I may well be wrong here…

Anton Bassov

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

4

Thanks Doron.

We actually debated these 2 DDIs for quite awhile, in the end it was
decided
that the DPC level versions were used incorrectly more often than not and
confusing to the end user. They are also a bit harder to verify with
static tools. In the end, if you need the At/FromDpcLevel versions,
use a KSPIN_LOCK instead.

I suspected that was the explanation behind the omission.

In an NDIS/KMDF miniport, then, one has access to NDIS, KMDF, and WDM(NT)
primitives and DDIs. It is my understanding that WHQL is (will be)
requiring NDIS/KMDF for SW NDIS Miniports. I can see that if an NDIS
miniport is layered over a message (IRP) based bus like USB how this could
be a very good thing. I am not sure how it matters very much in a Miniport
that accesses PDO/BUS functionality via IRP_MN_QUERY_INTERFACE and direct
function pointers. The opportunity to leverage KMDF extensively is lessened
and what I perceive as the benefit of such a (WHQL) policy seems to be
diminished.

Again, I am probably missing some point and I make this comments fully
expecting that to turn out to be the case!

-dave

5

Doron,

Typically a WDM driver takes short cuts which can blow up in the edge cases
(like how the wdm toaster sample tracks pnp state) , but if you wrote a fully
functioning WDM driver with the same (I would say advanced) feature set, the
cycles come out to about the same.

Sorry, but the above statement looks really funny. According to MSFT side of a story, overhead that using of KMDF implies is not that significant, and I know that you will never say otherwise, no matter what and no matter how. However, it looks like in this particular case you went a way too far - you started arguing against both common sense and math…

KMDF does not have its own *actual* functionality, does it??? Instead, it just acts as a wrapper for the system exports that give an actual functionality to your driver. Therefore, if you want WDM and KMDF versions of your driver to be identical in terms of functionality, they have to make exactly the same system calls - the only difference is that WDM driver makes them directly while KMDF one goes via an extra layer, i.e. KMDF wrapper.

It is understandable that that more complex and advanced a driver is, the more system calls it makes. If calling a kernel export takes X cycles ( WDM drivers call kernel exports directly, don’t they) and KMDF wrapper adds Y extra cycles, by making N calls to KMDF functions, your are going to get (X+Y)*N cycles, as opposed to X*N cycles if you called kernel exports directly. Therefore, according to the basic rules of math, the higher the value of N, the more significant overhead becomes.

Therefore, according to the basic logic and math, the move advanced your driver is, the less beneficial use of KMDF becomes. However, you seem to claim exactly the opposite (which is hardly surprising, taking into account the fact that you are MSFT employee). The only thing left to find out is to which extent the above applies - after all, MSFT does not seem to be willing to disclose KMDF source, does it???

Anton Bassov

6

David R. Cattley wrote:

It is my understanding that WHQL is (will be) requiring NDIS/KMDF
for SW NDIS Miniports.

According to [1], NDIS-KMDF is only required for USB-based miniports. It specifically calls out NDIS miniports in general as NDIS-WDM.

[1] http://www.microsoft.com/whdc/driver/foundation/DrvRoadmap.mspx

7

Thanks Chris.

That makes more sense. I appreciate you pointing me to the latest policy
summary as I was working from WinHEC notes and rumor :wink:

-dave

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Wednesday, January 16, 2008 10:35 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] KMDF - Regarding WDFSPINLOCK and acquiring at
DISPATCH_LEVEL

David R. Cattley wrote:

It is my understanding that WHQL is (will be) requiring NDIS/KMDF for
SW NDIS Miniports.

According to [1], NDIS-KMDF is only required for USB-based miniports. It
specifically calls out NDIS miniports in general as NDIS-WDM.

[1] http://www.microsoft.com/whdc/driver/foundation/DrvRoadmap.mspx

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

8

KMDF does have actual functionality outside of its wrapper quality: a
default implementation of a complete integrated Pnp/power state
machine. One can implement a WDM driver that avoids ‘a complete
integrated Pnp/power state machine’ and that driver will indeed have
less overhead that a KMDF driver, it just won’t work correctly. The
complete WDM driver will have all of the KMDF provided functionality
minus the overhead of invoking KMDF functions that are simply
wrappers. That delta is basically negligible.

On Jan 16, 2008 10:24 AM, wrote:
> Doron,
>
> > Typically a WDM driver takes short cuts which can blow up in the edge cases
> > (like how the wdm toaster sample tracks pnp state) , but if you wrote a fully
> > functioning WDM driver with the same (I would say advanced) feature set, the
> > cycles come out to about the same.
>
>
> Sorry, but the above statement looks really funny. According to MSFT side of a story, overhead that using of KMDF implies is not that significant, and I know that you will never say otherwise, no matter what and no matter how. However, it looks like in this particular case you went a way too far - you started arguing against both common sense and math…
>
>
>
>
>
>
> KMDF does not have its own actual functionality, does it??? Instead, it just acts as a wrapper for the system exports that give an actual functionality to your driver. Therefore, if you want WDM and KMDF versions of your driver to be identical in terms of functionality, they have to make exactly the same system calls - the only difference is that WDM driver makes them directly while KMDF one goes via an extra layer, i.e. KMDF wrapper.
>
>
>
> It is understandable that that more complex and advanced a driver is, the more system calls it makes. If calling a kernel export takes X cycles ( WDM drivers call kernel exports directly, don’t they) and KMDF wrapper adds Y extra cycles, by making N calls to KMDF functions, your are going to get (X+Y)N cycles, as opposed to XN cycles if you called kernel exports directly. Therefore, according to the basic rules of math, the higher the value of N, the more significant overhead becomes.
>
> Therefore, according to the basic logic and math, the move advanced your driver is, the less beneficial use of KMDF becomes. However, you seem to claim exactly the opposite (which is hardly surprising, taking into account the fact that you are MSFT employee). The only thing left to find out is to which extent the above applies - after all, MSFT does not seem to be willing to disclose KMDF source, does it???
>
> Anton Bassov
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>

–
Mark Roddy

9

> KMDF does have actual functionality outside of its wrapper quality:

a default implementation of a complete integrated Pnp/power state machine.

Yes, but this is not an *actual* functionality -again, it is a higher-level abstraction that relies upon the system calls that provide an *actual* functionality to it. I do see what you mean - if you choose WDM option you will have to implement this higher-level abstraction anyway, so that overhead *of this particular subset* of your driver is going to be exactly the same as if you were using KMDF. However, this is just a subset - it is not yet a whole driver, is it???

That delta is basically negligible.

This is exactly what we have to discover yet - after all, MSFT has not disclosed KMDF source in so far, did it??? Therefore, the question is still open…

Anton Bassov

10

You seem to start out saying a complete default pnp/power state
machine is not actual functionality and then agree that it is. I’ll
take that to be a stream of consciousness essay where you end up
agreeing with me that obviously this is actual complex functionality
that takes a non-negligible amount of processor cycles.

One of the motivations for KMDF was that in fact 90% or so (number
PFA) of production WDM drivers did not implement anywhere near a
complete pnp/power state machine. Instead they implemented a minimal
and incomplete pnp/power state machine that managed to get them
through whatever qualification tests they needed to pass. In addition
the state machine implemented was typically broken in all sorts of
ways beyond being incomplete. So once again, a full WDM driver, of
which there were perhaps NONE, and a typical KMDF driver would have
almost the same overhead - as the difference would be the negligible
KMDF wrapper-like functions.

If you want concrete numbers why not conduct a test? There are several
comparable WDM/KMDF driver pairs - measure their performance. My guess
is that measurement error will overwhelm any real difference.

On Jan 16, 2008 11:41 AM, wrote:
> > KMDF does have actual functionality outside of its wrapper quality:
> > a default implementation of a complete integrated Pnp/power state machine.
>
> Yes, but this is not an actual functionality -again, it is a higher-level abstraction that relies upon the system calls that provide an actual functionality to it. I do see what you mean - if you choose WDM option you will have to implement this higher-level abstraction anyway, so that overhead of this particular subset of your driver is going to be exactly the same as if you were using KMDF. However, this is just a subset - it is not yet a whole driver, is it???
>
> > That delta is basically negligible.
>
> This is exactly what we have to discover yet - after all, MSFT has not disclosed KMDF source in so far, did it??? Therefore, the question is still open…
>
>
> Anton Bassov
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>

–
Mark Roddy

11

Exactly… Lets not discuss the KMDF overhead until someone brings numbers that back the overhead claim up. It was said that KMDF doesn’t provide any functionality? Well behond the full PNP/Power state machine, the built in design patterns it allows for(Parent/Child,Destroy/Cleanup,etc.) is quite powerful and would also fall into the catagory of functionality. My 2 cents.

12

Mark,

One of the motivations for KMDF was that in fact 90% or so (number PFA)
of production WDM drivers did not implement anywhere near a complete
pnp/power state machine.

Yes, but why do you think we need so MSFT-like solution to the problem, i.e. a framework???
Would not it be better to sort it out in a traditional C-like way, i.e. with a library??? For example, if you don’t want all the drudgery that PnP/power state handling involves, you could just delegate this job (or just a part of it - I believe you should have the right to choose) to the library, and that’s it. Since when are the libraries supposed to impose any rules upon their client code???

If you want concrete numbers why not conduct a test?

Actually, it is not that simple. In which context should we run our tests ??? Look - overhead that may be just funny at PASSIVE_LEVEL may be already not-so-funny if it happens within a section protected by spinlock that is always in high demand. Therefore, exactly the same test may produce rather different results on the machine with Pentium with no support for HT (i.e.
just 1 processor) and the one with 2 Quad Cores (i.e. already 8 logical processors) - the difference that is not-so-significant in the former case may be already much more significant
in the latter one…

Anton Bassov

13

Anton… You’re out of control, dude. Do you think this thread started out with a question about KMDF overhead? Or did you MAKE it about KMDF overhead, whether KMDF has any actual functionality, library versys wrapper, you aparent distaste for Frameworks, Microsoft, etc… without apparently taking the time to learn anything about KMDF.

I told you nicely just recently: These thread-hijacking rants are not healthy for the list and are not helpful for the vast majority of members in the community.

Now I’ll tell you as clearly as I can: STFU.

I’m tired of getting email complaining about your posts.

This time: Public warning.
Next time: Private moderation.
Time after next: Public bannification.

You have a problem with that take it off-line with me.

Peter
OSR

14

You are of course hijacking another thread.

“Since when are the libraries supposed to impose any rules upon their
client code?”

I actually don’t know when this started, but it was around the time
that event driven object oriented software became pervasive, that
‘libraries’ started imposing lots of rules and structures on their
clients beyond ‘call this function with these parameters’. That would
be at least 20 years or so ago.

You don’t like frameworks. Ok, that opinion is shared by others here
as well. The structure imposed by KMDF seems to me to be appropriate
to the object oriented event driven complex state machine of a
pnp/power enabled driver in the Windows OS. Having written my own
version of something like KMDF years before when W2K arrived, I
actually cannot quite picture the non-object oriented non-event driven
non-callback enabled non-framework structured library that would
accomplish ‘take care of all of the boilerplate pnp/power stuff for me
unless I opt out, I’ll just handle the device specific stuff’. Maybe
you should write that library?

On Jan 16, 2008 5:05 PM, wrote:
> Mark,
>
> > One of the motivations for KMDF was that in fact 90% or so (number PFA)
> > of production WDM drivers did not implement anywhere near a complete
> > pnp/power state machine.
>
> Yes, but why do you think we need so MSFT-like solution to the problem, i.e. a framework???
> Would not it be better to sort it out in a traditional C-like way, i.e. with a library??? For example, if you don’t want all the drudgery that PnP/power state handling involves, you could just delegate this job (or just a part of it - I believe you should have the right to choose) to the library, and that’s it. Since when are the libraries supposed to impose any rules upon their client code???
>
> > If you want concrete numbers why not conduct a test?
>
> Actually, it is not that simple. In which context should we run our tests ??? Look - overhead that may be just funny at PASSIVE_LEVEL may be already not-so-funny if it happens within a section protected by spinlock that is always in high demand. Therefore, exactly the same test may produce rather different results on the machine with Pentium with no support for HT (i.e.
> just 1 processor) and the one with 2 Quad Cores (i.e. already 8 logical processors) - the difference that is not-so-significant in the former case may be already much more significant
> in the latter one…
>
> Anton Bassov
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>

–
Mark Roddy

15

Peter Viscarola (OSR) wrote:

Anton… You’re out of control, dude. Do you think this thread started
out with a question about KMDF overhead? Or did you MAKE it about
KMDF overhead, whether KMDF has any actual functionality, library
versys wrapper, you aparent distaste for Frameworks, Microsoft,
etc… without apparently taking the time to learn anything about KMDF.

For what it’s worth, this all seemed to start around the time Anton started getting in fights with people over what it meant to be an MVP, whether or not this means you are a Microsoft shill, whether or not being an MVP means anything, and so on.

I’ve only been on this list for a year or two but this was the first time I saw personal attacks (from Mark Roddy, et. al). Maybe Anton feels this is the best way to fight back, just by being obnoxious in general (and gaming Mark into replying yet again, thus adding to the noise).

16



Peter,

> Do you think this thread started out with a question about KMDF overhead?

Actually, this thread started off asking why KMDF removed the equivalent of KeAcquireXxxSpinLockAtDpc() and KeyReleaseXxxSpinLockAtDpc(), which gives KMDF version some certain overhead against WDM if spinlock is acquired at DPC level (I really hope you are not going to argue about it, are you). The only question is how significant this overhead is…

> Or did you MAKE it about KMDF overhead, whether KMDF
> has any actual functionality, library versys wrapper,

Sorry, but how are we supposed to discuss things (unless by the term “discussion” you understand simply chanting “Long live KMDF!!! Long live MSFT!!!”, without going anywhere close to speaking about the technical pros and cons of any given technology) ???

> you aparent distaste for Frameworks, Microsoft, etc

Actually, on this particular thread I avoided speaking about this topic completely. Could you please show me where I said anything about it???

> I told you nicely just recently: These thread-hijacking rants are not healthy
> for the list and are not helpful for the vast majority of members in the community.

Did I move anywhere away from the topic of a discussion??? If you think so, please look at
the thread title again - it is “Regarding WDFSPINLOCK and acquiring at DISPATCH_LEVEL”.
Did I move anywhere away from it??? As I can see, saying that acquiring a spinlock at DPC level is more expensive in KMDF rather than in WDM is, in your opinion, a major crime. You’ve got quite interesting understanding of freedom of thought!!!.

Concerning thread hijacking, ironically, it happens to be you who actually hijacked it - your whole post is “ad hominem” argument…

> I’m tired of getting email complaining about your posts.

I somehow presume the vast majority of them arrives from MSFT and their “aficinados”, right?

> This time: Public warning.
> Next time: Private moderation.
> Time after next: Public bannification.

Well, you’ve got quite interesting reaction to criticism of MSFT technology…

> You have a problem with that take it off-line with me.

However, you somehow decided to do it in public, effectively hijacking the thread…

Anton Bassov

17

> Maybe you should write that library?

Actually, it is an interesting idea - to write a library and to disclose its source to everyone who may be interested in it. I will give it a thought - I have a feeling that doing it in plain-C style must be a feasible task . Thank you for suggestion…

Anton Bassov

18

Peter,

Please look at the post that below - it appeared immediately after yours:

[begin quote]

Peter Viscarola (OSR) wrote:

Anton… You’re out of control, dude. Do you think this thread started
> out with a question about KMDF overhead? Or did you MAKE it about
KMDF overhead, whether KMDF has any actual functionality, library
versys wrapper, you aparent distaste for Frameworks, Microsoft,
etc… without apparently taking the time to learn anything about KMDF.

For what it’s worth, this all seemed to start around the time Anton started getting in fights with people over what it meant to be an MVP, whether or not this means you are a Microsoft shill, whether or not being an MVP means anything, and so on. I’ve only been on this list for a year or two but this was the first time I saw personal attacks (from Mark Roddy, et. al). Maybe Anton feels this is the best way to fight back, just by being obnoxious in general (and gaming Mark into replying yet again, thus adding to the noise).

[end quote]

Do you have any doubts that it was *you* who hijacked this particular thread???

Please note that I don’t want to reply to a poster in order to avoid a stupid and absolutely unnecessary discussion…

Anton Bassov