Baffling issue with Queued spin-locks

Chris_Aseltine · January 10, 2013, 12:16pm

anton bassov wrote:

Yes, Mr. Kyler is just incredible in absolutely all respects…

Wasn’t he the guy who said you supported “genocide” because of your email address being soviet_bloke@? Or was that Mr. Terhell?

Peter_Viscarola_OSR · January 10, 2013, 3:28pm

Yes, THAT was Mr. Kyler.

Peter
OSR

OSR_Community_User · January 10, 2013, 8:05pm

Thanks Peter (and Phil and Kenny and Martin)! This helps.

About what Peter said:

A whitepaper gives you the opportunity to download a big chunk of information (50 or 80 pages) on one topic that reads like a story.
The key is, ALL the information for the whitepapers NEEDS to be rewritten and integrated back into the WDK. It’s just too hard to find otherwise.

We were kind of hoping to do one or the other. Here’s the thing: Generally while we’re in the process of documenting a new OS version, the feature team PMs are handing out whitepapers to NDA partners. At some point in the release cycle it becomes OK to publish the information in those whitepapers.

Here’s what I’m thinking: At that point, we can take each whitepaper, clean it up a bit, and insert it into the WDK docs. When time permits, we can rewrite the information from the whitepaper and assimilate it into the docs properly.

Once the assimilating is done, it would be great if we could delete the whitepaper. Otherwise, we end up with the same information in 2 places in the docs, 1 of which will grow stale over time. (We simply don’t have the resources to maintain whitepapers.)

Questions:

Is it OK to insert whitepapers into the WDK docs (preferably as a single “page” - even if it’s actually 80 printed pages)? Or does anyone really need them in downloadable form?
Once the information is assimilated into the WDK docs, is it OK to delete the whitepaper? Or is it better if we keep it around? (We’d have to put some kind of verbiage at the top declaring that it’s archived material that won’t be updated.)

Cheers,

–Diane

P.S.: I’m the one who created the 32-bit driver porting guide (http://msdn.microsoft.com/en-us/library/windows/hardware/ff559923.aspx) many years ago (maybe 2001?) at the request of David Golds.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Thursday, January 10, 2013 6:32 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Baffling issue with Queued spin-locks

There’s a place for whitepapers. That is, a detailed technical exploration of a topic for the community. This is useful, for example, when introducing a new technology (such as when MSI support was added to Windows) or when try to illustrate in a cogent end-to-end way some complex concept that the community is or will likely have trouble with. For example, “porting” 32-bit drivers to 64-bit. A whitepaper gives you the opportunity to download a big chunk of information (50 or 80 pages) on one topic that reads like a story.

The key is, ALL the information for the whitepapers NEEDS to be rewritten and integrated back into the WDK. It’s just too hard to find otherwise.

As a rule, I’d like to see the Remarks section limited to comments and guidance about a given DDI. I’d like to see the Remarks section POINT to conceptual sections in the WDK that explain overall concepts, and issues that apply to multiple DDIs. This appears to be the current approach, and I think it’s a good one. It’s not followed 100% consistently, but … you know… what is, right?

This may be a controversial idea, but I also don’t think the WDK needs to delve into ultimate depth on every issue. It’s find to say “In most cases, kernel developers should…”. For Queued Spin Locks, for example, it wouldn’t be unreasonable to provide guidance that Queued Spin Locks generally scale better in multiprocessor environments and provide equal or better performance to Executive Spin Locks. You can then note that in certain virtualized systems, particularly those running on Windows Server 2003 or earlier, Queue Spin Locks are likely to be less performant than Executive Spin Locks.

I’ll also take this opportunity to note that I think it’s *really* important that the WDK docs work harder to avoid providing guidance by making absolute pronouncements that are not literally correct. These sweeping generalizations appear all too frequently for my tastes in the WDK docs of the early 2000’s. In these cases, what is actually guidance is presented in the WDK as global truth. This is VERY confusing, and winds up diluting the WDK’s value. SOME things the WDK docs say you shouldn’t do are actually things you should NEVER do… other things the WDK say you shouldn’t do are things you shouldn’t do unless you’re clever, know what you’re doing, and follow the appropriate procedures. All this makes it difficult for less experienced kernel engineers who read the WDK which says “don’t do X”, and we more experienced folks have to explain “Well, they don’t really MEAN don’t do X… they mean that you shouldn’t do X unless you know what you’re doing and you’ve previously done A, B and C.”

Given the above, I think the docs shouldn’t shy away from qualifying statements such as “Most developers should…” or “In most drivers, the best approach will typically be…” – I know there are often “writing” issues with these qualifiers (WHICH developers should, how does the reader know if this applies to them, if MOST should then should we say who SHOULDN’T, if it’s good for MOST developers shouldn’t we just say EVERYONE should). But I think they are *really* important from an engineering standpoint. Guidance is merely guidance, not declaration of an archtectural precept. Those things need to be kept separate.

Of course, this ALSO takes us to the whole discussion about whether the WDK’s role is to present overarching architectural concepts and the DDI deatils and let people figure it out (the way the old DDK docs did back in the mid-90’s) or to present more specific task-based information and in so doing steer developers toward mainstream solutions and the associated DDIs (as has been the approach more recently). THAT, however, is a FAR longer discussion than can be had in a forum.

Peter
OSR

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Peter_Viscarola_OSR · January 10, 2013, 8:23pm

Hi Diane,

I think the process you describe would be just dandy.

In answer to your questions… just my opinion:

One big 80 printed page “page” – Fine with me.
This is a tough question, believe it or not. Should the whitepaper “go away”?? Probably, on balance, the best answer is that once the content is assimilated into the WDK docs, the whitepaper should go away. Those early whitepapers are often full of half-correct information (as I know too well, having written enough of them myself). You know… it WAS correct AT THE TIME, but when the code was all written, tested, and finalized not EVERYthing turned out out QUITE the way the whitepaper says. But SOME of the whitepaper is still accurate. It’s confusing, and it’s maddening. And having the whitepaper marked “archived… out of date… do not belive this” is not helpful. Because, you can clearly see SOME stuff that’s correct and useful. So, you don’t know WHAT to believe. There’s a super funny whitepaper that’s still floating around that talks about INF file values that you can use to set the priority of your ISR relative to other devices. It’s super funny because, to the best of my knowledge, the code for this feature was (thankfully) never implemented. I have to explain this to people from time to time. And I get the argument “but it’s in the documentation!!!” Arrrgh!

On the OTHER hand: It’s been my experience that when the whitepapers get incorporated or revised they sometimes get “additionally sanitized” to reflect some unintentionally unhelpful policy decision. Some stuff gets losts, intentionally or otherwise. And, to be clear: I *am* talking about differences from when the whitepaper is initially made public and later on. There are numerous examples of this, my favorite being the whitepaper on invokling ACPI methods where some content was removed and the examples were helpfully “cleaned-up” and “modernized” by converting them from WDM to KMDF.

ANyhow, I hope that’s helpful in some small way.

I hope others weigh-in with their opinions,

Peter
OSR

mm1 · January 10, 2013, 8:39pm

Thanks, Diane. Awesome work as usual.

Absolutely
Certainly deleting the whitepaper is a cleaner and more
conceptually appealing solution, but I’ve found old whitepapers to be
useful because they contain information that’s been removed from the
updates. As static entities, they’re generally free from this
problem. No point in labeling them as deprecated or whatever - that
won’t stop us.

Thanks again,

Mm

Sent from my Verizon Wireless 4G LTE DROID

Diane Olsen wrote:

Thanks Peter (and Phil and Kenny and Martin)! This helps.

About what Peter said:

> A whitepaper gives you the opportunity to download a big chunk of information (50 or 80 pages) on one topic that reads like a story.
> The key is, ALL the information for the whitepapers NEEDS to be rewritten and integrated back into the WDK. It’s just too hard to find otherwise.

We were kind of hoping to do one or the other. Here’s the thing:
Generally while we’re in the process of documenting a new OS version,
the feature team PMs are handing out whitepapers to NDA partners. At
some point in the release cycle it becomes OK to publish the
information in those whitepapers.

Here’s what I’m thinking: At that point, we can take each whitepaper,
clean it up a bit, and insert it into the WDK docs. When time permits,
we can rewrite the information from the whitepaper and assimilate it
into the docs properly.

Once the assimilating is done, it would be great if we could delete
the whitepaper. Otherwise, we end up with the same information in 2
places in the docs, 1 of which will grow stale over time. (We simply
don’t have the resources to maintain whitepapers.)

Questions:

1. Is it OK to insert whitepapers into the WDK docs (preferably as a
single “page” - even if it’s actually 80 printed pages)? Or does
anyone really need them in downloadable form?

2. Once the information is assimilated into the WDK docs, is it OK to
delete the whitepaper? Or is it better if we keep it around? (We’d
have to put some kind of verbiage at the top declaring that it’s
archived material that won’t be updated.)

Cheers,

–Diane

P.S.: I’m the one who created the 32-bit driver porting guide
(http://msdn.microsoft.com/en-us/library/windows/hardware/ff559923.aspx)
many years ago (maybe 2001?) at the request of David Golds.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@osr.com
Sent: Thursday, January 10, 2013 6:32 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Baffling issue with Queued spin-locks

There’s a place for whitepapers. That is, a detailed technical
exploration of a topic for the community. This is useful, for
example, when introducing a new technology (such as when MSI support
was added to Windows) or when try to illustrate in a cogent end-to-end
way some complex concept that the community is or will likely have
trouble with. For example, “porting” 32-bit drivers to 64-bit. A
whitepaper gives you the opportunity to download a big chunk of
information (50 or 80 pages) on one topic that reads like a story.

The key is, ALL the information for the whitepapers NEEDS to be
rewritten and integrated back into the WDK. It’s just too hard to
find otherwise.

As a rule, I’d like to see the Remarks section limited to comments and
guidance about a given DDI. I’d like to see the Remarks section POINT
to conceptual sections in the WDK that explain overall concepts, and
issues that apply to multiple DDIs. This appears to be the current
approach, and I think it’s a good one. It’s not followed 100%
consistently, but … you know… what is, right?

This may be a controversial idea, but I also don’t think the WDK needs
to delve into ultimate depth on every issue. It’s find to say “In
most cases, kernel developers should…”. For Queued Spin Locks, for
example, it wouldn’t be unreasonable to provide guidance that Queued
Spin Locks generally scale better in multiprocessor environments and
provide equal or better performance to Executive Spin Locks. You can
then note that in certain virtualized systems, particularly those
running on Windows Server 2003 or earlier, Queue Spin Locks are likely
to be less performant than Executive Spin Locks.

I’ll also take this opportunity to note that I think it’s really
important that the WDK docs work harder to avoid providing guidance by
making absolute pronouncements that are not literally correct. These
sweeping generalizations appear all too frequently for my tastes in
the WDK docs of the early 2000’s. In these cases, what is actually
guidance is presented in the WDK as global truth. This is VERY
confusing, and winds up diluting the WDK’s value. SOME things the WDK
docs say you shouldn’t do are actually things you should NEVER do…
other things the WDK say you shouldn’t do are things you shouldn’t do
unless you’re clever, know what you’re doing, and follow the
appropriate procedures. All this makes it difficult for less
experienced kernel engineers who read the WDK which says “don’t do X”,
and we more experienced folks have to explain “Well, they don’t really
MEAN don’t do X… they mean that you shouldn’t do X unless you know
what you’re doing and you’ve previously done A, B and C.”

Given the above, I think the docs shouldn’t shy away from qualifying
statements such as “Most developers should…” or “In most drivers,
the best approach will typically be…” – I know there are often
“writing” issues with these qualifiers (WHICH developers should, how
does the reader know if this applies to them, if MOST should then
should we say who SHOULDN’T, if it’s good for MOST developers
shouldn’t we just say EVERYONE should). But I think they are really
important from an engineering standpoint. Guidance is merely
guidance, not declaration of an archtectural precept. Those things
need to be kept separate.

Of course, this ALSO takes us to the whole discussion about whether
the WDK’s role is to present overarching architectural concepts and
the DDI deatils and let people figure it out (the way the old DDK docs
did back in the mid-90’s) or to present more specific task-based
information and in so doing steer developers toward mainstream
solutions and the associated DDIs (as has been the approach more
recently). THAT, however, is a FAR longer discussion than can be had
in a forum.

Peter
OSR

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Daniel_Terhell · January 10, 2013, 10:03pm

The problem that I am having with the separate docs is that they are
amazingly hard to find. You need to know in advance that a certain document
exists before you can possibly find it. A document index or hierarchy and
more links from the WDK to these documents would be most welcome. Another
problem with these separate white papers is because they are separate
downloadable entities, they are not easily updated so they often contain
information that has become FALSE over time. An example is the KMCS
walkthrough document that everyone still uses which now talks about
requirements and limitations that are no longer there.

Wasn’t he the guy who said you supported “genocide” because of your email
address being soviet_bloke@? Or was that Mr. Terhell?

Of course that was not me, such an unbelievably silly remark only suits an
idiot.

//Daniel

Prokash_Sinha · January 10, 2013, 10:24pm

What I’ve seen ( and did not like ) are -

Some of the old document were gone. IIRC, there were few PNP and
PCI related doc(s).
Quite a few of the links I’ve seen are broken.
As said earlier, very hard to find using search…

IMO, there is a TON of good informations ( but not easy to find ). The
white papers should not go away. A link from the wdk doc would be
great. Also some kind of version control, and indexing would be great.
But that might be too much to ask for, don’t know…

-pro

Phil_Barila · January 10, 2013, 10:45pm

+1 to what PeterGV et al said about this.

As I said before, WPs have their place, when they are used for a deep dive into a single topic, such as the previously mentioned 32->64-bit porting guide, KMCS walkthrough, and many others. That place, though, should be in the same body of content as all the rest, so it’s equally searchable (or not, as the case may be with some versions of DDK/WDK docs).

Although I don’t know if it’s possible, it would be really cool if future WPs took the form of conceptual exposition with links into the live doc tree. The “Programming Issues for 64-Bit Drivers” (http://msdn.microsoft.com/en-us/library/ff559923(v=vs.85).aspx) doc tree is sort of an embryonic form of that, but I’m envisioning a much more comprehensive documentation approach than that.

Then it just becomes a maintained part of the doc set, and is modified and pruned as necessary.

Thanks for asking, and for listening to the answers.

Phil

Phil Barila | Senior Software Engineer
720.881.5364 (w)
LogRhythm, Inc.
A LEADER 2012 SIEM Magic Quadrant
WINNER of SC Magazine’s 2012 SIEM Best Buy

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Diane Olsen
Sent: Thursday, January 10, 2013 6:05 PM
To: Windows System Software Devs Interest List
Subject: RE: RE:[ntdev] Baffling issue with Queued spin-locks

Thanks Peter (and Phil and Kenny and Martin)! This helps.

About what Peter said:

A whitepaper gives you the opportunity to download a big chunk of information (50 or 80 pages) on one topic that reads like a story.
The key is, ALL the information for the whitepapers NEEDS to be rewritten and integrated back into the WDK. It’s just too hard to find otherwise.

We were kind of hoping to do one or the other. Here’s the thing: Generally while we’re in the process of documenting a new OS version, the feature team PMs are handing out whitepapers to NDA partners. At some point in the release cycle it becomes OK to publish the information in those whitepapers.

Here’s what I’m thinking: At that point, we can take each whitepaper, clean it up a bit, and insert it into the WDK docs. When time permits, we can rewrite the information from the whitepaper and assimilate it into the docs properly.

Once the assimilating is done, it would be great if we could delete the whitepaper. Otherwise, we end up with the same information in 2 places in the docs, 1 of which will grow stale over time. (We simply don’t have the resources to maintain whitepapers.)

Questions:

Is it OK to insert whitepapers into the WDK docs (preferably as a single “page” - even if it’s actually 80 printed pages)? Or does anyone really need them in downloadable form?
Once the information is assimilated into the WDK docs, is it OK to delete the whitepaper? Or is it better if we keep it around? (We’d have to put some kind of verbiage at the top declaring that it’s archived material that won’t be updated.)