Gary,
I talk for myself, of course.
And I have no problem shoving the OS out of my way when I feel compelled to
do so.
As far as that spinlock thing went, I had a reason, let me try to explain.
The guy said he had a performance problem with his implementation. My first
reaction was, well, if you have high volume of traffic on one spinlock, you
may be seeing contention at front-side bus level. Now, I suspected that the
implementation of KeAcquireSpinlock( ) did a plain vanilla bts-based test
and set, and although I could not say for sure, it might be the case that a
lot of traffic on a single bts might generate a lot of bus traffic and hence
slow his SMP implementation. It is a simple test to replace a test-and-set
with a better spinlock, which is a test-and-test-and-set: it entails adding
a while loop in front of the bts loop. That takes what, replacing two lines
of code in his critical section with two while loops and one release. That
can be done and tested in less than five minutes, if it increases the
performance he would know one of his bottleneck, of not, just delete it and
go to the next stage.
However, you see, it is not quite possible to convert the Windows
implementation of KeAcquireSpinlock( ) from test-and-set to
test-and-test-and-set, unless one goes down one notch and operates within
the OS. If one would disassemble the code, one might find out that all
that’s needed is to insert the one while loop in front of the call to
KeAcquireSpinlock( ), but that would require disassembly, etc., which takes
time and energy: when all that’s needed is a three-line-of-code change and
test.
So, what do you suggest, that we don’t try it ? Neglect five minutes of
work, and for what, to toe the party line ? I don’t think you’re going to
convince me to suggest that kind of thing.
Alberto.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Gary G. Little
Sent: Monday, December 22, 2003 11:31 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Re: Synchronizing access to a linked list
Alberto,
Knowing that many of the posters here use, and speak highly of, SoftIce and
other Compuware products, I do believe that most, if not all, of our
egregious comments are not directed at any philosophy of Compuware, but
rather directly at Alberto Moreira. It has nothing to do with your
competence, either. Indeed your ability to hold your own in almost any or
all discourses speaks of your competence. I think what most of us find
distressing is your wanton disregard, or should I say disdain, for abiding
by the formats, protocols, structures, and other portions of the framework
of the OS in which we work. Most of us that have found times when we had to
do this, take the more conservative road of at least indicating we had no
other re-course to accomplish the task. You to frequently attempt to get raw
neophytes who have problems spelling “C” let alone working in assembly that
close to the hardware, to do the same thing. That encourages poor
programming practices, period. To many of us have had to pull the chestnuts
of such “I’m king of the world” programming, whether by us or others, out
of the fire to tolerate such wanton willingness to flaunt the OS.
The point here is to play well in the system that exists, not re-write the
system everytime something new comes around. We did that … I did that …
alot back in the 70’s and early 80’s. I can still do that when I have to,
but I would by far rather see a driver I have written playing well in the
200 million systems using it, than running 100% in only 1% of the market and
BSOD’ng in the other 99%.
It’s silly to encourage someone to re-write the basic spinlock used in the
DDK simply because “I don’t trust any API I didn’t write”. The chances are
the newbie will get it wrong, and spend more time than is needed in finally
getting it right, especially when the way you recommend they do it is the
way it is implemented in the OS. That really is silly, my friend.
–
Gary G. Little
Seagate Technologies, LLC
“Moreira, Alberto” wrote in message
news:xxxxx@ntdev…
Gary, our product’s alive and kicking, lots of people use it. And we’re not
known for dropping the ball that often either, nor are we bashful to bypass
the OS when we need to.
Alberto.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Gary G. Little
Sent: Friday, December 19, 2003 4:30 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Re: Synchronizing access to a linked list
Well, Bubba, lemming is as lemming does. Do I follow Alberto over the cliff,
or follow Jake Ohins, Nathan Obr, Peter Weiland, Walter Oney, Jamey Kirby,
Gary Little (oh that’s me) AWAY from the cliff.
'Nuff said.
–
Gary G. Little
Seagate Technologies, LLC
“Moreira, Alberto” wrote in message
news:xxxxx@ntdev…
Golly, Mark, I’ve seen lemming behavior before, but this is one step beyond.
I didn’t look inside their code, and I won’t, so, I’ll reverse the charges:
give me one good reason to use a facility I have no clue how it’s
implemented, specially in a performance-sensitive case !
In this case I have many questions: do they or do they not use
test-and-test-and-set ? Do they have exponential backoff after each
test-and-set, if so, which exponent do they use ? Do they use the pause
instruction, how, and where ? Do they use test-and-set, compare-replace,
exchange-add, what ?
And if they do it the most possible effective way, why is Inaki having
performance issues ?
There’s no such a thing as “correctly written” here. There’s a zillion ways
of implementing a lock. No harm done in experimenting and trying to find a
faster one.
Alberto.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Roddy, Mark
Sent: Friday, December 19, 2003 3:16 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: Synchronizing access to a linked list
Alberto, you are a one-trick pony these days. KeAcquireSpinlock is correctly
written, so there is no need to reinvent that particular wheel.
=====================
Mark Roddy
________________________________
From: Moreira, Alberto [mailto:xxxxx@compuware.com]
Sent: Friday, December 19, 2003 2:50 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: Synchronizing access to a linked list
Readers/Writers is a well known problem, get yourself a copy of any
standard OS book and copy the algorithm into your code; the strategy changes
depending on whether you want to give priority to reads, to writes, or no
special priority. You may be able to speed up your spinlocks by using
test-test-and-set instead of plain vanilla test-end-set, and you should
consider inserting a pause instruction into your code, as Intel recommends.
The test-test-and-set works like this:
while (yourlock);
while (testAndSet(yourlock));
It moves much of the testing to the local processor cache and
basically generates no bus traffic and little contention, the reason is, the
first while loop operates on the processor’s cached copy of that “yourlock”
synchronization variable.
Another trick is waiting a little bit before retrying , that again
reduces contention and may reduce latency as well. You can also try to
structure your list as a circular array of dword pointers, for example, use
a ticket algorithm and have it synchornize itself with a lock-exchange-add,
but that’s trickier because of the boundary conditions.
Hope this helps…
Alberto.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Iñaki Castillo
Sent: Friday, December 19, 2003 12:49 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Synchronizing access to a linked list
I have a linked list that grows dinamically. I use this list
to store data that is used in different threads in a kernel driver.
By one side, updating(writing) to the list occurs a few
times (say several times per minute) while reading the list occurs many
times (say thousand of times per minute).
The writing routine works always at passive level, while the
reading routine works always at above passive level (sometimes at DPC level,
sometimes at DISPATCH level).
I have detected that working on multi processor machines
this scheme does not work well. It does work, but it incurs in great
performance penalty.
If I remove the reading synchronization it works much
faster, but removing it is not a solution because the list can be updated at
any time from the writing thread.
I use SpinLocks to synchronize.
Having this picture, what would be the best method to
synchronize access to the list ?
I have been thinking on remove synchronization at reading
and replacing the linked dynamic list by a static list, though it sounds a
little dirty method.
Any suggestions ?
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as:
xxxxx@compuware.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
The contents of this e-mail are intended for the named addressee
only. It contains information that may be confidential. Unless you are the
named addressee or an authorized designee, you may not copy or use it, or
disclose it to anyone else. If you received it in error please notify us
immediately and then destroy it.
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.
