Re: Problem using queued spinlocks on single cpu comp uter

Problem is, you’re assuming a scope. I read the DDK documentation, where
they say, “Drivers should normally allocate the structure on the stack”,
well, again, that assumes a static scope. This may not be the way a spinlock
is used in real life, it’s nice and well behaved but it may be too
restrictive: the stack at the time the acquirer was created may not exist
any longer, so, in the general case I may need more permanent storage than a
stack: for example, what if thread A acquires a spinlock, forks threads B
through E, and then terminates, so that whoever finishes last, B, C, D or E,
releases the lock ? What if I want to use a spinlock as a barrier
synchronizer, and again, the last thread to arrive at the barrier releases
it ? And if I can delegate that kind of issue to the object itself, why
should I bother handling it ? I also think it’s a bad idea to rely on any
specific time to destroy objects, because well, we may want to implement
garbage collection for example: such a nice language as C# should have a
real compiler that allows us to write drivers, and with automatic garbage
collection you don’t want to mix object destruction with anything else. So,
I believe that releasing an object and deleting it should be two very
separate actions, and the original local context when the object was created
must not be assumed at the time the object is destroyed.

I hear your argument, and it kind of makes sense, but I still believe that
encapsulating everything within a single object is a better way.

Alberto.

-----Original Message-----
From: Don Burn [mailto:xxxxx@acm.org]
Sent: Thursday, September 25, 2003 1:34 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Re: Problem using queued spinlocks on single cpu comp
uter

I first saw this from Bjarne Stroustrup and the Bell Labs C++ team
many years ago. As Max points out, this has the property (good or
bad depending on your preference) of automatically releasing the
lock as part of the destructor, therefore guaranteeing release at the
end of a scope.

I have since seen this approach used by many of the recognized
gurus of OO design.

Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting

----- Original Message -----
From: “Moreira, Alberto”
To: “Windows System Software Devs Interest List”
Sent: Thursday, September 25, 2003 1:16 PM
Subject: [ntdev] Re: Problem using queued spinlocks on single cpu comp uter

> In object oriented design, resources are no longer pure data, they’re now
> objects ! The use of an object is internal to that object, only interfaces
> are published. Otherwise you must give up having a healthy encapsulation,
> just look at that “friend” statement you need in the class. Another
problem
> I see with this design, as I told Max, is that the queue handle is now
> attached to the Acquirer and not to the Lock: if two different pieces of
> code try to acquire the same spinlock, you will end up with two queues,
> which defeats the purpose of using a queued lock.
>
> Alberto.
>
>
> -----Original Message-----
> From: Chuck Batson [mailto:xxxxx@cbatson.com]
> Sent: Thursday, September 25, 2003 12:46 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] Re: Problem using queued spinlocks on single cpu comp
> uter
>
>
> > That’s an interesting statement, Chuck, what advantages do you see in
> > separating these two ?
>
> The conceptual separation of a resource from its use. We can use the
> current discussion as the basis for an example:
>
> class CSpinLockAcquire;
>
> class CSpinLock
> {
> public:
> CSpinLock() { KeInitializeSpinLock(&SpinLock); }
> private:
> Lock(KLOCK_QUEUE_HANDLE & h) {
> KeAcquireInStackQueuedSpinLock(&SpinLock, &h); }
> Unlock(KLOCK_QUEUE_HANDLE & h) {
> KeReleaseInStackQueuedSpinLock(&SpinLock, &h); }
> KSPIN_LOCK SpinLock;
> friend class CSpinLockAcquire;
> };
>
> class CSpinLockAcquire
> {
> public:
> CSpinLockAcquire(CSpinLock & s) : SpinLock(s) {
> s.Lock(Handle); }
> ~CSpinLockAcquire() { s.Unlock(Handle); }
> private:
> CSpinLock & SpinLock;
> KLOCK_QUEUE_HANDLE Handle;
> };
>
> This conceptual separation is useful any time you have a resource which
> must be acquired/released, locked/unlocked, etc. There are a few
> concrete benefits:
>
> 1. Exception safety. Since locking and unlocking is handled by the
> constructor/destructor of CSpinLockAcquire, you can never have a
> situation of a lock not being matched with a corresponding unlock, even
> in the presence of exceptions.
>
> 2. Human error and maintainability. For the same reason, it’s
> impossible for the programmer to “forget” to unlock a resource.
> Granted, if you forget to release a spin lock, you’re likely to know
> pretty quickly, but it still wastes time and effort diagnosing the
> problem. The same general principle applies to other types of resources
> (e.g. not necessarily spin locks) where usage may be more involved
> and/or subtle and hence more prone to difficult-to-diagnose “forgetting
> to unlock” errors.
>
> 3. Acquisition/release data is separate from resource data. This is
> good for two reasons:
>
> a. Memory storage for the resource object itself may be precious.
> Consider the “single class” implementation where the spin lock and its
> queue handle are stored together. The spin lock has to go into shared
> memory which is accessible to all those who might potentially acquire
> it. There’s no reason for the queue handle to be shared, since it’s
> only used by one acquirer at a time. So separating them is beneficial
> when use of shared memory has performance implications.
>
> b. Resource acquisition data sometimes must be separate from the
> resource, especially when it’s possible for more that one entity to
> acquire a given resource at the same time. This is less useful in this
> case, when only one processor can acquire a spinlock at any given time.
> (Though it’s still a bit aesthetically displeasing to me, when you think
> about two different processors calling KeAcquireInStackQueuedSpinLock()
> with a pointer to the same queue handle; “luckily” only one of them will
> succeed in acquiring the spin lock, so there will never be more than one
> user of the queue handle… but still makes me shudder.) But there are
> situations in which this is necessary. For example, a resource which
> has both read and write acquires, where multiple readers are allowed but
> only a single writer (with no readers) is allowed. Since you can have
> multiple readers, obviously it won’t do to put the acquisition handle
> (or whatever relevant acquisition-related information) in the resource
> object itself.
>
> Chuck
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@compuware.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
disclose
> it to anyone else. If you received it in error please notify us
immediately
> and then destroy it.
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@acm.org
> To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

It all depends on what you want to do with the bits and with the uchars as
you define them. If the functionality is complicated enough, yes, it may
make sense to encapsulate things that way. It all depends on what you want
to accomplish.

Furthermore, we have bit functionality in C, we have “unsigned char”, and so
on - why do we need BIT, UCHAR ? Waste of time, no ? Or maybe there’s some
additional effects people want to achieve that can’t be done the old way ?

And you know, who needs structs if you can make do with arrays ? Long live
Fortran II. But hey, at some point we might want to retire that Model T.

Alberto.

-----Original Message-----
From: Jamey Kirby [mailto:xxxxx@storagecraft.com]
Sent: Thursday, September 25, 2003 1:44 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Re: Problem using queued spinlocks on single cpu comp
uter

Oh, I got it… Let’s encapsulate the actual bits… That should make for
some very manageable and robust driver code; NOT!

class BIT
{
};

class UCHAR : BIT
{
};

class WCHAR : UCHAR
{
};

Oooo… Ahhhh… Thing are becoming more clear and more robust; don’t you
agree? Heck, my grand mother could maintain this code.

Jamey Kirby, Windows DDK MVP
StorageCraft Inc.
xxxxx@storagecraft.com
http://www.storagecraft.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Moreira, Alberto
Sent: Thursday, September 25, 2003 10:17 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Re: Problem using queued spinlocks on single cpu comp uter

In object oriented design, resources are no longer pure data, they’re now
objects ! The use of an object is internal to that object, only interfaces
are published. Otherwise you must give up having a healthy encapsulation,
just look at that “friend” statement you need in the class. Another problem
I see with this design, as I told Max, is that the queue handle is now
attached to the Acquirer and not to the Lock: if two different pieces of
code try to acquire the same spinlock, you will end up with two queues,
which defeats the purpose of using a queued lock.

Alberto.

-----Original Message-----
From: Chuck Batson [mailto:xxxxx@cbatson.com]
Sent: Thursday, September 25, 2003 12:46 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Re: Problem using queued spinlocks on single cpu comp
uter

That’s an interesting statement, Chuck, what advantages do you see in
separating these two ?

The conceptual separation of a resource from its use. We can use the
current discussion as the basis for an example:

class CSpinLockAcquire;

class CSpinLock
{
public:
CSpinLock() { KeInitializeSpinLock(&SpinLock); }
private:
Lock(KLOCK_QUEUE_HANDLE & h) {
KeAcquireInStackQueuedSpinLock(&SpinLock, &h); }
Unlock(KLOCK_QUEUE_HANDLE & h) {
KeReleaseInStackQueuedSpinLock(&SpinLock, &h); }
KSPIN_LOCK SpinLock;
friend class CSpinLockAcquire;
};

class CSpinLockAcquire
{
public:
CSpinLockAcquire(CSpinLock & s) : SpinLock(s) {
s.Lock(Handle); }
~CSpinLockAcquire() { s.Unlock(Handle); }
private:
CSpinLock & SpinLock;
KLOCK_QUEUE_HANDLE Handle;
};

This conceptual separation is useful any time you have a resource which
must be acquired/released, locked/unlocked, etc. There are a few
concrete benefits:

  1. Exception safety. Since locking and unlocking is handled by the
    constructor/destructor of CSpinLockAcquire, you can never have a
    situation of a lock not being matched with a corresponding unlock, even
    in the presence of exceptions.

  2. Human error and maintainability. For the same reason, it’s
    impossible for the programmer to “forget” to unlock a resource.
    Granted, if you forget to release a spin lock, you’re likely to know
    pretty quickly, but it still wastes time and effort diagnosing the
    problem. The same general principle applies to other types of resources
    (e.g. not necessarily spin locks) where usage may be more involved
    and/or subtle and hence more prone to difficult-to-diagnose “forgetting
    to unlock” errors.

  3. Acquisition/release data is separate from resource data. This is
    good for two reasons:

a. Memory storage for the resource object itself may be precious.
Consider the “single class” implementation where the spin lock and its
queue handle are stored together. The spin lock has to go into shared
memory which is accessible to all those who might potentially acquire
it. There’s no reason for the queue handle to be shared, since it’s
only used by one acquirer at a time. So separating them is beneficial
when use of shared memory has performance implications.

b. Resource acquisition data sometimes must be separate from the
resource, especially when it’s possible for more that one entity to
acquire a given resource at the same time. This is less useful in this
case, when only one processor can acquire a spinlock at any given time.
(Though it’s still a bit aesthetically displeasing to me, when you think
about two different processors calling KeAcquireInStackQueuedSpinLock()
with a pointer to the same queue handle; “luckily” only one of them will
succeed in acquiring the spin lock, so there will never be more than one
user of the queue handle… but still makes me shudder.) But there are
situations in which this is necessary. For example, a resource which
has both read and write acquires, where multiple readers are allowed but
only a single writer (with no readers) is allowed. Since you can have
multiple readers, obviously it won’t do to put the acquisition handle
(or whatever relevant acquisition-related information) in the resource
object itself.

Chuck


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

Yes, you’re right, I figured that out by rereading the DDK page. In other
words, move it from the constructor to the Acquire function: grab the
structure at Acquire time, free it at Release time. That way you guarantee
you don’t depend on assumptions about the user’s block structuring. Yet I
still don’t see the need for a separate object.

Alberto.

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Thursday, September 25, 2003 4:11 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Re: Problem using queued spinlocks on single cpu comp
uter

KLOCK_QUEUE_HANDLE must be 1 per acquisition attempt, not 1 per spinlock.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Moreira, Alberto”
To: “Windows System Software Devs Interest List”
Sent: Thursday, September 25, 2003 9:09 PM
Subject: [ntdev] Re: Problem using queued spinlocks on single cpu comp uter

> I don’t understand, I thought the queue was attached to the spinlock ? Not
> to the acquisitor ? With this design, if I have two separate pieces of
code
> that issue two locks to the same spinlock I get two queues, one at each
> acquisitor, while what I may want is both accessors to end up at the same
> queue, so that they’re serviced in the same sequence they applied their
> respective locks.
>
> Alberto.
>
>
> -----Original Message-----
> From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
> Sent: Thursday, September 25, 2003 12:23 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] Re: Problem using queued spinlocks on single cpu comp
> uter
>
>
> > class CSpinLock
> > {
> > private:
> > KLOCK_QUEUE_HANDLE m_LockQueue;
> > PKSPIN_LOCK m_oSpinLock;
> > CString m_name;
> > public:
> > CSpinLock (const CString & name)
> > : m_name(name)
> > {
> > KeInitializeSpinLock(&m_oSpinLock);
> > }
>
> This will not work. The correct way is:
>
> class CSpinLock
> {
> friend class CSpinLockAcquisitor;
> public:
> CSpinLock() { KeInitializeSpinLock(&m_Lock); }
> private:
> KSPIN_LOCK m_Lock;
> };
>
> class CSpinLockAcquisitor
> {
> public:
> CSpinLockAcquisitor(CSpinLock* Lock) {
> KeAcquireInStackQueuedSpinLock(Lock->m_Lock, &m_Locker); }
> ~CSpinLockAcquisitor { KeReleaseInStackQueuedSpinLock(&m_Locker); }
> private:
> KLOCK_QUEUE_HANDLE m_Locker;
> };
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@compuware.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
disclose
> it to anyone else. If you received it in error please notify us
immediately
> and then destroy it.
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

You can grab the LOCK_QUEUE_HANDLE structure inside the Acquire, store it in
a member variable in the spinlock. Come release time, delete the structure.
All invisible to all but the spinlock object, and no assumptions about the
caller’s structures. But you see, this whole design already taxes the
spinlock logic beyond what it should be, because now you have to create and
destroy an object every time you use the Spinlock, be it the Acquisitor or
the Queue Handle. I would have expected the facility to have been designed
so that I could grab whatever data structures I needed at object creation
time, and just reuse it over and over, like a file handle.

Alberto.

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Thursday, September 25, 2003 4:09 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Re: Problem using queued spinlocks on single cpu comp uter

Must be a question of personal preference, because I don’t see why

CSpinLockAcquisitor(&SomeGlobalLock);

is better than

SomeGlobalLock.Acquire( );

Because, for queued spinlock, you will need a LOCK_QUEUE_HANDLE.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

> ----------

From: xxxxx@storagecraft.com[SMTP:xxxxx@storagecraft.com]
Reply To: xxxxx@lists.osr.com
Sent: Thursday, September 25, 2003 10:14 PM
To: xxxxx@lists.osr.com
Subject: [ntdev] Re: Problem using queued spinlocks on single cpu
comp uter

Good sides: you cannot forget the lock release. Exception safety can also
be
important (though not for spinlocks for sure - throwing a C++ exception on
DISPATCH_LEVEL is a maniac idea :slight_smile: ).

I’m sure Alberto would like it :slight_smile: On the other hand, there is no principal
difference between throwing C++ exception and returning status code. Both
serve the same purpose and in both cases lock must be released before
leaving block. Exception handling with cleverly designed classes (as lock
acquisitor is) allows to do it automagically. The price is a lot of hidden
code and maybe stack usage not suitable for kernel code.

Exception safety is very important when C++ exceptions are used for error
handling (in user mode, for sure :). Without it every locked code section
would have to be enclosed inside try/catch block and extra logic used to
decide if lock should be released inside catch section. With C++ even
innocently looking statement (as assignment) can throw an exception.

Best regards,

Michal Vodicka
STMicroelectronics Design and Application s.r.o.
[michal.vodicka@st.com, http:://www.st.com]