Are callbacks guaranteed to run consecutively?

[anton_bassov]

Don,

Yes that is synchronization since you can create a contract based on this, is
it a general mutex like sync, NO.

You gave a very, very precise answer to the question about synchronization in itself , which is simply NO - there is just no such thing as partial synchronization, so that if you want your code to be thread-safe you have to provide your own synch mechanism in your callback.
Unfortunately, MSDN, again, creates a confusion by making ambiguous statements that can be easily misunderstood and misinterpreted…

Anton Bassov

[Don_Burn_1]

BULLSHIT!!!

Anton,

It is a contract like everything else. By your definition SPINLOCKS do
not nessecarily synchronize, since one thread can decide to release a
spinlock that another took, or by a thread not taking the spinlock that is
intended for managing a piece of data. Yes, callbacks are more restrictive,
but that means for instance if you can say only one thread will call the
callback, then the callback can assume that the thread doing the callback is
not accessing the data, and if one creates a contract that says a set of
data is in a valid consistant state then you can trust it.

Callbacks have been a synchronization mechanism long before windows.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply

wrote in message news:xxxxx@ntdev…
> Don,
>
> > Yes that is synchronization since you can create a contract based on
> > this, is
>> it a general mutex like sync, NO.
>
> You gave a very, very precise answer to the question about synchronization
> in itself , which is simply NO - there is just no such thing as partial
> synchronization, so that if you want your code to be thread-safe you have
> to provide your own synch mechanism in your callback.
> Unfortunately, MSDN, again, creates a confusion by making ambiguous
> statements that can be easily misunderstood and misinterpreted…
>
> Anton Bassov
>
>

[anton_bassov]

Don,

By your definition SPINLOCKS do not nessecarily synchronize, since one thread can
decide to release a spinlock that another took,

How on Earth may something like that possibly happen??? If thread X takes a spinlock, no other thread should have a chance to run on a given CPU until spinlock gets released. This is why spinlock holders always run at elevated IRQL. This is not just some implementation feature, but, instead, just a conceptual thing - if context switches were possible while spinlock is being held, it would make the very concept of a spinlock unworkable…

Callbacks have been a synchronization mechanism long before windows.

…but only on uniprocessor systems (that were prevalent at the time)

Anton Bassov

[Mark_Roddy]

On Dec 22, 2007 4:46 PM, wrote:

> Don,
>
> > By your definition SPINLOCKS do not nessecarily synchronize, since one
> thread can
> > decide to release a spinlock that another took,
>
> How on Earth may something like that possibly happen??? If thread X takes
> a spinlock, no other thread should have a chance to run on a given CPU until
> spinlock gets released.

So thread y runs on some other cpu and simply releases the spinlock. It
certainly can be done, it is just wrong.

> This is why spinlock holders always run at elevated IRQL. This is not just
> some implementation feature, but, instead, just a conceptual thing - if
> context switches were possible while spinlock is being held, it would make
> the very concept of a spinlock unworkable…

If I remember correctly the AIX version of Unix had just such a spinlock. It
spun callers at raised interrupt for a while, and then it put those threads
to sleep. NT’s merging of IRQL and spinlock are features of NT and are not
an essential requirement of a spinlock implementation.

–
Mark Roddy

[Don_Burn_1]

As Mark Roddy pointed out, a thread on a second processor could release the
lock. You seem to be thinking uni-processoor which shows a great gap in
thinking about synchronization. By the way, there have been many spin-lock
implementations that do not block scheduling, many of them test the spinlock
n times, and if they do not get it yield. As far as IRQL remember this is
a Windows (and I think VMS concept), it is definitely not a generic OS
concept.

Every synchronization mechanism is a contract, if you violate the contract
you fail. Saying something is not a sync mechanism just because you don’t
like the contract, is just garbage tha same as saying that the OS does not
have the right to dictate what your driver can do. Perhaps you can get
away with it some time, but sooner or later people catch on to bad code, or
stupid claims.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply

wrote in message news:xxxxx@ntdev…
> Don,
>
>> By your definition SPINLOCKS do not nessecarily synchronize, since one
>> thread can
>> decide to release a spinlock that another took,
>
> How on Earth may something like that possibly happen??? If thread X takes
> a spinlock, no other thread should have a chance to run on a given CPU
> until spinlock gets released. This is why spinlock holders always run at
> elevated IRQL. This is not just some implementation feature, but, instead,
> just a conceptual thing - if context switches were possible while spinlock
> is being held, it would make the very concept of a spinlock unworkable…
>
>
>> Callbacks have been a synchronization mechanism long before windows.
>
> …but only on uniprocessor systems (that were prevalent at the time)
>
> Anton Bassov
>

[anton_bassov]

Don,

As Mark Roddy pointed out, a thread on a second processor could release the lock.

First of all, let’s decide upon the topic of our discussion. I speak about synchronization mechanism that ensures * thread safety* and not about the one that may protect programmers against their own stupidity or malicious intentions. What I am saying here is that OS-provided mechanism of a queued callback object without either spinlock or additional custom synchronization mechanism in a callback function is not sufficient for providing thread safety in MP environment. This is the only thing I am saying. Therefore, I really have no clue why you speak about misusing spinlocks…

By the way, there have been many spin-lock implementations that do not
block scheduling many of them test the spinlock n times, and if they do not get it yield.

You seem to confuse the process of spinlock acquisition with being a spinlock holder. Assuming the code that tries to acquire a spinlock is allowed to block, it can handle acquisition process in any way it wishes - as long as you don’t yet hold a spinlock you can yield execution without any problem. However, after having actually acquired a spinlock, you cannot already block until you release it. This is true for Windows, Linux, and, apparently, all other MP OSes.

As far as IRQL remember this is a Windows (and I think VMS concept), it is
definitely not a generic OS concept.

AFAIK, the concept of IRQL ( as it is known under Windows) had been around at the time MSFT did not even exists - it was known as splx() family under various flavors of UNIX back in early 1970s. Subsequently it was simplified/abandoned by UNIX, due to the unnecessary complexity that it adds without providing any significant benefits. This concept still exists under modern Linux, although in significantly simplified form ( modern Linux makes no difference between hardware interrupt priority, but software interrupts are still prioritized over one another). Therefore, IRQL ( as it is known under Windows) is not an indicator of some superiority of Windows to other OSes - instead, it is a sign of old-fashioned system. In fact, this is not the question of IRQL in itself, but rather of “may block” vs “cannot block”, which is a generic OS concept that applies everywhere (although the way this concept is implemented may vary under various OSes).

Every synchronization mechanism is a contract, if you violate the contract you fail.

Sure. However, I don’t see how it is related to this discussion

Saying something is not a sync mechanism just because you don’t like the contract,

Can you show me any statement saying it??? The only thing I said is that supposed “synchronization” that callbacks provide is not sufficient for ensuring thread safety in MP environment - in order to ensure it you need to introduce some additional mechanisms (i.e. spinlock or mutex) into your callback itself…

Anton Bassov

[Maxim_S_Shatskih]

> If I remember correctly the AIX version of Unix had just such a spinlock. It

spun callers at raised interrupt for a while, and then it put those threads
to sleep.

I think that user-mode CRITICAL_SECTION and maybe even FAST_MUTEX are the same.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

[Don_Burn_1]

Comments inline:

wrote in message news:xxxxx@ntdev…
> Don,
>
>> As Mark Roddy pointed out, a thread on a second processor could release
>> the lock.
>
> First of all, let’s decide upon the topic of our discussion. I speak about
> synchronization mechanism that ensures * thread safety* and not about the
> one that may protect programmers against their own stupidity or malicious
> intentions. What I am saying here is that OS-provided mechanism of a
> queued callback object without either spinlock or additional custom
> synchronization mechanism in a callback function is not sufficient for
> providing thread safety in MP environment. This is the only thing I am
> saying. Therefore, I really have no clue why you speak about misusing
> spinlocks…
>
I agree that by itself it is not MP thread safe, but disagree about your
statements about this therefore not being a synchronization mechanism. The
requirement of total MP safety was never part of the definition of
synchronization (at least during the time people looked at a lot of OS’es
and not just Windows or Unix clones).

>
>> By the way, there have been many spin-lock implementations that do not
>> block scheduling many of them test the spinlock n times, and if they do
>> not get it yield.
>
> You seem to confuse the process of spinlock acquisition with being a
> spinlock holder. Assuming the code that tries to acquire a spinlock is
> allowed to block, it can handle acquisition process in any way it wishes -
> as long as you don’t yet hold a spinlock you can yield execution without
> any problem. However, after having actually acquired a spinlock, you
> cannot already block until you release it. This is true for Windows,
> Linux, and, apparently, all other MP OSes.

Sorry wrong, there are OS’es that allow just that. Yes deadlock can be a
royal pain in these systems, but they were out there and worked.

>> As far as IRQL remember this is a Windows (and I think VMS concept), it
>> is
>> definitely not a generic OS concept.
>
> AFAIK, the concept of IRQL ( as it is known under Windows) had been around
> at the time MSFT did not even exists - it was known as splx() family under
> various flavors of UNIX back in early 1970s. Subsequently it was
> simplified/abandoned by UNIX, due to the unnecessary complexity that it
> adds without providing any significant benefits. This concept still exists
> under modern Linux, although in significantly simplified form ( modern
> Linux makes no difference between hardware interrupt priority, but
> software interrupts are still prioritized over one another). Therefore,
> IRQL ( as it is known under Windows) is not an indicator of some
> superiority of Windows to other OSes - instead, it is a sign of
> old-fashioned system. In fact, this is not the question of IRQL in itself,
> but rather of “may block” vs “cannot block”, which is a generic OS concept
> that applies everywhere (although the way this concept is implemented may
> vary under various OSes).

Go take a Windows driver class, IRQL is a software concept that to a small
degree spills over to hardware, splx() at least when UNIX was first comming
out of Bell Labs was a hardware only concept and in a number of early
implementations was a write to a control register…

>> Every synchronization mechanism is a contract, if you violate the
>> contract you fail.
>
> Sure. However, I don’t see how it is related to this discussion
>
>> Saying something is not a sync mechanism just because you don’t like the
>> contract,
>
> Can you show me any statement saying it??? The only thing I said is that
> supposed “synchronization” that callbacks provide is not sufficient for
> ensuring thread safety in MP environment - in order to ensure it you need
> to introduce some additional mechanisms (i.e. spinlock or mutex) into your
> callback itself…
>
Go find an operating system book from the pre “Linux/Unix is the perfect
creation days”, or better yet find a book on synchronization mechanisms (yes
there were a few).

I am dropping out of this, since you arguments are framed by your lack of
knowledge and experience outside of the Linux/Windows world. Rob Pike when
he said “Operating System research is dead” had it almost right but nowadays
it should be “Operating System Knowledge is dead”

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply

[anton_bassov]

Don,

I agree that by itself it is not MP thread safe, but disagree about your statements
about this therefore not being a synchronization mechanism. The requirement of total
MP safety was never part of the definition of synchronization…

Sorry, but what is what is synchronization then??? I would be glad to hear your definition of it…

…(at least during the time people looked at a lot of OS’es and not just Windows or Unix clones).

Probably, just because MP system was a rare beast (probably, even non-existent one) at the time??? Sorry, but when it comes to computers, you just cannot live by the textbooks that were written 40 years ago…

> You seem to confuse the process of spinlock acquisition with being a
> spinlock holder. Assuming the code that tries to acquire a spinlock is
> allowed to block, it can handle acquisition process in any way it wishes -
> as long as you don’t yet hold a spinlock you can yield execution without

Sorry wrong, there are OS’es that allow just that. Yes deadlock can be a royal
pain in these systems, but they were out there and worked.

If spinlock holder is allowed to yield while holding a spinlock…well, then this is not a spinlock but already a mutex. Sorry, but any reasonably designed OS would make a distinction between these two in order to make sure the code that is not allowed to block is not going to wait for the one. Apparently, you had some first-hand experience with OSes that did not make the above
distinction, but they are dead by now, right? Actually, I think this is one of the reasons why they
are gone…

Go take a Windows driver class, IRQL is a software concept that to a small degree
spills over to hardware, splx() at least when UNIX was first comming out of Bell Labs
was a hardware only concept and in a number of early implementations was a
write to a control register…

I am afraid your biggest problem is that you took too many Windows driver classes, instead of doing your own research…

Sorry, but the thing you said is just bullshit - you just blindly repeated something that you have
learned in some driver class or read on MSDN . However, if you did some “independent research”, instead of thoughtlessly repeating things that you were told by MSFT, you would have known that, exactly like its early UNIX splx() counterpart, any Windows operation that involves changing IRQL involves writing to a register ( which may be specific to a given hardware architecture - for example, on APIC HAL it is local APIC’s Task Priority Register), and IRQL itself is just an index into the array that holds values to be written to this register. Therefore, IRQL is not a software-only concept, as you were told in some Windows driver class - instead, it requires hardware cooperation, because it is CPU itself and not OS software who accepts an interrupt and keeps it pending until IRQL drops below specified level (i.e. appropriate value is written into the corresponding register by the OS software)…

I am dropping out of this, since you arguments are framed by your lack of
knowledge and experience outside of the Linux/Windows world.

Sure - in fact, I just had no chance to experience anything else, simply because these 2 systems are prevalent today.

Rob Pike when he said “Operating System research is dead” had it almost
right but nowadays it should be “Operating System Knowledge is dead”

Well, it is not dead, but it is dying ( at least as far as Windows world is concerned), and you are the one who actively encourages this process (long live KMDF; do what MSFT says, instead of learning things; etc). Therefore, you’ve got nothing to complain about - instead, you should be glad…

Anton Bassov

[Don_Burn_1]

I wasn’t going but this is so bad I have to reply.
wrote in message news:xxxxx@ntdev…

> Sorry, but the thing you said is just bullshit - you just blindly repeated
> something that you have
> learned in some driver class or read on MSDN . However, if you did some
> “independent research”, instead of thoughtlessly repeating things that
> you were told by MSFT, you would have known that, exactly like its early
> UNIX splx() counterpart, any Windows operation that involves changing IRQL
> involves writing to a register ( which may be specific to a given hardware
> architecture - for example, on APIC HAL it is local APIC’s Task Priority
> Register), and IRQL itself is just an index into the array that holds
> values to be written to this register. Therefore, IRQL is not a
> software-only concept, as you were told in some Windows driver class -
> instead, it requires hardware cooperation, because it is CPU itself and
> not OS software who accepts an interrupt and keeps it pending until IRQL
> drops below specified level (i.e. appropriate value is written into the
> corresponding register by the OS software)…

I learned about splx from working on Unix operating systems in the 70’s and
early 80’s and there is a hell of a difference from the pure interrupt
concept of Unix splx and the scheduling/software interrupt/hardware
interrupt/disable interrupt callback of IRQL’s. Sorry one is a complex
concept with a rich set of capabilities, and the other is hardware interrupt
levels period. Now whether IRQL is a good idea or not can be argued, but
they ain’t the same.

I based my comments on 35 years of system software writing including drivers
for over 10 differing OS’es and using a couple of dozen. My independant
research on IRQL has included reading the source on much of this.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply

[OSR_Community_User]

I’m not sure I follow you guys. If thread A grabs a lock, no other thread
should be able to release it, or it’s not really a lock. But then, as you
yourself have pointed out, the IRQL implementation is far from a
straightfoward interrupt management, so, hey, everything’s possible.

Synchronization constructs shouldn’t be regarded as contracts: they’re
engines that should have highly predictable behavior. It’s the *usage* of
synchronization constructs that may be regarded as a contract, and even
then, the better the synchronization construct the less one has to treat it
as a contract. For example, if I have a self-synchronizing object or data
structure - also called a “Monitor” in computer science parlance - accesses
to its data are synchronized as a matter of fact, by the object itself,
contract or no contract. The mutual exclusion is enforced by the data
itself, not by the users of the data. And that can permeate all the way to
the language level, if you doubt it I invite you to read the Concurrent
Pascal language spec.

On the other hand, if you have a simple-minded construct such as a mutex or
even a semaphore, there’s a protocol that users of the mechanism have to
adhere to, otherwise synchronization is not going to work. But jot this one
down to the weakness of the mechanism, not on the fact that its usage might
or not require something like a “contract” among threads.

As far as the OS goes, its duty is to implement the mechanism so that users
can take advantage of it. I do not thing it’s an OS call to tell users what
they can or cannot do, as long as applications use the mechanism through the
OS’s published APIs. It is also not the OS’s call to tell drivers what they
will or will not do with well published and well specified synchronization
mechanisms, again, as long as drivers respect the API and use it properly.
And, in a well designed OS drivers sit between the hardware and the OS.
Hardware should be driven by drivers, not by the OS, and IMHO the OS has no
business sitting between a driver and its hardware. OS’s aren’t the owners
of the machine - they’re a service layer. An OS isn’t a ruler or a
president, but a public servant that is at all times at the command of its
users.

If you want to shield the OS from driver problems you should begin by
running them in different protection rings. I find it mind boggling that
even though the i386 architecture implements such a large array of security
oriented features, the OS uses none of them, all under the excuse of being
compatible with machines that do not have nearly a fraction of that
functionality. Go figure!

Alberto.

----- Original Message -----
From: “Don Burn”
Newsgroups: ntdev
To: “Windows System Software Devs Interest List”
Sent: Saturday, December 22, 2007 6:10 PM
Subject: Re:[ntdev] Are callbacks guaranteed to run consecutively?

> As Mark Roddy pointed out, a thread on a second processor could release
> the lock. You seem to be thinking uni-processoor which shows a great gap
> in thinking about synchronization. By the way, there have been many
> spin-lock implementations that do not block scheduling, many of them test
> the spinlock n times, and if they do not get it yield. As far as IRQL
> remember this is a Windows (and I think VMS concept), it is definitely not
> a generic OS concept.
>
> Every synchronization mechanism is a contract, if you violate the contract
> you fail. Saying something is not a sync mechanism just because you don’t
> like the contract, is just garbage tha same as saying that the OS does not
> have the right to dictate what your driver can do. Perhaps you can get
> away with it some time, but sooner or later people catch on to bad code,
> or stupid claims.
>
>
> –
> Don Burn (MVP, Windows DDK)
> Windows 2k/XP/2k3 Filesystem and Driver Consulting
> Website: http://www.windrvr.com
> Blog: http://msmvps.com/blogs/WinDrvr
> Remove StopSpam to reply
>
>
>
> wrote in message news:xxxxx@ntdev…
>> Don,
>>
>>> By your definition SPINLOCKS do not nessecarily synchronize, since one
>>> thread can
>>> decide to release a spinlock that another took,
>>
>> How on Earth may something like that possibly happen??? If thread X
>> takes a spinlock, no other thread should have a chance to run on a given
>> CPU until spinlock gets released. This is why spinlock holders always run
>> at elevated IRQL. This is not just some implementation feature, but,
>> instead, just a conceptual thing - if context switches were possible
>> while spinlock is being held, it would make the very concept of a
>> spinlock unworkable…
>>
>>
>>> Callbacks have been a synchronization mechanism long before windows.
>>
>> …but only on uniprocessor systems (that were prevalent at the time)
>>
>> Anton Bassov
>>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

[anton_bassov]

Don,

I wasn’t going but this is so bad I have to reply.

Sorry, I did not mean to hurt you in any possible way. However, I still maintain that your excessive reliance upon the “official” sources and unwillingness to discover how things *actually* work under Windows does not do you any good - sometimes it results in totally erroneous statements…

My independant research on IRQL has included reading the source on much of this.

Actually, the only thing I meant by “independent research” is simply dumping HAL.DLL and looking at what IRQL-related function do (not that much, delegating the bulk of the work to the CPU hardware, as you would see with your own eyes). This is why I put “independent research” in quotation marks.

I learned about splx from working on Unix operating systems in the 70’s
and early 80’s and there is a hell of a difference from the pure interrupt
concept of Unix splx and the scheduling/software interrupt/hardware
interrupt/disable interrupt callback of IRQL’s.

…

I based my comments on 35 years of system software writing including drivers
for over 10 differing OS’es and using a couple of dozen.

Are you sure you have not confused Windows implementation of IRQL with some other OS???
Look - the only IRQL-related thing that OS software does under Windows is writing to TPR (I assume APIC HAL) in order to disable/reenable interrupts of lower priority. The rest is done by the CPU itself - there is no software-implemented callback mechanism whatsoever…

Anton Bassov

[Mark_Roddy]

Maybe we are having a terminology problem. “Should” does not mean “can’t”.
Spinlocks are software constructs. It is up to the implementation to decide
if, for example, it is going to refuse to honor a release request from
anyone except the current thread owner. The spinlock implementation in NT
does not enforce such a rule, consequently you can release a spinlock from a
non-owning thread. That does not mean that you should do that, it means that
you can do that but that doing so violates the contract, just as modifying
the object covered by the spinlock without holding that spinlock violates
the contract. An assertion was made that one can’t release a spinlock this
way, that assertion is wrong.

“a protocol that users of the mechanism have to adhere to” has approximately
the same meaning to me as “a contract”, so I fail to see any real difference
here.

Finally, another assertion was made (not by you) that one cannot block after
acquiring a spinlock. This is of course not true at all. A spinlock owning
thread is free to acquire another spinlock and will of course block waiting
for the current owner of that lock to release it. Once again perhaps we all
are just having a terminology problem, and that ‘block’ meant ‘wait on an
event’ in the poster’s terminology.

On Dec 23, 2007 2:11 AM, Alberto Moreira wrote:

> I’m not sure I follow you guys. If thread A grabs a lock, no other thread
> should be able to release it, or it’s not really a lock. But then, as you
> yourself have pointed out, the IRQL implementation is far from a
> straightfoward interrupt management, so, hey, everything’s possible.
>
> Synchronization constructs shouldn’t be regarded as contracts: they’re
> engines that should have highly predictable behavior. It’s the usage of
> synchronization constructs that may be regarded as a contract, and even
> then, the better the synchronization construct the less one has to treat
> it
> as a contract. For example, if I have a self-synchronizing object or data
> structure - also called a “Monitor” in computer science parlance -
> accesses
> to its data are synchronized as a matter of fact, by the object itself,
> contract or no contract. The mutual exclusion is enforced by the data
> itself, not by the users of the data. And that can permeate all the way to
> the language level, if you doubt it I invite you to read the Concurrent
> Pascal language spec.
>
> On the other hand, if you have a simple-minded construct such as a mutex
> or
> even a semaphore, there’s a protocol that users of the mechanism have to
> adhere to, otherwise synchronization is not going to work. But jot this
> one
> down to the weakness of the mechanism, not on the fact that its usage
> might
> or not require something like a “contract” among threads.
>
> As far as the OS goes, its duty is to implement the mechanism so that
> users
> can take advantage of it. I do not thing it’s an OS call to tell users
> what
> they can or cannot do, as long as applications use the mechanism through
> the
> OS’s published APIs. It is also not the OS’s call to tell drivers what
> they
> will or will not do with well published and well specified synchronization
> mechanisms, again, as long as drivers respect the API and use it properly.
> And, in a well designed OS drivers sit between the hardware and the OS.
> Hardware should be driven by drivers, not by the OS, and IMHO the OS has
> no
> business sitting between a driver and its hardware. OS’s aren’t the owners
> of the machine - they’re a service layer. An OS isn’t a ruler or a
> president, but a public servant that is at all times at the command of its
> users.
>
> If you want to shield the OS from driver problems you should begin by
> running them in different protection rings. I find it mind boggling that
> even though the i386 architecture implements such a large array of
> security
> oriented features, the OS uses none of them, all under the excuse of being
> compatible with machines that do not have nearly a fraction of that
> functionality. Go figure!
>
>
> Alberto.
>
>
>
> ----- Original Message -----
> From: “Don Burn”
> Newsgroups: ntdev
> To: “Windows System Software Devs Interest List”
> Sent: Saturday, December 22, 2007 6:10 PM
> Subject: Re:[ntdev] Are callbacks guaranteed to run consecutively?
>
>
> > As Mark Roddy pointed out, a thread on a second processor could release
> > the lock. You seem to be thinking uni-processoor which shows a great
> gap
> > in thinking about synchronization. By the way, there have been many
> > spin-lock implementations that do not block scheduling, many of them
> test
> > the spinlock n times, and if they do not get it yield. As far as IRQL
> > remember this is a Windows (and I think VMS concept), it is definitely
> not
> > a generic OS concept.
> >
> > Every synchronization mechanism is a contract, if you violate the
> contract
> > you fail. Saying something is not a sync mechanism just because you
> don’t
> > like the contract, is just garbage tha same as saying that the OS does
> not
> > have the right to dictate what your driver can do. Perhaps you can get
> > away with it some time, but sooner or later people catch on to bad code,
> > or stupid claims.
> >
> >
> > –
> > Don Burn (MVP, Windows DDK)
> > Windows 2k/XP/2k3 Filesystem and Driver Consulting
> > Website: http://www.windrvr.com
> > Blog: http://msmvps.com/blogs/WinDrvr
> > Remove StopSpam to reply
> >
> >
> >
> > wrote in message news:xxxxx@ntdev…
> >> Don,
> >>
> >>> By your definition SPINLOCKS do not nessecarily synchronize, since one
> >>> thread can
> >>> decide to release a spinlock that another took,
> >>
> >> How on Earth may something like that possibly happen??? If thread X
> >> takes a spinlock, no other thread should have a chance to run on a
> given
> >> CPU until spinlock gets released. This is why spinlock holders always
> run
> >> at elevated IRQL. This is not just some implementation feature, but,
> >> instead, just a conceptual thing - if context switches were possible
> >> while spinlock is being held, it would make the very concept of a
> >> spinlock unworkable…
> >>
> >>
> >>> Callbacks have been a synchronization mechanism long before windows.
> >>
> >> …but only on uniprocessor systems (that were prevalent at the time)
> >>
> >> Anton Bassov
> >>
> >
> >
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at
> > http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

–
Mark Roddy

[anton_bassov]

Mark,

Finally, another assertion was made (not by you) that one cannot block after
acquiring a spinlock. This is of course not true at all. A spinlock owning thread is
free to acquire another spinlock and will of course block waiting for the current
owner of that lock to release it.
Once again perhaps we all are just having a terminology problem, and that ‘block’ meant ‘wait > on an event’ in the poster’s terminology.

AFAIK, the very term “blocking” comes from UNIX, which makes a very clear distinction between blocking and non-blocking IO (you can put your target file/socket/ into non-blocking mode, and if you don’t do it, you can always find out whether your IO operation is going to block by making poll() or select() system call). In UNIX terms, “blocking” means yielding the CPU ( a call to schedule() under Linux). Therefore, if you translate it into Windows terms, blocking means calling WaitXXX() with non-zero interval…

Anton Bassov

[David_J_Craig]

Interesting definition of ‘blocking’ that I think doesn’t match the common
English usage. I do agree that a call to schedule() and the WaitXXX()
functions are fairly equivalent though since I know little about the actual
Unix implementation. From the English word, ‘block’, a SpinLock does block
that CPU but the thread remains in control of the CPU compared to the Wait
functions where another thread is usually scheduled. In fact, blocking such
as in football has more to do with spinlocks than the WaitXXX() functions.
Maybe ‘yielding’ is a better word for the WaitXXX() functions?

I was attending a course in a Honeywell facility in Phoenix, AZ in 1973
where I saw a Multics system which predates the SMP stuff in NT by almost
two decades. I don’t know when DEC’s SMP systems began and we know they
were from where the designers of NT came.

wrote in message news:xxxxx@ntdev…
> Mark,
>
>> Finally, another assertion was made (not by you) that one cannot block
>> after
>> acquiring a spinlock. This is of course not true at all. A spinlock
>> owning thread is
>> free to acquire another spinlock and will of course block waiting for
>> the current
>> owner of that lock to release it.
>> Once again perhaps we all are just having a terminology problem, and that
>> ‘block’ meant ‘wait > on an event’ in the poster’s terminology.
>
> AFAIK, the very term “blocking” comes from UNIX, which makes a very clear
> distinction between blocking and non-blocking IO (you can put your target
> file/socket/ into non-blocking mode, and if you don’t do it, you can
> always find out whether your IO operation is going to block by making
> poll() or select() system call). In UNIX terms, “blocking” means yielding
> the CPU ( a call to schedule() under Linux). Therefore, if you translate
> it into Windows terms, blocking means calling WaitXXX() with non-zero
> interval…
>
> Anton Bassov
>
>

[Don_Burn_1]

wrote in message news:xxxxx@ntdev…
> AFAIK, the very term “blocking” comes from UNIX, which makes a very clear
> distinction between blocking and non-blocking IO (you can put your target
> file/socket/ into non-blocking mode, and if you don’t do it, you can
> always find out whether your IO operation is going to block by making
> poll() or select() system call). In UNIX terms, “blocking” means yielding
> the CPU ( a call to schedule() under Linux). Therefore, if you translate
> it into Windows terms, blocking means calling WaitXXX() with non-zero
> interval…
>

Actually the early Unix papers were careful to identify the concept of
waiting which is what you describe from blocking which they included spin
locks. I believe this was because of the Multic’s heritage, but I cannot
find the reference.

I did find some stuff in the early IBM SMP papers from the mid-60’s and I
believe I remember seeing it mentioned in CDC documentation that could be
traced to the early 60’s. Of course the CDC stuff was for I/O processors
where the OS ran, coordinating with the main CPU where applications ran, if
you can accept that a multi-processors then they are hitting 50 years old.
I wanted to check the references before I refuted you claim made earlier
that there were not multi-processors 40 years ago, there certainly were!

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply

[Mark_Roddy]

blocking and non-blocking IO are very clear, however we were not discussing
IO we were discussing thread execution, and in that context threads can be
blocked from continuing execution from many different sorts of things. Like
I said, this appears to be a terminology issue only.

On Dec 23, 2007 5:44 PM, wrote:

> Mark,
>
> > Finally, another assertion was made (not by you) that one cannot block
> after
> > acquiring a spinlock. This is of course not true at all. A spinlock
> owning thread is
> > free to acquire another spinlock and will of course block waiting for
> the current
> > owner of that lock to release it.
> > Once again perhaps we all are just having a terminology problem, and
> that ‘block’ meant ‘wait > on an event’ in the poster’s terminology.
>
> AFAIK, the very term “blocking” comes from UNIX, which makes a very clear
> distinction between blocking and non-blocking IO (you can put your target
> file/socket/ into non-blocking mode, and if you don’t do it, you can always
> find out whether your IO operation is going to block by making poll() or
> select() system call). In UNIX terms, “blocking” means yielding the CPU ( a
> call to schedule() under Linux). Therefore, if you translate it into Windows
> terms, blocking means calling WaitXXX() with non-zero interval…
>
> Anton Bassov
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

–
Mark Roddy

[anton_bassov]

> Interesting definition of ‘blocking’ that I think doesn’t match the common English usage.

Absolutely agree with you here. However, it is very typical of people of any '“white-collar” profession to speak the language that a “blue-collar commoner” would not understand. First they try to come up with some sophisticated term (preferably, French or Latin one) that no one can understand. If they fail to find a “clever” term, they achieve their goal simply by taking a standard English word and slightly changing its meaning…

Maybe ‘yielding’ is a better word for the WaitXXX() functions?

Somehow it got generally accepted by the UNIX community that “block”==“sleep”. Furthermore, when it comes to TCPIP, this definition of blocking got accepted by the Windows community as well. For example, WSAEWOULDBLOCK definition in ws2_32.h speaks for itself - this return value meas that a thread that calls recv() on a blocking socket will be put to sleep until data becomes available…

Anton Bassov

[anton_bassov]

> blocking and non-blocking IO are very clear, however we were not discussing

IO we were discussing thread execution

Although blocking/non-blocking IO and thread execution are not necessarily related from the IO request initiator’s perspective, they are directly related from the driver’s one. If you have received non-blocking IO request that cannot be immediately satisfied, you are required to return right on the spot - you cannot (sorry, I meant “you should not”) yield execution. However, if you are unable to satisfy a blocking request, you are required to yield and return to the caller only when data becomes available or timeout expires (if you have entered interruptible sleep and it gets aborted by receiving a signal, request will get resubmitted by the system transparently to the actual caller)…

in that context threads can be blocked from continuing execution from many
different sorts of things.

, but spinlock acquisition is not among them. Why? Simply because spinlock acquisition is all about spinning in a loop, and spinning in a loop is just a particular form of execution - the target thread keeps on running (without doing anything particularly useful, but still running)…

Anton Bassov

[Maxim_S_Shatskih]

> Interesting definition of ‘blocking’ that I think doesn’t match the common

English usage.

It matches the UNIX usage, Anton was correct in this.

The UNIX usage was developed due to non-preemptive nature of the kernel of most
UNIXen, where “blocking” (dispatcher-implemented waiting) is the only way to
switch the current thread while executing the kernel code.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com