mutex, semaphores: same or different ?

NTDEV
1

Till today I thought of mutex as a semaphore with limit = 2 but their definitions inside wdm.h states some thing else.

A mutex or a mutent is defined as

typedef struct _KMUTANT {
DISPATCHER_HEADER Header;
LIST_ENTRY MutantListEntry;
struct _KTHREAD *OwnerThread;
BOOLEAN Abandoned;
UCHAR ApcDisable;
} KMUTANT, *PKMUTANT, *PRKMUTANT, KMUTEX, *PKMUTEX, *PRKMUTEX;

where as a semaphore definition is

typedef struct _KSEMAPHORE {
DISPATCHER_HEADER Header;
LONG Limit;
} KSEMAPHORE, *PKSEMAPHORE, *PRKSEMAPHORE;

So a there is some concept of thread ownership associated with a mutex which is not with semaphore. Also msdn states that when a mutex is acquired delivery of APCs will be disabled until it releases that mutex, this is also not applied to semaphore.(At least the structure definition has nothing which points to APC)

So what all other uses a mutex provide which a semaphore can not, in fact it looks like that a semaphore is actually a event with limit.

Thanks
Aditya

2

A semaphore is a counter. Acquiring it is only possible when the count is
non-zero. Acquiring causes the count to be decremented. If the count is
zero, attempting to acquire it blocks the thread.

A mutex allows exactly one thread to acquire it. It has a concept of
ownership to support recursive acquisition meaning that the owner can
continue to (re-acquire) the mutex without blocking. The owner, however,
must release the mutex as many times as it acquires it.

So, where a thread acquiring a semaphore repeatedly (in a nested way) will
exhaust the count in the semaphore, a thread acquiring a mutex (in a nested
way) would continue to do so without blocking until it hit the internal
limit of the nesting count (which likely causes an bugcheck but I don’t know
this is the case).

They are most definitely *not* interchangeable.

Sempahore: Often used with managed allocation of limited resource like a
pool of objects.
Mutex: Exclusive access to a single resource.

Good Luck,
Dave Cattley

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Monday, November 02, 2009 7:26 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] mutex, semaphores: same or different ?

Till today I thought of mutex as a semaphore with limit = 2 but their
definitions inside wdm.h states some thing else.

A mutex or a mutent is defined as

typedef struct _KMUTANT {
DISPATCHER_HEADER Header;
LIST_ENTRY MutantListEntry;
struct _KTHREAD *OwnerThread;
BOOLEAN Abandoned;
UCHAR ApcDisable;
} KMUTANT, *PKMUTANT, *PRKMUTANT, KMUTEX, *PKMUTEX, *PRKMUTEX;

where as a semaphore definition is

typedef struct _KSEMAPHORE {
DISPATCHER_HEADER Header;
LONG Limit;
} KSEMAPHORE, *PKSEMAPHORE, *PRKSEMAPHORE;

So a there is some concept of thread ownership associated with a mutex which
is not with semaphore. Also msdn states that when a mutex is acquired
delivery of APCs will be disabled until it releases that mutex, this is also
not applied to semaphore.(At least the structure definition has nothing
which points to APC)

So what all other uses a mutex provide which a semaphore can not, in fact it
looks like that a semaphore is actually a event with limit.

Thanks
Aditya

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

3

So the owner of a mutex can acquire it recursively. Concerned system component(executive or dispatcher) can actually refer to OwnerThread field when a call to acquire a mutex reached them, if it is from owner they’ll allow it or else the calling thread will start waiting on it.

Any guesses on what MutantListEntry role is inside the list. Are all mutent in system linked?

Thanks
Aditya

4

From web I found that because semaphores do not have a owner associated with it and hence can have issues like when another thread accidentally release them or acquire them against the protocol. Mutex address this problem by including ownership, the thread which owns it is only be able to release it.

5

The role of the internal structure elements of these items is private to the
Kernel. In reality, only the ‘size’ of the structure is public.

You really should only be concerned with the ‘contract’ behavior of the
object given its supporting DDIs.

The systems use of the field MutantListEntry is, well, undocumented. I
could guess but what good would that be? I might be right and yet it might
change.

You might surmise that the executive acquires a lock, compares the owner
with the calling thread, and makes a decision as to allow the recursive
acquisition (or initial acquisition) based on the comparison. If the
acquisition cannot be performed then the thread is suspended. Whether or
not the thread is placed somehow on a list of waiters to ensure FiFo
acquisition, I don’t know. But all of this is just speculation really based
on how ‘typically’ these things are done in an OS and not based on what
actually happens in NTOSKRNL. Maybe somebody else will be happy to tell you
how it is implemented in a specific version of NT.

Good Luck,
Dave Cattley

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Monday, November 02, 2009 8:12 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] mutex, semaphores: same or different ?

So the owner of a mutex can acquire it recursively. Concerned system
component(executive or dispatcher) can actually refer to OwnerThread field
when a call to acquire a mutex reached them, if it is from owner they’ll
allow it or else the calling thread will start waiting on it.

Any guesses on what MutantListEntry role is inside the list. Are all mutent
in system linked?

Thanks
Aditya

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

6

>>You really should only be concerned with the ‘contract’ behavior of the object given its supporting DDIs.

Yes I totally agree to that and it has nothing to do with any type of product work. Just curious to understand how exactly it works.

>I could guess but what good would that be?

I thought I read somewhere in OSR that MVPs have windows source code so just felt a bit greedy and asked this.

Thanks for your time,
Aditya

7

“David R. Cattley” wrote in message news:xxxxx@ntdev…
> A mutex allows exactly one thread to acquire it. It has a concept of
> ownership to support recursive acquisition meaning that the owner can
> continue to (re-acquire) the mutex without blocking. The owner, however,
> must release the mutex as many times as it acquires it.
>

Let’s note that this holds true for standard mutexes only but not for fast
mutex or guarded mutexes which will deadlock if acquire recursively.

//Daniel

8

While reading win intrnl I noticed a line written for fast_mutex

“they avoid waiting for the event object (and therefore the spinlocks on which an event object is based)”

*Event based on spinlocks ???*

Event is a dispatcher object and are used at below dispatch level IRQL. Where as A spinlock is based on a atomic test and reset instruction and also has a IRQL associated with it and can be used at dispatch or above level.

I can understand that two threads on different processors may try to change signal state at same time and hence synchronization across processors is required.

Does it means that when both the processors try to change the state of an event from user mode, thread executing on them will switch to kernel mode reaches dispatcher and than it try to acquire some specific spinlock and than change the signaledstate ?

Thanks
Aditya

9

>>Let’s note that this holds true for standard mutexes only but not for fast mutex or guarded mutexes which will deadlock if acquire recursively.

Yes I read this but only detail they mentioned is "it will not wait on an event if there’s no contention for the fast mutex.

dt _FAST_MUTEX
+0x000 Count : Int4B
+0x004 Owner : Ptr32 _KTHREAD
+0x008 Contention : Uint4B
+0x00c Event : _KEVENT
+0x01c OldIrql : Uint4B

Does that means that if no other thread is asking for this fast_mutex, they will just not use event? it implies that when some other thread wants this object at same time they do use events. If so how actually it delivers better performance on multi-processor as mentioned in the book? Does it mean that chances of two threads asking for same synchronization objects at same time are sparsed?

Thanks
Aditya

10

xxxxx@gmail.com wrote:

I thought I read somewhere in OSR that MVPs have windows source code so just felt a bit greedy and asked this.

DDK MVPs have the option of requesting source code access. The access
comes with rather severe usage and disclosure restrictions, and many of
us have decided so far that the reward wasn’t worth the risk.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

11

> Does it means that when both the processors try to change the state of an event from user mode,

thread executing on them will switch to kernel mode reaches dispatcher and than it try to acquire some
specific spinlock and than change the signaledstate ?

Pre-Win7 - yes, in Win7, events are lock free data structures IIRC.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

12

> Does that means that if no other thread is asking for this fast_mutex, they will just not use event?

Yes.

ExAcquireFastMutex:
if( InterlockedIncrement(T->Count) != 1 )
WaitOnSyncEvent

ExReleaseFastMutex:
if( InterlockedDecrement(T->Count) != 0 )
SetEvent

With no contention, there is no calls to dispatcher at all and no acquisitions of KiDispatcherLock.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

13

> DDK MVPs have the option of requesting source code access. The access

comes with rather severe usage and disclosure restrictions, and many of
us have decided so far that the reward wasn’t worth the risk.

Country-based restrictions, for instance, (only the rich Western countries and their close political friends like the 3 Baltian countries).

Ex-USSR - no. Asian-Pacific (except Japan) - I don’t know. India - I don’t know.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

14

Thanks Maxim, you are a great help.