shared-to-exclusive resource question

NTFSD
1

Hi,

Is there a standard / highly efficient way to acquire a resource in
exclusive fashion, if we already hold it in shared way, without allowing
any other thread to acquire it in exclusive way before us? (semantically
this would be somehow the inverse of ExConvertExclusiveToSharedLite)

The scenario where I would need this is the following: I have a quite
huge list of objects, protected by a resource. I typically only need to
get a reference to one of those objects, if it is already in the list. For
this, I acquire the resource in a shared fashion. Now, in certain
situations,
when I walk through the list and do NOT found the particular object I’m
looking for, I need to allocate a new object and insert it into the list.
Now, for this I need exclusive access. But, if I already have shared access,
then I must previously release the shared resource. After this, if I get the
exclusive access I can’t be sure, that some other thread meanwhile didn’t
already inserted that object into the thread (in a highly concurrent way).
What I want to avoid is the re-verification of the list (looking for the
object) after I get the exclusive access.

any suggestions how to solve this?

thank you, have a nice day,

Sandor LUKACS
Virus Analyst, SOFTWIN

2

Hello,
if you use ERESOURCE, the same thread cannot convert shared access to
exclusive access. ExConvertExclusiveToSharedLite changes thread’s access to
shared and all threads that are waiting for shared access.

It’s possible to use NDIS_RW_LOCK
(NdisInitializeReadWriteLock/NdisAcquireReadWriteLock) - both shared and
exclusive access raises to DPC_LEVEL (readers can re-enter), but if your
driver was FS you wouldn’t like to raise IRQL to DISPATCH_LEVEL.

So, how often do you update the list? How long does it take and what
structure do you use? Wouldn’t be easier to look up the object again with
the exclusive access if it doesnt take too much time ? I guess you use some
kind of hash table, red-black/avl tree.

Petr Kurtin
ALWIL Software

“Sandor LUKACS” wrote in message
news:xxxxx@ntfsd…
> Hi,
>
> Is there a standard / highly efficient way to acquire a resource in
> exclusive fashion, if we already hold it in shared way, without allowing
> any other thread to acquire it in exclusive way before us? (semantically
> this would be somehow the inverse of ExConvertExclusiveToSharedLite)
>
> The scenario where I would need this is the following: I have a quite
> huge list of objects, protected by a resource. I typically only need to
> get a reference to one of those objects, if it is already in the list. For
> this, I acquire the resource in a shared fashion. Now, in certain
> situations,
> when I walk through the list and do NOT found the particular object I’m
> looking for, I need to allocate a new object and insert it into the list.
> Now, for this I need exclusive access. But, if I already have shared
> access,
> then I must previously release the shared resource. After this, if I get
> the
> exclusive access I can’t be sure, that some other thread meanwhile didn’t
> already inserted that object into the thread (in a highly concurrent way).
> What I want to avoid is the re-verification of the list (looking for the
> object) after I get the exclusive access.
>
> any suggestions how to solve this?
>
> thank you, have a nice day,
>
> Sandor LUKACS
> Virus Analyst, SOFTWIN
>
>

3

Hi,

thank you for the reply,

Petr Kurtin wrote:

Hello,
if you use ERESOURCE, the same thread cannot convert shared access to
exclusive access. ExConvertExclusiveToSharedLite changes thread’s access to
shared and all threads that are waiting for shared access.

It’s possible to use NDIS_RW_LOCK
(NdisInitializeReadWriteLock/NdisAcquireReadWriteLock) - both shared and
exclusive access raises to DPC_LEVEL (readers can re-enter), but if your
driver was FS you wouldn’t like to raise IRQL to DISPATCH_LEVEL.

I will check this, but yes, I’m working on a FS stuff, so it is unlikely
I will use it.

So, how often do you update the list? How long does it take and what
structure do you use? Wouldn’t be easier to look up the object again with
the exclusive access if it doesnt take too much time ? I guess you use some
kind of hash table, red-black/avl tree.

Actually the time when I need to update the list varies significantly,
from scenario to
scenario. Let’s say, 80% of the cases are ‘good’ ones, when I don’t
receive a lot of inserts,
but there are real life scenarios, which will result in an almost
constant insert of a couple
of hundreds of objects, and I just wanted to optimize for those cases.
But, if there is no
clean way to optimize this, I think I will remain on reverifying this
when we get the
exclusive access.

have a nice day,

Sandor LUKACS
Virus Analyst, SOFTWIN

Petr Kurtin
ALWIL Software

“Sandor LUKACS” wrote in message
> news:xxxxx@ntfsd…
>
>> Hi,
>>
>> Is there a standard / highly efficient way to acquire a resource in
>> exclusive fashion, if we already hold it in shared way, without allowing
>> any other thread to acquire it in exclusive way before us? (semantically
>> this would be somehow the inverse of ExConvertExclusiveToSharedLite)
>>
>> The scenario where I would need this is the following: I have a quite
>> huge list of objects, protected by a resource. I typically only need to
>> get a reference to one of those objects, if it is already in the list. For
>> this, I acquire the resource in a shared fashion. Now, in certain
>> situations,
>> when I walk through the list and do NOT found the particular object I’m
>> looking for, I need to allocate a new object and insert it into the list.
>> Now, for this I need exclusive access. But, if I already have shared
>> access,
>> then I must previously release the shared resource. After this, if I get
>> the
>> exclusive access I can’t be sure, that some other thread meanwhile didn’t
>> already inserted that object into the thread (in a highly concurrent way).
>> What I want to avoid is the re-verification of the list (looking for the
>> object) after I get the exclusive access.
>>
>> any suggestions how to solve this?
>>
>> thank you, have a nice day,
>>
>> Sandor LUKACS
>> Virus Analyst, SOFTWIN
>>
>>
>>
>
>
>
> —
> Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@bitdefender.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>

4

Just acquire it exclusive, while holding shared. Then release both
exclusive and shared acquisitions.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

“Sandor LUKACS” wrote in message news:xxxxx@ntfsd…
> Hi,
>
> Is there a standard / highly efficient way to acquire a resource in
> exclusive fashion, if we already hold it in shared way, without allowing
> any other thread to acquire it in exclusive way before us? (semantically
> this would be somehow the inverse of ExConvertExclusiveToSharedLite)
>
> The scenario where I would need this is the following: I have a quite
> huge list of objects, protected by a resource. I typically only need to
> get a reference to one of those objects, if it is already in the list. For
> this, I acquire the resource in a shared fashion. Now, in certain
> situations,
> when I walk through the list and do NOT found the particular object I’m
> looking for, I need to allocate a new object and insert it into the list.
> Now, for this I need exclusive access. But, if I already have shared access,
> then I must previously release the shared resource. After this, if I get the
> exclusive access I can’t be sure, that some other thread meanwhile didn’t
> already inserted that object into the thread (in a highly concurrent way).
> What I want to avoid is the re-verification of the list (looking for the
> object) after I get the exclusive access.
>
> any suggestions how to solve this?
>
> thank you, have a nice day,
>
> Sandor LUKACS
> Virus Analyst, SOFTWIN
>
>

5

thank you for the reply,

Maxim S. Shatskih wrote:

Just acquire it exclusive, while holding shared. Then release both
exclusive and shared acquisitions.
I don’t think that is the right way to do it, as

http://www.osronline.com/showThread.cfm?link=29375

If you acquire a resource shared and in the same thread you acquire it
>again exclusive that will cause your thread to deadlock. You cannot
>convert shared to exclusive.
>
>The otherway around is possible. You can acquire exclusive and then
>if you acquire shared you will be granted recursive exclusive access.
>This is documented in the DDK

now, may I really acquire it exclusive while holding shared?

Sandor LUKACS
Virus Analyst, SOFTWIN

6

> now, may I really acquire it exclusive while holding shared?

call ExAcquireResourceExclusiveLite with Wait=TRUE

7

well, but from different thread
it’s not possible to convert shared access to exclusive

“Petr Kurtin” wrote in message news:xxxxx@ntfsd…
>> now, may I really acquire it exclusive while holding shared?
>
> call ExAcquireResourceExclusiveLite with Wait=TRUE
>
>
>

8

Petr Kurtin wrote:

> now, may I really acquire it exclusive while holding shared?
>

call ExAcquireResourceExclusiveLite with Wait=TRUE

now, I’m really confused :frowning:
the WDK says: “If the caller has shared access to the resource, the
caller *must*
release the lock before attempting to reacquire it exclusively.”

now, if before I call ExAcquireResourceExclusiveLite, I do release the lock
by calling ExReleaseResourceLite, then isn’t it possible for any other
thread
to acquire the resource exclusively BEFORE the current thread?
OR, are the WDK docs wrong, and I do NOT need to release the already
owned (shared) resource before calling ExAcquireResourceExclusiveLite?

thank you for you replies,

Sandor LUKACS
Virus Analyst, SOFTWIN

9

Petr Kurtin wrote:

well, but from different thread

ok, that explains all :slight_smile:

have a nice day,

Sandor LUKACS
Virus Analyst, SOFTWIN

it’s not possible to convert shared access to exclusive

“Petr Kurtin” wrote in message news:xxxxx@ntfsd…
>
>>> now, may I really acquire it exclusive while holding shared?
>>>
>> call ExAcquireResourceExclusiveLite with Wait=TRUE
>>
>>
>>

10

There is no way to upgrade from shared to exclusive lock in a deadlock-free
way. Suppose two concurrent threads are holding shared resource and are
trying to upgrade it to exclusive - since you can’t have exclusive lock when
another thread is holding shared lock these two threads would wait each
other. The only possible approach is the one you are using - release shared,
acquire exclusive lock and re-verify the conditions.

Alexei.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Sandor LUKACS
Sent: Monday, May 14, 2007 1:22 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] shared-to-exclusive resource question

Hi,

Is there a standard / highly efficient way to acquire a resource in
exclusive fashion, if we already hold it in shared way, without allowing
any other thread to acquire it in exclusive way before us? (semantically
this would be somehow the inverse of ExConvertExclusiveToSharedLite)

The scenario where I would need this is the following: I have a quite
huge list of objects, protected by a resource. I typically only need to
get a reference to one of those objects, if it is already in the list. For
this, I acquire the resource in a shared fashion. Now, in certain
situations,
when I walk through the list and do NOT found the particular object I’m
looking for, I need to allocate a new object and insert it into the list.
Now, for this I need exclusive access. But, if I already have shared access,
then I must previously release the shared resource. After this, if I get the
exclusive access I can’t be sure, that some other thread meanwhile didn’t
already inserted that object into the thread (in a highly concurrent way).
What I want to avoid is the re-verification of the list (looking for the
object) after I get the exclusive access.

any suggestions how to solve this?

thank you, have a nice day,

Sandor LUKACS
Virus Analyst, SOFTWIN

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@vmware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

11

Aside: How I miss LCK$M_CONVERT! Mind you the associated semantics were
often challenging to understand and easy to exploit wrongly…