releasing a spinlock when not at dispatch level

This is a strange bugcheck that verifier is throwing.

I have three global linked lists, all initialised at DriverLoad with their associated spinlocks. There is a worker thread that processes the lists.

The problem comes at driver unload when I get this error.

Basically I acquire all three spinlocks during the unload procedure, this should halt the worker threads, and then set a boolean flag to true and release the spinlocks. The worker threads can then perform their shutdown procedures by emptying the lists.

It was working fine for 2 lists, now I have added a third list and it gives me this problem.

I have checked the code over thoroughly and there is no “double releasing” going on so I have no idea why the dispatch level would lower to passive when I have 3 spinlocks. I thought that acquiring a spinlock raised the IRQL to dispatch anyway.

Any ideas?

Did you release the spin locks in the reverse order than you acquired
them? If not, this might be the problem.

xxxxx@yahoo.co.uk
Sent by: xxxxx@lists.osr.com
08/24/2010 10:57 AM
Please respond to
“Windows System Software Devs Interest List”

To
“Windows System Software Devs Interest List”
cc

Subject
[ntdev] releasing a spinlock when not at dispatch level

This is a strange bugcheck that verifier is throwing.

I have three global linked lists, all initialised at DriverLoad with their
associated spinlocks. There is a worker thread that processes the lists.

The problem comes at driver unload when I get this error.

Basically I acquire all three spinlocks during the unload procedure, this
should halt the worker threads, and then set a boolean flag to true and
release the spinlocks. The worker threads can then perform their shutdown
procedures by emptying the lists.

It was working fine for 2 lists, now I have added a third list and it
gives me this problem.

I have checked the code over thoroughly and there is no “double releasing”
going on so I have no idea why the dispatch level would lower to passive
when I have 3 spinlocks. I thought that acquiring a spinlock raised the
IRQL to dispatch anyway.

Any ideas?


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

You might want to post the code.

Mark Roddy

On Tue, Aug 24, 2010 at 10:59 AM, wrote:

> This is a strange bugcheck that verifier is throwing.
>
> I have three global linked lists, all initialised at DriverLoad with their
> associated spinlocks. There is a worker thread that processes the lists.
>
> The problem comes at driver unload when I get this error.
>
> Basically I acquire all three spinlocks during the unload procedure, this
> should halt the worker threads, and then set a boolean flag to true and
> release the spinlocks. The worker threads can then perform their shutdown
> procedures by emptying the lists.
>
> It was working fine for 2 lists, now I have added a third list and it gives
> me this problem.
>
> I have checked the code over thoroughly and there is no “double releasing”
> going on so I have no idea why the dispatch level would lower to passive
> when I have 3 spinlocks. I thought that acquiring a spinlock raised the IRQL
> to dispatch anyway.
>
> Any ideas?
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

@Jerry: You are of course right. Schoolboy error. Thanks.

@Mark: Trust me you would not like to see my code :wink:

I have in fact sorted out the problem with a far more simple solution. I have restructured the shut down procedure so that the worker thread just exits, and nothing more. It then leaves the lists populated and in the current state. Then I have written a function to unload the lists one by one, and call this after the worker thread has been shut down. Works like a champ, and I have no idea why the microsoft example tries to do it any other way. I am pleased I had the confidence to try another approach and use my own logic. I think I read on here some time ago about the semantics of unloading linked lists on driver shutdown and I think it is considered good practice to have one function that initialises the lists and one that tears them down.

Thanks for your help. Can this thread be locked?

It may look prettier to release in reverse order of acquisition, and
prefast might shutup if you do that, but I do not know of any functional
requirement to do this, as release cannot deadlock. You do have to maintain
the correct IRQL state, and if you release and re-acquire a set of locks
acquisition order of course has to be maintained.

Mark Roddy

On Tue, Aug 24, 2010 at 12:04 PM, wrote:

> @Jerry: You are of course right. Schoolboy error. Thanks.
>
>
> @Mark: Trust me you would not like to see my code :wink:
>
> I have in fact sorted out the problem with a far more simple solution. I
> have restructured the shut down procedure so that the worker thread just
> exits, and nothing more. It then leaves the lists populated and in the
> current state. Then I have written a function to unload the lists one by
> one, and call this after the worker thread has been shut down. Works like a
> champ, and I have no idea why the microsoft example tries to do it any other
> way. I am pleased I had the confidence to try another approach and use my
> own logic. I think I read on here some time ago about the semantics of
> unloading linked lists on driver shutdown and I think it is considered good
> practice to have one function that initialises the lists and one that tears
> them down.
>
> Thanks for your help. Can this thread be locked?
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Well … each call to KeAcquireSpinLock is going to return an OldIrql. So
the first call is going to return the lowest level, potentially lower than
DISPATCH, and the others are going to return DISPATCH. I assume he is
going to release each spin lock using the OldIrql that was returned when
he acquired that lock. So if he releases the spin locks in the order he
acquired them, the first is going to potentially lower IRQL, and the next
ones are going to try to raise it back to DISPATCH, a no-no.

Mark Roddy
Sent by: xxxxx@lists.osr.com
08/24/2010 12:39 PM
Please respond to
“Windows System Software Devs Interest List”

To
“Windows System Software Devs Interest List”
cc

Subject
Re: [ntdev] releasing a spinlock when not at dispatch level

It may look prettier to release in reverse order of acquisition, and
prefast might shutup if you do that, but I do not know of any functional
requirement to do this, as release cannot deadlock. You do have to
maintain the correct IRQL state, and if you release and re-acquire a set
of locks acquisition order of course has to be maintained.

Mark Roddy

On Tue, Aug 24, 2010 at 12:04 PM, wrote:
@Jerry: You are of course right. Schoolboy error. Thanks.

@Mark: Trust me you would not like to see my code :wink:

I have in fact sorted out the problem with a far more simple solution. I
have restructured the shut down procedure so that the worker thread just
exits, and nothing more. It then leaves the lists populated and in the
current state. Then I have written a function to unload the lists one by
one, and call this after the worker thread has been shut down. Works like
a champ, and I have no idea why the microsoft example tries to do it any
other way. I am pleased I had the confidence to try another approach and
use my own logic. I think I read on here some time ago about the semantics
of unloading linked lists on driver shutdown and I think it is considered
good practice to have one function that initialises the lists and one that
tears them down.

Thanks for your help. Can this thread be locked?


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Also, make sure your DriverUnload waits for the thread(s) to finish, by waiting on their thread object(s).

Once at DISPATCH_LEVEL it is recommended to use KeAcquireSpinlockAtDPCLevel
and of course the Release version. As was stated you must release the locks
in reverse order, making sure that the return to PASSIVE_LEVEL is the last
one released.

Gary G. Little

H (952) 223-1349

C (952) 454-4629

xxxxx@comcast.net

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@attotech.com
Sent: Tuesday, August 24, 2010 11:59 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] releasing a spinlock when not at dispatch level

Well … each call to KeAcquireSpinLock is going to return an OldIrql. So
the first call is going to return the lowest level, potentially lower than
DISPATCH, and the others are going to return DISPATCH. I assume he is going
to release each spin lock using the OldIrql that was returned when he
acquired that lock. So if he releases the spin locks in the order he
acquired them, the first is going to potentially lower IRQL, and the next
ones are going to try to raise it back to DISPATCH, a no-no.

Mark Roddy
Sent by: xxxxx@lists.osr.com

08/24/2010 12:39 PM

Please respond to
“Windows System Software Devs Interest List”

To

“Windows System Software Devs Interest List”

cc

Subject

Re: [ntdev] releasing a spinlock when not at dispatch level

It may look prettier to release in reverse order of acquisition, and
prefast might shutup if you do that, but I do not know of any functional
requirement to do this, as release cannot deadlock. You do have to maintain
the correct IRQL state, and if you release and re-acquire a set of locks
acquisition order of course has to be maintained.

Mark Roddy

On Tue, Aug 24, 2010 at 12:04 PM, wrote:
@Jerry: You are of course right. Schoolboy error. Thanks.

@Mark: Trust me you would not like to see my code :wink:

I have in fact sorted out the problem with a far more simple solution. I
have restructured the shut down procedure so that the worker thread just
exits, and nothing more. It then leaves the lists populated and in the
current state. Then I have written a function to unload the lists one by
one, and call this after the worker thread has been shut down. Works like a
champ, and I have no idea why the microsoft example tries to do it any other
way. I am pleased I had the confidence to try another approach and use my
own logic. I think I read on here some time ago about the semantics of
unloading linked lists on driver shutdown and I think it is considered good
practice to have one function that initialises the lists and one that tears
them down.

Thanks for your help. Can this thread be locked?


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the
List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the
List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Just to be pedantic, you do not have to release them in the reverse order to
insure that you return to the correct irql, you have to insure that you
return to the correct irql when your last lock is released.

The first acquisition gets the oldirql value. The last release restores the
old irql value. The irql value to restore is a variable you control and
provide.

But indeed your code looks prettier if you make it symmetric.

Mark Roddy

On Tue, Aug 24, 2010 at 1:46 PM, Gary G. Little wrote:

> Once at DISPATCH_LEVEL it is recommended to use
> KeAcquireSpinlockAtDPCLevel and of course the Release version. As was stated
> you must release the locks in reverse order, making sure that the return to
> PASSIVE_LEVEL is the last one released.
>
>
>
> Gary G. Little
>
> H (952) 223-1349
>
> C (952) 454-4629
>
> xxxxx@comcast.net
>
>
>
> From: xxxxx@lists.osr.com [mailto:
> xxxxx@lists.osr.com] *On Behalf Of xxxxx@attotech.com
> Sent: Tuesday, August 24, 2010 11:59 AM
>
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] releasing a spinlock when not at dispatch level
>
>
>
>
> Well … each call to KeAcquireSpinLock is going to return an OldIrql. So
> the first call is going to return the lowest level, potentially lower than
> DISPATCH, and the others are going to return DISPATCH. I assume he is going
> to release each spin lock using the OldIrql that was returned when he
> acquired that lock. So if he releases the spin locks in the order he
> acquired them, the first is going to potentially lower IRQL, and the next
> ones are going to try to raise it back to DISPATCH, a no-no.
>
>
> Mark Roddy
> Sent by: xxxxx@lists.osr.com
>
> 08/24/2010 12:39 PM
>
> Please respond to
> “Windows System Software Devs Interest List”
>
> To
>
> “Windows System Software Devs Interest List”
>
> cc
>
> Subject
>
> Re: [ntdev] releasing a spinlock when not at dispatch level
>
>
>
>
>
>
> It may look prettier to release in reverse order of acquisition, and
> prefast might shutup if you do that, but I do not know of any functional
> requirement to do this, as release cannot deadlock. You do have to maintain
> the correct IRQL state, and if you release and re-acquire a set of locks
> acquisition order of course has to be maintained.
>
> Mark Roddy
>
>
> On Tue, Aug 24, 2010 at 12:04 PM, wrote:
> @Jerry: You are of course right. Schoolboy error. Thanks.
>
>
> @Mark: Trust me you would not like to see my code :wink:
>
> I have in fact sorted out the problem with a far more simple solution. I
> have restructured the shut down procedure so that the worker thread just
> exits, and nothing more. It then leaves the lists populated and in the
> current state. Then I have written a function to unload the lists one by
> one, and call this after the worker thread has been shut down. Works like a
> champ, and I have no idea why the microsoft example tries to do it any other
> way. I am pleased I had the confidence to try another approach and use my
> own logic. I think I read on here some time ago about the semantics of
> unloading linked lists on driver shutdown and I think it is considered good
> practice to have one function that initialises the lists and one that tears
> them down.
>
> Thanks for your help. Can this thread be locked?
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:

> *http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit
> the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

> Just to be pedantic, you do not have to release them in the reverse order to insure that you

return to the correct irql, you have to insure that you return to the correct irql when your last lock
is released.

Consider the following scenario:

KIRQL irql1,irql2;

KeAcquireSpinLock(&spinlock1, &irql1);

KeAcquireSpinLock(&spinlock2, &irql2);

KeReleaseSpinLock(&spinlock1, Irql1);

Assuming that the original call was made at IRQL< DPC_LEVEL the above line will set IRQL to its original level, despite the fact that spinlock2 is still being held…

Anton Bassov

> KIRQL irql1,irql2;

KeAcquireSpinLock(&spinlock1, &irql1);

KeAcquireSpinLock(&spinlock2, &irql2);

KeReleaseSpinLock(&spinlock1, Irql1);

Assuming that the original call was made at IRQL< DPC_LEVEL the above line will set IRQL to its original level, despite the fact that spinlock2 is still being held…

…and then the release of spinlock2 will try to raise back to DISPATCH, since irql2 is DISPATCH.

Voila the bug. Spinlock release cannot raise the IRQL.


Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

As I said - you have to maintain irql correctly, but that does not require
release ordering.

KIRQL originalIrql;

KeAcquireSpinLock(&spinlock1, &originalIrql);
KeAcquireSpinLockAtDpcLevel(&spinlock2);

… do some stuff

KeReleaseSpinLockFromDpcLevel(&spinlock1);

… do some more stuff

KeReleaseSpinLock(&spinlock2, originalIrql);

What violation of anything other than our affinity for symmetry happened?

Mark Roddy

On Tue, Aug 24, 2010 at 7:27 PM, wrote:

> > Just to be pedantic, you do not have to release them in the reverse order
> to insure that you
> > return to the correct irql, you have to insure that you return to the
> correct irql when your last lock
> > is released.
>
> Consider the following scenario:
>
> KIRQL irql1,irql2;
>
> KeAcquireSpinLock(&spinlock1, &irql1);
>
> KeAcquireSpinLock(&spinlock2, &irql2);
>
>
> …
>
>
>
> KeReleaseSpinLock(&spinlock1, Irql1);
>
>
> Assuming that the original call was made at IRQL< DPC_LEVEL the above line
> will set IRQL to its original level, despite the fact that spinlock2 is
> still being held…
>
>
> Anton Bassov
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

>

As I said - you have to maintain irql correctly, but that does not
require
release ordering.

KIRQL originalIrql;

KeAcquireSpinLock(&spinlock1, &originalIrql);
KeAcquireSpinLockAtDpcLevel(&spinlock2);

… do some stuff

KeReleaseSpinLockFromDpcLevel(&spinlock1);

… do some more stuff

KeReleaseSpinLock(&spinlock2, originalIrql);

What violation of anything other than our affinity for symmetry
happened?

A violation of the rules. You are doing exactly what the docs say not to
do. From KeReleaseSpinLockFromDpcLevel:

"
It is an error to call KeReleaseSpinLockFromDpcLevel if the given spin
lock was acquired by calling KeAcquireSpinLock because the caller’s
original IRQL is not restored, which can cause deadlocks or fatal page
faults.
"

And from KeReleaseSpinLock:

"
VOID
KeReleaseSpinLock(
IN PKSPIN_LOCK SpinLock,
IN KIRQL NewIrql
);

NewIrql
Specifies the IRQL value saved from the preceding call to
KeAcquireSpinLock.
"

Now I know and you know that the current implementation of those
functions means that what you have done above gives the correct result,
and it’s probably reasonable to assume that it will never cause a
problem in any future version of Windows, but it’s a violation of the
contract and therefore should be frowned upon unless there is a very
good reason for doing it. I’d like to see a legitimate reason for
releasing the locks in any order than the reverse of the order in which
they were acquired.

Many regressions are caused by people thinking they can violate the
rules because they know how it works and know they will get the correct
result, even if the rules appear dumb.

James

wrote in message news:xxxxx@ntdev…
>> Just to be pedantic, you do not have to release them in the reverse order
>> to insure that you
>> return to the correct irql, you have to insure that you return to the
>> correct irql when your last lock
>> is released.

DEC operating systems, IIRC, did not return IRQL to caller,
maybe exactly because of this concern. All functions that raise IRQL
worked thru callbacks - like KeSynchronizeExecution vs.
KeAcquireInterruptSpinLock.
Saving and restoring IRQL was hidden from users. Also, because of
callbacks,
changes of IRQL occurred in stack order, and, naturally, restored in
reverse order.

In NT, they thought that callbacks are too much overhead (true!) and had to
expose previous IRQL to programmer.
Everything has its price.

Regards,
– pa

> Consider the following scenario:
>
> KIRQL irql1,irql2;
>
> KeAcquireSpinLock(&spinlock1, &irql1);
>
> KeAcquireSpinLock(&spinlock2, &irql2);
>
> …
>
>
> KeReleaseSpinLock(&spinlock1, Irql1);
>
>
> Assuming that the original call was made at IRQL< DPC_LEVEL the above line
> will set IRQL to its original level, despite the fact that spinlock2 is
> still being held…
>
>
> Anton Bassov
>

The correct IRQL is maintained entirely by the dispatch spinlock user which
is why KeAcquireSpinLockAtDpcLevel can exist. Yes I understand what the docs
say. And as I have said repeatedly, you do have to maintain the IRQL state
correctly, but what does that actually mean?

I am claiming that it means this:
In a sequence of dispatch spinlock acquisitions, the first acquire sets the
IRQL state from <= DISPATCH_LEVEL to DISPATCH_LEVEL and saves the initial
IRQL state. The last release restores the IRQL level from DISPATCH_LEVEL to
the saved IRQL state. The optimization KeAcquireSpinLockAtDpcLevel exists
because the client, the user of spinlocks, knows the IRQL state I just
described and can make an informed decision at acquire time regarding which
form of acquire to use.

There is no functional requirement to do the release of the locks in any
specific order. Docs and prefast may differ. This isn’t an implementation
detail it is fundamental to dispatch spinlocks. You cannot cause a spinlock
deadlock on lock release, but you do have to maintain the IRQL correctly.

Mark Roddy

On Wed, Aug 25, 2010 at 12:34 AM, James Harper <
xxxxx@bendigoit.com.au> wrote:

>
> As I said - you have to maintain irql correctly, but that does not
require
> release ordering.
>
> KIRQL originalIrql;
>
> KeAcquireSpinLock(&spinlock1, &originalIrql);
> KeAcquireSpinLockAtDpcLevel(&spinlock2);
>
> … do some stuff
>
> KeReleaseSpinLockFromDpcLevel(&spinlock1);
>
> … do some more stuff
>
> KeReleaseSpinLock(&spinlock2, originalIrql);
>
> What violation of anything other than our affinity for symmetry
happened?
>

A violation of the rules. You are doing exactly what the docs say not to
do. From KeReleaseSpinLockFromDpcLevel:

"
It is an error to call KeReleaseSpinLockFromDpcLevel if the given spin
lock was acquired by calling KeAcquireSpinLock because the caller’s
original IRQL is not restored, which can cause deadlocks or fatal page
faults.
"

And from KeReleaseSpinLock:

"
VOID
KeReleaseSpinLock(
IN PKSPIN_LOCK SpinLock,
IN KIRQL NewIrql
);

NewIrql
Specifies the IRQL value saved from the preceding call to
KeAcquireSpinLock.
"

Now I know and you know that the current implementation of those
functions means that what you have done above gives the correct result,
and it’s probably reasonable to assume that it will never cause a
problem in any future version of Windows, but it’s a violation of the
contract and therefore should be frowned upon unless there is a very
good reason for doing it. I’d like to see a legitimate reason for
releasing the locks in any order than the reverse of the order in which
they were acquired.

Many regressions are caused by people thinking they can violate the
rules because they know how it works and know they will get the correct
result, even if the rules appear dumb.

James


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

I’ll simplify the problem:

//
// always called at DISPATCH_LEVEL
//
foo()
{
KeAcquireSpinLockAtDpcLevel(&spinlock1);
KeAcquireSpinLockAtDpcLevel(&spinlock2);
stuffRequiringLock1andLock2();
KeReleaseSpinLockFromDpcLevel(&spinlock1);
stuffRequiringLock2();
KeReleaseSpinLockFromDpcLevel(&spinlock2);
}

ok - now that IRQL is out of the picture, what exactly is the problem?

Mark Roddy

On Wed, Aug 25, 2010 at 10:46 AM, Mark Roddy wrote:

> The correct IRQL is maintained entirely by the dispatch spinlock user which
> is why KeAcquireSpinLockAtDpcLevel can exist. Yes I understand what the docs
> say. And as I have said repeatedly, you do have to maintain the IRQL state
> correctly, but what does that actually mean?
>
> I am claiming that it means this:
> In a sequence of dispatch spinlock acquisitions, the first acquire sets the
> IRQL state from <= DISPATCH_LEVEL to DISPATCH_LEVEL and saves the initial
> IRQL state. The last release restores the IRQL level from DISPATCH_LEVEL to
> the saved IRQL state. The optimization KeAcquireSpinLockAtDpcLevel exists
> because the client, the user of spinlocks, knows the IRQL state I just
> described and can make an informed decision at acquire time regarding which
> form of acquire to use.
>
> There is no functional requirement to do the release of the locks in any
> specific order. Docs and prefast may differ. This isn’t an implementation
> detail it is fundamental to dispatch spinlocks. You cannot cause a spinlock
> deadlock on lock release, but you do have to maintain the IRQL correctly.
>
> Mark Roddy
>
>
> On Wed, Aug 25, 2010 at 12:34 AM, James Harper <
> xxxxx@bendigoit.com.au> wrote:
>
>> >
>> > As I said - you have to maintain irql correctly, but that does not
>> require
>> > release ordering.
>> >
>> > KIRQL originalIrql;
>> >
>> > KeAcquireSpinLock(&spinlock1, &originalIrql);
>> > KeAcquireSpinLockAtDpcLevel(&spinlock2);
>> >
>> > … do some stuff
>> >
>> > KeReleaseSpinLockFromDpcLevel(&spinlock1);
>> >
>> > … do some more stuff
>> >
>> > KeReleaseSpinLock(&spinlock2, originalIrql);
>> >
>> > What violation of anything other than our affinity for symmetry
>> happened?
>> >
>>
>> A violation of the rules. You are doing exactly what the docs say not to
>> do. From KeReleaseSpinLockFromDpcLevel:
>>
>> "
>> It is an error to call KeReleaseSpinLockFromDpcLevel if the given spin
>> lock was acquired by calling KeAcquireSpinLock because the caller’s
>> original IRQL is not restored, which can cause deadlocks or fatal page
>> faults.
>> "
>>
>> And from KeReleaseSpinLock:
>>
>> "
>> VOID
>> KeReleaseSpinLock(
>> IN PKSPIN_LOCK SpinLock,
>> IN KIRQL NewIrql
>> );
>>
>> NewIrql
>> Specifies the IRQL value saved from the preceding call to
>> KeAcquireSpinLock.
>> "
>>
>> Now I know and you know that the current implementation of those
>> functions means that what you have done above gives the correct result,
>> and it’s probably reasonable to assume that it will never cause a
>> problem in any future version of Windows, but it’s a violation of the
>> contract and therefore should be frowned upon unless there is a very
>> good reason for doing it. I’d like to see a legitimate reason for
>> releasing the locks in any order than the reverse of the order in which
>> they were acquired.
>>
>> Many regressions are caused by people thinking they can violate the
>> rules because they know how it works and know they will get the correct
>> result, even if the rules appear dumb.
>>
>> James
>>
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>
>

Generally, to avoid deadlock situations, it is considered best practice to
release locks in the inverse order they were acquired. It has been 40 years
since we had to do the formal proof for this, so I no longer recall all the
details. But I have this bit set that says “unlock in inverse order”.
joe


From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Mark Roddy
Sent: Wednesday, August 25, 2010 10:54 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] releasing a spinlock when not at dispatch level

I’ll simplify the problem:

//
// always called at DISPATCH_LEVEL
//
foo()
{
KeAcquireSpinLockAtDpcLevel(&spinlock1);
KeAcquireSpinLockAtDpcLevel(&spinlock2);
stuffRequiringLock1andLock2();
KeReleaseSpinLockFromDpcLevel(&spinlock1);
stuffRequiringLock2();
KeReleaseSpinLockFromDpcLevel(&spinlock2);
}

ok - now that IRQL is out of the picture, what exactly is the problem?

Mark Roddy

On Wed, Aug 25, 2010 at 10:46 AM, Mark Roddy wrote:
The correct IRQL is maintained entirely by the dispatch spinlock user which
is why KeAcquireSpinLockAtDpcLevel can exist. Yes I understand what the docs
say. And as I have said repeatedly, you do have to maintain the IRQL state
correctly, but what does that actually mean?

I am claiming that it means this:
In a sequence of dispatch spinlock acquisitions, the first acquire sets the
IRQL state from <= DISPATCH_LEVEL to DISPATCH_LEVEL and saves the initial
IRQL state. The last release restores the IRQL level from DISPATCH_LEVEL to
the saved IRQL state. The optimization KeAcquireSpinLockAtDpcLevel exists
because the client, the user of spinlocks, knows the IRQL state I just
described and can make an informed decision at acquire time regarding which
form of acquire to use.

There is no functional requirement to do the release of the locks in any
specific order. Docs and prefast may differ. This isn’t an implementation
detail it is fundamental to dispatch spinlocks. You cannot cause a spinlock
deadlock on lock release, but you do have to maintain the IRQL correctly.

Mark Roddy

On Wed, Aug 25, 2010 at 12:34 AM, James Harper
wrote:
>
> As I said - you have to maintain irql correctly, but that does not
require
> release ordering.
>
> KIRQL originalIrql;
>
> KeAcquireSpinLock(&spinlock1, &originalIrql);
> KeAcquireSpinLockAtDpcLevel(&spinlock2);
>
> … do some stuff
>
> KeReleaseSpinLockFromDpcLevel(&spinlock1);
>
> … do some more stuff
>
> KeReleaseSpinLock(&spinlock2, originalIrql);
>
> What violation of anything other than our affinity for symmetry
happened?
>
A violation of the rules. You are doing exactly what the docs say not to
do. From KeReleaseSpinLockFromDpcLevel:


It is an error to call KeReleaseSpinLockFromDpcLevel if the given spin
lock was acquired by calling KeAcquireSpinLock because the caller’s
original IRQL is not restored, which can cause deadlocks or fatal page
faults.


And from KeReleaseSpinLock:


VOID
KeReleaseSpinLock(
IN PKSPIN_LOCK SpinLock,
IN KIRQL NewIrql
);

NewIrql
Specifies the IRQL value saved from the preceding call to
KeAcquireSpinLock.


Now I know and you know that the current implementation of those
functions means that what you have done above gives the correct result,
and it’s probably reasonable to assume that it will never cause a
problem in any future version of Windows, but it’s a violation of the
contract and therefore should be frowned upon unless there is a very
good reason for doing it. I’d like to see a legitimate reason for
releasing the locks in any order than the reverse of the order in which
they were acquired.

Many regressions are caused by people thinking they can violate the
rules because they know how it works and know they will get the correct
result, even if the rules appear dumb.

James


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the
List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

This message has been scanned for viruses and
dangerous content by http:</http:> MailScanner, and is
believed to be clean.

The problem with the code below is that it violates a fundamental principle
of locking: you unlock in the INVERSE order of locking. Therefore, it makes
no sense at all to unlock lock1 while lock2 is still being held. Not only
does it cause the problem you observe, but it can lead to potential deadlock
situations.

Therefore, rewrite the code to be correct, and do not unlock the locks in
any but the inverse order of acquisition.
joe

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Pavel A.
Sent: Wednesday, August 25, 2010 6:51 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] releasing a spinlock when not at dispatch level

wrote in message news:xxxxx@ntdev…
>> Just to be pedantic, you do not have to release them in the reverse order

>> to insure that you
>> return to the correct irql, you have to insure that you return to the
>> correct irql when your last lock
>> is released.

DEC operating systems, IIRC, did not return IRQL to caller,
maybe exactly because of this concern. All functions that raise IRQL
worked thru callbacks - like KeSynchronizeExecution vs.
KeAcquireInterruptSpinLock.
Saving and restoring IRQL was hidden from users. Also, because of
callbacks,
changes of IRQL occurred in stack order, and, naturally, restored in
reverse order.

In NT, they thought that callbacks are too much overhead (true!) and had to
expose previous IRQL to programmer.
Everything has its price.

Regards,
– pa

> Consider the following scenario:
>
> KIRQL irql1,irql2;
>
> KeAcquireSpinLock(&spinlock1, &irql1);
>
> KeAcquireSpinLock(&spinlock2, &irql2);
>
> …
>
>
> KeReleaseSpinLock(&spinlock1, Irql1);
>
>
> Assuming that the original call was made at IRQL< DPC_LEVEL the above line

> will set IRQL to its original level, despite the fact that spinlock2 is
> still being held…
>
>
> Anton Bassov
>


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Actually, I had almost the same requirement of doing the formal proof that
locks did not have to be released in reverse the order they were acquired.
Once proven this significantly sped up a research operating system we were
working on.

It is a nice thing to do to use reverse order, and it is clean in that it
can be viewed like a structured programming model but there are reasons to
release locks in differing order as long as you deal with the side effects
such as IRQL that can occur.


Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

“Joseph M. Newcomer” wrote in message
news:xxxxx@ntdev…
> Generally, to avoid deadlock situations, it is considered best practice to
> release locks in the inverse order they were acquired. It has been 40
> years
> since we had to do the formal proof for this, so I no longer recall all
> the
> details. But I have this bit set that says “unlock in inverse order”.
> joe
>
>

Information from ESET NOD32 Antivirus, version of virus signature database 5397 (20100825)

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

No, it is not required in all cases. We’ve had this discussion about
fiftyfour times now.

mm

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Joseph M. Newcomer
Sent: Wednesday, August 25, 2010 12:55 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] releasing a spinlock when not at dispatch level

The problem with the code below is that it violates a fundamental principle
of locking: you unlock in the INVERSE order of locking. Therefore, it makes
no sense at all to unlock lock1 while lock2 is still being held. Not only
does it cause the problem you observe, but it can lead to potential deadlock
situations.

Therefore, rewrite the code to be correct, and do not unlock the locks in
any but the inverse order of acquisition.
joe

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Pavel A.
Sent: Wednesday, August 25, 2010 6:51 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] releasing a spinlock when not at dispatch level

wrote in message news:xxxxx@ntdev…
>> Just to be pedantic, you do not have to release them in the reverse
>> order

>> to insure that you
>> return to the correct irql, you have to insure that you return to
>> the correct irql when your last lock is released.

DEC operating systems, IIRC, did not return IRQL to caller, maybe exactly
because of this concern. All functions that raise IRQL worked thru callbacks
- like KeSynchronizeExecution vs.
KeAcquireInterruptSpinLock.
Saving and restoring IRQL was hidden from users. Also, because of callbacks,
changes of IRQL occurred in stack order, and, naturally, restored in reverse
order.

In NT, they thought that callbacks are too much overhead (true!) and had to
expose previous IRQL to programmer.
Everything has its price.

Regards,
– pa

> Consider the following scenario:
>
> KIRQL irql1,irql2;
>
> KeAcquireSpinLock(&spinlock1, &irql1);
>
> KeAcquireSpinLock(&spinlock2, &irql2);
>
> …
>
>
> KeReleaseSpinLock(&spinlock1, Irql1);
>
>
> Assuming that the original call was made at IRQL< DPC_LEVEL the above
> line

> will set IRQL to its original level, despite the fact that spinlock2
> is still being held…
>
>
> Anton Bassov
>


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


This message has been scanned for viruses and dangerous content by
MailScanner, and is believed to be clean.


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer