Acquiring fast mutex in read/write dispatch routine

NTDEV
1

When I try to acquire a fast mutex inside a routine invoked from read/write Dispatch routine I get the following error. Is it not valid to introduce wait in Read/Write dispatch routines? Help. Thanks.

ATTEMPTED_SWITCH_FROM_DPC (b8)
A wait operation, attach process, or yield was attempted from a DPC routine.
This is an illegal operation and the stack track will lead to the offending
code and original DPC routine.
Arguments:
Arg1: 818f8300, Original thread which is the cause of the failure
Arg2: 84a5b408, New thread
Arg3: 818f2000, Stack address of the original thread
Arg4: 00000000

Debugging Details:

FAULTING_THREAD: 818f8300

DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT

BUGCHECK_STR: 0xB8

PROCESS_NAME: System

CURRENT_IRQL: 1b

LAST_CONTROL_TRANSFER: from 818ac9cf to 81890e46

STACK_TEXT:
87653ac0 818ac9cf 00000000 818f8300 84a5b34c nt!KiSwapContext+0x26
87653afc 818bb125 818f8300 00000004 84a5b340 nt!KiSwapThread+0x389
87653b30 818236d8 00000000 ee670000 84bcd002 nt!KeWaitForGate+0x198
87653b48 81876035 84607130 84bcd0e8 8779da84 nt!KiAcquireGuardedMutex+0x53
87653b54 8779da84 3f3f3f3f 3f3f3f3f 3f3f3f3f nt!ExAcquireFastMutex+0x1e
87653bf0 8779e10f 84607130 83ff2008 0c773b80 Myfltr!MyReadWriteLBARange+0xd4
87653c14 87799896 84607130 83ff2008 00000003 Myfltr!MyReadWriteProcess+0x6f
87653c48 87799967 84607130 00000000 8779e760 Myfltr!MyFltrReadWriteIrp+0x106
87653c64 81827f83 84607130 83ff2008 83ff2008 MyFltr!MyFltrReadWrite+0x97

2

If your read/write routines can be called at dispatch (which they can be if you are in the storage stack), you must use a synchronization primitive that you can acquire at dispatch level (basically this leaves you with a spin lock), a fast mutex is not acquirable at dispatch level

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.com
Sent: Monday, August 18, 2008 8:16 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Acquiring fast mutex in read/write dispatch routine

When I try to acquire a fast mutex inside a routine invoked from read/write Dispatch routine I get the following error. Is it not valid to introduce wait in Read/Write dispatch routines? Help. Thanks.

ATTEMPTED_SWITCH_FROM_DPC (b8)
A wait operation, attach process, or yield was attempted from a DPC routine.
This is an illegal operation and the stack track will lead to the offending
code and original DPC routine.
Arguments:
Arg1: 818f8300, Original thread which is the cause of the failure
Arg2: 84a5b408, New thread
Arg3: 818f2000, Stack address of the original thread
Arg4: 00000000

Debugging Details:

FAULTING_THREAD: 818f8300

DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT

BUGCHECK_STR: 0xB8

PROCESS_NAME: System

CURRENT_IRQL: 1b

LAST_CONTROL_TRANSFER: from 818ac9cf to 81890e46

STACK_TEXT:
87653ac0 818ac9cf 00000000 818f8300 84a5b34c nt!KiSwapContext+0x26
87653afc 818bb125 818f8300 00000004 84a5b340 nt!KiSwapThread+0x389
87653b30 818236d8 00000000 ee670000 84bcd002 nt!KeWaitForGate+0x198
87653b48 81876035 84607130 84bcd0e8 8779da84 nt!KiAcquireGuardedMutex+0x53
87653b54 8779da84 3f3f3f3f 3f3f3f3f 3f3f3f3f nt!ExAcquireFastMutex+0x1e
87653bf0 8779e10f 84607130 83ff2008 0c773b80 Myfltr!MyReadWriteLBARange+0xd4
87653c14 87799896 84607130 83ff2008 00000003 Myfltr!MyReadWriteProcess+0x6f
87653c48 87799967 84607130 00000000 8779e760 Myfltr!MyFltrReadWriteIrp+0x106
87653c64 81827f83 84607130 83ff2008 83ff2008 MyFltr!MyFltrReadWrite+0x97

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

3

Thanks Doron. I’m also using ERESOURCES (by calling ExAcquireResourceExclusiveLite etc) to implement reader/writers lock in some of the functions invoked by read/write dispatch routine. MSDN says IRQL <= APC_LEVEL but does not say if the IRQL is raised when ExAcquireResourceExclusiveLite and other ERESOURCE routines are called (unlike specifically mentioned for fast mutex routines). Is it OK to use ERESOURCES or will I see the same bugcheck as observed while using the fast mutex in dispatch routines? Thanks again.

4

> Is it OK to use ERESOURCES or will I see the same bugcheck as observed while using

the fast mutex in dispatch routines?

The only synchronization construct that can be used at IRQL==DPC level is spinlock. All dispatcher-level synchronization constructs imply the possibility of blocking, and blocking calls cannot be made at elevated IRQL…

Anton Bassov

5

The only way you can do that at elevated IRQL is to put timeout 0.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@hotmail.com
Sent: Tuesday, August 19, 2008 1:59 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Acquiring fast mutex in read/write dispatch routine

Is it OK to use ERESOURCES or will I see the same bugcheck as observed
while using
the fast mutex in dispatch routines?

The only synchronization construct that can be used at IRQL==DPC level is
spinlock. All dispatcher-level synchronization constructs imply the
possibility of blocking, and blocking calls cannot be made at elevated
IRQL…

Anton Bassov

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

6

Setting the timeout to 0 (zero) is only used to quickly test the wait condition without waiting.

Regards,
Razvan

— On Tue, 8/19/08, Bercea Gabriel wrote:

> From: Bercea Gabriel
> Subject: RE: [ntdev] Acquiring fast mutex in read/write dispatch routine
> To: “Windows System Software Devs Interest List”
> Date: Tuesday, August 19, 2008, 8:42 PM
> The only way you can do that at elevated IRQL is to put
> timeout 0.

7

Thanks Anton. I need to implement readers/writers locks while processing the incoming irps. If there is no other sync construct available then the only other method I can think of achieving this is to create a separate thread (e.g a worker item) from the dispatch routine which can use ERESOURCES and can process these irps. Will this be the best way or can you guys suggest any better option? Thank you.

k

8

Anton, Thanks. I’m implementing readers/writers lock in my driver read/write dispatch routine. If I cannot use ERESOURCES there then the only other way I can think of is creating a separate thread (like work item) which can make use of ERESOURCES. But I’m not sure if this is the best thing to do performance wise. Do you think it is ok or if you can suggest a better solution? Thanks.

Hi Gabriel, I need to use ERESOURCES. Where is the timeout specified for an ERESOURCE? Thanks.

9

Other than probably killing perf, it might work. You need to be very very careful not to cause a page fault while processing a paging i/o. what do you need to add a reader/writer to the i/o path? You can probably roll your own lock if you can do async processing when the conditions are correct to let a reader or writer in instead of blocking the calling the thread, but that requires a pretty good understanding of async/multi threaded programming which you have not really shown you have yet…

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.com
Sent: Tuesday, August 19, 2008 12:20 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Acquiring fast mutex in read/write dispatch routine

Thanks Anton. I need to implement readers/writers locks while processing the incoming irps. If there is no other sync construct available then the only other method I can think of achieving this is to create a separate thread (e.g a worker item) from the dispatch routine which can use ERESOURCES and can process these irps. Will this be the best way or can you guys suggest any better option? Thank you.

k

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

10

>> but that requires a pretty good understanding of async/multi threaded programming which you have not really shown you have yet…

yes you are correct I do not have strong back ground in this arena. :frowning:

> You can probably roll your own lock if you can do async processing when the conditions are correct to let a reader or writer in instead of blocking the calling the thread,

But if I can do async processing then why do I need to write my own lock? I can as well use ERESOURCES. My question is how can I do async processing? My original idea was to queue the incoming irps in a driver queue and return STATUS_PENDING for them in the dispatch routine so that the calling thread is not blocked and then a separate thread (which I will create during device start time) will consume these Irps from the driver queue and initiate their async processing (setting completion routines for each of these Irps) but you said it will kill the performance :frowning: SO any other suggestions - how can I do async processing of these Irps where I have to use readers/writers lock? FYI: r/w locks are required to synchronize access to a data structure that my driver maintains for storing information about incoming Irps.

Thanks.

11

So why are you using a R/W lock to protect the data structure with “information about incoming Irps”? Do you have a real reason for doing this (e.g. you have so many IRPs that exclusively locking the list measurably hurts you)? Or was it just a fun lock to use?

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.com
Sent: Tuesday, August 19, 2008 1:23 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Acquiring fast mutex in read/write dispatch routine

> but that requires a pretty good understanding of async/multi threaded programming which you have not really shown you have yet…

yes you are correct I do not have strong back ground in this arena. :frowning:

> You can probably roll your own lock if you can do async processing when the conditions are correct to let a reader or writer in instead of blocking the calling the thread,

But if I can do async processing then why do I need to write my own lock? I can as well use ERESOURCES. My question is how can I do async processing? My original idea was to queue the incoming irps in a driver queue and return STATUS_PENDING for them in the dispatch routine so that the calling thread is not blocked and then a separate thread (which I will create during device start time) will consume these Irps from the driver queue and initiate their async processing (setting completion routines for each of these Irps) but you said it will kill the performance :frowning: SO any other suggestions - how can I do async processing of these Irps where I have to use readers/writers lock? FYI: r/w locks are required to synchronize access to a data structure that my driver maintains for storing information about incoming Irps.

Thanks.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

12

Hi Peter, this driver is an disk class upper filter driver so all the disk access is through is driver and there is no typical traffic pattern. I may have lot of Irps and if I have exclusive lock this may really hurt the performance.

Thanks.

13

Have you measured it? Going to this much trouble without some real performance analysis is not advisable in general.

  • S

-----Original Message-----
From: xxxxx@yahoo.com
Sent: Tuesday, August 19, 2008 16:01
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Acquiring fast mutex in read/write dispatch routine

Hi Peter, this driver is an disk class upper filter driver so all the disk access is through is driver and there is no typical traffic pattern. I may have lot of Irps and if I have exclusive lock this may really hurt the performance.

Thanks.

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

14

So your proposal is to use a “fast” lock and then to shunt all I/O operations over to a limited pool of threads, incurring at least one and possibly two unnecessary context switches per I/O operation as well as dispatching delays if the number of I/O operations exceeds the number of threads.

Here is my suggestion:

Start with the exclusive lock. Get your code working. Try to hold the exclusive lock for a short of a time as possible so that you keep contention on the lock low. Don?t do wasteful things like allocate or free memory while holding the lock - ideally you should only be finding what you need in your data structure.

If/when you have a working solution, do some actual performance measurement and if that lock shows up as a performance issue then look at an alternate locking strategy. At that point you’ll have a working solution to test alternative strategies with, so this won’t be an academic exercise anymore and you can come back with some more specific questions.

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.com
Sent: Tuesday, August 19, 2008 1:58 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Acquiring fast mutex in read/write dispatch routine

Hi Peter, this driver is an disk class upper filter driver so all the disk access is through is driver and there is no typical traffic pattern. I may have lot of Irps and if I have exclusive lock this may really hurt the performance.

Thanks.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

15

Thanks Peter. let me work on your suggestion first :slight_smile: