critical regions

NTFSD
1

I’m doing an fsd proof-of-concept, and I got into a discussion with
another dev about apc delivery and critical regions. A couple of
questions:

  1. Why, in general, are FSDs not reentrant, such that disabling normal
    kernel and user APCs is required? You obviously have to manage
    synchronization to specific data structures, but this global disabling
    of APCs sort of violates the idea of “hold the fewest locks for the
    least time possible”. It’s clearly not a problem to enter the code
    multiple times by different threads - only reentry by the same thread is
    prevented here. What’s the deal?

  2. The opposite question is also on my mind: why not just KeRaiseIrql
    to APC_LEVEL, instead of needing a critical region? I realize io
    completion is a special kernel apc, but the fsd would presumably lower
    the irql back to passive before queuing that apc – or does this open
    you up to the very race you were trying to prevent, hence the need for
    the critical region level?

It seems that critical regions are kind of an add-on to the irql model -
they don’t quite fit cleanly in with the rest of the architecture.
Regardless, I’m definitely missing something about reentrancy here.

Thanks.

-sd

2

Answering my own question, partially, I just thought through the obvious
deadlock:

  • Thread starts enters an FSD and acquires some lock before proceeding
  • Thread gets interrupted by an APC to the same thread
  • Thread re-enters FSD in context of APC, trying to grab the same lock

Because it’s an apc, it will run to completion before resuming mainline
processing, but the mainline code has the lock, so you’re deadlocked.

Is that it in a nutshell?

-sd

On Thu, 2004-08-05 at 12:48, Steve Dispensa wrote:

I’m doing an fsd proof-of-concept, and I got into a discussion with
another dev about apc delivery and critical regions. A couple of
questions:

  1. Why, in general, are FSDs not reentrant, such that disabling normal
    kernel and user APCs is required? You obviously have to manage
    synchronization to specific data structures, but this global disabling
    of APCs sort of violates the idea of “hold the fewest locks for the
    least time possible”. It’s clearly not a problem to enter the code
    multiple times by different threads - only reentry by the same thread is
    prevented here. What’s the deal?

  2. The opposite question is also on my mind: why not just KeRaiseIrql
    to APC_LEVEL, instead of needing a critical region? I realize io
    completion is a special kernel apc, but the fsd would presumably lower
    the irql back to passive before queuing that apc – or does this open
    you up to the very race you were trying to prevent, hence the need for
    the critical region level?

It seems that critical regions are kind of an add-on to the irql model -
they don’t quite fit cleanly in with the rest of the architecture.
Regardless, I’m definitely missing something about reentrancy here.

Thanks.

-sd

Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@positivenetworks.net
To unsubscribe send a blank email to xxxxx@lists.osr.com

3

> 2) The opposite question is also on my mind: why not just KeRaiseIrql

to APC_LEVEL, instead of needing a critical region? I realize io
completion is a special kernel apc, but the fsd would presumably lower

Critical regions are mainly to prevent SuspendThread while holding FSD locks.

Entering a critical region is a good idea for any ERESOURCE acquisition and not
only FSDs.

Other types of locks like FAST_MUTEX or spinlock are even more rough - they
raise the IRQL.

The drawback is that sending synchronous IRPs is impossible while holding them
since IopCompleteRequest is also blocked. Such a drawback is too bad for FSD
internals, so, they are milder and only use the critical region.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

4

On Thu, 2004-08-05 at 14:16, Maxim S. Shatskih wrote:

> 2) The opposite question is also on my mind: why not just KeRaiseIrql
> to APC_LEVEL, instead of needing a critical region? I realize io
> completion is a special kernel apc, but the fsd would presumably lower

Critical regions are mainly to prevent SuspendThread while holding FSD locks.

What sort of APC does SuspendThread use?

This seems like it should be a big problem for any driver with a
top-level interface such as an ioctl handler, which doesn’t use
spinlocks for synch. None of the other locking primitives raise the
irql to the point that a context switch cannot occur, so thread
suspension could be a big problem - sounds to me like a good opportunity
for a system-wide DoS. Also, what about NtTerminateThread and friends?

Entering a critical region is a good idea for any ERESOURCE acquisition and not
only FSDs.

Should this apply to things besides ERESOURCE as well that don’t
automatically disable APCs?

Also, entering a critical region doesn’t prevent special kernel apcs, so
what’s to stop someone from breaking things this way? Is it just
protocol that no special kernel apcs shall be written in such a way to
cause trouble here?

Other types of locks like FAST_MUTEX or spinlock are even more rough - they
raise the IRQL. The drawback is that sending synchronous IRPs is impossible while holding them
since IopCompleteRequest is also blocked. Such a drawback is too bad for FSD
internals, so, they are milder and only use the critical region.

Why must completion of a synchronous request require an APC? I was
under the impression that the IO manager would, in that case, still be
in the context of the original caller, so no APC should be required.

Thanks.

-sd

5

> > Critical regions are mainly to prevent SuspendThread while holding FSD
locks.

What sort of APC does SuspendThread use?

Some special kind of APC.
SuspendThread queues an APC for a thread.

for a system-wide DoS. Also, what about NtTerminateThread and friends?

Again it queues a killer APC (PspExitSpecialApc or such) to a thread. So, if
APCs are blocked, NtTerminateThread is blocked too.

Should this apply to things besides ERESOURCE as well that don’t
automatically disable APCs?

FAST_MUTEX disables APCs.
Spinlock disables context switches.

You must be careful with a KMUTEX though.

Also, entering a critical region doesn’t prevent special kernel apcs,

Which is ONLY IopCompleteRequest (from what I know).

Why must completion of a synchronous request require an APC? I was
under the impression that the IO manager would, in that case, still be
in the context of the original caller, so no APC should be required.

It is so only if the underlying driver did not pend the IRP.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

6

On Thu, 2004-08-05 at 16:23, Maxim S. Shatskih wrote:

> for a system-wide DoS. Also, what about NtTerminateThread and friends?

Again it queues a killer APC (PspExitSpecialApc or such) to a thread. So, if
APCs are blocked, NtTerminateThread is blocked too.

So that covers suspension and termination of threads; are there any
other apc-related deadlock cases?

> Why must completion of a synchronous request require an APC? I was
> under the impression that the IO manager would, in that case, still be
> in the context of the original caller, so no APC should be required.

It is so only if the underlying driver did not pend the IRP.

So the event is signalled *after* copy-over of the data? I would have
assumed it would be the other way (i.e. copy over after the wait,
because that guarantees the right context and avoids the APC). This
makes me think that I still don’t understand KeEnterCriticalRegion vs.
KeRaiseIrql(APC_LEVEL). :slight_smile:

Interesting discussion, regardless.

-sd