KM APC and aquiring resource

Hi everyone:

Could anyone verify that KM APC should be disabled (by calling
KeEnterCriticalRegion)
all the time while my driver has a ERESOURCE acquired? Or it’s enough to
disable
APC just for acquiring and releasing the resource?
In either case why?

TIA,

Vladimir

I’m assuming this is related to the Driver Verifier’s check for this
condition, right?

The issue is one of lock ordering. A normal kernel mode APC might enter the
file system to perform an ARBITRARY operation. Thus, once you start
acquiring locks you should not allow normal kernel mode APCs until you have
released all those locks in order to avoid the possibility of reentrancy.

Thus, the short answer to your question: block normal kernel APCs for the
duration of time that you own a reentrant lock (notably an ERESOURCE.)

Doing this JUST around the acquisition does not resolve the problem,
although it does make the driver verifier work correctly (for now - I
suspect that the driver verifier folks will likely add a check in
KenLeaveCriticalRegion to confirm that you aren’t holding any reentrant
locks when APC delivery is re-enabled.)

Do also note that this does not block special kernel APCs (since they don’t
reenter the FSD this is OK.) Thus, I/O completion can be processed as
normal (which is NOT the case when you raise to APC_LEVEL.)

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com http:

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 24, 2000 11:16 AM
To: NT Developers Interest List
Subject: [ntdev] KM APC and aquiring resource

Hi everyone:

Could anyone verify that KM APC should be disabled (by calling
KeEnterCriticalRegion)
all the time while my driver has a ERESOURCE acquired? Or it’s enough to
disable
APC just for acquiring and releasing the resource?
In either case why?

TIA,

Vladimir</http:>

Tony:

Thanks for the explanation.
You were right in your assumptions about Verifier. Of course, my driver
BSODed
(BTW, a nice way to report about that type of problem :slight_smile: and quick fix was
to
guard resource acquiring/releasing with FsRtlEnter/LeaveFileSystem. Seems to
be
not right :frowning:

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, May 24, 2000 11:17 AM
To: NT Developers Interest List
Subject: [ntdev] RE: KM APC and aquiring resource

I’m assuming this is related to the Driver Verifier’s check for this
condition, right?

The issue is one of lock ordering. A normal kernel mode APC might enter the
file system to perform an ARBITRARY operation. Thus, once you start
acquiring locks you should not allow normal kernel mode APCs until you have
released all those locks in order to avoid the possibility of reentrancy.

Thus, the short answer to your question: block normal kernel APCs for the
duration of time that you own a reentrant lock (notably an ERESOURCE.)

Doing this JUST around the acquisition does not resolve the problem,
although it does make the driver verifier work correctly (for now - I
suspect that the driver verifier folks will likely add a check in
KenLeaveCriticalRegion to confirm that you aren’t holding any reentrant
locks when APC delivery is re-enabled.)

Do also note that this does not block special kernel APCs (since they don’t
reenter the FSD this is OK.) Thus, I/O completion can be processed as
normal (which is NOT the case when you raise to APC_LEVEL.)

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com http:

-----Original Message-----
From: Chtchetkine, Vladimir [mailto:xxxxx@Starbase.com]
Sent: Wednesday, May 24, 2000 11:16 AM
To: NT Developers Interest List
Subject: [ntdev] KM APC and aquiring resource

Hi everyone:

Could anyone verify that KM APC should be disabled (by calling
KeEnterCriticalRegion)
all the time while my driver has a ERESOURCE acquired? Or it’s enough to
disable
APC just for acquiring and releasing the resource?
In either case why?

TIA,

Vladimir


You are currently subscribed to ntdev as: xxxxx@Starbase.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)</http:>

Kernel mode APCs (mormal) can be disabled by KeEnterCriticalRegion.
This routine can be called recursively and every time it is called
appropriate release routine KeLeaveCriticalSection must be called.
Last release will enable kernel mode APCs and if the current
thread’s APC queue is not empty, software interrupt at APC level
is requested. Special kernel APCs (they which do not have Normal routine)
are not affected by this - they can be disabled by raising the Irql
to APC_LEVEL only (Kernel routine is always executed at APC_LEVEL,
while Normal routine is executed at PASSIVE_LEVEL).
While holding an executive resource in FSD the current thread must
have kernel APCs disabled (using KeEnterCriticalRegion or equivalent
macro FsRtlEnterFileSystem). Special kernel APCs (they which are used
for completing I/O requests - they call either IopCompleteRequest or
IopCompeletePageWrite) can occur because they do not reenter FSD.
But if the kernel code calls e.g. Nt(Zw)ReadFile with valid ApcRoutine,
the IopCompleteRequest will queue this APC as normal. The caller’s
ApcRoutine potentially can issue another request (because it is called
at PASSIVE_LEVEL and thus it may call e.g. any of Nt(Zw)Xxxx routines).
which can reenter FSD and can make some kind of synchronization problems.
By disabling kernel APCs this scenario cannot occur because normal APCs
are called after all FSD resources are released.

Code should look like this:

FsRtlEnterFileSystem();

ExAcquireResourceXxxx(…, …);

ExReleaseResourceXxxx(…, …);

FsRtlExitFileSystem();

Same scenario is maintained also in FSD callbacks for pre-acquiring
FSD resources. E.g. FsRtlAcquireFileExclusive first calls
FsRtlEnterFileSystem and then calls either FSD callback (if exist) or
tries to acquire both resources. FsRtlReleaseFile first releases acquired
resources (in FSD callback or directly) and then calls FsRtlExitFileSystem.

Paul

PS: If any developer can know more about APCs I am prepared to
give as many informations as I know (and I think I know enough).

-----P?vodn? zpr?va-----
Od: Chtchetkine, Vladimir [SMTP:xxxxx@Starbase.com]
Odesl?no: 24. kv?tna 2000 17:16
Komu: NT Developers Interest List
P?edm?t: [ntdev] KM APC and aquiring resource

Hi everyone:

Could anyone verify that KM APC should be disabled (by calling
KeEnterCriticalRegion)
all the time while my driver has a ERESOURCE acquired? Or it’s enough to
disable
APC just for acquiring and releasing the resource?
In either case why?

TIA,

Vladimir

> at PASSIVE_LEVEL and thus it may call e.g. any of Nt(Zw)Xxxx routines).

which can reenter FSD and can make some kind of synchronization >problems.

This is strange for me - resource locks - not like fast mutexes - can be
acquired recursively by the same thread.

Same scenario is maintained also in FSD callbacks for pre-acquiring
FSD resources. E.g. FsRtlAcquireFileExclusive first calls
FsRtlEnterFileSystem and then calls either FSD callback (if exist) or
tries to acquire both resources. FsRtlReleaseFile first releases acquired

FsRtlAcquireFileExclusive:

  • calls KeEnterCriticalRegion (or FsRtlEnterFileSystem? they are
    synonyms).
  • then tries to call AcquireFileForNtCreateSection fastio routine if any
  • otherwise, acquires only the MainResource in common FCB header
    (if both MainResource and FileObject->FsContext are not NULL) exclusively.

It is called from MmCreateSection to acquire the lock which guards the
section object pointers structure which holds 2 MM-private pointers, also it
is
called from some Ccxxx routines.

Max