Hi,
It seems that I do not understand APC mechanism clearly.
For instance, FSDs protect their dispatch routines by
FsRtlEnter/ExitFileSystem(), normal kernel-mode APC
should be disabled before calling ExAcquireResourceXxx(),
but I do not understand why.
Could anybody explain me this and/or provide any basic
example, which illustrates when it is necessary to disable
APC and what could happen if it remains enabled?
Thank you for any help.
Kind regards,
Leonid.
Tony will probably provide a much more expressive explanation…
The basic idea of disabling normal APC’s is to avoid resource acquisition deadlock. For example, suppose a thread is performing operations in the file system with a file system resource acquired exclusively. An APC is delivered to that thread that interrupts the file system operation. The APC happens to make another call into the FSD. The FSD tries to acquire the same file system resource exclusively, and the thread deadlocks. Since almost every thread in the system tries to do file IO, the system will grind to a halt.
As you imply in your e-mail, special kernel APC’s are still enabled in order to allow the IO completion APC’s to run, IopCompleteRequest and friends. These are an exception because it is known they won’t re-enter the file system.
-----Original Message-----
From: Leonid Zhigunov [mailto:xxxxx@progate.spb.ru]
Sent: Tuesday, May 21, 2002 9:51 AM
To: File Systems Developers
Subject: [ntfsd] Kernel-mode APC
Hi,
It seems that I do not understand APC mechanism clearly.
For instance, FSDs protect their dispatch routines by
FsRtlEnter/ExitFileSystem(), normal kernel-mode APC
should be disabled before calling ExAcquireResourceXxx(),
but I do not understand why.
Could anybody explain me this and/or provide any basic
example, which illustrates when it is necessary to disable
APC and what could happen if it remains enabled?
Thank you for any help.
Kind regards,
Leonid.
You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%
Thank you,
As I understand, the ExAcquireResourceExclusive() can not be acquired
recursively. So, the following sequence will deadlock the thread:
ExAcquireResourceExclusive(myResource, TRUE);
ExAcquireResourceExclusive(myResource, TRUE);
Am I right?
No, that’s okay.
ExAcquireResourceShared followed by ExAcquireResourceExclusive will cause a deadlock.
When I posted about acquiring resources, I was speaking generally, not specifically in terms of ERESOURCE objects.
-----Original Message-----
From: Leonid Zhigunov [mailto:xxxxx@progate.spb.ru]
Sent: Tuesday, May 21, 2002 11:06 AM
To: File Systems Developers
Subject: [ntfsd] Re: Kernel-mode APC
Thank you,
As I understand, the ExAcquireResourceExclusive() can not be acquired
recursively. So, the following sequence will deadlock the thread:
ExAcquireResourceExclusive(myResource, TRUE);
ExAcquireResourceExclusive(myResource, TRUE);
Am I right?
You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%
No. This is false. The only ERESOURCE sequence that will deadlock is:
ExAcquireResourceShared(myResource, TRUE);
ExAcquireResourceExclusive(myResource, TRUE);
Acquiring it in the opposite order (exclusive, then shared) works without
deadlock, as do combinations shared/shared, and exclusive/exclusive.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Hope to see you at the next OSR file systems class October 7, 2002!
-----Original Message-----
From: Leonid Zhigunov [mailto:xxxxx@progate.spb.ru]
Sent: Tuesday, May 21, 2002 12:06 PM
To: File Systems Developers
Subject: [ntfsd] Re: Kernel-mode APC
Thank you,
As I understand, the ExAcquireResourceExclusive() can not be acquired
recursively. So, the following sequence will deadlock the thread:
ExAcquireResourceExclusive(myResource, TRUE);
ExAcquireResourceExclusive(myResource, TRUE);
Am I right?
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%
Also note that any lock order inversion will cause a deadlock. For example, suppose you normally acquire resource A before resource B, but you don’t always acquire A when you acquire B. Suppose your thread owns resource B. An APC causes re-entry into your file system, where the thread now tries to acquire resource A with B already owned. Because your prescribed locking order is now inverted, deadlock is likely.
-----Original Message-----
From: Fuller, Rob
Sent: Tuesday, May 21, 2002 11:12 AM
To: File Systems Developers
Subject: [ntfsd] Re: Kernel-mode APC
No, that’s okay.
ExAcquireResourceShared followed by ExAcquireResourceExclusive will cause a deadlock.
When I posted about acquiring resources, I was speaking generally, not specifically in terms of ERESOURCE objects.
-----Original Message-----
From: Leonid Zhigunov [mailto:xxxxx@progate.spb.ru]
Sent: Tuesday, May 21, 2002 11:06 AM
To: File Systems Developers
Subject: [ntfsd] Re: Kernel-mode APC
Thank you,
As I understand, the ExAcquireResourceExclusive() can not be acquired
recursively. So, the following sequence will deadlock the thread:
ExAcquireResourceExclusive(myResource, TRUE);
ExAcquireResourceExclusive(myResource, TRUE);
Am I right?
You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%
You are currently subscribed to ntfsd as: xxxxx@inin.com
To unsubscribe send a blank email to %%email.unsub%%
No. Resources are recursively acquirable.
One of the reasons for disabling APCs is because suspend is implemented
via an APC. If say, your filesystem acquires a lock on every i/o & the
thread is suspended from user-mode before you release it (imagine a
rogue user mode process sending down an I/O on on thread, and issuing
SuspendThread() on that thread from another in a loop until this race
happens), the system will quickly come to a halt - as no other thread
will be able to do i/o directed towards the filesystem.
Disabling kernel APCs before acquiring locks prevents this situation.
-----Original Message-----
From: Leonid Zhigunov [mailto:xxxxx@progate.spb.ru]
Sent: Tuesday, May 21, 2002 9:06 AM
To: File Systems Developers
Subject: [ntfsd] Re: Kernel-mode APC
Thank you,
As I understand, the ExAcquireResourceExclusive() can not be acquired
recursively. So, the following sequence will deadlock the thread:
ExAcquireResourceExclusive(myResource, TRUE);
ExAcquireResourceExclusive(myResource, TRUE);
Am I right?
You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%
Actually I think Rob’s description is an excellent one!
The one point I would add is the rationale for having “APC_LEVEL” at all.
Specifically, certain operations are required to be performed IN THREAD
CONTEXT. That is because requiring that something be done in thread context
eliminates the “multi-processor” issue since a single thread can only run on
a single processor at one instant in time - it can hop around like a frog on
a hot griddle between instants, but can only be on one at a single instant.
This is a HUGE simplification.
For example, IoQueueThreadIrp (in the Windows XP IFS Kit) relies upon this
when inserting a newly built IRP into the thread’s IRP queue (notice it only
takes the IRP, not the thread AND the IRP? You cannot enqueue to a
DIFFERENT thread!) That is the #1 reason why we have IopCompleteRequest
(the APC used for I/O completion in thread context). We have to get into
process context ANYWAY (to copy data into the right address space) but we
also need to remove this IRP from the thread’s list, which must be done in
thread context (that’s why a mere KeStackAttachProcess is not sufficient to
finish up I/O completion).
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Hope to see you at the next OSR file systems class October 7, 2002!
-----Original Message-----
From: Fuller, Rob [mailto:xxxxx@inin.com]
Sent: Tuesday, May 21, 2002 11:02 AM
To: File Systems Developers
Subject: [ntfsd] RE: Kernel-mode APC
Tony will probably provide a much more expressive explanation…
The basic idea of disabling normal APC’s is to avoid resource acquisition
deadlock. For example, suppose a thread is performing operations in the
file system with a file system resource acquired exclusively. An APC is
delivered to that thread that interrupts the file system operation. The APC
happens to make another call into the FSD. The FSD tries to acquire the
same file system resource exclusively, and the thread deadlocks. Since
almost every thread in the system tries to do file IO, the system will grind
to a halt.
As you imply in your e-mail, special kernel APC’s are still enabled in order
to allow the IO completion APC’s to run, IopCompleteRequest and friends.
These are an exception because it is known they won’t re-enter the file
system.
-----Original Message-----
From: Leonid Zhigunov [mailto:xxxxx@progate.spb.ru]
Sent: Tuesday, May 21, 2002 9:51 AM
To: File Systems Developers
Subject: [ntfsd] Kernel-mode APC
Hi,
It seems that I do not understand APC mechanism clearly.
For instance, FSDs protect their dispatch routines by
FsRtlEnter/ExitFileSystem(), normal kernel-mode APC should be disabled
before calling ExAcquireResourceXxx(), but I do not understand why.
Could anybody explain me this and/or provide any basic
example, which illustrates when it is necessary to disable
APC and what could happen if it remains enabled?
Thank you for any help.
Kind regards,
Leonid.
You are currently subscribed to ntfsd as: xxxxx@inin.com To unsubscribe
send a blank email to %%email.unsub%%
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%
Thank you very much for all.
Now is is clear.
The disabling of KM APC looks like a mechanism
for synchronization within the thread itself! Cool 
I did not know that Suspend is implemented via an APC.
In this case, if you use global lock (any type of objects
that allow APC to be delivered) to serialize access to
some resource, you can come to the situation when all
threads requesting the protected resource will be blocked,
if someone calls SuspendThread() while the thread is holding
the resource. Am I right?
Kind regards,
Leonid.
Yes.
Ravi
-----Original Message-----
From: Leonid Zhigunov [mailto:xxxxx@progate.spb.ru]
Sent: Tuesday, May 21, 2002 10:11 AM
To: File Systems Developers
Subject: [ntfsd] Re: Kernel-mode APC
Thank you very much for all.
Now is is clear.
The disabling of KM APC looks like a mechanism
for synchronization within the thread itself! Cool
I did not know that Suspend is implemented via an APC.
In this case, if you use global lock (any type of objects
that allow APC to be delivered) to serialize access to
some resource, you can come to the situation when all
threads requesting the protected resource will be blocked,
if someone calls SuspendThread() while the thread is holding the
resource. Am I right?
Kind regards,
Leonid.
You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%
IIRC any resource acquisition can be recursive.
“Leonid Zhigunov” wrote in message
news:LYRIS-2023-53339-2002.05.21-12.02.40–maxim#xxxxx@lists.osr.com…
> Thank you,
>
> As I understand, the ExAcquireResourceExclusive() can not be acquired
> recursively. So, the following sequence will deadlock the thread:
>
> ExAcquireResourceExclusive(myResource, TRUE);
> ExAcquireResourceExclusive(myResource, TRUE);
>
> Am I right?
>
>
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to %%email.unsub%%
>