IoCompletion Question

NTDEV
1

This really is a “Read The Document” kinda question, but it is not happening
the way the DOCs are saying it should, hence the question:

Is it possible that the IO completion routine is called in the same thread
context as the one that originally called IoCallDriver?

DDK docs seem to suggest otherwise as it says “A driver’s IoCompletion
routine executes in an arbitrary thread or DPC ontext” (Ref: IoCompletion in
DDK doc).

In my TDI client driver, with one specific NIC card, I see that the send
completion routine is called in the same thread context as the one doing
the send (thread that called IoCallDriver).

I need to acquire a lock in the completion routine which, if already
acquired in the send thread, leads to a deadlock if the completion routine
is called in the same context.

I can always queue a work item in the completion routine to avoid this but I
am curious if it is a problem with the NIC driver (as I see it happen only
with one particular NIC card) or am I reading the DOCs wrong?

TIA,
Bandeep

2

somewhere in the docs it says (so or similar):
' do not call IoCallDriver while holding locks '...

Norbert.

"Only in America will you find the Department of the Interior in
charge of everything that's outside."
---- snip ----

This really is a "Read The Document" kinda question, but it is not happening
the way the DOCs are saying it should, hence the question:

Is it possible that the IO completion routine is called in the same thread
context as the one that originally called IoCallDriver?

DDK docs seem to suggest otherwise as it says "A driver's IoCompletion
routine executes in an arbitrary thread or DPC ontext" (Ref: IoCompletion in
DDK doc).

In my TDI client driver, with one specific NIC card, I see that the send
completion routine is called in the same thread context as the one doing
the send (thread that called IoCallDriver).

I need to acquire a lock in the completion routine which, if already
acquired in the send thread, leads to a deadlock if the completion routine
is called in the same context.

I can always queue a work item in the completion routine to avoid this but I
am curious if it is a problem with the NIC driver (as I see it happen only
with one particular NIC card) or am I reading the DOCs wrong?

TIA,
Bandeep

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@stollmann.de
To unsubscribe send a blank email to xxxxx@lists.osr.com

---- snip ----

3

No, it is not a problem, depending on how the driver does I/O you will
commonly see this. What you are seeing is the call to the send completing
the request, so the context is the calling context. But there is nothing
that says the driver cannot queue the request and some other context hold
it.

It is very bad design to hold a lock while calling IoCallDriver to send down
the IRP. As you have seen this can cause nasty deadlocks. I would redesign
your driver to eliminate this need.

–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply

“Bandeep Singh” wrote in message
news:xxxxx@ntdev…
> This really is a “Read The Document” kinda question, but it is not
happening
> the way the DOCs are saying it should, hence the question:
>
> Is it possible that the IO completion routine is called in the same thread
> context as the one that originally called IoCallDriver?
>
> DDK docs seem to suggest otherwise as it says “A driver’s IoCompletion
> routine executes in an arbitrary thread or DPC ontext” (Ref: IoCompletion
in
> DDK doc).
>
> In my TDI client driver, with one specific NIC card, I see that the send
> completion routine is called in the same thread context as the one doing
> the send (thread that called IoCallDriver).
>
> I need to acquire a lock in the completion routine which, if already
> acquired in the send thread, leads to a deadlock if the completion routine
> is called in the same context.
>
> I can always queue a work item in the completion routine to avoid this but
I
> am curious if it is a problem with the NIC driver (as I see it happen only
> with one particular NIC card) or am I reading the DOCs wrong?
>
> TIA,
> Bandeep
>
>
>

4

You are interpreting it incorrectly. An arbitrary thread context means
any context, including the thread in which the PIRP is sent. Queueing a
work item is not the answer here. You must redesign your driver *NOT*
to call IoCallDriver with a lock held. Once you call IoCallDriver, you
must be reentrant on the same thread at any time. What is being guarded
by your lock that you need to hold onto the lock while calling
IoCallDriver?

d

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bandeep Singh
Sent: Wednesday, July 28, 2004 7:53 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] IoCompletion Question

This really is a “Read The Document” kinda question, but it is not
happening
the way the DOCs are saying it should, hence the question:

Is it possible that the IO completion routine is called in the same
thread
context as the one that originally called IoCallDriver?

DDK docs seem to suggest otherwise as it says “A driver’s IoCompletion
routine executes in an arbitrary thread or DPC ontext” (Ref:
IoCompletion in
DDK doc).

In my TDI client driver, with one specific NIC card, I see that the send
completion routine is called in the same thread context as the one
doing
the send (thread that called IoCallDriver).

I need to acquire a lock in the completion routine which, if already
acquired in the send thread, leads to a deadlock if the completion
routine
is called in the same context.

I can always queue a work item in the completion routine to avoid this
but I
am curious if it is a problem with the NIC driver (as I see it happen
only
with one particular NIC card) or am I reading the DOCs wrong?

TIA,
Bandeep

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

5

“arbitrary thread or DPC context” includes the thread that initiated the
request.

In general it’s a bad idea to hold locks while sending I/O requests to
the drivers below you (ie. calling IoCallDriver). You might take a hard
look at why you’re holding the lock, what it’s protecting, and whether
you could get the same behavior by doing something like queueing
incoming requests and processing them one at a time when this operation
is taking place.

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bandeep Singh
Sent: Wednesday, July 28, 2004 7:53 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] IoCompletion Question

This really is a “Read The Document” kinda question, but it is not
happening the way the DOCs are saying it should, hence the question:

Is it possible that the IO completion routine is called in the same
thread context as the one that originally called IoCallDriver?

DDK docs seem to suggest otherwise as it says “A driver’s IoCompletion
routine executes in an arbitrary thread or DPC ontext” (Ref:
IoCompletion in DDK doc).

In my TDI client driver, with one specific NIC card, I see that the send
completion routine is called in the same thread context as the one
doing the send (thread that called IoCallDriver).

I need to acquire a lock in the completion routine which, if already
acquired in the send thread, leads to a deadlock if the completion
routine is called in the same context.

I can always queue a work item in the completion routine to avoid this
but I am curious if it is a problem with the NIC driver (as I see it
happen only with one particular NIC card) or am I reading the DOCs
wrong?

TIA,
Bandeep

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

6

> Is it possible that the IO completion routine is called in the same thread

context as the one that originally called IoCallDriver?

Yes.

DDK docs seem to suggest otherwise as it says “A driver’s IoCompletion
routine executes in an arbitrary thread or DPC ontext” (Ref: IoCompletion in
DDK doc).

“Arbitrary thread” means “thread context is not known”, so, it can be your
original thread.

In my TDI client driver, with one specific NIC card, I see that the send
completion routine is called in the same thread context as the one doing
the send (thread that called IoCallDriver).

I need to acquire a lock in the completion routine which, if already
acquired in the send thread, leads to a deadlock if the completion routine
is called in the same context.

Never ever call IoCallDriver or IoCompleteRequest while holding any kind of a
lock (the only exception is filesystem’s ERESOURCE locks which guard the file
size). This is a gross mis-design.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

7

> Never ever call IoCallDriver or IoCompleteRequest while

holding any kind of a
lock (the only exception is filesystem’s ERESOURCE locks
which guard the file
size). This is a gross mis-design.

Max, are you sure that was the ONLY exception in NT code?

It’s observed that w32k IoCalls video miniport with a mutex held in may
cases.

Calvin Guan Software Engineer
ATI Technologies Inc. www.ati.com

8

RE: [ntdev] IoCompletion Question The port-miniport model is usually full of such things, SCSIPORT also, serialiazed NDIS also. In these environments, the whole code of yours is running under the lock taken above you.

Nevertheless, the statement of “never do any calls out of your binary while holding a lock” is true in these environments too.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: Calvin Guan
To: Windows System Software Devs Interest List
Sent: Thursday, July 29, 2004 6:51 PM
Subject: RE: [ntdev] IoCompletion Question

Never ever call IoCallDriver or IoCompleteRequest while
> holding any kind of a
> lock (the only exception is filesystem’s ERESOURCE locks
> which guard the file
> size). This is a gross mis-design.
>

Max, are you sure that was the ONLY exception in NT code?

It’s observed that w32k IoCalls video miniport with a mutex held in may cases.

Calvin Guan Software Engineer
ATI Technologies Inc. www.ati.com

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com