on IoAcquireRemoveLock

massoud_navab · April 29, 2009, 5:42pm

This is a revisit of an earlier question.

Can a remove lock be reused, after a call to release and wait, or does it need a new re-init?
Does it make sense to acquire remove lock in irp_mj_close? or will it fail a legitimate close request?

OSR_Community_User · April 29, 2009, 5:55pm

For the first question:

I believe the lock cannot be reused, and it should not be re-initialized either. Here’s a quote where we tried to explain this - inside the paper from http://www.microsoft.com/whdc/devtools/tools/Win7DriverVer.mspx:

In Windows 7, Driver Verifier I/O Verification can detect another common driver bug - reinitializing remove locks. Remove locks data structures should be allocated inside device extensions. This ensures that the I/O manager frees the memory that holds the IO_REMOVE_LOCK structure only when the device object is deleted. If the driver performs the following three steps, it is possible that after step 2 an application or driver still holds a reference to Device1:

Allocates the IO_REMOVE_LOCK structure that corresponds to Device1 but outside Device1’s extension.
Calls IoReleaseRemoveLockAndWait when Device1 is being removed.
Calls IoInitializeRemoveLock for the same lock to reuse it as a remove lock for Device2.

It is possible that after step 2 an application or driver still holds a reference to Device1. The application or driver can still send requests to Device1, even though this device was removed. Therefore, it is not safe to reuse the same memory as a new remove lock until I/O manager deletes Device1. Reinitializing the same lock while another thread is trying to acquire it can result in the corruption of the lock, with unpredictable results for the driver and the entire system.

Dan

Doron_Holan · April 29, 2009, 5:59pm

you must reinitialize it (and that requires passive_level). An io remove lock serves a very specific purpose, guard io against the lifetime of a device object. While it looks nice and shiny and could possibly be used for other objects/allocations, that is not the case (I have tried). At that point, your own io counter/state tracking struct is going to serve you better
yes in the sense that it makes sure that your release and wait blocks until you release the remlock at the end of the close dispatch routine (which is after you have completed the close irp which would have let the remove come in while still in the close dispatch routine)

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.com
Sent: Wednesday, April 29, 2009 2:44 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] on IoAcquireRemoveLock

This is a revisit of an earlier question.

Can a remove lock be reused, after a call to release and wait, or does it need a new re-init?
Does it make sense to acquire remove lock in irp_mj_close? or will it fail a legitimate close request?

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

massoud_navab · April 29, 2009, 6:03pm

Thanks very much. I’ll check out the Win7 white paper.

OSR_Community_User · April 29, 2009, 6:07pm

From:
> 1) Can a remove lock be reused, after a call to release and wait, or does
> it need a new re-init?

Yes. You normally initialize it during AddDevice, at the same time as you
initialize everything else in your device extension.

> 2) Does it make sense to acquire remove lock in irp_mj_close? or will it
> fail a legitimate close request?

The short answer is that you want to “acquire” your remove lock before you
pass any IRP down the PNP stack to another driver and release it when that
IRP completes. You’re not passing IRP_MJ_CLOSE down if you’re a function
driver, so you don’t need the remove lock.

Plus, you won’t be getting an IRP_MJ_CLOSE after getting an
IRP_MN_REMOVE_DEVICE in the first place, so the remove lock call shouldn’t
fail.

If you want a complete explanation, read pp. 332-37 of my book. Presumably,
someone will now flame me for daring to analyze this…

Walter Oney
Consulting and Training
www.oneysoft.com

massoud_navab · April 29, 2009, 8:17pm

I’d like to use remove lock for the following condtions:

Replace the +1 based i/o counters that many drivers use to signal stop/remove events. That is to call IoReleaseRemoveLockAndWait in my irp_mn_xx_stop and irp_mn_xx_remove. For example,

Call IoReleaseRemoveLockAndWait to handle stop, then handle restart, call
acquire/release over and over, then finall call IoReleaseRemoveLockAndWait to handle remove.

Use removelock when i issue dcp, workitem, thread, etc.

Doron_Holan · April 29, 2009, 11:19pm

1 not what remlocks were designed for. If you could use a remlock for stop processing, all of the samples would have done it already. Remlocks are designed to drain io, stop needs to be transparent to the app and needs to pend io

2 that is fine

If you are at the stage that you are adding remlocks and stop support, i strongly recommend that you look at using kmdf which does all of this for you

d

Sent from my phone with no t9, all spilling mistakes are not intentional.

-----Original Message-----
From: xxxxx@yahoo.com
Sent: Wednesday, April 29, 2009 5:17 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] on IoAcquireRemoveLock

I’d like to use remove lock for the following condtions:

1) Replace the +1 based i/o counters that many drivers use to signal stop/remove events. That is to call IoReleaseRemoveLockAndWait in my irp_mn_xx_stop and irp_mn_xx_remove. For example,

Call IoReleaseRemoveLockAndWait to handle stop, then handle restart, call
acquire/release over and over, then finall call IoReleaseRemoveLockAndWait to handle remove.

2) Use removelock when i issue dcp, workitem, thread, etc.

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Maxim_S_Shatskih · April 30, 2009, 6:38am

>1) you must reinitialize it (and that requires passive_level). An io remove lock serves a very specific

purpose, guard io against the lifetime of a device object.

What is the need in remove lock at all?

PnP will not send MN_REMOVE_DEVICE to a devnode till there are any file objects existing on it.

Any IO (except shutdown notifications) sent from top by the IO manager requires a file object, so, if we are not speaking about self-managed IO originated somewhere in the devnode (and MJ_SHUTDOWN too) - then we are safe even without the remove lock.

As about self-managed IO (including MJ_SHUTDOWN and timers) - am I wrong that the drivers must shut it down and wait for it to actually shut down before passing MN_REMOVE_DEVICE down the devnode?

Looks like remove lock is a redundant thing (this was first noticed by Walter Oney in his book). Am I wrong?

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

OSR_Community_User · April 30, 2009, 8:10am

From: “Maxim S. Shatskih”
> Looks like remove lock is a redundant thing (this was first noticed by
> Walter Oney in his book). Am I wrong?

Except for power IRPs, I think that’s right for most device drivers. I
wouldn’t want to speculate concerning file system or mass storage
drivers/filters.

The key point is that no driver can protect ITSELF from being unmapped. Any
mechanism you can think of leaves the driver vulnerable to being unmapped
while it still has a RET instruction to execute, at the very least. That’s
the problem solved by IoSetCompletionRoutineEx and IoQueueWorkItem. Apart
from situations where those are appropriate, each driver has to rely on
whoever calls it to make sure it stays mapped until it returns.

Walter Oney
Consulting and Training
www.oneysoft.com

Doron_Holan · April 30, 2009, 9:31am

Max, i thought the same thing while developing kmdf, but as walter points out, this is incorrect. You can get pnp (non state changing), power (when you are not pwr policy policy owner), and wmi irps during and after remove irp processing. Each of these are initiated outside of the io manager and are not fileobj based, BUT each takes a ref on the stack to make sure the devobj is not gone (but could be removed)

d

Sent from my phone with no t9, all spilling mistakes are not intentional.

-----Original Message-----
From: Walter Oney
Sent: Thursday, April 30, 2009 5:11 AM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] on IoAcquireRemoveLock

From: “Maxim S. Shatskih”
> Looks like remove lock is a redundant thing (this was first noticed by
> Walter Oney in his book). Am I wrong?

Except for power IRPs, I think that’s right for most device drivers. I
wouldn’t want to speculate concerning file system or mass storage
drivers/filters.

The key point is that no driver can protect ITSELF from being unmapped. Any
mechanism you can think of leaves the driver vulnerable to being unmapped
while it still has a RET instruction to execute, at the very least. That’s
the problem solved by IoSetCompletionRoutineEx and IoQueueWorkItem. Apart
from situations where those are appropriate, each driver has to rely on
whoever calls it to make sure it stays mapped until it returns.

Walter Oney
Consulting and Training
www.oneysoft.com

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

OSR_Community_User · April 30, 2009, 11:06am

From: “Doron Holan”
> Max, i thought the same thing while developing kmdf, but as walter points
> out, this is incorrect. You can get pnp (non state changing), power (when
> you are not pwr policy policy owner), and wmi irps during and after remove
> irp processing. Each of these are initiated outside of the io manager and
> are not fileobj based, BUT each takes a ref on the stack to make sure the
> devobj is not gone (but could be removed)

I thought that deregistering for WMI (something I’ve always done at
STOP_DEVICE time) would wait until all WMI IRPs were drained through and
would prevent any new ones from being sent. Not so?

Walter Oney
Consulting and Training
www.oneysoft.com

Doron_Holan · April 30, 2009, 11:20am

You can still get query registration and as a filter, you can see wmi irps going down the stack in the case where the lower stack has not yet unregistered. Unregistering in stop is not semantically correct btw, stop should be transparent to the app, io or wmi

d

Sent from my phone with no t9, all spilling mistakes are not intentional.

-----Original Message-----
From: Walter Oney
Sent: Thursday, April 30, 2009 8:06 AM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] on IoAcquireRemoveLock

From: “Doron Holan”
> Max, i thought the same thing while developing kmdf, but as walter points
> out, this is incorrect. You can get pnp (non state changing), power (when
> you are not pwr policy policy owner), and wmi irps during and after remove
> irp processing. Each of these are initiated outside of the io manager and
> are not fileobj based, BUT each takes a ref on the stack to make sure the
> devobj is not gone (but could be removed)

I thought that deregistering for WMI (something I’ve always done at
STOP_DEVICE time) would wait until all WMI IRPs were drained through and
would prevent any new ones from being sent. Not so?

Walter Oney
Consulting and Training
www.oneysoft.com

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Maxim_S_Shatskih · April 30, 2009, 3:24pm

Then am I correct that the remove lock is needed only for non-state-changing PnP, power and WMI IRPs?

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

“Doron Holan” wrote in message news:xxxxx@ntdev…
Max, i thought the same thing while developing kmdf, but as walter points out, this is incorrect. You can get pnp (non state changing), power (when you are not pwr policy policy owner), and wmi irps during and after remove irp processing. Each of these are initiated outside of the io manager and are not

Doron_Holan · April 30, 2009, 4:46pm

As long as your reads/writes/ioctls/internal ioctls are file object based, yes. If you get any of these IOs that are not fileobject based, like from the stack above you, a remlock, while not required 100% of the time, is a pretty good idea most of the time depending on the behavior of the senders.

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Thursday, April 30, 2009 12:23 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Re:on IoAcquireRemoveLock

Then am I correct that the remove lock is needed only for non-state-changing PnP, power and WMI IRPs?

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

“Doron Holan” wrote in message news:xxxxx@ntdev…
Max, i thought the same thing while developing kmdf, but as walter points out, this is incorrect. You can get pnp (non state changing), power (when you are not pwr policy policy owner), and wmi irps during and after remove irp processing. Each of these are initiated outside of the io manager and are not

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer