Control Device Inhibits Unload

NTDEV
21

>This leads me to a conclusion that if driver calls ZwUnloadDriver() on

itself in context of a driver-created thread, its >DrvUnload() will get
invoked in context of another thread.

Unassemble the function, in a quick scan of my Vista VM I don’t see this to
be the case.

It also appears to be a synchronous operation, so if you wait for the thread
you posted in your unload routine I think you’d deadlock (if the I/O manager
would even allow it, that is. What happens if you try to trigger an unload
while you’re already unloading? Don’t know, don’t particularly care.).

It’s complex because it’s not a clearly defined scenario and thus subject to
lots of, "I suspect"s, "I think"s, and "it should"s. Luckily Tim has already
moved on to bigger and better things, so no one needs to spend time
modelling the behavior of the scenario on crufty old versions of the O/S
(though I think I still have a dusty copy of XP SP2 lying around if someone
wanted to check).

-scott

Scott Noone
Software Engineer
OSR Open Systems Resources, Inc.
http://www.osronline.com

wrote in message news:xxxxx@ntdev…
>> I’m not aware of any way for a non-PnP driver to ask for itself to be
>> unloaded.
>
> ZwUnloadDriver()…
>
> This function can be called by anyone, but DrvUnload() gets invoked in
> context of a system thread. This leads me to a conclusion that if driver
> calls ZwUnloadDriver() on itself in context of a driver-created thread,
> its DrvUnload() will get invoked in context of another thread.
>
> Therefore, the solution seems to be pretty easy. Create a thread in
> DriverEntry() that works like:
>
> KeWait( &event,…);
> ZwUnloadDriver(RegPath);
> PsTerminateSystemThread(0);
>
> When your driver wants to unload itself, it can signal the event, so that
> the above lines get executed. In DrvUnload() wait until the “unloader”
> thread gets signaled before you return - by doing so you will ensure
> that driver image cannot get unloaded from RAM until “unloader” thread
> terminates. What is so complex here???
>
> Anton Bassov
>
>

22

Nope, doesn’t work like that. A driver cannot unload itself by calling ZwUnloadDriver in its own code. Tim is exactly right, it will return into empty (or rather unmapped) memory and blow up. Furthermore, while this function can be called by anyone, it can only be called by the component which called ZwLoadDriver (or the equivalent loading routine). There are no load reference counts in the driver object (unlike a user mode DLL), so you cannot call ZwLoadDriver twice and then ZwUnloadDriver once and have the image still be present.

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Monday, April 07, 2008 12:14 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Control Device Inhibits Unload

I’m not aware of any way for a non-PnP driver to ask for itself to be unloaded.

ZwUnloadDriver()…

This function can be called by anyone, but DrvUnload() gets invoked in context of a system thread. This leads me to a conclusion that if driver calls ZwUnloadDriver() on itself in context of a driver-created thread, its DrvUnload() will get invoked in context of another thread.

Therefore, the solution seems to be pretty easy. Create a thread in DriverEntry() that works like:

KeWait( &event,…);
ZwUnloadDriver(RegPath);
PsTerminateSystemThread(0);

When your driver wants to unload itself, it can signal the event, so that the above lines get executed. In DrvUnload() wait until the “unloader” thread gets signaled before you return - by doing so you will ensure
that driver image cannot get unloaded from RAM until “unloader” thread terminates. What is so complex here???

Anton Bassov

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

23

Doron,

A driver cannot unload itself by calling ZwUnloadDriver in its own code. Tim is exactly right, it will return >into empty (or rather unmapped) memory and blow up.

Hold on - an image just cannot get unloaded from RAM before DrvUnload() returns control, can it??? Therefore, the “maximum” that one can expect in the solution that I have described is a deadlock, which is going to happen if ZwUnloadDriver() calls DrvUnload() synchronously - a thread is going to wait for itself to get signaled, which will never happen, for understandable reasons…

Anton Bassov

24

wrote in message news:xxxxx@ntdev…
>> I personally regard this as a fault in the Windows driver architecture
>> and feel there should be some way to > signal the PnP manager manually to
>> check for a driver unload in the event that your
>> control device realizes that it is the last device being shut down.
>
> Yes, but why do you think PnP should be bothered about non-PnP devices???
> In our particular case, there is one-to-one relationship between PnP
> device and control one, but this relationship is defined strictly
> by the logic of a driver, so it could be totally different, which is
> completely out of PnP’s scope. Once the only one who knows the logical
> relationship between devices that a driver creates is a driver itself, it
> makes a perfect sense to d make it responsible for tracking these
> devices…

The PnP device manager is holding the driver hostage basically since it has
a reference on the driver object. As such it is master of the driver object
so any request to unload has to be handled by it. The driver itself is in a
position to understand its device relationships and know exactly when it is
ready to be unloaded as you say, so it should be able to humbly notify the
master that it believes it is ready to be unloaded. Windows provides no
such mechanism that I am aware of and only relies on the PnP device manager
to automatically make the check when a PnP device is removed. That leaves a
driver stuck in memory in the event that the last device removed is not a
PnP device. That simply is not right, but that is the way the Windows
driver architecture works.

25

I tried something like this myself way back when and I think I recall that
ZwUnloadDriver checks the reference count before it will just pluck out a
driver. When the PnP manager creates the driver object due to a device
being plugged in it is holding a reference or lock on it, and it wont
release it until it decides to remove the driver itself. Because of that
outstanding reference ZwUnloadDriver will always fail, unless it is the PnP
manager that tries to do it right after it removes that reference. As far
as I know there’s no way to ask the PnP manager to perform the check to see
if it can unload the driver. It does so only in response to removal of a
PnP device owned by the driver.

Its been a long time since I did this, so I could be wrong but that is my
recollection.

xxxxx@hotmail.com> wrote in message news:xxxxx@ntdev…
Therefore, the solution seems to be pretty easy. Create a thread in
DriverEntry() that works like:

KeWait( &event,…);
ZwUnloadDriver(RegPath);
PsTerminateSystemThread(0);

When your driver wants to unload itself, it can signal the event, so that
the above lines get executed. In DrvUnload() wait until the “unloader”
thread gets signaled before you return - by doing so you will ensure
that driver image cannot get unloaded from RAM until “unloader” thread
terminates. What is so complex here???

26

The image will be unloaded in the middle of ZwUnloadDriver. So if the call stack looks like this

Yourdriver!DriverUnload
Nt!ZwDriverUnload
Yourdriver!TryToCleaup+0x123 <– calls ZwUnloadDriver

ZwDriverUnload will return to Yourdriver!TryToCleaup+0x123, but it will be unmapped and you will blow up. The os does not do reference count on other threads to know that it should rundown and wait or even generic rundown for you in general. This is why a driver in its unload routine must do its own rundown in unload() regardless of the caller invokes ZwUnloadDriver

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Monday, April 07, 2008 2:32 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Control Device Inhibits Unload

Doron,

A driver cannot unload itself by calling ZwUnloadDriver in its own code. Tim is exactly right, it will return >into empty (or rather unmapped) memory and blow up.

Hold on - an image just cannot get unloaded from RAM before DrvUnload() returns control, can it??? Therefore, the “maximum” that one can expect in the solution that I have described is a deadlock, which is going to happen if ZwUnloadDriver() calls DrvUnload() synchronously - a thread is going to wait for itself to get signaled, which will never happen, for understandable reasons…

Anton Bassov

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

27

“Tim Roberts” wrote in message news:xxxxx@ntdev…
> What we have here is a weird hybrid of a PnP and a non-PnP driver. If
> this was strictly a non-PnP driver, I would not expect it to be
> automatically unloaded. I have to request that via the service manager.
> In this case, when PnP wants to unload it, it can’t because there is still
> an open handle. When I close the handle, it’s a close on a non-PnP device
> object, so the unload check is never triggered. I’m not aware of any way
> for a non-PnP driver to ask for itself to be unloaded.

I think you are correct Tim, the service manager would be used (it would
make a call ZwUnloadDriver) to close down a non-PnP driver, but even if you
have your control panel application try to manually achieve that, it will
fail in this instance because the service manager will detect that the
driver is a PnP driver. It will not allow you to unload a PnP service
(driver) manually. They are removed automatically when the last PnP device
object is released, so long as there are no other references open. In your
case if the control device is the last device open, it is not PnP so will
not trigger an unload when you close it, but if you close it before the last
PnP device the driver will unload from memory.

Neither PnP nor the service control manager can unload your driver when the
last device it closed was non-PnP so it is stuck there in memory. Its an
unfortunate situation that requires more effort than it is worth to work
around.

28

Matthew Carter wrote:

Neither PnP nor the service control manager can unload your driver when the
last device it closed was non-PnP so it is stuck there in memory. Its an
unfortunate situation that requires more effort than it is worth to work
around.

Absolutely correct. And as Peter pointed out earlier today, my
curiosity has now been satisfied, and I’ve moved on to problems that matter.

I feel like I should apologize for presenting a molehill and offering
others the opportunity to turn it into a mountain.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

29

Then how they do this in Linux?
IIRC they run a task periodically to detect unowned/unreferenced modules and
unload them.
So a driver may remain loaded for few seconds after all it’s references go
away.

In Windows - ok, it seems like naïve solutions won’t work,
you’ll need to invent a bit more complex communication
between kernel and usermode parts, that does work
and works reliably :slight_smile:

Regards,
–PA

30

Sometimes we need more Everests. There is nothing that says we can’t be
entertained while we work. I found it interesting that a ‘solution’ just
creates more problems. If I end up doing drivers that don’t exist in a
miniport prison, I may find this information useful. I would say from the
thread that the following is true:

  1. If a PnP driver needs to receive special queries from user mode, hope
    your driver is the top driver and can receive any IoCtls without them being
    blocked by something that thinks it knows better. This means using the PnP
    device to receive those requests instead of a ‘control device object’ that
    is not part of the PnP world.
  2. If the first item is not true, then ‘hang on to your hat’ as you got
    some fun coming at you. If the traffic from user land is infrequent, then
    make sure user land does an open, IoCtl, and close for each sequence (more
    than one IoCtl could be required for a sequence to complete, but only if it
    is still closed quickly). Another option is to have an always pending IoCtl
    that can be cancelled in the PnP path to indicate that the app should close
    any handle immediately. An event could also be used if the only information
    needed to be conveyed is that it must terminate immediately. Maybe a delay
    in the PnP stop request can give it time to work, so when PnP gets it back
    the control device object will be gone.

Most of the above depends upon the design of the application and driver as
to how well they work together within the limitations of the Windows
universe.

“Tim Roberts” wrote in message news:xxxxx@ntdev…
> Matthew Carter wrote:
>> Neither PnP nor the service control manager can unload your driver when
>> the last device it closed was non-PnP so it is stuck there in memory.
>> Its an unfortunate situation that requires more effort than it is worth
>> to work around.
>>
>
> Absolutely correct. And as Peter pointed out earlier today, my curiosity
> has now been satisfied, and I’ve moved on to problems that matter.
>
> I feel like I should apologize for presenting a molehill and offering
> others the opportunity to turn it into a mountain.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>

31

Doron,

The image will be unloaded in the middle of ZwUnloadDriver. So if the call stack looks like this

Yourdriver!DriverUnload
Nt!ZwDriverUnload
Yourdriver!TryToCleaup+0x123 <– calls ZwUnloadDriver

If the stack looks like that, then it means that DrvUnload() is invoked in context of the same thread that calls ZwUnloadDriver(), rather than that of a system thread (which contradicts MSDN, BTW). If this is the case, then the whole thing that I described does not make sense, although for a different reason - we are going to deadlock , rather than bluescreen . DrvUnload() will wait until the thread’s state becomes signaled, but it may happen only after ZwUnloadDriver() returns and TryToCleaup() calls PsTerminateSystemThread(), and ZwUnloadDriver() may return only after DrvUnload() does. Impasse…

Therefore, the whole thing that I described is based upon the assumption that DrvUnload() gets invoked in context of a system thread the way MSDN says, rather that the one that calls DrvUnload() (once this routine can be called by anyone, then DrvUnload() is called in an arbitrary context, which contradicts MSDN)

Anton Bassov

32

In other words, dear Anton, you’re gonna deadlock if you wait in
DriverUnload for a call to ZwUnloadDriver to complete, regardless of the
implementation details of ZwUnloadDriver. And if you don’t wait, you’ll
crash, either with an inexplicable bugcheck caused by returning to nowhere,
or driver unloaded without cancelling blah, blah, blah, if verifier is on.

My advice is DON’T DO THAT.

Yer welcome.

  • Dan.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@hotmail.com
Sent: Monday, April 07, 2008 6:18 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Control Device Inhibits Unload

Doron,

The image will be unloaded in the middle of ZwUnloadDriver. So if the
call stack looks like this

Yourdriver!DriverUnload
Nt!ZwDriverUnload
Yourdriver!TryToCleaup+0x123 <– calls ZwUnloadDriver

If the stack looks like that, then it means that DrvUnload() is invoked in
context of the same thread that calls ZwUnloadDriver(), rather than that of
a system thread (which contradicts MSDN, BTW). If this is the case, then the
whole thing that I described does not make sense, although for a different
reason - we are going to deadlock , rather than bluescreen . DrvUnload()
will wait until the thread’s state becomes signaled, but it may happen only
after ZwUnloadDriver() returns and TryToCleaup() calls
PsTerminateSystemThread(), and ZwUnloadDriver() may return only after
DrvUnload() does. Impasse…

Therefore, the whole thing that I described is based upon the assumption
that DrvUnload() gets invoked in context of a system thread the way MSDN
says, rather that the one that calls DrvUnload() (once this routine can be
called by anyone, then DrvUnload() is called in an arbitrary context, which
contradicts MSDN)

Anton Bassov

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

33

> In other words, dear Anton, you’re gonna deadlock if you wait in DriverUnload for a call to ZwUnloadDriver > to complete , regardless of the implementation details of ZwUnloadDriver.

Really??? Even if DrvUnload() gets invoked in context of a thread that is different from the one that calls
ZwUnloadDriver()???

And if you don’t wait, you’ll crash,

This is out of question - indeed, if you don’t wait you will crash regardless of ZwUnloadDriver()'s implementation, because ZwUnloadDriver() will return to the middle of nowhere…

However, please note that, for this or that reason MSDN claims that DrvUnload() is invoked in context of a system thread, rather than arbitrary one. If DrvUnload() is invoked in context of some thread that is dedicated for this purpose, the trick with waiting should work just fine…

Anton Bassov

34

I do think that a non-PnP driver can cause itself to be unloaded by creating
a system thread that calls ZwUnloadDriver. It couldn’t sit and wait for
that call to complete, but I don’t see why it would need to. However, in
this particular case calling ZwUnloadDriver is completely out of the
question if the MSDN documentation is accurate. Part of the documentation
on MSDN says, “If DriverName is the name of a PnP device driver,
ZwUnloadDriver returns STATUS_INVALID_DEVICE_REQUEST and does not unload the
driver.” Here the device driver is PnP (although it does also create a
control device that is not a PnP device), so ZwDriverUnload should always
fail to have any effect at all. When the driver is PnP there’s nothing to
be done but wait for the PnP manager to unload it, which it won’t ever do if
the last device it closed was non-PnP.

35

> documentation on MSDN says, "If DriverName is the name of a PnP device driver,

ZwUnloadDriver returns STATUS_INVALID_DEVICE_REQUEST and does not unload the driver."
Here the device driver is PnP

The only things that make a driver PnP-compliant is, first, AddDevice(), and, second, IRP_MJ_PNP handler - PnP-compliant drivers fill these fields in DRIVER_OBJECT, while non-PnP ones leave them blank. Service description entry in a registry does not make any distinction between these two.Therefore, it looks like PnP-compliant driver can turn itself into non-PnP one one the fly simply by clearing
AddDevice() and IRP_MJ_PNP handler fields in its DRIVER_OBJECT. Certainly, this trick is dirty and ugly, so that one should not use it in production drivers even if it works (I am not so sure about it), but it may be interesting thing to try just for the sake of doing an experiment…

Anton Bassov

36

That does sound like an interesting experiment. It is certainly possible
that if the PnP manager doesn’t keep track of its devices except by those
two means that it could work. My gut instinct is that the PnP device
manager is keeping track of its device nodes and drivers in a tree somewhere
in its memory, so even if I could fool Windows into thinking that a driver
is not a PnP driver and removing it, I may cause some kind of memory leak or
later invalid memory access by the PnP device manager when it tried to
access it without realizing that it is gone. I’m not privy to the Windows
internals so I’d be afraid to do that on any computer except my own even if
it appeared to work.

wrote in message news:xxxxx@ntdev…
>> documentation on MSDN says, “If DriverName is the name of a PnP device
>> driver,
>> ZwUnloadDriver returns STATUS_INVALID_DEVICE_REQUEST and does not unload
>> the driver.”
>> Here the device driver is PnP
>
> The only things that make a driver PnP-compliant is, first, AddDevice(),
> and, second, IRP_MJ_PNP handler - PnP-compliant drivers fill these fields
> in DRIVER_OBJECT, while non-PnP ones leave them blank. Service description
> entry in a registry does not make any distinction between these
> two.Therefore, it looks like PnP-compliant driver can turn itself into
> non-PnP one one the fly simply by clearing
> AddDevice() and IRP_MJ_PNP handler fields in its DRIVER_OBJECT.
> Certainly, this trick is dirty and ugly, so that one should not use it in
> production drivers even if it works (I am not so sure about it), but it
> may be interesting thing to try just for the sake of doing an
> experiment…
>
> Anton Bassov
>

37

Sorry, won’t work. the io manager has to figure out that your driver is a pnp driver right after DriverEntry returns b/c there are 2 different correct behaviors. For a legacy NT4 driver, if DriverEntry returns success and does not create any devices, it is unloaded if DriverUnload is set. For a pnp driver, it is valid (neh, a requirement) to return success w/out any creating device objects. This means that before processing a successful return from DriverEntry, the io manager determines if it is a pnp driver (through a variety of tests, AddDevice and IRP_MJ_PNP being set are 2 of them) and caches it internally w/in the PDRIVER_OBJECT. Mucking with these fields later on will not have any effect since the kernel uses the internal flag.

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Matthew Carter
Sent: Monday, April 07, 2008 8:28 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Control Device Inhibits Unload

That does sound like an interesting experiment. It is certainly possible
that if the PnP manager doesn’t keep track of its devices except by those
two means that it could work. My gut instinct is that the PnP device
manager is keeping track of its device nodes and drivers in a tree somewhere
in its memory, so even if I could fool Windows into thinking that a driver
is not a PnP driver and removing it, I may cause some kind of memory leak or
later invalid memory access by the PnP device manager when it tried to
access it without realizing that it is gone. I’m not privy to the Windows
internals so I’d be afraid to do that on any computer except my own even if
it appeared to work.

wrote in message news:xxxxx@ntdev…
>> documentation on MSDN says, “If DriverName is the name of a PnP device
>> driver,
>> ZwUnloadDriver returns STATUS_INVALID_DEVICE_REQUEST and does not unload
>> the driver.”
>> Here the device driver is PnP
>
> The only things that make a driver PnP-compliant is, first, AddDevice(),
> and, second, IRP_MJ_PNP handler - PnP-compliant drivers fill these fields
> in DRIVER_OBJECT, while non-PnP ones leave them blank. Service description
> entry in a registry does not make any distinction between these
> two.Therefore, it looks like PnP-compliant driver can turn itself into
> non-PnP one one the fly simply by clearing
> AddDevice() and IRP_MJ_PNP handler fields in its DRIVER_OBJECT.
> Certainly, this trick is dirty and ugly, so that one should not use it in
> production drivers even if it works (I am not so sure about it), but it
> may be interesting thing to try just for the sake of doing an
> experiment…
>
> Anton Bassov
>

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

38

> Then how they do this in Linux? IIRC they run a task periodically to detect

unowned/unreferenced modules and unload them. So a driver may remain loaded for few seconds > after all it’s references go away.

I am afraid you are much too optimistic…

For the fun of doing it, I ran lsmod (64-bit Fedora 8), and got around 15 loaded modules that are not used by anyone. I repeated the experiment half an hour later…but still I see exactly the same unused modules loaded…

Anton Bassov

39

DriverUnload is always called in the context of a system process. If a driver calls ZwUnloadDriver w/in the context of the system process, DriverUnload is called directly inline. If not, ZwUnloadDriver will synchronously defer the call to a thread in the system process. Either way, to the caller of ZwUnloadDriver the behavior is synchronous and DriverUnload() has been called by the time it returns. The question is now…returning to what? If it is the driver which attempted to unload itself, it will return to unmapped memory. What it boils down to is a driver cannot unload itself, some other agent must unload the driver so that ZwUnloadDriver has something to return to.

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Monday, April 07, 2008 6:47 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Control Device Inhibits Unload

In other words, dear Anton, you’re gonna deadlock if you wait in DriverUnload for a call to ZwUnloadDriver > to complete , regardless of the implementation details of ZwUnloadDriver.

Really??? Even if DrvUnload() gets invoked in context of a thread that is different from the one that calls
ZwUnloadDriver()???

And if you don’t wait, you’ll crash,

This is out of question - indeed, if you don’t wait you will crash regardless of ZwUnloadDriver()'s implementation, because ZwUnloadDriver() will return to the middle of nowhere…

However, please note that, for this or that reason MSDN claims that DrvUnload() is invoked in context of a system thread, rather than arbitrary one. If DrvUnload() is invoked in context of some thread that is dedicated for this purpose, the trick with waiting should work just fine…

Anton Bassov

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

40

Doron,

If not, ZwUnloadDriver will synchronously defer the call to a thread in the system process.
Either way, to the caller of ZwUnloadDriver the behavior is synchronous and DriverUnload()
has been called by the time it returns. The question is now…returning to what?

Actually, in our case the question is not “returns where” but “does it return at all”…

Consider the scenario when DrvUnload() waits for something that never happens. What is going to happen from ZwUnloadDriver() caller’s perspective??? Will thread that calls ZwUnloadDriver() go dead for good??? In other words, does ZwUnloadDriver() defer a call to another thread and wait for it to complete, or does it return to the caller straight away? Judging from the fact that DrvUnload() is void function that does not return any value, the status that ZwUnloadDriver() returns to a caller should not depend on DrvUnload() call in any possible way, so that I would rather expect the latter scenario. Could you please check it…

Anton Bassov