While working with reference counting with device interfaces, it looks to me as-if target driver can get unloaded even if the client driver does not call InterfaceDereference() on its device interface. Is it a known issue ? If yes, then what's use of InterfaceReference if it cannot prevent target driver to unload until clients call InterfaceDerefence() ?
Scenario overview:
-> I have a PnP driver DrvA which exposes a device interface, and a non-PnP driver DrvB which consumes that.
-> DrvA is a KMDF driver and DrvB is WDM non-PnP driver.
-> DrvA initializes the device interface using WDF_QUERY_INTERFACE_CONFIG_INIT(), and specify InterfaceHeader.InterfaceReference=WdfDeviceInterfaceReferenceNoOp and InterfaceHeader.InterfaceDereference=WdfDeviceInterfaceDereferenceNoOp.
-> DrvB gets the device object of DrvA and get its device interface using IRP_MN_QUERY_INTERFACE IRP. I believe internally WdfDeviceInterfaceReferenceNoOp() got called before giving interface to DrvB.
-> When DrvA device is disabled from devmgr, DrvB handles IRP_TARGET_QUERY_REMOVE and releases reference to DrvA device object it has but have not called (devInterface.InterfaceDereference()) to release the interface. So why do I see DrvA get unloaded if there is pending reference to its device interface which is not yet released by DrvB ?
From: xxxxx@gmail.com Sent: Tuesday, October 20, 2015 4:46 PM To: Windows System Software Devs Interest List Subject: [ntdev] DeviceInterface.InterfaceReference not preventing driverUnload
Hi All,
While working with reference counting with device interfaces, it looks to me as-if target driver can get unloaded even if the client driver does not call InterfaceDereference() on its device interface. Is it a known issue ? If yes, then what’s use of InterfaceReference if it cannot prevent target driver to unload until clients call InterfaceDerefence() ?
Scenario overview: ----------------------- -> I have a PnP driver DrvA which exposes a device interface, and a non-PnP driver DrvB which consumes that. -> DrvA is a KMDF driver and DrvB is WDM non-PnP driver. -> DrvA initializes the device interface using WDF_QUERY_INTERFACE_CONFIG_INIT(), and specify InterfaceHeader.InterfaceReference=WdfDeviceInterfaceReferenceNoOp and InterfaceHeader.InterfaceDereference=WdfDeviceInterfaceDereferenceNoOp. -> DrvB gets the device object of DrvA and get its device interface using IRP_MN_QUERY_INTERFACE IRP. I believe internally WdfDeviceInterfaceReferenceNoOp() got called before giving interface to DrvB. -> When DrvA device is disabled from devmgr, DrvB handles IRP_TARGET_QUERY_REMOVE and releases reference to DrvA device object it has but have not called (devInterface.InterfaceDereference()) to release the interface. So why do I see DrvA get unloaded if there is pending reference to its device interface which is not yet released by DrvB ?
Hi Doron, thanks for your comment. In that case I have reached below conclusions. Can you please confirm if my understanding is right ?
It looks like its solely up-to the server driver to implement reference counting and ensure that it does not unload while (any client still have an active reference count to its interface but do not have active reference to DO of the server driver). WDF/WDM framework does not ensure anything like that.
Also, there are 2 ways in which such peer driver communication can ensure that server cannot be unloaded while client is using its device interface:
Document to clients that server do not handle ref-counting for device interface, and in that case clients have to keep reference to DO associated with device-interface alive as long as it is using the server's interface. If server is remote stack PnP device, then clients have to release DO in response to TARGET_QUERY_REMOVE notification, and before that also ensure it NULLIFY its direct-call-interface copy which it received from the server.
Document to clients that server handle ref-counting on the interface. In such case:
(a) client can release server DO as soon as it got the direct-call-interface via IRP_MN_QUERY_INTERFACE.
(b) server *have-to* implement its own 'InterfaceReference/InterfaceDereference' and their implementation can simply as follows:
While working with reference counting with device interfaces, it looks to me as-if target driver can get unloaded even if the client driver does not call InterfaceDereference() on its device interface. Is it a known issue ? If yes, then what’s use of InterfaceReference if it cannot prevent target driver to unload until clients call InterfaceDerefence() ?
It CAN be used prevent drivers from unloading, but it’s not automatic –
the serving driver has to do it.
-> DrvA initializes the device interface using WDF_QUERY_INTERFACE_CONFIG_INIT(), and specify InterfaceHeader.InterfaceReference=WdfDeviceInterfaceReferenceNoOp and InterfaceHeader.InterfaceDereference=WdfDeviceInterfaceDereferenceNoOp.
And by doing so, you are telling the system “my driver doesn’t have any
reference dependency issues”, as the documentation for
WdfDeviceInterfaceReferenceNoOp clearly states.
If you want to use the interface reference count to prevent unloading,
then you need to supply callbacks for InterfaceReference and
InterfaceDereference that do reference counting.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Let’s take a step back for a second. InterfaceReference/Deref does not maintain state, it maintains memory lifetime. That means you can’t use it to prevent the device stack to be torn down or the driver unloaded (even ObRef won’t prevent DriverUnload from being called, it will just prevent the image from being unloaded).
There are two types of callers to query interface. First is within the stack. These callers know to stop using the interface when the get the query remove/remove. They intrinsically know who to send the query call to (top of stack) . The caller’s state matches the interface’s.
The second is outside the stack. These callers must open a handle to send the query and then are required to register for state notifications on the handle and stop using the interface when the query remove comes. They CANNOT close the handle immediately after querying for the interface b/c then there is no way to know when the underlying implementation goes away. By keeping the handle open you can then prevent query remove from succeeding (if desired).
Where does that leave you? InterfaceReference doesn’t do what you want and there other physics in the system that fulfill the need, thus leaving InterfaceReference to do nothing
d
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Wednesday, October 21, 2015 6:01 AM
To: Windows System Software Devs Interest List Subject: RE:[ntdev] DeviceInterface.InterfaceReference not preventing driverUnload
Hi Doron, thanks for your comment. In that case I have reached below conclusions. Can you please confirm if my understanding is right ?
----------------------------------------------------------------------------------------------------------------- It looks like its solely up-to the server driver to implement reference counting and ensure that it does not unload while (any client still have an active reference count to its interface but do not have active reference to DO of the server driver). WDF/WDM framework does not ensure anything like that.
Also, there are 2 ways in which such peer driver communication can ensure that server cannot be unloaded while client is using its device interface:
1) Document to clients that server do not handle ref-counting for device interface, and in that case clients have to keep reference to DO associated with device-interface alive as long as it is using the server’s interface. If server is remote stack PnP device, then clients have to release DO in response to TARGET_QUERY_REMOVE notification, and before that also ensure it NULLIFY its direct-call-interface copy which it received from the server.
2) Document to clients that server handle ref-counting on the interface. In such case:
(a) client can release server DO as soon as it got the direct-call-interface via IRP_MN_QUERY_INTERFACE.
(b) server have-to implement its own ‘InterfaceReference/InterfaceDereference’ and their implementation can simply as follows:
(c) client needs to call InterfaceDerefence() in TARGET_QUERY_REMOVE to trigger removal of internal reference count keep on the server device-object. ---------------------------------------------------------------------------------------------------------------
Thanks Doron and Tim for your comments. My scenario is on ‘callers outside the server device stack’ ? both PnP server driver and non-PnP client driver are owned by me.
Hi Doron, I experimented on drivers a bit to understand your comments and I think I got what you mean. Below is what I see if I try disabling/enabling device from devmgr and I explicitly leak reference to the server driver while client processes TARGET_DEVICE_QUERY_REMOVE:
(1) If I do not release reference to fileObject corresponding to server driver?s DO while query_remove, query_remove IRP fails and in devmgr it says “u need to restart PC as hardware settings have changed”. So I guess when you said “By keeping the handle open you can prevent query remove from succeeding (if desired)”, then you are saying about fileObject.
(2) If I release reference to fileObject in query_remove handling at client side, but before that do ObReferenceObject() on deviceObject. Then I can see that even EvtDeviceContextCleanup/ EvtDriverCleanup are getting called but driver image is not unloaded. In this case, from devmgr I can see device disabled successfully. This also looks to be in-line to your explanation on what will happen if we take reference to DO ? it will just ensure memory lifetime is secured but state will be torned down.
Q. Is my above understanding correct now ?
Q. Also, in case (2) above, I am seeing problems post device disable. Device is disabled properly and driver image also not unloaded, but when I try to re-enable the device from devmgr, it says me “u need to restart PC as hardware settings have changed” and device cannot be re-enabled. Can you please help understand what can be the reason behind that ?
[Tim]: It CAN be used prevent drivers from unloading, but it’s not automatic – the serving driver has to do it.
Considering the case I want to prevent driver image to get unloaded when device is disabled, and I want device re-enabling also to work fine, what should I do in my custom InterfaceIncrement/ InterfaceDecrement ? Referring above experiment I did, if I take and release reference to my own DO in these functions, looks like some driver state is disturbed and while device is able to get disabled, re-enabling it is giving errors on devmgr and its saying reboot is required.
Considering the case I want to prevent driver image to get unloaded when device is disabled, and I want device re-enabling also to work fine, what should I do in my custom InterfaceIncrement/ InterfaceDecrement ? Referring above experiment I did, if I take and release reference to my own DO in these functions, looks like some driver state is disturbed and while device is able to get disabled, re-enabling it is giving errors on devmgr and its saying reboot is required.
You don’t want to interfere with normal PnP operation. If the OS wants
unload the driver, better let it do this.
To avoid unload, you have the root-enumerated instance which is not
bound to your real hardware and underlying bus (PCI?).
It will pin the driver in memory when no real devices are present.
However, if user decides to disable this device too, the driver has to
unload.
Hi Pavel, the scenario I have taken here is different that what I have discussed in the other OSR thread (using controller device object to aggregate multiple devices). Here it just a normal PnP driver exposing one device and one non-PnP client to that. I am just trying to understand use of InterfaceReference/ InterfaceDereference here and if it can help in this scenario.
Tim, Doron, please do provide your comments on follow-up questions I have asked.
The re-enable//reload fails after unload//image still loaded because the image is still loaded and the kernel loader can’t load another copy of the image. To reload a driver the image must unload and it hasn’t.
From: xxxxx@gmail.com Sent: Wednesday, October 21, 2015 5:09 PM To: Windows System Software Devs Interest List Subject: RE:[ntdev] DeviceInterface.InterfaceReference not preventing driverUnload
Thanks Doron and Tim for your comments. My scenario is on ‘callers outside the server device stack’ ? both PnP server driver and non-PnP client driver are owned by me.
Hi Doron, I experimented on drivers a bit to understand your comments and I think I got what you mean. Below is what I see if I try disabling/enabling device from devmgr and I explicitly leak reference to the server driver while client processes TARGET_DEVICE_QUERY_REMOVE:
(1) If I do not release reference to fileObject corresponding to server driver?s DO while query_remove, query_remove IRP fails and in devmgr it says “u need to restart PC as hardware settings have changed”. So I guess when you said “By keeping the handle open you can prevent query remove from succeeding (if desired)”, then you are saying about fileObject.
(2) If I release reference to fileObject in query_remove handling at client side, but before that do ObReferenceObject() on deviceObject. Then I can see that even EvtDeviceContextCleanup/ EvtDriverCleanup are getting called but driver image is not unloaded. In this case, from devmgr I can see device disabled successfully. This also looks to be in-line to your explanation on what will happen if we take reference to DO ? it will just ensure memory lifetime is secured but state will be torned down.
Q. Is my above understanding correct now ? Q. Also, in case (2) above, I am seeing problems post device disable. Device is disabled properly and driver image also not unloaded, but when I try to re-enable the device from devmgr, it says me “u need to restart PC as hardware settings have changed” and device cannot be re-enabled. Can you please help understand what can be the reason behind that ?
> [Tim]: It CAN be used prevent drivers from unloading, but it’s not automatic – the serving driver has to do it.
Considering the case I want to prevent driver image to get unloaded when device is disabled, and I want device re-enabling also to work fine, what should I do in my custom InterfaceIncrement/ InterfaceDecrement ? Referring above experiment I did, if I take and release reference to my own DO in these functions, looks like some driver state is disturbed and while device is able to get disabled, re-enabling it is giving errors on devmgr and its saying reboot is required.
[Don]: To reload a driver, the image must unload.
[Peter]: You can’t LOAD the new driver because the OLD driver hasn’t been unloaded.
Then how does multiple devices sharing the same driver image works ? For example, if dev0 and dev1 belongs to same driver and we enable dev0 first, how are we able to enable dev1 also after that ? Shoudn’t this also fail because it will try to load the driver but driver is already loaded by dev0 ? As far as I know, all devices exposed from same sys file share same driver image.
Thanks Maxim for your comments, in that case if I understand correctly:
-> PnP keeps track of DriverEntry/DriverUnload of PnP drivers as part of state maintenance.
-> If a new device appear and PnP come to know that its DriverEntry has already been called by another device already enabled, it does not try to reload driver and just call EvtDeviceAdd.
-> In my case where client had forcefully taken reference to device-object and hence not allowed driver image to be released from memory, DriverUnload routine have also completed By PnP. Hence when we try to re-enable the device, then PnP will expect to do [load driver image again + call DriverEntry]. And it will fail.