Multifunction A reloaded while B is active

NTDEV
1

Hi,

I have a PCIe device and, since I am not a PCIe expert, do allow me to ask a question:

  • what’s the reason for which the OS reloads a function belonging to an MF device? Is the reason logged somewhere? setupapi.dev.log does not reveal why this is happening.
  • the reload happens more often on some HW platforms.

Thank you for taking the time,
? Calin

2

What does “reload a function” means? Does it mean “driver restart”? What do you see in the system event log?

3

> Hi,

I have a PCIe device and, since I am not a PCIe expert, do allow me to ask
a question:

  • what’s the reason for which the OS reloads a function belonging to an MF
    device? Is the reason logged somewhere? setupapi.dev.log does not reveal
    why this is happening.
  • the reload happens more often on some HW platforms.

There is no possible way for the system to “reload a function”. However,
if you meant “reload a function driver” then, yes, this can happen. It
only happens if the driver is unloaded. To unload a driver, you have to
make sure that all the pending IRPs are completed with error indications,
because the IRPs cannot be “remembered” across a driver reload. You must
also ensure that if your driver is handling devices A and B that, if they
are handled in a single driver, you don’t just ensure that A has cleaned
up properly and leave lots of uncompleted IRPs for B; you have to make
sure both devices are properly quiesced and both drivers have properly
cleaned up before allowing the unload to proceed. The PNP Manager might
handle all this for you by sending the correct PnP/Power notifications to
both the devices you are supporting. I don’t know how this is handled.
joe

joe

Thank you for taking the time,
  Calin

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

4

I suspect he means “reload a driver for a given function” – He has a multi-function PCIe device, and apparently the function driver for one of the functions on this device is periodically getting unloaded and reloaded.

OP… how did I go? Good guess on my part??

The only reason this should be the case is if the function is dropping off-line (with Windows subsequently unloading the driver) and coming back online (with Windows subsequently REloading the driver). We’ve seen other reports of this here on NTDEV… if this is what’s happening, it’s a fundamental hardware problem.

There really is no other reason Windows would unload a driver and then reload it.

Peter
OSR

5

Is PCIe “hot plug” by default? I only ask this because it would be the
bus driver that detects the device being missing, does all the protocols
via the PnP Manager (“Hey, remember that device I said we had…?”) to get
the driver unloaded, then sees it reappear and of course reloads the
driver. I didn’t think that the stock non-hotplug PCI bus driver could do
that, but PCI is not hotpluggable except on certain motherboards. For
these, as I had understood it, a special PCI hotplug bus driver is used.
So, my question on PCIe, does it have two bus drivers, one for hotplug and
one for non-hotplug? And if so, does the board the OP is using support
PCIe hotplug? Of course, if PCIe is intrinsically hotpluggable, There Can
Be Only One™ driver, and yes, I agree, it looks like a hardware
problem, although it could be a software issue of the bus driver
overreacting. I’d think some logging statements at various key PnP and/or
Power callbacks might be revealing.
joe

“There Can Be Only One” is a trademark of Highlander Software Ltd.,
headquartered in Glasgow, Scotland, and is used here without permission.

I suspect he means “reload a driver for a given function” – He has a
multi-function PCIe device, and apparently the function driver for one of
the functions on this device is periodically getting unloaded and
reloaded.

OP… how did I go? Good guess on my part??

The only reason this should be the case is if the function is dropping
off-line (with Windows subsequently unloading the driver) and coming back
online (with Windows subsequently REloading the driver). We’ve seen other
reports of this here on NTDEV… if this is what’s happening, it’s a
fundamental hardware problem.

There really is no other reason Windows would unload a driver and then
reload it.

Peter
OSR

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

6

> There really is no other reason Windows would unload a driver and then reload it.

IoInvalidateDeviceState call allows the devnode to commit suicide.

I think that, after this, the next bus rescan (unless you’re using the root enumerator and thus the PDO is really gone forever) will resurrect it.

–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

7

Joe,

PCI Express is hot-plug by default, at least at the “device” end of the wire. (PCIe is a point-to-point technology, called a “bus” only for name recognition.) The “slot” end of the wire, which may be just a trace on the motherboard, can optionally implement a hot-plug controller, which then gives software a state machine to manage for applying power and clocks to the slot, etc. All common chipsets implement these hot-plug controllers and most BIOSes will configure those chipsets to hide the hot-plug control registers for slots which do not have independent power control and for devices physically embedded on the motherboard.

The stock PCI driver (the one included in Windows) will manage PCI Express hot-plug if two things are true: first that the BIOS gives up control of PCI Express to Windows and second that the BIOS configures the slot to expose the hot-plug controller. There are not two bus drivers.

If the BIOS fails to give up control of PCI Express to Windows, it can still expose hot-plug through ACPI, managing PCI Express internally. This works, though the quality of implementation is spotty. When this comes into play, there is still only one bus driver, though ACPI.sys, acting as a filter, performs some of the work.

The most common implementation of PCI Express hot-plug is called “Express Card.” It’s a laptop expansion slot similar to Cardbus, except that it’s PCI Express instead of an amalgam of conventional PCI and ISAPnP (also called PCMCIA.) Thunderbolt is a newer form of PCIe hot-plug.

  • Jake Oshins
    (occasional PCI guy)
    Windows Kernel Team

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@flounder.com
Sent: Tuesday, November 19, 2013 2:09 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Multifunction A reloaded while B is active

Is PCIe “hot plug” by default? I only ask this because it would be the
bus driver that detects the device being missing, does all the protocols
via the PnP Manager (“Hey, remember that device I said we had…?”) to get
the driver unloaded, then sees it reappear and of course reloads the
driver. I didn’t think that the stock non-hotplug PCI bus driver could do
that, but PCI is not hotpluggable except on certain motherboards. For
these, as I had understood it, a special PCI hotplug bus driver is used.
So, my question on PCIe, does it have two bus drivers, one for hotplug and
one for non-hotplug? And if so, does the board the OP is using support
PCIe hotplug? Of course, if PCIe is intrinsically hotpluggable, There Can
Be Only One™ driver, and yes, I agree, it looks like a hardware
problem, although it could be a software issue of the bus driver
overreacting. I’d think some logging statements at various key PnP and/or
Power callbacks might be revealing.
joe

“There Can Be Only One” is a trademark of Highlander Software Ltd.,
headquartered in Glasgow, Scotland, and is used here without permission.

I suspect he means “reload a driver for a given function” – He has a
multi-function PCIe device, and apparently the function driver for one of
the functions on this device is periodically getting unloaded and
reloaded.

OP… how did I go? Good guess on my part??

The only reason this should be the case is if the function is dropping
off-line (with Windows subsequently unloading the driver) and coming back
online (with Windows subsequently REloading the driver). We’ve seen other
reports of this here on NTDEV… if this is what’s happening, it’s a
fundamental hardware problem.

There really is no other reason Windows would unload a driver and then
reload it.

Peter
OSR

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

8

Thanks for the explanation. I was not at all sure about it.

Note that this suggests that to analyze the OP’s problem, we need to know
something about how control of PCIe is managed; that “somewhat spotty”
case may be the one the OP is facing.
joe

Joe,

PCI Express is hot-plug by default, at least at the “device” end of the
wire. (PCIe is a point-to-point technology, called a “bus” only for name
recognition.) The “slot” end of the wire, which may be just a trace on
the motherboard, can optionally implement a hot-plug controller, which
then gives software a state machine to manage for applying power and
clocks to the slot, etc. All common chipsets implement these hot-plug
controllers and most BIOSes will configure those chipsets to hide the
hot-plug control registers for slots which do not have independent power
control and for devices physically embedded on the motherboard.

The stock PCI driver (the one included in Windows) will manage PCI Express
hot-plug if two things are true: first that the BIOS gives up control of
PCI Express to Windows and second that the BIOS configures the slot to
expose the hot-plug controller. There are not two bus drivers.

If the BIOS fails to give up control of PCI Express to Windows, it can
still expose hot-plug through ACPI, managing PCI Express internally. This
works, though the quality of implementation is spotty. When this comes
into play, there is still only one bus driver, though ACPI.sys, acting as
a filter, performs some of the work.

The most common implementation of PCI Express hot-plug is called “Express
Card.” It’s a laptop expansion slot similar to Cardbus, except that it’s
PCI Express instead of an amalgam of conventional PCI and ISAPnP (also
called PCMCIA.) Thunderbolt is a newer form of PCIe hot-plug.

  • Jake Oshins
    (occasional PCI guy)
    Windows Kernel Team

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@flounder.com
Sent: Tuesday, November 19, 2013 2:09 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Multifunction A reloaded while B is active

Is PCIe “hot plug” by default? I only ask this because it would be the
bus driver that detects the device being missing, does all the protocols
via the PnP Manager (“Hey, remember that device I said we had…?”) to get
the driver unloaded, then sees it reappear and of course reloads the
driver. I didn’t think that the stock non-hotplug PCI bus driver could do
that, but PCI is not hotpluggable except on certain motherboards. For
these, as I had understood it, a special PCI hotplug bus driver is used.
So, my question on PCIe, does it have two bus drivers, one for hotplug and
one for non-hotplug? And if so, does the board the OP is using support
PCIe hotplug? Of course, if PCIe is intrinsically hotpluggable, There Can
Be Only One™ driver, and yes, I agree, it looks like a hardware
problem, although it could be a software issue of the bus driver
overreacting. I’d think some logging statements at various key PnP and/or
Power callbacks might be revealing.
joe

“There Can Be Only One” is a trademark of Highlander Software Ltd.,
headquartered in Glasgow, Scotland, and is used here without permission.

> I suspect he means “reload a driver for a given function” – He has a
> multi-function PCIe device, and apparently the function driver for one
> of
> the functions on this device is periodically getting unloaded and
> reloaded.
>
> OP… how did I go? Good guess on my part??
>
> The only reason this should be the case is if the function is dropping
> off-line (with Windows subsequently unloading the driver) and coming
> back
> online (with Windows subsequently REloading the driver). We’ve seen
> other
> reports of this here on NTDEV… if this is what’s happening, it’s a
> fundamental hardware problem.
>
> There really is no other reason Windows would unload a driver and then
> reload it.
>
> Peter
> OSR
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

9

Hi Grigora,

? I am trying to narrow down to a replicable scenario. The sequence is this:

  1. driver for Function A is loaded
  2. driver for Function B is loaded
  3. driver for Function A is unloaded
  4. driver for Function A is loaded

Keep you posted and thank you for the replies,
? Calin

On Monday, November 18, 2013 9:00 PM, “xxxxx@broadcom.com” wrote:

What does “reload a function” means? Does it mean “driver restart”? What do you see in the system event log?

—
NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!!? See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

10

>1) driver for Function A is loaded

  1. driver for Function B is loaded
  2. driver for Function A is unloaded
  3. driver for Function A is loaded

Again, what does “loaded” mean?