EvtDeviceSurpriseRemoval not called

Hi
I’ve written a PCIe driver for an external PCie device that connected via PCI cable to PCI Adapter card which is on one of PCIe slots on the PC. Server 2008 R2 Enterprise.
The PCIe device the power supply is stand alone.
Thus user can power off the PCI device.
In such case I expect the framework will call to EvtDeviceSurpriseRemoval callback and then other callbacks in the removal sequence.

Unfortunately, nothing happens after a user turns power the PCI device down.
But when I open the device manager and scan manually for hardware changes , the removal sequence occurs and the Driver disappear.

I tried to enter the BIOS and validate for any P&P options , but haven’t found anything.
I will appreciate any suggestion ?

[quote]
Thus user can power off the PCI device. In such case

nothing happens. But when I open the device manager and scan manually for hardware changes , the
removal sequence occurs and the Driver disappear.

[quote]

Let’s start with the simple questions: What D-State is your device in when you remotely power it off? What S-State is the system in?

When you power your device down remotely, what are you trying to achieve? Do you want this to be a signal that you’re doing a “hot plug remove” operation? Or something else?

Normally, you can’t just “shut off” power on your PCIe device. Power state transitions are typically controlled from the host. If the card is present (plugged into the PCIe bus and responding) it needs to play by the rules of the bus. There is a (hardware) sequence by which a remote device can begin the hot plug remove process, but it’s non-trivial.

Peter
OSR
@OSRDrivers

Hi,
Unfortunately I can’t avoid from user to do this at any state. I do not control the physical power switch.
Usually the device in D0, S0 .
But I assume that the user isn’t so idiot to power down during he operates it (DMA transactions).
My question is , why no removal sequence callbacks come?
I think i can deal with any situation, once I have a notification about it.
Why the notification callbacks come only after I do “scan for hardware changes” from device manager?

> There is a sequence by which a remote device can begin the hot plug remove process … " >>
Where to start ? any reference ?

Because your PCI backplane doesn’t support surprise removal. It only sees the device is gone when asked (device manager scan for new hw). This is std behavior for PCI on PC hardware. Certain server classes support PCI removal with physical switches.

d

Bent from my phone


From: xxxxx@hotmail.commailto:xxxxx
Sent: ?12/?17/?2014 7:26 AM
To: Windows System Software Devs Interest Listmailto:xxxxx
Subject: RE:[ntdev] EvtDeviceSurpriseRemoval not called

Hi,
Unfortunately I can’t avoid from user to do this at any state. I do not control the physical power switch.
Usually the device in D0, S0 .
But I assume that the user isn’t so idiot to power down during he operates it (DMA transactions).
My question is , why no removal sequence callbacks come?
I think i can deal with any situation, once I have a notification about it.
Why the notification callbacks come only after I do “scan for hardware changes” from device manager?

>> There is a sequence by which a remote device can begin the hot plug remove process … " >>
Where to start ? any reference ?


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>

Thank you for quick response.
How you know that my PCI backplane doesn’t support surprise removal.
Do you assume it because this is Server HW?
This Server characteristic ?

> Certain server classes support PCI removal with physical switches.
What do you mean? can you explain?

Let’s start by being clear.

OK. We’re talking about an EXPRESS device…

Huh? PCI or Express? You can’t plug an Express device into a traditional PCI slot. What’s a PCI Cable?

Back to Express again.

So… I’m assuming you have an Express device and everything is Express.

I *think* Doron assumed you had a PCI device, or something.

Because they SHOULDN’T come, really, unless the device has specifically indicated that it’s been removed and made the link idle.

I’ll be the link is still up to your device.

Your device’s interface to the Express bus is almost certainly NOT externally powered, even if the piece of equipment it controls is. So… the device still appears to be present.

Peter
OSR
@OSRDrivers

Thank you for clarification call.
We speak only about PCIe.

Anyway, I’m not sure about any difference from PnP manager perspective either for PCI or PCIe.
I try to understand why PnP manager doesn’t send various remove IRPs to the driver, directing it to remove device object and so on …

Peter thanks you,
I can try to set PCI analyzer in order to find out if the link idle.

> Your device’s interface to the Express bus is almost certainly NOT externally powered …
> So… the device still appears to be present.

Yeah, probably this what happens. But if so , why it is absent after I do “scan hw changes” ?
And what can I do at that ?

If you have a driver for the device the driver can have a watchdog thread
that reads from the device memory periodically looking for an “all ffs”
result (and that cannot be the normal result) indicating that the device is
no longer present. It can use this to initiate the hardware rescan that
will in turn initiate surprise remove.

Mark Roddy

On Wed, Dec 17, 2014 at 10:37 AM, Doron Holan
wrote:
>
> Because your PCI backplane doesn’t support surprise removal. It only
> sees the device is gone when asked (device manager scan for new hw). This
> is std behavior for PCI on PC hardware. Certain server classes support PCI
> removal with physical switches.
>
> d
>
> Bent from my phone
> ------------------------------
> From: xxxxx@hotmail.com
> Sent: ‎12/‎17/‎2014 7:26 AM
> To: Windows System Software Devs Interest List
> Subject: RE:[ntdev] EvtDeviceSurpriseRemoval not called
>
> Hi,
> Unfortunately I can’t avoid from user to do this at any state. I do not
> control the physical power switch.
> Usually the device in D0, S0 .
> But I assume that the user isn’t so idiot to power down during he operates
> it (DMA transactions).
> My question is , why no removal sequence callbacks come?
> I think i can deal with any situation, once I have a notification about it.
> Why the notification callbacks come only after I do “scan for hardware
> changes” from device manager?
>
> >> There is a sequence by which a remote device can begin the hot plug
> remove process … " >>
> Where to start ? any reference ?
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Outstanding suggestion, Mr. Roddy. Excellent idea. And one that I did *not* consider.

A *lot* easier than implementing logic on the device to raise “ATTENTION” or whatever :wink:

Peter
OSR
@OSRDrivers

xxxxx@hotmail.com wrote:

Anyway, I’m not sure about any difference from PnP manager perspective either for PCI or PCIe.
I try to understand why PnP manager doesn’t send various remove IRPs to the driver, directing it to remove device object and so on …

Think about this from the system perspective. How would PnP know that
the device was removed? The only way it can know that is if the owning
bus driver calls IoInvalidateDeviceRelations to advertise that something
has changed. That is THE trigger. In response to that, PnP asks the
bus driver to scan its children and return the new list.

In order for the PCI driver to call IoInvalidateDeviceRelations, it has
to KNOW that something changed, and that means some piece of hardware
has to TELL it so. Some root complexes and bridge chips support
on-the-fly hot-plug detection, and will fire an interrupt that causes
the PCI driver to start this process. Some don’t. If they don’t, then
no one will ever know that a device has gone missing until some external
process forces a rescan of the children.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

xxxxx@osr.com wrote:

Outstanding suggestion, Mr. Roddy. Excellent idea. And one that I did *not* consider.

A *lot* easier than implementing logic on the device to raise “ATTENTION” or whatever :wink:

Well, there’s a “gotcha” in that plan. I have only encountered it once,
but there are some server-level machines with I/O architectures wherein
reading from a non-existent address causes a bus exception rather than
returning all FFs.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

10 years of working on fault tolerant windows platforms had its merits :slight_smile:

Mark Roddy

On Wed, Dec 17, 2014 at 2:41 PM, wrote:
>
>


>
> Outstanding suggestion, Mr. Roddy. Excellent idea. And one that I did
> not consider.
>
> A lot easier than implementing logic on the device to raise “ATTENTION”
> or whatever :wink:
>
> Peter
> OSR
> @OSRDrivers
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Certainly there are probably systems where this won’t work, but they would
be out of compliance for a PCI <- and I mean PCI -> bus. The Op is off in
the weeds of platform specificity anyway.

Mark Roddy

On Wed, Dec 17, 2014 at 2:59 PM, Tim Roberts wrote:
>
> xxxxx@osr.com wrote:
> >


> >
> > Outstanding suggestion, Mr. Roddy. Excellent idea. And one that I did
> not consider.
> >
> > A lot easier than implementing logic on the device to raise
> “ATTENTION” or whatever :wink:
>
> Well, there’s a “gotcha” in that plan. I have only encountered it once,
> but there are some server-level machines with I/O architectures wherein
> reading from a non-existent address causes a bus exception rather than
> returning all FFs.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Mark Roddy wrote:

Certainly there are probably systems where this won’t work, but they
would be out of compliance for a PCI <- and I mean PCI -> bus. The Op
is off in the weeds of platform specificity anyway.

I THINK we have established that the OP has a PCIExpress board.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

>card which is on one of PCIe slots on the PC. Server 2008 R2 Enterprise.

What about doing the same in 2012? 2012 R2?


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

> Huh? PCI or Express? You can’t plug an Express device into a traditional PCI slot. What’s a PCI

Cable?

I was also surprised, but yes, there are PCIe cables. You can google for them :slight_smile:


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

It all comes down to how “Unsupported Request” gets handled, I believe.

Haven’t seen an Express bus bugcheck on read of a bad address. Certainly have seen it on Conventional PCI. But it’s really rare.

Peter
OSR
@OSRDrivers

If I’m not mistaken, a bunch of conditions have to be met for the OS to
actually treat things as PCIe instead of PCI. One possibility is these
conditions are not being met, so the OS is treating the PCIe tree (device)
as a PCI tree (device).

Being that the OP is talking about some sort of device with a separate
power switch at the end of some sort of cable, the possibility of the
hardware not being treated as full PCIe by the OS seems like a real
possibility. The result would be you don’t get any of the typical PCIe
capability, like hot plug detection.

A PCIe bus analyzer certainly seems like a good strategy, as would
installing the checked pci.sys driver and turning on a bunch of tracing.

Jan

On 12/17/14, 8:18 PM, “Tim Roberts” wrote:

>Mark Roddy wrote:
>> Certainly there are probably systems where this won’t work, but they
>> would be out of compliance for a PCI <- and I mean PCI -> bus. The Op
>> is off in the weeds of platform specificity anyway.
>
>I THINK we have established that the OP has a PCIExpress board.
>
>–
>Tim Roberts, xxxxx@probo.com
>Providenza & Boekelheide, Inc.

Sure - I also think that the watchdog read will work on just about every
system out there and that the OP should take it for a test drive.

Mark Roddy

On Wed, Dec 17, 2014 at 3:18 PM, Tim Roberts wrote:
>
> Mark Roddy wrote:
> > Certainly there are probably systems where this won’t work, but they
> > would be out of compliance for a PCI <- and I mean PCI -> bus. The Op
> > is off in the weeds of platform specificity anyway.
>
> I THINK we have established that the OP has a PCIExpress board.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>