Devices attached to PCIe powered off/on

All,

I’m trying to recover (reinitialize) a device attached to a PCIe card after it has been power cycled. The framework doesn’t detect this event and so, I don’t get any of the pnpPowerCallbacks called back. It doesn’t call either the EvtDeviceD0Exit or EvtDeviceReleaseHardware callbacks. I can detect the event because I have a watchdog timer running that is looking at a register with a changing value on the device. So, is there a way from the driver to reset this device or does it require an external application?

Thanks,
Harry

Ford, Harry L wrote:

I’m trying to recover (reinitialize) a device attached to a PCIe card
after it has been power cycled. The framework doesn’t detect this
event and so, I don’t get any of the pnpPowerCallbackscalled back. It
doesn’t call either the EvtDeviceD0Exit or EvtDeviceReleaseHardware
callbacks. I can detect the event because I have a watchdog timer
running that is looking at a register with a changing value on the
device. So, is there a way from the driver to reset this device or
does it require an external application?

If you can still read the registers, then the PCIe connection itself
never went down. The operating system doesn’t know that anything
changed, so there’s nothing for PnP to report. This is just an activity
that is local to your driver.

The phrase “reset this device” has no global meaning in PCIExpress. If
your hardware has components that can be reset, then only you know how
to make that happen, and what will happen when you do.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

I can’t really still read the register’s, value (it’s a countdown value) and it remains the same between expirations of the watchdog so, I know the device is no longer functioning. The only way to recover seems to either do a disable/enable from the device manager or use devcon. I’m hoping there’s a way to get the driver to go back through its DriverEntry point programmatically from the driver.

-Harry

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Tuesday, November 11, 2014 12:23 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Devices attached to PCIe powered off/on

Ford, Harry L wrote:

I’m trying to recover (reinitialize) a device attached to a PCIe card after it has been power cycled. The framework doesn’t detect this event and so, I don’t get any of the pnpPowerCallbacks called back. It doesn’t call either the EvtDeviceD0Exit or EvtDeviceReleaseHardware callbacks. I can detect the event because I have a watchdog timer running that is looking at a register with a changing value on the device. So, is there a way from the driver to reset this device or does it require an external application?

If you can still read the registers, then the PCIe connection itself never went down. The operating system doesn’t know that anything changed, so there’s nothing for PnP to report. This is just an activity that is local to your driver.

The phrase “reset this device” has no global meaning in PCIExpress. If your hardware has components that can be reset, then only you know how to make that happen, and what will happen when you do.

Tim Roberts, xxxxx@probo.commailto:xxxxx

Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx>

Ford, Harry L wrote:

I can’t really still read the register’s, value (it’s a countdown
value) and it remains the same between expirations of the watchdog so,
I know the device is no longer functioning.

If your device had dropped off of the bus, the address your driver has
stored would no longer be valid. Usually, you’ll read back FFFFFFFF.
If you’re reading data, then your PCIExpress IP is responding. In that
case, whatever problem you have is strictly a problem on your device.

The only way to recover seems to either do a disable/enable from the
device manager or use devcon. I’m hoping there’s a way to get the
driver to go back through its DriverEntry point programmatically from
the driver.

No, there isn’t, but you don’t need that. Disable and enable don’t do
ANYTHING to the hardware. That’s just a software operation. What you
need to do is do the same things you did in those callbacks. You don’t
need to remap the space, but repeat whatever register initialization you
did.

So, take the code from your EvtDeviceD0Entry and EvtDeviceD0Exit
routines, and put them in separate functions called “PowerDown” and
“PowerUp”. Now, in D0Entry you can call PowerUp, and whenever you
detect that your watchdog has stopped counting, you can call PowerDown()
then PowerUp().


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Try calling WdfDeviceSetFailed(WdfDeviceFailedAttemptRestart). If PCI supports GUID_REENUMERATE_SELF_INTERFACE_STANDARD on the OS you are running, your device will be surprise removed and reenumerated. If not supported, your only recourse is to disable/enable in user mode

d

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ford, Harry L
Sent: Tuesday, November 11, 2014 1:15 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Devices attached to PCIe powered off/on

I can’t really still read the register’s, value (it’s a countdown value) and it remains the same between expirations of the watchdog so, I know the device is no longer functioning. The only way to recover seems to either do a disable/enable from the device manager or use devcon. I’m hoping there’s a way to get the driver to go back through its DriverEntry point programmatically from the driver.

-Harry

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Tuesday, November 11, 2014 12:23 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Devices attached to PCIe powered off/on

Ford, Harry L wrote:

I’m trying to recover (reinitialize) a device attached to a PCIe card after it has been power cycled. The framework doesn’t detect this event and so, I don’t get any of the pnpPowerCallbacks called back. It doesn’t call either the EvtDeviceD0Exit or EvtDeviceReleaseHardware callbacks. I can detect the event because I have a watchdog timer running that is looking at a register with a changing value on the device. So, is there a way from the driver to reset this device or does it require an external application?

If you can still read the registers, then the PCIe connection itself never went down. The operating system doesn’t know that anything changed, so there’s nothing for PnP to report. This is just an activity that is local to your driver.

The phrase “reset this device” has no global meaning in PCIExpress. If your hardware has components that can be reset, then only you know how to make that happen, and what will happen when you do.



Tim Roberts, xxxxx@probo.commailto:xxxxx

Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>

EvtDeviceD0Exit
Does this

switch (TargetState) {
case WdfPowerDeviceD1:
case WdfPowerDeviceD2:
case WdfPowerDeviceD3:

break;

Plus some other empty case statements and TraceEvents

EvtDeviceD0Entry

Just has some traceevent statements.

-Harry

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Tuesday, November 11, 2014 1:28 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Devices attached to PCIe powered off/on

Ford, Harry L wrote:
I can’t really still read the register’s, value (it’s a countdown value) and it remains the same between expirations of the watchdog so, I know the device is no longer functioning.

If your device had dropped off of the bus, the address your driver has stored would no longer be valid. Usually, you’ll read back FFFFFFFF. If you’re reading data, then your PCIExpress IP is responding. In that case, whatever problem you have is strictly a problem on your device.

The only way to recover seems to either do a disable/enable from the device manager or use devcon. I’m hoping there’s a way to get the driver to go back through its DriverEntry point programmatically from the driver.

No, there isn’t, but you don’t need that. Disable and enable don’t do ANYTHING to the hardware. That’s just a software operation. What you need to do is do the same things you did in those callbacks. You don’t need to remap the space, but repeat whatever register initialization you did.

So, take the code from your EvtDeviceD0Entry and EvtDeviceD0Exit routines, and put them in separate functions called “PowerDown” and “PowerUp”. Now, in D0Entry you can call PowerUp, and whenever you detect that your watchdog has stopped counting, you can call PowerDown() then PowerUp().

Tim Roberts, xxxxx@probo.commailto:xxxxx

Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx>

I’m running windows 7, I’ll give it a try.

Thanks,

-Harry

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Doron Holan
Sent: Tuesday, November 11, 2014 1:28 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Devices attached to PCIe powered off/on

Try calling WdfDeviceSetFailed(WdfDeviceFailedAttemptRestart). If PCI supports GUID_REENUMERATE_SELF_INTERFACE_STANDARD on the OS you are running, your device will be surprise removed and reenumerated. If not supported, your only recourse is to disable/enable in user mode

d

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Ford, Harry L
Sent: Tuesday, November 11, 2014 1:15 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Devices attached to PCIe powered off/on

I can’t really still read the register’s, value (it’s a countdown value) and it remains the same between expirations of the watchdog so, I know the device is no longer functioning. The only way to recover seems to either do a disable/enable from the device manager or use devcon. I’m hoping there’s a way to get the driver to go back through its DriverEntry point programmatically from the driver.

-Harry

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Tuesday, November 11, 2014 12:23 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Devices attached to PCIe powered off/on

Ford, Harry L wrote:

I’m trying to recover (reinitialize) a device attached to a PCIe card after it has been power cycled. The framework doesn’t detect this event and so, I don’t get any of the pnpPowerCallbacks called back. It doesn’t call either the EvtDeviceD0Exit or EvtDeviceReleaseHardware callbacks. I can detect the event because I have a watchdog timer running that is looking at a register with a changing value on the device. So, is there a way from the driver to reset this device or does it require an external application?

If you can still read the registers, then the PCIe connection itself never went down. The operating system doesn’t know that anything changed, so there’s nothing for PnP to report. This is just an activity that is local to your driver.

The phrase “reset this device” has no global meaning in PCIExpress. If your hardware has components that can be reset, then only you know how to make that happen, and what will happen when you do.



Tim Roberts, xxxxx@probo.commailto:xxxxx

Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx></mailto:xxxxx>

Pci Doesn’t support guid on win7

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Ford, Harry L
Sent: Tuesday, November 11, 2014 1:44 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Devices attached to PCIe powered off/on

I’m running windows 7, I’ll give it a try.

Thanks,

-Harry

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Doron Holan
Sent: Tuesday, November 11, 2014 1:28 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Devices attached to PCIe powered off/on

Try calling WdfDeviceSetFailed(WdfDeviceFailedAttemptRestart). If PCI supports GUID_REENUMERATE_SELF_INTERFACE_STANDARD on the OS you are running, your device will be surprise removed and reenumerated. If not supported, your only recourse is to disable/enable in user mode

d

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Ford, Harry L
Sent: Tuesday, November 11, 2014 1:15 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Devices attached to PCIe powered off/on

I can’t really still read the register’s, value (it’s a countdown value) and it remains the same between expirations of the watchdog so, I know the device is no longer functioning. The only way to recover seems to either do a disable/enable from the device manager or use devcon. I’m hoping there’s a way to get the driver to go back through its DriverEntry point programmatically from the driver.

-Harry

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Tuesday, November 11, 2014 12:23 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Devices attached to PCIe powered off/on

Ford, Harry L wrote:

I’m trying to recover (reinitialize) a device attached to a PCIe card after it has been power cycled. The framework doesn’t detect this event and so, I don’t get any of the pnpPowerCallbacks called back. It doesn’t call either the EvtDeviceD0Exit or EvtDeviceReleaseHardware callbacks. I can detect the event because I have a watchdog timer running that is looking at a register with a changing value on the device. So, is there a way from the driver to reset this device or does it require an external application?

If you can still read the registers, then the PCIe connection itself never went down. The operating system doesn’t know that anything changed, so there’s nothing for PnP to report. This is just an activity that is local to your driver.

The phrase “reset this device” has no global meaning in PCIExpress. If your hardware has components that can be reset, then only you know how to make that happen, and what will happen when you do.



Tim Roberts, xxxxx@probo.commailto:xxxxx

Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx>

Ford, Harry L wrote:

EvtDeviceD0Exit

Does this

switch(TargetState) {

caseWdfPowerDeviceD1:

caseWdfPowerDeviceD2:

caseWdfPowerDeviceD3:

break;

Plus some other empty case statements and TraceEvents

EvtDeviceD0Entry

Just has some traceevent statements.

Well, then, your PrepareHardware and UnprepareHardware. I would hope
you could figure that out. What I’m telling you is this: if the code
that your driver executes during disable and enable fixes the problem,
then you can just run that code yourself when you detect the need for
it. The system isn’t doing any magic on your behalf.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Really? Hmmm… So, the PCI driver doesn’t do anything to reset/reprogram the device as a result of an disable/enable sequence?

I wouldn’t have guessed that. But, then again, I’ve never followed the code in the PCI driver (in ANY version of Windows) all the way through these processes either.

Also, doesn’t this depend on whether the device supports D3-Cold or not?

Peter
OSR
@OSRDrivers

xxxxx@osr.com wrote:

Really? Hmmm… So, the PCI driver doesn’t do anything to reset/reprogram the device as a result of an disable/enable sequence?

No. There is no PCI traffic during disable/enable other than whatever
the function driver does. There’s really nothing to do. There is no
“soft reset” signal on PCI or PCIe.

Also, doesn’t this depend on whether the device supports D3-Cold or not?

Hmm. Let me get out my tap shoes. I haven’t actually played with a
PCIe device that supports D3cold. However, even in this case, there
shouldn’t be anything special about disable/enable. It’s really “going
idle” that triggers the shift into D3, and disable is just another way
to “go idle”.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

On 11/12/2014 9:43 AM, Tim Roberts wrote:

xxxxx@osr.com wrote:
> [quote]
> The system isn’t doing any magic on your behalf.
> [/quote]
>
> Really? Hmmm… So, the PCI driver doesn’t do anything to reset/reprogram the device as a result of an disable/enable sequence?
No. There is no PCI traffic during disable/enable other than whatever
the function driver does. There’s really nothing to do. There is no
“soft reset” signal on PCI or PCIe.
I think I understand your “There is no ‘soft reset’ signal on PCI or
PCIe” statement within the context of the previous two sentences, but on
it’s own I think the wording is less than optimal. In theory, if you
disabled the device, then Windows could reset the device with say a
function level reset to each function or hot reset via the bus driver to
make sure the device is quiesced. To me that’s a soft reset signal. I
think of PERST has a hard reset. There’s a wire for that one. But then
nearly everything defined in the spec is a packet, so maybe FLR and Hot
reset are hard resets if that’s your perspective.

Jeff Glass wrote:

I think I understand your “There is no ‘soft reset’ signal on PCI or
PCIe” statement within the context of the previous two sentences, but on
it’s own I think the wording is less than optimal. In theory, if you
disabled the device, then Windows could reset the device with say a
function level reset to each function or hot reset via the bus driver to
make sure the device is quiesced. To me that’s a soft reset signal.

I don’t know how many devices support FLR. It’s relatively new, and
optional. Nor, I guess, do I know if Windows ever asserts it.

I think of PERST has a hard reset. There’s a wire for that one.

Right, and it’s system-wide.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

On 11/12/2014 10:48 AM, Tim Roberts wrote:

Jeff Glass wrote:
> I think I understand your “There is no ‘soft reset’ signal on PCI or
> PCIe” statement within the context of the previous two sentences, but on
> it’s own I think the wording is less than optimal. In theory, if you
> disabled the device, then Windows could reset the device with say a
> function level reset to each function or hot reset via the bus driver to
> make sure the device is quiesced. To me that’s a soft reset signal.
I don’t know how many devices support FLR. It’s relatively new, and
optional. Nor, I guess, do I know if Windows ever asserts it.
I’ve been supporting it in the firmware for products I work on, but when
I’m working on firmware I don’t pay much attention to Windows. When I’m
doing Windows drivers I’m above the level where this would be
happening. I doubt its ever used. I agree with you when you say that
Windows is going to do anything that would kick a hung device back into
action.

But, but, but…

SOMEbody has to move the “slot” the device is into D3, though, depending on the power plane configuration, right? When I disable my device, I move it to D3. If I’m willing to move to D3-Cold, Vcc (!) is removed from the device, assuming there are no other devices sharing the power plane with my slot.

That feels like it results in a reset to me… am I mistaken about this?

Peter
OSR
@OSRDrivers

xxxxx@osr.com wrote:

But, but, but…

SOMEbody has to move the “slot” the device is into D3, though, depending on the power plane configuration, right? When I disable my device, I move it to D3. If I’m willing to move to D3-Cold, Vcc (!) is removed from the device, assuming there are no other devices sharing the power plane with my slot.

Yes.

That feels like it results in a reset to me… am I mistaken about this?

Well, I think we are quibbling over words here. Yes, it’s clear that
cycling power will reset the device. My point is that the transition
into D3Cold is not associated with disable and re-enable. My device
might already have been in a low-power state, in which case the disable
might do nothing.

So, yes, it’s possible that doing a driver disable and enable does some
board activity. But that board activity is not BECAUSE of the disable
and enable – it’s because of a change in power state, which might or
might not occur during a reload.

Now, if you’ll give me a ladder so I can climb out of the corner I’ve
painted myself into, I’d appreciate it.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

I am SO not trying to be a jerk here or to pedantically parse words… I was legitimately wondering if I was missing something, somehow.

Whew. I feel much better.

Peter
OSR
@OSRDrivers

xxxxx@osr.com wrote:

I am SO not trying to be a jerk here or to pedantically parse words…

I did not intend to imply you were being a jerk. This is simply a case
where one’s interpretation depends on one’s pont of view.

Put another way, it’s possible I was defining my terms narrowly in order
to support my position.

Also, there are few things I like better then pedantically parsing words.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.