PCI config space uninitialized after resuming from S3

On 26/03/2013 14:20, xxxxx@osr.com wrote:

http://bit.ly/UFN2lB

Unfortunately Google doesn’t give what’s probably the simplest option on
the first page - http://7-zip.org/ .


Bruce Cran

Mounting the .iso file isn’t the issue - it’s where to actually find/extract
pci.sys from within the mounted Win7 SP1 checked build .iso.

And then, once I have it extracted, what’s the trick for substituting it in
place of the existing free build pci.sys on a system that has Win7 SP1
installed?

Thanks,

  • Jay

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529827-
xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Tuesday, March 26, 2013 7:20 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] PCI config space uninitialized after resuming from S3

http://bit.ly/UFN2lB

(or use Win8, which mounts ISOs transparently)

Peter
OSR


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Jay Talbott wrote:

Mounting the .iso file isn’t the issue - it’s where to actually find/extract
pci.sys from within the mounted Win7 SP1 checked build .iso.

The 7zip tool can extract files from the install.wim archive in the iso.

And then, once I have it extracted, what’s the trick for substituting it in
place of the existing free build pci.sys on a system that has Win7 SP1
installed?

Trick?
C:\tmp> copy pci.sys c:\windows\system32\drivers
C:\tmp> reboot

Might want to copy the original file to a safe place.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

On 27/03/2013 17:16, Tim Roberts wrote:

Trick?
C:\tmp> copy pci.sys c:\windows\system32\drivers
C:\tmp> reboot

Might want to copy the original file to a safe place.

And if you want to recreate the ISO file, I’d recommend using ImgBurn.


Bruce Cran

Ok, I’ve extracted pci.sys. It was actually located in boot.wim.

As for substituting it on my test system, don’t I need to somehow disable
Windows Resource Protection before simply replacing the file?

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529891-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, March 27, 2013 10:16 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
> Mounting the .iso file isn’t the issue - it’s where to actually
find/extract
> pci.sys from within the mounted Win7 SP1 checked build .iso.

The 7zip tool can extract files from the install.wim archive in the iso.

> And then, once I have it extracted, what’s the trick for substituting it
in
> place of the existing free build pci.sys on a system that has Win7 SP1
> installed?

Trick?
C:\tmp> copy pci.sys c:\windows\system32\drivers
C:\tmp> reboot

Might want to copy the original file to a safe place.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

That amounts to these commands:

cd \windows\system32\drivers
takeown /f pci.sys
icacls pci.sys /grant domain\username:(F)

  • Jake Oshins

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?11?:?53? ?AM
To: Windows System Software Devs Interest List

Ok, I’ve extracted pci.sys. It was actually located in boot.wim.

As for substituting it on my test system, don’t I need to somehow disable
Windows Resource Protection before simply replacing the file?

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529891-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, March 27, 2013 10:16 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
> Mounting the .iso file isn’t the issue - it’s where to actually
find/extract
> pci.sys from within the mounted Win7 SP1 checked build .iso.

The 7zip tool can extract files from the install.wim archive in the iso.

> And then, once I have it extracted, what’s the trick for substituting it
in
> place of the existing free build pci.sys on a system that has Win7 SP1
> installed?

Trick?
C:\tmp> copy pci.sys c:\windows\system32\drivers
C:\tmp> reboot

Might want to copy the original file to a safe place.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

You could probably also change the ImageName to something other than pci.sys (ie pcichk.sys) and forgo the whole ownership problem.

d

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 12:44 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

That amounts to these commands:

cd \windows\system32\drivers
takeown /f pci.sys
icacls pci.sys /grant domain\username:(F)

  • Jake Oshins

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?11?:?53? ?AM
To: Windows System Software Devs Interest List

Ok, I’ve extracted pci.sys. It was actually located in boot.wim.

As for substituting it on my test system, don’t I need to somehow disable
Windows Resource Protection before simply replacing the file?

-----Original Message-----
From: xxxxx@lists.osr.commailto:xxxxx [mailto:bounce-529891-
> xxxxx@lists.osr.commailto:xxxxx] On Behalf Of Tim Roberts
> Sent: Wednesday, March 27, 2013 10:16 AM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3
>
> Jay Talbott wrote:
> > Mounting the .iso file isn’t the issue - it’s where to actually
find/extract
> > pci.sys from within the mounted Win7 SP1 checked build .iso.
>
> The 7zip tool can extract files from the install.wim archive in the iso.
>
>
> > And then, once I have it extracted, what’s the trick for substituting it
in
> > place of the existing free build pci.sys on a system that has Win7 SP1
> > installed?
>
> Trick?
> C:\tmp> copy pci.sys c:\windows\system32\drivers
> C:\tmp> reboot
>
> Might want to copy the original file to a safe place.
>
> –
> Tim Roberts, xxxxx@probo.commailto:xxxxx
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx></mailto:xxxxx>

Thanks Jake. That was the ticket I needed.

So, according to the debug output of the checked version of PCI.sys, it
looks like it?s not restoring the PCI config space of the devices when
resuming from sleep. I see where it restores the PCI config space of the
upstream and downstream ports of the PCIe switch to which the devices are
connected, and I even see an output where it?s setting the power state of
the devices:

PCI: PDO(b=0x3, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

PCI: PDO(b=0x4, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

But there are no writes (or reads for that matter) to either device?s PCI
config space like I see for all of the other devices in the system.
Obviously there is something going on here that is convincing the PCI bus
driver that it doesn?t need to restore the PCI config space for these
devices.

Which brings me back to my prior question:

What are all of the possible situations that could cause the PCI bus driver
to NOT restore the PCI config data for a device when resuming from sleep?

What?s even more confusing is that the problem is not reproducible on all
systems. So what would be causing the PCI bus driver to NOT restore the PCI
config data on some systems, but not others, with the same devices
installed?

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 12:44 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

That amounts to these commands:

cd \windows\system32\drivers

takeown /f pci.sys

icacls pci.sys /grant domain\username:(F)

  • Jake Oshins

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?11?:?53? ?AM
To: Windows System Software Devs Interest List

Ok, I’ve extracted pci.sys. It was actually located in boot.wim.

As for substituting it on my test system, don’t I need to somehow disable
Windows Resource Protection before simply replacing the file?

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529891-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, March 27, 2013 10:16 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
> Mounting the .iso file isn’t the issue - it’s where to actually
find/extract
> pci.sys from within the mounted Win7 SP1 checked build .iso.

The 7zip tool can extract files from the install.wim archive in the iso.

> And then, once I have it extracted, what’s the trick for substituting it
in
> place of the existing free build pci.sys on a system that has Win7 SP1
> installed?

Trick?
C:\tmp> copy pci.sys c:\windows\system32\drivers
C:\tmp> reboot

Might want to copy the original file to a safe place.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

As I?ve said before, the only thing that will cause the PCI driver to abandon this restoration is if it thinks that the device that was present before it went to D3 is no longer present. This usually amounts to a failure to read the exact same vendor ID, device ID and subsystem IDs from the device when moving back to D0.

In many cases, devices in slots can be changed while the system is hibernated, or even while the device is just powered off. The code doesn?t want to attempt to restore the state of a device that is no longer present.

  • Jake Oshins
    Windows Kernel Team

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Wednesday, March 27, 2013 2:54 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

Thanks Jake. That was the ticket I needed.

So, according to the debug output of the checked version of PCI.sys, it looks like it?s not restoring the PCI config space of the devices when resuming from sleep. I see where it restores the PCI config space of the upstream and downstream ports of the PCIe switch to which the devices are connected, and I even see an output where it?s setting the power state of the devices:

PCI: PDO(b=0x3, d=0x0, f=0x0)<-SET_POWERPCI: 0x0
PCI: PDO(b=0x4, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

But there are no writes (or reads for that matter) to either device?s PCI config space like I see for all of the other devices in the system. Obviously there is something going on here that is convincing the PCI bus driver that it doesn?t need to restore the PCI config space for these devices.

Which brings me back to my prior question:
What are all of the possible situations that could cause the PCI bus driver to NOT restore the PCI config data for a device when resuming from sleep?

What?s even more confusing is that the problem is not reproducible on all systems. So what would be causing the PCI bus driver to NOT restore the PCI config data on some systems, but not others, with the same devices installed?

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 12:44 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

That amounts to these commands:

cd \windows\system32\drivers
takeown /f pci.sys
icacls pci.sys /grant domain\username:(F)

- Jake Oshins

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?11?:?53? ?AM
To: Windows System Software Devs Interest List

Ok, I’ve extracted pci.sys. It was actually located in boot.wim.

As for substituting it on my test system, don’t I need to somehow disable
Windows Resource Protection before simply replacing the file?

> -----Original Message-----
> From: xxxxx@lists.osr.commailto:xxxxx [mailto:bounce-529891-
> xxxxx@lists.osr.commailto:xxxxx] On Behalf Of Tim Roberts
> Sent: Wednesday, March 27, 2013 10:16 AM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3
>
> Jay Talbott wrote:
> > Mounting the .iso file isn’t the issue - it’s where to actually
find/extract
> > pci.sys from within the mounted Win7 SP1 checked build .iso.
>
> The 7zip tool can extract files from the install.wim archive in the iso.
>
>
> > And then, once I have it extracted, what’s the trick for substituting it
in
> > place of the existing free build pci.sys on a system that has Win7 SP1
> > installed?
>
> Trick?
> C:\tmp> copy pci.sys c:\windows\system32\drivers
> C:\tmp> reboot
>
> Might want to copy the original file to a safe place.
>
> –
> Tim Roberts, xxxxx@probo.commailto:xxxxx
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx>

Well, per the log of debug output I was analyzing earlier, the PCI bus
driver is definitely NOT restoring the PCI config data. However, it?s also
not treating the devices as removed, as it doesn?t initiate a device removal
or surprise removal in the driver. So that contradicts what you are telling
me.

One thing about the device is that it doesn?t support ASPM (yes, they know
it?s out of spec, and they had to get a WLK/HCK contingency filter for it).
The ASPM bits in the PCIe Link Capabilities register are both 0, when, they
should be either 01 or 11 to indicate the level of ASPM support.
Furthermore, the No_Soft_Reset bit is set in the Power Management
Control/Status register. Could the combination of those two things make the
PCI bus driver think the device never goes from D3hot to D3cold, and thus it
doesn?t restore the PCI config data when resuming from sleep (because it
doesn?t think it needs to)?

I?m reaching for an answer here that fits the observed data?

  • Jay

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 5:15 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

As I?ve said before, the only thing that will cause the PCI driver to
abandon this restoration is if it thinks that the device that was present
before it went to D3 is no longer present. This usually amounts to a
failure to read the exact same vendor ID, device ID and subsystem IDs from
the device when moving back to D0.

In many cases, devices in slots can be changed while the system is
hibernated, or even while the device is just powered off. The code doesn?t
want to attempt to restore the state of a device that is no longer present.

  • Jake Oshins

Windows Kernel Team

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Wednesday, March 27, 2013 2:54 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

Thanks Jake. That was the ticket I needed.

So, according to the debug output of the checked version of PCI.sys, it
looks like it?s not restoring the PCI config space of the devices when
resuming from sleep. I see where it restores the PCI config space of the
upstream and downstream ports of the PCIe switch to which the devices are
connected, and I even see an output where it?s setting the power state of
the devices:

PCI: PDO(b=0x3, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

PCI: PDO(b=0x4, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

But there are no writes (or reads for that matter) to either device?s PCI
config space like I see for all of the other devices in the system.
Obviously there is something going on here that is convincing the PCI bus
driver that it doesn?t need to restore the PCI config space for these
devices.

Which brings me back to my prior question:

What are all of the possible situations that could cause the PCI bus driver
to NOT restore the PCI config data for a device when resuming from sleep?

What?s even more confusing is that the problem is not reproducible on all
systems. So what would be causing the PCI bus driver to NOT restore the PCI
config data on some systems, but not others, with the same devices
installed?

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 12:44 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

That amounts to these commands:

cd \windows\system32\drivers

takeown /f pci.sys

icacls pci.sys /grant domain\username:(F)

  • Jake Oshins

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?11?:?53? ?AM
To: Windows System Software Devs Interest List

Ok, I’ve extracted pci.sys. It was actually located in boot.wim.

As for substituting it on my test system, don’t I need to somehow disable
Windows Resource Protection before simply replacing the file?

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529891-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, March 27, 2013 10:16 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
> Mounting the .iso file isn’t the issue - it’s where to actually
find/extract
> pci.sys from within the mounted Win7 SP1 checked build .iso.

The 7zip tool can extract files from the install.wim archive in the iso.

> And then, once I have it extracted, what’s the trick for substituting it
in
> place of the existing free build pci.sys on a system that has Win7 SP1
> installed?

Trick?
C:\tmp> copy pci.sys c:\windows\system32\drivers
C:\tmp> reboot

Might want to copy the original file to a safe place.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Those things shouldn’t matter. PCI will also skip restoring the device if it thinks that it never left D0. Does the log show that the device was actually moved to D3?

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?5?:?32? ?PM
To: Windows System Software Devs Interest List

Well, per the log of debug output I was analyzing earlier, the PCI bus driver is definitely NOT restoring the PCI config data. However, it?s also not treating the devices as removed, as it doesn?t initiate a device removal or surprise removal in the driver. So that contradicts what you are telling me.

One thing about the device is that it doesn?t support ASPM (yes, they know it?s out of spec, and they had to get a WLK/HCK contingency filter for it). The ASPM bits in the PCIe Link Capabilities register are both 0, when, they should be either 01 or 11 to indicate the level of ASPM support. Furthermore, the No_Soft_Reset bit is set in the Power Management Control/Status register. Could the combination of those two things make the PCI bus driver think the device never goes from D3hot to D3cold, and thus it doesn?t restore the PCI config data when resuming from sleep (because it doesn?t think it needs to)?

I?m reaching for an answer here that fits the observed data?

  • Jay

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 5:15 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

As I?ve said before, the only thing that will cause the PCI driver to abandon this restoration is if it thinks that the device that was present before it went to D3 is no longer present. This usually amounts to a failure to read the exact same vendor ID, device ID and subsystem IDs from the device when moving back to D0.

In many cases, devices in slots can be changed while the system is hibernated, or even while the device is just powered off. The code doesn?t want to attempt to restore the state of a device that is no longer present.

  • Jake Oshins
    Windows Kernel Team

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Wednesday, March 27, 2013 2:54 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

Thanks Jake. That was the ticket I needed.

So, according to the debug output of the checked version of PCI.sys, it looks like it?s not restoring the PCI config space of the devices when resuming from sleep. I see where it restores the PCI config space of the upstream and downstream ports of the PCIe switch to which the devices are connected, and I even see an output where it?s setting the power state of the devices:

PCI: PDO(b=0x3, d=0x0, f=0x0)<-SET_POWERPCI: 0x0
PCI: PDO(b=0x4, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

But there are no writes (or reads for that matter) to either device?s PCI config space like I see for all of the other devices in the system. Obviously there is something going on here that is convincing the PCI bus driver that it doesn?t need to restore the PCI config space for these devices.

Which brings me back to my prior question:
What are all of the possible situations that could cause the PCI bus driver to NOT restore the PCI config data for a device when resuming from sleep?

What?s even more confusing is that the problem is not reproducible on all systems. So what would be causing the PCI bus driver to NOT restore the PCI config data on some systems, but not others, with the same devices installed?

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 12:44 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

That amounts to these commands:

cd \windows\system32\drivers
takeown /f pci.sys
icacls pci.sys /grant domain\username:(F)

- Jake Oshins

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?11?:?53? ?AM
To: Windows System Software Devs Interest List

Ok, I’ve extracted pci.sys. It was actually located in boot.wim.

As for substituting it on my test system, don’t I need to somehow disable
Windows Resource Protection before simply replacing the file?

> -----Original Message-----
> From: xxxxx@lists.osr.commailto:xxxxx [mailto:bounce-529891-
> xxxxx@lists.osr.commailto:xxxxx] On Behalf Of Tim Roberts
> Sent: Wednesday, March 27, 2013 10:16 AM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3
>
> Jay Talbott wrote:
> > Mounting the .iso file isn’t the issue - it’s where to actually
find/extract
> > pci.sys from within the mounted Win7 SP1 checked build .iso.
>
> The 7zip tool can extract files from the install.wim archive in the iso.
>
>
> > And then, once I have it extracted, what’s the trick for substituting it
in
> > place of the existing free build pci.sys on a system that has Win7 SP1
> > installed?
>
> Trick?
> C:\tmp> copy pci.sys c:\windows\system32\drivers
> C:\tmp> reboot
>
> Might want to copy the original file to a safe place.
>
> –
> Tim Roberts, xxxxx@probo.commailto:xxxxx
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx>

The debug log includes lines like this:

PCI: PDO(b=0x3, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

PCI: PDO(b=0x4, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

(those are the bus/device/function numbers for the two devices in my test
system)

However, the debug output in the log does not indicate what actual device
power state the device is being set to wherever those statements appear.

But, when going to sleep the driver?s EvtDeviceD0Exit definitely gets called
with a TargetState of WdfPowerDeviceD3, and similarly on resume the drivers?
EvtDeviceD0Entry gets called with a PreviousState of WdfPowerDeviceD3, so
that confirms that it is definitely going in and out of D3. However, since
there?s no ASPM support, I?m assuming that the PCIe link never goes out of
L0 (the debug log doesn?t provide any info about PCIe link PM state
transitions).

Right now my workaround is that in those event callbacks the driver does its
own save and restore of the PCI config space when going into and out of D3,
since the PCI driver is not restoring it on behalf of the device. But the
bigger issue is why the PCI driver isn?t doing the restore like it does for
all of the other devices in the system. There obviously must be something
that?s causing the PCI bus driver to skip the restoration of the PCI config
space for this device?

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 5:40 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Those things shouldn’t matter. PCI will also skip restoring the device if
it thinks that it never left D0. Does the log show that the device was
actually moved to D3?

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?5?:?32? ?PM
To: Windows System Software Devs Interest List

Well, per the log of debug output I was analyzing earlier, the PCI bus
driver is definitely NOT restoring the PCI config data. However, it?s also
not treating the devices as removed, as it doesn?t initiate a device removal
or surprise removal in the driver. So that contradicts what you are telling
me.

One thing about the device is that it doesn?t support ASPM (yes, they know
it?s out of spec, and they had to get a WLK/HCK contingency filter for it).
The ASPM bits in the PCIe Link Capabilities register are both 0, when, they
should be either 01 or 11 to indicate the level of ASPM support.
Furthermore, the No_Soft_Reset bit is set in the Power Management
Control/Status register. Could the combination of those two things make the
PCI bus driver think the device never goes from D3hot to D3cold, and thus it
doesn?t restore the PCI config data when resuming from sleep (because it
doesn?t think it needs to)?

I?m reaching for an answer here that fits the observed data?

  • Jay

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 5:15 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

As I?ve said before, the only thing that will cause the PCI driver to
abandon this restoration is if it thinks that the device that was present
before it went to D3 is no longer present. This usually amounts to a
failure to read the exact same vendor ID, device ID and subsystem IDs from
the device when moving back to D0.

In many cases, devices in slots can be changed while the system is
hibernated, or even while the device is just powered off. The code doesn?t
want to attempt to restore the state of a device that is no longer present.

  • Jake Oshins

Windows Kernel Team

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Wednesday, March 27, 2013 2:54 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

Thanks Jake. That was the ticket I needed.

So, according to the debug output of the checked version of PCI.sys, it
looks like it?s not restoring the PCI config space of the devices when
resuming from sleep. I see where it restores the PCI config space of the
upstream and downstream ports of the PCIe switch to which the devices are
connected, and I even see an output where it?s setting the power state of
the devices:

PCI: PDO(b=0x3, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

PCI: PDO(b=0x4, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

But there are no writes (or reads for that matter) to either device?s PCI
config space like I see for all of the other devices in the system.
Obviously there is something going on here that is convincing the PCI bus
driver that it doesn?t need to restore the PCI config space for these
devices.

Which brings me back to my prior question:

What are all of the possible situations that could cause the PCI bus driver
to NOT restore the PCI config data for a device when resuming from sleep?

What?s even more confusing is that the problem is not reproducible on all
systems. So what would be causing the PCI bus driver to NOT restore the PCI
config data on some systems, but not others, with the same devices
installed?

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 12:44 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

That amounts to these commands:

cd \windows\system32\drivers

takeown /f pci.sys

icacls pci.sys /grant domain\username:(F)

  • Jake Oshins

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?11?:?53? ?AM
To: Windows System Software Devs Interest List

Ok, I’ve extracted pci.sys. It was actually located in boot.wim.

As for substituting it on my test system, don’t I need to somehow disable
Windows Resource Protection before simply replacing the file?

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529891-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, March 27, 2013 10:16 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
> Mounting the .iso file isn’t the issue - it’s where to actually
find/extract
> pci.sys from within the mounted Win7 SP1 checked build .iso.

The 7zip tool can extract files from the install.wim archive in the iso.

> And then, once I have it extracted, what’s the trick for substituting it
in
> place of the existing free build pci.sys on a system that has Win7 SP1
> installed?

Trick?
C:\tmp> copy pci.sys c:\windows\system32\drivers
C:\tmp> reboot

Might want to copy the original file to a safe place.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Well, no. Not really. It confirms that the DRIVER is being told that the device MIGHT go in and out of D3.

Consider the case where a power plane is shared among multiple PCIe slots. Each device attached to that power plane would have to be in D3 before the slots attached to that power plane – and hence the devices in those slots – could have main power removed and thereby physically transition the devices to D3.

Peter
OSR

Well, in this case it’s pretty clear the board is actually getting powered
down, since the whole issue is that it comes back up out of sleep with the
PCI config space uninitialized. Also, it’s clear that power is being removed
from the board since the fans shut off and all the LEDs go dark while the
system is sleeping.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529969-
xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Thursday, March 28, 2013 6:02 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] PCI config space uninitialized after resuming from S3

Well, no. Not really. It confirms that the DRIVER is being told that the
device
MIGHT go in and out of D3.

Consider the case where a power plane is shared among multiple PCIe slots.
Each device attached to that power plane would have to be in D3 before the
slots attached to that power plane – and hence the devices in those slots

could have main power removed and thereby physically transition the
devices to D3.

Peter
OSR


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Ah, OK… THAT makes it clear.

Also, you note that the system was going into Sx. Sorry… THAT’s definitive that the board is being powered-off as well (well… practically speaking). I had lost that point (my bad) in the history of the thread. It’s not D0->Dx->D0 during device idle we’re discussing, it’s D0->Dx->D0 during S0->Sx that we’re discussing.

Sorry to have lost the plot there…

Peter
OSR

I simply don?t know. Sorry.

  • Jake

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Wednesday, March 27, 2013 9:26 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

The debug log includes lines like this:

PCI: PDO(b=0x3, d=0x0, f=0x0)<-SET_POWERPCI: 0x0
PCI: PDO(b=0x4, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

(those are the bus/device/function numbers for the two devices in my test system)

However, the debug output in the log does not indicate what actual device power state the device is being set to wherever those statements appear.

But, when going to sleep the driver?s EvtDeviceD0Exit definitely gets called with a TargetState of WdfPowerDeviceD3, and similarly on resume the drivers? EvtDeviceD0Entry gets called with a PreviousState of WdfPowerDeviceD3, so that confirms that it is definitely going in and out of D3. However, since there?s no ASPM support, I?m assuming that the PCIe link never goes out of L0 (the debug log doesn?t provide any info about PCIe link PM state transitions).

Right now my workaround is that in those event callbacks the driver does its own save and restore of the PCI config space when going into and out of D3, since the PCI driver is not restoring it on behalf of the device. But the bigger issue is why the PCI driver isn?t doing the restore like it does for all of the other devices in the system. There obviously must be something that?s causing the PCI bus driver to skip the restoration of the PCI config space for this device?

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 5:40 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Those things shouldn’t matter. PCI will also skip restoring the device if it thinks that it never left D0. Does the log show that the device was actually moved to D3?

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?5?:?32? ?PM
To: Windows System Software Devs Interest List

Well, per the log of debug output I was analyzing earlier, the PCI bus driver is definitely NOT restoring the PCI config data. However, it?s also not treating the devices as removed, as it doesn?t initiate a device removal or surprise removal in the driver. So that contradicts what you are telling me.

One thing about the device is that it doesn?t support ASPM (yes, they know it?s out of spec, and they had to get a WLK/HCK contingency filter for it). The ASPM bits in the PCIe Link Capabilities register are both 0, when, they should be either 01 or 11 to indicate the level of ASPM support. Furthermore, the No_Soft_Reset bit is set in the Power Management Control/Status register. Could the combination of those two things make the PCI bus driver think the device never goes from D3hot to D3cold, and thus it doesn?t restore the PCI config data when resuming from sleep (because it doesn?t think it needs to)?

I?m reaching for an answer here that fits the observed data?

- Jay

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 5:15 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

As I?ve said before, the only thing that will cause the PCI driver to abandon this restoration is if it thinks that the device that was present before it went to D3 is no longer present. This usually amounts to a failure to read the exact same vendor ID, device ID and subsystem IDs from the device when moving back to D0.

In many cases, devices in slots can be changed while the system is hibernated, or even while the device is just powered off. The code doesn?t want to attempt to restore the state of a device that is no longer present.

- Jake Oshins
Windows Kernel Team

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Wednesday, March 27, 2013 2:54 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

Thanks Jake. That was the ticket I needed.

So, according to the debug output of the checked version of PCI.sys, it looks like it?s not restoring the PCI config space of the devices when resuming from sleep. I see where it restores the PCI config space of the upstream and downstream ports of the PCIe switch to which the devices are connected, and I even see an output where it?s setting the power state of the devices:

PCI: PDO(b=0x3, d=0x0, f=0x0)<-SET_POWERPCI: 0x0
PCI: PDO(b=0x4, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

But there are no writes (or reads for that matter) to either device?s PCI config space like I see for all of the other devices in the system. Obviously there is something going on here that is convincing the PCI bus driver that it doesn?t need to restore the PCI config space for these devices.

Which brings me back to my prior question:
What are all of the possible situations that could cause the PCI bus driver to NOT restore the PCI config data for a device when resuming from sleep?

What?s even more confusing is that the problem is not reproducible on all systems. So what would be causing the PCI bus driver to NOT restore the PCI config data on some systems, but not others, with the same devices installed?

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 12:44 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

That amounts to these commands:

cd \windows\system32\drivers
takeown /f pci.sys
icacls pci.sys /grant domain\username:(F)

- Jake Oshins

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?11?:?53? ?AM
To: Windows System Software Devs Interest List

Ok, I’ve extracted pci.sys. It was actually located in boot.wim.

As for substituting it on my test system, don’t I need to somehow disable
Windows Resource Protection before simply replacing the file?

> -----Original Message-----
> From: xxxxx@lists.osr.commailto:xxxxx [mailto:bounce-529891-
> xxxxx@lists.osr.commailto:xxxxx] On Behalf Of Tim Roberts
> Sent: Wednesday, March 27, 2013 10:16 AM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3
>
> Jay Talbott wrote:
> > Mounting the .iso file isn’t the issue - it’s where to actually
find/extract
> > pci.sys from within the mounted Win7 SP1 checked build .iso.
>
> The 7zip tool can extract files from the install.wim archive in the iso.
>
>
> > And then, once I have it extracted, what’s the trick for substituting it
in
> > place of the existing free build pci.sys on a system that has Win7 SP1
> > installed?
>
> Trick?
> C:\tmp> copy pci.sys c:\windows\system32\drivers
> C:\tmp> reboot
>
> Might want to copy the original file to a safe place.
>
> –
> Tim Roberts, xxxxx@probo.commailto:xxxxx
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx>

Anybody else have any ideas before I go eat another one of my MSDN support
incidents?

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Thursday, March 28, 2013 10:17 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

I simply don?t know. Sorry.

  • Jake

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Wednesday, March 27, 2013 9:26 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

The debug log includes lines like this:

PCI: PDO(b=0x3, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

PCI: PDO(b=0x4, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

(those are the bus/device/function numbers for the two devices in my test
system)

However, the debug output in the log does not indicate what actual device
power state the device is being set to wherever those statements appear.

But, when going to sleep the driver?s EvtDeviceD0Exit definitely gets called
with a TargetState of WdfPowerDeviceD3, and similarly on resume the drivers?
EvtDeviceD0Entry gets called with a PreviousState of WdfPowerDeviceD3, so
that confirms that it is definitely going in and out of D3. However, since
there?s no ASPM support, I?m assuming that the PCIe link never goes out of
L0 (the debug log doesn?t provide any info about PCIe link PM state
transitions).

Right now my workaround is that in those event callbacks the driver does its
own save and restore of the PCI config space when going into and out of D3,
since the PCI driver is not restoring it on behalf of the device. But the
bigger issue is why the PCI driver isn?t doing the restore like it does for
all of the other devices in the system. There obviously must be something
that?s causing the PCI bus driver to skip the restoration of the PCI config
space for this device?

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 5:40 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Those things shouldn’t matter. PCI will also skip restoring the device if
it thinks that it never left D0. Does the log show that the device was
actually moved to D3?

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?5?:?32? ?PM
To: Windows System Software Devs Interest List

Well, per the log of debug output I was analyzing earlier, the PCI bus
driver is definitely NOT restoring the PCI config data. However, it?s also
not treating the devices as removed, as it doesn?t initiate a device removal
or surprise removal in the driver. So that contradicts what you are telling
me.

One thing about the device is that it doesn?t support ASPM (yes, they know
it?s out of spec, and they had to get a WLK/HCK contingency filter for it).
The ASPM bits in the PCIe Link Capabilities register are both 0, when, they
should be either 01 or 11 to indicate the level of ASPM support.
Furthermore, the No_Soft_Reset bit is set in the Power Management
Control/Status register. Could the combination of those two things make the
PCI bus driver think the device never goes from D3hot to D3cold, and thus it
doesn?t restore the PCI config data when resuming from sleep (because it
doesn?t think it needs to)?

I?m reaching for an answer here that fits the observed data?

  • Jay

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 5:15 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

As I?ve said before, the only thing that will cause the PCI driver to
abandon this restoration is if it thinks that the device that was present
before it went to D3 is no longer present. This usually amounts to a
failure to read the exact same vendor ID, device ID and subsystem IDs from
the device when moving back to D0.

In many cases, devices in slots can be changed while the system is
hibernated, or even while the device is just powered off. The code doesn?t
want to attempt to restore the state of a device that is no longer present.

  • Jake Oshins

Windows Kernel Team

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Wednesday, March 27, 2013 2:54 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

Thanks Jake. That was the ticket I needed.

So, according to the debug output of the checked version of PCI.sys, it
looks like it?s not restoring the PCI config space of the devices when
resuming from sleep. I see where it restores the PCI config space of the
upstream and downstream ports of the PCIe switch to which the devices are
connected, and I even see an output where it?s setting the power state of
the devices:

PCI: PDO(b=0x3, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

PCI: PDO(b=0x4, d=0x0, f=0x0)<-SET_POWERPCI: 0x0

But there are no writes (or reads for that matter) to either device?s PCI
config space like I see for all of the other devices in the system.
Obviously there is something going on here that is convincing the PCI bus
driver that it doesn?t need to restore the PCI config space for these
devices.

Which brings me back to my prior question:

What are all of the possible situations that could cause the PCI bus driver
to NOT restore the PCI config data for a device when resuming from sleep?

What?s even more confusing is that the problem is not reproducible on all
systems. So what would be causing the PCI bus driver to NOT restore the PCI
config data on some systems, but not others, with the same devices
installed?

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 27, 2013 12:44 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

That amounts to these commands:

cd \windows\system32\drivers

takeown /f pci.sys

icacls pci.sys /grant domain\username:(F)

  • Jake Oshins

From: Jay Talbott
Sent: ?Wednesday?, ?March? ?27?, ?2013 ?11?:?53? ?AM
To: Windows System Software Devs Interest List

Ok, I’ve extracted pci.sys. It was actually located in boot.wim.

As for substituting it on my test system, don’t I need to somehow disable
Windows Resource Protection before simply replacing the file?

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529891-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Wednesday, March 27, 2013 10:16 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
> Mounting the .iso file isn’t the issue - it’s where to actually
find/extract
> pci.sys from within the mounted Win7 SP1 checked build .iso.

The 7zip tool can extract files from the install.wim archive in the iso.

> And then, once I have it extracted, what’s the trick for substituting it
in
> place of the existing free build pci.sys on a system that has Win7 SP1
> installed?

Trick?
C:\tmp> copy pci.sys c:\windows\system32\drivers
C:\tmp> reboot

Might want to copy the original file to a safe place.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Yes, the plot is that when resuming from sleeping the system, the device’s
PCI config space is not getting restored by the PCI bus driver, as verified
by examining the debug output of the checked build of pci.sys. Instead, the
device comes back out of sleep with its PCI config space uninitialized, and
then any attempt by the driver to interact with the device has bad results.

The question is, why is it not getting restored for this device, when it
restores it for all of the other devices in the system? What about the
device is triggering the PCI bus driver to skip the PCI config restoration?

And, to further complicate things, why does this only seem to be a problem
on some systems, but not others? My client has a system that could reproduce
the issue, and a BIOS update made it go away. They’ve also had systems where
they could reproduce the problem in Vista, but not in Windows 7 or Windows
8, whereas I can reproduce it on my test system in all three versions of
Windows 100% of the time. Oh, and it can vary depending on how long the
system sleeps too, although I think that variability ultimately falls under
needing to sleep long enough for all of the power supply capacitors and such
to discharge so that the device is truly in powered off long enough that the
previous PCI config data is lost.

Again, I implemented a workaround in the driver where the driver takes care
of saving/restoring the PCI config data itself, but that only masks the
issue and doesn’t address the underlying problem which we are trying to
understand so my client can fix it in a future revision of their hardware.

  • Jay

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529977-
xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Thursday, March 28, 2013 8:15 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] PCI config space uninitialized after resuming from S3

Ah, OK… THAT makes it clear.

Also, you note that the system was going into Sx. Sorry… THAT’s
definitive
that the board is being powered-off as well (well… practically
speaking). I
had lost that point (my bad) in the history of the thread. It’s not
D0->Dx->D0
during device idle we’re discussing, it’s D0->Dx->D0 during S0->Sx that
we’re
discussing.

Sorry to have lost the plot there…

Peter
OSR


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Jay Talbott wrote:

Anybody else have any ideas before I go eat another one of my MSDN
support incidents?

If (A) you know your hardware is not spec compliant, and (B) you have a
hacky but acceptable workaround, then why are you spending any more time
and money on this? Microsoft is not going to invest very much time
chasing down an issue on non-compliant hardware. Put it away and enter
a bug report in your internal database so that the next round of the
hardware gets fixed.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

The issue is knowing if the spec non-compliance is the root cause of this particular issue, or if there’s something else going on that will need to be addressed in the next revision of the silicon. They don’t want to respin the silicon to fix one thing and find out that this particular problem still exists.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529986-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Thursday, March 28, 2013 10:56 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
>
> Anybody else have any ideas before I go eat another one of my MSDN
> support incidents?
>

If (A) you know your hardware is not spec compliant, and (B) you have a
hacky but acceptable workaround, then why are you spending any more
time
and money on this? Microsoft is not going to invest very much time
chasing down an issue on non-compliant hardware. Put it away and enter
a bug report in your internal database so that the next round of the
hardware gets fixed.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer