PCI config space uninitialized after resuming from S3

NTDEV
1

A client of mine has an issue with their PCIe device where after waking up
the system from an S3 sleep state, their device’s PCI config space is
uninitialized, resulting in bad things happening when the driver tries to
resume any interaction with the device. Is a PCI device supposed to maintain
its PCI config space data while in D3, or does the PCI config space get
restored by Windows during resume from sleep?

If it’s the former, then it looks like it’s a hardware problem. But if it’s
the latter, what would be causing Windows not to be restoring the PCI config
space for my client’s device, when the rest of the PCIe devices in the
system are getting properly reinitialized?

Although I know that generally a driver should only READ its device’s PCI
config data, would it be a reasonable solution to this problem to have the
driver’s EvtDeviceD0Exit save the PCI config space when going into D3, and
then have EvtDeviceD0Entry restore it when waking back up from D3? Or is
this as much of a really bad idea as I think it is?

  • Jay
2

The first thing that you need to understand is that EvtDeviceD0Entry is called when the PCI driver completes the device’s D0 IRP, allowing your driver (or in this case, WDF on your behalf) to know that the device is actually ready to be used.

The PCI driver does a full configuration space restoration just before completing that D0 IRP. The only thing that can stop it from completing that restoration is a hardware failure at the device. The most common type of failure involves not responding to the D0 transition within the 100ms timeout mandated by the PCI spec.

Your strategy is unlikely to be successful, simply because what you’re suggested has just failed once.

  • Jake Oshins
    (occasional PCI guy)
    Windows Kernel Team

From: xxxxx@lists.osr.com on behalf of Jay Talbott
Sent: Wednesday, March 13, 2013 8:37 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] PCI config space uninitialized after resuming from S3

A client of mine has an issue with their PCIe device where after waking up the system from an S3 sleep state, their device’s PCI config space is uninitialized, resulting in bad things happening when the driver tries to resume any interaction with the device. Is a PCI device supposed to maintain its PCI config space data while in D3, or does the PCI config space get restored by Windows during resume from sleep?

If it’s the former, then it looks like it’s a hardware problem. But if it’s the latter, what would be causing Windows not to be restoring the PCI config space for my client’s device, when the rest of the PCIe devices in the system are getting properly reinitialized?

Although I know that generally a driver should only READ its device’s PCI config data, would it be a reasonable solution to this problem to have the driver’s EvtDeviceD0Exit save the PCI config space when going into D3, and then have EvtDeviceD0Entry restore it when waking back up from D3? Or is this as much of a really bad idea as I think it is?

  • Jay

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

3

HI Jake,

Is there anything that could cause the PCI driver to not restore the PCI
config space on a particular device? For example, if there is some sort of a
bug in the device’s PCI PM capability info? Or is restoration of the PCI
config space applied universally?

Thanks,

  • Jay

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 13, 2013 10:36 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

The first thing that you need to understand is that EvtDeviceD0Entry is
called when the PCI driver completes the device’s D0 IRP, allowing your
driver (or in this case, WDF on your behalf) to know that the device is
actually ready to be used.

The PCI driver does a full configuration space restoration just before
completing that D0 IRP. The only thing that can stop it from completing
that restoration is a hardware failure at the device. The most common type
of failure involves not responding to the D0 transition within the 100ms
timeout mandated by the PCI spec.

Your strategy is unlikely to be successful, simply because what you’re
suggested has just failed once.

  • Jake Oshins

(occasional PCI guy)

Windows Kernel Team

From: xxxxx@lists.osr.com on behalf of Jay Talbott
Sent: Wednesday, March 13, 2013 8:37 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] PCI config space uninitialized after resuming from S3

A client of mine has an issue with their PCIe device where after waking up
the system from an S3 sleep state, their device’s PCI config space is
uninitialized, resulting in bad things happening when the driver tries to
resume any interaction with the device. Is a PCI device supposed to maintain
its PCI config space data while in D3, or does the PCI config space get
restored by Windows during resume from sleep?

If it’s the former, then it looks like it’s a hardware problem. But if it’s
the latter, what would be causing Windows not to be restoring the PCI config
space for my client’s device, when the rest of the PCIe devices in the
system are getting properly reinitialized?

Although I know that generally a driver should only READ its device’s PCI
config data, would it be a reasonable solution to this problem to have the
driver’s EvtDeviceD0Exit save the PCI config space when going into D3, and
then have EvtDeviceD0Entry restore it when waking back up from D3? Or is
this as much of a really bad idea as I think it is?

  • Jay

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

4

Jay Talbott wrote:

A client of mine has an issue with their PCIe device where after
waking up the system from an S3 sleep state, their device’s PCI config
space is uninitialized, resulting in bad things happening when the
driver tries to resume any interaction with the device. Is a PCI
device supposed to maintain its PCI config space data while in D3, or
does the PCI config space get restored by Windows during resume from
sleep?

Is their driver interfering with power management in some way? We had
someone come around with a similar issue several months ago. In their
case, they had a WDM driver that was arbitrarily completing all power
IRPs. The PCI bus driver only restores the PCI configuration space in
response to a device power transition IRP coming through the function
driver. That is, your driver gets a request to transition to S0. In
response, your driver is supposed to generate a request to transition
the device to D0. That’s what triggers the restoration of config space
(or so it appears). If you don’t generate that D0 IRP, it won’t get
restored.

Is this actually a KMDF driver, as implied? KMDF should be handle all
of that for them, unless they are interfering.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

5

It’s applied universally. It is possible to confuse the PCI driver if you mis-use the WDM power IRPs. But you seem to be using KMDF, which will do things properly, so that’s not your problem.

  • Jake

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Thursday, March 14, 2013 9:16 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

HI Jake,

Is there anything that could cause the PCI driver to not restore the PCI config space on a particular device? For example, if there is some sort of a bug in the device’s PCI PM capability info? Or is restoration of the PCI config space applied universally?

Thanks,

  • Jay

From: xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 13, 2013 10:36 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

The first thing that you need to understand is that EvtDeviceD0Entry is called when the PCI driver completes the device’s D0 IRP, allowing your driver (or in this case, WDF on your behalf) to know that the device is actually ready to be used.

The PCI driver does a full configuration space restoration just before completing that D0 IRP. The only thing that can stop it from completing that restoration is a hardware failure at the device. The most common type of failure involves not responding to the D0 transition within the 100ms timeout mandated by the PCI spec.

Your strategy is unlikely to be successful, simply because what you’re suggested has just failed once.

- Jake Oshins
(occasional PCI guy)
Windows Kernel Team
________________________________
From: xxxxx@lists.osr.commailto:xxxxx on behalf of Jay Talbott
Sent: Wednesday, March 13, 2013 8:37 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] PCI config space uninitialized after resuming from S3

A client of mine has an issue with their PCIe device where after waking up the system from an S3 sleep state, their device’s PCI config space is uninitialized, resulting in bad things happening when the driver tries to resume any interaction with the device. Is a PCI device supposed to maintain its PCI config space data while in D3, or does the PCI config space get restored by Windows during resume from sleep?

If it’s the former, then it looks like it’s a hardware problem. But if it’s the latter, what would be causing Windows not to be restoring the PCI config space for my client’s device, when the rest of the PCIe devices in the system are getting properly reinitialized?

Although I know that generally a driver should only READ its device’s PCI config data, would it be a reasonable solution to this problem to have the driver’s EvtDeviceD0Exit save the PCI config space when going into D3, and then have EvtDeviceD0Entry restore it when waking back up from D3? Or is this as much of a really bad idea as I think it is?

- Jay

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx>

6

It’s KMDF, and the driver isn’t interfering with the PM.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-528545-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Thursday, March 14, 2013 9:19 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
>
> A client of mine has an issue with their PCIe device where after
> waking up the system from an S3 sleep state, their device’s PCI config
> space is uninitialized, resulting in bad things happening when the
> driver tries to resume any interaction with the device. Is a PCI
> device supposed to maintain its PCI config space data while in D3, or
> does the PCI config space get restored by Windows during resume from
> sleep?
>

Is their driver interfering with power management in some way? We had
someone come around with a similar issue several months ago. In their
case, they had a WDM driver that was arbitrarily completing all power
IRPs. The PCI bus driver only restores the PCI configuration space in
response to a device power transition IRP coming through the function
driver. That is, your driver gets a request to transition to S0. In
response, your driver is supposed to generate a request to transition
the device to D0. That’s what triggers the restoration of config space
(or so it appears). If you don’t generate that D0 IRP, it won’t get
restored.

Is this actually a KMDF driver, as implied? KMDF should be handle all
of that for them, unless they are interfering.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

7

Going back to my idea of having the driver save/restore PCI config space,
what about inserting a delay (1 sec?) in EvtDeviceD0Entry before restoring
the PCI config space?

Or is having the driver restore PCI config space still a very bad idea?

I’m just trying to come up with some sort of workaround to help out my
client, since the product is already shipping to customers.

Frankly, I’m surprised that the WLK/HCK testing didn’t discover this. The
device and driver pass with flying colors. Of course, it’s an unclassified
device, so the tests don’t actually exercise any of the I/O functionality of
the driver, but still you’d think this would be something that would be
checked during PM testing.

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Thursday, March 14, 2013 9:47 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

It’s applied universally. It is possible to confuse the PCI driver if you
mis-use the WDM power IRPs. But you seem to be using KMDF, which will do
things properly, so that’s not your problem.

  • Jake

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Thursday, March 14, 2013 9:16 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

HI Jake,

Is there anything that could cause the PCI driver to not restore the PCI
config space on a particular device? For example, if there is some sort of a
bug in the device’s PCI PM capability info? Or is restoration of the PCI
config space applied universally?

Thanks,

  • Jay

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 13, 2013 10:36 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

The first thing that you need to understand is that EvtDeviceD0Entry is
called when the PCI driver completes the device’s D0 IRP, allowing your
driver (or in this case, WDF on your behalf) to know that the device is
actually ready to be used.

The PCI driver does a full configuration space restoration just before
completing that D0 IRP. The only thing that can stop it from completing
that restoration is a hardware failure at the device. The most common type
of failure involves not responding to the D0 transition within the 100ms
timeout mandated by the PCI spec.

Your strategy is unlikely to be successful, simply because what you’re
suggested has just failed once.

  • Jake Oshins

(occasional PCI guy)

Windows Kernel Team

From: xxxxx@lists.osr.com on behalf of Jay Talbott
Sent: Wednesday, March 13, 2013 8:37 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] PCI config space uninitialized after resuming from S3

A client of mine has an issue with their PCIe device where after waking up
the system from an S3 sleep state, their device’s PCI config space is
uninitialized, resulting in bad things happening when the driver tries to
resume any interaction with the device. Is a PCI device supposed to maintain
its PCI config space data while in D3, or does the PCI config space get
restored by Windows during resume from sleep?

If it’s the former, then it looks like it’s a hardware problem. But if it’s
the latter, what would be causing Windows not to be restoring the PCI config
space for my client’s device, when the rest of the PCIe devices in the
system are getting properly reinitialized?

Although I know that generally a driver should only READ its device’s PCI
config data, would it be a reasonable solution to this problem to have the
driver’s EvtDeviceD0Exit save the PCI config space when going into D3, and
then have EvtDeviceD0Entry restore it when waking back up from D3? Or is
this as much of a really bad idea as I think it is?

  • Jay

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

8

Ok, for the sake of trying to come up with a solution, I tried the
following:

I added a PCI_COMMON_CONFIG member to the device extension.

In EvtDeviceD0Exit, I added:

// Check if going into D3

if (TargetState == WdfPowerDeviceD3)

{

// Save the PCI configuration state

pDevExt->BusInterface.GetBusData(pDevExt->BusInterface.Context,

PCI_WHICHSPACE_CONFIG,

&pDevExt->PciConfig,

0,

sizeof(PCI_COMMON_CONFIG));

}

and in EvtDeviceD0Entry, I added:

// Check if coming out of D3

if (PreviousState == WdfPowerDeviceD3)

{

// Delay for a second

LARGE_INTEGER DelayTime;

DelayTime.QuadPart = WDF_REL_TIMEOUT_IN_SEC(1);

KeDelayExecutionThread(KernelMode, FALSE, &DelayTime);

// Restore the PCI configuration state

pDevExt->BusInterface.SetBusData(pDevExt->BusInterface.Context,

PCI_WHICHSPACE_CONFIG,

&pDevExt->PciConfig,

0,

sizeof(PCI_COMMON_CONFIG));

}

Initial testing shows that this works, but I’m still concerned that this
might be a really bad idea to be doing independently of the PCI bus driver.

Also, is there a driver verifier time limit for how much time can be spent
in EvtDeviceD0Entry that will be violated with this delay?

  • Jay

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Thursday, March 14, 2013 10:01 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

Going back to my idea of having the driver save/restore PCI config space,
what about inserting a delay (1 sec?) in EvtDeviceD0Entry before restoring
the PCI config space?

Or is having the driver restore PCI config space still a very bad idea?

I’m just trying to come up with some sort of workaround to help out my
client, since the product is already shipping to customers.

Frankly, I’m surprised that the WLK/HCK testing didn’t discover this. The
device and driver pass with flying colors. Of course, it’s an unclassified
device, so the tests don’t actually exercise any of the I/O functionality of
the driver, but still you’d think this would be something that would be
checked during PM testing.

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Thursday, March 14, 2013 9:47 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

It’s applied universally. It is possible to confuse the PCI driver if you
mis-use the WDM power IRPs. But you seem to be using KMDF, which will do
things properly, so that’s not your problem.

  • Jake

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Thursday, March 14, 2013 9:16 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

HI Jake,

Is there anything that could cause the PCI driver to not restore the PCI
config space on a particular device? For example, if there is some sort of a
bug in the device’s PCI PM capability info? Or is restoration of the PCI
config space applied universally?

Thanks,

  • Jay

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jake Oshins
Sent: Wednesday, March 13, 2013 10:36 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

The first thing that you need to understand is that EvtDeviceD0Entry is
called when the PCI driver completes the device’s D0 IRP, allowing your
driver (or in this case, WDF on your behalf) to know that the device is
actually ready to be used.

The PCI driver does a full configuration space restoration just before
completing that D0 IRP. The only thing that can stop it from completing
that restoration is a hardware failure at the device. The most common type
of failure involves not responding to the D0 transition within the 100ms
timeout mandated by the PCI spec.

Your strategy is unlikely to be successful, simply because what you’re
suggested has just failed once.

  • Jake Oshins

(occasional PCI guy)

Windows Kernel Team

From: xxxxx@lists.osr.com on behalf of Jay Talbott
Sent: Wednesday, March 13, 2013 8:37 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] PCI config space uninitialized after resuming from S3

A client of mine has an issue with their PCIe device where after waking up
the system from an S3 sleep state, their device’s PCI config space is
uninitialized, resulting in bad things happening when the driver tries to
resume any interaction with the device. Is a PCI device supposed to maintain
its PCI config space data while in D3, or does the PCI config space get
restored by Windows during resume from sleep?

If it’s the former, then it looks like it’s a hardware problem. But if it’s
the latter, what would be causing Windows not to be restoring the PCI config
space for my client’s device, when the rest of the PCIe devices in the
system are getting properly reinitialized?

Although I know that generally a driver should only READ its device’s PCI
config data, would it be a reasonable solution to this problem to have the
driver’s EvtDeviceD0Exit save the PCI config space when going into D3, and
then have EvtDeviceD0Entry restore it when waking back up from D3? Or is
this as much of a really bad idea as I think it is?

  • Jay

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

9

Jay Talbott wrote:

Ok, for the sake of trying to come up with a solution, I tried the
following:

…

// Check if coming out of D3

if(PreviousState== WdfPowerDeviceD3)

{

// Delay for a second

LARGE_INTEGERDelayTime;

DelayTime.QuadPart = WDF_REL_TIMEOUT_IN_SEC(1);

KeDelayExecutionThread(KernelMode, FALSE, &DelayTime);

// Restore the PCI configuration state

pDevExt->BusInterface.SetBusData(pDevExt->BusInterface.Context,

PCI_WHICHSPACE_CONFIG,

&pDevExt->PciConfig,

0,

sizeof(PCI_COMMON_CONFIG));

}

Initial testing shows that this works, but I’m still concerned that
this might be a really bad idea to be doing independently of the PCI
bus driver.

Why do you need the one-second delay? Is that because your hardware
isn’t prepared to listen to configuration space requests at that point?
If so, that could be the root of the problem, as Jake pointed out last
night. Your device is required to handle configuration requests within
100ms of restoring power. If your device can’t meet that, then the
hardware is broken. Just that simple.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

10

Tim,

Yes, I agree that this is probably a flaw is in the hardware. But since this
hardware is already shipping, I am thinking outside of the box for a
solution that will work with the existing hardware - thus the delay in the
code.

At least in initial testing on one test machine, it’s working as desired.
But I don’t know how well that translates to general case.

  • Jay

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-528581-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Thursday, March 14, 2013 11:00 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
>
> Ok, for the sake of trying to come up with a solution, I tried the
> following:
>
> …
>
> // Check if coming out of D3
>
> if(PreviousState== WdfPowerDeviceD3)
>
> {
>
> // Delay for a second
>
> LARGE_INTEGERDelayTime;
>
> DelayTime.QuadPart = WDF_REL_TIMEOUT_IN_SEC(1);
>
> KeDelayExecutionThread(KernelMode, FALSE, &DelayTime);
>
>
>
> // Restore the PCI configuration state
>
> pDevExt->BusInterface.SetBusData(pDevExt->BusInterface.Context,
>
> PCI_WHICHSPACE_CONFIG,
>
> &pDevExt->PciConfig,
>
> 0,
>
> sizeof(PCI_COMMON_CONFIG));
>
> }
>
>
>
> Initial testing shows that this works, but I’m still concerned that
> this might be a really bad idea to be doing independently of the PCI
> bus driver.
>

Why do you need the one-second delay? Is that because your hardware
isn’t prepared to listen to configuration space requests at that point?
If so, that could be the root of the problem, as Jake pointed out last
night. Your device is required to handle configuration requests within
100ms of restoring power. If your device can’t meet that, then the
hardware is broken. Just that simple.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

11

Jay Talbott wrote:

Yes, I agree that this is probably a flaw is in the hardware. But since this
hardware is already shipping, I am thinking outside of the box for a
solution that will work with the existing hardware - thus the delay in the
code.

At least in initial testing on one test machine, it’s working as desired.
But I don’t know how well that translates to general case.

Oh, I assume there are circumstances in which this still fails. Have
you checked all of the sleep/standby/hibernate transitions? (The WHQL
tests certainly do.)

However, I don’t think this needs to concern you very much. You have
other, bigger problems. For example, if you are violating the power-up
specs, there are motherboards that will disable your board at boot
time. You can’t fix that, because it happens before the operating
system is even loaded.

This also means you can’t use the PCI logos on your packaging, because
you would not pass the PCI Compliance Suite.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

12

So far I’ve tested Sleep, Hibernate, and Hybrid Sleep on x64 Win7, and it’s
working in all cases (where before it would fail Sleep and Hybrid Sleep).
Obviously I need to test other OS variants, as this is just my initial proof
of concept testing of the solution to the problem.

The WLK and HCK tests certain put the device through all of the power state
transitions, but clearly don’t check the PCI config space to verify it was
correctly restored when returning from S3 to S0, as the device has passed
the WLK and HCK tests. As an unclassified device, the WLK and HCK tests
never exercise the actual I/O functionality of the device, so they didn’t
uncover that the driver could no longer communicate with the hardware
following a sleep.

There’s no question that this is a big problem as far as the hardware is
concerned which my client will have to address. Fortunately, it must be
close enough to the spec that motherboards aren’t having an issue with it,
as the device has been tested in numerous different systems and now that
it’s shipping and many are in the hands of customers, there have been no
reports of any motherboard issues - at least not related to this particular
issue.

The bottom line is that we are looking for a short-term solution in the
driver that addresses this issue for existing hardware in the field while my
client addresses the flaws in their hardware.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-528583-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Thursday, March 14, 2013 11:17 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
> Yes, I agree that this is probably a flaw is in the hardware. But since
this
> hardware is already shipping, I am thinking outside of the box for a
> solution that will work with the existing hardware - thus the delay in
the
> code.
>
> At least in initial testing on one test machine, it’s working as
desired.
> But I don’t know how well that translates to general case.

Oh, I assume there are circumstances in which this still fails. Have
you checked all of the sleep/standby/hibernate transitions? (The WHQL
tests certainly do.)

However, I don’t think this needs to concern you very much. You have
other, bigger problems. For example, if you are violating the power-up
specs, there are motherboards that will disable your board at boot
time. You can’t fix that, because it happens before the operating
system is even loaded.

This also means you can’t use the PCI logos on your packaging, because
you would not pass the PCI Compliance Suite.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

13

> Oh, I assume there are circumstances in which this still fails. Have

you checked all of the sleep/standby/hibernate transitions? (The WHQL
tests certainly do.)

PNPDTEST/PWRTEST alone will do this, provided the BIOS supports all S states.

–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

14

Assuming that you decide to go with a workaround like this, I suggest that
you disable the behaviour by default and activate it with a registry setting
(or other configuration) so that when this version of your driver is
installed on corrected hardware, the arbitrary delay is not present unless
specifically activated. You could also do this by detecting specific
versions of the board a la the floating point errata workarounds.

“Jay Talbott” wrote in message news:xxxxx@ntdev…

So far I’ve tested Sleep, Hibernate, and Hybrid Sleep on x64 Win7, and it’s
working in all cases (where before it would fail Sleep and Hybrid Sleep).
Obviously I need to test other OS variants, as this is just my initial proof
of concept testing of the solution to the problem.

The WLK and HCK tests certain put the device through all of the power state
transitions, but clearly don’t check the PCI config space to verify it was
correctly restored when returning from S3 to S0, as the device has passed
the WLK and HCK tests. As an unclassified device, the WLK and HCK tests
never exercise the actual I/O functionality of the device, so they didn’t
uncover that the driver could no longer communicate with the hardware
following a sleep.

There’s no question that this is a big problem as far as the hardware is
concerned which my client will have to address. Fortunately, it must be
close enough to the spec that motherboards aren’t having an issue with it,
as the device has been tested in numerous different systems and now that
it’s shipping and many are in the hands of customers, there have been no
reports of any motherboard issues - at least not related to this particular
issue.

The bottom line is that we are looking for a short-term solution in the
driver that addresses this issue for existing hardware in the field while my
client addresses the flaws in their hardware.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-528583-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Thursday, March 14, 2013 11:17 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PCI config space uninitialized after resuming from S3

Jay Talbott wrote:
> Yes, I agree that this is probably a flaw is in the hardware. But since
this
> hardware is already shipping, I am thinking outside of the box for a
> solution that will work with the existing hardware - thus the delay in
the
> code.
>
> At least in initial testing on one test machine, it’s working as
desired.
> But I don’t know how well that translates to general case.

Oh, I assume there are circumstances in which this still fails. Have
you checked all of the sleep/standby/hibernate transitions? (The WHQL
tests certainly do.)

However, I don’t think this needs to concern you very much. You have
other, bigger problems. For example, if you are violating the power-up
specs, there are motherboards that will disable your board at boot
time. You can’t fix that, because it happens before the operating
system is even loaded.

This also means you can’t use the PCI logos on your packaging, because
you would not pass the PCI Compliance Suite.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

15

Further investigation of the hardware suggests that it should be powered up
long before the 100 ms timeout mandated by the PCI spec…

So I’m hoping that someone at Microsoft can answer this one: What are all of
the possible situations that could cause the PCI bus driver to NOT restore
the PCI config data for a device when resuming from sleep?

The actual PCIe link is getting re-established without any issues. But the
PCI config data is not always getting restored by the PCI bus driver. So
either there’s still some problem with the PCI config data not “sticking”
after being restored, or there’s a reason why the PCI bus driver is not
restoring the data at all.

The problem appears like it’s some sort of race condition, as it’s only
reproducible on some systems, some of the time. On one system where the
problem was reproducible, a BIOS update made it go away. And before that
BIOS update was applied on that particular system, the problem was
observable in Vista, but not in Win7 or Win8. On another test system, it’s
reproducible 100% of the time in Vista, Win7, and Win8. On other systems,
the problem can’t be reproduced at all.

The driver change I made so that the driver itself saves and restores the
device’s PCI config space appears to solve the problem on systems where the
problem can be reproduced, although it’s just masking the underlying problem
and my client wants to understand what’s ultimately the root cause so that
it can be properly resolved.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-528584-
xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Thursday, March 14, 2013 12:01 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

So far I’ve tested Sleep, Hibernate, and Hybrid Sleep on x64 Win7, and
it’s
working in all cases (where before it would fail Sleep and Hybrid Sleep).
Obviously I need to test other OS variants, as this is just my initial
proof
of concept testing of the solution to the problem.

The WLK and HCK tests certain put the device through all of the power
state
transitions, but clearly don’t check the PCI config space to verify it was
correctly restored when returning from S3 to S0, as the device has passed
the WLK and HCK tests. As an unclassified device, the WLK and HCK tests
never exercise the actual I/O functionality of the device, so they didn’t
uncover that the driver could no longer communicate with the hardware
following a sleep.

There’s no question that this is a big problem as far as the hardware is
concerned which my client will have to address. Fortunately, it must be
close enough to the spec that motherboards aren’t having an issue with it,
as the device has been tested in numerous different systems and now that
it’s shipping and many are in the hands of customers, there have been no
reports of any motherboard issues - at least not related to this
particular
issue.

The bottom line is that we are looking for a short-term solution in the
driver that addresses this issue for existing hardware in the field while
my
client addresses the flaws in their hardware.

> -----Original Message-----
> From: xxxxx@lists.osr.com [mailto:bounce-528583-
> xxxxx@lists.osr.com] On Behalf Of Tim Roberts
> Sent: Thursday, March 14, 2013 11:17 AM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] PCI config space uninitialized after resuming from
S3
>
> Jay Talbott wrote:
> > Yes, I agree that this is probably a flaw is in the hardware. But
since
this
> > hardware is already shipping, I am thinking outside of the box for a
> > solution that will work with the existing hardware - thus the delay in
the
> > code.
> >
> > At least in initial testing on one test machine, it’s working as
desired.
> > But I don’t know how well that translates to general case.
>
> Oh, I assume there are circumstances in which this still fails. Have
> you checked all of the sleep/standby/hibernate transitions? (The WHQL
> tests certainly do.)
>
> However, I don’t think this needs to concern you very much. You have
> other, bigger problems. For example, if you are violating the power-up
> specs, there are motherboards that will disable your board at boot
> time. You can’t fix that, because it happens before the operating
> system is even loaded.
>
> This also means you can’t use the PCI logos on your packaging, because
> you would not pass the PCI Compliance Suite.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

16

As I remember if you run the checked version of the PCI driver and turn on tracing for it you get a ton of details about what it’s doing. I believe in years past there have details on how to turn on PCI driver tracing on this list.

Jan

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Friday, March 22, 2013 4:28 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

Further investigation of the hardware suggests that it should be powered up long before the 100 ms timeout mandated by the PCI spec…

So I’m hoping that someone at Microsoft can answer this one: What are all of the possible situations that could cause the PCI bus driver to NOT restore the PCI config data for a device when resuming from sleep?

The actual PCIe link is getting re-established without any issues. But the PCI config data is not always getting restored by the PCI bus driver. So either there’s still some problem with the PCI config data not “sticking”
after being restored, or there’s a reason why the PCI bus driver is not restoring the data at all.

The problem appears like it’s some sort of race condition, as it’s only reproducible on some systems, some of the time. On one system where the problem was reproducible, a BIOS update made it go away. And before that BIOS update was applied on that particular system, the problem was observable in Vista, but not in Win7 or Win8. On another test system, it’s reproducible 100% of the time in Vista, Win7, and Win8. On other systems, the problem can’t be reproduced at all.

The driver change I made so that the driver itself saves and restores the device’s PCI config space appears to solve the problem on systems where the problem can be reproduced, although it’s just masking the underlying problem and my client wants to understand what’s ultimately the root cause so that it can be properly resolved.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-528584-
xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Thursday, March 14, 2013 12:01 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming
from S3

So far I’ve tested Sleep, Hibernate, and Hybrid Sleep on x64 Win7, and
it’s
working in all cases (where before it would fail Sleep and Hybrid Sleep).
Obviously I need to test other OS variants, as this is just my initial
proof
of concept testing of the solution to the problem.

The WLK and HCK tests certain put the device through all of the power
state
transitions, but clearly don’t check the PCI config space to verify it
was correctly restored when returning from S3 to S0, as the device has
passed the WLK and HCK tests. As an unclassified device, the WLK and
HCK tests never exercise the actual I/O functionality of the device,
so they didn’t uncover that the driver could no longer communicate
with the hardware following a sleep.

There’s no question that this is a big problem as far as the hardware
is concerned which my client will have to address. Fortunately, it
must be close enough to the spec that motherboards aren’t having an
issue with it, as the device has been tested in numerous different
systems and now that it’s shipping and many are in the hands of
customers, there have been no reports of any motherboard issues - at
least not related to this
particular
issue.

The bottom line is that we are looking for a short-term solution in
the driver that addresses this issue for existing hardware in the
field while
my
client addresses the flaws in their hardware.

> -----Original Message-----
> From: xxxxx@lists.osr.com [mailto:bounce-528583-
> xxxxx@lists.osr.com] On Behalf Of Tim Roberts
> Sent: Thursday, March 14, 2013 11:17 AM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] PCI config space uninitialized after resuming
> from
S3
>
> Jay Talbott wrote:
> > Yes, I agree that this is probably a flaw is in the hardware. But
since
this
> > hardware is already shipping, I am thinking outside of the box for
> > a solution that will work with the existing hardware - thus the
> > delay in
the
> > code.
> >
> > At least in initial testing on one test machine, it’s working as
desired.
> > But I don’t know how well that translates to general case.
>
> Oh, I assume there are circumstances in which this still fails.
> Have you checked all of the sleep/standby/hibernate transitions?
> (The WHQL tests certainly do.)
>
> However, I don’t think this needs to concern you very much. You
> have other, bigger problems. For example, if you are violating the
> power-up specs, there are motherboards that will disable your board
> at boot time. You can’t fix that, because it happens before the
> operating system is even loaded.
>
> This also means you can’t use the PCI logos on your packaging,
> because you would not pass the PCI Compliance Suite.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

17

A bus analyzer should be able to tell you if the PCI bus driver is restoring you config space or not, right? So you can tentatively determine where your device or PCI is at fault?

Peter
OSR

18

On 3/22/2013 7:28 PM, Jay Talbott wrote:

and my client wants to understand what’s ultimately the root cause so that
it can be properly resolved.

Wish I had more clients like that!

– mkj

“I can answer any question, so long as you agree that
‘I don’t know’ is an answer.”
– Isaac Asimov

//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//_______________________________________________

19

What’s the magic trick for extracting just the PCI driver from an MSDN
checked build .iso without having to install the complete checked build?

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529543-
xxxxx@lists.osr.com] On Behalf Of Jan Bottorff
Sent: Friday, March 22, 2013 4:45 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

As I remember if you run the checked version of the PCI driver and turn on
tracing for it you get a ton of details about what it’s doing. I believe
in years
past there have details on how to turn on PCI driver tracing on this list.

Jan

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-529541-
xxxxx@lists.osr.com] On Behalf Of Jay Talbott
Sent: Friday, March 22, 2013 4:28 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] PCI config space uninitialized after resuming from S3

Further investigation of the hardware suggests that it should be powered
up
long before the 100 ms timeout mandated by the PCI spec…

So I’m hoping that someone at Microsoft can answer this one: What are all
of
the possible situations that could cause the PCI bus driver to NOT restore
the
PCI config data for a device when resuming from sleep?

The actual PCIe link is getting re-established without any issues. But the
PCI
config data is not always getting restored by the PCI bus driver. So
either
there’s still some problem with the PCI config data not “sticking”
after being restored, or there’s a reason why the PCI bus driver is not
restoring the data at all.

The problem appears like it’s some sort of race condition, as it’s only
reproducible on some systems, some of the time. On one system where the
problem was reproducible, a BIOS update made it go away. And before that
BIOS update was applied on that particular system, the problem was
observable in Vista, but not in Win7 or Win8. On another test system, it’s
reproducible 100% of the time in Vista, Win7, and Win8. On other systems,
the problem can’t be reproduced at all.

The driver change I made so that the driver itself saves and restores the
device’s PCI config space appears to solve the problem on systems where
the problem can be reproduced, although it’s just masking the underlying
problem and my client wants to understand what’s ultimately the root cause
so that it can be properly resolved.

> -----Original Message-----
> From: xxxxx@lists.osr.com [mailto:bounce-528584-
> xxxxx@lists.osr.com] On Behalf Of Jay Talbott
> Sent: Thursday, March 14, 2013 12:01 PM
> To: Windows System Software Devs Interest List
> Subject: RE: [ntdev] PCI config space uninitialized after resuming
> from S3
>
> So far I’ve tested Sleep, Hibernate, and Hybrid Sleep on x64 Win7, and
it’s
> working in all cases (where before it would fail Sleep and Hybrid
Sleep).
> Obviously I need to test other OS variants, as this is just my initial
proof
> of concept testing of the solution to the problem.
>
> The WLK and HCK tests certain put the device through all of the power
state
> transitions, but clearly don’t check the PCI config space to verify it
> was correctly restored when returning from S3 to S0, as the device has
> passed the WLK and HCK tests. As an unclassified device, the WLK and
> HCK tests never exercise the actual I/O functionality of the device,
> so they didn’t uncover that the driver could no longer communicate
> with the hardware following a sleep.
>
> There’s no question that this is a big problem as far as the hardware
> is concerned which my client will have to address. Fortunately, it
> must be close enough to the spec that motherboards aren’t having an
> issue with it, as the device has been tested in numerous different
> systems and now that it’s shipping and many are in the hands of
> customers, there have been no reports of any motherboard issues - at
> least not related to this
particular
> issue.
>
> The bottom line is that we are looking for a short-term solution in
> the driver that addresses this issue for existing hardware in the
> field while
my
> client addresses the flaws in their hardware.
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com [mailto:bounce-528583-
> > xxxxx@lists.osr.com] On Behalf Of Tim Roberts
> > Sent: Thursday, March 14, 2013 11:17 AM
> > To: Windows System Software Devs Interest List
> > Subject: Re: [ntdev] PCI config space uninitialized after resuming
> > from
S3
> >
> > Jay Talbott wrote:
> > > Yes, I agree that this is probably a flaw is in the hardware. But
since
> this
> > > hardware is already shipping, I am thinking outside of the box for
> > > a solution that will work with the existing hardware - thus the
> > > delay in
> the
> > > code.
> > >
> > > At least in initial testing on one test machine, it’s working as
> desired.
> > > But I don’t know how well that translates to general case.
> >
> > Oh, I assume there are circumstances in which this still fails.
> > Have you checked all of the sleep/standby/hibernate transitions?
> > (The WHQL tests certainly do.)
> >
> > However, I don’t think this needs to concern you very much. You
> > have other, bigger problems. For example, if you are violating the
> > power-up specs, there are motherboards that will disable your board
> > at boot time. You can’t fix that, because it happens before the
> > operating system is even loaded.
> >
> > This also means you can’t use the PCI logos on your packaging,
> > because you would not pass the PCI Compliance Suite.
> >
> > –
> > Tim Roberts, xxxxx@probo.com
> > Providenza & Boekelheide, Inc.
> >
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > OSR is HIRING!! See http://www.osr.com/careers
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at
> > http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

20

http://bit.ly/UFN2lB

(or use Win8, which mounts ISOs transparently)

Peter
OSR