Notify Callback function never called when using ACPI RegisterForDeviceNotifications

NTDEV
1

We are working on a SCSI STORPORT driver for Windows 8 (and others if it really matters but this discussion is being limited to Win8).

The driver implements RAID support utilizing the motherboard AHCI controller.

We are attempting to implement power control for an optical device - ZPODD to be exact.

For reasons I won’t go into here we are doing this via method calls to ACPI. [This as opposed to utilizing WIN8 PoFxDevice support… Leave it that we have a support ticket open with Microsoft to work that aspect that remains unresolved after several months.]

The driver can successfully control the power to the ODD by invoking the appropriate _PS0 and _PS3 methods but this assumes that the driver makes the decision to turn power off or on.

Our problem is that when the user attempts to wake the device (by pressing the eject button on the front), we have been unsuccessful in getting any notification from ACPI. [The driver needs to receive a notification that the user wishes to power on the device and then the driver needs to invoke the _PS0 method to actually power on the ODD.]

Using windbg, we can step through the code and verify that the call to RegisterForDeviceNotifications() is returning a status of SUCCESS, and that ACPI is posting a NOTIFY for the press of the eject button.

The callback function (RC_HandleAcpiNotification()) is never called and I’m not sure where to go from here…

Any pointers or helpful knowledge would be greatly appreciated.

Thanks.

-Joe Thomas

I’ve include relevant code, as well as trace messages from windbg and my observations of trying to step through the ACPI code…
For complete information, the PCI VID is 1022, DID is 7805, and device class/subclass is 0104 (Mass Storage, SCSI Controller). [The thought at Microsoft is that there may be problems with the PoFxDevice support and ACPI when the device subclass does NOT equal 06 (SATA Controller) so the fact that the subclass is different may be pertinent.]

We currently register for ACPI notifications as follows:

VOID
RC_HandleAcpiNotification(
PVOID Context,
ULONG Code
)
{
ULONG tmp;

tmp = Code;
//RC_HW_OS_DebugPrintf(“RC_HandleAcpiNotification(): Context = 0x%x, Code = 0x%x\n”,
// Context, Code);
}

ULONG
RC_RegisterForAcpiNotifications(
IN PVOID hwDevExt,
IN PVOID Pdo
)
{
ULONG status = STATUS_SUCCESS;
ACPI_INTERFACE_STANDARD AcpiInterfaces = { 0 };

if ((status = RC_GetAcpiInterfaces((PDEVICE_OBJECT) Pdo, &AcpiInterfaces)) == STATUS_SUCCESS)
{
if (AcpiInterfaces.RegisterForDeviceNotifications)
{
status = AcpiInterfaces.RegisterForDeviceNotifications(
(PDEVICE_OBJECT) AcpiInterfaces.Context,
RC_HandleAcpiNotification,
(PDEVICE_OBJECT) Pdo
);
}
}

return status;
}

BOOLEAN
HwInitialization(
IN PVOID HwDeviceExtension
)
{
PVOID adapterDeviceObject;
PVOID physicalDeviceObject;
PVOID lowerDeviceObject;

StorPortGetDeviceObjects(
HwDeviceExtension,
&adapterDeviceObject,
&physicalDeviceObject,
&lowerDeviceObject
);

(VOID) RC_RegisterForAcpiNotifications(
HwDeviceExtension,
lowerDeviceObject
);

}

Windbg shows the following when the button is pressed:
ACPIWriteGpeEnableRegister: Writing GPE Enable register 0 = 0
ACPIWriteGpeEnableRegister: Writing GPE Enable register 1 = 0
ACPIWriteGpeEnableRegister: Writing GPE Enable register 2 = 0
ACPIWriteGpeEnableRegister: Writing GPE Enable register 3 = 0
AMLI: FFFFFA8008228B00: AsyncEvalObject(_GPE._L06)

AMLI: FFFFFA8008228B00: _GPE._L06()
ACPIWriteGpeEnableRegister: Writing GPE Enable register 0 = 28
ACPIWriteGpeEnableRegister: Writing GPE Enable register 1 = 38
ACPIWriteGpeEnableRegister: Writing GPE Enable register 2 = 1
ACPIWriteGpeEnableRegister: Writing GPE Enable register 3 = 10

fffffa800880d1a1: {
fffffa800880d1a1: | If(LAnd(LNot(LEqual(ODZC=0x1,Zero)=0x0)=0xffffffff,LEqual(SB.PCI0.SMBS.GE16=0x0,Zero)=0xffffffff)=0xffffffff)
fffffa800880d1c1: | {
fffffa800880d1c1: | | If(SB.PCI0.SMBS.G06T=0x0)
fffffa800880d36e: | | {
fffffa800880d36e: | | | Sleep(0x14)
fffffa800880d372: | | | If(LNot(SB.PCI0.SMBS.GE06=0x0)=0xffffffff)
fffffa800880d389: | | | {
fffffa800880d389: | | | | If(LEqual(SB.PCI0.SATA.STCL
OpRegion Access on region FFFFFA800886E768 device FFFFFA800886E438
DeviceHandle 0000000000000000
Return from OR handler - status 0
OpRegion Access on region FFFFFA800886E768 device FFFFFA800886E438
DeviceHandle 0000000000000000
Return from OR handler - status 0
=0x104,0x101)=0x0)
fffffa800880d3a2: | | | | If(LEqual(SB.PCI0.SATA.STCL
OpRegion Access on region FFFFFA800886E768 device FFFFFA800886E438
DeviceHandle 0000000000000000
Return from OR handler - status 0
OpRegion Access on region FFFFFA800886E768 device FFFFFA800886E438
DeviceHandle 0000000000000000
Return from OR handler - status 0
=0x104,0x106)=0x0)
fffffa800880d3f4: | | | | If(LOr(LEqual(SB.PCI0.SATA.SDID
OpRegion Access on region FFFFFA800886E768 device FFFFFA800886E438
DeviceHandle 0000000000000000
Return from OR handler - status 0
OpRegion Access on region FFFFFA800886E768 device FFFFFA800886E438
DeviceHandle 0000000000000000
Return from OR handler - status 0
=0x7805,0x7803)=0x0,LEqual(SB.PCI0.SATA.SDID
OpRegion Access on region FFFFFA800886E768 device FFFFFA800886E438
DeviceHandle 0000000000000000
Return from OR handler - status 0
OpRegion Access on region FFFFFA800886E768 device FFFFFA800886E438
DeviceHandle 0000000000000000
Return from OR handler - status 0
=0x7805,0x7802)=0x0)=0x0)
fffffa800880d45e: | | | | If(LEqual(SB.PCI0.SATA.SDID
OpRegion Access on region FFFFFA800886E768 device FFFFFA800886E438
DeviceHandle 0000000000000000
Return from OR handler - status 0
OpRegion Access on region FFFFFA800886E768 device FFFFFA800886E438
DeviceHandle 0000000000000000
Return from OR handler - status 0
=0x7805,0x7805)=0xffffffff)
fffffa800880d478: | | | | {
fffffa800880d478: | | | | | If(LNot(LLess(TPOS=0x60,0x60)=0x0)=0xffffffff)
fffffa800880d482: | | | | | {
fffffa800880d482: | | | | | | Notify(SB.PCI0.SATA.ODDZ,0x2)
ACPINotifyHandler: Notify on 88760c8 value 2, object type 6
FFFFFA8007F941E0 1ffff OSNotifyDeviceWake - 0xFFFFFA80088760C8 (ODDZ)
ACPIWriteGpeEnableRegister: Writing GPE Enable register 0 = 28
ACPIWriteGpeEnableRegister: Writing GPE Enable register 1 = 38
ACPIWriteGpeEnableRegister: Writing GPE Enable register 2 = 1
ACPIWriteGpeEnableRegister: Writing GPE Enable register 3 = 10

fffffa800880d498: | | | | | }
fffffa800880d4b0: | | | | }
fffffa800880d4b0: | | | | Sleep(0x1f4)

I’ve tried tracing through ACPI!ACPIRegisterForDeviceNotifications and the code is placing the callback address and context into a (unknown) table. I’m somewhat curious about the fact that this function “jmp”'s into ACPIRegisterForDeviceNotificationsByPowerInfo which then “ret”'s to the caller. To me, this implies that there is some type of PowerInfo structure and/or setting that might be required from the driver?

On the back end, the ACPI “Notify(_SB.PCI0.SATA.ODDZ,0x2)” generates a call to ACPI!Notify() which calls ACPI!NotifyHandler() which calls ACPI!OSNotifyDeviceWake() which calls ACPI!ACPIWakeRemoveDevicesAndUpdate(). ACPI!NotifyHandler() also calls ACPI!OSPowerFindPowerInfo() which again makes me question if there’s a power requirement we’re not aware of. Finally, ACPI!NotifyHandler() calls ACPI!DispatchNotification() before (eventually) returning to the caller.

Throughout this, I haven’t been able to discern where the checks are being made to see if my callback handler should be invoked, or if the check is being made at all.

// Joseph Thomas
// Principal Software Engineer
// Dot Hill Systems
// 2905 NorthWest Blvd., Suite 20
// Plymouth, MN 55441
// 763.226.2640

2

Maybe ACPI detects the event for a separate (than your CD) device?

3

Yuck.

So, I’m sure you realize that support for ZPODs is entirely new to Win8, right?

Have you talked to AMD about this? Any chance of easily changing the subclass of the device… Is there an accessible EEPROM on the board?

Your Register succeeds… But are you sure it succeeds for the same device for which the Notify is being raised (pci0.sata.oddz)?

A wild hope, I admit…

Peter
OSR

4

Peter,

Thanks for the reply…

Changing the subclass has it’s own set of issues some of which have to do with how our driver is presented to the OS (SCSI) and WHCK qualification. If it were simply a problem of supporting Win8, this might work but since we’re being asked also support Win7 and the 2K8/2K12, we need something that’s more “universal” as opposed to the Win8 only ZPODD implementation (i.e. creating PoFxPower structures and relying on CDROM.SYS to handle idle’ing the device, etc.)

AMD is engaged in this. Part of the problem is that this all works if its a straight forward AHCI driver. When you try to layer RAID on top of that, you change a lot of the assumptions being made because you’re no longer presenting the “real” physical view. Simple things like Unit 0 doesn’t really mean physical port 0, and the fact that StorPort found two devices, doesn’t mean that there are only two physical ports active. We’re really pushing the envelope for how all this was “thought” to work. Trying to use the StorPort calls w/BTL8 address structures becomes a lot more complicated.

For development purposes, we can and have run our driver using both subclasses and from a high level view, see no difference in behavior. By high level, I mean that the callback function isn’t called in either case but there may be a different reason “under the wraps” so to speak.

I admit there’s a problem in debugging this in that I don’t know for sure that it’s the same device. Documentation on ACPI seems to be vary sparse at best. I’ve actually found more information on what the BIOS developer needs that what a device driver developer needs. From the ACPI standpoint, ODDZ is a sub device under SATA which is the PCI address of the device the driver is controlling. Similarly, since I can control the _PS0 and _PS3 (actually ODDZ._PS0 and ODDZ._PS3) methods using the same StorPortGetDeviceObjects() call to find the lowerDeviceObject, etc. (for setting up an IRP), I am assuming that the two “device” do in fact refer to the same device.

Is there some method I can use to verify this? The Notify()'s are using ACPI objects and not driver Device Objects so I’m pretty clueless on how ACPI.SYS relates the two to each other… [Dumping the addresses from the notify messages return data that clearly isn’t a DeviceObject. Some data looks almost like “flag” type data – lots of small values (<10 in ulongs) while other data appears to be ACPI node information – that is, it contains strings representing the ACPI node, such as ‘ODDZ’, with some pointers to “something” – maybe links to next node?]

From: xxxxx@lists.osr.com [xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com [xxxxx@osr.com]
Sent: Friday, April 12, 2013 5:29 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Notify Callback function never called when using ACPI RegisterForDeviceNotifications

Yuck.

So, I’m sure you realize that support for ZPODs is entirely new to Win8, right?

Have you talked to AMD about this? Any chance of easily changing the subclass of the device… Is there an accessible EEPROM on the board?

Your Register succeeds… But are you sure it succeeds for the same device for which the Notify is being raised (pci0.sata.oddz)?

A wild hope, I admit…

Peter
OSR

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

5

(Just saying: You have a complex problem, in a complex environment, involving many variables, and a little part of the system that few people have much experience with. Hoping for significant insight from an internet forum, even one as broad and deep as NTDEV can be at times, is a pretty big ask. Even if one of the MSFT dev owners reads your post, this is the kind of thing that takes time and investigation.)

OK, back to seeing if I can help in any way. I doubt it, but let’s try:

We need to cover a number of basic questions: How are identifying the ACPI device that “owns” the methods you’re calling. Do you have a separate (ACPI) device defined in the BIOS that has the methods you’re invoking, and a separate function driver that claims that device (hardware ID ACPI\xxxx)? Or are these methods associated with the larger device, such as the AHCI Controller. I think I know the answer, but we need to make sure.

I assume you have control of the BIOS? You can modify the BIOS as you see fit? If so… adding some debugging code should be pretty straight forward, no??

Aren’t ZPODs PnP devices that are dynamically detected on the AHCI bus? Assuming so, I admit to being a bit confused as to how they would be dynamically identified and associated with static methods in the BIOS.

Eh? You’re able to get the Notify if you configure your StorPort as a (non-raided) AHCI driver… or if you hack the Register/Notify into StorAHCI or whatever?

If that’s the case, I would guess THAT would point to hanging the notify on the wrote unit, don’t you think?

Architecturally, are you stuffing all this RAIDing in the StorPort driver… consuming the AHCI controller and all the attached LUs and surfacing “virtual” devices out the top? If so, with respect, I’m not a fan of that architecture. It can be very messy and hard to debug.

Quite right. “people who write drivers” and “people who know something about ACPI” in the Windows world are typically sets that have a *very* small intersection. That’s because, in general in Windows, most things where people think “I can call an ACPI method to do/get/set this” are either (a) already abstracted and exported by the OS or (b) better done another way.

Now, let me hasten to add: I am *not* saying this is the case with which you’re trying to do, which – assuming I understand at all, you’re trying to add ZPOD support to down-level platforms – seems entirely reasonable and justified.

I’m not sure… I can try to do some spelunking.

But, IIRC, at its core it’s really pretty simple: It’s not like there’s a lookup table. You have a (ACPI) device namespace on which your driver is instantiated. ACPI serves as the bus driver for this device when it enumerates devices in its namespace. You register a callback with that (ACPI) device. When that device raises a Notify, you get the callback.

I’m not sure I’m helping at all… I wish I could help more. But for a Saturday morning, that’s the best I can come up with right now.

Peter
OSR

6

Peter,

Thanks for the feedback!

First, I hope I haven’t come across as expecting someone to hand me the answer… I understand that what we’re attempting is unusual to say the least. I greatly appreciate any pointers/insights/help anyone offers but understand that ultimately, it’s my problem to solve.

To address the items you pointed out…

  1. These are methods that are associated with the AHCI (SATA) controller. Not that it matters, but the tree looks like (with lots of removed elements):
    Device(SATA)
    {
    Name(_ADR, 0x00110000)
    Field(BAR5,…)
    Device(PRID)
    {
    Name(_ADR, 0)
    }
    Device(SECD)
    {
    Name(_ADR, 2)
    }
    Device(ODDZ)
    {
    Name(_ADR, 0x1FFFF)
    Method(_PS0)
    {
    // Turn power on
    }
    Method(_PS3)
    {
    // Turn power off
    }
    }
    }

  2. We do not have control over the BIOS. To a small extent, we can ask for a one off build that has some debug code or changes but that requires a fair amount of work to make it happen.

  3. ZPODDs are specific to Optical Devices. There are two or three main requirements to determine if a device supports ZPODD – 1st the device must support AsyncNotifications, 2nd the device must support DeviceAttention (redefinition of a signal on the SlimLine power connecter) both of which are checked by the driver. The 3rd requirement is that it be a tray or slot loading device per the spec. The unspoken requirement is that the BIOS/ACPI must support it as well but conversely, just because the BIOS supports ZPODD, doesn’t mean that the connected device does. Anyways, the association is made by the _ADR element which for SATA/ACHI means physical port number. So from the above table, device ODDZ is on physical port 1 of the AHCI controller. [The 0xffff means that this port isn’t connected to a port multiplier.]

  4. Sorry if I implied that our code was working. What I was trying to convey is that (for Win8) using the inbox StorAhci.sys driver, someone (seems to be part of the “power” subsystem - not storahci) registers for the ACPI notification and when the notification occurs, receives the event and acts on it.

StorAhci creates some power control structures (PoFxDevice) for the controller and each “unit”, report the ability of the unit to support AsyncNotification and DeviceAttention (StorPortSetUnitCapabilities), and then signals StorPort when ever an AsyncNotification is received. The inbox CdRom.sys driver will check that the various requirements are met and if so, registers with the runtime power support. StorPort calls CdRom.sys when the AsyncNotification arrives, cdrom checks that the “idle” condition is met (no media, tray closed) and sends a device idle notification to the runtime power system. Runtime power then starts a timer and after a defined period has someone invoke the ODDZ._PS3() method to power off. [I haven’t gotten through all the threads yet to see which driver makes the actual call, who “owns” the acpi method, etc. Much of this is done in worker threads to deal with IRQL issues, etc.] We can accomplish all this with our driver today.

When the user presses the eject button, a DeviceAttention is signaled across the SlimLine connecter, and however the MB vendor decides (connected to a GPIO pin in our test platform) is picked up by BIOS and causes ACPI to issue a NOTIFY. The NOTIFY is seen by the inbox ACPI.sys and handled by ACPI!Notify. With the StorAhci driver, a worker thread is eventually created/woken and the callback routine that whatever driver registered is called and someone (runtime power?) invokes the ODDZ._PS0() method to turn the power on. [This is combined with StorPort UnitControl calls to inform the driver that power is being turned on or off so that the driver is aware of what the current power state is.]

Our problem is that when our driver registers for the notification callback, ACPI.sys does NOT call us.

This tells me that a) registrations can and do work (just not ours), b) NOTIFY events are being received by Windows from the BIOS ACPI and someone can map that to the correct ACPI method, and c) calling ODDZ._PS0() and ODDZ._PS3() do in fact turn the power on or off.

Note that if any of the support checks fail, then the notification callback isn’t ever invoked. (Which looks much like what we see in our driver…)

What I need to determine to solve our problem is:

  1. What is the runtime power subsystem (?) doing that we are not? (Is there a feature/capability set that’s the magic key?)
  2. Is the runtime power subsystem (?) preempting my attempt to register? (If someone else “owns” the method, is my register really working?)

I say runtime power subsystem because I don’t know for sure how is controlling the method when StorAhci is used but is somehow involved. It might be StorPort or ACPI or some other driver that either owns the ACPI method and/or invokes it.

Further debugging with windbg and stepping through assembly code it appears that there is a linked list that’s created with StorAhci that isn’t created with our driver. I’m currently assuming there’s a feature or capability that’s missing such that with our driver, the code acts like the power on/off feature isn’t supported. [Maybe it’s something that happens when the PoFxDevice is created, maybe it’s something passed in the SetCapabilities, maybe it’s something that CDROM does when registering with the runtime power subsystem.] We’ll eventually get to the bottom.

For anyone that cares:

The BIOS/ACPI NOTIFY is tied to ACPI!Notify. Upon Device Attention (user press of eject button), BIOS ACPI signals NOTIFY(ODDZ, 0x2) and Windows enters ACPI!Notify().

ACPI!Notify() calls ACPI!ACPINotifyHandler()

ACPI!ACPINotifyHandler() calls ACPI!OSNotifyDeviceWake()

ACPI!OSNotifyDeviceWake() allocates memory (ExAllocatePoolWithTag) to create an empty linked list. [Next == Prev == allocated memory addr]

ACPI!OSNotifyDeviceWake() calls ACPI!ACPIWakeRemoveDevicesAndUpdate()

ACPI!ACPIWakeRemoveDevicesAndUpdate() operates on some structures and possibly updates the linked list passed in.

  • For StorAhci, elements are added to the list
  • For our driver, the list remains unchanged

ACPI!OsNotifyDeviceWake() checks if the list is empty.

  • If empty, no callbacks are made
  • If not empty, calls ACPI!ACPIWakeDisableAsync() which eventually starts a thread that invokes ACPI!OSNotifyDeviceWakeCallBack() and along the way to completion, calls the registered callback.

I need to further decode what’s happening in ACPI!ACPIWakeRemoveDevicesAndUpdate() to see if I can find the condition that causes that linked list to be updated. If I can figure that out, I should be able to determine how to get our driver to set the same condition(s) and then we should see our callback invoked.

Again, thanks for the feedback/insights you have offered! If nothing else, it makes one rethink assumptions we’ve made and actually work through everything when trying to explain it to someone unfamiliar with the product/problem. This by itself often leads to “aha” moments along the way.

-Joe Thomas

// Joseph Thomas
// Principal Software Engineer
// Dot Hill Systems
// 2905 NorthWest Blvd., Suite 20
// Plymouth, MN 55441
// 763.226.2640

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Saturday, April 13, 2013 11:10 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Notify Callback function never called when using ACPI RegisterForDeviceNotifications

(Just saying: You have a complex problem, in a complex environment, involving many variables, and a little part of the system that few people have much experience with. Hoping for significant insight from an internet forum, even one as broad and deep as NTDEV can be at times, is a pretty big ask. Even if one of the MSFT dev owners reads your post, this is the kind of thing that takes time and investigation.)

OK, back to seeing if I can help in any way. I doubt it, but let’s try:

We need to cover a number of basic questions: How are identifying the ACPI device that “owns” the methods you’re calling. Do you have a separate (ACPI) device defined in the BIOS that has the methods you’re invoking, and a separate function driver that claims that device (hardware ID ACPI\xxxx)? Or are these methods associated with the larger device, such as the AHCI Controller. I think I know the answer, but we need to make sure.

I assume you have control of the BIOS? You can modify the BIOS as you see fit? If so… adding some debugging code should be pretty straight forward, no??

Aren’t ZPODs PnP devices that are dynamically detected on the AHCI bus? Assuming so, I admit to being a bit confused as to how they would be dynamically identified and associated with static methods in the BIOS.

Eh? You’re able to get the Notify if you configure your StorPort as a (non-raided) AHCI driver… or if you hack the Register/Notify into StorAHCI or whatever?

If that’s the case, I would guess THAT would point to hanging the notify on the wrote unit, don’t you think?

[quote]
When you try to layer RAID on top of that, you change a lot of the assumptions being made because you’re no longer presenting the “real” physical view [/quote]

Architecturally, are you stuffing all this RAIDing in the StorPort driver… consuming the AHCI controller and all the attached LUs and surfacing “virtual” devices out the top? If so, with respect, I’m not a fan of that architecture. It can be very messy and hard to debug.

Quite right. “people who write drivers” and “people who know something about ACPI” in the Windows world are typically sets that have a *very* small intersection. That’s because, in general in Windows, most things where people think “I can call an ACPI method to do/get/set this” are either (a) already abstracted and exported by the OS or (b) better done another way.

Now, let me hasten to add: I am *not* saying this is the case with which you’re trying to do, which – assuming I understand at all, you’re trying to add ZPOD support to down-level platforms – seems entirely reasonable and justified.

I’m not sure… I can try to do some spelunking.

But, IIRC, at its core it’s really pretty simple: It’s not like there’s a lookup table. You have a (ACPI) device namespace on which your driver is instantiated. ACPI serves as the bus driver for this device when it enumerates devices in its namespace. You register a callback with that (ACPI) device. When that device raises a Notify, you get the callback.

I’m not sure I’m helping at all… I wish I could help more. But for a Saturday morning, that’s the best I can come up with right now.

Peter
OSR

NTDEV is sponsored by OSR

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer