Question on rebalancing of PCIe memory (exposed via BARs) at run time in a PCIe function driver

I am trying to figure out if KMDF and other kernel mode frameworks (NDIS, StorPort, WDDM) have a facility whereby the driver managing the device can handle rebalancing of PCIe resources specifically PCIe MMIO bars.

As an example, let us say I have a PCie function requesting 2 GB 64-bit BAR. BIOS has assigned a physical address and wrote the address in the BAR.
My driver for the PCIe function is happy (It does MmMapIOSpace and gets a CPU virtual address).

Now another PCIe device is hot plugged into the system and this PCIe function requests a large physical address space (physically contiguous space).
OS sees it does not have that much of a hole in 64-bit address space and wants to move my PCIe function to another 2 GB hole in the physical address space so it can satisfy the requirements.

This means my driver has to be notified of this change but the driver has to go through the whole AddDevice() so it can release the existing mappings and do a new MMapIoSpace.

Are there facilities in the KMDF and other kernel driver frameworks to let the driver know of these changes (obviously traffic has to be quiesced and other details are there)?
If so, what would they be?. For now, I am curious about KMDF driver framework.

Thanks,
RK

Sure. It’s always been supported, since Windows NT v3.1

In KMDF, you see this as a call to your EvtDeviceReleaseHardware closely followed by a call to EvtDevicePrepareHardware.

Google ”resource rebalancing WDF” or “resource rebalancing KMDF” you’ll see lots of info.

Peter

@“Peter_Viscarola_(OSR)” thanks a lot for the quick help.

OS sees it does not have that much of a hole in 64-bit address space

You’re confusing physical addressing and virtual addressing here. PCIe works at the physical level, where there’s nothing needed to get “physically contiguous” space. It’s all “physically contiguous”. There will ALWAYS be a large enough hole in physical address space. Even if the chipset only supports 48 bits, that’s still 281 terabytes of physical space. Although chipsets and the operating system do support it, there should NEVER be a need to move your BARs.

“Physically contiguous” only becomes an issue when you’re allocating RAM memory that will be used for DMA in a device without scatter/gather support, and that only affects RAM. It doesn’t affect devices and BARs.