IOMMU fault in KMDF driver when pcie device dma reads the MmAllocateContiguousMemory buffer

Rocky_Marciano

Hi,

I am writing a KMDF driver for a PCIe device. Device has dma controller which can read/write system memory. Below are steps used
1. Allocated a contiguous buffer using MmAllocateContiguousMemory API. Get the physical address by MmGetPhysicalAddress()
2. Pass the physical address to device register.

When the device is making the access to passed physical address, iommu fault happens.
Do I need to configure iommu? If so which APIs are available in WDF framework to set iommu?

This is my initial experiment. Requirement is to directly read/write user space buffer from pcie device dma. For that I am planning to
1. IoAllocateMdl for user buffer
2. MmProbeAndLockPages
3. Get all the physical address, size list from the mdl and pass to pcie device

Please let me know if I am doing this is correct way.

Mark_Roddy

you need to use the kmdf dma apis and allocate a common buffer rather than
using mmallocatecontiguousmemory.
Mark Roddy

Peter_Viscarola_(OSR)

AND you need to post in the right forum: I suspect the forum labelled "Announcements and Administration" isn't EXACTLY the best possible spot for your post.

(Post moved to NTDEV)

Rocky_Marciano

Sorry for posting in wrong forum.

Thanks Mark. Understood the first part. But for reading/writing user space buffer directly from device, is there any iommu settings need to be done to allow device access this memory or just below steps are sufficient? In this scenario dma apis are not needed right?

1. IoAllocateMdl for user buffer
2. MmProbeAndLockPages
3. Get all the physical address, size list from the mdl and pass to pcie device

Peter_Viscarola_(OSR)

In this scenario dma apis are not needed right

Not right.

In Windows, you are (pretty much) never allowed to DMA from/to a Physical Address. This has been an invariant principle of Windows (NT) architecture since it was first released. And this isn't even unique to Windows.

The address you give to your DMA device needs to be a "device bus logical address" -- one that is subject to translation, such as by an IO MMU.

So... (a) You can allocate a common buffer as Mr. Roddy suggested, and use the value returned from WdfCommonBufferGetAlignedLogicalAddress to program your hardware, or (b) You can use the DMA APIs (Get an Enabler, Build a Transaction, etc...) and use the S/G List returned to you via those APIs to program your hardware.

That is the entire universe of your choices. There are no other alternatives.

Peter

Tim_Roberts

The scary thing is that, on many systems, cheating will work just fine, but there are servers and other high-end machines that do have extra hardware that does address rewriting for specialized purposes. If you don't do it correctly, you'll release a time bomb into the world.

Peter_Viscarola_(OSR)

Too true.

That's why enabling DMA Verification in Driver Verifier is always a good thing to do...

Peter

Rocky_Marciano

Thank you all. Appreciate your expert advice. Will use DMA APIs to pass S/G list for my device as user buffer can be in GBs

Scott_Noone_(OSR)

And don't forget to set the magic DmaRemappingCompatible registry value:

https://learn.microsoft.com/en-us/windows-hardware/drivers/pci/enabling-dma-remapping-for-device-drivers