64 bit addressing with WdfCommonBufferCreate

I am working on a DMA implementation with a device that does not support scatter-gather. The device does support 64 bit addressing. When I create my dma enabler, If I use WdfDmaProfilePacket or WdfDmaProfilePacket64, WdfCommonBufferCreate only returns buffers with a logical address in the first 4 GB of memory (my test machine has 16 GB of installed RAM). My device functions normally in this case. If I use WdfDmaProfileScatterGather64, I now get buffers with logical addresses > 16 GB. When my device tries to write to these bufferes I get random bsod.

What is the correct way to allocate contiguous memory buffers accessible by the device without being limited to memory in the 32bit address range? I have found lots of similar questions but nothing seems to explain WdfCommonBufferCreate returning buffers with addresses outside the physical installed RAM size.

Thanks!

Ok I figured it out. Probably to no one’s surprise the problem was that my device (which is an FPGA) was truncating write addresses to 32 bits. The random BSOD was therefore due to corrupting physical memory with writes to the wrong address.

So the answer is that it is apparently normal to get memory buffers as high as 0x42f4e6000 on a system with 16 GB RAM.

WdfDmaProfilePacket64, WdfCommonBufferCreate only returns buffers with a logical address in the first 4 GB of memory

That’s not what you should expect. SOMEthing is incorrectly configured somewhere. With a 64-bit profile, your buffers should not be limited to low-memory. Unless you have Driver Verifier enabled with DMA Verifier, then… I think you’ll see all low memory buffers. I think.

WdfCommonBufferCreate returning buffers with addresses outside the physical installed RAM size

Hmmmmm… If there’s an active IOMMU, you can definitely see this… right? Remember, the value you get back from WdfCommonBufferCreate is NOT a physical address… it’s a Device Bus Logical Address suitable for use with DMA to/from your device (only).

Peter

That’s not what you should expect. SOMEthing is incorrectly configured somewhere. With a 64-bit profile, your buffers should not be limited to low-memory. Unless you have Driver Verifier enabled with DMA Verifier, then… I think you’ll see all low memory buffers. I think.
Interesting. Someone else observed the same behavior I see in this thread:
https://community.osr.com/discussion/234733/allocating-a-huge-amount-of-dma-memory
Unless driver verifier automatically gets enabled, it is not enabled on my machine. My test machine is provisioned for kernel debugging with VS2017 with basically the default settings. I have not tried using Driver Verifier.

Remember, the value you get back from WdfCommonBufferCreate is NOT a physical address… it’s a Device Bus Logical Address suitable for use with DMA to/from your device (only).
Good point. I was thinking of the logical address as if it was a physical address but you are right, there is still the IOMMU between my device and the physical memory. I saw blue screens and I saw buffer addresses >16GB and I got worried.

Someone else observed the same behavior I see in this thread…

Well, the issue in that thread was different. That had to do with the AMOUNT of memory that could be allocated, not its addressing.

Still, the DMA APIs in general, and the WDF DMA APIs in particular, can sometimes result in behavior that folks might not expect. Let’s say some of what they do can be so “helpful” it becomes a little odd/esoteric.

I have not tried using Driver Verifier.

As a side note, you definitely want to try that… and KMDF verifier as well. Both well worth your time as they’ll help you locate potentially serious bugs.

Peter

Well, the issue in that thread was different. That had to do with the AMOUNT of memory that could be allocated, not its addressing.
Ah, I distinction I missed.

As a side note, you definitely want to try that… and KMDF verifier as well. Both well worth your time as they’ll help you locate potentially serious bugs.
I had a feeling you would say that :). I will test with driver verifier and KMDF verifier before going to production.

Thanks for all of your input!