Physical address of contiguous buffer

zvivered

Hello,

I'm using the following code to allocate a 28MB of contiguous buffer. This buffer is used for DMA from FGPA to RAM.

Due to limitation/bug in the firmware, the MSBit (#31) of the physical address must be 0.

Can you please advise if I can some how allocate buffer with bit #31 = 0 ?

Thank you,
Zvika

/**********************************************************************************/
NTSTATUS AllocateContinuousPhysicalMemory(IN PDEVICE_EXTENSION devExt, int Channel, int BufferSize)
{
WDF_DMA_ENABLER_CONFIG dmaConfig;
NTSTATUS status;

//
// Configure the DMA  object
//

WDF_DMA_ENABLER_CONFIG_INIT(&dmaConfig,
    WdfDmaProfilePacket64,
    COMMON_BUFFER_SIZE);

status = WdfDmaEnablerCreate(devExt->Device,
    &dmaConfig,
    WDF_NO_OBJECT_ATTRIBUTES,
    &devExt->DmaEnabler[Channel]);
if (!NT_SUCCESS(status))
{
    KdPrint(("WdfDmaEnblerCreate failed: %08X\n", status));
    return status;
}

status = **WdfCommonBufferCreate**(devExt->DmaEnabler[Channel],
    COMMON_BUFFER_SIZE,
    WDF_NO_OBJECT_ATTRIBUTES,
    &devExt->CommonBuffer[Channel]);

if (!NT_SUCCESS(status))
{
    KdPrint(("WdfCommonBufferCreate failed: %08X\n", status));
    return status;
}

devExt->KernelCommonBuffer[Channel] = WdfCommonBufferGetAlignedVirtualAddress(devExt->CommonBuffer[Channel]);
**devExt->PhysicalKernelCommonBuffer[Channel] = WdfCommonBufferGetAlignedLogicalAddress(devExt->CommonBuffer[Channel]);

**
RtlFillMemory(devExt->KernelCommonBuffer[Channel], BufferSize, 0x0);

devExt->CommonBufferMdl[Channel] = IoAllocateMdl(devExt->KernelCommonBuffer[Channel], BufferSize, FALSE, FALSE, NULL);

if (!devExt->CommonBufferMdl)
{
    KdPrint(("IoAllocateMdl failed.\n"));
    return STATUS_INSUFFICIENT_RESOURCES;
}

MmBuildMdlForNonPagedPool(devExt->CommonBufferMdl[Channel]);

KdPrint(("Channel %d: PhysicalDataAddressLow=%08x\n", Channel, devExt->PhysicalKernelCommonBuffer[Channel].LowPart));
KdPrint(("Channel %d: PhysicalDataAddressHigh=%08x\n", Channel, devExt->PhysicalKernelCommonBuffer[Channel].HighPart));

return STATUS_SUCCESS;

}

MBond2

I don't think you can fix this problem from the driver.

zvivered

Hi MBond2,

Thank you very much !

Best regards,
Zvika

Mark_Roddy

Hmmm, there always used to be a way to specify address limitations. In WDM the DEVICE_DESCRIPTION.DmaAddressWidth would do this. You could set that to 63, indicating that the MSB was out of range. There is very likely a way to do this through WDF as well. The common buffer allocation has to respect your DMA adapter properties. The WDF_DMA_ENABLER_CONFIG object has an AddressWidthOverride property that would appear to provide this capability. Then again I've never tried to do this so I don't know if it does what it appears to claim to do.

Tim_Roberts

You can set AddressWidthOverride in the WDF_DMA_ENABLER_CONFIG structure to 31.

If you need to work earlier than Windows 8, then the only available options for that fields are 64, 32, and 24, but the newer systems should accept any value.

MBond2

Do values other than 32 or 64 really work when there are different bus relative addresses?

Tim_Roberts

It shouldn't matter. We're talking about actual physical memory here, not about the device mapping.

Mark_Roddy

Just as an aside: NT has always supported device physical memory address restrictions, as ISA could not address anything outside of a 24bit address. And as we know from other discussions here, there are still plenty of NT systems with ISA devices.

MBond2

I must be missing something, but I thought the problem was that the device can't properly address memory when performing DMA. If that's the case, then the only thing that matters is the address from the point of view of the device?

Mark_Roddy

"the only thing that matters is the address from the point of view of the device?"

Yes but bus address == physical address in general, and when it doesn't that is a HAL problem not a driver problem. As long as the driver goes through the correct procedures, in this case by allocating a common buffer using a correctly configure dma adapter object, the platform guarantees that the 'bus logical address' will meet the dma adapter requirements.

All of which brings back memories of when common buffers were all fighting for 'low memory'.

MBond2

That's where I wonder if arbitrary lengths or patterns can really work?

Maybe I am being overly cautious, but I hesitate to say that this will work reliably

Tim_Roberts

The situation you're worried about (which can happen on high-end systems with odd I/O mappings) can be a problem when accessing DEVICE memory addresses, where the bus address is different than the physical address used by the CPU. But when a device needs access to the actual physical memory of the computer, there shouldn't be any remapping.

In 33 years of PC driver work, I've only encountered one system where physical address did not equal bus address, and that was the very, very strange machines that ran Windows NT 3.51 on an Alpha processor. Imagine, if you will, a system where PCI bus addresses had to be shifted left by 3, so consecutive DWORDs were actually separated by 32 bytes. That's why Alpha support didn't last very long.

MBond2

Ahh yes, NT 3.51 on Alpha. Those were fun.

zvivered

Hi All,

Thank you very much for your replies.
Before having the chance to try the solution, the firmware engineer fixed the problem.

Best regards,
Zvika

Mark_Roddy

Yeah but we still haven't finished arguing about this. :-)

Peter_Viscarola_(OSR)

In 33 years of PC driver work, I've only encountered one system where physical address did not equal bus address

Yes, historically correct. But "the times, they are a-changing" (I just made that up). IOMMUs are becoming increasingly common... so your Device Bus Logical Address won't BE your Physical Address for the purposes of DMA.

Peter

Mark_Roddy

The times might be a changing (and they are but quite obviously not in a good way) , as long as the driver goes through the documented procedures, the bus logical address is going to meet the device requirements, IOMMU or no IOMMU, and how that happens is a problem only for the HAL.

MBond2

The OP has landed in the right place in any case - fix the hardware

Peter_Viscarola_(OSR)

as long as the driver goes through the documented procedures, the bus logical address is going to meet the device requirements, IOMMU or no

IOMMU, and how that happens is a problem only for the HAL

Absolutely agree.

After 30 years of banging on the table telling people that Device Bus Logical Addresses are not necessarily Physical Addresses, it's nice to finally have something other than an esoteric example (and a driver verifier mode) that "proves" this.

Peter