KMDF 1.1 DMA with pre-allocated SGL, and WdfMemoryCreate == ExAllocatePool?

NTDEV
1

First, I have a mostly WDF driver in which I use ExAllocatePoolWithTag()
in my init code. This DDI isn’t in Win2K, but it can be worked around.
I’ve been wondering about replacing it with a WDF equivalent before I
tried to build this thing for Win2K anyway, so this a convenient time to
figure it out. It appears that WdfMemoryCreate is the WDF equivalent of
ExAllocatePool() (the doc even suggests that WDF_MEMORY_OBJECTS are
preferable because of the garbage collection on parent object delete) and
the variants of it. Is my understanding correct?

Second, I’m currently using the WDM DMA_OPERATIONS::AllocateCommonBuffer
and BuildScatterGatherList to pre-allocate an SGL buffer and use it at
runtime. This won’t even compile in the Windows 2000 build environment of
the 3790.1830 DDK, since BuildSGL is only available in XP and newer. I’ve
looked at the KMDF 1.1 docs, and I can see that I can use a
WdfCommonBuffer to replace the AllocateCommonBuffer call. However, I
can’t find any WDF function that appears to accept a pre-allocated SGL
buffer like BuildSGL does. Did I miss it, or is that not in WDF yet? Any
plans to put it in, if isn’t now?

Now to state the “problem we are really trying to solve”. As mentioned
before, this is a lab test harness. Among other stupid pet tricks we do
is a METHOD_BUFFERED command descriptor which has embedded user-mode
pointers that we IoAllocateMdl() and ProbeAndLock(). An aside: If
METHOD_OUT_DIRECT allowed returning data back to the calling app, we
wouldn’t need to do this, as we could then use two IOCTLs, a
METHOD_IN_DIRECT one that moved a lot of data from the device, in response
to a little data to the device, and a METHOD_OUT_DIRECT that moved a large
amount of data to the device and returned a small amount of status data
back. But I digress.

The reason I use BuildSGL is so that we can pre-allocate as many SGL
buffers as we need for the worst case of really big, overlapped transfers.
I know, it doesn’t make any sense to actually do command queuing with
giant transfers, but we have to test it anyway. Given that all I really
care about is not getting STATUS_INSUFFICIENT_RESOURCES or some other
failure when I call the routine, is there a WdfDmaTransactionYYY routine
that will (almost) always succeed, like BuildSGL will? Alternatively, is
there something else that I can use (MapTransfer() loop? That’s in Win2K,
but it seems like it’s diving deeper into the raw DDIs, instead of
simplifying my things) that won’t fail due to scarce map registers or
other resource limitations? Do keep in mind, this is an abusive lab test
harness, not a production driver. We aren’t ever going to release this on
the unsuspecting public, so techniques that don’t play nicely in general
purpose systems are OK here, though we’d like to keep stuff like this as
simple as possible while still meeting our requirements.

I guess I’m asking if it’s possible to do a pure WDF “Extreme DMA”
implementation that won’t randomly fail IOs due to resource limitations.
Is it?

Thanks,

Phil

Philip D. Barila
Seagate Technology LLC
(720) 684-1842

2

I’ll get you answers about the DMA questions. I know that the SG list is allocated when you call WdfDmaTransactionExecute(), not when you create the transaction.

ExAllocatePoolWithTag is available on win2k (KMDF uses it when running on win2k). If all you want is a pointer to use to initialize DMA, WDFMEMORY may be a bit overkill. WDFMEMORY is used for async I/O and memory lifetime issues associated with it. You can certainly use a WDFMEMORY if you want the auto cleanup. ExFreePoolWithTag is not available on win2k though.

If you are going to embed pointers in your structure, you will need to register an in caller callback via WdfDeviceInitSetIoInCallerContextCallback () to capture the pointers in the right context. WdfRequestProbeAndLockUserBufferForRead/Write will do the heavy work for you as well. Note that these calls can fail, but that is not a departure from WDM b/c ProbeAndLock and can fail as well.

d

– I can spell, I just can’t type.
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@seagate.com
Sent: Thursday, May 18, 2006 3:52 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] KMDF 1.1 DMA with pre-allocated SGL, and WdfMemoryCreate == ExAllocatePool?

First, I have a mostly WDF driver in which I use ExAllocatePoolWithTag() in my init code.? This DDI isn’t in Win2K, but it can be worked around.? I’ve been wondering about replacing it with a WDF equivalent before I tried to build this thing for Win2K anyway, so this a convenient time to figure it out.? It appears that WdfMemoryCreate is the WDF equivalent of ExAllocatePool() (the doc even suggests that WDF_MEMORY_OBJECTS are preferable because of the garbage collection on parent object delete) and the variants of it.? Is my understanding correct?
Second, I’m currently using the WDM DMA_OPERATIONS::AllocateCommonBuffer and BuildScatterGatherList to pre-allocate an SGL buffer and use it at runtime.? This won’t even compile in the Windows 2000 build environment of the 3790.1830 DDK, since BuildSGL is only available in XP and newer.? I’ve looked at the KMDF 1.1 docs, and I can see that I can use a WdfCommonBuffer to replace the AllocateCommonBuffer call.? However, I can’t find any WDF function that appears to accept a pre-allocated SGL buffer like BuildSGL does.? Did I miss it, or is that not in WDF yet?? Any plans to put it in, if isn’t now?
Now to state the “problem we are really trying to solve”.? As mentioned before, this is a lab test harness.? Among other stupid pet tricks we do is a METHOD_BUFFERED command descriptor which has embedded user-mode pointers that we IoAllocateMdl() and ProbeAndLock().? An aside:? If METHOD_OUT_DIRECT allowed returning data back to the calling app, we wouldn’t need to do this, as we could then use two IOCTLs, a METHOD_IN_DIRECT one that moved a lot of data from the device, in response to a little data to the device, and a METHOD_OUT_DIRECT that moved a large amount of data to the device and returned a small amount of status data back.? But I digress.
The reason I use BuildSGL is so that we can pre-allocate as many SGL buffers as we need for the worst case of really big, overlapped transfers.? I know, it doesn’t make any sense to actually do command queuing with giant transfers, but we have to test it anyway.? Given that all I really care about is not getting STATUS_INSUFFICIENT_RESOURCES or some other failure when I call the routine, is there a WdfDmaTransactionYYY routine that will (almost) always succeed, like BuildSGL will?? Alternatively, is there something else that I can use (MapTransfer() loop? That’s in Win2K, but it seems like it’s diving deeper into the raw DDIs, instead of simplifying my things) that won’t fail due to scarce map registers or other resource limitations?? Do keep in mind, this is an abusive lab test harness, not a production driver.? We aren’t ever going to release this on the unsuspecting public, so techniques that don’t play nicely in general purpose systems are OK here, though we’d like to keep stuff like this as simple as possible while still meeting our requirements.
I guess I’m asking if it’s possible to do a pure WDF “Extreme DMA” implementation that won’t randomly fail IOs due to resource limitations.? Is it?
Thanks,
Phil
Philip D. Barila
Seagate Technology LLC
(720) 684-1842

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer