WdfDmaProfileScatterGather behaves like WdfDmaProfilePacket

Phil_McHugh · November 25, 2009, 11:22am

Hi,

I am converting a legacy device driver (ie WinNT) to WDF so it can be
used on Vista onwards. The operation involves DMA of large buffers of data
to the device.

I get unexpected behaviour in the DMA.

My device is capable of hardware scatter gather DMA so I would like to use
that feature for performance. However when I set the DMA profile to
WdfDmaProfileScatterGather the call back always gets a single large
ScatterGather element for the size of the whole transaction. ie it seems
to be behaving as if I had configured WdfDmaProfilePacket.

My start up code looks like:

WDF_DMA_ENABLER_CONFIG_INIT( &dmaConfig,
WdfDmaProfileScatterGather, maxXfrLen );
status = WdfDmaEnablerCreate( hDevice, &dmaConfig,
WDF_NO_OBJECT_ATTRIBUTES, &pRWC->DmaEnablerHandle );

After executing windbg shows that the dmaconfig is setup as expected ie

0: kd> dt dmaConfig
Local var @ 0xf7909608 Type _WDF_DMA_ENABLER_CONFIG
+0x000 Size : 0x24
+0x004 Profile : 2 ( WdfDmaProfileScatterGather )
+0x008 MaximumLength : 0x400000
+0x00c EvtDmaEnablerFill : (null)
+0x010 EvtDmaEnablerFlush : (null)
+0x014 EvtDmaEnablerDisable : (null)
+0x018 EvtDmaEnablerEnable : (null)
+0x01c EvtDmaEnablerSelfManagedIoStart : (null)
+0x020 EvtDmaEnablerSelfManagedIoStop : (null)

But further down the line after I call WdfDmaTransactionExecute my
EvtProgramDmaFunction get a scatter gather list with one element which is
the size of the whole transaction.

ie:
0: kd> dt ScatterGather
Local var @ 0xf78c9e74 Type _SCATTER_GATHER_LIST*
0x85c50000
+0x000 NumberOfElements : 1
+0x004 Reserved : 1
+0x008 Elements : [0] _SCATTER_GATHER_ELEMENT

where the size of the only element is 0x3ed00 bytes - much too big an
element to be in a ‘real’ scatter gather list entry.

Its as if the DMA profile is WdfDmaProfilePacket instead of
WdfDmaProfileScatterGather. ie WDF is assuming my device cannot do
hardware DMA and so is using map registers to make the (almost certainly
non-contiguous) 0x3ed00 bytes appear contiguous.

Can any one help. Have I misunderstood the operation of scatter gather
within WDF or have I just got something wrong with my code.

Thanks for any advice.

Rgds

Phil McHugh
Newspaper Software Solutions
Agfa Graphics

Tel: +44 (0) 1525 850649
Mob: +44 (0) 7836 364655

DISCLAIMER

This e-mail and the information contained in it may be confidential,
privileged and protected by law. Access by the intended recipient only is
authorised. If you are not an intended recipient, please notify the
sender immediately and do not disclose, copy, or distribute the contents
to any other person or take any action in reliance on the contents. Any
liability arising from any third party acting on any information contained
in this e-mail is hereby excluded. No warranties or assurances are made
in relation to the content of this email and the information contained in
it. We cannot guarantee that this email is virus free and has not been
intercepted or amended.

Peter_Wieland · November 25, 2009, 11:43am

Where was the buffer you’re trying to send to the device allocated from? Is it application memory? A pool allocation? Or is it common buffer?

-p

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@agfa.com
Sent: Wednesday, November 25, 2009 8:22 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] WdfDmaProfileScatterGather behaves like WdfDmaProfilePacket

Hi,

I am converting a legacy device driver (ie WinNT) to WDF so it can be used on Vista onwards. The operation involves DMA of large buffers of data to the device.

I get unexpected behaviour in the DMA.

My device is capable of hardware scatter gather DMA so I would like to use that feature for performance. However when I set the DMA profile to WdfDmaProfileScatterGather the call back always gets a single large ScatterGather element for the size of the whole transaction. ie it seems to be behaving as if I had configured WdfDmaProfilePacket.

My start up code looks like:

WDF_DMA_ENABLER_CONFIG_INIT( &dmaConfig, WdfDmaProfileScatterGather, maxXfrLen );
status = WdfDmaEnablerCreate( hDevice, &dmaConfig, WDF_NO_OBJECT_ATTRIBUTES, &pRWC->DmaEnablerHandle );

After executing windbg shows that the dmaconfig is setup as expected ie

0: kd> dt dmaConfig
Local var @ 0xf7909608 Type _WDF_DMA_ENABLER_CONFIG
+0x000 Size : 0x24
+0x004 Profile : 2 ( WdfDmaProfileScatterGather )
+0x008 MaximumLength : 0x400000
+0x00c EvtDmaEnablerFill : (null)
+0x010 EvtDmaEnablerFlush : (null)
+0x014 EvtDmaEnablerDisable : (null)
+0x018 EvtDmaEnablerEnable : (null)
+0x01c EvtDmaEnablerSelfManagedIoStart : (null)
+0x020 EvtDmaEnablerSelfManagedIoStop : (null)

But further down the line after I call WdfDmaTransactionExecute my EvtProgramDmaFunction get a scatter gather list with one element which is the size of the whole transaction.

ie:
0: kd> dt ScatterGather
Local var @ 0xf78c9e74 Type _SCATTER_GATHER_LIST*
0x85c50000
+0x000 NumberOfElements : 1
+0x004 Reserved : 1
+0x008 Elements : [0] _SCATTER_GATHER_ELEMENT

where the size of the only element is 0x3ed00 bytes - much too big an element to be in a ‘real’ scatter gather list entry.

Its as if the DMA profile is WdfDmaProfilePacket instead of WdfDmaProfileScatterGather. ie WDF is assuming my device cannot do hardware DMA and so is using map registers to make the (almost certainly non-contiguous) 0x3ed00 bytes appear contiguous.

Can any one help. Have I misunderstood the operation of scatter gather within WDF or have I just got something wrong with my code.

Thanks for any advice.

Rgds

Phil McHugh
Newspaper Software Solutions
Agfa Graphics

Tel: +44 (0) 1525 850649
Mob: +44 (0) 7836 364655

DISCLAIMER

This e-mail and the information contained in it may be confidential, privileged and protected by law. Access by the intended recipient only is authorised. If you are not an intended recipient, please notify the sender immediately and do not disclose, copy, or distribute the contents to any other person or take any action in reliance on the contents. Any liability arising from any third party acting on any information contained in this e-mail is hereby excluded. No warranties or assurances are made in relation to the content of this email and the information contained in it. We cannot guarantee that this email is virus free and has not been intercepted or amended.
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Phil_McHugh · November 25, 2009, 12:30pm

Hi,

Its an application buffer as delivered to my EvtIoWrite callback.

I initialize the DMA transaction with:

ntstatus =
WdfDmaTransactionInitializeUsingRequest(pRWC->DmaTransactionHandle,
nextRequest,
AiEvtProgramDma,
WdfDmaDirectionWriteToDevice );

where nextRequest is the WDFREQUEST passed into the EvtIoWrite.

Rgds

Phil McHugh

xxxxx@lists.osr.com wrote on 25/11/2009 16:39:12:

Where was the buffer you?re trying to send to the device allocated
from? Is it application memory? A pool allocation? Or is it common
buffer?

-p

From: xxxxx@lists.osr.com [mailto:
xxxxx@lists.osr.com] On Behalf Of xxxxx@agfa.com
Sent: Wednesday, November 25, 2009 8:22 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] WdfDmaProfileScatterGather behaves like
WdfDmaProfilePacket

Hi,

I am converting a legacy device driver (ie WinNT) to WDF so it can
be used on Vista onwards. The operation involves DMA of large
buffers of data to the device.

I get unexpected behaviour in the DMA.

My device is capable of hardware scatter gather DMA so I would like
to use that feature for performance. However when I set the DMA
profile to WdfDmaProfileScatterGather the call back always gets a
single large ScatterGather element for the size of the whole
transaction. ie it seems to be behaving as if I had configured
WdfDmaProfilePacket.

My start up code looks like:

WDF_DMA_ENABLER_CONFIG_INIT( &dmaConfig,
WdfDmaProfileScatterGather, maxXfrLen );
status = WdfDmaEnablerCreate( hDevice, &dmaConfig,
WDF_NO_OBJECT_ATTRIBUTES, &pRWC->DmaEnablerHandle );

After executing windbg shows that the dmaconfig is setup as expected ie

0: kd> dt dmaConfig
Local var @ 0xf7909608 Type _WDF_DMA_ENABLER_CONFIG
+0x000 Size : 0x24
+0x004 Profile : 2 ( WdfDmaProfileScatterGather )
+0x008 MaximumLength : 0x400000
+0x00c EvtDmaEnablerFill : (null)
+0x010 EvtDmaEnablerFlush : (null)
+0x014 EvtDmaEnablerDisable : (null)
+0x018 EvtDmaEnablerEnable : (null)
+0x01c EvtDmaEnablerSelfManagedIoStart : (null)
+0x020 EvtDmaEnablerSelfManagedIoStop : (null)

But further down the line after I call WdfDmaTransactionExecute my
EvtProgramDmaFunction get a scatter gather list with one element
which is the size of the whole transaction.

ie:
0: kd> dt ScatterGather
Local var @ 0xf78c9e74 Type _SCATTER_GATHER_LIST*
0x85c50000
+0x000 NumberOfElements : 1
+0x004 Reserved : 1
+0x008 Elements : [0] _SCATTER_GATHER_ELEMENT

where the size of the only element is 0x3ed00 bytes - much too big
an element to be in a ‘real’ scatter gather list entry.

Its as if the DMA profile is WdfDmaProfilePacket instead of
WdfDmaProfileScatterGather. ie WDF is assuming my device cannot do
hardware DMA and so is using map registers to make the (almost
certainly non-contiguous) 0x3ed00 bytes appear contiguous.

Can any one help. Have I misunderstood the operation of scatter
gather within WDF or have I just got something wrong with my code.

Thanks for any advice.

Rgds

Phil McHugh
Newspaper Software Solutions
Agfa Graphics

Tel: +44 (0) 1525 850649
Mob: +44 (0) 7836 364655

DISCLAIMER

This e-mail and the information contained in it may be confidential,
privileged and protected by law. Access by the intended recipient
only is authorised. If you are not an intended recipient, please
notify the sender immediately and do not disclose, copy, or
distribute the contents to any other person or take any action in
reliance on the contents. Any liability arising from any third
party acting on any information contained in this e-mail is hereby
excluded. No warranties or assurances are made in relation to the
content of this email and the information contained in it. We
cannot guarantee that this email is virus free and has not been
intercepted or amended.
— NTDEV is sponsored by OSR For our schedule of WDF, WDM,
debugging and other seminars visit: http://www.osr.com/seminars To
unsubscribe, visit the List Server section of OSR Online at http:
//www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http:
//www.osronline.com/page.cfm?name=ListServer

Peter_Wieland · November 25, 2009, 2:00pm

You’re not specifying WdfDmaProfileScatterGather64. Is this a 32-bit system or a 64-bit system? Does this machine have more than 4 GB of physical memory in it?

I’ll agree it seems unlikely that the system gave your application 8 physically-contiguous pages, but it’s not impossible. Especially if the machine has a lot of memory in it. You could dump the MDL’s PfnArray and see if the page-frame numbers are sequential. That would imply the original buffer was contiguous.

What does !wdfkd.wdfdmaenabler say about your DMA enabler? How about !wdfkd.wdfdmatransaction?

-p

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@agfa.com
Sent: Wednesday, November 25, 2009 9:30 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] WdfDmaProfileScatterGather behaves like WdfDmaProfilePacket

Hi,

Its an application buffer as delivered to my EvtIoWrite callback.

I initialize the DMA transaction with:

ntstatus = WdfDmaTransactionInitializeUsingRequest(pRWC->DmaTransactionHandle,
nextRequest,
AiEvtProgramDma,
WdfDmaDirectionWriteToDevice );

where nextRequest is the WDFREQUEST passed into the EvtIoWrite.

Rgds

Phil McHugh

xxxxx@lists.osr.com mailto:xxxxx wrote on 25/11/2009 16:39:12:

> Where was the buffer you’re trying to send to the device allocated
> from? Is it application memory? A pool allocation? Or is it common buffer?
>
> -p
>
> From: xxxxx@lists.osr.com mailto:xxxxx [mailto:
> xxxxx@lists.osr.com mailto:xxxxx] On Behalf Of xxxxx@agfa.com mailto:xxxxx
> Sent: Wednesday, November 25, 2009 8:22 AM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] WdfDmaProfileScatterGather behaves like WdfDmaProfilePacket
>
>
> Hi,
>
> I am converting a legacy device driver (ie WinNT) to WDF so it can
> be used on Vista onwards. The operation involves DMA of large
> buffers of data to the device.
>
> I get unexpected behaviour in the DMA.
>
> My device is capable of hardware scatter gather DMA so I would like
> to use that feature for performance. However when I set the DMA
> profile to WdfDmaProfileScatterGather the call back always gets a
> single large ScatterGather element for the size of the whole
> transaction. ie it seems to be behaving as if I had configured
> WdfDmaProfilePacket.
>
> My start up code looks like:
>
> WDF_DMA_ENABLER_CONFIG_INIT( &dmaConfig,
> WdfDmaProfileScatterGather, maxXfrLen );
> status = WdfDmaEnablerCreate( hDevice, &dmaConfig,
> WDF_NO_OBJECT_ATTRIBUTES, &pRWC->DmaEnablerHandle );
>
> After executing windbg shows that the dmaconfig is setup as expected ie
>
>
> 0: kd> dt dmaConfig
> Local var @ 0xf7909608 Type _WDF_DMA_ENABLER_CONFIG
> +0x000 Size : 0x24
> +0x004 Profile : 2 ( WdfDmaProfileScatterGather )
> +0x008 MaximumLength : 0x400000
> +0x00c EvtDmaEnablerFill : (null)
> +0x010 EvtDmaEnablerFlush : (null)
> +0x014 EvtDmaEnablerDisable : (null)
> +0x018 EvtDmaEnablerEnable : (null)
> +0x01c EvtDmaEnablerSelfManagedIoStart : (null)
> +0x020 EvtDmaEnablerSelfManagedIoStop : (null)
>
> But further down the line after I call WdfDmaTransactionExecute my
> EvtProgramDmaFunction get a scatter gather list with one element
> which is the size of the whole transaction.
>
> ie:
> 0: kd> dt ScatterGather
> Local var @ 0xf78c9e74 Type _SCATTER_GATHER_LIST*
> 0x85c50000
> +0x000 NumberOfElements : 1
> +0x004 Reserved : 1
> +0x008 Elements : [0] _SCATTER_GATHER_ELEMENT
>
>
> where the size of the only element is 0x3ed00 bytes - much too big
> an element to be in a ‘real’ scatter gather list entry.
>
> Its as if the DMA profile is WdfDmaProfilePacket instead of
> WdfDmaProfileScatterGather. ie WDF is assuming my device cannot do
> hardware DMA and so is using map registers to make the (almost
> certainly non-contiguous) 0x3ed00 bytes appear contiguous.
>
> Can any one help. Have I misunderstood the operation of scatter
> gather within WDF or have I just got something wrong with my code.
>
> Thanks for any advice.
>
>
>
> Rgds
>
> Phil McHugh
> Newspaper Software Solutions
> Agfa Graphics
>
> Tel: +44 (0) 1525 850649
> Mob: +44 (0) 7836 364655
>
> DISCLAIMER
>
> This e-mail and the information contained in it may be confidential,
> privileged and protected by law. Access by the intended recipient
> only is authorised. If you are not an intended recipient, please
> notify the sender immediately and do not disclose, copy, or
> distribute the contents to any other person or take any action in
> reliance on the contents. Any liability arising from any third
> party acting on any information contained in this e-mail is hereby
> excluded. No warranties or assurances are made in relation to the
> content of this email and the information contained in it. We
> cannot guarantee that this email is virus free and has not been
> intercepted or amended.
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM,
> debugging and other seminars visit: http://www.osr.com/seminars To
> unsubscribe, visit the List Server section of OSR Online at http:
> //www.osronline.com/page.cfm?name=ListServer
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http:
> //www.osronline.com/page.cfm?name=ListServer — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer</mailto:xxxxx></mailto:xxxxx></mailto:xxxxx></mailto:xxxxx>

Phil_McHugh · November 26, 2009, 5:32am

Hi,

Problem solved…

Although I did not get an answer from the group the discussion changed my
direction of thought and lead to the solution.

Its was not an issue of WdfDmaProfileScatterGather vs
WdfDmaProfilePacket.

Instead the buffers were contiguous because I had not called
WdfDeviceInitSetIoType(DeviceInit, WdfDeviceIoDirect);

Therefore write IO defaults to WdfDeviceIoBuffered, ie the application
buffers are copied to a contiguous kernel buffer by the framework before
being passed to the driver.

Thanks for help given

Rgds

Phil McHugh

Doron_Holan · November 26, 2009, 11:01am

For your own knowledge, the framework is not copying the app’s buffer, the io manager is doing that. Also, this kernel buffer which holds the copy is not guaranteed to be physically contiguous, so you were getting lucky

d

From: xxxxx@agfa.com
Sent: Thursday, November 26, 2009 2:32 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] WdfDmaProfileScatterGather behaves like WdfDmaProfilePacket

Hi,

Problem solved…

Although I did not get an answer from the group the discussion changed my direction of thought and lead to the solution.

Its was not an issue of WdfDmaProfileScatterGather vs WdfDmaProfilePacket.

Instead the buffers were contiguous because I had not called WdfDeviceInitSetIoType(DeviceInit, WdfDeviceIoDirect);

Therefore write IO defaults to WdfDeviceIoBuffered, ie the application buffers are copied to a contiguous kernel buffer by the framework before being passed to the driver.

Thanks for help given

Rgds

Phil McHugh
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer