RE: Contiguous memory and dma transfer

You want to read up on MDL based DMA transfers in the DDK. Whatever you are
doing now, by the way, appears to be wrong, as all DMA transfers require the
use of either MDLs or special kernel memory buffers.

MDLs will avoid data copy operations and will give you contiguous physical
memory chunks for your user buffer. They do have size limitations (<64MB per
MDL,) but this doesn’t appear to be an issue for your use.

Mark Roddy
xxxxx@hollistech.com
www.hollistech.com
WindowsNT Windows 2000 Consulting Services

-----Original Message-----
From: Hesham Desokey [mailto:xxxxx@yahoo.com]
Sent: Wednesday, January 24, 2001 3:09 AM
To: NT Developers Interest List
Subject: [ntdev] Contigous memory and dma transfer

I am writing a pci wdm driver that communicate with pci and transfer data
using dma transfer as a master (the pci card has dma chip on it)
I have writen primary driver that transfer a little ammount of data (not
exceeded one page of memory) by direct communication with the dma chip and
it works successfully. But when the data become larger than the one page of
memory the problem of contigous memory appears.
so I need a method to force the memory to be contigouse.
note: The buffers where the data must be transfered must come from the
application ( speed issue - no copymemoy).


You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Hi,

I’m working with a driver that does large packet transfer that is upto 32K buffer in one DMA Xfer and i’m also using Contigious memory and I’m not facing any problems.

Ramit.
----- Original Message -----
From: Roddy, Mark
To: NT Developers Interest List
Sent: Wednesday, January 24, 2001 9:33 PM
Subject: [ntdev] RE: Contiguous memory and dma transfer

You want to read up on MDL based DMA transfers in the DDK. Whatever you are doing now, by the way, appears to be wrong, as all DMA transfers require the use of either MDLs or special kernel memory buffers.

MDLs will avoid data copy operations and will give you contiguous physical memory chunks for your user buffer. They do have size limitations (<64MB per MDL,) but this doesn’t appear to be an issue for your use.
Mark Roddy
xxxxx@hollistech.com
www.hollistech.com
WindowsNT Windows 2000 Consulting Services

-----Original Message-----
From: Hesham Desokey [mailto:xxxxx@yahoo.com]
Sent: Wednesday, January 24, 2001 3:09 AM
To: NT Developers Interest List
Subject: [ntdev] Contigous memory and dma transfer

I am writing a pci wdm driver that communicate with pci and transfer data using dma transfer as a master (the pci card has dma chip on it)
I have writen primary driver that transfer a little ammount of data (not exceeded one page of memory) by direct communication with the dma chip and it works successfully. But when the data become larger than the one page of memory the problem of contigous memory appears.
so I need a method to force the memory to be contigouse.
note: The buffers where the data must be transfered must come from the application ( speed issue - no copymemoy).


You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntdev as: xxxxx@wipro.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

thanks for you fast replay
but MDLS doesn’t appear to be contigouse beyond the page limit (4KB)
because my buffer is not larger than < 40 KB .
following what I have did:
1- take userbuffer using METHOD_NEITHER
2- Construct mdl object from virtual address passed
3- look mdl object
4- get physical address from the mdl object
5- use this physical address to transfer data from the dma on pci.
note: i increment this physical addres manually since i have assumed it is contigouse memory
that all what I have did but I have results of uncontigouse memory
thanks
bye

----- Original Message -----
From: Roddy, Mark
To: NT Developers Interest List
Sent: Wednesday, January 24, 2001 6:03 PM
Subject: [ntdev] RE: Contiguous memory and dma transfer

You want to read up on MDL based DMA transfers in the DDK. Whatever you are doing now, by the way, appears to be wrong, as all DMA transfers require the use of either MDLs or special kernel memory buffers.

MDLs will avoid data copy operations and will give you contiguous physical memory chunks for your user buffer. They do have size limitations (<64MB per MDL,) but this doesn’t appear to be an issue for your use.
Mark Roddy
xxxxx@hollistech.com
www.hollistech.com
WindowsNT Windows 2000 Consulting Services

-----Original Message-----
From: Hesham Desokey [mailto:xxxxx@yahoo.com]
Sent: Wednesday, January 24, 2001 3:09 AM
To: NT Developers Interest List
Subject: [ntdev] Contigous memory and dma transfer

I am writing a pci wdm driver that communicate with pci and transfer data using dma transfer as a master (the pci card has dma chip on it)
I have writen primary driver that transfer a little ammount of data (not exceeded one page of memory) by direct communication with the dma chip and it works successfully. But when the data become larger than the one page of memory the problem of contigous memory appears.
so I need a method to force the memory to be contigouse.
note: The buffers where the data must be transfered must come from the application ( speed issue - no copymemoy).


You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

You are currently subscribed to ntdev as: xxxxx@yahoo.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

This is a common mistake - using physical addresses to program a DMA
controller - and does NOT work under Windows NT.

Oh, you can find systems where it will work, but then you move to a system
with hardware map registers and WHAM! Your driver doesn’t work right *at
all*.

On NT 4.0 use the IoAllocateAdapterChannel/IoMapTransfer mechanism.

On Windows 2000 use the GetScatterGatherList function from your DMA_ADAPTER.
They will CONSTRUCT the list of LOGICAL addresses you should use. Yes, on
some systems they will be physical addresses. On others they will not.

For example, if you have a peripheral bus with an address size smaller than
that needed to access physical memory, you won’t work right. This includes
a 32 bit PCI card on a machine with > 4GB of memory. It will include
machines where the peripheral bus is interesting and/or complicated (and
that is a function of the bridge chip.)

As for your observation that MDLs don’t appear to be physically contiguous,
that is often the case, although it depends upon how the memory was
allocated. Virtually contiguous memory need not be physically contiguous.
That’s the whole point of the DMA model.

Hence, Mark’s original point - study up on the DMA model - is on point.

Regards,

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

----- Original Message -----
From: “Hesham Desokey”
To: “NT Developers Interest List”
Sent: Thursday, January 25, 2001 6:52 AM
Subject: [ntdev] RE: Contiguous memory and dma transfer

> thanks for you fast replay
> but MDLS doesn’t appear to be contigouse beyond the page limit (4KB)
> because my buffer is not larger than < 40 KB .
> following what I have did:
> 1- take userbuffer using METHOD_NEITHER
> 2- Construct mdl object from virtual address passed
> 3- look mdl object
> 4- get physical address from the mdl object
> 5- use this physical address to transfer data from the dma on pci.
> note: i increment this physical addres manually since i have assumed it
> is contigouse memory
> that all what I have did but I have results of uncontigouse memory
> thanks
> bye
>
>
> ----- Original Message -----
>
> From: Roddy, mailto:xxxxx Mark
> To: NT Developers Interest List mailto:xxxxx
> Sent: Wednesday, January 24, 2001 6:03 PM
> Subject: [ntdev] RE: Contiguous memory and dma transfer
>
> You want to read up on MDL based DMA transfers in the DDK. Whatever you
> are doing now, by the way, appears to be wrong, as all DMA transfers
> require the use of either MDLs or special kernel memory buffers.
>
> MDLs will avoid data copy operations and will give you contiguous
> physical memory chunks for your user buffer. They do have size
> limitations (<64MB per MDL,) but this doesn’t appear to be an issue for
> your use.
>
> Mark Roddy
> xxxxx@hollistech.com
> www.hollistech.com
> WindowsNT Windows 2000 Consulting Services
>
> -----Original Message-----
> From: Hesham Desokey [mailto:xxxxx@yahoo.com]
> Sent: Wednesday, January 24, 2001 3:09 AM
> To: NT Developers Interest List
> Subject: [ntdev] Contigous memory and dma transfer
>
>
> I am writing a pci wdm driver that communicate with pci and transfer
> data using dma transfer as a master (the pci card has dma chip on it)
> I have writen primary driver that transfer a little ammount of data (not
> exceeded one page of memory) by direct communication with the dma chip
> and it works successfully. But when the data become larger than the one
> page of memory the problem of contigous memory appears.
> so I need a method to force the memory to be contigouse.
> note: The buffers where the data must be transfered must come from the
> application ( speed issue - no copymemoy).
>
> —
> You are currently subscribed to ntdev as: xxxxx@stratus.com
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
> —
> You are currently subscribed to ntdev as: xxxxx@yahoo.com
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
> —
> You are currently subscribed to ntdev as: xxxxx@osr.com
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com</mailto:xxxxx></mailto:xxxxx>

Basically, Mark told you right – Read the DDK, and Viscarola & Mason, and
Baker, because you are doing it wrong, and hove no understanding on how to
get there.

First you cannot “increment this physical address manually” because it is
not necessarily contiguous. You can however, call IoMapTransfer to get the
next contiguous chunk, which sounds like what you want to be doing.
IoMapTransfer will return the physical address of the next chunk and it’s
length.

Second, if the buffer is of any size (> 256), the best would be to use
METHOD_IN/OUT_DIRECT since it will give you an MDL for the buffer in the
OutBuffer parameter of DeviceIoControl. If you use Scatter/Gather you simply
call IoMapTransfer (or GetScatterGatherList) to generate the SG list and
pass it to your hardware.

Gary G. Little
Sr. Staff Engineer
Broadband Storage, LLC
glittle#broadstor.com
glittle#delphieng.com

-----Original Message-----
From: Hesham Desokey [mailto:xxxxx@yahoo.com]
Sent: Thursday, January 25, 2001 6:53 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Contiguous memory and dma transfer

thanks for you fast replay
but MDLS doesn’t appear to be contigouse beyond the page limit (4KB)
because my buffer is not larger than < 40 KB .
following what I have did:
1- take userbuffer using METHOD_NEITHER
2- Construct mdl object from virtual address passed
3- look mdl object
4- get physical address from the mdl object
5- use this physical address to transfer data from the dma on pci.
note: i increment this physical addres manually since i have assumed it is
contigouse memory
that all what I have did but I have results of uncontigouse memory
thanks
bye

----- Original Message -----

From: Roddy, mailto:xxxxx Mark
To: NT Developers mailto:xxxxx Interest List
Sent: Wednesday, January 24, 2001 6:03 PM
Subject: [ntdev] RE: Contiguous memory and dma transfer

You want to read up on MDL based DMA transfers in the DDK. Whatever you are
doing now, by the way, appears to be wrong, as all DMA transfers require the
use of either MDLs or special kernel memory buffers.

MDLs will avoid data copy operations and will give you contiguous physical
memory chunks for your user buffer. They do have size limitations (<64MB per
MDL,) but this doesn’t appear to be an issue for your use.

Mark Roddy
xxxxx@hollistech.com
www.hollistech.com
WindowsNT Windows 2000 Consulting Services

-----Original Message-----
From: Hesham Desokey [mailto:xxxxx@yahoo.com]
Sent: Wednesday, January 24, 2001 3:09 AM
To: NT Developers Interest List
Subject: [ntdev] Contigous memory and dma transfer

I am writing a pci wdm driver that communicate with pci and transfer data
using dma transfer as a master (the pci card has dma chip on it)
I have writen primary driver that transfer a little ammount of data (not
exceeded one page of memory) by direct communication with the dma chip and
it works successfully. But when the data become larger than the one page of
memory the problem of contigous memory appears.
so I need a method to force the memory to be contigouse.
note: The buffers where the data must be transfered must come from the
application ( speed issue - no copymemoy).


You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: xxxxx@yahoo.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: xxxxx@delphieng.com
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com</mailto:xxxxx></mailto:xxxxx>

>following what I have did:

1- take userbuffer using METHOD_NEITHER
2- Construct mdl object from virtual address passed

Not necessary, just use DO_DIRECT_IO and the IO manager will do this for you
(Irp->MdlAddress).

4- get physical address from the mdl object
5- use this physical address to transfer data from the dma on pci.

And these steps must be done by IoMapTransfer or the WDM
GetScatterGatherList, NOT by any other means like looking to MDL tail
directly.

Max


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

I know all this stuff about IoMapTransfer (which is obselete)
but I need to talk to the hardware directly using physical addresses of user
mode allocated memory after allocating it.
note: the code work fine if id the user mode buffers devided into small
chunks.
I need to know if there is something to map the user mode buffer to
contigouse locked memory or not, without go deeply in details of wdm DMA
techniques.
thanks for you rupport
Eng. Hesham Desokey
Design Engineer
R and D Medical Devision IE
----- Original Message -----
From: “Maxim S. Shatskih”
To: “NT Developers Interest List”
Sent: Friday, January 26, 2001 6:36 PM
Subject: [ntdev] RE: Contiguous memory and dma transfer

> >following what I have did:
> >1- take userbuffer using METHOD_NEITHER
> >2- Construct mdl object from virtual address passed
>
> Not necessary, just use DO_DIRECT_IO and the IO manager will do this for
you
> (Irp->MdlAddress).
>
> >4- get physical address from the mdl object
> >5- use this physical address to transfer data from the dma on pci.
>
> And these steps must be done by IoMapTransfer or the WDM
> GetScatterGatherList, NOT by any other means like looking to MDL tail
> directly.
>
> Max
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@yahoo.com
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

_________________________________________________________

Do You Yahoo!?

Get your free @yahoo.com address at http://mail.yahoo.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

I believe the simple answer you seek is “No”.
Only a driver is capable of allocating contiguous memory, and there are no
guarantees that an allocation request will succeed.

Regards,

Paul Bunn, UltraBac.com, 425-644-6000
Microsoft MVP - WindowsNT/2000
http://www.ultrabac.com

-----Original Message-----
From: Hesham Desoky [mailto:xxxxx@yahoo.com]
Sent: Sunday, January 28, 2001 1:59 PM
To: NT Developers Interest List
Subject: [ntdev] RE: Contiguous memory and dma transfer

I know all this stuff about IoMapTransfer (which is obselete)
but I need to talk to the hardware directly using physical addresses of user
mode allocated memory after allocating it.
note: the code work fine if id the user mode buffers devided into small
chunks.
I need to know if there is something to map the user mode buffer to
contigouse locked memory or not, without go deeply in details of wdm DMA
techniques.


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Hesham,

You might look at the mapmem sample in the NT 4.0
DDK. The technique shown there will allow you to create
a contiguous memory area in kernel mode, then map it to
user mode. for Windows NT/2000.

Don Burn
Windows 2000 Device Driver and Filesystem consulting.

----- Original Message -----
From: “Hesham Desoky”
To: “NT Developers Interest List”
Sent: Sunday, January 28, 2001 4:59 PM
Subject: [ntdev] RE: Contiguous memory and dma transfer

> I know all this stuff about IoMapTransfer (which is obselete)
> but I need to talk to the hardware directly using physical addresses of
user
> mode allocated memory after allocating it.
> note: the code work fine if id the user mode buffers devided into small
> chunks.
> I need to know if there is something to map the user mode buffer to
> contigouse locked memory or not, without go deeply in details of wdm DMA
> techniques.
> thanks for you rupport
> Eng. Hesham Desokey
> Design Engineer
> R and D Medical Devision IE


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

woooooooow What a fast replay

where can I found more infos about wdm DMA Trasnsfer
the win2k DDK is not clear in this topic
thanks

----- Original Message -----
From: “Paul Bunn”
To: “NT Developers Interest List”
Sent: Monday, January 29, 2001 12:05 AM
Subject: [ntdev] RE: Contiguous memory and dma transfer

> I believe the simple answer you seek is “No”.
> Only a driver is capable of allocating contiguous memory, and there are no
> guarantees that an allocation request will succeed.
>
> Regards,
>
> Paul Bunn, UltraBac.com, 425-644-6000
> Microsoft MVP - WindowsNT/2000
> http://www.ultrabac.com
>
>
> -----Original Message-----
> From: Hesham Desoky [mailto:xxxxx@yahoo.com]
> Sent: Sunday, January 28, 2001 1:59 PM
> To: NT Developers Interest List
> Subject: [ntdev] RE: Contiguous memory and dma transfer
>
>
> I know all this stuff about IoMapTransfer (which is obselete)
> but I need to talk to the hardware directly using physical addresses of
user
> mode allocated memory after allocating it.
> note: the code work fine if id the user mode buffers devided into small
> chunks.
> I need to know if there is something to map the user mode buffer to
> contigouse locked memory or not, without go deeply in details of wdm DMA
> techniques.
>
> —
> You are currently subscribed to ntdev as: xxxxx@yahoo.com
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

_________________________________________________________

Do You Yahoo!?

Get your free @yahoo.com address at http://mail.yahoo.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Don
thanks for your valuable information, I will do this now.
but I need a source to increase my knowledge in DDK, book literatur …
thanks

----- Original Message -----
From: “Don Burn”
To: “NT Developers Interest List”
Sent: Monday, January 29, 2001 12:16 AM
Subject: [ntdev] RE: Contiguous memory and dma transfer

> Hesham,
>
> You might look at the mapmem sample in the NT 4.0
> DDK. The technique shown there will allow you to create
> a contiguous memory area in kernel mode, then map it to
> user mode. for Windows NT/2000.
>
> Don Burn
> Windows 2000 Device Driver and Filesystem consulting.
>
> ----- Original Message -----
> From: “Hesham Desoky”
> To: “NT Developers Interest List”
> Sent: Sunday, January 28, 2001 4:59 PM
> Subject: [ntdev] RE: Contiguous memory and dma transfer
>
>
> > I know all this stuff about IoMapTransfer (which is obselete)
> > but I need to talk to the hardware directly using physical addresses of
> user
> > mode allocated memory after allocating it.
> > note: the code work fine if id the user mode buffers devided into small
> > chunks.
> > I need to know if there is something to map the user mode buffer to
> > contigouse locked memory or not, without go deeply in details of wdm DMA
> > techniques.
> > thanks for you rupport
> > Eng. Hesham Desokey
> > Design Engineer
> > R and D Medical Devision IE
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@yahoo.com
> To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

_________________________________________________________

Do You Yahoo!?

Get your free @yahoo.com address at http://mail.yahoo.com


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Hesham,

On books there are three excellent ones (I teach Windows 2000/NT
device driver programming so I get this question alot):

Title: “Developing Windows NT Device Drivers - A Programmer’s Handbook”
Authors: Edward N. Dekker & Joseph M Newcomer
Publisher: Addison-Wesley
ISBN: 0-201-69590-1

This book uses a layered approach, i.e. look at all the problems from a high
level, then look at the problem from a more detailed view, some people love
this approach others don’t. The feature I like the most about the book is
the
organization of the later chapters, where everything you wanted to know
about
a subject is well laid out in one place. The other outstanding item in this
book is
the index, look for something there and it will point you to the real meat.

Title: “Windows NT Device Driver Development”
Authors: Peter Viscarola and W. Anthony Mason
Publisher: MacMillan Technical
ISBN: 1-57870-058-2

Since I learned NT device driver writing from the authors of this book, I’m
most
comfortable with it. This is the book with the highest density of data per
page, and
explanations you can find no where else including the DDK documentation.
This book
does assumes some knowledge of kernel programming.

Title: “Programming the Microsoft Windows Driver Model”
Author: Walter Oney
Publisher: Microsoft Press
ISBN: 0-7356-0588-2

This is definitely the best book on WDM driver programming out there.

Don Burn
Windows 2000 Device Driver and Filesystem consulting.

----- Original Message -----
From: “Hesham Desoky”
To: “NT Developers Interest List”
Sent: Sunday, January 28, 2001 5:32 PM
Subject: [ntdev] RE: Contiguous memory and dma transfer

> Don
> thanks for your valuable information, I will do this now.
> but I need a source to increase my knowledge in DDK, book literatur …
> thanks
>


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com

> I know all this stuff about IoMapTransfer (which is obselete)

but I need to talk to the hardware directly using physical addresses of
user
mode allocated memory after allocating it.

This is for what IoMapTransfer and GetScatterGatherList are exactly
intended.

I need to know if there is something to map the user mode buffer to
contigouse locked memory or not, without go deeply in details of wdm DMA
techniques.

Sorry, but when you tell some physical memory addresses for your hardware -
then this is DMA, and thus you must use the OS’s DMA support for it.

Max


You are currently subscribed to ntdev as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntdev-$subst(‘Recip.MemberIDChar’)@lists.osr.com