About MDL

OSR_Community_User · March 21, 2003, 12:23am

Hi,all
I am a newbie to NT kernel device develop,and I am confused about MDL (Memory descriptor list).

the difference between MDL and MDL chain.can I build directly a MDL chain?
Can i define a instance of MDL and then map it to some memory?How?
ExAllocatePool VS IoAllocateMdl ?
when to use MmBuildMdlForNonPagedPool?
use of MmInitializeMdl?

Maybe too many questions,
thanks

Jamie_Hanrahan · March 21, 2003, 1:02am

> 1. the

difference between MDL and MDL chain.can I build directly a
MDL chain?

An MDL describes a single virtually contiguous buffer and associates it with the (probably, but not necessarily) series of physical page numbers which “contain” those virtual addresses.

In normal use (if we’re not building our own MDLs) the fact that the MDL exists is also evidence that the buffer is locked into those physical pages (“pinned” is the usual Unix term) so that it can’t move until it’s unlocked. When you complete an IO that has an MDL attached to its IRP, the buffer will be unlocked and the MDL deallocated, “all at once” from our point of view.

An MDL chain is simply multiple MDLs in a singly-linked list. Each describes a different virtually contiguous buffer.

We normally don’t see MDL chains coming to us with IRPs from normal application IO requests, because the normal application IO APIs only permit the specification of a single buffer that will be set up for “direct IO”. MDL chains are usually (I hesitate to say “always”, but certainly “usually”) built by other drivers. The NDIS stack uses

The first MDL in an MDL chain is normally pointed to by the MdlAddress pointer in an IRP. I don’t think there’s provision for MDL chains not pointed to by an IRP. The data structures certainly allow it, but there’s nothing that would automatically

Can you build your own MDL chain? Sure - just call IoAllocateMdl with a non-null Irp pointer, where the Irp already has an MDL that it points to. Your new MDL gets added to the tail of the list, or to the head if you specified False for the SecondaryBuffer argument.

Can i define a instance of MDL and then map it to some memory? How?

One of my pet topics. The “how” depends on where the memory came from. If it’s originally a per-process address you use IoAllocateMdl, then MmProbeAndLockPages. Same if it’s in pageable system-wide memory like the paged pool. If it’s a system-space address for something in nonpaged pool, you can use IoAllocateMdl and then MmBuildMdlForNonPagedPool.

ExAllocatePool VS IoAllocateMdl

Pools are like heaps. ExAllocatePool allocates a chunk of “heap” storage. The result is a pointer to memory that you can dereference and store stuff in, just like the result of HeapAlloc out in user mode (or “malloc” in Unix terms).

IoAllocateMdl allocates space (from the nonpaged pool, though we’re not supposed to know that) for an MDL, initializes its header and various field members, and links it to an IRP if the Irp pointer is non-null. It does not however fill in the Pfn array that follows the fixed-szied portion of the MDL. That’s the job of MmProbeAndLockPages or MmBuildMdlForNonPagedPool.

when to use MmBuildMdlForNonPagedPool?

When you have an MDL that you have previously allocated and initialized with IoInitializeMdl, or allocated some other way and initialized with MmInitializeMdl - and which was initialized with the virtual address of a buffer that was previously allocated out of nonpaged pool. It will actually work on any nonpageable memory, but we’re not supposed to rely on that.

use of MmInitializeMdl?

Unnecessary if you’ve used IoAllocateMdl. However if you were allocating your own MDLs with ExAllocatePool, MmInitializeMdl would be a likely next step.

— Jamie Hanrahan
Azius Developer Training http://www.azius.com/
Kernel Mode Systems http://www.cmkrnl.com/
Windows Driver Consulting and Training

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Comstar Yan
Sent: Thursday, March 20, 2003 21:21
To: NT Developers Interest List
Subject: [ntdev] About MDL

Hi,all
I am a newbie to NT kernel device develop,and I am
confused about MDL (Memory descriptor list). 1. the
difference between MDL and MDL chain.can I build directly a
MDL chain? 2. Can i define a instance of MDL and then map it
to some memory?How? 3. ExAllocatePool VS IoAllocateMdl ? 4.
when to use MmBuildMdlForNonPagedPool? 5. use of MmInitializeMdl?

Maybe too many questions,
thanks
b嫮绠范\够?v诘譲葏僧y韬{.n?壏瑉wZnV闅奫h曟痾{]z遯鍔薼⑹

Jamie_Hanrahan · March 21, 2003, 3:11am

(Dang Outlook processing ctrl-Enter as “send”…)

I wrote:

Can you build your own MDL chain? Sure - just call
IoAllocateMdl with a non-null Irp pointer, where the Irp
already has an MDL that it points to. Your new MDL gets
added to the tail of the list, or to the head if you
specified False for the SecondaryBuffer argument.

Um, with the very large caveat that False here means “just make this one
the head”. It does NOT maintain the chain. i.e. if you say
SecondaryBuffer=False and there was already an Mdl chain, you’ve lost
the chain! So you can either create the first and (at the moment) only
MDL (SecondaryBuffer=False), or add the new MDL to the end of the chain
(SecondaryBuffer=True). You can’t insert a new one at the head of the
existing chain with this API. Or any other that I’ve seen.

The NDIS stack uses

…uses a chain of MDLs, NOT attached to an IRP, as a way to hand a
“complex buffer” to a miniport driver. This allows the data being sent
by the top-level client, the “payload”, to be wrapped by successive
protocol layers, each protocol header described by its own MDL, without
having to copy anything around in memory to prepend payloads to headers.
Clever programming of a scatter-gather-capable DMA controller, if
available, allows the NIC to just collect all these pieces together and
send it down the wire – without it ever having to be contiguous
virtually, let alone physically.

I don’t think there’s
provision for MDL chains not pointed to by an IRP. The data
structures certainly allow it, but there’s nothing that would
automatically

…that would automatically take the chain apart, as IoCompleteRequest
will for the chain of MDLs on an IRP. MmUnlockPages won’t do it.
Obviously NDIS has its own code to take the MDL chain apart and unlock
all the buffers.

(I think that’s everything. I’m absolutely certain that someone will
correct me if there are any more goofs. *grin* )

— Jamie Hanrahan
Azius Developer Training http://www.azius.com/
Windows Driver Consulting and Training

Maxim_S_Shatskih · March 22, 2003, 2:17pm

MDL is a structure with a tail, the structure itself describes some memory range, while its tail is an array of ULONGs which holds the physical page numbers for pages backing this memory range.

MDL chain is a list of MDLs linked by ->Next field. Some stacks like TDI supports MDL chains in Irp->MdlAddress. Others like storage - do not.
MDL chains are useful for “write gather” or “read scatter” functionality, if the data transfer area is not contiguous. They are usual in networking, since the networking stack needs to add headers to the packet as it travels down, so, it adds the MDLs to the beginning of the list.

ExAllocatePool allocates memory, while IoAllocateMdl allocates a descriptor for this memory.

MmInitializeMdl is used only if the space for MDL is allocated as a part of some larger structure. This space must be at least MmSizeOfMdl size, not sizeof(MDL).

To build the MDL properly, you must first call IoAllocateMdl or MmInitializeMdl.
Then, if the memory is guaranteed to be nonpaged - nonpaged pool alloc or the nonpaged data section of the module - call MmBuildMdlForNonPagedPool.
Otherwise, you must call MmProbeAndLockPages, which must be called from inside the __try/__except block to handle errors. MmProbeAndLockPages requires the corresponding call to MmUnlockPages before IoFreeMdl. It is also safe to call MmProbeAndLockPages for nonpaged memory, though there are talks that this can cease in next NT versions. The talks are hard to believe, since this will result in inability to call ZwReadFile on nonpaged memory.

MDLs are used a) for DMA by passing them to DMA routines b) to build another MDLs which describe the sub-ranges by IoBuildPartialMdl c) if there is a need to access the data by the CPU - map them to the system memory by MmGetSystemAddressForMdlSafe. Unmap is done automatically by MmUnlockPages.MmGetSystemAddressForMdlSafe is a no-op for nonpaged MDLs.

Now notes on the MDL which comes to you from the above in Irp-MdlAddress. The IO subsystem have already fully initialized it. You can pass it to DMA, or call IoBuildPartialMdl, or call MmGetSystemAddressForMdlSafe without any initialization. You also must not call MmUnlockPages or IoFreeMdl on it - this will be done in IoCompleteRequest.

Max

----- Original Message -----
From: Comstar Yan
To: NT Developers Interest List
Sent: Friday, March 21, 2003 8:21 AM
Subject: [ntdev] About MDL

Hi,all
I am a newbie to NT kernel device develop,and I am confused about MDL (Memory descriptor list).

the difference between MDL and MDL chain.can I build directly a MDL chain?
Can i define a instance of MDL and then map it to some memory?How?
ExAllocatePool VS IoAllocateMdl ?
when to use MmBuildMdlForNonPagedPool?
use of MmInitializeMdl?

Maybe too many questions,
thanks
b嫮绠范\够?v诘譲善)秺佺id韬{.n?壏瑉wZnV闅奫h曟痾{]z齯弋F?-矉+