Can any body tell me what is map register?
And what it is used?
DDK help tell us it works as mapping between device/logical address to physical address?
How does this mapping created?
And how many map register is needed for DMA transfer?
A “map register” is an abstract concept. On some platforms, this
abstraction is implemented by real registers on the machine, whose purpose
is to allow DMA to access addresses > 4GB. Essentially, it is seen as a
“page map” for DMA, which normally uses physical addresses, but the
mapping is done in the DMA hardware, not the CPU. High-end server chips
that expect to run 32-bit operating systems with 32-bit peripherals
require these mapping registers to access high-order memory.
There are a limited number of these registers, so if you find yourself on
one of these platforms, you will be told how many of these registers are
available to your device for a given transfer. This value is potentially
dynamic, based on how many active DMA transfers are going on. You have to
limit your DMA transfer to a number of pages that can be represented by
the available mapping registers.
Under other conditions, the DMA transfer does not actually happen to user
pages (for example, if the device does not implement scatter/gather DMA,
or the Driver Verifier is running with DMA checking enabled). There is a
“bounce buffer” which is a set of physically contiguous pages allocated in
the kernel, and the size of this buffer is indicated by giving a limited
number of “map register” values that limit your DMA transfer to the size
of this buffer. For input, the OS “lies” to you about the memory
location, telling you the address of the bounce buffer. After the input
completes, the data will be copied to the user space via MOVS-class
instructions. For output, the same thing happens, but the data is copied
from the user pages to the bounce buffer before writing, and again the OS
“lies” about the address of the buffer, giving the address of the bounce
buffer instead of the address of the first address of the user buffer.
By applying the concept uniformly across all platforms, you can write a
driver whose binary image will work with any of the existing platforms,
and even with the Driver Verifier doing its thing.
joe
Can any body tell me what is map register?
And what it is used?
DDK help tell us it works as mapping between device/logical address to
physical address?How does this mapping created?
And how many map register is needed for DMA transfer?
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
hi, Joe
What you said bounce buffer, is the Common buffer concept, right?
BTW, what’s the content in the map register?
For example, if a PTE or MDL, there is a mapping between PHY and Virtual add mapping.
So what is the mapping in the map register, even it is not construct by CPU, but by DMA controller.
No, a “common buffer” is something the driver explcitly allocates and
manages. The entity referred to as a “bounce buffer” is created and
manged by mechanisms invisible to and outside the control of your driver.
A “map register” holds whatever it is supposed to hold; as far as I know,
this can vary with the chipset you have. It doesn’t matter in the
slightest to a driver writer what this is. And that’s only if the map
registers are actually impemented in hardware. If they are fictitious,
they hold whatever fictitious map registers hold, and the important thing
is that you do not eed to know what this is.
For the morbidly curious, consult the detailed hardware documentation for
your particular chipset, most probably found on the Intel web site.
Generally you should not be messing around worrying about PTEs (which I
think are also chipset-dependent).
I only know the abstraction: that the mapping registers transform what
appear to be 32-bit physical addresses into nominally 36-bit physical
addresses. Note that when such mappings are in effect, the actual memory
described by 32-bit physical address cannot be concurrently used by
another DMA transfer.
The simplest description is that it is VM mapping for DMA, but does not
employ the concept of “page fault”. This is a bit of oversimplification.
I once worked on a machine with a 16-bit logical address but an 18-bit
address bus, and we had to program the two high-order bits into the
device. A machine could have 262,144 bytes of physical memory, but could
only address a subset in the running program. The memory map converted
16-bit virtual adresses into 18-bit physical addresses, and had “separate
I and D spaces”, meaning instruction fetches used one page table and data
fetches used a second page table, so you could have 64K bytes of program
and a separate 64K bytes of data (and no, you could not write
self-modifying code!).
joe
hi, Joe
What you said bounce buffer, is the Common buffer concept, right?
BTW, what’s the content in the map register?
For example, if a PTE or MDL, there is a mapping between PHY and Virtual
add mapping.So what is the mapping in the map register, even it is not construct by
CPU, but by DMA controller.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
>DDK help tell us it works as mapping between device/logical address to physical address?
“Device address” is the address sent to the device. When the device will do DMA cycles, it will set device addresses to the bus.
“Physical address” is the address required for the RAM/frontside bus.
There can be a mapping between the two.
-
First, there can be a hardware mapper in the north bridge, which accepts device addresses from the DMAing device and translates them to the physical addresses to be used for the RAM. Such hardware is generally called IOMMU, and AGP GART is a good sample of it.
-
Second, there can be “bounce buffers”. This means that the special memory chunks are allocated for DMA by the adapter object. “Device addresses” are the physical addresses of these chunks, and “physical addresses” are the addresses of the MDL’s pages. There is a memcpy() between the MDL’s pages and bounce buffers either at buffer creation (write) or at DMA transaction flush (read).
-
Third, there is system DMA (often considered to be obsolete ISA artifact, but still used on modern system-on-chips). System DMA means - the device is not aware of any addresses, the transaction’s physical address should be sent to the system DMA controller which is a separate logic. In this case, there is no “device addresses”.
The polymorphic abstraction over all of these is called “adapter object”, which also defined the notion of “map register”.
For 1), the “map register” is IOMMU’s table entry.
For 2), the “map register” is a bounce buffer.
For 3), there is 1 hardcoded “map register” for each system DMA channel.
The DMA logic is:
- AllocateMapRegisters(NumberOfPages)
- MapTransfer(MDL, MapRegisterHandle) // here memcpy() is done for bounce buffer on write
- send device addresses to the device
- ping the device to start the operation
- wait for completion
- FlushAdapterBuffers(MapRegisterHandle) // here memcpy() is done for bounce buffer on read
- free map registers.
Why is this stuff?
- to solve the issue of running DMA on physically discontiguous MDLs (one of the possible solutions of this problem).
- to allow the 32bit device (unable to do 64bit DMA addressing) to access RAM > 4GB (bounce buffer is used)
- to allow the old ISA device to access above 16M of RAM
- to support system DMA
- to do some new virtualization stuff about exposing the whole (sub)device to the guest - called something like SR/V or such. In this case, the guest physical addresses are DMA addresses, and IOMMU contains the part of the GPhys->Phys map.
Windows DMA APIs support map registers fully for years.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
Very good descriptions, all.
Here’s a (relatively recent) whole article from The NT Insider explaining Windows DMA concepts… with diagrams, even:
http://www.osronline.com/article.cfm?article=539
Peter
OSR
workingmailing@163.com wrote:
What you said bounce buffer, is the Common buffer concept, right?
BTW, what’s the content in the map register?
For example, if a PTE or MDL, there is a mapping between PHY and Virtual add mapping.
So what is the mapping in the map register, even it is not construct by CPU, but by DMA controller.
For the vast majority of the computers you are likely to encounter
today, the “map registers” are a software concept that do not exist in
hardware anywhere. The computer on your desk almost certainly has a 1:1
correspondence between the physical addresses as known by the CPU and
the physical addresses as seen on the buses.
However, it is quite possible to construct machines where that is not
the case, where some bus has an address space that doesn’t match the
physical addresses. As a non-real-world example, consider a 32-bit PCI
bus attached to a 64-bit machine. Say that the motherboard hardware has
the ability to “relocate” that entire PCI bus, so that physical address
range 1234_0000_0000 to 1234_ffff_ffff maps to that extra PCI bus.
In that case, the hardware would have map registers to map the CPU’s
physical address 1234_0000_0000 to that bus’s physical address 0. The
driver would work with physical addresses in the 123400000000 range, but
the DMA abstraction would convert that to physical address 0 for you to
send to the hardware.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Thank you for so many windows experts’ reply.
Now I am more clear about map register.
Most important:
- Map register is used when requestor’s buffer have physical address >4G
in this case, DMA abstraction will create map register(bounce buffer) to do the DMA
One 4G RAM system, we could get 1024 map register, but 8G not, can any one figure this issue?
> Thank you for so many windows experts’ reply.
Now I am more clear about map register.
Most important:
- Map register is used when requestor’s buffer have physical address >4G
in this case, DMA abstraction will create map register(bounce buffer) to
do the DMA
****
Almost true. In those architectures that actually have map registers, the
map registers will be used to relocate the DMA transfer to physical
addressess > 4GB. The important thing is that you, as the driver writer,
do not need to know which of these has happened; you are given an address
and you program it to the DMA chip. It might be the direct user buffer <
4GB, it might be the direct user buffer > 4GB, or it may be a bounce
buffer depending on lots of conditions (for example, in DMA checking in
Driver Verifier, the Driver Verifier manages a bounce buffer whose sole
purpose is to check for proper DMA transfers, and this is true even if the
user buffers are < 4GB or the platform has actual mapping registers!)
joe
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
That’s pretty easy. On a 4GB system, all memory is addressible with
32-bit addresses, so the map registers are merely a fiction, and you can
have a lot of them (4MB is a pretty tiny piece of 4GB). Since there is no
need to support 32-bit DMA addresses (since every buffer is addressible by
32 bits) then this merely sets a limit on the largest buffer you can
transfer in one DMA operation.
On the 8GB system, you probably have a high-end chipset that has REAL map
registers, and there are far fewer than 1024 of these real registers.
joe
One 4G RAM system, we could get 1024 map register, but 8G not, can any one
figure this issue?
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
workingmailing@163.com wrote:
Now I am more clear about map register.
Most important:
- Map register is used when requestor’s buffer have physical address >4G
in this case, DMA abstraction will create map register(bounce buffer) to do the DMA
The situation is not quite that simple. This is a picky point, but it’s
important, so I want to be very clear here.
In “commodity” hardware – the kind of computers you are likely to
encounter on a daily basis – the situation is pretty much as you have
described. Map registers are a software concept, essentially embodied
in the physical bounce buffers.
However, there ARE computers today in which map registers are genuine
hardware constructs that are used for every DMA transfer. There are
high-end machines with very complicated I/O architectures, with multiple
buses and multiple controllers, where addresses are getting rewritten
several times. These machines DO exist. I have not had the pleasure of
encountering one, but they are out there.
So, make no assumptions and follow the rules, and your driver will
continue to work when it happens to run on one of those weird machines.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
thank you Joe and Tim, and every one try to help me.
But I still wonder the issue about:
One 4G RAM system, we could get 1024 map register, but 8G not, can any one
figure this issue?
But what I done is just add the 4G ram to 8G, the total system(motherboard, cpu,…) are all the same.
BTW, this device is 32bit support, not support 64bit address.
RAM might be located above 4GB physical, even if there is less than 4GB of RAM in a particular system. A particular machine is not required to populate physical address space with RAM starting from address 0 contiguously.
There may be other consumers of address space, such as for device memory-mapped I/O, that may cause portions of RAM to be assigned addresses above 4GB (assuming the hardware supports > 32 physical address bits).
- S (Msft)
From: xxxxx@lists.osr.com [xxxxx@lists.osr.com] on behalf of workingmailing@163.com [workingmailing@163.com]
Sent: Monday, April 16, 2012 7:32 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] what is map register
thank you Joe and Tim, and every one try to help me.
But I still wonder the issue about:
One 4G RAM system, we could get 1024 map register, but 8G not, can any one
figure this issue?
But what I done is just add the 4G ram to 8G, the total system(motherboard, cpu,…) are all the same.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
thank you Ken, you give me a very important concept about RAM and its mapping.
I agree with you, even only 4G RAM exist, they may not all mapping to the address <4G address.
Some part of the RAM may mapping to the address higher than 0xffff,ffff
But come back to the question and does any encounter before and have solution?
Mr. Newcomer answered your question already.
In Windows, on the 8GB system, the map registers are physical buffers allocated by the HAL. On the 4GB system, you PROBABLY have < 4GB of physical memory, and therefore there are no physical map registers. THerefore you can have an unlimited number (or, in your csse 1024) of the resources that don’t really exist.
Clear now?
Peter
OSR
Thank you Peter, what you mean is this map register is something located lower than 4G memory, if that do not exist when no needed, so OS could say – lie, that it is unlimited.
But when it is needed, such as 8G system, so OS tell you the true, not so much bounce buffer I could allocated for you, it is in fact it is a warming.
But in fact I still could not master what the reason,
As I known, map register is bounce buffer allocated by DMA abstraction or HAL or somethings else in the Windows XXX manager lower than 4G RAM phy address.
The system are all the same, only different is I add another 4G RAM to totally 8G, the former case is only with 4G.
So in my thinking, map register (each is 4k/one page) is allocate by OS in the lower 4G phy memory working as bounce buffer, which will add one copy from bounce buffer to requestor buffer if the requestor buffer located in PHY memory higher than 4G.
So windows have a limit on map register numbers?
Yes. Exactly correct.
Yes. They are shared and should generally be considered a “scarce resource.”
Peter
OSR
> Thank you Peter, what you mean is this map register is something located
lower than 4G memory, if that do not exist when no needed, so OS could say
– lie, that it is unlimited.
****
No, a map register is used to map 32-bit DMA to physical addresses on a
wider bus. It is not “located” anywhere except in the I/O controller
(usually in the bus interface chip, commonly referred to as the “North
Bridge” but is officially the “Hub Control Interface” (HCI) chip.
****But when it is needed, such as 8G system, so OS tell you the true, not so
much bounce buffer I could allocated for you, it is in fact it is a
warming.
*****
No, it is just information. It does not “warn” you of anything. As in
all systems, it tells you the maximum transfer you can do using DMA on
that device. Don’t try to read more into the concept than that.
*****But in fact I still could not master what the reason,
As I known, map register is bounce buffer allocated by DMA abstraction or
HAL or somethings else in the Windows XXX manager lower than 4G RAM phy
address.
*****
It is up to the particular version of the OS what is done. It may or may
not exist; it may or may not have an intermediate bounch buffer; etc. The
point is, YOU DON’T CARE!
*****The system are all the same, only different is I add another 4G RAM to
totally 8G, the former case is only with 4G.
*****
What matters is the chipset. If the OS finds more than 4GB available, it
will take some suitable action to allow 32-bit peripherals to run. and
YOU DON’T CARE!
******So in my thinking, map register (each is 4k/one page) is allocate by OS in
the lower 4G phy memory working as bounce buffer, which will add one copy
from bounce buffer to requestor buffer if the requestor buffer located in
PHY memory higher than 4G.
*****
No, not even close. A map register is an abstraction, which has a variety
of implementations in both hardware and software. Do not think that any
one implementation is chosen, because YOU DON’T CARE. When implemented in
hardware, the DMA transfers directly to the physical pages, even those
above 4GB. Or maybe not. I repeat, YOU DON’T CARE.
*****So windows have a limit on map register numbers?
****
It is not Windows; it is the physical hardware if the registers are real.
You keep thinking there is more going on than was actually described.
Remember, map registers are an *abstraction* whose implementation is
unknown. So don’t keep saying that it involves a bounce buffer; that is a
mere detail of implementation and can vary from one system to the next.
joe
*****
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer