Accessing Memory Through Physical Address

In tools like Windbg, we can query the content of physical addresses by the ‘!db’ command. But how does it do that? From what I understand, the processor is using virtual address in all of its opcodes, which is not directly related to the underlying physical address.

windbg docs have a page explaining how virtual address are translated
into physical address
windbg also has a command !vtop 0 in the process
context to convert the virtual address of the current process context
into physical address

the docs also explain how you can convert any virtual address into
physical address

virtual address is a combination of bitfields
bits 00…11 are byte_index
bits 12…21 are page_table_index
bits 22…31 are page_directory_index

compute
PTE_BASE + page_directory_index * PAGE_SIZE + page_table_index * sizeof(MMPTE)

examine the contents of the dword at the computed address
the high 20 bits of the content + byte_index == physical address of
the given virtual address

for example in a running instance of notepad.exe
i will convert the virtual address 01013368 to its physical address

(base 16) (hex) 0x01013368 == (base 2)(binary)0000001000000010011001101101000
split intp bit fields 0000001000—0000100110----01101101000
--------------------------------0x8--------------0x26----------------368-------
ptebase±-------0x8pagesize±-------0x26sizeof(mmpte)
c0000000±----0x8000±----------------0x98 == c0008098

c0008098 contains 3f2d0025
virtual address 1013368 == physical address
3f2d0 + 368 3f2d0368

an auto conversion using available commands

lkd> lm m notepad
start end module name
01000000 01014000 notepad

lkd> .foreach (place {s -[1l 20]u 1000000 1014000 NotepadE }) {du place }
01013368 “NotepadEFailed to Initialize Fil”

lkd> !vtop 0 1013368
Virtual address 1013368 translates to physical address 3f2d0368.

lkd> !db 3f2d0368
#3f2d0368 4e 00 6f 00 74 00 65 00-70 00 61 00 64 00 45 00 N.o.t.e.p.a.d.E.
#3f2d0378 46 00 61 00 69 00 6c 00-65 00 64 00 20 00 74 00 F.a.i.l.e.d. .t.
#3f2d0388 6f 00 20 00 49 00 6e 00-69 00 74 00 69 00 61 00 o. .I.n.i.t.i.a.
#3f2d0398 6c 00 69 00 7a 00 65 00-20 00 46 00 69 00 6c 00 l.i.z.e. .F.i.l.

On 10/26/15, xxxxx@hotmail.com wrote:
> In tools like Windbg, we can query the content of physical addresses by the
> ‘!db’ command. But how does it do that? From what I understand, the
> processor is using virtual address in all of its opcodes, which is not
> directly related to the underlying physical address.
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

xxxxx@hotmail.com wrote:

In tools like Windbg, we can query the content of physical addresses by the ‘!db’ command. But how does it do that? From what I understand, the processor is using virtual address in all of its opcodes, which is not directly related to the underlying physical address.

Windbg creates a page table entry on the fly that maps to the physical
address in your command, then reads from the virtual address for the
page it created.

There can be problems if an address range is mapped twice with two
different caching attributes, and the windbg page warns about that.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

>Windbg creates a page table entry on the fly that maps to the physical address in your command, then reads from the virtual address for the page it created.

Which APIs allow us to do that ?

There can be problems if an address range is mapped twice with two different caching attributes, and the windbg page warns about that.

Yes, I heard of that many times,too. But isn’t it that by copying the the cache attirbutes used in the first mapping we can ensure the 2 are the same? Why people keep saying this is an issue?

> Yes, I heard of that many times,too. But isn’t it that by copying the the cache attirbutes used in the

first mapping we can ensure the 2 are the same?

How can you find the first mapping given the PhysAddr only?

Are you sure Windows has such a data structure (index) now?


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

xxxxx@hotmail.com wrote:

> Windbg creates a page table entry on the fly that maps to the physical address in your command, then reads from the virtual address for the page it created.
Which APIs allow us to do that ?

A kernel driver can use MmMapIoSpace to do this. A user-mode app can
use VirtualAlloc to reserve a page table entry with no memory, but it is
going to need kernel help to fill in the page table.

> There can be problems if an address range is mapped twice with two different caching attributes, and the windbg page warns about that.
Yes, I heard of that many times,too. But isn’t it that by copying the the cache attirbutes used in the first mapping we can ensure the 2 are the same?

You can’t copy the cache attributes from the first mapping, because you
don’t know where it is. The page tables are indexed by the VIRTUAL
address, not by the PHYSICAL address. Given a physical address, the
only way to find the matching virtual addresses (and there might be
several) is by doing an exhaustive search. That’s not practical.

Why people keep saying this is an issue?

Because it is an issue. I have actually encountered this problem during
debugging. I don’t remember whether it was a crash or a hang, but it
was not happy.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Each physical page has some kind of descriptor in a table somewhere.
When OS maps a page which is mapped somewhere else already, it will make sure the page is mapped with the same cache attributes.

When WinDbg does that, it doesn’t have such luxury. Or with the modern OS it may actually know the cache attributes.

David Solomon and Mark Russinovich’s Book 6th ed Part II:

***
The PFN database consists of an array of structures that represent each physical page of memory
on the system. The PFN database and its relationship to page tables are shown in Figure 10-37. As this
figure shows, valid PTEs usually point to entries in the PFN database, and the PFN database entries
(for nonprototype PFNs) point back to the page table that is using them
***

So given a physical address, couldn’t we locate the page table entry from the PFN database, take a look at the cache attribute currently used and copy it for any new PTE mapping we are going to create?

xxxxx@hotmail.com wrote:

David Solomon and Mark Russinovich’s Book 6th ed Part II:

***
The PFN database consists of an array of structures that represent each physical page of memory
on the system. The PFN database and its relationship to page tables are shown in Figure 10-37. As this
figure shows, valid PTEs usually point to entries in the PFN database, and the PFN database entries
(for nonprototype PFNs) point back to the page table that is using them
***

So given a physical address, couldn’t we locate the page table entry from the PFN database, take a look at the cache attribute currently used and copy it for any new PTE mapping we are going to create?

Absolutely, and we said that. The problem is that the process involves
a linear search of the entire page table, one at a time. A system with
8 GB of memory has 2 million page table entries. And, since a single
page can be mapped multiple times, you have to search the entire table.
In addition, what do you do if there are multiple entries and they disagree?

There is too much policy to automate, so the API simply leaves it up to
the caller.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Not to mention, there is ONE reference from the PFN entry to a PTE that uses that page. So, you merely get ONE example. And it doesn’t include prototype PTEs.

For a diagnostic situation, why bother?

Peter
OSR
@OSRDrivers