PhysicalMemory and Address Space on x64

I’m trying to work on a driver that reads physical memory on an x64 Windows OS. I’ve had no problems creating and signing a test driver that does the usual, ZwOpenSection, ZwMapViewOfSection, ZwUnmapViewOfSection, ZwClose on \Device\PhysicalMemory. However, on systems that I run this on it always crashes around the 3GB mark (even on systems that say have 8GB of RAM). In fact, when I compile and run my driver on an x86 system it completes the scan - mainly because it completes scanning UNDER the 3GB mark. Now, it may be that my RAM is not contiguously mapped in physical address space. I imagine my Video card is taking up some of that address space since it shares with RAM on the northbridge.

Is anyone aware of a way in which I can determine the exact Physical Address blocks that Physical Memory is assigned to? I’m aware that Physical Memory is not always contiguously mapped in physical address space.

Also, why would it be crashing…I’m not an expert on memory dumps, but even if I was trying to map a view of a section of physical address space that doesn’t below to Physical Memory, why would this cause it to crash?

I suppose we shouldn’t ask why you’re doing this, or what ultimate result it is that you’re trying to accomplish? Cuz I can’t imagine a reason in the world (except for writing a memory diagnostic) that what you’re trying to do makes much sense. Humor me… what’s your goal?

Sigh…

ANYhow… There’s a registry key that describes the physical memory blocks… If you can’t find it, and nobody else here knows off the top of their heads, I’ll look it up for you.

Peter
OSR

The registry value Peter is trying to remember is:

\REGISTRY\MACHINE\HARDWARE\RESOURCEMAP\System Resources\Physical Memory

He is the one who taught this to me over 15 years ago.

BUT AS PETER ASKED: Why are you doing this? Most of us were relieved
with the demise of the Physical Memory mapping security hole that
Microsoft finally closed.

Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

-----Original Message-----
From: xxxxx@osr.com [mailto:xxxxx@osr.com] Posted At: Thursday,
April 08, 2010 9:54 AM Posted To: ntdev
Conversation: PhysicalMemory and Address Space on x64
Subject: RE: PhysicalMemory and Address Space on x64

I suppose we shouldn’t ask why you’re doing this, or what ultimate
result it is that you’re trying to accomplish? Cuz I can’t imagine a
reason in the world (except for writing a memory diagnostic) that what

you’re trying to do makes much sense. Humor me… what’s your goal?

Sigh…

ANYhow… There’s a registry key that describes the physical memory
blocks…
If you can’t find it, and nobody else here knows off the top of their
heads, I’ll look it up for you.

Peter
OSR

__________ Information from ESET Smart Security, version of virus
signature database 5010 (20100408) __________

The message was checked by ESET Smart Security.

http://www.eset.com

__________ Information from ESET Smart Security, version of virus
signature database 5010 (20100408) __________

The message was checked by ESET Smart Security.

http://www.eset.com

Why is this a security hole and what was done to fix this? Shouldn’t drivers be able to read directly from any location within PhysicalMemory? If not, what locations within Memory are drivers allowed to map to?

Thanks for the quick response, nonetheless.

The drivers are allowed to only touch the memory they allocate or get in MDL
from the kernel. They are not allowed to touch arbitrary memory. Just like
usermode code, it doesn’t do you much good if you start peeking the
arbitrary memory addresses.

wrote in message news:xxxxx@ntdev…
> Why is this a security hole and what was done to fix this? Shouldn’t
> drivers be able to read directly from any location within PhysicalMemory?
> If not, what locations within Memory are drivers allowed to map to?
>
> Thanks for the quick response, nonetheless.
>

I can understand from the perspective on security holes, but aren’t drivers trusted if they are signed? Rather than debate the details of how this relates to Windows Security, I am more curious at this point if it’s even possible. There are memory dumping tools out there, that say this can be done and their tools support it - for example supporting Windows Vista x64 memory dumps. What is the harm in reading from physical addresses point to memory not referenced by the driver or from MDL? Is their any harm in that and are they valid pointers?

Drivers are trusted, but a driver should be well behaved and only access
memory it was allocated except in very special circumstances. What is
not trusted are user space applications, but by your initial description
what your driver is doing is mapping physical memory which it does not
own into user space. THAT IS A SECURITY HOLE! You still have not
explained what problem you are trying to solve.

Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

-----Original Message-----
From: xxxxx@guerrillamailblock.com
[mailto:xxxxx@guerrillamailblock.com]
Posted At: Thursday, April 08, 2010 10:43 AM
Posted To: ntdev
Conversation: PhysicalMemory and Address Space on x64
Subject: RE: PhysicalMemory and Address Space on x64

I can understand from the perspective on security holes, but aren’t
drivers
trusted if they are signed? Rather than debate the details of how this
relates
to Windows Security, I am more curious at this point if it’s even
possible.
There are memory dumping tools out there, that say this can be done
and their
tools support it - for example supporting Windows Vista x64 memory
dumps. What
is the harm in reading from physical addresses point to memory not
referenced
by the driver or from MDL? Is their any harm in that and are they
valid
pointers?

__________ Information from ESET Smart Security, version of virus
signature
database 5010 (20100408) __________

The message was checked by ESET Smart Security.

http://www.eset.com

Well… I hate to disagree with Mr. Grig, but all drivers are trusted parts of the system. Period. They’re inherently part of the trusted computing base, and they have access to any location in the kernel virtual address space, whether they’ve allocated it or not, by virtue of the fact that they’re RUNNING in the kernel virtual address space.

Now, that doesn’t mean it’s good POLICY to actively facilitate drivers doing undisciplined things and accessing all of physical memory whenever they want.

The “memory dumping tools” that are “out there” are mostly pure shite. I mean… consider the problem of capturing all of physical memory WHILE YOU’RE EXECUTING IN IT and while interrupts are enabled. Is the state of physical memory likely to remain consistent throughout your capturing process? The answer to that would be “no” – and thus, what IS it that you’re really capturing. In my opinion, most of these “capture and analyze memory” programs are nothing more than a way to blow a whole lot of smoke up somebody’s ass who doesn’t truly understand what they’re getting.

So… YES, it can be made to work. You CAN do what you’re asking. There are multiple ways to do it (I mean, heck… why not just change the contents of one of your PTEs??)… What I – and a couple of other folks here – have been trying to ask is: “What are you trying to accomplish, and why do you think this is a good idea?” as well as “Have you thought this through” and the implicit question “who are you and what are your intentions”??

These being your first posts here, Mr. “Oh My” from “guerrillamailblock”… you should EXPECT us to ask you a few questions, you know? This isn’t rootkit.com and we’re not in the business of actively facilitating the development of malware,

Peter
OSR

Say I want to write a memory dumping tool (not for nefarious reasons)…should this not be possible in Windows x64? Again, I understand its a security hole to pass on all physical memory to user-space through a device driver, but if one needs to write an application to dump out all memory to a file for inspection there is no other choice, right? Either I can open up a driver and map a view of memory to an address referenced in the MDL or use MmMapIoSpace to do a similar approach to map to non-paged system space.

If your goal is a memory dump to a file, then you do not need to pass
the data to user space first you could write the file directly to user
space. This is still a security hole but a somewhat smaller one in that
an application cannot actively monitor a region of kernel memory. Also,
it stops the inevitable, “well since I can read the memory, adding a
write function is easy…”.

You still have not said what you are doing, right now I think it
probably a good idea to give no active help to you till you at least
give us a clue.

Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

-----Original Message-----
From: xxxxx@guerrillamailblock.com
[mailto:xxxxx@guerrillamailblock.com]
Posted At: Thursday, April 08, 2010 11:04 AM
Posted To: ntdev
Conversation: PhysicalMemory and Address Space on x64
Subject: RE: PhysicalMemory and Address Space on x64

Say I want to write a memory dumping tool (not for nefarious
reasons)…should
this not be possible in Windows x64? Again, I understand its a
security hole
to pass on all physical memory to user-space through a device driver,
but if
one needs to write an application to dump out all memory to a file for
inspection there is no other choice, right? Either I can open up a
driver and
map a view of memory to an address referenced in the MDL or use
MmMapIoSpace
to do a similar approach to map to non-paged system space.

__________ Information from ESET Smart Security, version of virus
signature
database 5010 (20100408) __________

The message was checked by ESET Smart Security.

http://www.eset.com

I’m NOT doing anything nefarious, I’m NOT trying to write a rootkit. I’m simply trying to understand memory storage for static analysis. I use guerrillamailblock when I write posts because I don’t like spam mail.

I simply want to research how memory is stored in Windows x64 machines through static analysis (thus reading it out to file). Sure I can learn about the structures of process memory and how that’s stored, but I want to learn about the bigger picture. Being, memory that is referenced for active processes and what remains in memory that is no longer actively being utilized. This is useful because it helps with operating system development research, memory management research, and malicious code detection.

> why would it be crashing…

This is because you, apparently, don’t check the result of ZwMapViewOfSection() - it may fail under some circumstances, for example, when the target range is reserved by BIOS or is just refers to something that is not RAM( memory-mapped device, ROM, IOAPIC,etc). This is why you crash.

In my experience, CPU running in protected mode with paging enabled would not raise an exception as long as Present and RW bits in PTE are OK - even if the physical address specified in PTE is invalid or inconsistent with the access type nothing happens. Therefore, I believe the problem here lies with PTE, i.e. mapping just fails for some reason…

Anton Bassov

Anton…I’m checking the NTSTATUS after the call to ZwMapViewOfSection, but your correct in that I’m likely trying to access an area of physical address space that is not mapped to PhysicalMemory (Such as Video card RAM or what not). Now I used a tool (MemInfo) to determine which physical addresses are mapped for PhysicalMemory and I’m reading only those regions of the physical address space - this at least solved the crashing problem and I’m able to ATTEMPT to read through all physical addresses mapped to physical memory. But now I have a new problem and that’s a good number of those regions are giving me ZwOpenViewOfSection failures - as a note I’m only reading a page at a time. This could be due to a number of reasons…1. being the regions I discovered that are for PhysicalMemory are incorrect, 2. they are inaccessible somehow. Any thoughts?

Also, how would I read the PTE tables, aren’t those only for active processes? So what if I wanted to read inactive regions of PhysicalMemory?

If you’re going to use the dump for static analysis, why don’t you just create a crash dump? It can be done from user mode, and then you could use the existing (very capable, though terribly documented) API to examine it.

This is the same question that I have with all the ‘memory capture’ tools. In particular, it could be a common format/backend.

mm

Wouldn’t it be a mistake to assume that even reading arbitrary physical
memory is safe? If a hardware device is memory mapped it may have for
example a FIFO buffer where a read will remove the data so the next data
item pops up. Some nefarious driver reading from this memory would break the
device.

What would be the policy if a signed driver appeared that did this? Would MS
revoke the certificate?

I assume that WHQL even might not detect this, presumably the driver could
do this very rarely, either by mistake or by malintention…

Mike Kemp, Sintefex Audio Lda.

>>>>

----- Original Message -----
From: Don Burn
Newsgroups: ntdev
To: Windows System Software Devs Interest List
Sent: Thursday, April 08, 2010 4:07 PM
Subject: RE:[ntdev] PhysicalMemory and Address Space on x64

If your goal is a memory dump to a file, then you do not need to pass
the data to user space first you could write the file directly to user
space. This is still a security hole but a somewhat smaller one in that
an application cannot actively monitor a region of kernel memory. Also,
it stops the inevitable, “well since I can read the memory, adding a
write function is easy…”.

You still have not said what you are doing, right now I think it
probably a good idea to give no active help to you till you at least
give us a clue.

Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

-----Original Message-----
From: xxxxx@guerrillamailblock.com
[mailto:xxxxx@guerrillamailblock.com]
Posted At: Thursday, April 08, 2010 11:04 AM
Posted To: ntdev
Conversation: PhysicalMemory and Address Space on x64
Subject: RE: PhysicalMemory and Address Space on x64

Say I want to write a memory dumping tool (not for nefarious
reasons)…should
this not be possible in Windows x64? Again, I understand its a
security hole
to pass on all physical memory to user-space through a device driver,
but if
one needs to write an application to dump out all memory to a file for
inspection there is no other choice, right? Either I can open up a
driver and
map a view of memory to an address referenced in the MDL or use
MmMapIoSpace
to do a similar approach to map to non-paged system space.

__________ Information from ESET Smart Security, version of virus
signature
database 5010 (20100408) __________

The message was checked by ESET Smart Security.

http://www.eset.com


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

LOL… interesting. I guess it depends on whether the driver would be classified as malware. And then, of course, they’d revoke the certificate for the COMPANY that signed the driver… so that EVERY driver signed with that same cert would be barred.

Oh, quite right. I assume WHQL wouldn’t detect this if your driver did it once every second (depending on HOW you did it).

As long as your driver passes the tests for your device class,I think you’d be all good.

Peter
OSR

> Wouldn’t it be a mistake to assume that even reading arbitrary physical memory is safe?

If a hardware device is memory mapped it may have for example a FIFO buffer where a read willr
remove the data so the next data item pops up.

Please note that memory-mapped devices are supposed to support memory protocols so that they appear as memory ranges. Can reading a memory range destroy it data??? Therefore, I think your concern is unfounded. When it comes to IO space the situation may, indeed, be somehow different, but as long as device
registers are mapped as memory reads from them must appear as normal reads, without any “side effects”…

Anton Bassov

You must never map physical memory that you do not own. To do so will introduce the potential for internal processor TLB corruption which may corrupt or bring down the system. This is not a technique that you can ship in a production product.

  • S

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@guerrillamailblock.com
Sent: Thursday, April 08, 2010 6:48 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] PhysicalMemory and Address Space on x64

I’m trying to work on a driver that reads physical memory on an x64 Windows OS. I’ve had no problems creating and signing a test driver that does the usual, ZwOpenSection, ZwMapViewOfSection, ZwUnmapViewOfSection, ZwClose on \Device\PhysicalMemory. However, on systems that I run this on it always crashes around the 3GB mark (even on systems that say have 8GB of RAM). In fact, when I compile and run my driver on an x86 system it completes the scan - mainly because it completes scanning UNDER the 3GB mark. Now, it may be that my RAM is not contiguously mapped in physical address space. I imagine my Video card is taking up some of that address space since it shares with RAM on the northbridge.

Is anyone aware of a way in which I can determine the exact Physical Address blocks that Physical Memory is assigned to? I’m aware that Physical Memory is not always contiguously mapped in physical address space.

Also, why would it be crashing…I’m not an expert on memory dumps, but even if I was trying to map a view of a section of physical address space that doesn’t below to Physical Memory, why would this cause it to crash?


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Not “destroy” but, rather, “consume”… that’s the problem to which I understood Mr. Kemp was referring.

Eh? Define “map.” Define “own.” (for that matter, define “never”… no, seriously.)

If you mean “Point a PTE to”, I certainly agree. At least if you want to keep the system running after you do so. You might, for example, want to “roll your own” memory dump prior to halting the machine.

If you mean “map using the Memory Manager DDIs” I’d say that using these API makes you at least partial “owner” of the memory… and – as long as the memory manager does the mapping – I don’t see how that could least to TLB corruption.

Peter
OSR

Yes, I assumed that you wanted to create a program which did not risk corrupting the system (i.e. guarantees that the system will continue running properly afterwards). Hence the remarks about that risk included in the first post.

For this discussion, I would define ‘map’ as in to create (or cause to be created) a virtual -> physical mapping that could end up in a processor TLB, i.e. creating a PTE mapping some virtual space to that physical address, even if the PTE is never referenced by real program flow (speculative execution could fetch it).

If everyone follows the rules, MM will guarantee that there won’t be TLB corruption. The rules are that you must only map memory that you own (i.e. that has been handed back to you by some allocation scheme which gives you exclusive access to the entire set of pages and guarantees that nobody else has created virtual to physical mappings for said pages), such that the cache characteristics of mappers of that memory can be tracked and coordinated. The simplest case of this is that there is only one mapper and they promise not to create multiple conflicting mappings after given exclusive use of a set of physical pages.

Mapping arbitrary physical memory regions pulled out of the hat (i.e. not explicitly allocated for your exclusive usage by a proper allocator of some sort) with \Device\PhysicalMemory does not count as following the rules and you are not guaranteed that there won’t now (or in the future) be a caching attribute conflict that will cause TLB corruption. For instance you are not protected against some innocent party creating a future mapping through whichever allocator rightfully owns that physical memory region with conflicting attributes with respect to an illegally created mapping.

The documentation for ZwMapViewOfSection alludes to this. “Do not use ZwMapViewOfSection to map a memory range from \Device\PhysicalMemory […] unless your driver has directly allocated the memory range through MmAllocatePagesForMdl or another method guaranteeing that no other system component has mapped the same memory range”.

I would avoid doing this even in the bug check path unless you really have no choice at all in the matter. While there may be some waiving of these risks when you are down the path where the system is already going to crash, it’s still in your best interest to make sure that the system is most likely to successfully complete writing the crash dump intact rather than experiencing a secondary failure along the way.

  • S

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Thursday, April 08, 2010 10:51 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] PhysicalMemory and Address Space on x64

Not “destroy” but, rather, “consume”… that’s the problem to which I understood Mr. Kemp was referring.

Eh? Define “map.” Define “own.” (for that matter, define “never”… no, seriously.)

If you mean “Point a PTE to”, I certainly agree. At least if you want to keep the system running after you do so. You might, for example, want to “roll your own” memory dump prior to halting the machine.

If you mean “map using the Memory Manager DDIs” I’d say that using these API makes you at least partial “owner” of the memory… and – as long as the memory manager does the mapping – I don’t see how that could least to TLB corruption.

Peter
OSR


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer