> Why is this a security hole and what was done to fix this? Shouldn’t drivers be able to read directly
from any location within PhysicalMemory?
For what?
Windows is a virtual memory OS. No code except DMA code should ever know anything about physical memory (except its size and ranges).
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
>through a device driver, but if one needs to write an application to dump out all memory to a file for
inspection there is no other choice, right?
CrashOnCtrlScroll registry value, then hit these keys.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
> In my experience, CPU running in protected mode with paging enabled would not raise an exception
as long as Present and RW bits in PTE are OK - even if the physical address specified in PTE is
invalid or inconsistent with the access type nothing happens.
I think at least some such accesses will cause PCI Master Abort and thus a crash.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
> Not “destroy” but, rather, “consume”… that’s the problem to which I understood Mr. Kemp was referring.
Well, in this context the above terms are pretty much synonymous, don’t you think - if “read-once” data gets “consumed” by the illegitimate party it is, from its legitimate recipient 's point of view, simply destroyed. This cannot happen with RAM, can it - no matter how many times you read data from the memory buffer you will always get the same results as long as no writes to the target memory take place. Therefore, it should not happen with devices that support memory protocols either - otherwise, the target device just would not appear as RAM. Therefore, as long as we speak about memory-mapped device “consumption” must involve more than a simple memory read by CPU - instead, it must involve modifying device registers somehow…
Anton Bassov
The only thing that I can say here is that you are confusing TLBs with caching type…
Anton Bassov
The security hole was from user-mode.
But the thing that you’ve got to learn is that reading from device regions
can have side effects. Lots of devices have FIFOs, and “read-to-clear”
bits. So if you really do read all of physical memory, there’s a reasonable
chance you’ll cause a device driver to crash, or you’ll cause a device to
corrupt memory.
–
Jake Oshins
Hyper-V I/O Architect
Windows Kernel Group
This post implies no warranties and confers no rights.
wrote in message news:xxxxx@ntdev…
> Why is this a security hole and what was done to fix this? Shouldn’t
> drivers be able to read directly from any location within PhysicalMemory?
> If not, what locations within Memory are drivers allowed to map to?
>
> Thanks for the quick response, nonetheless.
>
That depends on how the machine is configured, Max. Mostly high-end servers
will do that.
–
Jake Oshins
Hyper-V I/O Architect
Windows Kernel Group
This post implies no warranties and confers no rights.
“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
>> In my experience, CPU running in protected mode with paging enabled would
>> not raise an exception
>>as long as Present and RW bits in PTE are OK - even if the physical
>>address specified in PTE is
>>invalid or inconsistent with the access type nothing happens.
>
> I think at least some such accesses will cause PCI Master Abort and thus a
> crash.
>
> –
> Maxim S. Shatskih
> Windows DDK MVP
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
xxxxx@guerrillamailblock.com wrote:
I’m trying to work on a driver that reads physical memory on an x64 Windows OS. I’ve had no problems creating and signing a test driver that does the usual, ZwOpenSection, ZwMapViewOfSection, ZwUnmapViewOfSection, ZwClose on \Device\PhysicalMemory. However, on systems that I run this on it always crashes around the 3GB mark (even on systems that say have 8GB of RAM). In fact, when I compile and run my driver on an x86 system it completes the scan - mainly because it completes scanning UNDER the 3GB mark. Now, it may be that my RAM is not contiguously mapped in physical address space. I imagine my Video card is taking up some of that address space since it shares with RAM on the northbridge.
Nope. The issue is the PCI hole. The BIOS is required to assign a
section of physical address space for PCI mappings, and it has to be
below the 4GB mark. So, on a system with 8 GB RAM, you might find
physical memory mapped from 0 to 3 GB, and then from 4 GB to 9 GB. The
region from 3 GB to 4 GB is assigned to PCI devices.
The exact address of the hole varies by BIOS.
Also, why would it be crashing…I’m not an expert on memory dumps, but even if I was trying to map a view of a section of physical address space that doesn’t below to Physical Memory, why would this cause it to crash?
You probably mapped a block that has already been mapped for a device,
and your caching parameters conflicted with the caching setup of the
existing block.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
xxxxx@osr.com wrote:
Oh, quite right. I assume WHQL wouldn’t detect this if your driver did it once every second (depending on HOW you did it).
As long as your driver passes the tests for your device class,I think you’d be all good.
Yes. After all, WHQL doesn’t actually RUN the drivers it certifies.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Well, the distinction here goes between “may” and “can”.
Can a driver screw any arbitrary memory? Of course. May it? No, it should
not do that. Precisely because it’s trusted NOT to do that.
wrote in message news:xxxxx@ntdev…
> Well… I hate to disagree with Mr. Grig, but all drivers are trusted
> parts of the system. Period. They’re inherently part of the trusted
> computing base, and they have access to any location in the kernel virtual
> address space, whether they’ve allocated it or not, by virtue of the fact
> that they’re RUNNING in the kernel virtual address space.
>
> Now, that doesn’t mean it’s good POLICY to actively facilitate drivers
> doing undisciplined things and accessing all of physical memory whenever
> they want.
>
> The “memory dumping tools” that are “out there” are mostly pure shite. I
> mean… consider the problem of capturing all of physical memory WHILE
> YOU’RE EXECUTING IN IT and while interrupts are enabled. Is the state of
> physical memory likely to remain consistent throughout your capturing
> process? The answer to that would be “no” – and thus, what IS it that
> you’re really capturing. In my opinion, most of these “capture and
> analyze memory” programs are nothing more than a way to blow a whole lot
> of smoke up somebody’s ass who doesn’t truly understand what they’re
> getting.
>
> So… YES, it can be made to work. You CAN do what you’re asking. There
> are multiple ways to do it (I mean, heck… why not just change the
> contents of one of your PTEs??)… What I – and a couple of other folks
> here – have been trying to ask is: “What are you trying to accomplish,
> and why do you think this is a good idea?” as well as “Have you thought
> this through” and the implicit question “who are you and what are your
> intentions”??
>
> These being your first posts here, Mr. “Oh My” from
> “guerrillamailblock”… you should EXPECT us to ask you a few questions,
> you know? This isn’t rootkit.com and we’re not in the business of
> actively facilitating the development of malware,
>
> Peter
> OSR
>
>
TLB also keeps the caching attributes for a page, besides from its base
address.
But it’s not a danger. If the same physical page is double-mapped with
different caching, I’m afraid it may cause very “interesting” side effects
if it’s accessed through these two addresses.
wrote in message news:xxxxx@ntdev…
> [quote]
>
> Mapping arbitrary physical memory regions pulled out of the hat (i.e. not
> explicitly allocated for your exclusive usage by a proper allocator of
> some sort) with \Device\PhysicalMemory does not count as following the
> rules and you are not guaranteed that there won’t now (or in the future)
> be a caching attribute conflict that will cause TLB corruption. For
> instance you are not protected against some innocent party creating a
> future mapping through whichever allocator rightfully owns that physical
> memory region with conflicting attributes with respect to an illegally
> created mapping.
>
> [/quote]
>
> The only thing that I can say here is that you are confusing TLBs with
> caching type…
>
>
> Anton Bassov
>
A well documented example of which, as my friend Andrew Rogers mentioned to me some time ago, could be found by searching for “linux cache attribute conflict”.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alexander Grigoriev
Sent: Thursday, April 08, 2010 7:02 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] PhysicalMemory and Address Space on x64
TLB also keeps the caching attributes for a page, besides from its base
address.
But it’s not a danger. If the same physical page is double-mapped with
different caching, I’m afraid it may cause very “interesting” side effects
if it’s accessed through these two addresses.
wrote in message news:xxxxx@ntdev…
> [quote]
>
> Mapping arbitrary physical memory regions pulled out of the hat (i.e. not
> explicitly allocated for your exclusive usage by a proper allocator of
> some sort) with \Device\PhysicalMemory does not count as following the
> rules and you are not guaranteed that there won’t now (or in the future)
> be a caching attribute conflict that will cause TLB corruption. For
> instance you are not protected against some innocent party creating a
> future mapping through whichever allocator rightfully owns that physical
> memory region with conflicting attributes with respect to an illegally
> created mapping.
>
> [/quote]
>
> The only thing that I can say here is that you are confusing TLBs with
> caching type…
>
>
> Anton Bassov
>
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
> Now I used a tool (MemInfo) to determine which physical addresses
are mapped for PhysicalMemory and I’m reading only those regions
of the physical address space - this at least solved the crashing
problem and I’m able to ATTEMPT to read through all physical
addresses mapped to physical memory. But now I have a new
problem and that’s a good number of those regions are giving me
ZwOpenViewOfSection failures - as a note I’m only reading a page
at a time.
We had this discussion here several times before. Skywing is right,
there is no way to safely map physical memory that you don’t own.
This thread has more details, including references to Intel
documentation:
http://www.osronline.com/showThread.cfm?link=173359
And this one explains why ZwMapViewOfSection might be failing
on some physical ranges:
https://www.osronline.com/showthread.cfm?link=164773
–
Pavel Lebedinsky/Windows Fundamentals Test
This posting is provided “AS IS” with no warranties, and confers no rights.
Well, I appreciate the time Ken and Pavel have taken to attempt to clarify this issue so far.
So, I think most of us are aware of the problem with using MmMapLockedPagesSpecifyCache with the wrong cache type. And I think we also get the (theoretical) problem with MmMapIoSpace (which ignores what you specify as caching type anyhow… but that’s a matter of implementation and not architecture).
However, I’m not sure that I understand the issue with \Device\PhysicalMemory… but I MIGHT have learned something today. Let’s see if I understand correctly what you guys (Ken and Pavel) have been saying:
I hear you saying is that \Device\PhysicalMemory is useless for mapping arbitrary physical pages into kernel virtual address space (such as, ah, to access parts of the BIOS or whatever). Because, even though you open the section and have a handle to it, you don’t “own” (for some value of “own”) the underlying physical memory pages… is that correct?
So, the ONE AND ONLY proper use of \Device\PhysicalMemory is to allow you to get a handle to the section, and then map a view to the PART of that section back to a given user mode process, IF (and only if) the part of the section you map back to user mode is memory that you’ve previously explicitly allocated or referenced through the memory manager.
And this is all simply because we can’t map the page as Uncached or WriteCombined and WriteBack at the same time, because to do so will cause memory corruption (which we all know)… and the Mm doesn’t keep track of this attribute anywhere for arbitrary pages… even though MmMapViewOfSection doesn’t even ALLOW us to specify the caching type?
But… wait… This is where I get confused:
The Mm DOES know the caching type (UC, WC, or WB) of each page of physical memory (the PFN has a pointer to a PTE that maps the page) at any given point in time. And given that when I call MmMapViewOfSection I don’t CARE about the memory caching type, and by making this call the Mm knows that I have a view to a set of pages… he can ensure my view is consistent. So, exactly when can this be a problem, in terms of mapping an arbitrary page from \Device\PhysicalMemory into kernel virtual address space?
If the “owner” frees a given page Mm STILL knows I’ve got the page mapped. If, later, somebody tries to change the mapping (and I’m not sure how common this even IS), the Mm STILL knows what my mapping is and could either (a) silently convert the other reference to my mapping type, or (b) fail the other mapping operation. Because, in the end, UC, WC or WB is in fact only something that we specify or care about for VERY FEW pages, right?
Again… to be clear: I’m not trying to argue. I’m trying to understand WHY this particular proscription is necessary. I am *not* trying to defend the OPs concept of mapping arbitrary pages back to user mode. I’m trying to understand how this works in Mm and why this is an issue. And I feel like I’m missing something.
Peter
OSR
I’d like to thank EVERYONE for participating in such a lively debate! As a novice in kernel development, I have learned quite a bit from this discussion and why it tends to be unsafe reading from arbitrary sections of physical address space - in particular physical memory’s physical address blocks.
Peter’s last comment opened up some thought discussions on how the Memory Manager deals with changes in the caching type and when the Mm knows who still has ownership or even a view to physical pages. What confuses me however, is in requesting a view, how that interferes with another kernel process accessing that same view and if that is what can cause memory corruption to the PTEs.
Everyone keeps talking about PFNs, aren’t those just the blocks of physical memory address space? And can one reference them from kernel-mode. I understand about using PDEs and PTEs to determine the physical addresses for process memory, but what is the relation to PFNs?
PTEs map virtual addresses to PFN’s (in general; they don’t have to).
You might want to search ‘Converting Virtual Addresses to Physical Addresses’ in the windbg docs.
mm
Well, as you can see, this abbreviation means multiple things. Mr. O’Brient is entirely correct, of course. PTEs map VAs to pages, AKA PFNs.
However, MY reference to “The PFN” was specifically referring to the Windows Page Frame Number Database, which is conventionally referred to as the PFN. This is the vector the Memory Manager uses to track the usage of each physical page in memory that is (unsurprisingly) indexed by page frame number. The PFN entry for each page has things like a count of active I/O references to the page (incremented when you call, for example, MmProbeAndLockPagesXxx) as well as a back pointer to ONE (typically, the first, but this can actually vary) PTE that references the page.
See Windows Internals by Russinovich and Soloman, look under Page Frame Number Database in the index… there’s a little diagram and everything, IIRC. If you don’t have this book, go out and get it.
Peter
OSR
> -----Original Message-----
From: xxxxx@osr.com [mailto:xxxxx@osr.com]
See Windows Internals by Russinovich and Soloman, look under Page
Frame Number
Database in the index… there’s a little diagram and everything,
IIRC. If
you don’t have this book, go out and get it.
The OP stated yesterday that he wanted to understand Windows memory
layout more. If he does not have that book he is wasting his time with
bits, when much of the data he may be looking for is explained there.
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
> And given that when I call MmMapViewOfSection I don’t CARE about the memory caching
type, and by making this call the Mm knows that I have a view to a set of pages… he can ensure
my view is consistent
This is a good question…
On one hand, there is no such thing as reverse mapping under Windows (at least pre-Win7 versions). Therefore, MM cannot ensure cache consistency between different mappings simply because it cannot
detect all PTEs that map a given physical page into the virtual memory- just consider what happens if MmMapViewOfSection() maps with WC attribute a page that had been earlier mapped by, say, MmMapIoSpace() as uncacheable.
OTOH, not so long ago Jake said that MmMapIoSpace() tries to resolve the cache type conflict,which implies that Windows implements at least some form of reverse mapping (for example, on range basis). The most interesting thing is that at the same time, he was insisting that MmMapIoSpace() does not care if the physical address in itself is valid so that it does not look at PFN…
Anton Bassov
There has traditionally been no COMPREHENSIVE reverse mapping (from PFN entry to every view of that page). I’m personally not sure if that changed in Win7… I’ve heard some rumors.
But, in any case, as I said in my previous two posts there is a SINGLE reverse mapping (from PFN to PTE) and it seems to ME that’s all you’d need to keep the caching consistent. When a page is first instantiated, the caching attribute is established. Then each new mapping of that page has to be consistent with that mapping.
So, clearly, I’m missing something.
Peter
OSR