Anyway IIRC when driver’s ISR code is running (in interrupt context) it shouldn’t access controlled device’s registers via VAs because it could incurr in a BSOD (page tables could be paget out) So i guess ISR should use always CPU-relative physical addresses
Does it make sense ?
No. When your driver maps its device registers into a virtual address,
it is mapping into KERNEL virtual space. A kernel virtual address is
valid in all contexts. Plus, device registers cannot be paged out.
Programs and drivers never uses physical addresses. I defer to Joe’s
eloquent and lengthy proof of this assertion.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
PFN 0 may or may not have an (initialized) entry in the PFN database, depending on what it’s used for and how it is marked by the firmware and the boot loader.
In your example below, PFN 0 does have a valid entry. On other systems its entry may be uninitialized, containing all zeroes.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@alice.it
Sent: Sunday, April 22, 2012 1:21 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] PCI bus-relative (physical) addresses
It doesn’t include device ranges or even pages that are in RAM but not part of the physical memory map (e.g. PFN 0)
Using lkd (windbg) on my win xp 32-bit client laptop I can see:
lkd> !pfn 0
PFN 00000000 at address 81DA6000
flink 00000000 blink / share count 00000001 pteaddress C07FE848
reference count 0001 Cached color 0
restore pte 00000000 containing page 000741 Active
lkd> !pte C07FE848
VA ffd09000
PDE at C0603FF0 PTE at C07FE848
contains 0000000000741163 contains 0000000000000163
pfn 741 -G-DA–KWEV pfn 0 -G-DA–KWEV
lkd> !pte ffd09000
VA ffd09000
PDE at C0603FF0 PTE at C07FE848
contains 0000000000741163 contains 0000000000000163
pfn 741 -G-DA–KWEV pfn 0 -G-DA–KWEV
Now it seem to me PFN 0 exist in PFN database (IIUC PFN 0 is page frame starting at CPU-relative physical address 0x00000000)
> No. When your driver maps its device registers into a virtual address, it is mapping into KERNEL virtual space. A kernel virtual address is valid in all contexts. Plus, device registers cannot be paged out
Maybe I was unclear…of course paging in not applicable, by definition, to device’s memory-mapped ranges. My question was related to page tables which map VAs to these physical regions (AFAIK, quite general, page tables pages themselves can be paged out including page table mapping system (kernel) VAs )
However as Anton pointed out “It is OS’s responsibility to ensure that accessing memory that had been mapped with MmMapIoSpace() does not result in page faults. Therefore, all page tables that are used for mapping the target page are going to be locked in RAM…”
So driver’s code can safety access its mapped device registers into system (kernel) virtual space…
> Can you find that conversation? AFAIK, that still doesn’t exist
in Windows 7. If you can find that support somewhere I’m sure
you’ll make lots of people happy though.
In what scenarios would this be useful?
There was some semi-related support added in Windows 7 to track
mappings on a per-control area basis. This allows support for a
new API that invalidates user mapped views when flushing/purging
cached data from a file system, but it’s a far cry from a
complete reverse lookup table.
This is what Anton was talking about.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Scott Noone
Sent: Monday, April 23, 2012 7:51 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] PCI bus-relative (physical) addresses
“Pavel Lebedynskiy” wrote in message news:xxxxx@ntdev…
> Can you find that conversation? AFAIK, that still doesn’t exist
> in Windows 7. If you can find that support somewhere I’m sure
> you’ll make lots of people happy though.
In what scenarios would this be useful?
Aside from various flavors of the coherency problem, this comes up often in
relation to forensic analysis.
Often you have a memory dump that you scan for interesting patterns/data.
Given you find something interesting, the next step is to figure out who is
actually using this data. All of the existing forensics toolkits achieve
this using a brute force approach, but an O/S supported mechanism would be
useful.
-scott
–
Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc. http://www.osronline.com
forensics toolkits achieve this using a brute force approach, but an O/S
supported mechanism would be useful.
-scott
“All,” including the toolkits to which you do not have access?
There is the page frame database, and IIRC that does have a reference
> count for a page, but that is as far as it goes. <
In many cases you also get a pointer to a page table entry which
references the page and from which you can determine the VA and the
process(es) which references the page. Unfortunately, this does not
tell you whether or not the page has been remapped to additional VA’s.
OK, you can maintain a refcount in a prototype PTE, rather than PFN entry, if you wish - you would still need
to store a pointer to prototype PTE in PFN entry itself, right. Otherwise you would be unable to track the usage of the page…
Well, I made it clear that I was speaking about the reverse lookup mechanism and not about the conventional list/table, which, indeed, would be pretty naive implementation of above mentioned mechanism.
I don’t really know what Windows 7 actually does, but judging from your description of it (i.e tracking mappings on a per-control area basis) it seems to be pretty similar to Linux approach of priority search trees. The idea is to, instead of naively linking PTEs that point to a page, link memory regions that map a file
(in Windows terms, file views) into a “variation on the theme” of McCreight’s tree, which is a hybrid of a heap and a balanced search tree that allows you to search the intervals that may overlap one another. Therefore, as long as you know that a given physical page maps a file ABC at the offset XYZ you are able to quickly locate all PTEs that map a given page without actually maintaining a list of these PTEs.
Concerning the conventional list/table, Linux uses this approach only for anonymous pages, i.e. stacks, heaps and bss. Once a page that backs up an anonymous mapping is very unlikely to have
been mapped into more than just a few address spaces, the simplistic list suffices. However, it would be pretty cumbersome and inefficient approach to take with pages that back up memory-mapped files - such a page that may map, without exaggeration, into thousands of different address spaces…
I checked PFN database with undocumented !pfn 0 1 command
Page column number (starting from 1) is contiguous (653262 entries). Maybe some of them refers to physical memory regions in which device’s registers are mapped to…in some sense these PFN entries has to be considered as “invalid”
I checked PFN database with undocumented !pfn 0 1 command
Page column number (starting from 1) is contiguous (653262 entries). Maybe some of them refers to physical memory regions in which device’s registers are mapped to…in some sense these PFN entries has to be considered as “invalid”
Actually, a PFN database entry either belongs to the OS physical memory
map or not depending on the nt!MiPfnBitMap. It is binary, 0 or 1 for
each page up to nt!MmHighestPossiblePhysicalPage. Starting with Windows
7 the PFN database can be sparse. There may be whole ranges of PFN
entries that aren’t mapped into virtual memory at all. You need to
check the nt!MiPfnBitMap first to see if an entry is valid before trying
to access it.
Why do you think that you think that the OS has to work the same way as
the !pfn debugger extension?
“George M. Garner Jr.” wrote in message news:xxxxx@ntdev…
“All,” including the toolkits to which you do not have access?
Always dangerous to speak in absolutes, of course, but even if it’s not the
only option I would hope that it’s an option. As this whole conversation
centers around the fact that Windows does not reliably track this
information, it makes sense to want to see things the way the hardware sees
them. Even if Windows *did* track this information I wouldn’t trust it as a
source of reliable information, though it would be interesting as a cross
reference.
In many cases you also get a pointer to a page table entry which references
the page and from which you can determine the VA and the process(es) which
references the page
Through what mechanism do you turn that single PTE tracked by the PFN into
all of the references to the page? That’s really the key point of the
conversation. In many cases that single PTE is a Prototype PTE, which is
usually the end of the line as it doesn’t have anything to do with other
valid mappings to the page (it only comes into play when another mapping
becomes invalid).
Regards,
-scott
–
Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc. http://www.osronline.com
wrote in message news:… >…you would still needstore a pointer to prototype PTE in PFN entry >itself, right. >Otherwise you would be unable to track the usage of the page…
Right. But that’s a single PTE (well, sort of a PTE), not all PTEs.
>The idea is to, instead of naively linking PTEs that point to a page, link >memory regions that map a file
Exactly, it’s not really a mapping of PFNs->PTEs, but a mapping of Control Area->Views.
-scott
– Scott Noone Consulting Associate and Chief System Problem Analyst OSR Open Systems Resources, Inc. http://www.osronline.com
As this whole
conversation centers around the fact that Windows does not reliably
track this information, it makes sense to want to see things the way the
hardware sees them. Even if Windows *did* track this information I
wouldn’t trust it as a source of reliable information, though it would
be interesting as a cross reference.
I see your point. At most the PFN database gives you a pointer to one
page table entry. Depending on how the page was allocated it may not
even give you that. So there is no *reliable* Windows track
information. Even if Windows did attempt to track all VA-to-page
mappings it would not be reliable in the context of malicious code. If
I am running at ring 0 I can create my own PTE with whatever contents I
like. As long as the PTE contents follow the format specified by the
hardware reference manual the processor will dereference the
corresponding VA to access the contents of the page.
Still, the PFN database does give us some very valuable information
about who is using a page in a lot of cases.
“George M. Garner Jr.” wrote in message news:xxxxx@ntdev…
Still, the PFN database does give us some very valuable information about
who is using a page in a lot of cases.
Absolutely, I didn’t mean to indicate that brute force was the only way to
get interesting information. And in a lot of cases just going to the PFN
entry provides enough information, though it’s important to keep in mind the
caveats (and the caveats are where all the fun stuff is anyway :)).
Regards,
-scott
–
Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc. http://www.osronline.com
>>The idea is to, instead of naively linking PTEs that point to a page, link
>memory regions that map a file
Exactly, it’s not really a mapping of PFNs->PTEs, but a mapping of Control Area->Views.
However, some well-known individuals still have the audacity to claim that Linux supposedly violates some of their patents patents…and to do so while shamelessly reimplementing the ideas and concepts that had been implemented by Linux kernel ages ago…
As you can see, bit 0 is not set on this legacy BIOS-based system . However, as Pavel alluded, Windows runs on a lot of different architectures now which may not follow the traditional paradigm.
There is also the nt!MmPhysicalMemoryBlock which records the physical address ranges which form a part of the Windows physical memory map. In theory, the nt!MmPhysicalMemoryBlock should be coherent with the PFN bitmap. However, you know…
On 4/24/12, George M. Garner Jr. wrote: > On 4/24/2012 10:46 AM, xxxxx@alice.it wrote: >>> Actually, a PFN database entry either belongs to the OS physical memory >>> map >> >> So, using other tools is it possible to view OS (Windows) physical memory >> map ? >> > > Is there something wrong with Windbg? > > kd> x nt!MiPfnBitMap > 827a37c0 nt!MiPfnBitMap = > kd> dt -b nt!_RTL_BITMAP 827a37c0 > +0x000 SizeOfBitMap : 0x77ef0 > +0x004 Buffer : 0x8433f000 > kd> db 8433f000 > 8433f000 fe ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff … > 8433f010 ff ff ff 7f 00 00 00 00-00 00 00 00 00 00 00 00 … > 8433f020 ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff … > 8433f030 ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff … > 8433f040 ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff … > 8433f050 ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff … > 8433f060 ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff … > 8433f070 ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff … > > As you can see, bit 0 is not set on this legacy BIOS-based system . > However, as Pavel alluded, Windows runs on a lot of different > architectures now which may not follow the traditional paradigm. > > There is also the nt!MmPhysicalMemoryBlock which records the physical > address ranges which form a part of the Windows physical memory map. In > theory, the nt!MmPhysicalMemoryBlock should be coherent with the PFN > bitmap. However, you know… > > — > NTDEV is sponsored by OSR > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at > http://www.osronline.com/page.cfm?name=ListServer >
lkd> x nt!MiPfnBitMap
8055d6a8 nt!MiPfnBitMap = > > lkd> dt -b nt!_RTL_BITMAP 8055d6a8 > +0x000 SizeOfBitMap : 0x9f7d0 > +0x004 Buffer : 0x8af01000 > > Now 0x9f7d0 = 653264 I think is the size of bitmap in bit… So 653264/8 = 81658 is the bitmap’s lenght in Byte, right ? >