> I am not aware that any such “database” actually exists.
…
but having worked in the low-level guts of several operating systems in my life, I never once
saw anything resembling the table you propose.
Of course it does exist - otherwise the OS of any description will be unable to track the info of pages
(for example, how many VAs a given physical pages maps to, discover what these addresses actually are and to what address spaces they belong, etc). What the OP asks is whether this list describes only actual RAM or just everything that can be accessed as memory. …
Having attended talks on Windows memory subsystem internals multiple
times, the one constant is “there is no inverted page table”, i.e. the
is no physical to virtual database. There is the page frame database,
and IIRC that does have a reference count for a page, but that is as far
as it goes.
> > I am not aware that any such “database” actually exists. > > … > > > but having worked in the low-level guts of several operating systems in my life, I never once > > saw anything resembling the table you propose. > > Of course it does exist - otherwise the OS of any description will be unable to track the info of pages > (for example, how many VAs a given physical pages maps to, discover what these addresses actually are and to what address spaces they belong, etc). What the OP asks is whether this list describes only actual RAM or just everything that can be accessed as memory. … > > Anton Bassov
Having attended talks on Windows memory subsystem internals multiple times, the one constant
is “there is no inverted page table”,
Indeed, historically this has been the case. However, IIRC, someone from MSFT was saying in this NG that they had finally implemented reverse mapping (i.e the feature that some other OSes have had for many years) in Windows 7…
i.e. the is no physical to virtual database.
Well, it does not really have to be a “database”, i.e. a list of all PTEs that point to a page, because it would be terribly inefficient approach (just imagine the length of such list for a page from, say, NTDLL’s executable section). Therefore, some more sophisticated solution is needed. For example, Linux provides its implementation of reverse mapping mechanism that is based upon priority search trees. This mechanism allows one to quickly locate everyone who maps a given page without actually maintaining a list of PTEs…
The OP made an invalid assumption: that the mapping is 1:1. This is
simply not true. Using memory-mapped files, shared-segment DLLs, and
shared-segment executables, plus MmGetSystemAddressForMdlSafe (or whatever
wraps it in KMDF), any given page frame could have a nominally unbounded
set of virtual addresses. So a “virtual-to-physical” map, besides being
useless except as a snapshot which could be obsolete before it is
processed, tells you nothing interesting except if it is a snapshot in a
particular context (one process, for example). Only the locked pages are
interesting, and since this state is itself dynamic, except for the
nonpaged pool, the information is largely untrustworthy in an active
execution environment. When the virtual-to-physical mapping required for
DMA is performed, it essentially uses the PFNs of the specific MDL to
derive these addresses, and the PFNs are established by
MmProbeAndLockPages, so those pages are locked.
Which goes back to the original question, of why the OP thinks this
information might have value.
joe
Anton,
Having attended talks on Windows memory subsystem internals multiple
times, the one constant is “there is no inverted page table”, i.e. the
is no physical to virtual database. There is the page frame database,
and IIRC that does have a reference count for a page, but that is as far
as it goes.
“xxxxx@hotmail.com” wrote in message > news:xxxxx@ntdev: > >> > I am not aware that any such “database” actually exists. >> >> … >> >> > but having worked in the low-level guts of several operating systems >> in my life, I never once >> > saw anything resembling the table you propose. >> >> Of course it does exist - otherwise the OS of any description will be >> unable to track the info of pages >> (for example, how many VAs a given physical pages maps to, discover what >> these addresses actually are and to what address spaces they belong, >> etc). What the OP asks is whether this list describes only actual RAM or >> just everything that can be accessed as memory. … >> >> Anton Bassov > > > — > NTDEV is sponsored by OSR > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at > http://www.osronline.com/page.cfm?name=ListServer >
> The OP made an invalid assumption: that the mapping is 1:1.
You still don’t understand the OP’s question…
He does not ask us about “virtual-to-physical” map, which, indeed, would be just totally useless. He asks about
the list that describes physical page frames (in Linux terms, the list of struct PAGE structures) - he wonders if
it lists memory-mapped device ranges as well…
I do not ask about “virtual-to-physical” translation (performed via PTE’s) but about Page Frame Number (PFN) database.
From Windows Internals book PFN database has entries for each physical page frame (normally 4KB each) containing some page frame informations and its state. If it is included either in a working set list (process or system) or a non-paged system pool page maps to it, then its status is Active/Valid.
My guess is this is quite general: PFN entries exist for RAM page frame as well device memory-mapped ranges. Furthermore I guess page frame entries for memory-mapped ranges are always in Active/Valid state (as well RAM page frames non-paged system pool maps to)
So in Anton’s laptop there will exist PFN entries also for last Gigabyte memory hole starting at C0000000 where device’s register are mapped to…
The PFN database only includes pages that are part of the physical
memory map. It doesn’t include device ranges or even pages that are in
RAM but not part of the physical memory map (e.g. PFN 0).
> Which tool can i use on Win XP system (32 bit) to check how system RAM page frames are mapped
in CPU-relative phy address space ?
System RAM is mapped directly to the CPU.
More so, PCI BARs are mapped directly to the CPU. So, the address in a BAR is a CPU’s physical address.
Mapping only exists in DMA scenarios, if you have IOMMU logic in your north bridge, which is rather rare now. In this scenario, the DMA addresses set by the device go through the hardware mapping before being used as the RAM physical addresses.
> Anyway IIRC when driver’s ISR code is running (in interrupt context) it shouldn’t access controlled
device’s registers
Wrong. It should at least access the device status register to determine whether the device is actually interrupting. Also, in the end, it should update the device state so it will stop asserting the INT line.
via VAs because it could incurr in a BSOD (page tables could be paget out)
They can, but not for the nonpaged kernel space, where the MmMapIoSpace’d BAR’s live.
s/i see pfn/ i see pfn 0/
s / statred/ started/
s / i am not if / i am not sure if/
On 4/23/12, raj_r wrote: > in my laptop i see pfn as zeroed > > but there is an undocumented command !pfn 0 1 which would yield a very > very long output > and iirc it always statred with page 1 and not 0 for for me > > i am not if what lkd shows could be used as i think many of the values > might be stale not current etc > > > lkd> !pfn 0 1 > > Page Flink Blk/Shr Ref V PTE Address SavedPTE Frame State > 1 17702 1e6c0 0 e1661048 0 4c0 2dff7 Standby P > 2 2e090 145c9 0 e3f109e8 0 460 2aec4 Standby P > 3 1192 155c 0 e2c205e0 0 460 14aad Standby P > 4 0 1 1 c0029899 5313200 80 ffedcb Active > RW > 5 54a 1 1 c0001a20 344000 80 2ba2f Active M > 6 1e947 2efc5 0 e477bbf0 0 4c0 7b30 Standby P > > > > > On 4/23/12, xxxxx@alice.it wrote: >>>It doesn’t include device ranges or even pages that are in RAM but not >>> part >>> of the physical memory map (e.g. PFN 0) >> >> Using lkd (windbg) on my win xp 32-bit client laptop I can see: >> >> lkd> !pfn 0 >> PFN 00000000 at address 81DA6000 >> flink 00000000 blink / share count 00000001 pteaddress >> C07FE848 >> reference count 0001 Cached color 0 >> restore pte 00000000 containing page 000741 Active >> >> lkd> !pte C07FE848 >> VA ffd09000 >> PDE at C0603FF0 PTE at C07FE848 >> contains 0000000000741163 contains 0000000000000163 >> pfn 741 -G-DA–KWEV pfn 0 -G-DA–KWEV >> >> lkd> !pte ffd09000 >> VA ffd09000 >> PDE at C0603FF0 PTE at C07FE848 >> contains 0000000000741163 contains 0000000000000163 >> pfn 741 -G-DA–KWEV pfn 0 -G-DA–KWEV >> >> Now it seem to me PFN 0 exist in PFN database (IIUC PFN 0 is page frame >> starting at CPU-relative physical address 0x00000000) >> >> Can you help me ? >> >> — >> NTDEV is sponsored by OSR >> >> For our schedule of WDF, WDM, debugging and other seminars visit: >> http://www.osr.com/seminars >> >> To unsubscribe, visit the List Server section of OSR Online at >> http://www.osronline.com/page.cfm?name=ListServer >> >
but there is an undocumented command !pfn 0 1 which would yield a very
very long output
and iirc it always statred with page 1 and not 0 for for me
i am not if what lkd shows could be used as i think many of the values
might be stale not current etc
lkd> !pfn 0 1
Page Flink Blk/Shr Ref V PTE Address SavedPTE Frame State
1 17702 1e6c0 0 e1661048 0 4c0 2dff7 Standby P
2 2e090 145c9 0 e3f109e8 0 460 2aec4 Standby P
3 1192 155c 0 e2c205e0 0 460 14aad Standby P
4 0 1 1 c0029899 5313200 80 ffedcb Active RW
5 54a 1 1 c0001a20 344000 80 2ba2f Active M
6 1e947 2efc5 0 e477bbf0 0 4c0 7b30 Standby P
On 4/23/12, xxxxx@alice.it wrote: >>It doesn’t include device ranges or even pages that are in RAM but not part >> of the physical memory map (e.g. PFN 0) > > Using lkd (windbg) on my win xp 32-bit client laptop I can see: > > lkd> !pfn 0 > PFN 00000000 at address 81DA6000 > flink 00000000 blink / share count 00000001 pteaddress C07FE848 > reference count 0001 Cached color 0 > restore pte 00000000 containing page 000741 Active > > lkd> !pte C07FE848 > VA ffd09000 > PDE at C0603FF0 PTE at C07FE848 > contains 0000000000741163 contains 0000000000000163 > pfn 741 -G-DA–KWEV pfn 0 -G-DA–KWEV > > lkd> !pte ffd09000 > VA ffd09000 > PDE at C0603FF0 PTE at C07FE848 > contains 0000000000741163 contains 0000000000000163 > pfn 741 -G-DA–KWEV pfn 0 -G-DA–KWEV > > Now it seem to me PFN 0 exist in PFN database (IIUC PFN 0 is page frame > starting at CPU-relative physical address 0x00000000) > > Can you help me ? > > — > NTDEV is sponsored by OSR > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at > http://www.osronline.com/page.cfm?name=ListServer >
On 4/23/12, raj_r wrote: > s/i see pfn/ i see pfn 0/ > s / statred/ started/ > s / i am not if / i am not sure if/ > > On 4/23/12, raj_r wrote: >> in my laptop i see pfn as zeroed >> >> but there is an undocumented command !pfn 0 1 which would yield a very >> very long output >> and iirc it always statred with page 1 and not 0 for for me >> >> i am not if what lkd shows could be used as i think many of the values >> might be stale not current etc >> >> >> lkd> !pfn 0 1 >> >> Page Flink Blk/Shr Ref V PTE Address SavedPTE Frame State >> 1 17702 1e6c0 0 e1661048 0 4c0 2dff7 Standby >> P >> 2 2e090 145c9 0 e3f109e8 0 460 2aec4 Standby >> P >> 3 1192 155c 0 e2c205e0 0 460 14aad Standby >> P >> 4 0 1 1 c0029899 5313200 80 ffedcb Active >> RW >> 5 54a 1 1 c0001a20 344000 80 2ba2f Active >> M >> 6 1e947 2efc5 0 e477bbf0 0 4c0 7b30 Standby >> P >> >> >> >> >> On 4/23/12, xxxxx@alice.it wrote: >>>>It doesn’t include device ranges or even pages that are in RAM but not >>>> part >>>> of the physical memory map (e.g. PFN 0) >>> >>> Using lkd (windbg) on my win xp 32-bit client laptop I can see: >>> >>> lkd> !pfn 0 >>> PFN 00000000 at address 81DA6000 >>> flink 00000000 blink / share count 00000001 pteaddress >>> C07FE848 >>> reference count 0001 Cached color 0 >>> restore pte 00000000 containing page 000741 Active >>> >>> lkd> !pte C07FE848 >>> VA ffd09000 >>> PDE at C0603FF0 PTE at C07FE848 >>> contains 0000000000741163 contains 0000000000000163 >>> pfn 741 -G-DA–KWEV pfn 0 -G-DA–KWEV >>> >>> lkd> !pte ffd09000 >>> VA ffd09000 >>> PDE at C0603FF0 PTE at C07FE848 >>> contains 0000000000741163 contains 0000000000000163 >>> pfn 741 -G-DA–KWEV pfn 0 -G-DA–KWEV >>> >>> Now it seem to me PFN 0 exist in PFN database (IIUC PFN 0 is page frame >>> starting at CPU-relative physical address 0x00000000) >>> >>> Can you help me ? >>> >>> — >>> NTDEV is sponsored by OSR >>> >>> For our schedule of WDF, WDM, debugging and other seminars visit: >>> http://www.osr.com/seminars >>> >>> To unsubscribe, visit the List Server section of OSR Online at >>> http://www.osronline.com/page.cfm?name=ListServer >>> >> >
wrote in message… >Of course it does exist - otherwise the OS of any description will be >unable to track the info of pages >(for example, how many VAs a given physical pages maps to, discover what >these addresses actually >are and to what address spaces they belong, etc).
Why do you assume that this information is required on Windows? Windows needs to know that multiple addresses point to a single page, but it doesn’t necessarily need to know which pages point to the physical page. The former is dealt with through the Prototype PTE concept, the latter is only addressed partially starting in Windows 7. Which brings me to the next point:
wrote in message… >Indeed, historically this has been the case. However, IIRC, someone from >MSFT was saying in this >NG that they had finally implemented reverse mapping (i.e the feature that >some other OSes have >had for many years) in Windows 7…
Can you find that conversation? AFAIK, that still doesn’t exist in Windows 7. If you can find that support somewhere I’m sure you’ll make lots of people happy though.
There was some semi-related support added in Windows 7 to track mappings on a per-control area basis. This allows support for a new API that invalidates user mapped views when flushing/purging cached data from a file system, but it’s a far cry from a complete reverse lookup table.
-scott
– Scott Noone Consulting Associate and Chief System Problem Analyst OSR Open Systems Resources, Inc. http://www.osronline.com
On 4/23/12, xxxxx@alice.it wrote: >> here is an output from my laptop lkd> .shell -ci “!address” grep -A 2 -B 2 >> -i “pfn” > > raj, .shell command on my XP laptop does not find “grep” utility…how can I > solve it ? >
a couple of ways
download install and add it to path so it is accessible