PFN Database, Original PTE field

OSR_Community_User · September 23, 2004, 2:24pm

The field Original PTE in each PFN Database entry is documented as being loaded with the original PTE contents.
However, it seems to contain only ‘invalid’ page states. I mean this field never contains a valid page state.
It preserves an invalid state all the time even if the corresponding PTE is valid currently.

According to Inside Microsoft Windows, page 475, ‘saving the contents of the PTE allows it to be restored when the physical page is no longer resident’. However, I cannot see it saves anything other than ‘invalid states’.

Which is the exact purpose of this field in the PFN database entries ? When is it used ?

Thanks.
Inaki.

Maxim_S_Shatskih · September 24, 2004, 2:21am

From what I know, the PFN entry is NEVER documented.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: I?aki Castillo
To: Windows System Software Devs Interest List
Sent: Thursday, September 23, 2004 10:21 PM
Subject: [ntdev] PFN Database, Original PTE field

The field Original PTE in each PFN Database entry is documented as being loaded with the original PTE contents.
However, it seems to contain only ‘invalid’ page states. I mean this field never contains a valid page state.
It preserves an invalid state all the time even if the corresponding PTE is valid currently.

According to Inside Microsoft Windows, page 475, ‘saving the contents of the PTE allows it to be restored when the physical page is no longer resident’. However, I cannot see it saves anything other than ‘invalid states’.

Which is the exact purpose of this field in the PFN database entries ? When is it used ?

Thanks.
Inaki.

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · September 24, 2004, 10:36am

Maybe, you are right, “for what you know”.

-----Mensaje original-----
De: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com]En nombre de Maxim S. Shatskih
Enviado el: viernes, 24 de septiembre de 2004 8:21
Para: Windows System Software Devs Interest List
Asunto: Re: [ntdev] PFN Database, Original PTE field

From what I know, the PFN entry is NEVER documented.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: I?aki Castillo mailto:xxxxx
To: Windows System Software Devs Interest mailto:xxxxx List
Sent: Thursday, September 23, 2004 10:21 PM
Subject: [ntdev] PFN Database, Original PTE field

The field Original PTE in each PFN Database entry is documented as being loaded with the original PTE contents.
However, it seems to contain only ‘invalid’ page states. I mean this field never contains a valid page state.
It preserves an invalid state all the time even if the corresponding PTE is valid currently.

According to Inside Microsoft Windows, page 475, ‘saving the contents of the PTE allows it to be restored when the physical page is no longer resident’. However, I cannot see it saves anything other than ‘invalid states’.

Which is the exact purpose of this field in the PFN database entries ? When is it used ?

Thanks.
Inaki.

—
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com</mailto:xxxxx></mailto:xxxxx>

OSR_Community_User · September 24, 2004, 11:55am

The format of the PFN entry is in the standard symbols on the symbol server:

lkd> dt nt!_MMPFN -b

+0x000 u1 : __unnamed

+0x000 Flink : Uint4B

+0x000 WsIndex : Uint4B

+0x000 Event : Ptr32

+0x000 ReadStatus : Int4B

+0x000 NextStackPfn : _SINGLE_LIST_ENTRY

+0x000 Next : Ptr32

+0x004 PteAddress : Ptr32

+0x008 u2 : __unnamed

+0x000 Blink : Uint4B

+0x000 ShareCount : Uint4B

+0x00c u3 : __unnamed

+0x000 e1 : _MMPFNENTRY

+0x000 Modified : Pos 0, 1 Bit

+0x000 ReadInProgress : Pos 1, 1 Bit

+0x000 WriteInProgress : Pos 2, 1 Bit

+0x000 PrototypePte : Pos 3, 1 Bit

+0x000 PageColor : Pos 4, 3 Bits

+0x000 ParityError : Pos 7, 1 Bit

+0x000 PageLocation : Pos 8, 3 Bits

+0x000 RemovalRequested : Pos 11, 1 Bit

+0x000 CacheAttribute : Pos 12, 2 Bits

+0x000 Rom : Pos 14, 1 Bit

+0x000 LockCharged : Pos 15, 1 Bit

+0x000 DontUse : Pos 16, 16 Bits

+0x000 e2 : __unnamed

+0x000 ShortFlags : Uint2B

+0x002 ReferenceCount : Uint2B

+0x010 OriginalPte : _MMPTE

+0x000 u : __unnamed

+0x000 Long : Uint4B

+0x000 Hard : _MMPTE_HARDWARE

+0x000 Valid : Pos 0, 1 Bit

+0x000 Writable : Pos 1, 1 Bit

+0x000 Owner : Pos 2, 1 Bit

+0x000 WriteThrough : Pos 3, 1 Bit

+0x000 CacheDisable : Pos 4, 1 Bit

+0x000 Accessed : Pos 5, 1 Bit

+0x000 Dirty : Pos 6, 1 Bit

+0x000 LargePage : Pos 7, 1 Bit

+0x000 Global : Pos 8, 1 Bit

+0x000 CopyOnWrite : Pos 9, 1 Bit

+0x000 Prototype : Pos 10, 1 Bit

+0x000 Write : Pos 11, 1 Bit

+0x000 PageFrameNumber : Pos 12, 20 Bits

+0x000 Flush : _HARDWARE_PTE

+0x000 Valid : Pos 0, 1 Bit

+0x000 Write : Pos 1, 1 Bit

+0x000 Owner : Pos 2, 1 Bit

+0x000 WriteThrough : Pos 3, 1 Bit

+0x000 CacheDisable : Pos 4, 1 Bit

+0x000 Accessed : Pos 5, 1 Bit

+0x000 Dirty : Pos 6, 1 Bit

+0x000 LargePage : Pos 7, 1 Bit

+0x000 Global : Pos 8, 1 Bit

+0x000 CopyOnWrite : Pos 9, 1 Bit

+0x000 Prototype : Pos 10, 1 Bit

+0x000 reserved : Pos 11, 1 Bit

+0x000 PageFrameNumber : Pos 12, 20 Bits

+0x000 Proto : _MMPTE_PROTOTYPE

+0x000 Valid : Pos 0, 1 Bit

+0x000 ProtoAddressLow : Pos 1, 7 Bits

+0x000 ReadOnly : Pos 8, 1 Bit

+0x000 WhichPool : Pos 9, 1 Bit

+0x000 Prototype : Pos 10, 1 Bit

+0x000 ProtoAddressHigh : Pos 11, 21 Bits

+0x000 Soft : _MMPTE_SOFTWARE

+0x000 Valid : Pos 0, 1 Bit

+0x000 PageFileLow : Pos 1, 4 Bits

+0x000 Protection : Pos 5, 5 Bits

+0x000 Prototype : Pos 10, 1 Bit

+0x000 Transition : Pos 11, 1 Bit

+0x000 PageFileHigh : Pos 12, 20 Bits

+0x000 Trans : _MMPTE_TRANSITION

+0x000 Valid : Pos 0, 1 Bit

+0x000 Write : Pos 1, 1 Bit

+0x000 Owner : Pos 2, 1 Bit

+0x000 WriteThrough : Pos 3, 1 Bit

+0x000 CacheDisable : Pos 4, 1 Bit

+0x000 Protection : Pos 5, 5 Bits

+0x000 Prototype : Pos 10, 1 Bit

+0x000 Transition : Pos 11, 1 Bit

+0x000 PageFrameNumber : Pos 12, 20 Bits

+0x000 Subsect : _MMPTE_SUBSECTION

+0x000 Valid : Pos 0, 1 Bit

+0x000 SubsectionAddressLow : Pos 1, 4 Bits

+0x000 Protection : Pos 5, 5 Bits

+0x000 Prototype : Pos 10, 1 Bit

+0x000 SubsectionAddressHigh : Pos 11, 20 Bits

+0x000 WhichPool : Pos 31, 1 Bit

+0x000 List : _MMPTE_LIST

+0x000 Valid : Pos 0, 1 Bit

+0x000 OneEntry : Pos 1, 1 Bit

+0x000 filler0 : Pos 2, 8 Bits

+0x000 Prototype : Pos 10, 1 Bit

+0x000 filler1 : Pos 11, 1 Bit

+0x000 NextEntry : Pos 12, 20 Bits

+0x014 u4 : __unnamed

+0x000 EntireFrame : Uint4B

+0x000 PteFrame : Pos 0, 26 Bits

+0x000 InPageError : Pos 26, 1 Bit

+0x000 VerifierAllocation : Pos 27, 1 Bit

+0x000 AweAllocation : Pos 28, 1 Bit

+0x000 LockCharged : Pos 29, 1 Bit

+0x000 KernelStack : Pos 30, 1 Bit

+0x000 Reserved : Pos 31, 1 Bit

Note that the OriginalPTE field has seven different formats. Interpreting the contents of that would thus depend upon knowing the correct context of the PFN entry itself. I *can* assure you that there are no unused fields in the PFN. On the contrary, the fields of the PFN are often overloaded and used to mean different things in different contexts, depending upon the state of the specific page.

Actually, after considering the text you quoted I would agree that what you observe is consistent with the text. If the page is removed from the page table, there wouldn’t be cases where the “original PTE” would ever be valid. Thus, the PTE that it would restore would need to tell the memory manager how to locate the data for that page (since the page itself is not referenced in the PTE).

Regards,

Tony

Tony Mason

Consulting Partner

OSR Open Systems Resources, Inc.

http://www.osr.com

Looking forward to seeing you at the Next OSR File Systems Class October 18, 2004 in Silicon Valley!

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of I?aki Castillo
Sent: Friday, September 24, 2004 10:33 AM
To: ntdev redirect
Subject: RE: [ntdev] PFN Database, Original PTE field

Maybe, you are right, “for what you know”.

-----Mensaje original-----
De: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com]En nombre de Maxim S. Shatskih
Enviado el: viernes, 24 de septiembre de 2004 8:21
Para: Windows System Software Devs Interest List
Asunto: Re: [ntdev] PFN Database, Original PTE field

From what I know, the PFN entry is NEVER documented.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----

From: I?aki Castillo mailto:xxxxx

To: Windows System Software Devs Interest List mailto:xxxxx

Sent: Thursday, September 23, 2004 10:21 PM

Subject: [ntdev] PFN Database, Original PTE field

The field Original PTE in each PFN Database entry is documented as being loaded with the original PTE contents.

However, it seems to contain only ‘invalid’ page states. I mean this field never contains a valid page state.

It preserves an invalid state all the time even if the corresponding PTE is valid currently.

According to Inside Microsoft Windows, page 475, ‘saving the contents of the PTE allows it to be restored when the physical page is no longer resident’. However, I cannot see it saves anything other than ‘invalid states’.

Which is the exact purpose of this field in the PFN database entries ? When is it used ?

Thanks.

Inaki.

—
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com</mailto:xxxxx></mailto:xxxxx>

OSR_Community_User · September 24, 2004, 12:40pm

Thanks for your answer. But I have some doubts:

I agree that if the page is invalid the PFN entry should contain a value according to this.
Although PFNs have different structures depending on the ‘state’, the fields PTE and Original PTE are always at the same offsets.
Suppose I have a known virtual address that is valid and is present and belongs to the current working set.
If you look at the PTE for such address you will find it is valid, it has some attributes, and it has a physical address that you can read.

Now you get the physical address from that PTE and from that value you calculate the PFB Database entry. Just multiply the page frame number on the PTE by 24 and add the PFN Database pointer. You will get an address on the PFN database. And yes the ‘PTE address’ field points back to a virtual address that contains the PTE descriptor, which is right and points to the same page.
However the field ‘Original PTE’ does not contain the PTE value nor anything alike. It contains a typical ‘invalid page’ value. How is that ? If the page is present and valid and being used by a process in that precise moment ?

I mean, if a page is valid, present and being used by the current process at a certain time, the corresponding field ‘OriginalPTE’ in the PFN entry should not be of the type MMPTE_HARDWARE?

While a PTE is valid and present, should not the PFN entry at a certain moment contain the same value that the PTE that corresponds to it ?

On the other hand, the entry ‘PFN of PTE’ (DWORD 5 in the PFN struct) that is supposed to contain the page frame numbger value for a given PTE what does it contain ? I cannot figure out what is it.

Thanks.
Inaki.

OSR_Community_User · September 24, 2004, 1:13pm

I can’t see any reason the PFN entry would contain the same PTE as the page table PTE - in that instance, the OS can just use the page table PTE. I’d have to verify, but instead, I would expect this entry to be used (for instance) when the page becomes transitional, so the OS can quickly restore the correct PTE entry. Similarly, if this were shared memory (such as a file backed page) I’m assuming this is going to point back to the proto pte. But these are very fine levels of detail for Mm. So, I guess the underlying question is: are you asking an academic question (“why does it work this way”) or are you trying to rely upon this for some implementation. If the latter, it will take more work to figure out the various states for this field and the use made of it.

Regards,

Tony

Tony Mason

Consulting Partner

OSR Open Systems Resources, Inc.

http://www.osr.com

Looking forward to seeing you at the Next OSR File Systems Class October 18, 2004 in Silicon Valley!

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of I?aki Castillo
Sent: Friday, September 24, 2004 12:37 PM
To: ntdev redirect
Subject: RE: [ntdev] PFN Database, Original PTE field

Thanks for your answer. But I have some doubts:

I agree that if the page is invalid the PFN entry should contain a value according to this.
Although PFNs have different structures depending on the ‘state’, the fields PTE and Original PTE are always at the same offsets.
Suppose I have a known virtual address that is valid and is present and belongs to the current working set.

If you look at the PTE for such address you will find it is valid, it has some attributes, and it has a physical address that you can read.

Now you get the physical address from that PTE and from that value you calculate the PFB Database entry. Just multiply the page frame number on the PTE by 24 and add the PFN Database pointer. You will get an address on the PFN database. And yes the ‘PTE address’ field points back to a virtual address that contains the PTE descriptor, which is right and points to the same page.

However the field ‘Original PTE’ does not contain the PTE value nor anything alike. It contains a typical ‘invalid page’ value. How is that ? If the page is present and valid and being used by a process in that precise moment ?

I mean, if a page is valid, present and being used by the current process at a certain time, the corresponding field ‘OriginalPTE’ in the PFN entry should not be of the type MMPTE_HARDWARE?

While a PTE is valid and present, should not the PFN entry at a certain moment contain the same value that the PTE that corresponds to it ?

On the other hand, the entry ‘PFN of PTE’ (DWORD 5 in the PFN struct) that is supposed to contain the page frame numbger value for a given PTE what does it contain ? I cannot figure out what is it.

Thanks.

Inaki.

Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

Maxim_S_Shatskih · September 25, 2004, 4:28am

>Note that the OriginalPTE field has seven different formats.

From what I remember just now:

this field is used to keep the pagefile address of the currently valid (or
transition) page, so that MM will know how to trim this page away. Trimming
will be - copy Pfn->OriginalPte to the real PTE in the page tables, and then
free the physical page (with the PFN). The next page fault will use the PTE’s
value as a pagefile address to bring the page in.

The rest of the details are in the page fault handling code which I have
reverse engineered around 1998 (from NT4). I can email it to those who want.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

Maxim_S_Shatskih · September 25, 2004, 4:34am

>expect this entry to be used (for instance) when the page becomes
transitional, so the OS can

quickly restore the correct PTE entry.

The transition PTE still keeps the physical address (though with Valid bit
zero), so the OS can locate the necessary PFN easily.

Restoration is something like - addref the PFN’s ShareCount, then set the Valid
bit in the PTE.

Similarly, if this were shared memory (such as a file backed page) I’m
assuming this is going
to point back to the proto pte.

Yes, the PFN also contains a PTE pointer which, in case of the shared page,
points to a prototype PTE. This is a very key field for Mm.

As about PFN’s OriginalPTE, which is not a pointer but a copy of the value -
then I can imagine no scenarios except outswapping a pagefile-backed page where
this field can be useful.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

Prokash_Sinha-1 · September 25, 2004, 1:44pm

It might be a good idea to make it an article, and put it in NT Insider. A
pseudo code level discussion with relevant invokation,context to capture the
main idea would be nice. Detail might be slightly different from Os version
to versions, but still, perhaps, the main idea could be captured to know the
workings !

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Maxim S. Shatskih
Sent: Saturday, September 25, 2004 1:29 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] PFN Database, Original PTE field

Note that the OriginalPTE field has seven different formats.

From what I remember just now:

this field is used to keep the pagefile address of the currently valid (or
transition) page, so that MM will know how to trim this page away. Trimming
will be - copy Pfn->OriginalPte to the real PTE in the page tables, and then
free the physical page (with the PFN). The next page fault will use the
PTE’s
value as a pagefile address to bring the page in.

The rest of the details are in the page fault handling code which I have
reverse engineered around 1998 (from NT4). I can email it to those who want.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com