Are page tables paged out?

OSR_Community_User · April 1, 2000, 5:25pm

Hello, Developers!

I have a question that is not directly related to driver development,
but to internal workings on NT memory manager.

Nagar Rejeev in his book (First Edition) states several times
that Page Frame Database is allocated from non-paged pool [1].
(Page 201, last paragraph and page 211, last paragraph)

But on page 212, last paragraph he says
“To avoid consuming this significant amount of memory
for translation information, page tables are also paged in
and out of memory”.[2]

This leaves me with 2 questions:

If [1] is correct how non-paged memory
can be paged out? Does memory manager have
some internal mechnism to page out non-paged memory?
If [2] is correct and some process accesses page that was swapped
to disk, will I/O requests be generated not only to bring accessed page
to RAM, but also to bring other pages to memory (pages containing
page table)?

Regards, Dennis

Do You Yahoo!?
Talk to your friends online with Yahoo! Messenger.
http://im.yahoo.com

OSR_Community_User · April 1, 2000, 5:31pm

Dennis,

You appear to be a little bit confused.

The Page Fram Database has NOTHING to do with Page Tables. The PFN
database is a list of Physical Memory Pages. Therefore, this is
not a case of non-paged memory being paged out. I assume that the
PFN database can’t be paged (although I suppose theoretically you
could do a desing that allowed it even though that might not make sense).

Process Page Table can indeed be paged out if need be. The process
of translating a memory reference can generate a page fault to fault
in the needed page or even the needed page for a page table entry to
continue the translation process. Obviously this means that a page
reference could either fault in the desired pages and/or the page
table entries needed to find the page to fault in. Swapping has
nothing really to do with this (directly).

Rick Cadruvi…

From: SMTP%“xxxxx@lists.osr.com” 1-APR-2000 14:20:31.83
To: “NT Developers Interest List”
CC:
Subj: [ntdev] Are page tables paged out?

Hello, Developers!

I have a question that is not directly related to driver development,
but to internal workings on NT memory manager.

Nagar Rejeev in his book (First Edition) states several times
that Page Frame Database is allocated from non-paged pool [1].
(Page 201, last paragraph and page 211, last paragraph)

But on page 212, last paragraph he says
“To avoid consuming this significant amount of memory
for translation information, page tables are also paged in=20
and out of memory”.[2]

This leaves me with 2 questions:
1. If [1] is correct how non-paged memory=20
can be paged out? Does memory manager have
some internal mechnism to page out non-paged memory?

2. If [2] is correct and some process accesses page that was swapped
to disk, will I/O requests be generated not only to bring accessed page
to RAM, but also to bring other pages to memory (pages containing
page table)?

Regards, Dennis

__________________________________________________

Do You Yahoo!?

Talk to your friends online with Yahoo! Messenger.

http://im.yahoo.com

—
You are currently subscribed to ntdev as: xxxxx@rdperf.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · April 1, 2000, 5:47pm

Rick,
Thanks on your explanation. I was really confused by
page table and page frame database. I have to admit that
Nagar’s descrition on this topic is not completelly clear.

I now have another question:
Does it makes any sence to page out page table and
causing multiple I/O requests for serving single page fault.
It seem not to be very cheap.

Regars.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of xxxxx@rdperf.com
Sent: Sunday, April 02, 2000 12:22 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Are page tables paged out?

Dennis,

You appear to be a little bit confused.

The Page Fram Database has NOTHING to do with Page Tables. The PFN
database is a list of Physical Memory Pages. Therefore, this is
not a case of non-paged memory being paged out. I assume that the
PFN database can’t be paged (although I suppose theoretically you
could do a desing that allowed it even though that might not make sense).

Process Page Table can indeed be paged out if need be. The process
of translating a memory reference can generate a page fault to fault
in the needed page or even the needed page for a page table entry to
continue the translation process. Obviously this means that a page
reference could either fault in the desired pages and/or the page
table entries needed to find the page to fault in. Swapping has
nothing really to do with this (directly).

Rick Cadruvi…

From: SMTP%“xxxxx@lists.osr.com” 1-APR-2000 14:20:31.83
To: “NT Developers Interest List”
> CC:
> Subj: [ntdev] Are page tables paged out?
>
> Hello, Developers!
>
> I have a question that is not directly related to driver development,
> but to internal workings on NT memory manager.
>
> Nagar Rejeev in his book (First Edition) states several times
> that Page Frame Database is allocated from non-paged pool [1].
> (Page 201, last paragraph and page 211, last paragraph)
>
> But on page 212, last paragraph he says
> “To avoid consuming this significant amount of memory
> for translation information, page tables are also paged in=20
> and out of memory”.[2]
>
> This leaves me with 2 questions:
> 1. If [1] is correct how non-paged memory=20
> can be paged out? Does memory manager have
> some internal mechnism to page out non-paged memory?
>
> 2. If [2] is correct and some process accesses page that was swapped
> to disk, will I/O requests be generated not only to bring accessed page
> to RAM, but also to bring other pages to memory (pages containing
> page table)?
>
> Regards, Dennis
>
>
>
>
>
>
> Do You Yahoo!?
>
> Talk to your friends online with Yahoo! Messenger.
>
> http://im.yahoo.com
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@rdperf.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@yahoo.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>

Do You Yahoo!?
Talk to your friends online with Yahoo! Messenger.
http://im.yahoo.com

OSR_Community_User · April 2, 2000, 4:40am

>Does it makes any sence to page out page table and

causing multiple I/O requests for serving single page fault.
It seem not to be very cheap.

The page table page can be paged out only if all PTEs in it are not present.
So - such events are rather rare.

Max

OSR_Community_User · April 2, 2000, 5:16pm

Really? So in other words, a page in the page table can only be paged out
if all of the pages that it describes have been paged out themselves? i.e.
the page tables are the last thing to get paged out?

At 12:37 PM 4/2/00 +0400, you wrote:

>Does it makes any sence to page out page table and
>causing multiple I/O requests for serving single page fault.
>It seem not to be very cheap.

The page table page can be paged out only if all PTEs in it are not present.
So - such events are rather rare.

Max

You are currently subscribed to ntdev as: xxxxx@iag.net
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · April 3, 2000, 6:07am

> Really? So in other words, a page in the page table can only be paged out

if all of the pages that it describes have been paged out themselves?
i.e.
the page tables are the last thing to get paged out?

Certainly. And can you imagine any other ways of pageable page tables?
When a PTE is made valid, the reference count for the page table containing
the PTE is incremented. And vice versa. Page can be swapped out only if
refcount == 0.

Max

OSR_Community_User · April 3, 2000, 10:06am

All right, color me stupid, but why is it exactly that a page cannot be
paged out if its reference count is not zero? Wouldn’t the reference count
be the number of valid references to this particular virtual->physical
mapping? Wouldn’t a zero count indicate that the particular mapping, if it
is pageable, should be discarded? I thought, and perhaps I am very mistaken
here, that pageable virtual->physical mappings can always be paged out as
long as they are not explicitly ‘pinned down’, as in MmProbeAndLockPages.

-----Original Message-----
From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
Sent: Monday, April 03, 2000 3:42 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Are page tables paged out?

> Really? So in other words, a page in the page table can
only be paged out
> if all of the pages that it describes have been paged out
themselves?
i.e.
> the page tables are the last thing to get paged out?

Certainly. And can you imagine any other ways of pageable page tables?
When a PTE is made valid, the reference count for the page
table containing
the PTE is incremented. And vice versa. Page can be swapped
out only if
refcount == 0.

Max

You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · April 3, 2000, 10:06am

Pages are paged out on an LRU basis. If in fact the page table has not been
used recently, then neither have all of its associated pages (well this is
almost true.) Paging out the page table indeed does make sense in this case.

While the initial translation (in a series of same-page address operations)
is expensive (up to two page faults, always two translations,) the
translation lookaside buffer hardware makes the subsequent translations
optimal, at least for a while.

Given the size of physical memory there really isn’t any other practical way
to organize virtual memory other than an n-tier directory approach. If you
don’t want to clog up non-paged pool with unused intermediate translation
directories, then you need also to make your page tables pageable.

-----Original Message-----
From: Dennis [mailto:xxxxx@yahoo.com]
Sent: Saturday, April 01, 2000 5:48 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Are page tables paged out?

Rick,
Thanks on your explanation. I was really confused by
page table and page frame database. I have to admit that
Nagar’s descrition on this topic is not completelly clear.

I now have another question:
Does it makes any sence to page out page table and
causing multiple I/O requests for serving single page fault.
It seem not to be very cheap.

Regars.

> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com]On Behalf Of xxxxx@rdperf.com
> Sent: Sunday, April 02, 2000 12:22 AM
> To: NT Developers Interest List
> Subject: [ntdev] Re: Are page tables paged out?
>
>
> Dennis,
>
> You appear to be a little bit confused.
>
> The Page Fram Database has NOTHING to do with Page Tables. The PFN
> database is a list of Physical Memory Pages. Therefore, this is
> not a case of non-paged memory being paged out. I assume that the
> PFN database can’t be paged (although I suppose theoretically you
> could do a desing that allowed it even though that might
not make sense).
>
> Process Page Table can indeed be paged out if need be. The process
> of translating a memory reference can generate a page fault to fault
> in the needed page or even the needed page for a page table entry to
> continue the translation process. Obviously this means that a page
> reference could either fault in the desired pages and/or the page
> table entries needed to find the page to fault in. Swapping has
> nothing really to do with this (directly).
>
>
> Rick Cadruvi…
>
>
> From: SMTP%“xxxxx@lists.osr.com” 1-APR-2000 14:20:31.83
> To: “NT Developers Interest List”
> > CC:
> > Subj: [ntdev] Are page tables paged out?
> >
> > Hello, Developers!
> >
> > I have a question that is not directly related to driver
> development,
> > but to internal workings on NT memory manager.
> >
> > Nagar Rejeev in his book (First Edition) states several times
> > that Page Frame Database is allocated from non-paged pool [1].
> > (Page 201, last paragraph and page 211, last paragraph)
> >
> > But on page 212, last paragraph he says
> > “To avoid consuming this significant amount of memory
> > for translation information, page tables are also paged in=20
> > and out of memory”.[2]
> >
> > This leaves me with 2 questions:
> > 1. If [1] is correct how non-paged memory=20
> > can be paged out? Does memory manager have
> > some internal mechnism to page out non-paged memory?
> >
> > 2. If [2] is correct and some process accesses page that was swapped
> > to disk, will I/O requests be generated not only to bring
> accessed page
> > to RAM, but also to bring other pages to memory (pages containing
> > page table)?
> >
> > Regards, Dennis
> >
> >
> >
> >
> >
> >
> > Do You Yahoo!?
> >
> > Talk to your friends online with Yahoo! Messenger.
> >
> > http://im.yahoo.com
> >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@rdperf.com
> > To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> >
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@yahoo.com
> > To unsubscribe send a blank email to $subst(‘Email.Unsub’)
> >
>
>
>
>
>
> Do You Yahoo!?
>
> Talk to your friends online with Yahoo! Messenger.
>
> http://im.yahoo.com
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@stratus.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>

OSR_Community_User · April 3, 2000, 11:36am

Please forgive any errors in this description. It has been a long
time since I studied this and I am doing this off the top of my head.

Page table entries are hardware dependent. In fact, it would be possible
(I think MIPS is this way) to NOT have a hardware PTE. The hardware
implementations that I have looked at tdo not have “reference counts”
in the PTEs. They would effectively be meaningless. They would have a “valid”
bit as well as some kind of “dirty” bit to indicate that there was a recent
write to the page so that page cleanup can make sure that section files
get written out to the disk before pages are removed.

The reference count referred to is probably the one in the Page Frame
Database. This reference count is used to “lock” down pages. The PFN
database is an OS construct and has NOTHING to do with the hardware
except that it helps the OS track physical memory pages. By OS definition,
a page with a non-zero reference count in it’s PFN database entry is NOT
pageable. This is essentially how MmProbeAndLockPages locks pages.
It’s a different mechanism, but similar that is used to lock pages within
a working set. A process has a working set list that works in a similar
manner to the PFN database to provide for working set trimming.

I would imagine that what Max was saying is that when a page mapped by a
PTE is paged in, the reference count for the page containing the PTE gets
incremented to prevent it’s being paged. I am not certain that this happens,
but it certainly makes sense. In theory, however, there would be no reason
why an OS couldn’t choose to page PTE pages without paging the pages that
they map. It wouldn’t make sense to do this since a page fault would
need to be rendered the next time a reference happened. Therefore, it makes
sense to only page out a PTE page when all the pages it maps are paged out.
I suppose that if you had some pages mapped by a PTE that were locked
(probably for a DMA transfer), you might decide to page the PTE page
since ALL the pages it maps cannot be paged. I wouldn’t even be surprised
if that happens in the case of a severe memory resource crisis. It certainly
would be okay in theory to do.

just my opinion based aon a potentially faulty recall mechanism. The old
brain is not quite what it used to be.

Rick Cadruvi…

All right, color me stupid, but why is it exactly that a page cannot be
paged out if its reference count is not zero? Wouldn’t the reference count
be the number of valid references to this particular virtual->physical
mapping? Wouldn’t a zero count indicate that the particular mapping, if it
is pageable, should be discarded? I thought, and perhaps I am very mistaken
here, that pageable virtual->physical mappings can always be paged out as
long as they are not explicitly ‘pinned down’, as in MmProbeAndLockPages.

> -----Original Message-----
> From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
> Sent: Monday, April 03, 2000 3:42 AM
> To: NT Developers Interest List
> Subject: [ntdev] Re: Are page tables paged out?
>
>
> > Really? So in other words, a page in the page table can
> only be paged out
> > if all of the pages that it describes have been paged out
> themselves?
> i.e.
> > the page tables are the last thing to get paged out?
>
> Certainly. And can you imagine any other ways of pageable page tables?
> When a PTE is made valid, the reference count for the page
> table containing
> the PTE is incremented. And vice versa. Page can be swapped
> out only if
> refcount == 0.
>
> Max
>
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@stratus.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>

You are currently subscribed to ntdev as: xxxxx@rdperf.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · April 3, 2000, 2:03pm

Sure I agree, but I’m still having trouble with Max saying:

"When a PTE is made valid, the reference count for the page table containing
the PTE is incremented. And vice versa. "

The reference on the Page Table might or might not be OS MM policy, but he
is saying here, unless I’m misreading, that the reference count on the PTE
is also incremented, and that would make all PTE’s effectively non-pageable.

-----Original Message-----
From: xxxxx@rdperf.com [mailto:xxxxx@rdperf.com]
Sent: Monday, April 03, 2000 11:13 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Are page tables paged out?

Please forgive any errors in this description. It has been a long
time since I studied this and I am doing this off the top of my head.

Page table entries are hardware dependent. In fact, it would
be possible
(I think MIPS is this way) to NOT have a hardware PTE. The hardware
implementations that I have looked at tdo not have “reference counts”
in the PTEs. They would effectively be meaningless. They
would have a “valid”
bit as well as some kind of “dirty” bit to indicate that
there was a recent
write to the page so that page cleanup can make sure that
section files
get written out to the disk before pages are removed.

The reference count referred to is probably the one in the Page Frame
Database. This reference count is used to “lock” down pages. The PFN
database is an OS construct and has NOTHING to do with the hardware
except that it helps the OS track physical memory pages. By
OS definition,
a page with a non-zero reference count in it’s PFN database
entry is NOT
pageable. This is essentially how MmProbeAndLockPages locks pages.
It’s a different mechanism, but similar that is used to lock
pages within
a working set. A process has a working set list that works
in a similar
manner to the PFN database to provide for working set trimming.

I would imagine that what Max was saying is that when a page
mapped by a
PTE is paged in, the reference count for the page containing
the PTE gets
incremented to prevent it’s being paged. I am not certain
that this happens,
but it certainly makes sense. In theory, however, there
would be no reason
why an OS couldn’t choose to page PTE pages without paging
the pages that
they map. It wouldn’t make sense to do this since a page fault would
need to be rendered the next time a reference happened.
Therefore, it makes
sense to only page out a PTE page when all the pages it maps
are paged out.
I suppose that if you had some pages mapped by a PTE that were locked
(probably for a DMA transfer), you might decide to page the PTE page
since ALL the pages it maps cannot be paged. I wouldn’t even
be surprised
if that happens in the case of a severe memory resource
crisis. It certainly
would be okay in theory to do.

just my opinion based aon a potentially faulty recall
mechanism. The old
brain is not quite what it used to be.

Rick Cadruvi…

>All right, color me stupid, but why is it exactly that a
page cannot be
>paged out if its reference count is not zero? Wouldn’t the
reference count
>be the number of valid references to this particular
virtual->physical
>mapping? Wouldn’t a zero count indicate that the particular
mapping, if it
>is pageable, should be discarded? I thought, and perhaps I
am very mistaken
>here, that pageable virtual->physical mappings can always be
paged out as
>long as they are not explicitly ‘pinned down’, as in
MmProbeAndLockPages.
>
>> -----Original Message-----
>> From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
>> Sent: Monday, April 03, 2000 3:42 AM
>> To: NT Developers Interest List
>> Subject: [ntdev] Re: Are page tables paged out?
>>
>>
>> > Really? So in other words, a page in the page table can
>> only be paged out
>> > if all of the pages that it describes have been paged out
>> themselves?
>> i.e.
>> > the page tables are the last thing to get paged out?
>>
>> Certainly. And can you imagine any other ways of pageable
page tables?
>> When a PTE is made valid, the reference count for the page
>> table containing
>> the PTE is incremented. And vice versa. Page can be swapped
>> out only if
>> refcount == 0.
>>
>> Max
>>
>>
>>
>> —
>> You are currently subscribed to ntdev as: xxxxx@stratus.com
>> To unsubscribe send a blank email to
$subst(‘Email.Unsub’)
>>
>
>—
>You are currently subscribed to ntdev as: xxxxx@rdperf.com
>To unsubscribe send a blank email to $subst(‘Email.Unsub’)

You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · April 3, 2000, 4:22pm

I think Max refers to shared pages only. These pages have a count reference
because actually they are not actual pages but pointers to prototype entry
pages.
The pages are not discarded except if its reference (in the Page Frame
Database) counter is zero.
However this has nothing to do with normal, non shared pages.

Inaki.

-----Original Message-----
From: Roddy, Mark
Sent: lunes 3 de abril de 2000 20:02
To: NT Developers Interest List
Subject: [ntdev] Re: Are page tables paged out?

Sure I agree, but I’m still having trouble with Max saying:

"When a PTE is made valid, the reference count for the page table
containing
the PTE is incremented. And vice versa. "

The reference on the Page Table might or might not be OS MM policy, but he
is saying here, unless I’m misreading, that the reference count on the PTE
is also incremented, and that would make all PTE’s effectively
non-pageable.

> -----Original Message-----
> From: xxxxx@rdperf.com [mailto:xxxxx@rdperf.com]
> Sent: Monday, April 03, 2000 11:13 AM
> To: NT Developers Interest List
> Subject: [ntdev] Re: Are page tables paged out?
>
>
> Please forgive any errors in this description. It has been a long
> time since I studied this and I am doing this off the top of my head.
>
> Page table entries are hardware dependent. In fact, it would
> be possible
> (I think MIPS is this way) to NOT have a hardware PTE. The hardware
> implementations that I have looked at tdo not have “reference counts”
> in the PTEs. They would effectively be meaningless. They
> would have a “valid”
> bit as well as some kind of “dirty” bit to indicate that
> there was a recent
> write to the page so that page cleanup can make sure that
> section files
> get written out to the disk before pages are removed.
>
> The reference count referred to is probably the one in the Page Frame
> Database. This reference count is used to “lock” down pages. The PFN
> database is an OS construct and has NOTHING to do with the hardware
> except that it helps the OS track physical memory pages. By
> OS definition,
> a page with a non-zero reference count in it’s PFN database
> entry is NOT
> pageable. This is essentially how MmProbeAndLockPages locks pages.
> It’s a different mechanism, but similar that is used to lock
> pages within
> a working set. A process has a working set list that works
> in a similar
> manner to the PFN database to provide for working set trimming.
>
> I would imagine that what Max was saying is that when a page
> mapped by a
> PTE is paged in, the reference count for the page containing
> the PTE gets
> incremented to prevent it’s being paged. I am not certain
> that this happens,
> but it certainly makes sense. In theory, however, there
> would be no reason
> why an OS couldn’t choose to page PTE pages without paging
> the pages that
> they map. It wouldn’t make sense to do this since a page fault would
> need to be rendered the next time a reference happened.
> Therefore, it makes
> sense to only page out a PTE page when all the pages it maps
> are paged out.
> I suppose that if you had some pages mapped by a PTE that were locked
> (probably for a DMA transfer), you might decide to page the PTE page
> since ALL the pages it maps cannot be paged. I wouldn’t even
> be surprised
> if that happens in the case of a severe memory resource
> crisis. It certainly
> would be okay in theory to do.
>
>
> just my opinion based aon a potentially faulty recall
> mechanism. The old
> brain is not quite what it used to be.
>
>
> Rick Cadruvi…
>
>
> >All right, color me stupid, but why is it exactly that a
> page cannot be
> >paged out if its reference count is not zero? Wouldn’t the
> reference count
> >be the number of valid references to this particular
> virtual->physical
> >mapping? Wouldn’t a zero count indicate that the particular
> mapping, if it
> >is pageable, should be discarded? I thought, and perhaps I
> am very mistaken
> >here, that pageable virtual->physical mappings can always be
> paged out as
> >long as they are not explicitly ‘pinned down’, as in
> MmProbeAndLockPages.
> >
> >> -----Original Message-----
> >> From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
> >> Sent: Monday, April 03, 2000 3:42 AM
> >> To: NT Developers Interest List
> >> Subject: [ntdev] Re: Are page tables paged out?
> >>
> >>
> >> > Really? So in other words, a page in the page table can
> >> only be paged out
> >> > if all of the pages that it describes have been paged out
> >> themselves?
> >> i.e.
> >> > the page tables are the last thing to get paged out?
> >>
> >> Certainly. And can you imagine any other ways of pageable
> page tables?
> >> When a PTE is made valid, the reference count for the page
> >> table containing
> >> the PTE is incremented. And vice versa. Page can be swapped
> >> out only if
> >> refcount == 0.
> >>
> >> Max
> >>
> >>
> >>
> >> —
> >> You are currently subscribed to ntdev as: xxxxx@stratus.com
> >> To unsubscribe send a blank email to
> $subst(‘Email.Unsub’)
> >>
> >
> >—
> >You are currently subscribed to ntdev as: xxxxx@rdperf.com
> >To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
> —
> You are currently subscribed to ntdev as: xxxxx@stratus.com
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>

You are currently subscribed to ntdev as: xxxxx@pandasoftware.es
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · April 3, 2000, 4:55pm

Hey I know, maybe we could all spend the next few days inventing
explanations that work or don’t work for ‘what max refers to’

I agree that shared pages are reference counted.

The point that interested me was the issue of does a page table ever page
out while there are paged-in PTE’s referenced from that page table. I’m
still not convinced that the page table is referenced counted by each paged
in PTE. However I am convinced that PTEs are not referenced counted by each
referring Page Table, the ‘vice versa’ portion of ‘what max referred to’.

More to the point, this thread at some point asserted that Page Tables would
rarely if ever page out, and I just don’t think that that is true.

-----Original Message-----
From: I?aki Castillo [mailto:xxxxx@pandasoftware.es]
Sent: Monday, April 03, 2000 2:18 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Are page tables paged out?

I think Max refers to shared pages only. These pages have a
count reference
because actually they are not actual pages but pointers to
prototype entry
pages.
The pages are not discarded except if its reference (in the Page Frame
Database) counter is zero.
However this has nothing to do with normal, non shared pages.

Inaki.

> -----Original Message-----
> From: Roddy, Mark
> Sent: lunes 3 de abril de 2000 20:02
> To: NT Developers Interest List
> Subject: [ntdev] Re: Are page tables paged out?
>
> Sure I agree, but I’m still having trouble with Max saying:
>
> "When a PTE is made valid, the reference count for the page table
> containing
> the PTE is incremented. And vice versa. "
>
> The reference on the Page Table might or might not be OS MM
policy, but he
> is saying here, unless I’m misreading, that the reference
count on the PTE
> is also incremented, and that would make all PTE’s effectively
> non-pageable.
>
>
> > -----Original Message-----
> > From: xxxxx@rdperf.com [mailto:xxxxx@rdperf.com]
> > Sent: Monday, April 03, 2000 11:13 AM
> > To: NT Developers Interest List
> > Subject: [ntdev] Re: Are page tables paged out?
> >
> >
> > Please forgive any errors in this description. It has been a long
> > time since I studied this and I am doing this off the top
of my head.
> >
> > Page table entries are hardware dependent. In fact, it would
> > be possible
> > (I think MIPS is this way) to NOT have a hardware PTE.
The hardware
> > implementations that I have looked at tdo not have
“reference counts”
> > in the PTEs. They would effectively be meaningless. They
> > would have a “valid”
> > bit as well as some kind of “dirty” bit to indicate that
> > there was a recent
> > write to the page so that page cleanup can make sure that
> > section files
> > get written out to the disk before pages are removed.
> >
> > The reference count referred to is probably the one in
the Page Frame
> > Database. This reference count is used to “lock” down
pages. The PFN
> > database is an OS construct and has NOTHING to do with
the hardware
> > except that it helps the OS track physical memory pages. By
> > OS definition,
> > a page with a non-zero reference count in it’s PFN database
> > entry is NOT
> > pageable. This is essentially how MmProbeAndLockPages
locks pages.
> > It’s a different mechanism, but similar that is used to lock
> > pages within
> > a working set. A process has a working set list that works
> > in a similar
> > manner to the PFN database to provide for working set trimming.
> >
> > I would imagine that what Max was saying is that when a page
> > mapped by a
> > PTE is paged in, the reference count for the page containing
> > the PTE gets
> > incremented to prevent it’s being paged. I am not certain
> > that this happens,
> > but it certainly makes sense. In theory, however, there
> > would be no reason
> > why an OS couldn’t choose to page PTE pages without paging
> > the pages that
> > they map. It wouldn’t make sense to do this since a page
fault would
> > need to be rendered the next time a reference happened.
> > Therefore, it makes
> > sense to only page out a PTE page when all the pages it maps
> > are paged out.
> > I suppose that if you had some pages mapped by a PTE that
were locked
> > (probably for a DMA transfer), you might decide to page
the PTE page
> > since ALL the pages it maps cannot be paged. I wouldn’t even
> > be surprised
> > if that happens in the case of a severe memory resource
> > crisis. It certainly
> > would be okay in theory to do.
> >
> >
> > just my opinion based aon a potentially faulty recall
> > mechanism. The old
> > brain is not quite what it used to be.
> >
> >
> > Rick Cadruvi…
> >
> >
> > >All right, color me stupid, but why is it exactly that a
> > page cannot be
> > >paged out if its reference count is not zero? Wouldn’t the
> > reference count
> > >be the number of valid references to this particular
> > virtual->physical
> > >mapping? Wouldn’t a zero count indicate that the particular
> > mapping, if it
> > >is pageable, should be discarded? I thought, and perhaps I
> > am very mistaken
> > >here, that pageable virtual->physical mappings can always be
> > paged out as
> > >long as they are not explicitly ‘pinned down’, as in
> > MmProbeAndLockPages.
> > >
> > >> -----Original Message-----
> > >> From: Maxim S. Shatskih [mailto:xxxxx@storagecraft.com]
> > >> Sent: Monday, April 03, 2000 3:42 AM
> > >> To: NT Developers Interest List
> > >> Subject: [ntdev] Re: Are page tables paged out?
> > >>
> > >>
> > >> > Really? So in other words, a page in the page table can
> > >> only be paged out
> > >> > if all of the pages that it describes have been paged out
> > >> themselves?
> > >> i.e.
> > >> > the page tables are the last thing to get paged out?
> > >>
> > >> Certainly. And can you imagine any other ways of pageable
> > page tables?
> > >> When a PTE is made valid, the reference count for the page
> > >> table containing
> > >> the PTE is incremented. And vice versa. Page can be swapped
> > >> out only if
> > >> refcount == 0.
> > >>
> > >> Max
> > >>
> > >>
> > >>
> > >> —
> > >> You are currently subscribed to ntdev as:
xxxxx@stratus.com
> > >> To unsubscribe send a blank email to
> > $subst(‘Email.Unsub’)
> > >>
> > >
> > >—
> > >You are currently subscribed to ntdev as: xxxxx@rdperf.com
> > >To unsubscribe send a blank email to
$subst(‘Email.Unsub’)
> >
> > —
> > You are currently subscribed to ntdev as: xxxxx@stratus.com
> > To unsubscribe send a blank email to
$subst(‘Email.Unsub’)
> >
>
> —
> You are currently subscribed to ntdev as: xxxxx@pandasoftware.es
> To unsubscribe send a blank email to $subst(‘Email.Unsub’)

You are currently subscribed to ntdev as: xxxxx@stratus.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

OSR_Community_User · April 4, 2000, 5:22am

> long as they are not explicitly ‘pinned down’, as in MmProbeAndLockPages.

Yes. And they are pinned down by incrementing the reference count.
You can think of the reference count as of the “pin count”.
The number of mapping - the number of PTEs referencing the page - is not a
reference count, but a share count - a different thing.

Max

OSR_Community_User · April 4, 2000, 5:22am

> I would imagine that what Max was saying is that when a page mapped by a

PTE is paged in, the reference count for the page containing the PTE gets
incremented to prevent it’s being paged.

This happens - I have spent some time investigating the assembly code of
MmAccessFault and other routines - so, I can say that this is for sure.

Throwing the page away from the physical memory required update to its
PTE to place a swapfile address in it. So, the PTE - and the page table
page - must be resident at this moment. That’s why it’s impossible to page
out the page table while it has even transition (standby list) PTEs, not to
say
present PTEs.

Max

OSR_Community_User · April 4, 2000, 5:22am

> The reference on the Page Table might or might not be OS MM policy, but he

is saying here, unless I’m misreading, that the reference count on the PTE
is also incremented, and that would make all PTE’s effectively non-

No. PTEs has no reference counts - only physical pages (PFN entries) has
such.

Max

OSR_Community_User · April 4, 2000, 5:22am

> The pages are not discarded except if its reference (in the Page Frame

Database) counter is zero.
However this has nothing to do with normal, non shared pages.

PFN database keeps both ShareCount (for shared pages. This is number of
present PTEs referencing the page) and ReferenceCount (this is “pin count”
incremented in MmProbeAndLockPages).
So, even non-shared pages have a reference count.
The idea described by me is something like - “MmProbeAndLock is done
on the page table the number of times equal to the number of
present/transition PTEs in it”.

Max