Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.More Info on Driver Writing and Debugging
The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.
Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/
Are page tables paged out?
OSR_Community_User
Member Posts: 110,217
in NTDEV
Hello, Developers!
I have a question that is not directly related to driver development,
but to internal workings on NT memory manager.
Nagar Rejeev in his book (First Edition) states several times
that Page Frame Database is allocated from non-paged pool [1].
(Page 201, last paragraph and page 211, last paragraph)
But on page 212, last paragraph he says
"To avoid consuming this significant amount of memory
for translation information, page tables are also paged in
and out of memory".[2]
This leaves me with 2 questions:
1. If [1] is correct how non-paged memory
can be paged out? Does memory manager have
some internal mechnism to page out non-paged memory?
2. If [2] is correct and some process accesses page that was swapped
to disk, will I/O requests be generated not only to bring accessed page
to RAM, but also to bring other pages to memory (pages containing
page table)?
Regards, Dennis
__________________________________________________
Do You Yahoo!?
Talk to your friends online with Yahoo! Messenger.
http://im.yahoo.com
I have a question that is not directly related to driver development,
but to internal workings on NT memory manager.
Nagar Rejeev in his book (First Edition) states several times
that Page Frame Database is allocated from non-paged pool [1].
(Page 201, last paragraph and page 211, last paragraph)
But on page 212, last paragraph he says
"To avoid consuming this significant amount of memory
for translation information, page tables are also paged in
and out of memory".[2]
This leaves me with 2 questions:
1. If [1] is correct how non-paged memory
can be paged out? Does memory manager have
some internal mechnism to page out non-paged memory?
2. If [2] is correct and some process accesses page that was swapped
to disk, will I/O requests be generated not only to bring accessed page
to RAM, but also to bring other pages to memory (pages containing
page table)?
Regards, Dennis
__________________________________________________
Do You Yahoo!?
Talk to your friends online with Yahoo! Messenger.
http://im.yahoo.com
Howdy, Stranger!
| Upcoming OSR Seminars | ||
|---|---|---|
| OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead! | ||
| Internals & Software Drivers | 7 February 2022 | Live, Online |
| Kernel Debugging | 21 March 2022 | Live, Online |
| Developing Minifilters | 23 May 2022 | Live, Online |
| Writing WDF Drivers | 12 September 2022 | Live, Online |
Comments
You appear to be a little bit confused.
The Page Fram Database has NOTHING to do with Page Tables. The PFN
database is a list of Physical Memory Pages. Therefore, this is
not a case of non-paged memory being paged out. I assume that the
PFN database can't be paged (although I suppose theoretically you
could do a desing that allowed it even though that might not make sense).
Process Page Table can indeed be paged out if need be. The process
of translating a memory reference can generate a page fault to fault
in the needed page or even the needed page for a page table entry to
continue the translation process. Obviously this means that a page
reference could either fault in the desired pages and/or the page
table entries needed to find the page to fault in. Swapping has
nothing really to do with this (directly).
Rick Cadruvi...
From: SMTP%"[email protected]" 1-APR-2000 14:20:31.83
To: "NT Developers Interest List"
CC:
Subj: [ntdev] Are page tables paged out?
Hello, Developers!
I have a question that is not directly related to driver development,
but to internal workings on NT memory manager.
Nagar Rejeev in his book (First Edition) states several times
that Page Frame Database is allocated from non-paged pool [1].
(Page 201, last paragraph and page 211, last paragraph)
But on page 212, last paragraph he says
"To avoid consuming this significant amount of memory
for translation information, page tables are also paged in=20
and out of memory".[2]
This leaves me with 2 questions:
1. If [1] is correct how non-paged memory=20
can be paged out? Does memory manager have
some internal mechnism to page out non-paged memory?
2. If [2] is correct and some process accesses page that was swapped
to disk, will I/O requests be generated not only to bring accessed page
to RAM, but also to bring other pages to memory (pages containing
page table)?
Regards, Dennis
__________________________________________________
Do You Yahoo!?
Talk to your friends online with Yahoo! Messenger.
http://im.yahoo.com
---
You are currently subscribed to ntdev as: [email protected]
To unsubscribe send a blank email to $subst('Email.Unsub')
Thanks on your explanation. I was really confused by
page table and page frame database. I have to admit that
Nagar's descrition on this topic is not completelly clear.
I now have another question:
Does it makes any sence to page out page table and
causing multiple I/O requests for serving single page fault.
It seem not to be very cheap.
Regars.
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]]On Behalf Of [email protected]
> Sent: Sunday, April 02, 2000 12:22 AM
> To: NT Developers Interest List
> Subject: [ntdev] Re: Are page tables paged out?
>
>
> Dennis,
>
> You appear to be a little bit confused.
>
> The Page Fram Database has NOTHING to do with Page Tables. The PFN
> database is a list of Physical Memory Pages. Therefore, this is
> not a case of non-paged memory being paged out. I assume that the
> PFN database can't be paged (although I suppose theoretically you
> could do a desing that allowed it even though that might not make sense).
>
> Process Page Table can indeed be paged out if need be. The process
> of translating a memory reference can generate a page fault to fault
> in the needed page or even the needed page for a page table entry to
> continue the translation process. Obviously this means that a page
> reference could either fault in the desired pages and/or the page
> table entries needed to find the page to fault in. Swapping has
> nothing really to do with this (directly).
>
>
> Rick Cadruvi...
>
>
> From: SMTP%"[email protected]" 1-APR-2000 14:20:31.83
> To: "NT Developers Interest List" <[email protected]>
> CC:
> Subj: [ntdev] Are page tables paged out?
>
> Hello, Developers!
>
> I have a question that is not directly related to driver development,
> but to internal workings on NT memory manager.
>
> Nagar Rejeev in his book (First Edition) states several times
> that Page Frame Database is allocated from non-paged pool [1].
> (Page 201, last paragraph and page 211, last paragraph)
>
> But on page 212, last paragraph he says
> "To avoid consuming this significant amount of memory
> for translation information, page tables are also paged in=20
> and out of memory".[2]
>
> This leaves me with 2 questions:
> 1. If [1] is correct how non-paged memory=20
> can be paged out? Does memory manager have
> some internal mechnism to page out non-paged memory?
>
> 2. If [2] is correct and some process accesses page that was swapped
> to disk, will I/O requests be generated not only to bring accessed page
> to RAM, but also to bring other pages to memory (pages containing
> page table)?
>
> Regards, Dennis
>
>
>
>
> __________________________________________________
>
> Do You Yahoo!?
>
> Talk to your friends online with Yahoo! Messenger.
>
> http://im.yahoo.com
>
>
> ---
> You are currently subscribed to ntdev as: [email protected]
> To unsubscribe send a blank email to $subst('Email.Unsub')
>
>
> ---
> You are currently subscribed to ntdev as: [email protected]
> To unsubscribe send a blank email to $subst('Email.Unsub')
>
__________________________________________________
Do You Yahoo!?
Talk to your friends online with Yahoo! Messenger.
http://im.yahoo.com
>causing multiple I/O requests for serving single page fault.
>It seem not to be very cheap.
The page table page can be paged out only if all PTEs in it are not present.
So - such events are rather rare.
Max
if all of the pages that it describes have been paged out themselves? i.e.
the page tables are the last thing to get paged out?
At 12:37 PM 4/2/00 +0400, you wrote:
> >Does it makes any sence to page out page table and
> >causing multiple I/O requests for serving single page fault.
> >It seem not to be very cheap.
>
>The page table page can be paged out only if all PTEs in it are not present.
>So - such events are rather rare.
>
> Max
>
>
>---
>You are currently subscribed to ntdev as: [email protected]
>To unsubscribe send a blank email to $subst('Email.Unsub')
> if all of the pages that it describes have been paged out themselves?
i.e.
> the page tables are the last thing to get paged out?
Certainly. And can you imagine any other ways of pageable page tables?
When a PTE is made valid, the reference count for the page table containing
the PTE is incremented. And vice versa. Page can be swapped out only if
refcount == 0.
Max
paged out if its reference count is not zero? Wouldn't the reference count
be the number of valid references to this particular virtual->physical
mapping? Wouldn't a zero count indicate that the particular mapping, if it
is pageable, should be discarded? I thought, and perhaps I am very mistaken
here, that pageable virtual->physical mappings can always be paged out as
long as they are not explicitly 'pinned down', as in MmProbeAndLockPages.
> -----Original Message-----
> From: Maxim S. Shatskih [mailto:[email protected]]
> Sent: Monday, April 03, 2000 3:42 AM
> To: NT Developers Interest List
> Subject: [ntdev] Re: Are page tables paged out?
>
>
> > Really? So in other words, a page in the page table can
> only be paged out
> > if all of the pages that it describes have been paged out
> themselves?
> i.e.
> > the page tables are the last thing to get paged out?
>
> Certainly. And can you imagine any other ways of pageable page tables?
> When a PTE is made valid, the reference count for the page
> table containing
> the PTE is incremented. And vice versa. Page can be swapped
> out only if
> refcount == 0.
>
> Max
>
>
>
> ---
> You are currently subscribed to ntdev as: [email protected]
> To unsubscribe send a blank email to $subst('Email.Unsub')
>
used recently, then neither have all of its associated pages (well this is
almost true.) Paging out the page table indeed does make sense in this case.
While the initial translation (in a series of same-page address operations)
is expensive (up to two page faults, always two translations,) the
translation lookaside buffer hardware makes the subsequent translations
optimal, at least for a while.
Given the size of physical memory there really isn't any other practical way
to organize virtual memory other than an n-tier directory approach. If you
don't want to clog up non-paged pool with unused intermediate translation
directories, then you need also to make your page tables pageable.
> -----Original Message-----
> From: Dennis [mailto:[email protected]]
> Sent: Saturday, April 01, 2000 5:48 PM
> To: NT Developers Interest List
> Subject: [ntdev] Re: Are page tables paged out?
>
>
> Rick,
> Thanks on your explanation. I was really confused by
> page table and page frame database. I have to admit that
> Nagar's descrition on this topic is not completelly clear.
>
> I now have another question:
> Does it makes any sence to page out page table and
> causing multiple I/O requests for serving single page fault.
> It seem not to be very cheap.
>
> Regars.
>
> > -----Original Message-----
> > From: [email protected]
> > [mailto:[email protected]]On Behalf Of [email protected]
> > Sent: Sunday, April 02, 2000 12:22 AM
> > To: NT Developers Interest List
> > Subject: [ntdev] Re: Are page tables paged out?
> >
> >
> > Dennis,
> >
> > You appear to be a little bit confused.
> >
> > The Page Fram Database has NOTHING to do with Page Tables. The PFN
> > database is a list of Physical Memory Pages. Therefore, this is
> > not a case of non-paged memory being paged out. I assume that the
> > PFN database can't be paged (although I suppose theoretically you
> > could do a desing that allowed it even though that might
> not make sense).
> >
> > Process Page Table can indeed be paged out if need be. The process
> > of translating a memory reference can generate a page fault to fault
> > in the needed page or even the needed page for a page table entry to
> > continue the translation process. Obviously this means that a page
> > reference could either fault in the desired pages and/or the page
> > table entries needed to find the page to fault in. Swapping has
> > nothing really to do with this (directly).
> >
> >
> > Rick Cadruvi...
> >
> >
> > From: SMTP%"[email protected]" 1-APR-2000 14:20:31.83
> > To: "NT Developers Interest List" <[email protected]>
> > CC:
> > Subj: [ntdev] Are page tables paged out?
> >
> > Hello, Developers!
> >
> > I have a question that is not directly related to driver
> development,
> > but to internal workings on NT memory manager.
> >
> > Nagar Rejeev in his book (First Edition) states several times
> > that Page Frame Database is allocated from non-paged pool [1].
> > (Page 201, last paragraph and page 211, last paragraph)
> >
> > But on page 212, last paragraph he says
> > "To avoid consuming this significant amount of memory
> > for translation information, page tables are also paged in=20
> > and out of memory".[2]
> >
> > This leaves me with 2 questions:
> > 1. If [1] is correct how non-paged memory=20
> > can be paged out? Does memory manager have
> > some internal mechnism to page out non-paged memory?
> >
> > 2. If [2] is correct and some process accesses page that was swapped
> > to disk, will I/O requests be generated not only to bring
> accessed page
> > to RAM, but also to bring other pages to memory (pages containing
> > page table)?
> >
> > Regards, Dennis
> >
> >
> >
> >
> > __________________________________________________
> >
> > Do You Yahoo!?
> >
> > Talk to your friends online with Yahoo! Messenger.
> >
> > http://im.yahoo.com
> >
> >
> > ---
> > You are currently subscribed to ntdev as: [email protected]
> > To unsubscribe send a blank email to $subst('Email.Unsub')
> >
> >
> > ---
> > You are currently subscribed to ntdev as: [email protected]
> > To unsubscribe send a blank email to $subst('Email.Unsub')
> >
>
>
>
> __________________________________________________
>
> Do You Yahoo!?
>
> Talk to your friends online with Yahoo! Messenger.
>
> http://im.yahoo.com
>
>
> ---
> You are currently subscribed to ntdev as: [email protected]
> To unsubscribe send a blank email to $subst('Email.Unsub')
>
time since I studied this and I am doing this off the top of my head.
Page table entries are hardware dependent. In fact, it would be possible
(I think MIPS is this way) to NOT have a hardware PTE. The hardware
implementations that I have looked at tdo not have "reference counts"
in the PTEs. They would effectively be meaningless. They would have a "valid"
bit as well as some kind of "dirty" bit to indicate that there was a recent
write to the page so that page cleanup can make sure that section files
get written out to the disk before pages are removed.
The reference count referred to is probably the one in the Page Frame
Database. This reference count is used to "lock" down pages. The PFN
database is an OS construct and has NOTHING to do with the hardware
except that it helps the OS track physical memory pages. By OS definition,
a page with a non-zero reference count in it's PFN database entry is NOT
pageable. This is essentially how MmProbeAndLockPages locks pages.
It's a different mechanism, but similar that is used to lock pages within
a working set. A process has a working set list that works in a similar
manner to the PFN database to provide for working set trimming.
I would imagine that what Max was saying is that when a page mapped by a
PTE is paged in, the reference count for the page containing the PTE gets
incremented to prevent it's being paged. I am not certain that this happens,
but it certainly makes sense. In theory, however, there would be no reason
why an OS couldn't choose to page PTE pages without paging the pages that
they map. It wouldn't make sense to do this since a page fault would
need to be rendered the next time a reference happened. Therefore, it makes
sense to only page out a PTE page when all the pages it maps are paged out.
I suppose that if you had some pages mapped by a PTE that were locked
(probably for a DMA transfer), you might decide to page the PTE page
since ALL the pages it maps cannot be paged. I wouldn't even be surprised
if that happens in the case of a severe memory resource crisis. It certainly
would be okay in theory to do.
just my opinion based aon a potentially faulty recall mechanism. The old
brain is not quite what it used to be.
Rick Cadruvi...
>All right, color me stupid, but why is it exactly that a page cannot be
>paged out if its reference count is not zero? Wouldn't the reference count
>be the number of valid references to this particular virtual->physical
>mapping? Wouldn't a zero count indicate that the particular mapping, if it
>is pageable, should be discarded? I thought, and perhaps I am very mistaken
>here, that pageable virtual->physical mappings can always be paged out as
>long as they are not explicitly 'pinned down', as in MmProbeAndLockPages.
>
>> -----Original Message-----
>> From: Maxim S. Shatskih [mailto:[email protected]]
>> Sent: Monday, April 03, 2000 3:42 AM
>> To: NT Developers Interest List
>> Subject: [ntdev] Re: Are page tables paged out?
>>
>>
>> > Really? So in other words, a page in the page table can
>> only be paged out
>> > if all of the pages that it describes have been paged out
>> themselves?
>> i.e.
>> > the page tables are the last thing to get paged out?
>>
>> Certainly. And can you imagine any other ways of pageable page tables?
>> When a PTE is made valid, the reference count for the page
>> table containing
>> the PTE is incremented. And vice versa. Page can be swapped
>> out only if
>> refcount == 0.
>>
>> Max
>>
>>
>>
>> ---
>> You are currently subscribed to ntdev as: [email protected]
>> To unsubscribe send a blank email to $subst('Email.Unsub')
>>
>
>---
>You are currently subscribed to ntdev as: [email protected]
>To unsubscribe send a blank email to $subst('Email.Unsub')
"When a PTE is made valid, the reference count for the page table containing
the PTE is incremented. And vice versa. "
The reference on the Page Table might or might not be OS MM policy, but he
is saying here, unless I'm misreading, that the reference count on the PTE
is also incremented, and that would make all PTE's effectively non-pageable.
> -----Original Message-----
> From: [email protected] [mailto:[email protected]]
> Sent: Monday, April 03, 2000 11:13 AM
> To: NT Developers Interest List
> Subject: [ntdev] Re: Are page tables paged out?
>
>
> Please forgive any errors in this description. It has been a long
> time since I studied this and I am doing this off the top of my head.
>
> Page table entries are hardware dependent. In fact, it would
> be possible
> (I think MIPS is this way) to NOT have a hardware PTE. The hardware
> implementations that I have looked at tdo not have "reference counts"
> in the PTEs. They would effectively be meaningless. They
> would have a "valid"
> bit as well as some kind of "dirty" bit to indicate that
> there was a recent
> write to the page so that page cleanup can make sure that
> section files
> get written out to the disk before pages are removed.
>
> The reference count referred to is probably the one in the Page Frame
> Database. This reference count is used to "lock" down pages. The PFN
> database is an OS construct and has NOTHING to do with the hardware
> except that it helps the OS track physical memory pages. By
> OS definition,
> a page with a non-zero reference count in it's PFN database
> entry is NOT
> pageable. This is essentially how MmProbeAndLockPages locks pages.
> It's a different mechanism, but similar that is used to lock
> pages within
> a working set. A process has a working set list that works
> in a similar
> manner to the PFN database to provide for working set trimming.
>
> I would imagine that what Max was saying is that when a page
> mapped by a
> PTE is paged in, the reference count for the page containing
> the PTE gets
> incremented to prevent it's being paged. I am not certain
> that this happens,
> but it certainly makes sense. In theory, however, there
> would be no reason
> why an OS couldn't choose to page PTE pages without paging
> the pages that
> they map. It wouldn't make sense to do this since a page fault would
> need to be rendered the next time a reference happened.
> Therefore, it makes
> sense to only page out a PTE page when all the pages it maps
> are paged out.
> I suppose that if you had some pages mapped by a PTE that were locked
> (probably for a DMA transfer), you might decide to page the PTE page
> since ALL the pages it maps cannot be paged. I wouldn't even
> be surprised
> if that happens in the case of a severe memory resource
> crisis. It certainly
> would be okay in theory to do.
>
>
> just my opinion based aon a potentially faulty recall
> mechanism. The old
> brain is not quite what it used to be.
>
>
> Rick Cadruvi...
>
>
> >All right, color me stupid, but why is it exactly that a
> page cannot be
> >paged out if its reference count is not zero? Wouldn't the
> reference count
> >be the number of valid references to this particular
> virtual->physical
> >mapping? Wouldn't a zero count indicate that the particular
> mapping, if it
> >is pageable, should be discarded? I thought, and perhaps I
> am very mistaken
> >here, that pageable virtual->physical mappings can always be
> paged out as
> >long as they are not explicitly 'pinned down', as in
> MmProbeAndLockPages.
> >
> >> -----Original Message-----
> >> From: Maxim S. Shatskih [mailto:[email protected]]
> >> Sent: Monday, April 03, 2000 3:42 AM
> >> To: NT Developers Interest List
> >> Subject: [ntdev] Re: Are page tables paged out?
> >>
> >>
> >> > Really? So in other words, a page in the page table can
> >> only be paged out
> >> > if all of the pages that it describes have been paged out
> >> themselves?
> >> i.e.
> >> > the page tables are the last thing to get paged out?
> >>
> >> Certainly. And can you imagine any other ways of pageable
> page tables?
> >> When a PTE is made valid, the reference count for the page
> >> table containing
> >> the PTE is incremented. And vice versa. Page can be swapped
> >> out only if
> >> refcount == 0.
> >>
> >> Max
> >>
> >>
> >>
> >> ---
> >> You are currently subscribed to ntdev as: [email protected]
> >> To unsubscribe send a blank email to
> $subst('Email.Unsub')
> >>
> >
> >---
> >You are currently subscribed to ntdev as: [email protected]com
> >To unsubscribe send a blank email to $subst('Email.Unsub')
>
> ---
> You are currently subscribed to ntdev as: [email protected]
> To unsubscribe send a blank email to $subst('Email.Unsub')
>
because actually they are not actual pages but pointers to prototype entry
pages.
The pages are not discarded except if its reference (in the Page Frame
Database) counter is zero.
However this has nothing to do with normal, non shared pages.
Inaki.
> -----Original Message-----
> From: Roddy, Mark
> Sent: lunes 3 de abril de 2000 20:02
> To: NT Developers Interest List
> Subject: [ntdev] Re: Are page tables paged out?
>
> Sure I agree, but I'm still having trouble with Max saying:
>
> "When a PTE is made valid, the reference count for the page table
> containing
> the PTE is incremented. And vice versa. "
>
> The reference on the Page Table might or might not be OS MM policy, but he
> is saying here, unless I'm misreading, that the reference count on the PTE
> is also incremented, and that would make all PTE's effectively
> non-pageable.
>
>
> > -----Original Message-----
> > From: [email protected] [mailto:[email protected]]
> > Sent: Monday, April 03, 2000 11:13 AM
> > To: NT Developers Interest List
> > Subject: [ntdev] Re: Are page tables paged out?
> >
> >
> > Please forgive any errors in this description. It has been a long
> > time since I studied this and I am doing this off the top of my head.
> >
> > Page table entries are hardware dependent. In fact, it would
> > be possible
> > (I think MIPS is this way) to NOT have a hardware PTE. The hardware
> > implementations that I have looked at tdo not have "reference counts"
> > in the PTEs. They would effectively be meaningless. They
> > would have a "valid"
> > bit as well as some kind of "dirty" bit to indicate that
> > there was a recent
> > write to the page so that page cleanup can make sure that
> > section files
> > get written out to the disk before pages are removed.
> >
> > The reference count referred to is probably the one in the Page Frame
> > Database. This reference count is used to "lock" down pages. The PFN
> > database is an OS construct and has NOTHING to do with the hardware
> > except that it helps the OS track physical memory pages. By
> > OS definition,
> > a page with a non-zero reference count in it's PFN database
> > entry is NOT
> > pageable. This is essentially how MmProbeAndLockPages locks pages.
> > It's a different mechanism, but similar that is used to lock
> > pages within
> > a working set. A process has a working set list that works
> > in a similar
> > manner to the PFN database to provide for working set trimming.
> >
> > I would imagine that what Max was saying is that when a page
> > mapped by a
> > PTE is paged in, the reference count for the page containing
> > the PTE gets
> > incremented to prevent it's being paged. I am not certain
> > that this happens,
> > but it certainly makes sense. In theory, however, there
> > would be no reason
> > why an OS couldn't choose to page PTE pages without paging
> > the pages that
> > they map. It wouldn't make sense to do this since a page fault would
> > need to be rendered the next time a reference happened.
> > Therefore, it makes
> > sense to only page out a PTE page when all the pages it maps
> > are paged out.
> > I suppose that if you had some pages mapped by a PTE that were locked
> > (probably for a DMA transfer), you might decide to page the PTE page
> > since ALL the pages it maps cannot be paged. I wouldn't even
> > be surprised
> > if that happens in the case of a severe memory resource
> > crisis. It certainly
> > would be okay in theory to do.
> >
> >
> > just my opinion based aon a potentially faulty recall
> > mechanism. The old
> > brain is not quite what it used to be.
> >
> >
> > Rick Cadruvi...
> >
> >
> > >All right, color me stupid, but why is it exactly that a
> > page cannot be
> > >paged out if its reference count is not zero? Wouldn't the
> > reference count
> > >be the number of valid references to this particular
> > virtual->physical
> > >mapping? Wouldn't a zero count indicate that the particular
> > mapping, if it
> > >is pageable, should be discarded? I thought, and perhaps I
> > am very mistaken
> > >here, that pageable virtual->physical mappings can always be
> > paged out as
> > >long as they are not explicitly 'pinned down', as in
> > MmProbeAndLockPages.
> > >
> > >> -----Original Message-----
> > >> From: Maxim S. Shatskih [mailto:[email protected]]
> > >> Sent: Monday, April 03, 2000 3:42 AM
> > >> To: NT Developers Interest List
> > >> Subject: [ntdev] Re: Are page tables paged out?
> > >>
> > >>
> > >> > Really? So in other words, a page in the page table can
> > >> only be paged out
> > >> > if all of the pages that it describes have been paged out
> > >> themselves?
> > >> i.e.
> > >> > the page tables are the last thing to get paged out?
> > >>
> > >> Certainly. And can you imagine any other ways of pageable
> > page tables?
> > >> When a PTE is made valid, the reference count for the page
> > >> table containing
> > >> the PTE is incremented. And vice versa. Page can be swapped
> > >> out only if
> > >> refcount == 0.
> > >>
> > >> Max
> > >>
> > >>
> > >>
> > >> ---
> > >> You are currently subscribed to ntdev as: [email protected]
> > >> To unsubscribe send a blank email to
> > $subst('Email.Unsub')
> > >>
> > >
> > >---
> > >You are currently subscribed to ntdev as: [email protected]
> > >To unsubscribe send a blank email to $subst('Email.Unsub')
> >
> > ---
> > You are currently subscribed to ntdev as: [email protected]
> > To unsubscribe send a blank email to $subst('Email.Unsub')
> >
>
> ---
> You are currently subscribed to ntdev as: [email protected]
> To unsubscribe send a blank email to $subst('Email.Unsub')
explanations that work or don't work for 'what max refers to' :-)
I agree that shared pages are reference counted.
The point that interested me was the issue of does a page table ever page
out while there are paged-in PTE's referenced from that page table. I'm
still not convinced that the page table is referenced counted by each paged
in PTE. However I am convinced that PTEs are not referenced counted by each
referring Page Table, the 'vice versa' portion of 'what max referred to'.
More to the point, this thread at some point asserted that Page Tables would
rarely if ever page out, and I just don't think that that is true.
> -----Original Message-----
> From: I?aki Castillo [mailto:[email protected]]
> Sent: Monday, April 03, 2000 2:18 PM
> To: NT Developers Interest List
> Subject: [ntdev] Re: Are page tables paged out?
>
>
> I think Max refers to shared pages only. These pages have a
> count reference
> because actually they are not actual pages but pointers to
> prototype entry
> pages.
> The pages are not discarded except if its reference (in the Page Frame
> Database) counter is zero.
> However this has nothing to do with normal, non shared pages.
>
> Inaki.
>
> > -----Original Message-----
> > From: Roddy, Mark
> > Sent: lunes 3 de abril de 2000 20:02
> > To: NT Developers Interest List
> > Subject: [ntdev] Re: Are page tables paged out?
> >
> > Sure I agree, but I'm still having trouble with Max saying:
> >
> > "When a PTE is made valid, the reference count for the page table
> > containing
> > the PTE is incremented. And vice versa. "
> >
> > The reference on the Page Table might or might not be OS MM
> policy, but he
> > is saying here, unless I'm misreading, that the reference
> count on the PTE
> > is also incremented, and that would make all PTE's effectively
> > non-pageable.
> >
> >
> > > -----Original Message-----
> > > From: [email protected] [mailto:[email protected]]
> > > Sent: Monday, April 03, 2000 11:13 AM
> > > To: NT Developers Interest List
> > > Subject: [ntdev] Re: Are page tables paged out?
> > >
> > >
> > > Please forgive any errors in this description. It has been a long
> > > time since I studied this and I am doing this off the top
> of my head.
> > >
> > > Page table entries are hardware dependent. In fact, it would
> > > be possible
> > > (I think MIPS is this way) to NOT have a hardware PTE.
> The hardware
> > > implementations that I have looked at tdo not have
> "reference counts"
> > > in the PTEs. They would effectively be meaningless. They
> > > would have a "valid"
> > > bit as well as some kind of "dirty" bit to indicate that
> > > there was a recent
> > > write to the page so that page cleanup can make sure that
> > > section files
> > > get written out to the disk before pages are removed.
> > >
> > > The reference count referred to is probably the one in
> the Page Frame
> > > Database. This reference count is used to "lock" down
> pages. The PFN
> > > database is an OS construct and has NOTHING to do with
> the hardware
> > > except that it helps the OS track physical memory pages. By
> > > OS definition,
> > > a page with a non-zero reference count in it's PFN database
> > > entry is NOT
> > > pageable. This is essentially how MmProbeAndLockPages
> locks pages.
> > > It's a different mechanism, but similar that is used to lock
> > > pages within
> > > a working set. A process has a working set list that works
> > > in a similar
> > > manner to the PFN database to provide for working set trimming.
> > >
> > > I would imagine that what Max was saying is that when a page
> > > mapped by a
> > > PTE is paged in, the reference count for the page containing
> > > the PTE gets
> > > incremented to prevent it's being paged. I am not certain
> > > that this happens,
> > > but it certainly makes sense. In theory, however, there
> > > would be no reason
> > > why an OS couldn't choose to page PTE pages without paging
> > > the pages that
> > > they map. It wouldn't make sense to do this since a page
> fault would
> > > need to be rendered the next time a reference happened.
> > > Therefore, it makes
> > > sense to only page out a PTE page when all the pages it maps
> > > are paged out.
> > > I suppose that if you had some pages mapped by a PTE that
> were locked
> > > (probably for a DMA transfer), you might decide to page
> the PTE page
> > > since ALL the pages it maps cannot be paged. I wouldn't even
> > > be surprised
> > > if that happens in the case of a severe memory resource
> > > crisis. It certainly
> > > would be okay in theory to do.
> > >
> > >
> > > just my opinion based aon a potentially faulty recall
> > > mechanism. The old
> > > brain is not quite what it used to be.
> > >
> > >
> > > Rick Cadruvi...
> > >
> > >
> > > >All right, color me stupid, but why is it exactly that a
> > > page cannot be
> > > >paged out if its reference count is not zero? Wouldn't the
> > > reference count
> > > >be the number of valid references to this particular
> > > virtual->physical
> > > >mapping? Wouldn't a zero count indicate that the particular
> > > mapping, if it
> > > >is pageable, should be discarded? I thought, and perhaps I
> > > am very mistaken
> > > >here, that pageable virtual->physical mappings can always be
> > > paged out as
> > > >long as they are not explicitly 'pinned down', as in
> > > MmProbeAndLockPages.
> > > >
> > > >> -----Original Message-----
> > > >> From: Maxim S. Shatskih [mailto:[email protected]]
> > > >> Sent: Monday, April 03, 2000 3:42 AM
> > > >> To: NT Developers Interest List
> > > >> Subject: [ntdev] Re: Are page tables paged out?
> > > >>
> > > >>
> > > >> > Really? So in other words, a page in the page table can
> > > >> only be paged out
> > > >> > if all of the pages that it describes have been paged out
> > > >> themselves?
> > > >> i.e.
> > > >> > the page tables are the last thing to get paged out?
> > > >>
> > > >> Certainly. And can you imagine any other ways of pageable
> > > page tables?
> > > >> When a PTE is made valid, the reference count for the page
> > > >> table containing
> > > >> the PTE is incremented. And vice versa. Page can be swapped
> > > >> out only if
> > > >> refcount == 0.
> > > >>
> > > >> Max
> > > >>
> > > >>
> > > >>
> > > >> ---
> > > >> You are currently subscribed to ntdev as:
> [email protected]
> > > >> To unsubscribe send a blank email to
> > > $subst('Email.Unsub')
> > > >>
> > > >
> > > >---
> > > >You are currently subscribed to ntdev as: [email protected]
> > > >To unsubscribe send a blank email to
> $subst('Email.Unsub')
> > >
> > > ---
> > > You are currently subscribed to ntdev as: [email protected]
> > > To unsubscribe send a blank email to
> $subst('Email.Unsub')
> > >
> >
> > ---
> > You are currently subscribed to ntdev as: [email protected]
> > To unsubscribe send a blank email to $subst('Email.Unsub')
>
> ---
> You are currently subscribed to ntdev as: [email protected]
> To unsubscribe send a blank email to $subst('Email.Unsub')
>
Yes. And they are pinned down by incrementing the reference count.
You can think of the reference count as of the "pin count".
The number of mapping - the number of PTEs referencing the page - is not a
reference count, but a _share_ count - a different thing.
Max
> PTE is paged in, the reference count for the page containing the PTE gets
> incremented to prevent it's being paged.
This happens - I have spent some time investigating the assembly code of
MmAccessFault and other routines - so, I can say that this is for sure.
Throwing the page away from the physical memory required update to its
PTE to place a swapfile address in it. So, the PTE - and the page table
page - must be resident at this moment. That's why it's impossible to page
out the page table while it has even transition (standby list) PTEs, not to
say
present PTEs.
Max
> is saying here, unless I'm misreading, that the reference count on the PTE
> is also incremented, and that would make all PTE's effectively non-
No. PTEs has no reference counts - only physical pages (PFN entries) has
such.
Max
> Database) counter is zero.
> However this has nothing to do with normal, non shared pages.
PFN database keeps both ShareCount (for shared pages. This is number of
present PTEs referencing the page) and ReferenceCount (this is "pin count"
incremented in MmProbeAndLockPages).
So, even non-shared pages have a reference count.
The idea described by me is something like - "MmProbeAndLock is done
on the page table the number of times equal to the number of
present/transition PTEs in it".
Max