Question on paged pool

NTDEV
1

Hi,

I have some doubts on paged pool (read some books/articles, but that didnt
give me clear picture). So thought of asking it here. Please note I am from
linux background.

This is about 32-bit windows 2k3 server having around 4GB of RAM where
default distribution is 2GB for user virtual address space and 2GB for
kernel virtual address space. For this configuration there is 256MB of
non-paged pool and upto 650MB of paged pool in the system (got these figures
from
http://blogs.technet.com/b/markrussinovich/archive/2009/03/26/3211216.aspx?PageIndex=2
).

Questions:

  1. Is non-paged pool similar to “directly mapped kernel” addresses in linux
    which is always resident in memory?

  2. When one allocates from non-paged pool, is that memory physically
    contiguous (which is the case with memory allocated by kmalloc in linux)?

  3. Is “Paged pool” actually the amount of kernel virtual memory (addresses)
    available for use, i.e. memory allocated from paged pool is virtually
    contiguous but may not be physically contiguous (similar to vmalloc() in
    linux with exception that “paged pool” is swappable and some part of it may
    reside on “swap device”)?

  4. If “paged pool” size is 650MB and hypothetically if there are 13 kernel
    modules each of which consume 50MB of paged pool (totaling 650MB), will the
    “paged pool” exhaust meaning we wont be able to allocate any more memory
    from “paged pool”, irrespective of how much RAM we put in the system,
    irrespective of how much swap space (page file) we have? Is this the
    “LowPagedPool” condition?

  5. If I have sufficient RAM (4 GB or more), will the system still need swap
    space (page file) to manage “kernel pageable pool” (I guess to manage user
    virtual address space “page file” will still be required), as sufficient
    amount of physical memory will be available for the “kernel paged pool” of
    size 650MB?

Thanks,
Sunil

2

Answers inline:

“Sunil Patil” wrote in message news:xxxxx@ntdev:

> Hi,
>
> I have some doubts on paged pool (read some books/articles, but that didnt
> give me clear picture). So thought of asking it here. Please note I am from
> linux background.
>
> This is about 32-bit windows 2k3 server having around 4GB of RAM where
> default distribution is 2GB for user virtual address space and 2GB for
> kernel virtual address space. For this configuration there is 256MB of
> non-paged pool and upto 650MB of paged pool in the system (got these figures
> from
> http://blogs.technet.com/b/markrussinovich/archive/2009/03/26/3211216.aspx?PageIndex=2
> ).
>
> Questions:
> 1. Is non-paged pool similar to “directly mapped kernel” addresses in linux
> which is always resident in memory?

The non-paged pool is always resident.
>
> 2. When one allocates from non-paged pool, is that memory physically
> contiguous (which is the case with memory allocated by kmalloc in linux)?

The memory is not contiguous for non-paged pool.
>
> 3. Is “Paged pool” actually the amount of kernel virtual memory (addresses)
> available for use, i.e. memory allocated from paged pool is virtually
> contiguous but may not be physically contiguous (similar to vmalloc() in
> linux with exception that “paged pool” is swappable and some part of it may
> reside on “swap device”)?

Paged pool is just that a heap where the memory is virtually contiguous
and the pages may not be resident.
>
> 4. If “paged pool” size is 650MB and hypothetically if there are 13 kernel
> modules each of which consume 50MB of paged pool (totaling 650MB), will the
> “paged pool” exhaust meaning we wont be able to allocate any more memory
> from “paged pool”, irrespective of how much RAM we put in the system,
> irrespective of how much swap space (page file) we have? Is this the
> “LowPagedPool” condition?

In systems prior to Win7 the limit was pretty fixed, if you had a number
of items that allocated large blocks of memory the pool was not going to
grow. Even worse the virtually contiguous nature of the memory means
that it will fragment and you cannot always get the space you want.
>
> 5. If I have sufficient RAM (4 GB or more), will the system still need swap
> space (page file) to manage “kernel pageable pool” (I guess to manage user
> virtual address space “page file” will still be required), as sufficient
> amount of physical memory will be available for the “kernel paged pool” of
> size 650MB?

The page file is still needed.

Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

3

> > Questions:

> 1. Is non-paged pool similar to “directly mapped kernel” addresses in
> linux which is always resident in memory?

The non-paged pool is always resident.

The nonpaged pool is always resident but it’s not directly mapped.

> 4. If “paged pool” size is 650MB and hypothetically if there are 13
> kernel modules each of which consume 50MB of paged pool (totaling
> 650MB), will the “paged pool” exhaust meaning we wont be able to
> allocate any more memory from “paged pool”, irrespective of how much
> RAM we put in the system, irrespective of how much swap space (page
> file) we have? Is this the “LowPagedPool” condition?

In systems prior to Win7 the limit was pretty fixed

This actually changed in Vista. The 32-bit kernel address space is now
allocated dynamically based on demand so in theory paged pool on 32-bit
Vista and Win7 could grow to almost 2 GB (assuming there is enough commit
available, nothing else is using a lot of system VA space, etc).

> 5. If I have sufficient RAM (4 GB or more), will the system still need
> swap space (page file) to manage “kernel pageable pool” (I guess to
> manage user virtual address space “page file” will still be required),
> as sufficient amount of physical memory will be available for the
> “kernel paged pool” of size 650MB?

The page file is still needed.

Running without a pagefile is not recommended but it should work as long as
the system doesn’t reach the commit limit.

With 4 GB of RAM and no pagefile the commit limit will be 4 GB. If the rest
of the system (user processes, nonpaged pool etc) consume say 2 GB then you
should be able to allocate significantly more than 650 MB of paged pool.

There are of course other reasons why you typically still want to use a
pagefile even if you have enough RAM to make sure you don’t run out of
commit under usual circumstances (such as not having to waste RAM on pages
that are rarely/never used, being able to create kernel crash dumps,
being able to handle unexpected memory usage spikes/leaks, etc).

4

On Sun, Jan 9, 2011 at 3:14 AM, Pavel Lebedynskiy wrote:

>
> > > 4. If “paged pool” size is 650MB and hypothetically if there are 13
> > > kernel modules each of which consume 50MB of paged pool (totaling
> > > 650MB), will the “paged pool” exhaust meaning we wont be able to
> > > allocate any more memory from “paged pool”, irrespective of how much
> > > RAM we put in the system, irrespective of how much swap space (page
> > > file) we have? Is this the “LowPagedPool” condition?
> >
> > In systems prior to Win7 the limit was pretty fixed
>
> This actually changed in Vista. The 32-bit kernel address space is now
> allocated dynamically based on demand so in theory paged pool on 32-bit
> Vista and Win7 could grow to almost 2 GB (assuming there is enough commit
> available, nothing else is using a lot of system VA space, etc).
>
>
May be I am missing something here. Given that kernel virtual address space
is only 2GB, how can paged pool in vista grow upto 2GB. I guess OS data
structures would consume at least 500MB or some part of kernel virtual
address space?

My understanding is that in win2k3 900MB of virtual address is used for
managing paged-pool and non-paged pool (non-paged = 256 MB and paged pool =
650MB) and remaining around 1.1GB used for OS data structures/book-keeping
which seems reasonable.

5

The statement was ‘almost 2G’. OS data structures do not consume 0.5G,
and not all of them are non-paged.

Mark Roddy

On Mon, Jan 10, 2011 at 4:03 AM, Sunil Patil wrote:
>
> On Sun, Jan 9, 2011 at 3:14 AM, Pavel Lebedynskiy
> wrote:
>>
>> > > 4. If “paged pool” size is 650MB and hypothetically if there are 13
>> > > kernel modules each of which consume 50MB of paged pool (totaling
>> > > 650MB), will the “paged pool” exhaust meaning we wont be able to
>> > > allocate any more memory from “paged pool”, irrespective of how much
>> > > RAM we put in the system, irrespective of how much swap space (page
>> > > file) we have? Is this the “LowPagedPool” condition?
>> >
>> > In systems prior to Win7 the limit was pretty fixed
>>
>> This actually changed in Vista. The 32-bit kernel address space is now
>> allocated dynamically based on demand so in theory paged pool on 32-bit
>> Vista and Win7 could grow to almost 2 GB (assuming there is enough commit
>> available, nothing else is using a lot of system VA space, etc).
>>
>
> May be I am missing something here. Given that kernel virtual address space
> is only 2GB, how can paged pool in vista grow upto 2GB. I guess OS data
> structures would consume at least 500MB or some part of kernel virtual
> address space?
>
> My understanding is that in win2k3 900MB of virtual address is used for
> managing paged-pool and non-paged pool (non-paged = 256 MB and paged pool =
> 650MB) and remaining around 1.1GB used for OS data structures/book-keeping
> which seems reasonable.
>
>
> — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and
> other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the
> List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

6

>1. Is non-paged pool similar to “directly mapped kernel” addresses in linux which is always resident in

memory?

NPP is resident, but it is not necessary directly mapped.

  1. When one allocates from non-paged pool, is that memory physically contiguous

Not necessary.

able to allocate any more memory from “paged pool”

Yes, though the limits were very much relaxed in Vista+.

“kernel pageable pool” (I guess to manage user virtual address space “page file” will still be
required)

There was some time since I was running Windows without a pagefile, but I think it is supported if there is plenty of RAM.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

7

>> May be I am missing something here. Given that kernel virtual address

> space is only 2GB, how can paged pool in vista grow upto 2GB. I guess
> OS data structures would consume at least 500MB or some part of kernel
virtual address space?
>
> My understanding is that in win2k3 900MB of virtual address is used
> for managing paged-pool and non-paged pool (non-paged = 256 MB and
> paged pool =
> 650MB) and remaining around 1.1GB used for OS data
> structures/book-keeping which seems reasonable.

The statement was ‘almost 2G’. OS data structures do not consume 0.5G,
and not all of them are non-paged.

!vm 21 debugger command shows kernel VA space usage by category (paged
pool, nonpaged pool, system PTE mappings etc). (this only works on 32-bit
Vista/Win7+ since these are the only Windows versions where the kernel
VA space is allocated dynamically)

Here’s the output from my test machine:

kd> !vm 21

VA Type CurrentUse Peak Limit Failures
Unused 1424 Mb 0 Mb OPEN 0
SessionSpace 32 Mb 36 Mb OPEN 0
ProcessSpace 16 Mb 0 Mb OPEN 0
BootLoaded 24 Mb 48 Mb OPEN 0
PfnDatabase 12 Mb 76 Mb OPEN 0
NonPagedPool 74 Mb 94 Mb OPEN 0
PagedPool 116 Mb 164 Mb OPEN 0
SpecialPool 0 Mb 0 Mb OPEN 0
SystemCache 258 Mb 440 Mb OPEN 0
SystemPtes 54 Mb 66 Mb OPEN 0
Hal 4 Mb 0 Mb OPEN 0
SessionGlobal 12 Mb 12 Mb OPEN 0
Driver Images 22 Mb 0 Mb OPEN 0
NPSpecialPool 0 Mb 0 Mb OPEN 0

The paged pool could potentially grow to almost 1800 MB here: 116 MB
currently consumed, plus 1424 MB that are currently not used for
anything, plus ~250 MB used by the system cache (the system cache will
release its mappings when available kernel VA space becomes too low).

Of course that doesn’t mean consuming that much paged pool on a 32 bit
system is actually a good idea.