Hi all,
I am newbie in windows driver development. I just want to know , a global variable in a driver will use paged pool memory or non paged pool memory ?
Thanks and Regards
Navaneeth
nonpaged by default
Mark Roddy
On Tue, Apr 28, 2009 at 6:33 AM, wrote:
> Hi all,
>
> I am newbie in windows driver development. I just want to know , a global variable in a driver will use paged pool memory or non paged pool memory ?
>
> Thanks and Regards
>
> Navaneeth
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
Nonpaged.
Declare it with #pragma data_seg(“PAGE”) or the similar pragma (alloc_data IIRC, but I’m not sure) for it to be paged.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
wrote in message news:xxxxx@ntdev…
> Hi all,
>
> I am newbie in windows driver development. I just want to know , a global variable in a driver will use paged pool memory or non paged pool memory ?
>
> Thanks and Regards
>
> Navaneeth
>
>
Neither one.
“Global variables” are not located in pool. They’re located with your driver image in main memory, in memory that is non-pageable by default.
They are not, however, located in either paged or non-paged pool.
Peter
OSR
I interpreted the newbie question to be ‘is this data paged or non-paged’.
Mark Roddy
On Tue, Apr 28, 2009 at 9:54 AM, wrote:
> Neither one.
>
> “Global variables” are not located in pool. ?They’re located with your driver image in main memory, in memory that is non-pageable by default.
>
> They are not, however, located in either paged or non-paged pool.
>
> Peter
> OSR
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
> “Global variables” are not located in pool.
They’re located with your driver image in main memory, in memory that is
non-pageable by default.
This is of course true, but I would bet that internally IopLoadDriver
[or MmLoadSystemImage or LdrLoadModule, whatever] allocates
non-paged memory “as everyone else”, through ExAllocate, so the
driver’s image does grab a chunk of a non-paged pool.
If my bet is correct, there is no error in the reponse “non-paged pool”
someone
gave.
They are not, however, located in either paged or non-paged pool.
… and when I unload, where does this now unused memory go then?
My guess - to the pool, where else…
----- Original Message -----
From:
To: “Windows System Software Devs Interest List”
Sent: Tuesday, April 28, 2009 9:54 AM
Subject: RE:[ntdev] Paged or Non Paged ???
> Neither one.
>
> “Global variables” are not located in pool. They’re located with your
> driver image in main memory, in memory that is non-pageable by default.
>
> They are not, however, located in either paged or non-paged pool.
>
> Peter
> OSR
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
Alex Shvedov wrote:
> “Global variables” are not located in pool.
> They’re located with your driver image in main memory, in memory that
> is non-pageable by default.
This is of course true, but I would bet that internally IopLoadDriver
[or MmLoadSystemImage or LdrLoadModule, whatever] allocates
non-paged memory “as everyone else”, through ExAllocate, so the
driver’s image does grab a chunk of a non-paged pool.
No. Remember, the driver image is not “copied” in to memory. It is
“mapped” in to memory. That’s an operation that is completely different
from pool allocations.
> They are not, however, located in either paged or non-paged pool.
… and when I unload, where does this now unused memory go then?
My guess - to the pool, where else…
No. The memory goes AWAY. The pages are deallocated. They no longer
exist.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
> No. Remember, the driver image is not “copied” in to memory. It is
“mapped” in to memory. That’s an operation that is completely different
from pool allocations.
Wow!
Wrong I was.
Could have guessed but did not
Thanks.
No. The memory goes AWAY. The pages are deallocated. They no longer
exist.
… as a consequence.
VAs are freed, not “memory”.
----- Original Message -----
From: “Tim Roberts”
To: “Windows System Software Devs Interest List”
Sent: Tuesday, April 28, 2009 12:39 PM
Subject: Re: [ntdev] Paged or Non Paged ???
> Alex Shvedov wrote:
>>> “Global variables” are not located in pool.
>>> They’re located with your driver image in main memory, in memory that
>>> is non-pageable by default.
>> This is of course true, but I would bet that internally IopLoadDriver
>> [or MmLoadSystemImage or LdrLoadModule, whatever] allocates
>> non-paged memory “as everyone else”, through ExAllocate, so the
>> driver’s image does grab a chunk of a non-paged pool.
>
> No. Remember, the driver image is not “copied” in to memory. It is
> “mapped” in to memory. That’s an operation that is completely different
> from pool allocations.
>
>
>>> They are not, however, located in either paged or non-paged pool.
>> … and when I unload, where does this now unused memory go then?
>> My guess - to the pool, where else…
>
> No. The memory goes AWAY. The pages are deallocated. They no longer
> exist.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
A very common newb mistake is to use the terms “paged pool” and “non-page pool” interchangeably with “pageable system memory” and “non-pageable system memory” – It is often a common newb mistake to assume that the system allocates ALL memory from one of the “pools” when in fact, the “pool” is a specific area reserved for the dynamic storage of OS data structures.
The best way to help these folks, I think, is to help them understand the correct terminology from the beginning. This isn’t a game of “catch the newb”… rather, it helps avoid major misconceptions later on.
Peter
OSR
Tim ,
Can you explain this a little more.
> No. Remember, the driver image is not “copied” in to memory. It is
> “mapped” in to memory. That’s an operation that is completely different
> from pool allocations.
When you say that it is “mapped” in to memory…where the “memory” comes from. Doesn’t the loader has to read the .sys file from the disk and bring it in to SOME memory? Can you please explain?
Thanks,
– Ajitabh
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Tuesday, April 28, 2009 9:40 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Paged or Non Paged ???
Alex Shvedov wrote:
> “Global variables” are not located in pool.
> They’re located with your driver image in main memory, in memory that
> is non-pageable by default.
This is of course true, but I would bet that internally IopLoadDriver
[or MmLoadSystemImage or LdrLoadModule, whatever] allocates
non-paged memory “as everyone else”, through ExAllocate, so the
driver’s image does grab a chunk of a non-paged pool.
No. Remember, the driver image is not “copied” in to memory. It is
“mapped” in to memory. That’s an operation that is completely different
from pool allocations.
> They are not, however, located in either paged or non-paged pool.
… and when I unload, where does this now unused memory go then?
My guess - to the pool, where else…
No. The memory goes AWAY. The pages are deallocated. They no longer
exist.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>When you say that it is “mapped” in to memory…where the “memory” comes from. Doesn’t the loader
has to read the .sys file from the disk and bring it in to SOME memory?
I think the loader reads the whole driver from .SYS file to the paged pool area and then locks all sections whose names do not start with “PAGE”.
At least you can delete and replace the loaded .SYS (not so with user-mode DLLs and EXEs).
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
>I think the loader reads the whole driver from .SYS file to the paged pool
area and then locks all sections whose names do not start >with “PAGE”.
Don’t forget that drivers can also go in session space.
And AFAIK it’s the other way around, system PTEs are allocated to map the
driver image, the driver image is copied into memory, and PTEs for PAGE
sections are marked pageable.
-scott
–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com
“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
>When you say that it is “mapped” in to memory…where the “memory” comes
>from. Doesn’t the loader
>has to read the .sys file from the disk and bring it in to SOME memory?
I think the loader reads the whole driver from .SYS file to the paged pool
area and then locks all sections whose names do not start with “PAGE”.
At least you can delete and replace the loaded .SYS (not so with user-mode
DLLs and EXEs).
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
> system PTEs are allocated to map the driver image, the driver image is
copied into memory
What kind of memory?
ExAllocated by driver loader?
common newb mistake is to use the terms “paged pool” and “non-page
pool” interchangeably with “pageable system memory” and “non-pageable
system memory”
Would it be too much to ask you to explain the difference?
Googling brought up nothing, really - except an obvious fact that
[non]pageable pool belongs to [non]pageable memory, like in
WDK’s POOL_TYPE description: “… Nonpaged pool, which is
nonpageable system memory”.
Not much.
Is there any way to confirm that this difference that
newbies forget about really exisst?
I don’t have Mark’s bible at hand…
----- Original Message -----
From: “Scott Noone”
Newsgroups: ntdev
To: “Windows System Software Devs Interest List”
Sent: Tuesday, April 28, 2009 4:24 PM
Subject: Re:[ntdev] Paged or Non Paged ???
> >I think the loader reads the whole driver from .SYS file to the paged
> >pool area and then locks all sections whose names do not start >with
> >“PAGE”.
>
> Don’t forget that drivers can also go in session space.
>
> And AFAIK it’s the other way around, system PTEs are allocated to map the
> driver image, the driver image is copied into memory, and PTEs for PAGE
> sections are marked pageable.
>
>
> -scott
>
> –
> Scott Noone
> Consulting Associate
> OSR Open Systems Resources, Inc.
> http://www.osronline.com
>
>
> “Maxim S. Shatskih” wrote in message
> news:xxxxx@ntdev…
>>When you say that it is “mapped” in to memory…where the “memory” comes
>>from. Doesn’t the loader
>>has to read the .sys file from the disk and bring it in to SOME memory?
>
> I think the loader reads the whole driver from .SYS file to the paged pool
> area and then locks all sections whose names do not start with “PAGE”.
>
> At least you can delete and replace the loaded .SYS (not so with user-mode
> DLLs and EXEs).
>
> –
> Maxim S. Shatskih
> Windows DDK MVP
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
Usually mapping means map the file to address space. In the past, it was way
difficult to hold the whole images into memory. Then came the paging
mechanism, so there is no need to get the whole image copied only to see the
respective pageable pages gets paged out… Something along this line. This
way many programs can be multiplexed over memory…
Read something about NT file ( including image) mapping …
-pro
On Tue, Apr 28, 2009 at 12:51 PM, Ajitabh Saxena wrote:
> Tim ,
> Can you explain this a little more.
>
> >> No. Remember, the driver image is not “copied” in to memory. It is
> >> “mapped” in to memory. That’s an operation that is completely different
> >> from pool allocations.
>
> When you say that it is “mapped” in to memory…where the “memory” comes
> from. Doesn’t the loader has to read the .sys file from the disk and bring
> it in to SOME memory? Can you please explain?
>
> Thanks,
> – Ajitabh
>
> -----Original Message-----
> From: xxxxx@lists.osr.com [mailto:
> xxxxx@lists.osr.com] On Behalf Of Tim Roberts
> Sent: Tuesday, April 28, 2009 9:40 AM
> To: Windows System Software Devs Interest List
> Subject: Re: [ntdev] Paged or Non Paged ???
>
> Alex Shvedov wrote:
> >> “Global variables” are not located in pool.
> >> They’re located with your driver image in main memory, in memory that
> >> is non-pageable by default.
> > This is of course true, but I would bet that internally IopLoadDriver
> > [or MmLoadSystemImage or LdrLoadModule, whatever] allocates
> > non-paged memory “as everyone else”, through ExAllocate, so the
> > driver’s image does grab a chunk of a non-paged pool.
>
> No. Remember, the driver image is not “copied” in to memory. It is
> “mapped” in to memory. That’s an operation that is completely different
> from pool allocations.
>
>
> >> They are not, however, located in either paged or non-paged pool.
> > … and when I unload, where does this now unused memory go then?
> > My guess - to the pool, where else…
>
> No. The memory goes AWAY. The pages are deallocated. They no longer
> exist.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
Looks like I found the answer to my own question at, for example,
http://codeidol.com/other/inside-windows-2000/Memory-Management/Address-Space-Layout/
Among other things, the picture shows space for mapping (ZwCreateSection and friends),
and yes, it is not inside a pool.
----- Original Message -----
From: Prokash Sinha
To: Windows System Software Devs Interest List
Sent: Tuesday, April 28, 2009 4:50 PM
Subject: Re: [ntdev] Paged or Non Paged ???
Usually mapping means map the file to address space. In the past, it was way difficult to hold the whole images into memory. Then came the paging mechanism, so there is no need to get the whole image copied only to see the respective pageable pages gets paged out… Something along this line. This way many programs can be multiplexed over memory…
Read something about NT file ( including image) mapping …
-pro
On Tue, Apr 28, 2009 at 12:51 PM, Ajitabh Saxena wrote:
Tim ,
Can you explain this a little more.
>> No. Remember, the driver image is not “copied” in to memory. It is
>> “mapped” in to memory. That’s an operation that is completely different
>> from pool allocations.
When you say that it is “mapped” in to memory…where the “memory” comes from. Doesn’t the loader has to read the .sys file from the disk and bring it in to SOME memory? Can you please explain?
Thanks,
– Ajitabh
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Tuesday, April 28, 2009 9:40 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Paged or Non Paged ???
Alex Shvedov wrote:
>> “Global variables” are not located in pool.
>> They’re located with your driver image in main memory, in memory that
>> is non-pageable by default.
> This is of course true, but I would bet that internally IopLoadDriver
> [or MmLoadSystemImage or LdrLoadModule, whatever] allocates
> non-paged memory “as everyone else”, through ExAllocate, so the
> driver’s image does grab a chunk of a non-paged pool.
No. Remember, the driver image is not “copied” in to memory. It is
“mapped” in to memory. That’s an operation that is completely different
from pool allocations.
>> They are not, however, located in either paged or non-paged pool.
> … and when I unload, where does this now unused memory go then?
> My guess - to the pool, where else…
No. The memory goes AWAY. The pages are deallocated. They no longer
exist.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Well you could trust the wise old folks who are telling you it really exists. That might be more sensible than trusting Google which knows something about everything but is an expert in nothing (except perhaps advertising :)). You could also buy a copy of the Windows Internals book which explains a lot of this.
There are three levels of memory management.
At one level MM manages physical pages of memory. MM keeps track of whether each physical page is in use, free, zeroed, locked, dirty, etc…
Above that is MM’s management of the virtual address space (VA) - per-process VA spaces, per-session VA space and the kernel VA space. MM manages the reservation and release of VA ranges and separately tracks which physical page is assigned to each virtual page. At this level MM manages which virtual addresses are being used in which address spaces and can allow their corresponding physical pages to be brought into or removed from the process’s working set.
When a physical page is evicted from a working set (either by being freed or being aged out) that drops its locked count. When the lock count drops to zero the physical page can be freed. If the page is dirty and backed by a file then MM will write the contents out to the file before freeing it. If the VA was evicted and is accessed by the process then a page fault occurs and MM will allocate a new physical page and read the appropriate contents back in from the backing file.
The equivalent of this in user-mode is VirtualAlloc, which allows an application to reserve a chunk of its own VA space and also to back that chunk by physical memory. Similarly CreateFileMapping allows the process to allocate a virtual address range which is backed by a file. When MM goes to make a page in that chunk resident it will load the contents from the backing file rather than just using a zeroed page (as it would with VirtualAlloc).
Pool is built on top of this mechanism. It’s the equivalent of the user-mode heap or the CRT’s malloc function (not the equivalent of a malloc system call). The pool allocates virtual memory from Mm and then subdivides that into individual pool allocations for the callers. Pool is just one way of getting virtual address space, and one of the only options exported by the kernel, but it is separate from the underlying virtual memory management layer that it depends on.
-p
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alex Shvedov
Sent: Tuesday, April 28, 2009 1:50 PM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] Paged or Non Paged ???
system PTEs are allocated to map the driver image, the driver image is
copied into memory
What kind of memory?
ExAllocated by driver loader?
common newb mistake is to use the terms “paged pool” and “non-page
pool” interchangeably with “pageable system memory” and “non-pageable
system memory”
Would it be too much to ask you to explain the difference?
Googling brought up nothing, really - except an obvious fact that
[non]pageable pool belongs to [non]pageable memory, like in
WDK’s POOL_TYPE description: “… Nonpaged pool, which is
nonpageable system memory”.
Not much.
Is there any way to confirm that this difference that
newbies forget about really exisst?
I don’t have Mark’s bible at hand…
----- Original Message -----
From: “Scott Noone”
Newsgroups: ntdev
To: “Windows System Software Devs Interest List”
Sent: Tuesday, April 28, 2009 4:24 PM
Subject: Re:[ntdev] Paged or Non Paged ???
> >I think the loader reads the whole driver from .SYS file to the paged
> >pool area and then locks all sections whose names do not start >with
> >“PAGE”.
>
> Don’t forget that drivers can also go in session space.
>
> And AFAIK it’s the other way around, system PTEs are allocated to map the
> driver image, the driver image is copied into memory, and PTEs for PAGE
> sections are marked pageable.
>
>
> -scott
>
> –
> Scott Noone
> Consulting Associate
> OSR Open Systems Resources, Inc.
> http://www.osronline.com
>
>
> “Maxim S. Shatskih” wrote in message
> news:xxxxx@ntdev…
>>When you say that it is “mapped” in to memory…where the “memory” comes
>>from. Doesn’t the loader
>>has to read the .sys file from the disk and bring it in to SOME memory?
>
> I think the loader reads the whole driver from .SYS file to the paged pool
> area and then locks all sections whose names do not start with “PAGE”.
>
> At least you can delete and replace the loaded .SYS (not so with user-mode
> DLLs and EXEs).
>
> –
> Maxim S. Shatskih
> Windows DDK MVP
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Alex Shvedov wrote:
> system PTEs are allocated to map the driver image, the driver image is
> copied into memory
What kind of memory?
ExAllocated by driver loader?> common newb mistake is to use the terms “paged pool” and “non-page
> pool” interchangeably with “pageable system memory” and “non-pageable
> system memory”
Would it be too much to ask you to explain the difference?
Not at all - that’s why he did so in the sentence you snipped which
followed the sentence you quoted.
“It is often a common newb mistake to assume that the system allocates
ALL memory from one of the “pools” when in fact, the “pool” is a
specific area reserved for the dynamic storage of OS data structures.”
Googling brought up nothing, really - except an obvious fact that
[non]pageable pool belongs to [non]pageable memory, like in
WDK’s POOL_TYPE description: “… Nonpaged pool, which is
nonpageable system memory”.Not much.
Is there any way to confirm that this difference that
newbies forget about really exisst?
If you are not prepared to believe the experts who know the Windows
source code intimately, your only option is to find some way to look at
the source code yourself.
This is great. Thanks Peter.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Peter Wieland
Sent: Tuesday, April 28, 2009 2:42 PM
To: Windows System Software Devs Interest List
Subject: RE: Re:[ntdev] Paged or Non Paged ???
Well you could trust the wise old folks who are telling you it really exists. That might be more sensible than trusting Google which knows something about everything but is an expert in nothing (except perhaps advertising :)). You could also buy a copy of the Windows Internals book which explains a lot of this.
There are three levels of memory management.
At one level MM manages physical pages of memory. MM keeps track of whether each physical page is in use, free, zeroed, locked, dirty, etc…
Above that is MM’s management of the virtual address space (VA) - per-process VA spaces, per-session VA space and the kernel VA space. MM manages the reservation and release of VA ranges and separately tracks which physical page is assigned to each virtual page. At this level MM manages which virtual addresses are being used in which address spaces and can allow their corresponding physical pages to be brought into or removed from the process’s working set.
When a physical page is evicted from a working set (either by being freed or being aged out) that drops its locked count. When the lock count drops to zero the physical page can be freed. If the page is dirty and backed by a file then MM will write the contents out to the file before freeing it. If the VA was evicted and is accessed by the process then a page fault occurs and MM will allocate a new physical page and read the appropriate contents back in from the backing file.
The equivalent of this in user-mode is VirtualAlloc, which allows an application to reserve a chunk of its own VA space and also to back that chunk by physical memory. Similarly CreateFileMapping allows the process to allocate a virtual address range which is backed by a file. When MM goes to make a page in that chunk resident it will load the contents from the backing file rather than just using a zeroed page (as it would with VirtualAlloc).
Pool is built on top of this mechanism. It’s the equivalent of the user-mode heap or the CRT’s malloc function (not the equivalent of a malloc system call). The pool allocates virtual memory from Mm and then subdivides that into individual pool allocations for the callers. Pool is just one way of getting virtual address space, and one of the only options exported by the kernel, but it is separate from the underlying virtual memory management layer that it depends on.
-p
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alex Shvedov
Sent: Tuesday, April 28, 2009 1:50 PM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] Paged or Non Paged ???
system PTEs are allocated to map the driver image, the driver image is
copied into memory
What kind of memory?
ExAllocated by driver loader?
common newb mistake is to use the terms “paged pool” and “non-page
pool” interchangeably with “pageable system memory” and “non-pageable
system memory”
Would it be too much to ask you to explain the difference?
Googling brought up nothing, really - except an obvious fact that
[non]pageable pool belongs to [non]pageable memory, like in
WDK’s POOL_TYPE description: “… Nonpaged pool, which is
nonpageable system memory”.
Not much.
Is there any way to confirm that this difference that
newbies forget about really exisst?
I don’t have Mark’s bible at hand…
----- Original Message -----
From: “Scott Noone”
Newsgroups: ntdev
To: “Windows System Software Devs Interest List”
Sent: Tuesday, April 28, 2009 4:24 PM
Subject: Re:[ntdev] Paged or Non Paged ???
> >I think the loader reads the whole driver from .SYS file to the paged
> >pool area and then locks all sections whose names do not start >with
> >“PAGE”.
>
> Don’t forget that drivers can also go in session space.
>
> And AFAIK it’s the other way around, system PTEs are allocated to map the
> driver image, the driver image is copied into memory, and PTEs for PAGE
> sections are marked pageable.
>
>
> -scott
>
> –
> Scott Noone
> Consulting Associate
> OSR Open Systems Resources, Inc.
> http://www.osronline.com
>
>
> “Maxim S. Shatskih” wrote in message
> news:xxxxx@ntdev…
>>When you say that it is “mapped” in to memory…where the “memory” comes
>>from. Doesn’t the loader
>>has to read the .sys file from the disk and bring it in to SOME memory?
>
> I think the loader reads the whole driver from .SYS file to the paged pool
> area and then locks all sections whose names do not start with “PAGE”.
>
> At least you can delete and replace the loaded .SYS (not so with user-mode
> DLLs and EXEs).
>
> –
> Maxim S. Shatskih
> Windows DDK MVP
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Well you could use the debugger and ask it which pool an address from
your driver’s code or data segments is in. But then you would have to
verify that the debugger wasn’t wrong, and that could be a real
problem. Probably the google would provide many opinions from you to
choose from.
Mark Roddy
On Tue, Apr 28, 2009 at 6:12 PM, J. J. Farrell wrote:
> Alex Shvedov wrote:
>>>
>>> system PTEs are allocated to map the driver image, the driver image is
>>> copied into memory
>>
>> What kind of memory?
>> ExAllocated by driver loader?
>>
>>> common newb mistake is to use the terms “paged pool” and “non-page
>>> pool” interchangeably with “pageable system memory” and “non-pageable
>>> system memory”
>>
>> Would it be too much to ask you to explain the difference?
>
> Not at all - that’s why he did so in the sentence you snipped which followed
> the sentence you quoted.
>
> “It is often a common newb mistake to assume that the system allocates ALL
> memory from one of the “pools” when in fact, the “pool” is a specific area
> reserved for the dynamic storage of OS data structures.”
>
>> Googling brought up nothing, really - except an obvious fact that
>> [non]pageable pool belongs to [non]pageable memory, like in
>> WDK’s POOL_TYPE description: “… Nonpaged pool, which is
>> nonpageable system memory”.
>>
>> Not much.
>>
>> Is there any way to confirm that this difference that
>> newbies forget about really exisst?
>
> If you are not prepared to believe the experts who know the Windows source
> code intimately, your only option is to find some way to look at the source
> code yourself.
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> Well you could trust the wise old folks who are telling you it really
exists
trusting Google
Trust? It was not and is not a matter of trust.
If you are not prepared to believe the experts who know the Windows
source code intimately
Believe? Well…
I in fact asked “what are memory regions not under mm’s control”.
There is no place for trust or belief here, the table (which
I in fact simply forgot) I googled out provided the answer.
Jic this does not mean that I believe/trust (or love or hate etc.) the
table.
your only option is to find some way to look at the source code yourself.
I did not have this option at the time. As it turns out, google may help,
just
do not look at the ads.
There are three levels of memory management.
Nice, useful (and correct) answer, but to a different question, the one I
did not ask.
Never hurts to have this memo handy anyway.
Well you could use the debugger and ask it which pool an address from
your driver’s code or data segments is in.
As you yourself note, it is not that simple. Besides, imagine that a
particular
driver is in such-n-such seg, what about ntos per se? non-driver mappings?
etc. etc. I would have to trust/believe in order to generalize…
Anyway, Xpaged pool is in fact a proper subset of Xpaged memory,
not that I trust or believe in this, I know (remembered?) now.
I trust/believe that you know the diff.
----- Original Message -----
From: “Peter Wieland”
To: “Windows System Software Devs Interest List”
Sent: Tuesday, April 28, 2009 5:42 PM
Subject: RE: Re:[ntdev] Paged or Non Paged ???
Well you could trust the wise old folks who are telling you it really
exists. That might be more sensible than trusting Google which knows
something about everything but is an expert in nothing (except perhaps
advertising :)). You could also buy a copy of the Windows Internals book
which explains a lot of this.
There are three levels of memory management.
At one level MM manages physical pages of memory. MM keeps track of whether
each physical page is in use, free, zeroed, locked, dirty, etc…
Above that is MM’s management of the virtual address space (VA) -
per-process VA spaces, per-session VA space and the kernel VA space. MM
manages the reservation and release of VA ranges and separately tracks which
physical page is assigned to each virtual page. At this level MM manages
which virtual addresses are being used in which address spaces and can allow
their corresponding physical pages to be brought into or removed from the
process’s working set.
When a physical page is evicted from a working set (either by being freed or
being aged out) that drops its locked count. When the lock count drops to
zero the physical page can be freed. If the page is dirty and backed by a
file then MM will write the contents out to the file before freeing it. If
the VA was evicted and is accessed by the process then a page fault occurs
and MM will allocate a new physical page and read the appropriate contents
back in from the backing file.
The equivalent of this in user-mode is VirtualAlloc, which allows an
application to reserve a chunk of its own VA space and also to back that
chunk by physical memory. Similarly CreateFileMapping allows the process to
allocate a virtual address range which is backed by a file. When MM goes to
make a page in that chunk resident it will load the contents from the
backing file rather than just using a zeroed page (as it would with
VirtualAlloc).
Pool is built on top of this mechanism. It’s the equivalent of the
user-mode heap or the CRT’s malloc function (not the equivalent of a malloc
system call). The pool allocates virtual memory from Mm and then subdivides
that into individual pool allocations for the callers. Pool is just one way
of getting virtual address space, and one of the only options exported by
the kernel, but it is separate from the underlying virtual memory management
layer that it depends on.
-p
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alex Shvedov
Sent: Tuesday, April 28, 2009 1:50 PM
To: Windows System Software Devs Interest List
Subject: Re: Re:[ntdev] Paged or Non Paged ???
> system PTEs are allocated to map the driver image, the driver image is
> copied into memory
What kind of memory?
ExAllocated by driver loader?
> common newb mistake is to use the terms “paged pool” and “non-page
> pool” interchangeably with “pageable system memory” and “non-pageable
> system memory”
Would it be too much to ask you to explain the difference?
Googling brought up nothing, really - except an obvious fact that
[non]pageable pool belongs to [non]pageable memory, like in
WDK’s POOL_TYPE description: “… Nonpaged pool, which is
nonpageable system memory”.
Not much.
Is there any way to confirm that this difference that
newbies forget about really exisst?
I don’t have Mark’s bible at hand…
----- Original Message -----
From: “Scott Noone”
Newsgroups: ntdev
To: “Windows System Software Devs Interest List”
Sent: Tuesday, April 28, 2009 4:24 PM
Subject: Re:[ntdev] Paged or Non Paged ???
> >I think the loader reads the whole driver from .SYS file to the paged
> >pool area and then locks all sections whose names do not start >with
> >“PAGE”.
>
> Don’t forget that drivers can also go in session space.
>
> And AFAIK it’s the other way around, system PTEs are allocated to map the
> driver image, the driver image is copied into memory, and PTEs for PAGE
> sections are marked pageable.
>
>
> -scott
>
> –
> Scott Noone
> Consulting Associate
> OSR Open Systems Resources, Inc.
> http://www.osronline.com
>
>
> “Maxim S. Shatskih” wrote in message
> news:xxxxx@ntdev…
>>When you say that it is “mapped” in to memory…where the “memory” comes
>>from. Doesn’t the loader
>>has to read the .sys file from the disk and bring it in to SOME memory?
>
> I think the loader reads the whole driver from .SYS file to the paged pool
> area and then locks all sections whose names do not start with “PAGE”.
>
> At least you can delete and replace the loaded .SYS (not so with user-mode
> DLLs and EXEs).
>
> –
> Maxim S. Shatskih
> Windows DDK MVP
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer