How can I obtain the Total & Available Pool Size (Paged and NonPaged)?
Either from Kernel mode or User mode. Thanks.
From user mode, it’s easy: http://msdn.microsoft.com/en-us/library/ms683210(VS.85).aspx
There may (probably) are other ways to do this, which may be better or worse; not sure really.
For kernel mode, I don’t believe that there is an official way to do this.
I’ve never really understood why this has is such a big secret (unless I’m just missing something). I mean, I get why this information could be misused pretty easily, but I don’t that as any worse that what goes on in the way of people trying to get it.
Good luck,
mm
xxxxx@evitechnology.com wrote:
I’ve never really understood why this has is such a big secret (unless I’m just missing something). I mean, I get why this information could be misused pretty easily, but I don’t that as any worse that what goes on in the way of people trying to get it.
Can you think of any use that is not misuse other than diagnostics? And
for that, isn’t usermode access sufficient?
- M
I’ve certainly never had need for this information; maybe others do; I don’t know. Either way, there are plenty of people who do want this information for good, bad or otherwise, and what’s left to them is either using the totally useless MmQuerySystemSize() or rolling their own, both of which are likely more problematic in practice, in my opinion, than what would be posed by the potential for misuse were there a documented API. If someone is hell bent on, say, allocating for their ramdrive to ‘improve performance,’ or whatever, it seems to me that at least giving them accurate information is better than having them try and figure it out on their own, guessing what ‘MmLargeSystem’ means, repeatedly allocating larger and larger blocks (or smaller and smaller) until they get something that they feel is a good value or relying on the occasionally updated kb articles out there that provide guidelines for limits based on OS version.
mm
But any returned information is never accurate unless there was a “query pool size and block all new pool allocations until your next pool allocation”-style monstrousity. And I do not see anyone wishing to try and support such an egregrious hack.
There’s no “big secret”; this sort of bookkeeping data is not a guarantee without locking the system in place while you decide to act on it.
- S
-----Original Message-----
From: xxxxx@evitechnology.com
Sent: Wednesday, November 18, 2009 16:12
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] Get available Kernel Pool space
I’ve certainly never had need for this information; maybe others do; I don’t know. Either way, there are plenty of people who do want this information for good, bad or otherwise, and what’s left to them is either using the totally useless MmQuerySystemSize() or rolling their own, both of which are likely more problematic in practice, in my opinion, than what would be posed by the potential for misuse were there a documented API. If someone is hell bent on, say, allocating for their ramdrive to ‘improve performance,’ or whatever, it seems to me that at least giving them accurate information is better than having them try and figure it out on their own, guessing what ‘MmLargeSystem’ means, repeatedly allocating larger and larger blocks (or smaller and smaller) until they get something that they feel is a good value or relying on the occasionally updated kb articles out there that provide guidelines for limits based on OS version.
mm
—
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Sure, but isn’t documenting that still likely to be ‘better’ for lack of a better word, than having people guess, use hardcoded values or in some cases not considering at all? I’m just speaking practically here, not about whether it’s a good idea or acurate, et. c. As long as people are going to do it, it is what it is, as I see it.
mm
Couple months ago there was similar query, and we tried our best to
explain this !.
I still wonder why some one would be in need of such a hard number -
from a central place?
There are water marks, and there are some other numbers that
statistically give an indication of the situation within certain epoch
of time ( by itself hard to calculate this epoch). And there is that
best effort scenario to pre-allocate and dynamically adjust, as long as
*forward progress* is guaranteed.
I would love to hear one concrete case where such a hard number is
necessary ( during the runtime of a driver etc) and can’t live
without it. May be I fail to understand!
-pro
Skywing wrote:
But any returned information is never accurate unless there was a “query pool size and block all new pool allocations until your next pool allocation”-style monstrousity. And I do not see anyone wishing to try and support such an egregrious hack.
There’s no “big secret”; this sort of bookkeeping data is not a guarantee without locking the system in place while you decide to act on it.
- S
-----Original Message-----
From: xxxxx@evitechnology.com
> Sent: Wednesday, November 18, 2009 16:12
> To: Windows File Systems Devs Interest List
> Subject: RE:[ntfsd] Get available Kernel Pool space
>
>
> I’ve certainly never had need for this information; maybe others do; I don’t know. Either way, there are plenty of people who do want this information for good, bad or otherwise, and what’s left to them is either using the totally useless MmQuerySystemSize() or rolling their own, both of which are likely more problematic in practice, in my opinion, than what would be posed by the potential for misuse were there a documented API. If someone is hell bent on, say, allocating for their ramdrive to ‘improve performance,’ or whatever, it seems to me that at least giving them accurate information is better than having them try and figure it out on their own, guessing what ‘MmLargeSystem’ means, repeatedly allocating larger and larger blocks (or smaller and smaller) until they get something that they feel is a good value or relying on the occasionally updated kb articles out there that provide guidelines for limits based on OS version.
>
> mm
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
>
Yes, I remember that conversation.
What I’m saying is that I don’t think that explaining the problem will stop people from doing it. There really isn’t all that much to explain, either - it’s not all that complicated really. If you accept my contention, then the question is whether what they do to arrive at answer by themselves is better or worse than what they could get from a documented API, were there one, because they’re going to do the same thing with the information, no matter where it comes from. It has to be more accurate than guessing what ‘MmLargeSystem’ means, for example, or using those hardcoded guidelines, and plenty definitely use this information in their design. I mean, there’s no question in my mind that there would obviously be potential for people to get in to trouble with this information - take MmIsAddressValid, for example - but that’s true either way.
I agree that mm does a great job and also that I can’t think of a good use for this information other than diagnostics, which as Malcom said can be done in user mode, but that doesn’t comment on whether other people agree, are stopped by the lack of an api, or whether the results are better or worse. My opinion is that it I don’t see how it would make things worse, if not better.
mm
I understand your point of view. If people relies on this very heavily
without understanding the mechanics then there is a problem in the
design of their product. Also what should we really expect from a
documented API in this case?
UINT or LONGLONG
GetAvailablePageableMemory ( )
{
// Do not trust me, since even before I return the available memory
could be vastly different ???
// With better insight you can use me to generate random number !!!
[…]
return AvailablePageableMemory;
}
OTHER APIS GO HERE …
xxxxx@evitechnology.com wrote:
Yes, I remember that conversation.
What I’m saying is that I don’t think that explaining the problem will stop people from doing it. There really isn’t all that much to explain, either - it’s not all that complicated really. If you accept my contention, then the question is whether what they do to arrive at answer by themselves is better or worse than what they could get from a documented API, were there one, because they’re going to do the same thing with the information, no matter where it comes from. It has to be more accurate than guessing what ‘MmLargeSystem’ means, for example, or using those hardcoded guidelines, and plenty definitely use this information in their design. I mean, there’s no question in my mind that there would obviously be potential for people to get in to trouble with this information - take MmIsAddressValid, for example - but that’s true either way.
I agree that mm does a great job and also that I can’t think of a good use for this information other than diagnostics, which as Malcom said can be done in user mode, but that doesn’t comment on whether other people agree, are stopped by the lack of an api, or whether the results are better or worse. My opinion is that it I don’t see how it would make things worse, if not better.
mm
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>If people relies on this very heavily without understanding the mechanics then there is a problem in >the design of their product.
How are they supposed to understand the mechanics if no one tells them what they are? Unless you have a way to prevent people from doing this, this sounds like a reason to document it to me.
Also what should we really expect from a documented API in this case?
Well, there’s:
MM_SYSTEMSIZE
MmQuerySystemSize( )
{
////
//// return something that means absolutely nothing to anyone and hope for the best
////
return (MmLargeSystem);
}
Regarding this // Do not trust me, since even before I return the available memory could be vastly different ???,’ again, how it could it possibly be worse than what people create on their own? Also, when all is said and done, how is this different in the case of user mode? The information that they get is dated the minute they get it.
mm
(Hello NTFSD… it’s another one of the misplaced NTDEV threads here. We came to haunt your useful technical exchanges, to show you what you’re missing by not checking NTDEV regularly.)
I don’t think anyone is proposing an API to query the available pageable memory.
I BELIEVE the question was related to the size of available paged/non-paged POOL.
VERY different things.
Given that the size of the pools is not static, I’m not even sure that querying its size makes much sense. Would MmGetFreePool(NonPagedPool) return the amount that’s free before the non-paged pool were extended? The amount of non-paged pool space if the non-paged pool were able to grow to its maximum boundary?
ON THE OTHER HAND…
Who NEEDS a guarantee? Saying that one shouldn’t create an API that returns some data, especially a STATISTIC, because the data might CHANGE is on its face silly. Shall we not have “KeQuerySystemTime” because the time will be different when we return (OMFG… it will change a VARIALBE AMOUNT prior to return as well. Maybe we need a “Query System Time and stop time for progressing” so we can query the time properly)??
How about “MmIsAddressValid”?? The address MIGHT be paged when we return, even if we get back TRUE.
C’mon…
Peter
OSR
Not providing API(s) is a good way to preserve the integrity IMO. An
incorrect API does not seem to have a place here.
For the document, what should be there?. May be you have an idea to
document ( but most of us, including you, know that this parameters or
statistic are random).
Examples, (1) Hooking: We tell everyone what is the danger ahead; (2)
Non-paged Allocation: What is the danger ahead (3) Interrupt processing:
Minimize the code to bare minimum, otherwise danger is called for …
(4) Relying on available pool sizes etc: Don’t rely on it, since by the
time you get the statistic it is obsolete with high probability.
Finally for the people creating their own hacks: I don’t see why they
would not understand this. There are moral responsibilities for the
people to tell the community that “Shit I don’t understand what the heck
mm, ken, pro and others are talking about, but here is my explicit
requirement how can I solve?”. There would always be an alternative
solution, if such a solution exist. Otherwise false promise.
-pro
xxxxx@evitechnology.com wrote:
> If people relies on this very heavily without understanding the mechanics then there is a problem in >the design of their product.
>How are they supposed to understand the mechanics if no one tells them what they are? Unless you have a way to prevent people from doing this, this sounds like a reason to document it to me.
> Also what should we really expect from a documented API in this case?
>Well, there’s:
MM_SYSTEMSIZE
MmQuerySystemSize( )
{
////
//// return something that means absolutely nothing to anyone and hope for the best
////
return (MmLargeSystem);
}Regarding this // Do not trust me, since even before I return the available memory could be vastly different ???,’ again, how it could it possibly be worse than what people create on their own? Also, when all is said and done, how is this different in the case of user mode? The information that they get is dated the minute they get it.
mm
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
xxxxx@osr.com wrote:
(Hello NTFSD… it’s another one of the misplaced NTDEV threads here. We came to haunt your useful technical exchanges, to show you what you’re missing by not checking NTDEV regularly.)
I don’t think anyone is proposing an API to query the available pageable memory.
I BELIEVE the question was related to the size of available paged/non-paged POOL.
VERY different things.
Well these are system defined static data, in that case it is okay to
have APIs.
Given that the size of the pools is not static, I’m not even sure that querying its size makes much sense. Would MmGetFreePool(NonPagedPool) return the amount that’s free before the non-paged pool were extended? The amount of non-paged pool space if the non-paged pool were able to grow to its maximum boundary?
ON THE OTHER HAND…
Who NEEDS a guarantee? Saying that one shouldn’t create an API that returns some data, especially a STATISTIC, because the data might CHANGE is on its face silly. Shall we not have “KeQuerySystemTime” because the time will be different when we return (OMFG… it will change a VARIALBE AMOUNT prior to return as well. Maybe we need a “Query System Time and stop time for progressing” so we can query the time properly)??
It is always the usecase. If you need to start a job based on
KeQuerySystemTime(), because for some reason we decided to make job
start right on time, then it is kinda wrong, well of course we have to
define the tolerance here.
How about “MmIsAddressValid”?? The address MIGHT be paged when we return, even if we get back TRUE.
Once again the purpose of its use…
C’mon…
Peter
OSR
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
As you note, both the extent and free bytes in the pools aren’t fixed. This is especially the case on Vista and later on NT32, where various kernel address space consumers will trade-off dynamically for the same regions of kernel address space.
If you would like information such as pool bytes used, they’re certainly available as a diagnostic statistic for user mode.
I am btw surprised to see you defend MmIsAddressValid. There are very nearly zero safe use cases for that outside of the memory manager itself. It’s IMO quite unfortunate that this API got exported at all given that external callers don’t have the right context to use it safely in general – most all of the time I see anyone using it, they’re writing code that’s broken by design and/or trying to do something inherently unsafe. It’s not something that should make its way into production code.
What specific use case were you looking for with this information that you can’t accomplish right now? I think it’s more useful to stick to that for the discussion.
- S
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Wednesday, November 18, 2009 6:43 PM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] Get available Kernel Pool space
(Hello NTFSD… it’s another one of the misplaced NTDEV threads here. We came to haunt your useful technical exchanges, to show you what you’re missing by not checking NTDEV regularly.)
I don’t think anyone is proposing an API to query the available pageable memory.
I BELIEVE the question was related to the size of available paged/non-paged POOL.
VERY different things.
Given that the size of the pools is not static, I’m not even sure that querying its size makes much sense. Would MmGetFreePool(NonPagedPool) return the amount that’s free before the non-paged pool were extended? The amount of non-paged pool space if the non-paged pool were able to grow to its maximum boundary?
ON THE OTHER HAND…
Who NEEDS a guarantee? Saying that one shouldn’t create an API that returns some data, especially a STATISTIC, because the data might CHANGE is on its face silly. Shall we not have “KeQuerySystemTime” because the time will be different when we return (OMFG… it will change a VARIALBE AMOUNT prior to return as well. Maybe we need a “Query System Time and stop time for progressing” so we can query the time properly)??
How about “MmIsAddressValid”?? The address MIGHT be paged when we return, even if we get back TRUE.
C’mon…
Peter
OSR
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Re MmIsAddressValid:
I *didn’t* “defend” MmIsAddress valid.
I agree that it’s a widely misunderstood function. I actually talk about this DDI in my seminars as “bug bait” (my favorite line is that the function should been named “MmWASAddressValidJustALittleBitAgo”) – I don’t agree that it’s QUITE as useless as you seem to say… there are certainly valid use cases outside the Memory Manager.
But, like I said, I think the function is poorly thought-out and widely misinterpreted.
Oh, who knows? It’s not that I think that MmGetAvailablePool(PoolType) would be an important DDI to add to Windows. In fact, I don’t. While I *could* concoct a use case about sizing data structures appropriately depending on the amount of free pool, as I said I don’t think such a function even makes a great deal of sense for a long list of reasons – the pool size is dynamic, the amount of free CONTIGUOUS pool isn’t the same as the amount of free pool, etc, etc, etc.
So, I agree… it’s not useful or even that interesting.
*I* was just reacting to the reason you and Pro appeared to cite that one couldn’t create the DDI because the returned information was subject to change befroe the caller received the information, and to create such a DDI that was useful you’d need to create “get the free pool size and stop allocating.” There may be a lot of reasons such a DDI doesn’t make sense, but that’s not one of them. The OS is in a constant state of flux, and ANYtime you query something it’s subject to immediate change. Like the time.
Peter
OSR