Is there a way to determine whether a given memory address is nonpaged or paged? I have a recurring need to take a pointer that has been handed to me and determine whether it is pageable.
On a similar note, does anyone know the technical difference between non-pageable memory and pageable memory that has been locked? I read an old post that said that locked memory can still cause page faults, because the pageable virtual address may refer to page tables that are pageable. I have been unable to find documentation that confirms that some page tables are nonpageable and others are pageable, although it makes sense that UM page tables have no need to be locked in memory.
Is there a way to determine whether a given memory address is nonpaged or paged?
Intrinsically paged or non-paged? No. There is not.
does anyone know the technical difference between non-pageable memory and pageable memory that has been locked?
Aside from the fact that “pageable memory that has been locked” could at some point in the future become UNlocked, there is no difference.
I have a recurring need to take a pointer that has been handed to me and determine whether it is pageable.
It’s a very difficult question to answer, right? Consider, you could have a page that’s locked (or “pinned” as they say on some other operating systems) when you check on it… and then 1ms later, the page could be unlocked.
There’s a function in NTDDK.H that’s often misused in exactly the situation you’re describing. That function is MmIsAddressValid – It’ll return TRUE if the address provided can be referenced without causing a pagefault. The problem is that the function should have been named MmWasAddressValidWhenIChecked because there’s no guarantee that when you return from the function, the memory pointed to is STILL locked.
It’s just the way of the world…
Peter
There’s a function in NTDDK.H that’s often misused in exactly the situation you’re describing. That function is MmIsAddressValid – It’ll return TRUE if the address provided can be referenced without causing a pagefault. The problem is that the function should have been named MmWasAddressValidWhenIChecked because there’s no guarantee that when you return from the function, the memory pointed to is STILL locked.
Does MmIsAddressValid() check whether the page is currently mapped (i.e. PTE.Valid is 1), or whether the memory is locked? I would settle for a check to see if a page is currently pinned. I can live with the possibility that this could change.
Is it a true statement that memory from the nonpaged pool is no different than memory from the paged pool that has been locked (other than the intended duration of the lock)?
Let me describe my latest reason for wanting this so the question has better definition. I am trying to find a way to map the memory of my driver into physical memory regions, skipping any ranges that are pageable.
Does MmIsAddressValid() check whether the page is currently mapped
Well, a quick look at the docs will show you that it takes a kernel virtual address as input… so, yeah… the memory has to be mapped.
Is it a true statement that memory from the nonpaged pool is no different than memory from the paged pool that has been locked
For some value of “no different”, yes.
I am trying to find a way to map the memory of my driver into physical memory regions, skipping any ranges that are pageable
Hmmmm… that sentence kinda hurts my head. You want to “map the memory of [your] driver” – OK… that’s the code and data comprising your driver – “into physical memory regions” – Huh? You can’t “map” your driver into physical memory. You can PUT your driver in physical memory that’s MAPPED by a virtual address space.
Can you maaaaybeee step back a couple of paces, and explain your larger goal? Mayhaps there’s a way we can help you… or I can at least keep posting and stall long enough for Mr. Roberts to get email access back and he can give you an answer 
Peter
does anyone know the technical difference between non-pageable memory and pageable memory that has been locked?
Pageable memory does not really get “locked” in a conventional sense (i.e. in a VirtualLock() -like fashion) as far as KM drivers are concerned.
You are not supposed to do it this way - instead, a more elaborate one is required
I read an old post that said that locked memory can still cause page faults, because the pageable virtual address may refer
to page tables that are pageable.
This is exactly the reason why KM components don’t “lock” memory the way their usermode components do - it would be simply unsafe.
If memory is pageable and you want to ensure that accessing it does not cause a page fault, the only way for dealing with it is
- Build an MDL for the target pageable range
- Probe and lock pages in MDL in order to ensure that they are all brought into RAM
- Map the locked MDL into the kernel address space. The address that MmMapLockedPagesSpecifyCache() returns is going to be different from the original one. Accessing the target buffer by this address is guaranteed not to cause the page faults. If you try to access it by the original address you are just looking for trouble. It may take you quite a while to get into one, but you rcode is going to be potentially unsafe
I am trying to find a way to map the memory of my driver into physical memory regions, skipping any ranges that are pageable
OMG…
I think that, first of all, you need to learn a bit about the general KM concepts. The above statement immediately reveals that you are not yet in a position to write drivers. To begin with, just grab a copy of “Windows Internals” and read it from A to Z…
Anton Bassov
Does MmIsAddressValid() check whether the page is currently mapped
Well, a quick look at the docs will show you that it takes a kernel virtual address as input… so, yeah… the memory has to be mapped.
Same question stated differently:
Given that Mm and/or CPU has some means of determining whether a given page is eligible for swapping. My first thought was a bit in the PTE, but it isn’t there, and besides, it doesn’t seem practical to search page tables to find out if a physical page can be swapped. Regardless of where it is, nonpaged pool has this set from the time it is set aside for the memory manager, and pinning a page sets this “bit”. This is the piece of information I am wanting to read/test.
I believe that MmIsAddressValid() likely looks up the PTE and checks bit 0 (Valid). Thus a page that is eligible for swapping might still return true.
I am trying to find a way to map the memory of my driver into physical memory regions, skipping any ranges that are pageable
Hmmmm… that sentence kinda hurts my head. You want to “map the memory of [your] driver” – OK… that’s the code and data comprising your driver – “into physical memory regions” – Huh? You can’t “map” your driver into physical memory. You can PUT your driver in physical memory that’s MAPPED by a virtual address space.
Can you maaaaybeee step back a couple of paces, and explain your larger goal? Mayhaps there’s a way we can help you… or I can at least keep posting and stall long enough for Mr. Roberts to get email access back and he can give you an answer
(takes two steps backward)
How about here? 
Darn, now I can’t reach my drink. 
When we code a driver, we allow for paged and nonpaged code segments. Using C++ is a little more nebulous than straight C. I have the base Va of my driver and the image size, and I would like to dump the layout of that code space in terms of pageable and nonpageable memory, as well as visualizing the physical layout of any non-pageable sections.
_Ron
I think that, first of all, you need to learn a bit about the general KM concepts. The above statement immediately reveals that you are not yet in a position to write drivers. To begin with, just grab a copy of “Windows Internals” and read it from A to Z…
You misunderstood my question. I never mentioned UM or VirtualLock(). I’m talking about ProbeAndLockPages() type of locking, as well as memory acquired through ExAllocatePoolWithTag().
What I am trying to find is a way to determine if a given page is currently locked/pinned. I know a method exists, or the memory manager could not function. I’m hoping there is a way my driver can test this for a given virtual address
_Ron
we allow for paged and nonpaged code segments
Just don’t specify that anything is pageable, it’ll all be non-pageable by default, and you’ll be good to go.
Problem solved?
Yes, I’m serious.
Peter
You misunderstood my question. I never mentioned UM or VirtualLock().
Sorry…
What I am trying to find is a way to determine if a given page is currently locked/pinned. I know a method exists,
or the memory manager could not function.
Well, the fact that a mechanism exists does not necessarily imply that it is available to “outsiders” - after all, this is not Linux that allows drivers to directly access all the fields of the PAGE structure. AFAIK, the functionality you are asking about is internal to the MM under Windows, and not exposed to anyone outside of it…
I’m hoping there is a way my driver can test this for a given virtual address
AFAIK, it can not, at least not by any “official” means…
The best thing it can do is to call MmIsAddressValid() , but even this function is not really reliable, because, as it has been already mentioned on this thread, the situation may change by the time you make an actual use of its return value. According to Doron, this function should have never been exposed to anyone outside MM, in the first place.
Anton Bassov
Just don’t specify that anything is pageable, it’ll all be non-pageable by default, and you’ll be good to go.
Well, I am afraid I may experience “the full power of the new platform’s troll-management functionality” ( BTW, you gave me a free pass for one “inflammatory” post, did not you), but I think that certain NTDEV members are going to be VERY upset by such a suggestion, taking into consideration its author. If it was suggested by someone else we would, probably, have a chance to see some “exciting” reaction (probably, even in capitals), but in this particular case it is made by " the ultimate authority figure", and, hence, cannot be questioned…
Anton Bassov
The best thing it can do is to call MmIsAddressValid() , but even this function is not really reliable, because, as it has been already mentioned on this thread, the situation may change by the time you make an actual use of its return value. According to Doron, this function should have never been exposed to anyone outside MM, in the first place.
That’s the conclusion I have been coming to. I was just hoping. Thanks for your help.
Do you know, does MmIsAddressValid() just tell you whether the page is currently mapped to a physical page (i.e. the PTE Valid bit)?
_Ron
Just don’t specify that anything is pageable, it’ll all be non-pageable by default, and you’ll be good to go.
Problem solved?
Yes, I’m serious.
That is my plan B for this case. I have other cases where I would like to know if a piece of memory is pageable or not though.
For example, and I guess I should have led with this since it drives me nuts that I can’t do it, annotations could be much more descriptive if we could read this simple bit. Possibly a majority of my functions, as well as a great many of those in the Windows kernel API, have to allow DISPATCH_LEVEL in the annotations, but resort to warning the user in the documentation that if this or this or this, then it is <= APC_LEVEL. If there were a low cost check for pageable memory, we could use:
When(KeIsMemoryPageable(Buffer), IRQL_requires_max(APC_LEVEL))
_Ron
Do you know, does MmIsAddressValid() just tell you whether the page is currently mapped to a physical page (i.e. the PTE Valid bit)?
IIRC, it just returns the state of the ‘Present’ bit of the PTE…
BTW, if you think carefully about MmIsAddressValid() you are (hopefully) going to realise that your original question about the “locked” attribute of a page cannot have any reliable positive answer for exactly the same reason that MmIsAddressValid() cannot be reliably used by anyone outside MM. Even if you find a way to read this attribute, the situation may change by the time you try to make any actual use of it, which is totally out of your control…
Anton Bassov
@anton_bassov said:
If it was suggested by someone else we would, probably, have a chance to see some “exciting” reaction (probably, even in capitals), but in this particular case it is made by " the ultimate authority figure", and, hence, cannot be questioned…
Just like King Edward in “A Knight’s Tale”. That’s quite a compliment, Peter.
_Ron
The point about annotations is that they’re entirely static, not dynamic. So we could never have an annotation that says “if the buffer is pinned, the the IRQL can be DISPATCH_LEVEL.”
I would like to know if a piece of memory is pageable or not
There is no such API available to Windows driver writers. Full stop. We’ve been writing drivers for Windows for almost 25 years now, and we haven’t felt the need for this API yet. If you’re feeling like you need it, I’d suggest perhaps you’re thinking about things in a “non-Windows way” architecturally.
Peter
but I think that certain NTDEV members are going to be VERY upset by such a >suggestion
I am a firm believer in the saying “In device drivers, all pool is non-paged and all locks are spin” — I need an actual reason to make my device driver code or data pageable. And “cuz it could run on a physical system in 100 VMs” is not a sufficient reason to me.
Peter
The point about annotations is that they’re entirely static, not dynamic. So we could never have an annotation that says “if the buffer is pinned, the the IRQL can be DISPATCH_LEVEL.”
So annotations are used only by the static analyzer, not by DV to check values at runtime? I suppose that makes sense, I just never realized it.
_Ron
annotations are used only by the static analyze
By Static Driver Verifier (SDV) and by VS Code Analysis (CA), yes.
Peter
annotations are used only by the static analyze
By Static Driver Verifier (SDV) and by VS Code Analysis (CA), yes.
I suppose that deep down I had to have known that. The annotation macros all expand to nothingness, so it would be unreasonable to think that they would modify the code to check things at runtime.
Can you think of no advantages to knowing if a pointer handed to you is nonpaged? What if you have a pointer and you want an mdl to describe it, but you don’t know if it is pageable? If it is nonpaged pool or locked, you can call MmBuildMdlForNonPagedPool(), if not you have to call MmProbeAndLockPages().
I don’t know what the code looks like, but “build mdl knowing it is non-paged” sounds more efficient than “probing and locking + adding a reference count that needs to be unlocked + building an mdl”. What would you do in this situation? Just call MmProbeAndLockPages() to be safe? Both functions are void, so it’s not like you can call MmBuildMdlForNonPagedPool() and check for an error code.
MmProbeAndLockPages() can be called at DISPATCH_LEVEL if the memory is nonpaged, else it has to be called at <= APC_LEVEL. Knowing if a pointer is pinned or not can therefore make the difference between having to pend until safe or not.
MmBuildMdlForNonPagedPool() only works for locked pages. The documentation doesn’t say what happens if you call it for a range that includes a non-pinned page, but I imagine it is a bugcheck. So knowing if the pointer includes paged memory can avoid a bugcheck.
If I’m way off base, just say so. I’ve got less than a year of kernel development under my belt, and most of what I know comes from you all, and the MS documentation that perpetually falls short. You are the definitive source for information, I’m just looking for something that feels satisfying.
_Ron
On Sep 20, 2018, at 9:20 PM, rstruempf wrote:
>
>
>>> annotations are used only by the static analyze
>>
>> By Static Driver Verifier (SDV) and by VS Code Analysis (CA), yes.
>
> Can you think of no advantages to knowing if a pointer handed to you is nonpaged? What if you have a pointer and you want an mdl to describe it, but you don’t know if it is pageable? If it is nonpaged pool or locked, you can call MmBuildMdlForNonPagedPool(), if not you have to call MmProbeAndLockPages().
>
> I don’t know what the code looks like, but “build mdl knowing it is non-paged” sounds more efficient than “probing and locking + adding a reference count that needs to be unlocked + building an mdl”. What would you do in this situation? Just call MmProbeAndLockPages() to be safe? Both functions are void, so it’s not like you can call MmBuildMdlForNonPagedPool() and check for an error code.
Yes, if you don’t know, then you assume the pointer is paged and call MmProbeAndLockPages.
> MmProbeAndLockPages() can be called at DISPATCH_LEVEL if the memory is nonpaged, else it has to be called at <= APC_LEVEL. Knowing if a pointer is pinned or not can therefore make the difference between having to pend until safe or not.
Yes. If you have the potential to work with paged data, then you cannot run at DISPATCH_LEVEL. It’s just that simple.
In real life, this is not a problem. There are almost never any occasions where a single piece of code will sometimes received paged and sometimes received non-paged address. You virtually always get one or the other, and you had better know which one it is.
—
Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.