Hello everyone,
I am writing a small (dummy) driver which does not sit in any stack (like TDI, NDIS, storage, etc.).
The machine has 8 GB of RAM and the PAE is enabled on it.
I need to access physical memory above 4 GB.
What is the correct way to get and manipulate it?
Thank you.
Tushar.
There is not special API as there is in user mode.
http://www.microsoft.com/whdc/system/platform/server/PAE/PAEdrv.mspx
Good luck,
mm
xxxxx@gmail.com wrote:
Hello everyone,
I am writing a small (dummy) driver which does not sit in any stack (like TDI, NDIS, storage, etc.).
The machine has 8 GB of RAM and the PAE is enabled on it.
I need to access physical memory above 4 GB.
What is the correct way to get and manipulate it?
Thank you.
Tushar.
Thank you for your reply.
I read that link but could not get much out of it. 
Can i use MmAllocatePagesForMdl or something like it?
Thank you.
Tushar.
MmAllocateContiguousMemorySpecifyCache or MmAllocatePagesForMdlEx. What is
your reason for requiring the physical memory to be above 4gb?
On Feb 5, 2008 9:56 AM, wrote:
> Thank you for your reply.
>
> I read that link but could not get much out of it.
>
> Can i use MmAllocatePagesForMdl or something like it?
>
> Thank you.
> Tushar.
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
–
Mark Roddy
Actually i want to implement a ramdisk that exposes a disk of 5 GB.
My machine has a physical RAM of 16 GB.
I read something about PAE and that physical memory above 4 GB can be used in this case.
But how to do it exactly? Can somebody please tell?
-tushar
Allocate the memory in user mode and share it with your driver. The ramdisk
project has been discussed here repeatedly. In my opinion it is a horrible
idea and not such a good product either. The system does a pretty good job
at caching what needs to be cached on multiple levels as it is.
On Feb 5, 2008 11:15 AM, wrote:
> Actually i want to implement a ramdisk that exposes a disk of 5 GB.
> My machine has a physical RAM of 16 GB.
> I read something about PAE and that physical memory above 4 GB can be used
> in this case.
>
> But how to do it exactly? Can somebody please tell?
>
> -tushar
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
–
Mark Roddy
Actually the objective is to use the physical memory above 4 GB which mostly remains unused in 32 bit OS.
How can i allocate memory of 5 GB in user space? Can it be done by the usual memory allocation functions?
If no then how should i do it?
-tushar
And if i have to do it from kernel mode, should i use MmAllocatePagesForMdl(Ex)?
How should i manipulate the mdl returned by MmAllocatePagesForMdl(Ex) & access the memory?
-tushar
xxxxx@gmail.com wrote:
Actually the objective is to use the physical memory above 4 GB which mostly remains unused in 32 bit OS.
How can i allocate memory of 5 GB in user space? Can it be done by the usual memory allocation functions?
If no then how should i do it?
The operating system simply refuses to see it, so none of the memory
allocation functions will get it. You have to treat it as device memory
and map it in as if it were sitting on a PCI device.
That means you have to be confident of where the memory is actually
located in physical address space. I’m not sure that you can assume
that it starts at 4GB and runs continuously upward from there.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
Tim,
If he uses the registry memory map, he will know the highest physical
address that the system sees. He can use that to get a base address, the
challenge is there is no way to ensure how much memory is there, so the
driver is running on a leap of faith. I have done this with an embedded
environment for a client before who needs ways to access the memory.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply
“Tim Roberts” wrote in message news:xxxxx@ntdev…
> xxxxx@gmail.com wrote:
>> Actually the objective is to use the physical memory above 4 GB which
>> mostly remains unused in 32 bit OS.
>> How can i allocate memory of 5 GB in user space? Can it be done by the
>> usual memory allocation functions?
>> If no then how should i do it?
>>
>
> The operating system simply refuses to see it, so none of the memory
> allocation functions will get it. You have to treat it as device memory
> and map it in as if it were sitting on a PCI device.
>
> That means you have to be confident of where the memory is actually
> located in physical address space. I’m not sure that you can assume that
> it starts at 4GB and runs continuously upward from there.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> How can i allocate memory of 5 GB in user space? Can it be done by the
usual memory allocation functions?
If no then how should i do it?
You do it with VirtualAlloc using some of the less common allocation flags.
And you have to access it in special ways, essentially doing your own page
management. Look at the stuff on PAE and the access functions in MSDN, it
is quite well documented. You are still limited by what portion of the
memory range you can have mapped into your address space. And there are
some nasty little features, such as being unable to share the memory mapping
between processes. For something like a ramdrive that could be accessed by
anyone that might be close to a killing feature.
BTW, its my belief (possibly incorrect) that the space above the 4G fence is
NOT “essentially unused” on a 32 bit PAE system. There are some management
nasties with the space due to addressing, but in general the space gets used
pretty much like memory elsewhere. Programs get loaded there and executed
from there. It is just that any one program cannot access more than a 4GB
address space without having to jump through hoops. But just because the
program address space is limited to 4GB of virtual address, there is no
reason to assume that all (or any) of that virtual address space lives below
a physical 4GB boundary in the machine.
Loren
Loren Wilton wrote:
BTW, its my belief (possibly incorrect) that the space above the 4G
fence is NOT “essentially unused” on a 32 bit PAE system. There are
some management nasties with the space due to addressing, but in
general the space gets used pretty much like memory elsewhere.
Programs get loaded there and executed from there. It is just that
any one program cannot access more than a 4GB address space without
having to jump through hoops. But just because the program address
space is limited to 4GB of virtual address, there is no reason to
assume that all (or any) of that virtual address space lives below a
physical 4GB boundary in the machine.
On Windows 2000, this may have been the case. On Windows XP, it is
definitely not the case. Microsoft representatives and the Microsoft
documentation clearly says that the client systems (i.e., XP) will
simply refuse to acknowledge any physical memory located above the 4GB mark.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
This doesn’t address the client 4GB issue, but he might be able to get
the physical memory information from SMBIOS. I’ve used it for a similar,
less stringent purpose once, and I really can’t say that I know how
accurate this information is going to be from system to system, if at all.
Good luck,
mm
Don Burn wrote:
Tim,
If he uses the registry memory map, he will know the highest physical
address that the system sees. He can use that to get a base address, the
challenge is there is no way to ensure how much memory is there, so the
driver is running on a leap of faith. I have done this with an embedded
environment for a client before who needs ways to access the memory.
The OP does not specify XP or Vista desktop 32 bit os releases, as far as I
can tell. The OS supports much more than 4gb of physical memory on 32 bit OS
server releases. If the OP thinks he has a product for all those 8gb desktop
systems running 32 bit XP and Vista, he is chasing a vanishingly small
market.
On Feb 5, 2008 8:06 PM, Tim Roberts wrote:
> Loren Wilton wrote:
> >
> > BTW, its my belief (possibly incorrect) that the space above the 4G
> > fence is NOT “essentially unused” on a 32 bit PAE system. There are
> > some management nasties with the space due to addressing, but in
> > general the space gets used pretty much like memory elsewhere.
> > Programs get loaded there and executed from there. It is just that
> > any one program cannot access more than a 4GB address space without
> > having to jump through hoops. But just because the program address
> > space is limited to 4GB of virtual address, there is no reason to
> > assume that all (or any) of that virtual address space lives below a
> > physical 4GB boundary in the machine.
>
> On Windows 2000, this may have been the case. On Windows XP, it is
> definitely not the case. Microsoft representatives and the Microsoft
> documentation clearly says that the client systems (i.e., XP) will
> simply refuse to acknowledge any physical memory located above the 4GB
> mark.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
–
Mark Roddy
> On Windows 2000, this may have been the case. On Windows XP, it is
definitely not the case. Microsoft representatives and the Microsoft
documentation clearly says that the client systems (i.e., XP) will simply
refuse to acknowledge any physical memory located above the 4GB mark.
Is that true with PAE enabled and a PAE HAL? Or has PAE been removed from
XP? The whole point with PAE was to be able to use up to 32GB of actual
memory by the system. I know W2K and W2K3 server both did this correctly.
Loren
> The whole point with PAE was to be able to use up to 32GB of actual memory by the system.
This is not the only feature that PAE offers. For example, if you want to take advantage of ExecuteDisable bit, you need to enable PAE. Therefore, PAE has its use even on those systems that have less than 4G of RAM…
Concerning your question, I see the reason why 32-bit OS may be made to refuse recognizing memory above 4G despite providing PAE support - probably, this is just an indirect “recommendation” from MSFT to move to 64-bit OS if your machine is able to run the one anyway (OK, I stop now - otherwise, Peter will accuse me of trying to hijack a thread)…
Anton Bassov
Thank you everybody.
So, if i am getting this correctly, following things apply:
I have a few more questions:
Thank you.
Tushar.
On Tue, Feb 05, 2008 at 08:38:49PM -0800, Loren Wilton wrote:
> On Windows 2000, this may have been the case. On Windows XP, it is
> definitely not the case. Microsoft representatives and the Microsoft
> documentation clearly says that the client systems (i.e., XP) will simply
> refuse to acknowledge any physical memory located above the 4GB mark.Is that true with PAE enabled and a PAE HAL?
Yes.
Or has PAE been removed from XP? The whole point with PAE was to be able to
use up to 32GB of actual memory by the system. I know W2K and W2K3 server
both did this correctly.
Tim Roberts, xxxxx@probo.com
Providenza & Boeklheide, Inc.
How can i specifically access physical memory that is above the upper limit of that which is used by Windows.
For example:
There is a machine with 16 GB of physical memory.
Windows 2000 Advanced Server is installed (which will support upto 8 GB of physical memory).
How can i use the physical memory above 8 GB in this case? Is there a foolproof way of doing this?
Thank you.
Tushar.
Tim,
Correct. I know that the Server systems do this correctly, and I thought that W2K also did,
but XP has been modified not to allow this.
What about ExecuteDisable bit??? If they have removed a support for PAE, then this feature is not going to work either. In other words, did they remove support for PAE itself ( i.e. for a PAE-style memory translation scheme), or they just refuse to recognize physical memory above 4G while PAE itself is enabled?
Anton Bassov