>this feature is not going to work either. In other words, did they remove
support for
PAE itself ( i.e. for a PAE-style memory translation scheme),
No, only for physical addresses not fitting 32bit. PAE itself is OK in XP SP2.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
> No, only for physical addresses not fitting 32bit. PAE itself is OK in XP SP2.
In other words, just one more pain for no reason whatsoever - now one has to worry that his code is going to work under W2K and W2K3 but not under XP…
Anton Bassov
NTKERNELAPI PHYSICAL_MEMORY_RANGE * MmGetPhysicalMemoryRanges (VOID);
struct PHYSICAL_MEMORY_RANGE
{
PHYSICAL_ADDRESS BaseAddress;
LARGE_INTEGER NumberOfBytes;
};
It returns an array of PHYSICAL_MEMORY_RANGE, with the last entry
indicated by BaseAddress==0 && NumberOfBytes==0. What is also not
documented as how you should free the memory that it returns, but on the
systems I’ve used it on, it is done with ExFreePool, however, I would
disassemble MmGetPhysicalMemoryRanges to confirm that it uses
ExAllocatePool.
I’m not sure I understand this question, because you definitely do
not want to map overlapping ranges, which I don’t believe will work
anyway. It sounds like you’re asking if you should commandeer OS
memory, which is definitely not something you wish to do in this fashion.
Yes: /burnmemory=. This is a very good idea to do, because it
will reserve memory from the top down, so you know it will be there (at
least on server models, for your purposes). If you do this, then you
just have to iterate through the PHYSICAL_MEMORY_RANGE * to find the
highest address, and the block you reserved should be after it, although
I suppose that there is nothing explicitly saying that there can not be
additional unclaimed memory between the two.
Good luck,
mm
http://www.jungo.com/support/tech_docs/td129.html
xxxxx@gmail.com wrote:
> Thank you everybody.
> So, if i am getting this correctly, following things apply:
>
> 1. I need to query the upper limit of physical memory that the OS is using by MmGetPhysicalMemoryRanges.
> 2. Then use the upper limit as the base address in the call to MmAllocatePagesForMdl(Ex).
> 3. Then map the mdl back to system virtual address using MmGetSystemAddressForMdlSafe and then use them.
>
> I have a few more questions:
> 1. MmGetPhysicalMemoryRanges i undocumented. How can i confidently/ accurately know the upper limit of memory that is being used by the OS.
> 2. Should i really care how much physical memory is being reserved by OS and map only the physical memory above that limit?
> 3. Can i somehow reduce the maximum physical memory limit that the OS uses? For ex: in a machine with 32 GB of RAM, and Windows 2003 Server Enterprise installed (which supports upto 32 GB of RAM) can i change some settings such that it uses only say 16 GB of RAM?
>
> Thank you.
> Tushar.
>
>How can i use the physical memory above 8 GB in this case?
Is there a foolproof way of doing this?
Sure. What you need here is a helper app that uses AWE and maps the target memory into its address space without doing anything else with it( if you want to use much “extra” RAM, you will have to run several instances of it ). After having mapped memory, your app can send an IOCTL to your driver and provide a list of memory ranges that it has mapped successfully, so that a driver can use them - at this point it will know that ranges are valid and not used by anyone else…
This is a perfectly safe and fully “supported” way of doing things - if, for this or that reason, your app fails to map some physical memory range, the system will politely return an error, rather than abruptly showing you are blue screen if the same error occurs in the kernel mode. …
Anton Bassov
Anton,
If i do it from a user app, then i suppose i have to be always in the context of that app to access tose pages.
Is there a way to do it completely from kernel mode?
Than you.
Tushar
How can i get the TOTAL physical memory available & the physical memory supported by the OS?
Thanks.
Tushar
There isn’t a documented one in the kernel for either, or at least one
that I know of. You can figure out the second from the aforementioned
registry keys or MmGetPhysicalMemoryRanges, making sure to not consider
reserved sections in the case of the former. For the second, the only
thing I know to suggest is looking at SMBIOS.
http://www.dmtf.org/standards/smbios/
Good luck,
mm
xxxxx@gmail.com wrote:
How can i get the TOTAL physical memory available & the physical memory supported by the OS?
Thanks.
Tushar
Thank you mm.
I have two more questions:
Lets take a scenario in which my machine has 16 GB of RAM. Windows 2000 Advanced Server is installed which supports upto 8 GB of physical memory. And the OS is currently using 6 GB of physical memory out of the supported 8 GB.
Suppose i call MmAllocatePagesForMdl and specify a size of 4 GB in it. What will happen?
Will the physical memory above 8 GB be also used or the call will fail?
Are the pages reserved using AWE APIs accessible from kernel mode in ALL process context or only in the context of the process that called the AllocateUserPhysicalPages function?
Thank you.
Tushar
> If i do it from a user app, then i suppose i have to be always in the context of that app
to access tose pages.
NO!!! The above applies only to accessing the user-mode virtual addresses, and you don’t have to be bothered about the virtual addresses here. The only reason why you use an app is to make sure that the PHYSICAL address that your driver maps is, first, valid (i.e. it exists and it is not marked by BIOS as a reserved range), and, second, is not used by anyone (apart from your app that does not actually use it anyway). If either of the above conditions is not met by some certain range, an app will just fail to map a given range into its address space. In other words, this is just trial-and-error validation, and, for the practical purposes, it is much better to do it in the user mode.
Is there a way to do it completely from kernel mode?
No - at least not safely. Consider the scenario when 2 drivers want to use memory above 4G.
Once the system does not provide any mechanism to do it, they both have to rely upon “unsupported” tricks, but none of them is aware of what its “collegue” is doing. Each of them may believe that the target range is exclusively used only by itself, while, in actuality, they share it. I hope there is no need to explain to you the potential consequences…
Anton Bassov
No problem.
I can’t say that I’ve ever tried this, but I would imagine that it
will not use the 8 GB that the OS doesn’t officially support. This is
just guess.
One process only. From MSDN:
‘Virtual address ranges allocated for the AWE are not sharable with
other processes (and therefore not inheritable). In fact, two different
AWE virtual addresses within the same process are not allowed to map the
same physical page. These restrictions provide fast remapping and
cleanup when memory is freed.’
Good luck,
mm
xxxxx@gmail.com wrote:
Thank you mm.
I have two more questions:
Lets take a scenario in which my machine has 16 GB of RAM. Windows 2000 Advanced Server is installed which supports upto 8 GB of physical memory. And the OS is currently using 6 GB of physical memory out of the supported 8 GB.
Suppose i call MmAllocatePagesForMdl and specify a size of 4 GB in it. What will happen?
Will the physical memory above 8 GB be also used or the call will fail?Are the pages reserved using AWE APIs accessible from kernel mode in ALL process context or only in the context of the process that called the AllocateUserPhysicalPages function?
Thank you.
Tushar
Now let me summarize what i have understood:
Again i will take the previous scenario:
my machine has 16 GB of RAM. Windows 2000
Advanced Server is installed which supports upto 8 GB of physical memory. And
the OS is currently using 6 GB of physical memory out of the supported 8 GB.
There is no method by which i can access physical memory above the OS supported 8 GB PURELY in kernel mode (i mean without any user app).
However, the physical memory above the supported 8 GB mark can be mapped in using AWE APIs in user mode. And then passed to kernel mode for use.
Is there any special reason for this behaviour of supporting it from user mode & not from kernel mode?
Thank you.
Tushar
I don’t think AWE is going to let you map the 8GB that server has kept out
of the system. It will let you map and reserve physical address spaces
within the other 8gb that w2k server has permitted, but it also does not
provide an interface for you to specify exactly which physical pages will be
used. Access to the 8gb you want has to use one of the awful other
techniques mentioned, and you will have to deal with the potential conflicts
with other entities doing the same thing.
On Feb 6, 2008 11:26 PM, wrote:
> Now let me summarize what i have understood:
>
> Again i will take the previous scenario:
> my machine has 16 GB of RAM. Windows 2000
> Advanced Server is installed which supports upto 8 GB of physical memory.
> And
> the OS is currently using 6 GB of physical memory out of the supported 8
> GB.
>
>
> 1. There is no method by which i can access physical memory above the OS
> supported 8 GB PURELY in kernel mode (i mean without any user app).
>
> 2. However, the physical memory above the supported 8 GB mark can be
> mapped in using AWE APIs in user mode. And then passed to kernel mode for
> use.
>
> Is there any special reason for this behaviour of supporting it from user
> mode & not from kernel mode?
>
> Thank you.
> Tushar
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
–
Mark Roddy
Mark,
So, then it comes to the same thing.
I can access the physical memory supported by the OS and not any physical address above it.
And this will vary from OS to OS. Right?
So why not do it in kernel mode itself using MmAllocatePagesForMdl instead of using the AWE APIs in user mode and then adding the overhead of user app & driver communication?
Thanks.
Tushar
> Now let me summarize what i have understood:
Again i will take the previous scenario: my machine has 16 GB of RAM. Windows 2000
Advanced Server is installed which supports upto 8 GB of physical memory.
And the OS is currently using 6 GB of physical memory out of the supported 8 GB. 1.
There is no method by which i can access physical memory above the OS supported 8 GB PURELY in kernel mode (i mean without any user app).
- However, the physical memory above the supported 8 GB mark can be
mapped in using AWE APIs in user mode. And then passed to kernel mode for use.
I am afraid you got it wrong. AWE allows you to use memory from 4G up to the OS -defined upper limit, i.e. 8G However, if you want to map memory above the system-defined upper limit,
then AWE is not going to help you in any possible way, because the system just does not recognize this memory- as far the OS is concerned, this memory just does not exist
(or marked as a reserved range by BIOS). Furthermore, MmGetPhysicalMemoryRanges() is, apparently, in exactly the same position as AWE. All suggestions that you have seen on this thread (apart from checking BIOS) assume that we are speaking about the memory that the OS actually recognizes.
If you want to use unrecognized memory, you are totally out of luck - assuming x86 platform, the only mechanism that allows you to query physical ranges is BIOS INT 15, function 0xE820, i.e. the code that may run only in real-address mode (IIRC, BIOS tables don’t give you the precise info about memory ranges). What you have to do here is to write your own bootloader, so that you can query available memory ranges before Windows gets loaded.
Anton Bassov
Thanks so much Anton for removing the confusion. 
Now what do you recommend of the following two things for using the physical memory in driver:
What are the advantages & disadvantages of using one method over other?
Thanks again.
Tushar
Tushar:
Just to be clear, as Anton already said, I don’t believe that either
of these will get you access to top 8GB of memory as you wish.
The AWE API’s, I’m told, will allow you to allocate a large amount
of memory in a single context without having to understand how to do
this in the kernel. I’ve never used them, but, give it a try and see if
it does what you wish, at least as first step.
I don’t believe that MmAllocatePagesForMdl will, in and of itself,
address your issue, as I doubt it will return to you a MDL describing a
buffer of the size you wish, but try it and see what happens. If you
haven’t already, you might wish to take a look at the ‘Allocating and
using large (1GB) buffer in driver’ thread. In it, Mark Roddy talks
about one way to do this involving IoAllocateMdl/MmInitializeMdl.
If this doesn’t work, you will be down to rolling your own MDL, or
doing very unwise things like attacking the problem at the architectural
level. It sounds like you are new to this; if so, you should go in to
this with the understanding that there’s going to be a lot of trial,
error and debugging involved, and on most of this you’re not going to be
on your own, because rolling your own, while very possible, is just not
a common to do, and the later is totally unadvisable.
I can certainly understand your desire to ascertain what it is the best
way to proceed before you begin. That notwithstanding, just give it a
try. In the case of MmAllocatePagesForMdl, you will very quickly be
able to figure out how much it will let you map on your particular system.
Good luck,
mm
xxxxx@gmail.com wrote:
Thanks so much Anton for removing the confusion.
Now what do you recommend of the following two things for using the physical memory in driver:
- Use AWE APIs
2 . Use MmAllocatePagesForMdlWhat are the advantages & disadvantages of using one method over other?
Thanks again.
Tushar
Thank you very much all of you for your help.
I will try out both the methods.
-Tushar
> Now what do you recommend of the following two things for using the physical
memory in driver:
- Use AWE APIs
2 . Use MmAllocatePagesForMdl
This is only for physical memory visible to the OS, but out of kernel pools.
To access the OS-invisible memory, MmMapIoSpace of its physical address is the
only chance. It is not in PFN database, so MmAllocatePagesForMdl and AWE will
not work.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
> To access the OS-invisible memory, MmMapIoSpace of its physical address
is the only chance.
Indeed, MmMapIoSpace() does not seem to care if the physical range itself is valid. For example, once I tried to map BIOS as writeable memory with MmMapIoSpace() (it does not allow you to specify page protection anyway, so that all memory it maps is writeable), and to write to it. Certainly, the operation did not have any effect, but there was no exception…
Anton Bassov
Thanks a lot Maxim.
This means that if somehow i can get the total physical memory present and the physical memory supported by the OS then i can use MmMapIoSpace on that address range.
Right?
Thank you.
Tushar