How does windows find available memory

NTDEV
1

Hi All,
I have gotten into a situation where I need to know how windows finds the
so-called “available memory”, and root cause area of bios that is buggy.

Background:
I have a new x86 based board with 4 core processor, and a 4GB RAM. The BIOS
for it is in house developed.
When installed 32-bit Win7, on this machine, The OS shows only 2.6 GB as
the available memory with or without PAE enabled.
I agree that OS has to get memory from BIOS. But this problem is not seen
with linux, where it reports 3.8GiB.

I understand that some devices(like integrated graphics) might/will steal
some amount(that can be configured in BIOS) of system memory, but in that
case, how come it works differently on different Operating systems.

Inference:
Possibly Windows discovers available memory in a different manner. as
opposed to how linux finds it.

Question:
If there are different ways of getting the available system memory, what
are they? and is there a spec from windows that says what BIOS should
implement for it to discover the available memory?

Thanks,
Madhusudan

2

For Windows, the only task performed by the BIOs is to load NtLdr. Once that is loaded and launched, the BIOs is effectively mapped out and never seen or used again. 32 bit windows only sees 2.6 gig of RAM because that is all it sees. For more than 2.6 gig you have to go to the 64bit platform, but I doubt that will solve your problem. Windows is Windows, and Linux is Linux, and how Linux may or may not do things has no bearing on how Windows may or may not do things. 4GB of RAM in 32 bit Windows, wastes nearly 1.5 Gig of RAM since Windows will never see it nor use it.

32 bit Windows address space can be extended by using Physical Address Extension (PAE). In boot.ini that would be adding the /PAE switch, or using bcdedit /set PAE.

Gary Little
H (952) 223-1349
C (952) 454-4629
xxxxx@comcast.net

On Dec 6, 2011, at 6:28 AM, Madhusudan Narayan wrote:

Hi All,
I have gotten into a situation where I need to know how windows finds the so-called “available memory”, and root cause area of bios that is buggy.

Background:
I have a new x86 based board with 4 core processor, and a 4GB RAM. The BIOS for it is in house developed.
When installed 32-bit Win7, on this machine, The OS shows only 2.6 GB as the available memory with or without PAE enabled.
I agree that OS has to get memory from BIOS. But this problem is not seen with linux, where it reports 3.8GiB.

I understand that some devices(like integrated graphics) might/will steal some amount(that can be configured in BIOS) of system memory, but in that case, how come it works differently on different Operating systems.

Inference:
Possibly Windows discovers available memory in a different manner. as opposed to how linux finds it.

Question:
If there are different ways of getting the available system memory, what are they? and is there a spec from windows that says what BIOS should implement for it to discover the available memory?

Thanks,
Madhusudan

— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

3

On 06/12/2011 14:38, Gary Little wrote:

For Windows, the only task performed by the BIOs is to load NtLdr.
Once that is loaded and launched, the BIOs is effectively mapped out
and never seen or used again. 32 bit windows only sees 2.6 gig of RAM
because that is all it sees. For more than 2.6 gig you have to go to
the 64bit platform, but I doubt that will solve your problem. Windows
is Windows, and Linux is Linux, and how Linux may or may not do things
has no bearing on how Windows may or may not do things. 4GB of RAM in
32 bit Windows, wastes nearly 1.5 Gig of RAM since Windows will never
see it nor use it.

I thought the issue was that Windows will see that there is 4GB RAM in
32-bit Windows but won’t be able to map it all because memory from other
devices such as graphics cards etc. will use the remaining address space?

–
Bruce Cran

4

Gary Little wrote:

Once that is loaded and launched, the BIOs is effectively
mapped out and never seen or used again. 32 bit windows
only sees 2.6 gig of RAM because that is all it sees. For
more than 2.6 gig you have to go to the 64bit platform,
but I doubt that will solve your problem.

Note that the 4 gig limit on x86 Windows is a *marketing* limitation for the consumer-level SKU’s. My Linux box, running 32-bit Linux, has 8GB of RAM and it is all happily consumed by the OS.

I think Server 2003 x86 will also support up to 32GB or 64GB of RAM, or whatever the real limit is.

5

Some details:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778(v=vs.85).aspx

–
Best regards,
Krystian Bigaj

6

Madhusudan Narayan wrote:

I have gotten into a situation where I need to know how windows finds
the so-called “available memory”, and root cause area of bios that is
buggy.

Background:
I have a new x86 based board with 4 core processor, and a 4GB RAM. The
BIOS for it is in house developed.
When installed 32-bit Win7, on this machine, The OS shows only 2.6 GB
as the available memory with or without PAE enabled.
I agree that OS has to get memory from BIOS. But this problem is not
seen with linux, where it reports 3.8GiB.

I understand that some devices(like integrated graphics) might/will
steal some amount(that can be configured in BIOS) of system memory,
but in that case, how come it works differently on different Operating
systems.

The BIOS allocates a chunk of the physical address space below 4GB to
PCI devices. The physical memory that would have been at that spot is
relocated above the 4GB mark.

The 32-bit Windows systems (non-server), for licensing reasons, have an
artificially imposed limitation that prevents them from using any
physical memory above the 4GB boundary, even with PAE. If you run Win7
64-bit, you’ll find your 3.8GB again.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

7

On 06-Dec-2011 14:28, Madhusudan Narayan wrote:
…

I understand that some devices(like integrated graphics) might/will
steal some amount(that can be configured in BIOS) of system memory, but
in that case, how come it works differently on different Operating systems.

So is there an integrated video? have you tried to disable it
and replace with add-on card?

– pa

8

>When installed 32-bit Win7, on this machine, The OS shows only 2.6 GB as the available memory

Try installing Windows Server 32bit there, or any 64bit Windows.

This is a political and marketing thing in 32bit client versions of Windows. They do not support physical addresses > 4GB.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

9

> The 32-bit Windows systems (non-server), for licensing reasons, have an

artificially imposed limitation that prevents them from using any
physical memory above the 4GB boundary, even with PAE.

I think Web edition server SKU is also such.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

10

Maxim S. Shatskih wrote:

This is a political and marketing thing in 32bit client versions of Windows. They do not support physical addresses > 4GB.

The restriction is a bit narrower than that. Those systems will simply
not use any physical memory with a physical address beyond 4GB. The
page tables do support addresses above 4GB, and they do support devices
that live above physical 4GB. An enterprising driver can map in that
abandoned high memory on its own, assuming it can live with the dangers.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

11

I though the deal was that 32-bit Windows was limited to only use physical
DMA addresses below the 4 Gbyte line because there are old buggy drivers for
some hardware that don’t correctly handle 64-bit physical addresses, even if
the OS can. This address limit was set to improve OS compatibility with
these old drivers. Since there is no 32-bit Win 2008 R2 (i.e. Win 7) server,
only 64-bit, this would only apply to Win 7 client. This is a real technical
reason for the limit, not just a marketing reason.

A recent Asus super netbook I bought for $350 (AMD E-350 processor based)
came with 64-bit Windows (using UEFI no less), so am not sure what machines
even ship with 32-bit Windows anymore. Of course my super network can expand
to 8 GB of ram (which I did), and can run VM’s A 32-bit OS would have just
been silly. The bottom end of machines has certainly moved up.

Since you generally don’t (or can’t) use old driver binaries with a newer
Linux release, Linux may not do this compatibility improvement. My
understanding of Linux is it makes no attempt to support a stable driver
binary interface, and drivers need to be rebuild or changed as the Linux
kernel evolves.

Jan

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-485588-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Tuesday, December 06, 2011 9:39 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] How does windows find available memory

Madhusudan Narayan wrote:
>
> I have gotten into a situation where I need to know how windows finds
> the so-called “available memory”, and root cause area of bios that is
> buggy.
>
> Background:
> I have a new x86 based board with 4 core processor, and a 4GB RAM. The
> BIOS for it is in house developed.
> When installed 32-bit Win7, on this machine, The OS shows only 2.6 GB
> as the available memory with or without PAE enabled.
> I agree that OS has to get memory from BIOS. But this problem is not
> seen with linux, where it reports 3.8GiB.
>
> I understand that some devices(like integrated graphics) might/will
> steal some amount(that can be configured in BIOS) of system memory,
> but in that case, how come it works differently on different Operating
> systems.

The BIOS allocates a chunk of the physical address space below 4GB to PCI
devices. The physical memory that would have been at that spot is
relocated
above the 4GB mark.

The 32-bit Windows systems (non-server), for licensing reasons, have an
artificially imposed limitation that prevents them from using any physical
memory above the 4GB boundary, even with PAE. If you run Win7 64-bit,
you’ll find your 3.8GB again.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

12

Jan Bottorff wrote:

I though the deal was that 32-bit Windows was limited to only use physical
DMA addresses below the 4 Gbyte line because there are old buggy drivers for
some hardware that don’t correctly handle 64-bit physical addresses, even if
the OS can. This address limit was set to improve OS compatibility with
these old drivers. Since there is no 32-bit Win 2008 R2 (i.e. Win 7) server,
only 64-bit, this would only apply to Win 7 client. This is a real technical
reason for the limit, not just a marketing reason.

I think you are being overly generous. The DMA abstraction has always
provided a way for drivers to report whether their hardware supports
64-bit addressing or not. The Windows PHYSICAL_ADDRESS structure has
always been a 64-bit value. This is not a new problem, by any stretch
of the imagination.

Plus, this restriction is only in the 32-bit *client* systems. The
complete list of affected systems is XP, Vista 32, and Win 7 32. All
other 21st Century systems, including Server 2003 32, handle it just fine.

No, I’m pretty confident this is a policy limitation rather than a
technical limitation.

Since you generally don’t (or can’t) use old driver binaries with a newer
Linux release, Linux may not do this compatibility improvement.

“Compatibility improvement”? You might just have a bright career ahead
of you as a Microsoft marketing representative. “In order to serve you
better, we have restricted your memory usage.”

My understanding of Linux is it makes no attempt to support a stable driver
binary interface, and drivers need to be rebuild or changed as the Linux
kernel evolves.

True. The issue is really that dynamically-loaded kernel drivers in
Linux are more tightly coupled to the kernel than is the case in
Windows. The concept is less like a DLL than it is like partially
deferred static linking. A kernel module is partially linked to a
specific kernel. If the kernel changes and the exported symbols change
(which they often do), the partial linking is no longer valid.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

13

Yes I know the PHYSICAL_ADDRESS structure allows 64-bit address as does the
Windows DMA architecture.

The blog entry at
http://blogs.technet.com/b/markrussinovich/archive/2008/07/21/3092070.aspx
in the paragraph “Windows Client Memory Limits”, written by Mark
Russinovich, describes just what I said, about older drivers being unstable
with memory above 4 GB. I assume I was probably just passing on the info
from Mark’s posting.

I tend to think of Mark Russinovich as a pretty credible source of Windows
information.

Jan

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-485659-
xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Tuesday, December 06, 2011 12:35 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] How does windows find available memory

Jan Bottorff wrote:
> I though the deal was that 32-bit Windows was limited to only use
> physical DMA addresses below the 4 Gbyte line because there are old
> buggy drivers for some hardware that don’t correctly handle 64-bit
> physical addresses, even if the OS can. This address limit was set to
> improve OS compatibility with these old drivers. Since there is no
> 32-bit Win 2008 R2 (i.e. Win 7) server, only 64-bit, this would only
> apply to Win 7 client. This is a real technical reason for the limit,
not just a
marketing reason.

I think you are being overly generous. The DMA abstraction has always
provided a way for drivers to report whether their hardware supports
64-bit
addressing or not. The Windows PHYSICAL_ADDRESS structure has always
been a 64-bit value. This is not a new problem, by any stretch of the
imagination.

Plus, this restriction is only in the 32-bit *client* systems. The
complete list
of affected systems is XP, Vista 32, and Win 7 32. All other 21st Century
systems, including Server 2003 32, handle it just fine.

No, I’m pretty confident this is a policy limitation rather than a
technical
limitation.

> Since you generally don’t (or can’t) use old driver binaries with a
> newer Linux release, Linux may not do this compatibility improvement.

“Compatibility improvement”? You might just have a bright career ahead of
you as a Microsoft marketing representative. “In order to serve you
better,
we have restricted your memory usage.”

> My understanding of Linux is it makes no attempt to support a stable
> driver binary interface, and drivers need to be rebuild or changed as
> the Linux kernel evolves.

True. The issue is really that dynamically-loaded kernel drivers in Linux
are
more tightly coupled to the kernel than is the case in Windows. The
concept
is less like a DLL than it is like partially deferred static linking. A
kernel module
is partially linked to a specific kernel. If the kernel changes and the
exported
symbols change (which they often do), the partial linking is no longer
valid.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

14

> Since you generally don’t (or can’t) use old driver binaries with a newer Linux release, Linux may not

do this compatibility improvement. My understanding of Linux is it makes no attempt to support a stable driver >binary interface, and drivers need to be rebuild or changed as the Linux kernel evolves.

Well, this is just a question of policy that is meant to discourage the distribution of driver binary images, and, instead, encourage distribution of source files that are meant to be built against the currently running kernel as a part of installation process. You still may choose to distribute driver binaries, but, by doing so, you dramatically limit availability of kernel exports that you may rely upon

Therefore, “compatibility improvement” here lies with the use of macros that are meant to relieve you from the necessity of directly accessing the fields of certain kernel structures - unlike kernel structures, these macros are meant to be very stable, so that properly-written driver does not need any modifications to its source even if kernel changes significantly…

Anton Bassov

15

Windows will use all memory reported through the INT 15 E820 interface (see
http://acpi.info.) Win7 32-bit versions (or any other 32-bit client
Windows) will ignore any memory above 4GB for greater driver compatibility.

Jake Oshins
(former ACPI guy)
Windows Kernel Group

This post implies no warranties and confers no rights.

“Madhusudan Narayan” wrote in message
news:xxxxx@ntdev…
Hi All,
I have gotten into a situation where I need to know how windows finds the
so-called “available memory”, and root cause area of bios that is buggy.

Background:
I have a new x86 based board with 4 core processor, and a 4GB RAM. The BIOS
for it is in house developed.
When installed 32-bit Win7, on this machine, The OS shows only 2.6 GB as the
available memory with or without PAE enabled.
I agree that OS has to get memory from BIOS. But this problem is not seen
with linux, where it reports 3.8GiB.

I understand that some devices(like integrated graphics) might/will steal
some amount(that can be configured in BIOS) of system memory, but in that
case, how come it works differently on different Operating systems.

Inference:
Possibly Windows discovers available memory in a different manner. as
opposed to how linux finds it.

Question:
If there are different ways of getting the available system memory, what are
they? and is there a spec from windows that says what BIOS should implement
for it to discover the available memory?

Thanks,
Madhusudan

16

Jake,
Thanks for the info.

Win7 32-bit versions (or any other 32-bit >client Windows) will ignore any
memory above 4GB for greater driver compatibility.

Is that true even if PAE is enabled?

Regards,
Madhusudan

On Wed, Dec 7, 2011 at 10:46 AM, Jake Oshins wrote:

> Windows will use all memory reported through the INT 15 E820 interface
> (see http://acpi.info.) Win7 32-bit versions (or any other 32-bit client
> Windows) will ignore any memory above 4GB for greater driver compatibility.
>
> Jake Oshins
> (former ACPI guy)
> Windows Kernel Group
>
> This post implies no warranties and confers no rights.
>
> ------------------------------ ------------------------------ –
> “Madhusudan Narayan” wrote in message
> news:xxxxx@ntdev…
> Hi All,
>
> I have gotten into a situation where I need to know how windows finds the
> so-called “available memory”, and root cause area of bios that is buggy.
>
> Background:
> I have a new x86 based board with 4 core processor, and a 4GB RAM. The
> BIOS for it is in house developed.
> When installed 32-bit Win7, on this machine, The OS shows only 2.6 GB as
> the available memory with or without PAE enabled.
> I agree that OS has to get memory from BIOS. But this problem is not seen
> with linux, where it reports 3.8GiB.
>
> I understand that some devices(like integrated graphics) might/will steal
> some amount(that can be configured in BIOS) of system memory, but in that
> case, how come it works differently on different Operating systems.
>
> Inference:
> Possibly Windows discovers available memory in a different manner. as
> opposed to how linux finds it.
>
> Question:
> If there are different ways of getting the available system memory, what
> are they? and is there a spec from windows that says what BIOS should
> implement for it to discover the available memory?
>
>
>
> Thanks,
> Madhusudan
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.**cfm?name=ListServerhttp:
></http:>

17

Yes. Busted drivers doesn’t become suddenly not-busted when you enable PAE.

Jake Oshins
Windows Kernel Group

This post implies no warranties and confers no rights.

“Madhusudan Narayan” wrote in message
news:xxxxx@ntdev…
Jake,
Thanks for the info.
>Win7 32-bit versions (or any other 32-bit >client Windows) will ignore any
>memory above 4GB for greater driver compatibility.

Is that true even if PAE is enabled?

Regards,
Madhusudan

On Wed, Dec 7, 2011 at 10:46 AM, Jake Oshins
wrote:
Windows will use all memory reported through the INT 15 E820 interface (see
http://acpi.info.) Win7 32-bit versions (or any other 32-bit client
Windows) will ignore any memory above 4GB for greater driver compatibility.

Jake Oshins
(former ACPI guy)
Windows Kernel Group

This post implies no warranties and confers no rights.

--------------------------------------------------------------
“Madhusudan Narayan” wrote in message
news:xxxxx@ntdev…
Hi All,

I have gotten into a situation where I need to know how windows finds the
so-called “available memory”, and root cause area of bios that is buggy.

Background:
I have a new x86 based board with 4 core processor, and a 4GB RAM. The BIOS
for it is in house developed.
When installed 32-bit Win7, on this machine, The OS shows only 2.6 GB as the
available memory with or without PAE enabled.
I agree that OS has to get memory from BIOS. But this problem is not seen
with linux, where it reports 3.8GiB.

I understand that some devices(like integrated graphics) might/will steal
some amount(that can be configured in BIOS) of system memory, but in that
case, how come it works differently on different Operating systems.

Inference:
Possibly Windows discovers available memory in a different manner. as
opposed to how linux finds it.

Question:
If there are different ways of getting the available system memory, what are
they? and is there a spec from windows that says what BIOS should implement
for it to discover the available memory?

Thanks,
Madhusudan

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

18

>I though the deal was that 32-bit Windows was limited to only use physical

DMA addresses below the 4 Gbyte line because there are old buggy drivers

Yes, but the political/business reasons were added to this, and thus the DMA limitation was executed as “no RAM at all above 4GB physical” limitation :slight_smile:

A recent Asus super netbook I bought for $350 (AMD E-350 processor based)
came with 64-bit Windows (using UEFI no less), so am not sure what machines
even ship with 32-bit Windows anymore.

All Atom/Poulsbo platforms, including the things like most Eee PCs (like the “Asus Eee PC” model which is Atom-based and is still for sale), and also these $150 book-size fan-less platform boxes.

With the current fashion of SystemOnChip, these (and similar) will be numerous.

understanding of Linux is it makes no attempt to support a stable driver
binary interface

Yes.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

19

> Plus, this restriction is only in the 32-bit *client* systems. The

complete list of affected systems is XP, Vista 32, and Win 7 32. All
other 21st Century systems, including Server 2003 32, handle it just fine.

I think Server Web Edition is also such.

True. The issue is really that dynamically-loaded kernel drivers in
Linux are more tightly coupled to the kernel than is the case in
Windows. The concept is less like a DLL than it is like partially
deferred static linking. A kernel module is partially linked to a
specific kernel. If the kernel changes and the exported symbols change
(which they often do)

IIRC Linux kernel exports all non-static functions, there are no such things like .DEF file or __declspec(dllexport) there.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

20

>Is that true even if PAE is enabled?

Yes.

On these OSes, PAE only gives you a) NX bit b) ability for the OS to map BARs above 4GB, if the device supports such a thing.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com