Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results
Before Posting...
Please check out the Community Guidelines in the Announcements and Administration Category.More Info on Driver Writing and Debugging
The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.
Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/
Get the unallocated memory on 32-bit Windows, 4GB sys
Bob_Kroeter
Member Posts: 65
in NTDEV
32-bit Windows systems with 4GB installed typically do not use all 4GB. Is there a way a driver can utilize the left over memory?
Howdy, Stranger!
| Upcoming OSR Seminars | ||
|---|---|---|
| OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead! | ||
| Internals & Software Drivers | 19-23 June 2023 | Live, Online |
| Writing WDF Drivers | 10-14 July 2023 | Live, Online |
| Kernel Debugging | 16-20 October 2023 | Live, Online |
| Developing Minifilters | 13-17 November 2023 | Live, Online |
Comments
d
tiny phone keyboard + fat thumbs = you do the muth
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Saturday, March 13, 2010 6:47 PM
To: Windows System Software Devs Interest List <[email protected]>
Subject: [ntdev] Get the unallocated memory on 32-bit Windows, 4GB sys
32-bit Windows systems with 4GB installed typically do not use all 4GB. Is there a way a driver can utilize the left over memory?
---
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
d
tiny phone keyboard + fat thumbs = you do the muth
-----Original Message-----
From: [email protected] <[email protected]>
Sent: Monday, March 15, 2010 6:02 PM
To: Windows System Software Devs Interest List <[email protected]>
Subject: RE:[ntdev] Get the unallocated memory on 32-bit Windows, 4GB sys
Since another device driver on the market is doing it, this is not a an acceptable answer to managment so perhaps undocumented techniques are the way forward.
---
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
>Since another device driver on the market is doing it, this is not a an
>acceptable answer to managment so perhaps undocumented techniques are the
>way forward.
The 32-bit versions of Windows XP and Vista are limited by licensing restrictions to using physical memory below the 4GB mark only. Even if it were possible to use that memory in Windows, it is not necessarily legal.
The actual size of the PCI hole depends on your BIOS, but is typically a few dozen megabytes. I guess it's up to you to decide whether you want to risk a licensing violation over 25c worth of RAM, especially since people seem to be quickly migrating to 64-bit systems, where this restriction does not exist.
--
Tim Roberts, [email protected]
Providenza & Boekelheide, Inc.
Tim Roberts, [email protected]
Providenza & Boekelheide, Inc.
undocumented techniques are the magic yellow brick road, to do something
that the Microsoft rep that monitors this list said you can't? How forward
compatible will your "undocumented" techniques be for Win 7.1 or Win 8?
Undocumented techniques are undocumented for a reason --- the code they are
based upon, may go away, without any warning, leaving you and all your
customers holding onto your pipe dream. Not a nice place to be in when all
you had to do was be truthful with mangement in the first place.
How do you know those undocumented techniques are the way to solve the
problem? Perhaps your competitor paid the bucks for the source to build
their code. If so, dude, you're screwed, since "they" can go places you can
only guess at and hope someone "undocuments" the next branch on those rather
pissy colored bricks.
But, Doron had a question you need to answer --- why is that memory so damn
important, and precisely WHAT are you trying to do? There just might be, oh
my god, DOCUMENTED, ways of doing it.
The personal opinion of
Gary G. Little
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of [email protected]
Sent: Monday, March 15, 2010 6:04 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Get the unallocated memory on 32-bit Windows, 4GB sys
Since another device driver on the market is doing it, this is not a an
acceptable answer to managment so perhaps undocumented techniques are the
way forward.
---
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
__________ Information from ESET Smart Security, version of virus signature
database 4947 (20100315) __________
The message was checked by ESET Smart Security.
http://www.eset.com
__________ Information from ESET Smart Security, version of virus signature
database 4947 (20100315) __________
The message was checked by ESET Smart Security.
http://www.eset.com
> Ok, so you would rather lie to management and lead them to believe
that
> undocumented techniques are the magic yellow brick road, to do
something
> that the Microsoft rep that monitors this list said you can't? How
forward
> compatible will your "undocumented" techniques be for Win 7.1 or Win
8?
> Undocumented techniques are undocumented for a reason --- the code
they are
> based upon, may go away, without any warning, leaving you and all your
> customers holding onto your pipe dream. Not a nice place to be in when
all
> you had to do was be truthful with mangement in the first place.
>
> How do you know those undocumented techniques are the way to solve the
> problem? Perhaps your competitor paid the bucks for the source to
build
> their code. If so, dude, you're screwed, since "they" can go places
you can
> only guess at and hope someone "undocuments" the next branch on those
rather
> pissy colored bricks.
Amen!
> But, Doron had a question you need to answer --- why is that memory so
damn
> important, and precisely WHAT are you trying to do? There just might
be, oh
> my god, DOCUMENTED, ways of doing it.
>
I think it's a pretty simple matter of numbers. My laptop has 4GB of
memory installed, yet XP says that it only has 3GB. The top 1GB is
inaccessible because there is 'stuff' in the way, and presumably it has
been remapped above the 4GB mark (or it's just not accessible). So on
the face of it, getting access to that extra 1GB could give you
effectively 33% more memory.
My recommendation to the OP is that if memory is so important to you, go
64 bit. Let your competition have the fun of milking the long tail of
the 32 bit XP market and go do something else.
James
2008 x86 where the license restrictions don't exist. Of course the memory
is still special in some ways, but there are documented ways to use it.
"Tim Roberts" <[email protected]> wrote in message news:[email protected]
> You wrote:
>>
>>Since another device driver on the market is doing it, this is not a an
>>acceptable answer to managment so perhaps undocumented techniques are the
>>way forward.
>
> The 32-bit versions of Windows XP and Vista are limited by licensing
> restrictions to using physical memory below the 4GB mark only. Even if it
> were possible to use that memory in Windows, it is not necessarily legal.
>
> The actual size of the PCI hole depends on your BIOS, but is typically a
> few dozen megabytes. I guess it's up to you to decide whether you want to
> risk a licensing violation over 25c worth of RAM, especially since people
> seem to be quickly migrating to 64-bit systems, where this restriction
> does not exist.
> --
> Tim Roberts, [email protected]
> Providenza & Boekelheide, Inc.
>
It is true that client versions of 32-bit Windows are restricted to 4GB of
physical address space. And you're right, this does appeal to the folks at
Microsoft who want you to upgrade your OS, or use server OSes.
But the fundamental reason we chose to limit 32-bit client Windows to 32-bit
physical addresses was driver compatibility. There are just too many
drivers in the world that are either lying about their capabilities for
performance reasons or just skipping the DMA APIs entirely.
Server OSes don't generally have this problem, as they tend to run mostly on
machines with device drivers that have been through a more rigorous
validation cycle, particularly in the presence of large amounts of RAM.
--
Jake Oshins
Hyper-V I/O Architect
Windows Kernel Group
This post implies no warranties and confers no rights.
--------------------------------------------------------------
"David Craig" <[email protected]> wrote in message news:[email protected]
> There is the obvious solution which Microsoft will love. Just use Server
> 2008 x86 where the license restrictions don't exist. Of course the memory
> is still special in some ways, but there are documented ways to use it.
>
>
> "Tim Roberts" <[email protected]> wrote in message news:[email protected]
>> You wrote:
>>>
>>>Since another device driver on the market is doing it, this is not a an
>>>acceptable answer to managment so perhaps undocumented techniques are the
>>>way forward.
>>
>> The 32-bit versions of Windows XP and Vista are limited by licensing
>> restrictions to using physical memory below the 4GB mark only. Even if
>> it were possible to use that memory in Windows, it is not necessarily
>> legal.
>>
>> The actual size of the PCI hole depends on your BIOS, but is typically a
>> few dozen megabytes. I guess it's up to you to decide whether you want
>> to risk a licensing violation over 25c worth of RAM, especially since
>> people seem to be quickly migrating to 64-bit systems, where this
>> restriction does not exist.
>> --
>> Tim Roberts, [email protected]
>> Providenza & Boekelheide, Inc.
>>
>
installed. I'd say all 4GB systems are shipped with 64 bit OS.
wrote in message news:[email protected]
> This is for a product similar to a ramdisk. The ability to utilize a gig
> of RAM that is sitting there unused is considered an important feature
> because it allow users to get a great deal more out of their system.
>
>
known of people who did this in highly controlled embedded environments,
but even then some of them ran into problems. You say this is ramdisk
like which means you a storing the users data, corruption should be your
first concern, and there is no way to protect against someone else
assuming the memory is available.
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
> -----Original Message-----
> From: [email protected] [mailto:[email protected]]
> Posted At: Tuesday, March 16, 2010 9:50 AM
> Posted To: ntdev
> Conversation: Get the unallocated memory on 32-bit Windows, 4GB sys
> Subject: RE: Get the unallocated memory on 32-bit Windows, 4GB sys
>
> This is for a product similar to a ramdisk. The ability to utilize a
> gig of RAM that is sitting there unused is considered an important
> feature because it allow users to get a great deal more out of their
> system.
>
>
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4949 (20100316) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
clients to be sure to avoid this crap at all costs.
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
> -----Original Message-----
> From: [email protected] [mailto:[email protected]]
> Posted At: Tuesday, March 16, 2010 10:48 AM
> Posted To: ntdev
> Conversation: Get the unallocated memory on 32-bit Windows, 4GB sys
> Subject: RE: Get the unallocated memory on 32-bit Windows, 4GB sys
>
> We do not have the luxury of only serving the OEM market. This is a
> retail product and there are substantial users who would purchase the
> product if it squeezes more out of their systems without upgrades.
> Getting this memory serves a sweet spot in todays market.
>
>
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4949 (20100316) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
Perhaps, but it doesn't appear to be a currently unoccupied 'sweet spot.' There's already at least one product that claims to be able to do this (>4GB & 3.2GB to 4GB), has gotten itself through the logo process, retails for about $60, supports formating as NTFS or any other Windows support FS and claims to be 'patent pending.' Taking all of that at face value, that's a lot of work for $60/copy, especially if you're playing catchup.
mm
> We do not have the luxury of only serving the OEM market. This is a retail product and there are substantial users who would purchase the product if it squeezes more out of their systems without upgrades. Getting this memory serves a sweet spot in todays market.
>
You are using a very unusual definition of the word "substantial". The
number of users who (1) have 4GB RAM, (2) have a motherboard that has a
PCI hole of sufficient size to make this worthwhile, (3) are running
32-bit operating systems, (4) are absolutely unwilling to upgrade to Win
7 64, and (5) have the technical wherewithal to know that there is a PCI
hole must be vanishingly small.
However, as they say, it's your funeral. Assuming you can learn that
there is a PCI hole, how large the hole is, and where the memory has
been mapped, all you have to do is map it in to kernel space, exactly
like mapping a PCI board's BAR. At that point, it's yours to use
(although, as Don quite correctly pointed out, it's also available to
any other utility like yours, and there is no arbitration).
--
Tim Roberts, [email protected]
Providenza & Boekelheide, Inc.
Tim Roberts, [email protected]
Providenza & Boekelheide, Inc.
Measuring the market opportunity is not my department. I would prefer to let it go and focus on features that appeal to all users, but that doesn't matter because I am not a chief, just an indian. As there is a similar product out there grabbing this memory and has logo and wows their customers with it, I really do not seem to have any technical case to present against this feature.
How can a driver detect this memory in preparation for mapping?
d
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of [email protected]
Sent: Wednesday, March 17, 2010 12:17 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Get the unallocated memory on 32-bit Windows, 4GB sys
So is it a correct summary to say it will work fine unless another device driver doing the same thing is present?
Measuring the market opportunity is not my department. I would prefer to let it go and focus on features that appeal to all users, but that doesn't matter because I am not a chief, just an indian. As there is a similar product out there grabbing this memory and has logo and wows their customers with it, I really do not seem to have any technical case to present against this feature.
How can a driver detect this memory in preparation for mapping?
---
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
have a cheap video device that is reported by the BIOS but may not show
up in your calculation, and I am sure there are more that I forgot. As
far as the chief and the Indian argument, if you are a professional it
is your responsibility to point out all the problems.
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
> -----Original Message-----
> From: [email protected] [mailto:[email protected]]
> Posted At: Wednesday, March 17, 2010 3:17 AM
> Posted To: ntdev
> Conversation: Get the unallocated memory on 32-bit Windows, 4GB sys
> Subject: RE: Get the unallocated memory on 32-bit Windows, 4GB sys
>
> So is it a correct summary to say it will work fine unless another
> device driver doing the same thing is present?
>
> Measuring the market opportunity is not my department. I would prefer
> to let it go and focus on features that appeal to all users, but that
> doesn't matter because I am not a chief, just an indian. As there is a
> similar product out there grabbing this memory and has logo and wows
> their customers with it, I really do not seem to have any technical
> case to present against this feature.
>
> How can a driver detect this memory in preparation for mapping?
>
>
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4951 (20100317) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
Now I am curious about this. Since it was important enough to trumpet this announcement in this thread can you let us know the outcome now that a released driver doing this has been identified?
wrote in message news:[email protected]
> "I want to tell all my clients to be sure to avoid this crap at all
> costs."
>
> Now I am curious about this. Since it was important enough to trumpet this
> announcement in this thread can you let us know the outcome now that a
> released driver doing this has been identified?
>
>
competitors product we will equally scoff, hurrah, and harangue both of
them. The OP did miss the point. Don wants to know BOTH products to avoid
BOTH, not just the OPs.
The personal opinion of
Gary G. Little
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Alexander Grigoriev
Sent: Thursday, March 18, 2010 9:22 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Get the unallocated memory on 32-bit Windows, 4GB sys
Any product that uses undocumented things should be avoided at all costs.
<[email protected]> wrote in message news:[email protected]
> "I want to tell all my clients to be sure to avoid this crap at all
> costs."
>
> Now I am curious about this. Since it was important enough to trumpet this
> announcement in this thread can you let us know the outcome now that a
> released driver doing this has been identified?
>
>
---
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
__________ Information from ESET Smart Security, version of virus signature
database 4955 (20100318) __________
The message was checked by ESET Smart Security.
http://www.eset.com
__________ Information from ESET Smart Security, version of virus signature
database 4955 (20100318) __________
The message was checked by ESET Smart Security.
http://www.eset.com
news:[email protected]
> Any product that uses undocumented things should be avoided at all costs.
Ts ts. A nice phrase, but.... are you ready to part with DebugView?
Quite a lot undocumented things eventually became documented - see
winternl.h.
Yet other things still are not documented completely and accurately, so
people dig and hack and manage to create interesting products anyway.
These days, it's hard to make money without taking risks.
And trust the management, they know better
Regards,
-- pa
1. Is this for a test system (such as DebugView)?
2. Is this for a well controlled environment, i.e. the hardware and the
OS version are stable?
Beyond the above, it has to depend on "how undocumented" something is.
But if I have the choice of a product that uses undocumented and one
that doesn't I agree with Alexander.
As far as the OP's product, I hope we get a clue what it is, since I
want nothing to do with that shit.
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
> -----Original Message-----
> From: Pavel A. [mailto:[email protected]]
> Posted At: Thursday, March 18, 2010 6:04 PM
> Posted To: ntdev
> Conversation: Get the unallocated memory on 32-bit Windows, 4GB sys
> Subject: Re: Get the unallocated memory on 32-bit Windows, 4GB sys
>
> "Alexander Grigoriev" <[email protected]> wrote in message
> news:[email protected]
> > Any product that uses undocumented things should be avoided at all
costs.
>
> Ts ts. A nice phrase, but.... are you ready to part with DebugView?
> Quite a lot undocumented things eventually became documented - see
> winternl.h.
> Yet other things still are not documented completely and accurately,
so
> people dig and hack and manage to create interesting products anyway.
> These days, it's hard to make money without taking risks.
> And trust the management, they know better
>
> Regards,
> -- pa
>
>
>
>
> __________ Information from ESET Smart Security, version of virus
signature
> database 4956 (20100318) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
>managment so perhaps undocumented techniques are the way forward.
You must know the exact upper bound of Windows-visible RAM, and then call MmMapIoSpace on the RAM above this bound.
How will you know the bounds - I don't know. Undocumented.
--
Maxim S. Shatskih
Windows DDK MVP
[email protected]
http://www.storagecraft.com
Do you understand that, if there will be another product doing the same, the major interops with crashes are inevitable?
--
Maxim S. Shatskih
Windows DDK MVP
[email protected]
http://www.storagecraft.com
Down to 3.2GB on some motherboards.
--
Maxim S. Shatskih
Windows DDK MVP
[email protected]
http://www.storagecraft.com
Sounds as if you are back peddling and will not warn all of your clients about the shipping product. No biggie, just checking.
> "Alexander Grigoriev" <[email protected]> wrote in message
> news:[email protected]
>>
>> Any product that uses undocumented things should be avoided at all costs.
>
> Ts ts. A nice phrase, but.... are you ready to part with DebugView?
> Quite a lot undocumented things eventually became documented - see
> winternl.h.
> Yet other things still are not documented completely and accurately, so
> people dig and hack and manage to create interesting products anyway.
> These days, it's hard to make money without taking risks.
> And trust the management, they know better
DebugView is not production ready kernel mode software for
heterogeneous and generic environment. I think that for debugging and
for learning stuff it is perfectly acceptable to use undocumented
features, write drivers in ASM, hook the kernel etc.
When it comes to delivering software to users, you must play by the
book in the most strict sense.
--
Aram Hăvărneanu
> to managment so perhaps undocumented techniques are the way forward.
Ironically you have just presented a key argument against use of "unconventional" stuff - if driver X is doing it and driver Y attempts to do the same (they have no chance to coordinate their "efforts", do they), what is going to happen to the target machine???
Anton Bassov
It is not complete and totally correct, as everyone who ever tried something
not trivial might learn.
There are certain well known don'ts like hooking.
But sometimes we have to rely on results of our own research, that are not
documented at all - not as do's neither as don'ts.
It is in a gray area, risky. There are certain known means to contain the
risk
and make it acceptable.
DebugView is a small example of something not readily taken from a book -
it was delivered to users first, and book writers explained it later.
Same with CD & DVD recording software, same with P2P networking,
same with NTFS hacks (Partition Magic, Ghost), VMware, and many many others.
Management knows what customers are willing to pay for. This is their job.
Developer's job is to do what management wants, and do it well enough
Regards,
--pa
"Aram Havarneanu" <[email protected]> wrote in message news:[email protected]
> On Fri, Mar 19, 2010 at 12:04 AM, Pavel A. <[email protected]> wrote:
>> "Alexander Grigoriev" <[email protected]> wrote in message
>> news:[email protected]
>>>
>>> Any product that uses undocumented things should be avoided at all
>>> costs.
>>
>> Ts ts. A nice phrase, but.... are you ready to part with DebugView?
>> Quite a lot undocumented things eventually became documented - see
>> winternl.h.
>> Yet other things still are not documented completely and accurately, so
>> people dig and hack and manage to create interesting products anyway.
>> These days, it's hard to make money without taking risks.
>> And trust the management, they know better
>
> DebugView is not production ready kernel mode software for
> heterogeneous and generic environment. I think that for debugging and
> for learning stuff it is perfectly acceptable to use undocumented
> features, write drivers in ASM, hook the kernel etc.
>
> When it comes to delivering software to users, you must play by the
> book in the most strict sense.
>
> --
> Aram HÄfvÄfrneanu