I thought that a 32-bit process could use at most 2GB RAM, since half the address space is reserved for the OS (maybe 3GB with the /3GB switch). However, StarCraft II manages to use nearly 4GB and it's a 32-bit EXE. Process Explorer shows:
4,000,896 KB private bytes
3,928,164 KB working set
How does it do that and how can I do it in my own programs? Is it possible in .NET? In unmanaged C++? In managed C++ somehow?
On 64bit OS 32bit process can use almost 4GB.
Processes marked as "large address space aware" can use up to 3GB in 32bit OS and up to full address space on 64bit OS.
Here is more discussion on it: Can a 32bit process access more memory on a 64bit windows OS? .
Related
When executing a 32-bit program on a 64-bit CPU, can we assume that the underlying hardware will use 32-bit values in memory and in the CPU registers to store an integer?
Given these setups:
Intel i5 + Windows 32 bit + 32 bit application
Intel i5 + Windows 64 bit + 32 bit application
Please explain the drawbacks of each setup. Would using the second one be less efficient because the CPU gets extra work to ignore halt of the register's value?
When an x86_64 processor executes 32-bit code it is effectively running in i686 mode. This is all achieved in hardware so there is no performance penalty.
A user-space 32-bit program will be executed in "32 bit mode" whether the operating system is 32 bits or 64 bits so the behaviour should be identical.
The only potential drawback is that the program might need to dynamically link to some 32-bit libraries which are not installed by default on the 64-bit OS. That sort of scenario is more common on Linux where it is much more common for programs to dynamically link to external libraries; the only example I can think of on Windows is if the program needs the Visual C++ runtime library then you would have to install the 32-bit version of that (possibly alongside the 64-bit version, if another program needed that). So in short you might have to install more stuff with a 64 bit set up.
Also the actual OS might consume more memory on a 64 bit setup, however one drawback of 32-bit system is that your entire system memory size is limited to 4GB (or practically about 3.5 GB due to some memory space being mapped to hardware).
64-bit Knowledge Base: How much memory can an application access in Win32 and Win64?
Both MS Windows and Oracle Linux allows 32bit applications to use >4GB of Memory. Windows method is AWE: Address Windowing Extensions and Linux's method is Very Large Memory.
How it works: 32-bit application can't directly address > 4 GB of Virtual memory; but 64-bit OS can and 4GB of memory is too small to some applications. So, VLM and AWE allow application to reserve huge amount of memory from 64bit OS (or even from 32bit OS with AWE). 32-bit application can't address this memory directly, but it can ask OS to setup mapping of some part of huge memory into first 4GB (into 32-bit virtual space), then this memory can be accessed, modified; then it is unmapped back (with OS request).
Question is: Is there something like VLM or AWE in Solaris OS (version 10 or 11; x86_64 or sparc64)?
There is no library I'm aware of but implementing it would be quite straightforward under Solaris (and all Unix/Unix like OSes supporting tmpfs and mmap).
Just create a file the size you want (eg: 16 GiB) in /tmp (assuming /tmp is on tmpfs, the default configuration) and have the process(es) mapping various areas of this file to access memory at the wanted offsets.
Should you really want to access physical memory and not virtual one, you can use Solaris ramdisk support (ramdiskadm) instead of tmpfs.
Solaris supports PAE (Physical Address Extension), but Googling around doesn't paint a pretty picture. There is very little information available, and most of it is dire warnings that a bunch of third-party drivers won't work with it.
I strongly expect this to end up as a duplication, but I can't seem to find it.
I've got a C++ program that I normally run on 64-bit MacOSX SnowLeopard.
When I try to run it on a 32-bit Windows 7, it runs out of memory. Probably, it really needs too much memory, but I want to make sure that I'm not missing some sort of option or another that allows me to squeeze out the maximum possible heap size.
32 bit Process have a 4GB memory limit, but it's spitted into 2GB for the user address space and the kernel address space.
There is a /3GB switch so that you can use 3GB for the user space and 1 GB for the kernel space. To do this you need to change a setting in the OS through boot.ini(Win 2000, XP, 2003) or the bcdedit utility(Win Vista an later). Also you need to make your exe aware of this switch linking it with the /LARGEADDRESSAWARE flag. You can do this with the editbin utility(it comes with the Windows SDK).
Other than that I'm afraid you have to make some changes to how your app runs so that it doesn't take as much memory.
According to this table, the limit per process should be 2GB or 3GB with some registry tampering. I don't think this is much different from previous versions of Windows. I recall XP limited addressable memory to 3GB.
Under Windows Server 2003, Enterprise Edition, SP2 (/3GB switch not enabled)
As I understand it, and I may be wrong, the maximum addressable memory for a process is 4GB.
Is that 2GB of private bytes and 2GB of virtual bytes?
Do you get "out of memory" errors when the private byte limit or virtual byte limit is reached?
It is correct that the maximum address space of a process is 4GB, in a sense. Half of the address space is, for each process, taken up by the operating system. This can be changed with the 3GB switch but it might cause system instability. So, we are left with 2GB of addressable memory for the process to use on its own. Well, not entirely. It turns out that a part of this space is taken up by other stuff such as DLLs an other common code. The actual memory available to you as a programmer is around 1.5GB - 1.7GB.
I'm not sure about how you can handle accidentally going above this limit but I know of games which crash in large multiplayer maps for this reason. Another thing to note is that a 32bit program cannot use more than the 2GB address space on a 64bit system unless they enable the /LARGEADDRESSAWARE:YES linker flag.
Mark Russinovich started a series of posts on this..
Pushing the Limits of Windows: Physical Memory
While 4GB is the licensed limit for 32-bit client SKUs, the effective limit is actually lower and dependent on the system's chipset and connected devices. The reason is that the physical address map includes not only RAM, but device memory as well, and x86 and x64 systems map all device memory below the 4GB address boundary to remain compatible with 32-bit operating systems that don't know how to handle addresses larger than 4GB. If a system has 4GB RAM and devices, like video, audio and network adapters, that implement windows into their device memory that sum to 500MB, 500MB of the 4GB of RAM will reside above the 4GB address boundary.
You can only access 2Gb of memory in total (without the 3Gb switch) on 32bit Windows platforms.
You could run multiple 32bit VMs on a 64bit OS so that each app has access to as much memory as possible if your machine has more than 4Gb.
A lot of people are just starting to hit these barriers, I guess it's easier if your app is in .net or Java as the VMs happily go up to 32Gb of memory on 64bit os.
On 32 bits, if there is enough physical memory and disk space for virtual memory, memory runs out around 3GB since the kernel reserves the address space above 0xC0000000 for itself. On a 64 bits kernel running a 64 bits application, the limit is at 8TB.
For more details, check out MSDN - Memory Limits for Windows Releases
Maximum addressable memory for a 32bit machine is 4GB, for a 64bit machine you can address loads more. (Although some 32bit machines have extension systems for accessing more, but I don't think this is worth bothering with or considering for use).
You get out of memory errors when the virtual limit is reached. On Windows Server 2003, task manager tells you the limit on the performance tab labelled 'Commit Charge Limit'.
Jeff covered this a while back on his blog in terms of 32 bit Vista.
Does the same 32 bit 4 GB memory cap that applies in 32 bit Vista apply to 32 bit Ubuntu? Are there any 32 bit operating systems that have creatively solved this problem?
Ubuntu server has PAE enabled in the kernel, the desktop version does not have this feature enabled by default.
This explains, by the way, why Ubuntu server does not work in some hardware emulators whereas the desktop edition does
Well, with windows, there's something called PAE, which means you can access up to 64 GB of memory on a windows machine. The downside is that most apps don't support actually using more than 4 GB of RAM. Only a small number of apps, like SQL Server are programmed to actually take advantage of all the extra memory.
Yes, 32 bit ubuntu has the same memory limitations.
There are exceptions to the 4GB limitation, but they are application specific... As in, Microsoft Sql Server can use 16 gigabytes with "Physical address Extensions" [PAE] configured and supported and... ugh
http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=3703755&SiteID=17
Also drivers in ubuntu and windows both reduce the amount of memory available from the 4GB address space by mapping memory from that 4GB to devices. Graphics cards are particularly bad at this, your 256MB graphics card is using up at least 256MB of your address space...
If you can [your drivers support it, and cpu is new enough] install a 64 bit os. Your 32 bit applications and games will run fine.
There seems to be some confusion around PAE. PAE is "Page Address Extension", and is by no means a Windows feature. It is a hack Intel put in their Pentium II (and newer) chips to allow machines to access 64GB of memory. On Windows, applications need to support PAE explicitely, but in the open source world, packages can be compiled and optimized to your liking. The packages that could use more than 4GB of memory on Ubuntu (and other Linux distro's) are compiled with PAE support. This includes all server-specific software.
In theory, all 32-bit OSes have that problem. You have 32 bits to do addressing.
2^32 bits / 2^10 (bits per kb) / 2^10 (kb per mb) / 2^10 (mb per gb) = 2^2 = 4gb.
Although there are some ways around it. (Look up the jump from 16-bit computing to 32-bit computing. They hit the same problem.)
Linux supports a technology called PAE that lets you use more than 4GB of memory, however I don't know whether Ubuntu has it on by default. You may need to compile a new kernel.
Edit: Some threads on the Ubuntu forums suggest that the server kernel has PAE on by default, you could try installing that.