I'm using Dev C++ to practice C programming, but while I was executing the program (no problems on compiling) I noticed that depending the data I allocate with malloc the program stops, saying:
the programm "name".exe stopped working
Any ideas why dev does not allow me to allocate much memory?
Related
Suppose there is a c++ code. It is compile to binary code during compiling. My question is where does my computer store binary code, in main memory (DRAM) of in program memory (inside CPU).
And I also want to know can the content of program memory be changed by computer user?
If you have a memory designed to holds X, that's where you need to put X in.
If the CPU of your reference architecture fetches instructions from a dedicated program memory, that's where the instructions must be stored since the CPU will look for them only there.
It's worth saying that modern processors are von Neumann, they have a unified program and data memory (internally they are not, e.g the caches are split) while
microcontroller are often Harvard.
I'll gloss over the advantages of each one and just say that each combination of attributes exists: you can have a CPU where the program memory is RO and programmed in factory, where it is programmable through an external interface, where it cannot be read by the program itself, where it can and where it can be written by the program itself.
I am dealing with a Fortran code with MPI parallelization where I'm running out of memory for intensive runs. I'm careful to allocate nearly all of the memory that is required at the beginning of my simulation. Subroutine static memory allocation is typically small, but if I was to run out of memory due to these subroutines, it would occur early in the simulation because the memory allocation should not grow as time progresses. My issue is that far into the simulation, I am running into memory errors such as:
Insufficient memory to allocate Fortran RTL message buffer, message #174 = hex 000000ae.
The only thing that I can think of is that my MPI calls are using memory that I cannot preallocate at the beginning of the simulation. I'm using mostly MPI_Allreduce, MPI_Alltoall, and MPI_Alltoallv while the simulation is running and sometimes I am passing large amounts of data. Could the memory issues be a result of internal buffers created by MPI? How can I prevent a surprise memory issue like this? Can this internal buffer grow during the simulation?
I've looked at Valgrind and besides the annoying MPI warnings, I'm not seeing any other memory issues.
It's a bit hard to tell if MPI is at fault here without knowing more details. You can try massif (one of the valgrind tools) to find out where memory is being allocated.
Be sure you don't introduce any resource leaks: If you create new MPI resources (communicators, groups, requests etc.), make sure to release them properly.
In general, be aware of the buffer sizes required for all-to-all communication, especially at large scale. Use MPI_IN_PLACE if possible, or send data in small chunks rather than as single large blocks.
I'm trying to allocate memory with my "program" - just to allocate it and stay there - for testing purposes. When I run it on my MacOS the Activity Monitor shows it allocates 1.6 gb, when I compile it for linux and run it there it does nothing - it prints the message but the ram isn't being used on the machine. Am I doing it wrong? Is there a better way? Here is my code:
package main
import (
"fmt"
"unsafe"
"time"
)
func main() {
var buffer [100 * 1024 * 1024]string
fmt.Printf("The size of the buffer is: %d bytes\n", unsafe.Sizeof(buffer))
time.Sleep(300 * time.Second)
}
First I used byte for my array type, but it did not worked event on my mac?
There's nothing whatsoever in your code that requires memory. The compiler is perfectly within its rights to optimize the whole allocation away, and even if the compiler doesn't do it, the OS will not commit the memory - you're never assigning anything, so it's likely just mirroring a zero-page.
I don't know anything about the subtle differences between whatever Linux you're using and whatever MacOS you're using, so there's little that can be said with certainty. It might very well be that your way of checking memory on your linux machine gives you only committed memory, while you're seeing all virtual memory on your MacOS, or there might be other subtle differences. In any case, since compilers became smart and since we had virtual memory on PCs, it's been getting harder and harder to get any meaningful benchmarks - the tools we work with are usually smart enough to avoid unnecessary waste; and most benchmarks you'll try are pretty much completely unnecessary waste.
Benchmarking is hard.
I have a 32-bit Delphi application running with /LARGEADDRESSAWARE flag on. This allows to allocate up to 4GB on a 64-bit system.
Am using threads (in a pool) to process files where each task loads a file in memory. When multiple threads are running (multiple files being loaded), at some point the EOutOfMemory hits me.
What would be the proper way to get the available address space so I can check if I have enough memory before processing the next file?
Something like:
if TotalMemoryUsed {from GetMemoryManagerState} + FileSize <
"AvailableUpToMaxAddressSpace" then NoOutOfMemory
I've tried using
TMemoryStatusEx.ullAvailVirtual for AvailableUpToMaxAddressSpace
but the results are not correct (sometimes 0, sometimes > than I actually have).
I don't think that you can reasonably and robustly expect to be able to predict ahead of time whether or not memory allocations will fail. At the very least you would probably need to write your own memory allocator that was dedicated to serving your application, and have a very strong understanding of the heap allocation requirements of your process.
Realistically the tractable way forward for you is to break free from the shackles of 32 bit address space. That is your fundamental problem. The way to escape from 32 bit address space is to compile for 64 bit. That requires XE2 or later.
You may need to continue supporting 32 bit versions of your application because you have users that are still on 32 bit systems. The modern versions of Delphi have 32 bit and 64 bit compilers and it is quite simple to write code that will compile and behave correctly under both scenarios.
For your 32 bit versions you are less likely to run into memory problems anyway because 32 bit systems tend to run on older hardware with fewer processors. In turn this means less demand on memory space because your thread pool tends to be smaller.
If you encounter machines with large enough processor counts to cause out of memory problems then one very simple and pragmatic approach is to give the user a mechanism to limit the number of threads used by your application's thread pool.
Since there are more processes running on the target system even if the necessary infrastructure would be available, if would be no use.
There is no guarantee that another process does not allocate the memory after you have checked its availability and before you actually allocate it. The right thing to do is writing code that will fail gracefully and catch the EOutOfMemory exception when it appears. Use it as a sign to stop creating more threads until some of them is already terminated.
Delphi is 32bit, so you can't allocate memory addresses larger than that.
Take a look at this:
What is a safe Maximum Stack Size or How to measure use of stack?
I am developing for the Arduino Due which has 96k SRAM and 512k flash memory for code. If I have a program that will compile to, say, 50k, when I run the code, how much sram will I use? will I use 50k immediately, or only the memory used by the functions I call? Is there a way to measure this memory usage before I upload the sketch to the arduino?
You can run
arm-none-eabi-size bin.elf
Where:
bin.elf is the generated binary (look it up in the compile log)
arm-none-eabi-size is a tool included with Arduino for arm which lets you know the memory distribution of your binary. This program can be found inside the Arduino directory. In my mac, this is /Applications/Arduino.app/Contents/Resources/Java/hardware/tools/g++_arm_none_eabi/bin
This command will output:
text data bss dec hex filename
9648 0 1188 10836 2a54 /var/folders/jz/ylfb9j0s76xb57xrkb605djm0000gn/T/build2004175178561973401.tmp/sketch_oct24a.cpp.elf
data + bss is RAM, text is program memory.
Very important: This doesn't account for dynamic memory (created in stack), this is only RAM memory for static and global variables. There are other techniques to check the RAM usage dynamically, like this one, but it will depend on the linker capabilities of the compiler suite you are using.
Your whole program is loaded into arduino, so atleast 50K flash memory will be used. Then on running the code, you will allocate some variables, some on stack, some global which will take some memory too but on SRAM.
I am not sure if there is a way to exactly measure the memory required but you can get a rough estimation based on the number and types of variables being allocated in the code. Remember, the global variables will take the space during the entire time the code is running on arduino, the local variables( the ones that are declared within a pair of {..}) remain in the memory till the '}' brace also known as the scope of the variables. Also remember, the compiled 50K code which you are mentioning is just the code portion, it does not include your variables, not even the global ones. The code is store in Flash memory and the variables are stored in the SRAM. The variables start taking memory only during runtime.
Also I curious to know how you are calculating that your code uses 50K memory?
Here is a little library to output the avalaible RAM memory.
I used it a lot when my program was crashing with no bug in the code. It turned out that I was running out of RAM.
So it's very handy!
Avalaible Memory Library
Hope it helps! :)