As I know, heaps such as malloc is stored in dynamic data. But is it right to say
that malloc function call may allocate the memory space in stack area?
malloc is not a feature of MARS or QtSpim.
There is a system call, #9, that mimics *nix sbrk, to allocate additional address space to the simulated process. It returns to you the next address available past the global data (and past any prior sbrk's). (However, it only allocates, and does not return memory the way a proper sbrk would do given a negative value.) Another name for this area of memory is heap / heap memory.
It does not allocate stack space as that is in substantially higher in the address space. On these simulators, the stack is limited to about 4MB, and the memory returned by sbrk does not reach anywhere near into that 4MB.
Heap memory is useful since a heap allocated memory will survive a function call — a function can return data in the heap but cannot return newly allocated data in the stack, since, by definition, a function that allocates stack space must release that exact same amount when it leaves.
Stack space is allocated simply by decrementing the stack pointer (and released by incrementing the stack pointer). By convention, all functions & subroutines agree to leave existing stack memory alone and allocate new stack memory if they want it, though they also must release it before returning to their caller. When those restrictions do not meet requirements, heap memory is a good choice. Global data is also an option for memory that survives a function call, though that can lead to problems with multithreading (and potentially with recursion).
Related
Say I have a big block of mapped memory I finished using. It came from mmaping anonymous memory or using MAP_PRIVATE. I could munmap it, then have malloc mmap again the next time I make a big enough allocation.
Could I instead give the memory to malloc directly? Could I say "Hey malloc, here's an address range I mapped. Go use it for heap space. Feel free to mprotect, mremap, or even munmap it as you wish."?
I'm using glibc on linux.
glibc malloc calls __morecore (a function pointer) to obtain more memory. See <malloc.h>. However, this will not work in general because the implementation assumes that the function behaves like sbrk and returns memory from a single, larger memory region. In practice, with glibc malloc, the only realistic way to make memory available for reuse by malloc is calling munmap.
Other malloc implementations allow donating memory (in some cases as internal interfaces). For example, musl's malloc has a function called __malloc_donate which should do what you are asking for.
I read things like "memory is allocated in a stack " or things like "these variable are placed in a heap ". I had once studied a book on microprocessor and can faintly remember that there had been topics or sections on something called as stack . And I do know that stacks also mean a kind of LIFO type data structure .
So , I feel confused as to what stacks imply . Are there memory locations in a every microprocessor other than the registers which are called as stack ?
I'll describe the most common situation.
In this context, stack is a dedicated memory for a program (more precisely, for a thread). This memory is allocated automatically by the operating system, when your program is started. Usually (but not always), stack is allocated from the main memory (so it is not a special memory in the CPU).
It's name is stack, because it is used "LIFO style". When a function is called, its local variables gets allocated from the stack ("pushed to the stack"). When it returns, these variables are freed ("pop from the stack").
About heap: heap is the place from where one can allocate memory in a more flexible manner than stack. Heap storage space is usually much larger than the stack. And the allocated space will be available even after the function (which allocated the space) returns. And for languages which doesn't have garbage collection, you have to manually free the allocated space. This heap is not to be confused with the data structure heap, which is a completely different thing.
char *var;
void example(int length) {
char stackVar[1024]; // a 1024 element char array allocated on the stack
char *heapVar = new char[length]; // a length sized variable allocated on the heap, and a pointer (heapVar) to this place allocated on the stack
var = heapVar; // store a pointer to the allocated space
// upon return, stackVar is automatically freed
// the pointer heapVar automatically freed
// the space that heapVar points to is not freed automatically, can be used afterwards (via the var pointer)
}
In the following procedure, will the array be allocated on the stack?
procedure One:
var
arr: array[0..1023] of byte;
begin
end;
What is the largest item that can go on the stack?
Is there a speed difference between accessing variable on the stack and on the heap?
In the following procedure, will the array be allocated on the stack?
Yes, provided that the local variable is not captured by an anonymous method. Such local variables reside on the heap.
What is the largest item that can go on the stack?
It depends on how large the stack is, and how much of the stack has already been used, and how much of the stack is used by calls made by the function itself. The stack is a fixed size, determined when the thread is created. The stack overflows if it grows beyond that size. On Windows at least, the default stack size is 1MB, so I would not expect you to encounter problems with a 1KB array as can be seen here.
Is there a speed difference between accessing variable on the stack and on the heap?
By and large no, but again this depends. Variables on the stack are probably more likely to be accessed frequently, and so probably easier to be cached. But for a decently sized object, like the 1KB array we can see here, I would not expect there to be any difference in access time. In terms of the underlying memory architecture, there's no difference between stack and heap, it's all just memory.
Now, where there is a difference in performance is in allocation. Heap allocation is more expensive than stack allocation. And especially if you have a multi-threaded application, heap allocation can be a bottleneck. In particular, the default Delphi memory manager does not scale well in multi-threaded use.
I am trying to find some useful information on the malloc function.
when I call this function it allocates memory dynamically. it returns the pointer (e.g. the address) to the beginning of the allocated memory.
the questions:
how the returned address is used in order to read/write into the allocated memory block (using inderect addressing registers or how?)
if it is not possible to allocate a block of memory it returns NULL. what is NULL in terms of hardware?
in order to allocate memory in heap we need to know which memory parts are occupied. where this information (about the occupied memory) is stored (if for example we use a small risc microcontroller)?
Q3 The usual way that heaps are managed are through a linked list. In the simplest case, the malloc function retains a pointer to the first free-space block in the heap, and each free-space block has a header that points to the next free space block in the heap. So the heap is in-effect self-defining in terms of knowing what is not occupied (and by inference what is therefore occupied); this minimizes the amount of overhead RAM needed to manage the heap.
When new space is needed via a malloc call, a large enough free-space block is found by traversing the linked list. That found free-space block is given to the malloc caller (with a small hidden header), and if needed a smaller free-space block is inserted into the linked list with any residual space between the original free space block and how much memory the malloc call asked for.
When a heap block is released by the application, its block is just formatted with the linked-list header, and added to the linked list, usually with some extra logic to combine consecutive free-space blocks into one larger free-space block.
Debugging versions of malloc usually do more, including retaining linked-lists of the allocated areas too, "guard zones" around the allocated heap areas to help detect memory overflows, etc. These take up extra heap space (making the heap effectively smaller in terms of usable space for the applications), but are extremely helpful when debugging.
Q2 A NULL pointer is effectively just a zero, which if used attempts to access memory starting at location 0 of RAM, which is almost always reserved memory of the OS. This is the cause of a significant quantity of memory violation aborts, all caused by programmer's lack of error checking for NULL returns from functions that allocate memory).
Because accessing memory location 0 by a non-OS application is never what is wanted, most hardware aborts any attempt to access location 0 by non-OS software. Even with page mapping such that the applications memory space (including location 0) is never mapped to real RAM location 0, since NULL is always zero, most CPUs will still abort attempts to access location 0 on the assumption that this is an access via a pointer that contains NULL.
Given your RISC processor, you will need to read its documentation to see how it handles attempts to access memory location 0.
Q1 There are many high-level language ways to use allocated memory, primarily through pointers, strings, and arrays.
In terms of assembly language and the hardware itself, the allocated heap block address just gets put into a register that is being used for memory indirection. You will need to see how that is handled in the RISC processor. However if you use C or C++ or such higher level language, then you don't need to worry about registers; the compiler handles all that.
Since you are using malloc, can we assume you are using C?
If so, you assign the result to a pointer variable, then you can access the memory by referencing through the variable. You don't really know how this is implemented in assembly. That depends on CPU you are using. malloc return 0 if it fails. Since usually NULL is defined as 0, you can test for NULL. You don't care how malloc tracks the free memory. If you really need this information, you should look at the source in glibc/malloc available on the net
char * c = malloc(10); // allocate 10 bytes
if (c == NULL)
// handle error case
else
*c = 'a' // write a in the first character on the block
I know I can reserve virtual memory using VirtualAlloc.
e.g. I can claim 1GB of virtual memory and then call in the first MB of that to put my a growing array into.
When the array grows beyond 1MB I call in the 2nd MB and so on.
This way I don't need to move the array around in memory when it grows, it just stays in place and the Intel/AMD virtual memory manager takes care of my problems.
However does FastMM support this structure, so I don't have to do my own memory management?
Pseudo code:
type
PBigarray = ^TBigarray;
TBigArray = array[0..0] of SomeRecord;
....
begin
VirtualMem:= FastMM.ReserveVirtualMemory(1GB);
PBigArray:= FastMM.ClaimPhysicalMemory(VirtualMem, 1MB);
....
procedure GrowBigArray
begin
FastMM.ClaimMorePhysicalMemory(PBigArray, 1MB {extra});
//will generate OOM exception when claim exceeds 1GB
Does FastMM support this?
No, FastMM4 (as of the latest version I looked at) does not explicitly support this. It's really not a functionality you would expect in a general purpose memory manager as it's trivially simple to do with VirtualAlloc calls.
NexusMM4 (which is part of NexusDB) does something that gives you a similar result, but without wasting all the address space before it is needed in the background.
If you make an initial large allocation (directly via GetMem, or indirectly via a dynamic array or such) the memory is allocated in just the size needed, via VirtualAlloc.
But if that allocation is then resized to a larger size, NexusMM will use a different way to allocate memory which allows it to simply unmap the allocation from the address space an remap it again, at a larger size, when further reallocs takes place.
This prevents the 2 major problems that most general purpose memory managers have when reallocating:
during a normal realloc the existing and new allocation need to be present in the address space at the same time, temporarily doubling the address space and physical memory requirements
during a normal realloc, the whole contents of the existing allocation needs to be copied
So with NexusMM you would get all the advantages of what you showed in your pseudo code (with the exception that the first realloc will involve a copy, and that growing your array might change it's address) by simply using normal GetMem/ReallocMem/FreeMem calls.