I tried to allocate 21,128 bytes using a wrapper function, which calls malloc internally.
Malloc Stats() :-
system bytes = 14618624
in use bytes = 13759424
Arena 1:
Arena 0:
system bytes = 14626816
in use bytes = 13759600
Arena 1:
system bytes = 135168
in use bytes = 3280
Arena 2:
system bytes = 135168
in use bytes = 13088
But still, I see malloc has failed. What could be possibly the reason?
*** Error in `./wr_acc': malloc(): memory corruption: 0x00007ff4747a2ff0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x82c86)[0x7ff48c9d5c86]
/lib64/libc.so.6(__libc_malloc+0x4c)[0x7ff48c9d884c]
./wr_acc[0xdf4c28]
Please help. I am a beginner.
The error message is clear:
*** Error in `./wr_acc': malloc(): memory corruption: 0x00007ff4747a2ff0 ***
malloc() detected an invalid state in its ancillary structures so it gave up trying to allocate memory and aborted the program to avoid potentially damaging side effects.
The private data malloc() uses to keep track of allocated and free blocks may have been overwritten by your program, for example by writing beyond the end of an allocated block or before its beginning. You could post the code and see if anyone can spot such problems.
Related
What could be the reason for the error "Failed to allocate 153 bytes in slab allocator for memtx_tuple" on the client when writing to Tarantool memtx?
This means that memtx engine runs out of memory. Memory available for data and indexes stored in memtx is limited by memtx_memory option of box.cfg, which is default to 256Mb. It's possible to increase this limit in runtime:
-- add 512Mb
box.cfg({memtx_memory = box.cfg.memtx_memory + 512 * 2^20})
Here is documentation section about function for monitoring memory usage:
https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_slab/#lua-function.box.slab.info
Please consider the following code:
char **ptr;
str = malloc(sizeof(char *) * 3); // Allocates enough memory for 3 char pointers
str[0] = malloc(sizeof(char) * 24);
str[1] = malloc(sizeof(char) * 25);
str[2] = malloc(sizeof(char) * 25);
When I use some printf to print the memory adresses of each pointer:
printf("str[0] = '%p'\nstr[1] = '%p'\nstr[2] = '%p'\n", str[0], str[1], str[2]);
I get this output:
str[0] = '0x1254030'
str[1] = '0x1254050'
str[2] = '0x1254080'
I expected the number corresponding to the second adress to be the sum of the number corresponding to the first one, and 24, which corresponds to the size of the string str[0] in bytes (since a char has a size of 1 byte). I expected the number corresponding to the second adress to be 0x1254047, considering that this number is expressed in base 16 (0123456789abcdef).
It seems to me that I spotted a pattern: from 24 characters in a string, for every 16 more characters contained in it, the memory used is 16 bytes larger. For example, a 45 characters long string uses 64 bytes of memory, a 77 characters long string uses 80 bytes of memory, and a 150 characters long string uses 160 bytes of memory.
Here is an illustration of the pattern:
I would like to understand why the memory allocated isn't equal to the size of the string. Why does it follow this pattern?
There at least two reasons malloc() may return memory in the manner you've noted.
Efficiency and peformance. By returning blocks of memory in only a small set of actual sizes, a request for memory is much more likely to find an already-existing block of memory that can be used, or wind up producing a block of memory that can be easily reused in the future. This will make the request return faster and has the side effect of limiting memory fragmentation.
Implications arising from the memory alignment requirements 7.22.3 Memory management functions of the C Standard states "The order and contiguity of storage allocated by successive calls to the
aligned_alloc, calloc,
malloc, and
realloc functions is unspecified. The
pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to
a pointer to any type of object with a fundamental alignment requirement ..." Note the italicized part. Since the malloc() implementation has no knowledge of what the memory is to be used for, the memory returned has to be suitably aligned for any possible use. This usually means 8- or 16-byte alignment, depending on the platform.
Three reasons:
(1) The string "ABC" contains 4 characters, because every string in C has to have a terminating '\0'.
(2) Many processors have memory address alignment issues that make it most efficient when any block allocated by malloc() starts on an address that is a multiple of 4, or 8, or whatever the "natural" memory size is.
(3) The malloc() function itself requires some memory to store information about what has been allocated where, so that free() knows what to do.
I'm attempting to do an exercise from "Expert C Programming" where the point is to see how much memory a program can allocate. It hinges on malloc returning NULL when it cannot allocate anymore.
#include <stdio.h>
#include <stdlib.h>
int main() {
int totalMB = 0;
int oneMeg = 1<<20;
while (malloc(oneMeg)) {
++totalMB;
}
printf("Allocated %d Mb total \n", totalMB);
return 0;
}
Rather than printing the total, I get a kernel panic after allocating ~8GB on my 16GB Macbook Pro.
Kernel panic log:
Anonymous UUID: 0B87CC9D-2495-4639-EA18-6F1F8696029F
Tue Dec 13 23:09:12 2016
*** Panic Report ***
panic(cpu 0 caller 0xffffff800c51f5a4): "zalloc: zone map exhausted while allocating from zone VM map entries, likely due to memory leak in zone VM map entries (6178859600 total bytes, 77235745 elements allocated)"#/Library/Caches/com.apple.xbs/Sources/xnu/xnu-3248.50.21/osfmk/kern/zalloc.c:2628
Backtrace (CPU 0), Frame : Return Address
0xffffff91f89bb960 : 0xffffff800c4dab12
0xffffff91f89bb9e0 : 0xffffff800c51f5a4
0xffffff91f89bbb10 : 0xffffff800c5614e0
0xffffff91f89bbb30 : 0xffffff800c5550e2
0xffffff91f89bbba0 : 0xffffff800c554960
0xffffff91f89bbd90 : 0xffffff800c55f493
0xffffff91f89bbea0 : 0xffffff800c4d17cb
0xffffff91f89bbf10 : 0xffffff800c5b8dca
0xffffff91f89bbfb0 : 0xffffff800c5ecc86
BSD process name corresponding to current thread: a.out
Mac OS version:
15F34
I understand that this can easily be fixed by the doctor's cliche of "It hurts when you do that? Then don't do that" but I want to understand why malloc isn't working as expected.
OS X 10.11.5
For the definitive answer to that question, you can look at the source code, which you'll find here:
zalloc.c source in XNU
In that source file find the function zalloc_internal(). This is the function that gives the kernel panic.
In the function you'll find a "for (;;) {" loop, which basically tries to allocate the memory you're requesting in the specified zone. If there isn't enough space, it immediately tries again. If that fails it does a zone_gc() (garbage collect) to try to reclaim memory. If that also fails, it simply kernel panics - effectively halting the computer.
If you want to understand how zalloc.c works, look up zone-based memory allocators.
Your program is making the kernel run out of space in the zone called "VM map entries", which is a predefined zone allocated at boot. You could probably get the result you are expecting from your program, without a kernel panic, if you allocated more than 1 MB at a time.
In essence it is not really a problem for the kernel to allocate you several gigabytes of memory. However, allocating thousands of smaller allocations summing up to those gigabytes is much harder.
Below is the C code
#include <stdio.h>
void read_input()
{
char input[512];
int c = 0;
while (read(0, input + c++,1) == 1);
}
int main ()
{
read_input();
printf("Done !\n");
return 0;
}
In the above code, there should be a buffer overflow of the array 'input'. The file we give it will have over 600 characters in it, all 2's ( ex. 2222222...) (btw, ascii of 2 is 32). However, when executing the code with the file, no segmentation fault is thrown, meaning program counter register was unchanged. Below is the screenshot of the memory of input array in gdb, highlighted is the address of the ebp (program counter) register, and its clear that it was skipped when writing:
LINK
The writing of the characters continues after the program counter, which is maybe why segmentation fault is not shown. Please explain why this is happening, and how to cause the program counter to overflow.
This is tricky! Both input[] and c are in stack, with c following the 512 bytes of input[]. Before you read the 513th byte, c=0x00000201 (513). But since input[] is over you are reading 0x32 (50) onto c that after reading is c=0x00000232 (562): in fact this is little endian and the least significative byte comes first in memory (if this was a big endian architecture it was c=0x32000201 - and it was going to segfault mostly for sure).
So you are actually jumping 562 - 513 = 49 bytes ahead. Than there is the ++ and they are 50. In fact you have exactly 50 bytes not overwritten with 0x32 (again... 0x3232ab64 is little endian. If you display memory as bytes instead of dwords you will see 0x64 0xab 0x32 0x32).
So you are writing in not assigned stack area. It doesn't segfault because it's in the process legal space (up to the imposed limit), and is not overwriting any vital information.
Nice example of how things can go horribly wrong without exploding! Is this a real life example or an assignment?
Ah yes... for the second question, try declaring c before input[], or c as static... in order not to overwrite it.
I am trying to find the memory map of an array or some memory allocated from malloc() using mmap() but it is showing invalid argument.
#include<stdio.h>
#include<sys/mman.h>
#include<stdlib.h>
int main()
{
int *var1=NULL;
size_t size=0;
size = 1000*sizeof(int);
var1 = (int*)malloc(size);
int i=0;
for(i=0;i<999;i++)
{
var1[i] = 1;
}
printf("%p\n",var1);
void *addr=NULL;
addr = mmap((void *)var1, size, PROT_EXEC|PROT_READ|PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS | MAP_FIXED, -1, 0); //to create memory map of var1
err(1,NULL); //to print error
return 0;
}
Error:
a.out: Invalid argument
Please help me.
Thank you in advance.
Proximate cause: mmap fails because you asked it do create a new memory mapping, you asked for the mapping to be placed at a specific address (var1's address), that address is already occupied (by the heap from which malloc got its memory), and you told the operating system it was not allowed to choose an alternate address in case var1 was not a suitable address (MAP_FIXED).
Analysis: What are you trying to do here? What does "find the memory map of an array" mean? Do you want to have your array of integers located in heap memory (returned by malloc()) or in an anonymous memory mapping created by mmap()? By the way, unless you fork() (create a child process) there is little functional difference: both are areas of memory that are private to your process. But they are not the same thing and you can't manipulate the heap with mmap() nor can you manage mapped memory with malloc().