I have a short program that generates a linked list by adding nodes to it, then frees the memory allocated by the linked list.
Valgrind does not report any memory leak errors, but the process continues to hold the allocated memory.
I was only able to fix the error after I changed the memory allocated from sizeof(structure_name) to fixed number 512. (see commented code)
Is this a bug or normal operation?
Here is the code:
#include <execinfo.h>
#include <stdlib.h>
#include <stdio.h>
typedef struct llist_node {
int ibody;
struct llist_node * next;
struct llist_node * previous;
struct llist * list;
}llist_node;
typedef struct llist {
struct llist_node * head;
struct llist_node * tail;
int id;
int count;
}llist;
llist_node * new_lnode (void) {
llist_node * nnode = (llist_node *) malloc ( 512 );
// llist_node * nnode = (llist_node *) malloc ( sizeof(llist_node) );
nnode->next = NULL;
nnode->previous = NULL;
nnode->list = NULL;
return nnode;
}
llist * new_llist (void) {
llist * nlist = (llist *) malloc ( 512 );
// llist * nlist = (llist *) malloc ( sizeof(llist) );
nlist->head = NULL;
nlist->tail = NULL;
nlist->count = 0;
return nlist;
}
void add_int_tail ( int ibody, llist * list ) {
llist_node * nnode = new_lnode();
nnode->ibody = ibody;
list->count++;
nnode->next = NULL;
if ( list->head == NULL ) {
list->head = nnode;
list->tail = nnode;
}
else {
nnode->previous = list->tail;
list->tail->next = nnode;
list->tail = nnode;
}
}
void destroy_list_nodes ( llist_node * nodes ) {
llist_node * llnp = NULL;
llist_node * llnpnext = NULL;
llist_node * llnp2 = NULL;
if ( nodes == NULL )
return;
for ( llnp = nodes; llnp != NULL; llnp = llnpnext ) {
llnpnext = llnp->next;
free (llnp);
}
return;
}
void destroy_list ( llist * list ) {
destroy_list_nodes ( list->head );
free (list);
}
int main () {
int i = 0;
int j = 0;
llist * list = new_llist ();
for ( i = 0; i < 100; i++ ) {
for ( j = 0; j < 100; j++ ) {
add_int_tail ( i+j, list );
}
}
printf("enter to continue and free memory...");
getchar();
destroy_list ( list );
printf("memory freed. enter to exit...");
getchar();
printf( "\n");
return 0;
}
If by "the process continues to hold the allocated memory" you mean that ps doesn't report a decrease in the process's memory usage, that's perfectly normal. Returning memory to your process's heap doesn't necessarily make the process return it to the operating system, for all sorts of reasons. If you create and destroy your list over and over again, in a big loop, and the memory usage of your process doesn't grow without limit, then you probably haven't got a real memory leak.
[EDITED to add: See also Will malloc implementations return free-ed memory back to the system? ]
[EDITED again to add: Incidentally, the most likely reason why allocating 512-byte blocks makes the problem go away is that your malloc implementation treats larger blocks specially in some way that makes it easier for it to notice when there are whole pages that are no longer being used -- which is necessary if it's going to return any memory to the OS.]
I discovered the answer to my question here:
http://linuxupc.upc.es/~pep/OLD/man/malloc.html
The memory after expanding the heap can be returned back to kernel if the conditions configured by __noshrink are satisfied. Only then the ps will notice that the memory is freed.
It is important to configure it sometimes particularly when the memory usage is small, but the heap size is bigger than the main memory available. Thus the program trashes even if the required memory is less than the available main memory.
Related
I'm trying to pass in a Professor struct into my professor function but I can't properly get the information stored in it to pass into the function. I suspect it has something to do with how I malloc'd p but I thought freeing after it's completed would solve that problem. I get a segfault when I try to print *professor->id, because apparently it decides to read p as memory location 0x0, even though it's not in main
typedef struct{
int *id;
int *assignings;
int *min_wait;
int *max_wait;
int *min_assignments;
int *max_assignments;
int *min_hrs;
int *max_hrs;
} Professor;
Professor* makeProfessor(){
Professor *professor = malloc(sizeof *professor);
return professor;
}
void * professorFunc(void *p){
Professor *professor = (Professor*)p;
fprintf(stdout,"Starting Professor %d\n", *professor->id);
pthread_exit(0);
}
int main(int argc, char **argv){
//Creating threads
pthread_t professor[num_professors];
Professor *p;
int i;
int id;
for(i = 0; i < num_professors; ++i){
id = i + 1;
p = malloc (sizeof *p);
p->id = &id;
if(pthread_create(&professor[i], NULL, professorFunc, (void*)p) != 0){
perror("pthread_create");
exit(1);
}
free(p);
}
for(i = 0; i < num_professors; ++i){
if(pthread_join(professor[i], NULL) != 0){
perror("pthread_join");
exit(1);
}
}
You are allocating the array of Professor structs, and immediately freeing them, likely before your thread has the chance to operate on them. A better way to implement this, would be to allocate the whole array, process them, and then free the memory, once you know the threads have exited (example below).
pthread_t professor[num_professors];
Professor *p;
int i;
int id;
p = malloc (sizeof(*p) * num_professors);
for(i = 0; i < num_professors; ++i){
id = i + 1;
p->id = &id;
if(pthread_create(&professor[i], NULL, professorFunc, (void*)p) != 0){
perror("pthread_create");
exit(1);
}
}
for(i = 0; i < num_professors; ++i){
if(pthread_join(professor[i], NULL) != 0){
perror("pthread_join");
exit(1);
}
}
free(p);
I want to get 2 programs to communicate, one (server) would store datas, and the other (client) would just access it.
I'll have to use a linked list to store datas because it won't stop storing, and then I was wondering if I could access to the whole linked list if only the first node is shared in memory.
What I mean is… are we allowed to access from the client program to the memory pointed by a shared pointer?
Sorry it seems obvious that we can not, so should I store my linked list into the shared memory, or do you think that would be awkward?
Because if I do so, I'll have to declare a shared memory for every node right?
So, to add shared memory to both programs I need the same keys, but I don't know how many keys there will be, and I can't just store it for both programs, unless I would have had already a linked list…
so I used a very very VERY awkward method that I don't even know if it works right, but I wish you can tell, which is to use ftok that is supposed to take an (url,pid) and return a key. So I assumed it would send the exact same key if I used the same url and pid, using a fake pid starting from 0 that I would increment for every element I add to the linked list… what do you think about it? Any other way to do it which would seem less… crap?
typedef struct s_shared_elem
{
char c;
struct s_shared_elem* next;
struct s_shared_elem* previous;
}shared_elem;
typedef struct s_shared_list
{
s_shared_elem* first;
s_shared_elem* last;
}shared_list;
int forthekey = 0;
char* url="/home/toor/Projet_cgi/";
shared_elem* shared_malloc(int pid, const char* url)
{
shared_elem* shm;
int shmid;
int key=ftok(url,pid);
if((shmid=shmget(key,1,IPC_CREAT | 0666)) < 0)
{
perror("shmget");
exit(1);
}
if ((shm = shmat(shmid,NULL,0)) == (shared_elem*)-1)
{
perror("shmat");
exit(1);
}
return shm;
}
void Init_shared_list(shared_list* liste)
{
liste->first = NULL;
liste->last = NULL;
}
void Add_elem(shared_list* liste)
{
shared_elem* new = shared_malloc(pid,url);
new->next = NULL;
new->previous = liste->last;
if(liste->first == NULL)
{
liste->first = new;
liste->last = new;
}
else
{
liste->last->next = new;
liste->last = new;
}
forthekey++;
}
void shared_free(shared_elem* todelete,int pid, const char* url)
{
shared_elem* shm;
int shmid;
int key=ftok(url,pid);
if((shmid=shmget(key,1,IPC_CREAT | 0666)) < 0)
{
perror("shmget");
exit(1);
}
shmdt(todelete);
shmctl(shmid,IPC_RMID,NULL);
forthekey--;
}
void Delete_list(shared_list* liste)
{
while(liste->last != liste->first)
{
shared_elem* tmp=liste->last;
liste->last=liste->last->previous;
Shared_free(tmp,pid,url);
}
Shared_free(liste->first,pid,url);
}
In share memory you can insert a whole linked list. It is useful in many cases. You do not need to create a linked list of share memory (e.g. using previous key, next key ). All you need to copy each node of linked list to the shared memory.
for example .....
process2.c
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<sys/types.h>
#include<sys/ipc.h>
#include<sys/shm.h>
int main(int argc, char *argv[])
{
int shmid,i;
node *data;
if ((shmid = shmget(10, SHM_SIZE, 0644 | IPC_CREAT)) == -1) {
perror("shmget");
exit(1);
}
data = (node *)shmat(shmid, (void *)0, 0); // node is linked list
for(i=0;i<2;i++)
printf("%d\n",(data++)->item_code);
if (shmdt(data) == -1) {
perror("shmdt");
exit(1);
}
return 0;
}
process1.c
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<sys/types.h>
#include<sys/ipc.h>
#include<sys/shm.h>
int main(int argc, char *argv[])
{
node *SELL=NULL; // node is linked list (structure) SELL is header
insert(&SELL,"Soap",1,12.5,10);
insert(&SELL,"Pen",2,20.75,8);
display(SELL);
int shmid,i;
node *data;
if ((shmid = shmget(10, 2*sizeof(node), 0644 | IPC_CREAT)) == -1) {
perror("shmget");
exit(1);
}
data = (node *) shmat(shmid, (void *)0, 0);
for(i=0;i<2;i++)
{
*(data++)=*SELL;
SELL=SELL->next;
}
getchar();
if (shmdt(data) == -1) {
perror("shmdt");
exit(1);
}
shmctl(shmid, IPC_RMID, NULL);
return 0;
}
Run process1.c 1st then run process2.c
I've seen it said multiple times that there is no way to limit a Lua script's memory usage, including people jumping through hoops to prevent Lua scripts from creating functions and tables. But given that lua_newstate allows you to pass a custom allocator, couldn't one just use that to limit memory consumption? At worst, one could use an arena-based allocator and put a hard limit even on the amount of memory that could be used by fragmentation.
Am I missing something here?
static void *l_alloc_restricted (void *ud, void *ptr, size_t osize, size_t nsize)
{
const int MAX_SIZE = 1024; /* set limit here */
int *used = (int *)ud;
if(ptr == NULL) {
/*
* <http://www.lua.org/manual/5.2/manual.html#lua_Alloc>:
* When ptr is NULL, osize encodes the kind of object that Lua is
* allocating.
*
* Since we don’t care about that, just mark it as 0.
*/
osize = 0;
}
if (nsize == 0)
{
free(ptr);
*used -= osize; /* substract old size from used memory */
return NULL;
}
else
{
if (*used + (nsize - osize) > MAX_SIZE) /* too much memory in use */
return NULL;
ptr = realloc(ptr, nsize);
if (ptr) /* reallocation successful? */
*used += (nsize - osize);
return ptr;
}
}
To make Lua use your allocator, you can use
int *ud = malloc(sizeof(int)); *ud = 0;
lua_State *L = lua_State *lua_newstate (l_alloc_restricted, ud);
Note: I haven't tested the source, but it should work.
I am trying madvise() to mark allocated memory as mergeable so that two applications having same pages can be merged.
While using the madvise() function it shows "invalid argument".
#include<stdio.h>
#include<sys/mman.h>
#include<stdlib.h>
#include<errno.h>
#define ADDR 0xf900f000
int main()
{
int *var1=NULL,*var2=NULL;
size_t size=0;
size = 1000*sizeof(int);
var1 = (int*)malloc(size);
var2 = (int *)malloc(size);
int i=0;
for(i=0;i<999;i++)
{
var1[i] = 1;
}
for(i=0;i<999;i++)
{
var2[i] = 1;
}
i = -1;
while(i<0)
{
i = madvise((void *)var1, size, MADV_MERGEABLE); //to declare mergeable
printf("%d %p\n", i, var1); //to print the output value
err(1,NULL); //to print the generated error
i = madvise((void *)var2, size, MADV_MERGEABLE); //to declare mergeable
printf("%d\n", i);
}
return 0;
}
Error:
a.out: Invalid argument
Please help me.
Thank You.
You can only merge whole pages. You can't merge arbitrary chunks of data.
malloc at Line A will consume more memory than Line B,
why?is it relevant to pthread?
int main()
{
char *buf = (char*)malloc(1024*1024*1024); //Line A
memset(buf,0,sizeof(1024*1024*1024));
pthread_t m_sockThreadHandle[8];
for (int i=0;i<8;i++)
{
if ( pthread_create(&m_sockThreadHandle[i], NULL, thread_run, NULL) != 0 )
{
perror("pthread_create");
}
}
sleep(10);
char *buf = (char*)malloc(1024*1024*1024);//Line B
memset(buf,0,sizeof(1024*1024*1024));
for (int i=0;i<8;i++)
{
pthread_join(m_sockThreadHandle[i],NULL);
}
}
Possibly because this isn't doing what you thought it was:
memset(buf,0,sizeof(1024*1024*1024));
sizeof(1024*1024*1024) is 4 on my compiler. I think you meant:
memset(buf,0, 1024*1024*1024);
From the code you post buf is unused, so it's not clear what you're trying to do, or why. But this at least is wrong....