I have an IPAD program designed to torture the thread creation process,
which is failing unexpectedly after a few thousand iterations of a
process like this:
pthread_create [a thread that does nothing and exits very quickly]
sleep(100);
Eventually, pthread_create fails with error 35 "too many threads"
The failure occurs, eventually, no matter how long the sleep interval.
Is there some arbitrary limit to threads over time, or is there some
resource I'm likely to be consuming without realizing it?
You need to either call pthread_join from another thread to reclaim the resources, or call pthread_detach to indicate that the resources should be reclaimed automatically. Like unix processes, threads have a result that needs to be reaped before it can completely go away.
Related
Is it possible to use mutex from pthread.h to synchronize processes created with fork() from unistd.h? Afaik, both in the end are using system call clone().
I am asking it in the scope of shared memory segment (from ipc.h, shm.h) with critical data, which should be protected against concurrent writes from different processes. In that memory then semaphores can be defined and later used in different processes. Why couldn't mutexes be used instead of semaphores?
Why am I asking?
First of all I was told that it won't work, without receiving any explanation for that. On the Internet I was not able to find any answer so I decided to ask here.
Second, forked process is safer than thread created with pthread_create - if forked process crashes, the rest of the program continues to work and if thread crashes then whole program exits.
Third, mutexes seem to be more human-friendly than semaphores in managing.
In my process i have created 10 threads and will use those threads till my application is alive. Each thread will perform some file input and output operation every time. So the problem is every time thread start executing then my process virtual memory is getting increased.
My analysis is that when one file input output task is allowcated to the thread then the file will be loaded to thread address space when thread start to copy the file and after copy is completed then the thread address space will not be cleared as still the thread is not exited. So if i once again assign another task to the thread then the new file will be loaded to the thread address space.
Hence the main process virtual memory address space will be increase. SO Please correct me if i am wrong and also help to know this has some problem if the process run for log time.
A few things here.
1) Threads do not have their own memory address space. Processes do. (However, threads do get their own thread local storage.)
2) In managed languages, objects are not cleaned up and the heap compacted until the garbage collector is run. The garbage collector is not run until it needs to (e.g. the program is close to running out of memory). As long as the object has no strong references to it (nothing running can reach it) then the object will get cleaned up when the program needs it to be cleaned up, and you don't need to do anything else. If you want the garbage collector to run early, however, tell it to.
By the way, if resources are needed commonly amongst many different threads, you could consider having some sort of global cache for them. However, early optimization is a grievous sin, so don't go to all that effort until you've determined it solves a REAL problem.
pthread_mutex_trylock detects deadlocks, doesn't block, then why would you even "need" pthread_mutex_lock?
Perhaps when you deliberately want the thread to block? But in that case it may result in a deadlock?
pthread_mutex_trylock does not detect deadlocks.
You can use it to avoid deadlocks but you have to do that by wrapping your own code around it, effectively multiple calls to pthread_mutex_trylock in a loop with a time-out, after which your thread releases all its resources.
In any case, you can avoid deadlocks even with pthread_mutex_lock if you just follow the simple rule that all threads allocate resources in the same order.
You use pthread_mutex_lock if you just want to efficiently wait until the resource is available, without having to spin on the mutex, something which is often very inefficient. Properly designed multi-threaded applications have no need for the pthread_mutex_trylock variant.
Locks should only be held for the absolute minimum time to do the work and, if that's too long, you can generally redesign things so the lock time is less (such as by using the mutex to only copy data to a thread's local data areas, and having the long-running bit work on that after the mutex is released).
The pseudo-code:
while not pthread_mutex_trylock:
yield
will continue to run your thread, waiting for the lock to be available, especially since there is no pthread_yield() in POSIX threads (though it's sometimes provided as a non-portable extension).
That means, at worst, the code segment above won't even be able to portably yield the CPU, therefore chewing up the rest of it's quantum every time through the scheduler cycle.
And at best, it will still activate the thread once per scheduler cycle just to see if the mutex can be obtained.
Whereas:
pthread_mutex_lock
will most likely totally pause your thread until the lock is made available, since it will move it to a waiting queue until the current lock holder releases the mutex.
That's probably the major reason why you should prefer pthread_mutex_lock to pthread_mutex_trylock.
Perhaps when you deliberately want the thread to block?
Yup, exactly in this case. But you can mimic pthread_mutex_lock() behavior with something like that
while(pthread_mutex_trylock(&mtx))
pthread_yield()
I want to create a lot of threads for a writing into a thread, and after writing I call exit... But, when I call exit do I free up the stack or do I still consume it??
In order to avoid resource leaks, you have to do one of these 2:
Make sure some other thread call pthread_join() on the thread
Create the thread as 'detached', which can either be done by setting the proper pthread attribute to pthread_create, or by calling the pthread_detach() function.
Failure to do so will often result in the entire stack "leaking" in many implementations.
The system allocates underlying storage for each thread, (thread ID, thread retval, stack), and this will remain in the process space (and not be recycled) until the thread has terminated and has been joined by other threads.
If you have a thread which you don't care how the thread terminates, and a detached thread is a good choice.
For detached threads, the system recycles its underlying resources automatically after the thread terminates.
source article: http://www.ibm.com/developerworks/library/l-memory-leaks/
I have multithreaded application and I've got a little problem when application ends: I can correctly terminate the thread by calling TThread.Terminate method in Form1.OnDestroy event handler, but the termination does take some time and so I can't free the memory (by TThread.Free method).
Unfortunately for some other reason I must have TThread.FreeOnTerminate property set to false, so the thread object isn't destroyed automatically after thread termination.
My question is probably a little silly and I should have known it a long time ago, but is this ok and the thread will be destroyed automatically (since the application just ends), or is it a problem and the memory would be "lost"? Thanks a lot for explanation.
You should wait for the thread to terminate before you begin the process off shutting down the rest of your application, otherwise shared resources may be freed under the threads feet, possibly leading to a string of access violations. After you have waited for thread termination, then you can free it. In fact, that's what the TThread destructor does for you.
If there are no shared resources, then sure, let it die by itself. Even if the thread terminates after the main thread, all that is required is that all your threads exit for the program to terminate. Any memory associated with the thread's object will just get cleaned up and given back to the OS with everything else.
BUT, be careful! If your thread is taking a while to exit, it can lead to a zombie process sitting there churning away without a GUI. That is why it is very important to check the Terminated flag very often in the thread loop, and exit the thread.
N#
Your question is not silly or simple - read the MSDN article. All in all, if you want to be on the safe side you are better to wait a background thread to terminate before exiting an application.
The thread will eventually terminate and Windows will clean up any memory left over. However, you might as well just wait for the thread to terminate, because that is exactly what Windows will do anyway. Your application may appear to have shut down because all windows may have been closed/hidden, but the application process won't terminate until all threads have finished...
When a process terminates the OS will reclaim all allocated memory and will close all open handles. You don't need to worry about MEMORY*) that leaks in the very special event of shutting down the application. The OS will also close all your open handles**), at least theoretically. All those taken into account, it might be safe for you to simply terminate your thread (using TerminateThread(MyThread.Handle)) from your forms destructor, before killing other shared resources. Ask yourself those questions:
What's the thread doing? Is it safe to terminate it at any time? Example: If the thread is doing any writing to disk, it's unsafe to just kill it, because you might live files on disk in an inconsistant state.
Are you using any resources that aren't automatically freed by Windows? Can't think of an good example here...
If you're on the safe side with both, you can use TerminateThread and not wait for the thread to naturally terminate. A safer approach might a combined approach, maybe you should give the thread a chance to naturally terminate and, if it didn't terminate withing 5 seconds, force-terminate it.
*) I'm talking about memory you can prove only leaks on process termination, like threads you kill without giving them a chance to properly shut down, or global singleton classes you don't free. All other unaccounted memory needs to be tracked down and fixed, because it's an bug.
**) Unfortunately the Windows OS is not bug-free. Example: Anyone that worked with serial devices on the Windows platform knows how easy it is to get the serial-device in a "locked" state, requiring an restart to get it working again. Technically that's also an Handle, end-processing the application that locked it should unlock it.
why you dont increment a variable when creating the thread, and on the destroy event wait until thread finish, decrement the variable, and on applicationterminate just do Application.processmessages ?
why your thread isn't freeonterminate=true ? all shared resources can be handled into a critical section.
best regards,