I'm trying to figure out how NPTL cleans up the resources (stack space, etc) of a
detached thread when it exits. Joinable threads are easy, there's a call to pthread_join
which waits for a specific thread to exit and then reclaims its resources. No problem, but
how does NPTL know that a detached thread has exited? Does anyone know or at least know who
would know?
thanks,
Rich
after the thread function returns, NPTL will check whether the thread is detached, if it is detached, it will free everything of the thread directly, if it is joined, it will leave thread control block(TCB) to pthread_join to free TCB.
for detail, http://raison.gegahost.net/?p=91 may helps.
Related
Is it possible to use mutex from pthread.h to synchronize processes created with fork() from unistd.h? Afaik, both in the end are using system call clone().
I am asking it in the scope of shared memory segment (from ipc.h, shm.h) with critical data, which should be protected against concurrent writes from different processes. In that memory then semaphores can be defined and later used in different processes. Why couldn't mutexes be used instead of semaphores?
Why am I asking?
First of all I was told that it won't work, without receiving any explanation for that. On the Internet I was not able to find any answer so I decided to ask here.
Second, forked process is safer than thread created with pthread_create - if forked process crashes, the rest of the program continues to work and if thread crashes then whole program exits.
Third, mutexes seem to be more human-friendly than semaphores in managing.
This is quite a general computer science question and not specific to any OS or framework.
So I am a little confused by the overhead associated with switching tasks on a thread pool. In many cases it doesn't make sense to give every job its own specific thread (we don't want to create too many hardware threads), so instead we put these jobs into tasks which can be scheduled to run on a thread. We setup up a pool of threads and then dynamically allocate the tasks to run on a thread taken from the thread pool.
I am just a little confused (can't find a in depth answer) on the overhead associated with switching tasks on a specific thread (in the thread pool). A DrDobbs article (sourced below) states it does but I need a more in depth answer to what is actually happening (a cite-able source would be fantastic :)).
By definition, SomeWork must be queued up in the pool and then run on
a different thread than the original thread. This means we necessarily
incur queuing overhead plus a context switch just to move the work to
the pool. If we need to communicate an answer back to the original
thread, such as through a message or Future or similar, we will incur
another context switch for that.
Source: http://www.drdobbs.com/parallel/use-thread-pools-correctly-keep-tasks-sh/216500409?pgno=1
What components of the thread are actually switching? The thread itself isn't actually switching, just the data that is specific to the thread. What is the overhead associated with this (more, less or the same)?
let´s clarify first 5 key concepts here and then discuss how they correlates in a thread pool context:
thread:
In a brief resume it can be described as a program execution context, given by the code that is being run, the data in cpu registries and the stack. when a thread is created it is assigned the code that should be executed in that thread context. In each cpu cycle the thread has an instruction to execute and the data in cpu registries and stack in a given state.
task:
Represents a unit of work. It's the code that is assigned to a thread to be executed.
context switch (from wikipedia):
Is the process of storing and restoring the state (context) of a thread so that execution can be resumed from the same point at a later time. This enables multiple processes to share a single CPU and is an essential feature of a multitasking operating system. What constitutes the context is as explained above is the code that is being executed, the cpu registries and the stack.
What is context switched is the thread. A task represents only a peace of work that can be assigned to a thread to be executed. At given moment a thread can be executing a task.
Thread Pool (from wikipedia):
In computer programming, the thread pool is where a number of threads are created to perform a number of tasks, which are usually organized in a queue.
Thread Pool Queue:
Where tasks are placed to be executed by threads in the pool. This data structure is a shared peace of memory where threads may compete to queue/dequeue, may lead to contention in high load scenarios.
Illustrating a thread pool usage scenario:
In your program (eventually running in the main thread), you create a task and schedules it to be executed in thread pool.
The task is queued in the thread pool queue.
When a thread from the pool executes it dequeues a task from the pool and starts to executed it.
If there is no free cpus to execute the thread from the pool, the operating system at some point (depending on thread scheduler policy and thread priorities) will stop a thread from executing, context switching to other thread.
the operating system can stop the execution of a thread at any time, context switching to another thread, returning latter to continue where it stopped.
The overhead of the context switching is augmented when the number of active threads that competes for cpus grows. Thus, ideally, a thread pool tries to use the minimum necessary threads to occupy all available cpus in a machine.
If your tasks haven't code that blocks somewhere, context switching is minimized because it is used no more threads than the available cpus on machine.
Of course if you have only one core, your main thread and the thread pool will compete for the same cpu.
The article probably talks about the case in which work is posted to the pool and the result of it is being waited for. Running a task on the thread-pool in general does not incur any context switching overhead.
Imagine queueing 1000 work items. A thread-pool thread will executed them one after the other. All of that without a single context switch in between.
Switching happens doe to waiting/blocking.
Is there a foolproof way to automatically release mutexes held by a thread when that thread is exiting (in its destructor)?
The approach I have been taking is to create a structure for each mutex which hold the identity of the thread that holds it, and then in the destructor to scan through this list and if any mutexes match the thread being finished, to release it then. But I'm thinking that this actually has a race condition: what happens if after I lock the mutex but before I set the data structure the destructor is called?
I've also looked at pthread_mutexattr_setrobust_np, but my understanding is that np functions are non-portable, and I have had issues with that in the past.
For reference, each thread is associated with a TCP/IP connection, and locking/unlocking occurs in response to requests over this connection. If the connection abnormally closes I need to clean up i.e. release any locks held.
I found a solution which appears to work. First, I use an error checking mutex (PTHREAD_ERRORCHECK_MUTEX_INITIALIZER or PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP).
Next, in the destructor, I trying to unlock all mutexes, with the idea being any mutex not owned by the thread will be left alone, but any mutex owned by the thread will be released.
For some reason even mutexes owned by the thread return EPERM, but a subsequent attempt to re-lock the mutex from another thread succeeds whereas without trying to unlock another attempt will deadlock. Conversely, other mutexes not owned by the destructed thread are still found to be locked after the destructor runs.
I want to create a lot of threads for a writing into a thread, and after writing I call exit... But, when I call exit do I free up the stack or do I still consume it??
In order to avoid resource leaks, you have to do one of these 2:
Make sure some other thread call pthread_join() on the thread
Create the thread as 'detached', which can either be done by setting the proper pthread attribute to pthread_create, or by calling the pthread_detach() function.
Failure to do so will often result in the entire stack "leaking" in many implementations.
The system allocates underlying storage for each thread, (thread ID, thread retval, stack), and this will remain in the process space (and not be recycled) until the thread has terminated and has been joined by other threads.
If you have a thread which you don't care how the thread terminates, and a detached thread is a good choice.
For detached threads, the system recycles its underlying resources automatically after the thread terminates.
source article: http://www.ibm.com/developerworks/library/l-memory-leaks/
I have multithreaded application and I've got a little problem when application ends: I can correctly terminate the thread by calling TThread.Terminate method in Form1.OnDestroy event handler, but the termination does take some time and so I can't free the memory (by TThread.Free method).
Unfortunately for some other reason I must have TThread.FreeOnTerminate property set to false, so the thread object isn't destroyed automatically after thread termination.
My question is probably a little silly and I should have known it a long time ago, but is this ok and the thread will be destroyed automatically (since the application just ends), or is it a problem and the memory would be "lost"? Thanks a lot for explanation.
You should wait for the thread to terminate before you begin the process off shutting down the rest of your application, otherwise shared resources may be freed under the threads feet, possibly leading to a string of access violations. After you have waited for thread termination, then you can free it. In fact, that's what the TThread destructor does for you.
If there are no shared resources, then sure, let it die by itself. Even if the thread terminates after the main thread, all that is required is that all your threads exit for the program to terminate. Any memory associated with the thread's object will just get cleaned up and given back to the OS with everything else.
BUT, be careful! If your thread is taking a while to exit, it can lead to a zombie process sitting there churning away without a GUI. That is why it is very important to check the Terminated flag very often in the thread loop, and exit the thread.
N#
Your question is not silly or simple - read the MSDN article. All in all, if you want to be on the safe side you are better to wait a background thread to terminate before exiting an application.
The thread will eventually terminate and Windows will clean up any memory left over. However, you might as well just wait for the thread to terminate, because that is exactly what Windows will do anyway. Your application may appear to have shut down because all windows may have been closed/hidden, but the application process won't terminate until all threads have finished...
When a process terminates the OS will reclaim all allocated memory and will close all open handles. You don't need to worry about MEMORY*) that leaks in the very special event of shutting down the application. The OS will also close all your open handles**), at least theoretically. All those taken into account, it might be safe for you to simply terminate your thread (using TerminateThread(MyThread.Handle)) from your forms destructor, before killing other shared resources. Ask yourself those questions:
What's the thread doing? Is it safe to terminate it at any time? Example: If the thread is doing any writing to disk, it's unsafe to just kill it, because you might live files on disk in an inconsistant state.
Are you using any resources that aren't automatically freed by Windows? Can't think of an good example here...
If you're on the safe side with both, you can use TerminateThread and not wait for the thread to naturally terminate. A safer approach might a combined approach, maybe you should give the thread a chance to naturally terminate and, if it didn't terminate withing 5 seconds, force-terminate it.
*) I'm talking about memory you can prove only leaks on process termination, like threads you kill without giving them a chance to properly shut down, or global singleton classes you don't free. All other unaccounted memory needs to be tracked down and fixed, because it's an bug.
**) Unfortunately the Windows OS is not bug-free. Example: Anyone that worked with serial devices on the Windows platform knows how easy it is to get the serial-device in a "locked" state, requiring an restart to get it working again. Technically that's also an Handle, end-processing the application that locked it should unlock it.
why you dont increment a variable when creating the thread, and on the destroy event wait until thread finish, decrement the variable, and on applicationterminate just do Application.processmessages ?
why your thread isn't freeonterminate=true ? all shared resources can be handled into a critical section.
best regards,