When using simple_one_to_one in a supervisor, the children haven't started automatically. By the examples, I should run supervisor:start_child(chat_liason_sup, DataForChild). to start a child. In the real world, we want to execute the erlang application, and not sit at the console to start a child. How can I get at least one child to start automatically? One trick I considered was to have a normal one_for_one started at the same time, which would, in turn, start the simple_one_for_one children. But it looks like I can only start one kind of child in a given supervisor...
I use the simple_one_for_one strategy when I need a process factory. Generally, I have a first supervisor using a one_for_one or one_for_all strategy, and whose children are
a server responsible for a part of the application, and whose job includes the start of multiple and identical processes
the simple_one_for_one supervisor.
Doing this I take advantage of the OTP to manage the life cycle of all the processes.
If you want to start more than one kind of child, do not use simple_one_for_one. supervisor:start_child/2 accepts child_spec() for other types of supervisor. If you want to start child automatically, use another process and more than one supervisor. One way is to use one supervisor for a launcher and second simple_one_for_one supervisor for children. But you can use as many supervisors as you want.
Related
I'm working with Elixir but I believe this question applies to Erlang as well.
I'm working on a system which might create dozens of thousands of groups processes of same kind. Each group will have 2 workers and a local supervisor of its own. The question is who will supervise the local supervisors ?
I can imagine two strategies
one big supervisor that will handle all local supervisors. This method is simple yet I believe the supervisor will need to traverse its huge list of children whenever something happens to a child which will be a heavy operation.
a partitioned tree. Say for example a set of intermediate supervisors supervising about 1000 local supervisors, then a global supervisor handling the intermediate ones. To create a new group, the global supervisor will need to find the intermediate supervisor with least children and delegate to that one the creation.
Does either make sense or is there any other way? Any advice is welcome
"It depends".
"huge list" and "thousands" really are in different realms. Simple iteration is fast on modern machines. Up to high five, low six items I would have no qualms with a system that regularly has to traverse a list this size, and probably over that I wouldn't really care either:
iex(2)> list = Enum.to_list 1..1_000_000; :timer.tc(fn -> Enum.sum list end)
{24497, 500000500000}
(that is 25 ms for the list traversal and some arithmetic - I'm usually happy if a crashed process gets restarted with such small delays)
Of course - at the end of the day you're expected to do your own performance testing, compare the outcomes with the expected local supervisor crash rate, look up your system's requirements, and compare all these figures to come to an answer.
In the meantime, use the simplest thing that can possibly work: a single global supervisor monitoring a flat hierarchy.
The approach one is perfectly efficient. The global supervisor would not need to traverse anything as soon as any subgroup has it’s own local supervisor and the latter it not intended to crash.
When something will happen with the leaf worker, this local supervisor will take care about restarting it, and the global supervisor wouldn’t even know that something wrong happened there down in the tree.
If, OTOH, you expect your local supervisors to be crashed from time to time on purpose, each local supervisor should be supervised with it’s own, say, intermediate supervisor, which will take care of it’s restarts. The global supervisor will in this case manage these intermediate supervisors, and everything will be cool again.
Use director with ETS mode and don't worry about number of children.
In ETS mode, you can read some info about children directly from Table too.
Can someone explain what's the difference between gen_server:start() and gen_server:start_link()?
I've been told that it's something about multi threading stuff.
EDIT:
If my gen_server is called from multiple threads, will it execute them all at once ? Or will it create concurrency between these threads?
Both functions start new gen_server instances as children of the calling process, but they differ in that the gen_server:start_link/3,4 atomically starts a gen_server child and links it to its parent process. Linking means that if the child dies, the parent will by default also die. Supervisors are parent processes that use links to take specific actions when their child processes exit abnormally, typically restarting them.
Other than the linking involved in the gen_server:start_link case, there are no multi-process aspects involved in these calls. Regardless of whether you use gen_server:start or gen_server:start_link to start a new gen_server, the new process has a single message queue, and it receives and processes those messages one at a time. There is nothing about gen_server:start_link that causes the new gen_server process to behave or perform differently than it would if started with gen_server:start.
When you use gen_server:start_link new process becomes "child" of calling process - it's part of supervision tree. It allows calling process to be notified if gen_server process dies.
Using gen_server:start will spawn process outside of supervision tree.
Nice description of supervision in Erlang is here: http://learnyousomeerlang.com/supervisors
I have number of supervised components that can stand alone as separate applications. But I would like to cascade them such that a call or event in a worker in one component starts the next component down in an inverted tree-like structure.
1) Can I package each of these components as separate applications?
2) If so,how do I write the calling code to start the child application?
3) Or do I need to do something else altogether and,if so, what?
Note: I'm still working on mastery of supervision trees. The chain of events following application:start(Mod) is still not burned well into my head.
Many thanks,
LRP
Supervision trees and applications are complex Erlang/OTP concepts. They are both documented in OTP Design Principles User's Guide and in particular in:
chapter 5: Supervisor behaviour
chapter 7: Applications.
Supervision trees are not dependency trees and should not be designed as such. Instead, a supervision tree should be defined based on the desired crash behavior, as well as the desired start order. As a reminder, every process running long enough must be in the supervision tree.
An application is a reusable component that can be started and stopped. Applications can depend on other applications. However, applications are meant to be started at boot-time, and not when an event occurs.
Processes can be started when a given event occurs. If such processes shall be supervised, simply call supervisor:start_child/2 on its supervisor when the event occurs. This will start the process and it will be inserted in the supervision tree. You will typically use a Simple-one-for-one supervisor which will initially have no child.
Consequently:
You can package components as separate applications. In this case, you will declare the dependencies of applications in each application's app(4) file. Applications could then only be started in the proper order, either with a boot script or interactively with application:start/1.
You can package all your components in a single application and have worker processes starting other worker processes with supervisor:start_child/2.
You can package components as separate applications and have worker processes in one application starting processes in another application. In this case, the best would be to define a module in the target application that will call supervisor:start_child/2 itself, as applications should have clean APIs.
When you have worker processes (parents) starting other worker processes (children), you probably will link those processes. Linking is achieved with link/1. Links are symmetric and are usually established from the parent since the parent knows the pid of the child. If the parent process exits abnormally, the child will be terminated, and reciprocally.
Links are the most common way to handle crashes, for example a child shall be terminated if the parent is no longer there. Links are actually the foundation of OTP supervision. Adding links between worker processes reveals that designing supervision trees is actually difficult. Indeed, with links, you will have both processes terminating if one crashes, and yet, you probably do not want the child process to be restarted by the supervisor, as a supervisor-restarted child process will not be known (or linked) to a supervisor-restarted parent process.
If the parent shall terminate when the child exits normally, then this is a totally different design. You can either have the child send a message to the parent (e.g. return a result) or the parent monitor the child.
Finally, the parent process can terminate a child process. If the child is supervised, use supervisor:terminate_child/2. Otherwise, you can simply send an exit signal to the child process. In either cases, you will need to unlink the child process to avoid an exit of the parent process.
Both links and monitors are documented in the Erlang Reference Manual User's Guide. Instead of monitors, you might be tempted to trap exits, something explained in the guide. However, the manual page for the function to achieve this (process_flag/2) specifically reads:
Application processes should normally not trap exits.
This is typical OTP design wisdom, spread here and there in the documentation. Use monitors or simple messages instead.
Is it possible to start a supervisor module in ejabberd which I can add a gen_fsm module to, per connection?
Specifically, I want to create a supervisor which I start when the server starts (or when connections come in). And I have a couple of gen_servers which I want to start, but the part I'm looking for some guidance on is how to dynamically add a gen_fsm module to my supervisor when I see this user's presence become available?
You might want to have a look to the Simple one for on supervisor, which:
is a simplified one_for_one supervisor, where all child processes are
dynamically added instances of the same process
...
When started, the supervisor will not start any child processes. Instead, all child
processes are added dynamically by calling:
supervisor:start_child(Sup, List)
...
Basically, you use this kind of supervisors when:
All the children are of the same type
You want to add children dynamically
Which appears to be your case.
in Erlang I have a supervisor-tree of processes, containing one that accepts tcp/ip connections. For each incoming connection I spawn a new process. Should this process be added to the supervisor tree or not?
Regards,
Steve
Yes, you should add these processes to the supervision heirarchy as you want them to be correctly/gracefully shutdown when your application is stopped. (Otherwise you end up leaking connections that will fail as the application infrastructure they depend on been shutdown).
You could create a simple_one_for_one strategy supervisor say yourapp_client_sup that has a child spec of {Id, {yourapp_client_connection, start_link_with_socket, []}, Restart, Shutdown, worker, temporary}. The temporary type here is important because there's normally no useful restart strategy for a connection handler - you can't connect out to the client to restart the connection. temporary here will cause the supervisor to report the connection handler exit but otherwise ignore it.
The process that does gen_tcp:accept will then create the connection handler process by doing supervisor:start_child(yourapp_client_sup, [Socket,Options,...]) rather than yourapp_client_sup:start_link(Socket, Options, ...). Ensure that the youreapp_client_connection:start_link_with_socket function starts the child via gen_server or proc_lib functions (a requirement of the supervisor module) and that the function transfers control of the socket to the child with gen_tcp:controlling_process otherwise the child won't be able to use the socket.
An alternate approach is to create a dummy yourapp_client_sup process that yourclient_connection_handler processes can link to at startup. The yourapp_client_sup process will just exist to propagate EXIT messages from its parent to the connection handler processes. It will need to trap exists and ignore all EXIT messages other than those from its parent. On the whole, I prefer to use the simple_one_for_one supervisor approach.
If you expect these processes to be many, it could be a good idea to add a supervisor under your main supervisor as to separate responsibility (and maybe use the simple_one_for_one setting to make things simpler, maybe even simpler than your current case).
The thing is, if you need to control these processes, it's always nice to have a supervisor. If it doesn't matter if they succeed or not, then you might not need one. But then again, I always argue that that is sloppy coding. ;-)
The only thing I wouldn't do, is to add them to your existing tree, unless it is very obvious where they come from and they're fairly few.