I'm learning Erlang, and have a Supervisor question...
I have a function which requires 3 parameters (string, string, number). I want to Supervise that and make sure that if it fails, it gets restarted with the 3 parameters I passed it.
Is this something a Supervisor can handle, or do I need to look into some other concept?
Thanks.
Update 1/23/2016
One thing I want to mention... I have a list of 1439 entries. I need to create a Supervisor for each entry in that list. Each entry will incur different arguments. For example, here's some psuedo-code (reminiscent of Ruby):
(360..1799).each do |index|
export(output_path, table_name, index) # Supervise this
end
This is triggered by a user interaction at runtime. The output_path and table_name are dynamic too, but won't change for a given batch. Unravelled, a run may look something like this:
export("/output/2016-01-23/", "temp1234", 360)
export("/output/2016-01-23/", "temp1234", 361)
export("/output/2016-01-23/", "temp1234", 362)
.
.
So if 361 fails, I need it restarted with "/output/2016-01-23/", temp1234, and 361.
Is this something I can do with a Supervisor?
Yes, this is what supervisor does, but you mean "arguments", not "parameters".
For ordinary (not simple_one_for_one) supervisors your init/1 implementation returns a list of so called child specifications, each of which specifies a child that will be spawned, and the arguments that will be passed are provided as part of this child specification.
With simple_one_for_one supervisors you still have to provide a child specification, but you provide the arguments when you start each supervised child.
In all cases your supervised children will be restarted with the same arguments.
General Concepts
Erlang systems are divided into modules.
Each module are composed of functions.
Works are done by processes and they use functions to do it.
A supervisor is a process which supervises other processes.
So your function will sure be executed inside a process, and you can specify and supervise that process by a supervisor. This way when your function fails and crashes, the process that are executing it will crash as well. When the process crashes, its supervisor will find out and can act upon it based on a predefined restart strategy. These strategies could be one_for_one, one_for_all and rest_for_one.
You can check its manual for further options.
Related
What is the difference between a series of mnesia:dirty_ commands executed within a function passed to mnesia:async_dirty() and those very same transactions executed "raw"?
I.e., is there any difference between doing:
mnesia:dirty_write({table, Rec1}),
mnesia:dirty_write({table, Rec1}),
mnesia:dirty_write({table, Rec1})
and
F = fun() ->
mnesia:dirty_write({table, Rec1}),
mnesia:dirty_write({table, Rec1}),
mnesia:dirty_write({table, Rec1})
end,
mnesia:async_dirty(F)
Thanks
Lets first quote the Users' Guide on async_dirty context:
By passing the same "fun" as argument to the function mnesia:async_dirty(Fun [, Args]) it will be performed in dirty context. The function calls will be mapped to the corresponding dirty functions.This will still involve logging, replication and subscriptions but there will beno locking, local transaction storage or commit protocols involved. Checkpointretainers will be updated but will be updated "dirty". Thus, they will be updatedasynchronously. The functions will wait for the operation to be performed on one node but not the others. If the table resides locally no waiting will occur.
The two options you have provided will be executed the same way. However, when you execute the dirty functions out of the fun as in option one, each one is a separate call into mnesia. With async_dirty, the 3 calls will be bundled and mnesia will only wait until the 3 are completed on the local node to return. However, the behavior of these two may differ in a mnesia multi-node cluster. Do some tests :)
Suppose a Pid is registered as follows.
register(foobar, Pid).
Now I want to replace the associated pid:
unregister(foobar),
register(foobar, NewPid).
How can I achieve this atomically?
Use gproc, https://github.com/uwiger/gproc
The advantage is that its registry is an ETS table and ETS tables have atomic updates where you can overwrite a name atomically like the thing you want. I am almost positive it can do this kind of thing.
I don't think this is possible, at least, using the register/2 and unregister/1 BIFs.
You need to serialize requests to the registry, for example using a gen_server or an ETS table.
Also, consider the following. Registered names for processes are atoms and atoms, in the Erlang VM, are limited and not garbage collected. If you're registering/unregistering processes dynamically a huge number of processes (e.g. one process per request) you might want to re-think to this approach, since you might run out of atoms at some point.
Given two Erlang nodes, "foo#host" and "bar#host", the following produces a print-out on "foo":
(foo#host) rpc:call('bar#host', io, format, ["~p", [test]]).
While the following prints out on "bar":
(foo#host) rpc:call('bar#host', erlang, display, [test]).
Even if erlang:display/1 is supposed to be used for debug only, both functions are supposed to send stuff to the standard output. Each process should inherit the group leader from its parent, so I would expect that the two functions would behave in a consistent way.
Is there any rationale for the above behaviour?
The reason for this difference in behaviour is where and by whom the output is done:
erlang:display/1 is a BIF and is handled directly by the BEAM which writes it straight out to its standard output without going anywhere near Erlang's io-system. So doing this on bar results in it printed to bar's standard output.
io:format/1/2 is handled by the Erlang io-system. As no IoDevice has been given it sends an io-request to its group leader. The rpc:call/4 is implemented in such away that the remotely spawned process inherits the group leader of the process doing the RPC call. So the output goes to the standard output of the calling process. So doing an RPC call on foo to the node bar results in the output going to foo's standard output.
Hence the difference. It is interesting to note that no special handling of this is needed in the Erlang io-system, once the group leader has been set it all works transparently.
I need to start multiple supervisor children in an atomic way. That is, if one of children in group fails at startup then none of them should be started.
I see this operation as a function:
start_children(SupRef, ChildSpecs)
ChildSpecs = List :: [child_spec()]
How should I implement this in a proper way? Any examples, libraries etc.? My intuition tells me that starting all children from the list, checking if all of them were successful and then killing remaining ones is not the way.
Or perhaps my design is flawed and I really should not need to do such things?
OTP's supervisor provides support for this with the one_for_all strategy. By default, if some process fails, all processes are restarted, but you can change this by using a suitable for your purpose Restart parameter (e.g. temporary).
Let's say we implement Pregel with Erlang. Why do we actually need supersteps? Isn't it better to just send messages from one supervisor to processes that represent nodes? They could just apply the calculation function to themselves, send messages to each other and then send a 'done' message to the supervisor.
What is the whole purpose of supersteps in concurrent Erlang implementation of Pregel?
The SuperStep concept as espoused by the Pregel model could be viewed as sort of a Barrier for parallel-y executing entities. At the end of each superstep, each worker, flushes it state to the persistent store.
The algorithm is check-pointed at the end of each SuperStep so that in case of failure, when a new node has to take over the function of a failed peer, it has a point to start from. Pregel guarantees that since the data of the node has been flushed to disk before the SuperStep started, it can reliably start from exactly that point.
It also in a way signifies "progress" of the algorithm. A pregel algorithm/job can be provided with a "max number of supersteps" after which the algorithm should terminate.
What you specified in your question (about superisors sending worker a calculation function and waiting for a "done") can definitely be implemented (although I dont think the current supervisor packaged with OTP can do stuff like that out of the box) but I guess the concept of a SuperStep is just a requirement of a Pregel model. If on the other hand, you were implementing something like a parallel mapper (like what Joe implements in his book) you wont need supersteps/