Erlang supervisor dynamic change to restart intensity - erlang

My question is, can one modify the restart intensity thresholds of an already running supervisor, apart from in a release upgrade scenario, and if so, how?
It's never come up before, but running a supervisor with initially no children, so that another process starts children by way of supervisor:start_child/2, so my sup init/1 being like this:
init([]) ->
RestartSt = {simple_one_for_one, 10, 10},
ChSpec = [foo, {foo,start_link,[]}, transient, 1000, worker, [foo]}],
{ok, {RestartSt, ChSpec}}.
At the time of supervisor start, the likely number of children is unknown; certainly it could vary dramatically from 10, to 10,000, or more.
A restart intensity of say 20 is generous enough for 10 children, but for say 10,000 children I would like to be able to increase it... and decrease it as the number of children drops due to normal terminations.

There's no API for doing this, so I believe you're stuck with the upgrade approach unless you want to propose a new API for this to the OTP team by submitting a pull request providing a complete patch with code changes, new tests, and documentation changes.
There's also a really dirty hack way of doing this that involves manipulating internal supervisor state, and so it's absolutely not something I would recommend for a production system but I think it's still interesting to look at. A supervisor stores restart intensity in its internal loop state. You can see this state by calling sys:get_state/1,2 on a supervisor process. For example, here's the state of a supervisor in the Yaws web server:
1> rr(supervisor).
[child,state]
2> sys:get_state(yaws_sup).
#state{name = {local,yaws_sup},
strategy = one_for_all,
children = [#child{pid = <0.67.0>,name = yaws_sup_restarts,
mfargs = {yaws_sup_restarts,start_link,[]},
restart_type = transient,shutdown = infinity,
child_type = supervisor,
modules = [yaws_sup_restarts]},
#child{pid = <0.42.0>,name = yaws_server,
mfargs = {yaws_server,start_link,
[{env,true,false,false,false,false,false,"default"}]},
restart_type = permanent,shutdown = 120000,
child_type = worker,
modules = [yaws_server]},
#child{pid = <0.39.0>,name = yaws_trace,
mfargs = {yaws_trace,start_link,[]},
restart_type = permanent,shutdown = 5000,
child_type = worker,
modules = [yaws_trace]},
#child{pid = <0.36.0>,name = yaws_log,
mfargs = {yaws_log,start_link,[]},
restart_type = permanent,shutdown = 5000,
child_type = worker,
modules = [yaws_log]}],
dynamics = undefined,intensity = 0,period = 1,restarts = [],
module = yaws_sup,args = []}
The initial rr command retrieves the record definitions from supervisor so we can see the field names when we get the state from yaws_sup, otherwise we would just get a tuple full of anonymous values.
The retrieved state shows the intensity in this case to be 0. We can change it using sys:replace_state/2,3:
3> sys:replace_state(yaws_sup, fun(S) -> S#state{intensity=2} end).
#state{name = {local,yaws_sup},
strategy = one_for_all,
children = [#child{pid = <0.67.0>,name = yaws_sup_restarts,
mfargs = {yaws_sup_restarts,start_link,[]},
restart_type = transient,shutdown = infinity,
child_type = supervisor,
modules = [yaws_sup_restarts]},
#child{pid = <0.42.0>,name = yaws_server,
mfargs = {yaws_server,start_link,
[{env,true,false,false,false,false,false,"default"}]},
restart_type = permanent,shutdown = 120000,
child_type = worker,
modules = [yaws_server]},
#child{pid = <0.39.0>,name = yaws_trace,
mfargs = {yaws_trace,start_link,[]},
restart_type = permanent,shutdown = 5000,
child_type = worker,
modules = [yaws_trace]},
#child{pid = <0.36.0>,name = yaws_log,
mfargs = {yaws_log,start_link,[]},
restart_type = permanent,shutdown = 5000,
child_type = worker,
modules = [yaws_log]}],
dynamics = undefined,intensity = 2,period = 1,restarts = [],
module = yaws_sup,args = []}
Our second argument to sys:replace_state/2 takes a state record as an argument and changes its intensity field to 2. The sys:replace_state/2,3 functions return the new state, and as you can see near the end of the result here, intensity is now 2 instead of 0.
As the sys:replace_state/2,3 documentation explains, these functions are intended only for debugging purposes, so using them to do this in a production system is definitely not something I recommend. The second argument to replace_state here shows that this approach requires knowledge of the details of the internal state record of supervisor, which we obtained here via the rr shell command, so if that record ever changes, this code may stop working. Even more fragile would be treating the supervisor state record as a tuple and counting on the intensity field to be in a particular tuple position so you can change its value. Therefore, if you really want this functionality of changing a supervisor's restart intensity, you're best off in the long run proposing to the OTP team that it be added; if you're going to take that route, I recommend first proposing the idea on the erlang-questions mailing list to gauge interest.

One solution would be to nest your supervisors. But the main question is what do you want to achieve by this restart intensities. The intensity when you want to kill the supervisor needs to be a indication for something very wrong e.g. a needed resource unexpectedly not being available.

Related

When I use $, it leaves no space with exams2blackboard

My University is testing blackboard ultra. I generate the exercises with
exams2blackboard("ProbEjemploPrueba9", n=10, fix_pre = TRUE, converter = "pandoc", encoding = "utf8", eval = list(partial = T, negative = T, rule = "false"))
but when I use the $ in $A$, it does not leave space, it appears like this;
"IfA is the event .... "instead of "If A is the event.... "
I attach the image.
The solution is to remove $, because in this case nothing happens, but maybe other times it is necessary.
Would there be any solution to fix that typo?

Kafka sink to InfluxDB

I'm trying to get data from my kafka topic into InfluxDB using the Confluent/Kafka stack. At the moment, the messages in the topic have a form of {"tag1":"123","tag2":"456"} (I have relatively good control over the message format, I chose the JSON to be as above, could include a timestamp etc if necessary).
Ideally, I would like to add many tags without needing to specify a schema/column names in the future.
I followed https://docs.confluent.io/kafka-connect-influxdb/current/influx-db-sink-connector/index.html (the "Schemaless JSON tags example") as this matches my use case quite closely. The "key" of each message is currently just the MQTT topic name (the topic's source is an MQTT connector). So I set the "key.converter" to "stringconverter" (instead of JSONconverter as in the example).
Other examples I've seen online seem to suggest the need for a schema to be set, which I'd like to avoid. Using InfluxDB v1.8, everything on Docker/maintained on Portainer.
I cannot seem to start the connector and never get any data to move across.
Below is the config for my InfluxDBSink Connector:
{
"name": "InfluxDBSinkKafka",
"config": {
"key.converter.schemas.enable": "false",
"value.converter.schemas.enable": "false",
"name": "InfluxDBSinkKafka",
"connector.class": "io.confluent.influxdb.InfluxDBSinkConnector",
"tasks.max": "1",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"topics": "KAFKATOPIC1",
"influxdb.url": "http://URL:PORT",
"influxdb.db": "tagdata",
"measurement.name.format": "${topic}"
}
}
The connector fails, and each time I click "start" (the play button) the following pops up in the connect container's logs:
[2022-03-22 15:46:52,562] INFO [Worker clientId=connect-1, groupId=compose-connect-group]
Connector InfluxDBSinkKafka target state change (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
[2022-03-22 15:46:52,562] INFO Setting connector InfluxDBSinkKafka state to STARTED (org.apache.kafka.connect.runtime.Worker)
[2022-03-22 15:46:52,562] INFO SinkConnectorConfig values:
config.action.reload = restart
connector.class = io.confluent.influxdb.InfluxDBSinkConnector
errors.deadletterqueue.context.headers.enable = false
errors.deadletterqueue.topic.name =
errors.deadletterqueue.topic.replication.factor = 3
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = class org.apache.kafka.connect.storage.StringConverter
name = InfluxDBSinkKafka
predicates = []
tasks.max = 1
topics = [KAFKATOPIC1]
topics.regex =
transforms = []
value.converter = class org.apache.kafka.connect.json.JsonConverter
(org.apache.kafka.connect.runtime.SinkConnectorConfig)
[2022-03-22 15:46:52,563] INFO EnrichedConnectorConfig values:
config.action.reload = restart
connector.class = io.confluent.influxdb.InfluxDBSinkConnector
errors.deadletterqueue.context.headers.enable = false
errors.deadletterqueue.topic.name =
errors.deadletterqueue.topic.replication.factor = 3
errors.log.enable = false
errors.log.include.messages = false
errors.retry.delay.max.ms = 60000
errors.retry.timeout = 0
errors.tolerance = none
header.converter = null
key.converter = class org.apache.kafka.connect.storage.StringConverter
name = InfluxDBSinkKafka
predicates = []
tasks.max = 1
topics = [KAFKATOPIC1]
topics.regex =
transforms = []
value.converter = class org.apache.kafka.connect.json.JsonConverter
(org.apache.kafka.connect.runtime.ConnectorConfig$EnrichedConnectorConfig)
I am feeling a little out of my depth and would appreciate any and all help.
The trick here is getting the data in the right format to Kafka in the first place. My MQTT source stream needed to have the value converter set to Bytearray with e schema url and schema = true. Then the Influx Sink started working when I used the jsonconverter, with schema=false. Then it started working. This is deceptive because the message queue looks the same with different valueconverters for the MQTT source connecter, so it took a while to figure out that was the problem.
After getting this working, and realising the confluent stack was perhaps a little overkill for this task, I went with the (much) easier route of pushing MQTT directly to Telegraf and having Telegraf push into InfluxDB. I would recommend this.

Lua: Concise expression of table scope

I'm working on a game where a bunch of characters will be generated on the fly, based on some constraints defined either in the project or externally via mod files. I am using MoonSharp Lua (5.2) interpreter for interfacing with my C# code, and Lua tables to store the constraint presets. As an example:
require "Defaults"
AgePresets = {}
-- Single value
AgePresets.Newborn = 0
-- Simple ranges
AgePresets.Default = defaultAgeRange --referring to the Defaults require
AgePresets.Child = {1, 12}
AgePresets.Teenager = {13, 19}
AgePresets.YoungAdult = {20, 29}
AgePresets.Adult = {30, 40}
AgePresets.MiddleAge = {40, 60}
AgePresets.Senior = {61, 80}
AgePresets.Elder = {81, 99}
AgePresets.Methuselah = {100, 150}
AgePresets.Methuselah2 = {150, 200}
-- Weighted ranges // again referring to previously defined elements to keep things concise
AgePresets.Tween = {
{weight = 1, minmax = AgePresets.Teenager },
{weight = 1, minmax = AgePresets.YoungAdult }
}
This works fine, but from an end-user point of view, there's a lot of unnecessary typing involved. We are clearly working on AgePresets here but it is still mentioned as a prefix before every member name.
I could of course define AgePresets as an array, like AgePresets = { Child = {}, Teenager = {} } but the problem with that is then I cannot refer to previously defined elements in the array.
This doesn't work:
AgePresets = {
Child = {1,12},
RefToChild = Child, //attempt to index a nil value exception
Teen = {13,19}
}
What I ideally want to achieve is a clean, concise way for users to enter this data in, like in the first example but without having to put AgePresets. prefix before everything. How do I go about declaring a scope in a file such that all succeeding members defined in the file will be within that scope, while maintaining the ability to refer to other members defined previously in the scope?
AgePresets = setmetatable({}, {__index = _G})
do
local _ENV = AgePresets
Newborn = 0
Child = {1,12}
RefToChild = Child -- this ref is Ok
Teen = {13,19}
YoungAdult = {20,29}
Tween = {
{weight = 1, minmax = Teen },
{weight = 1, minmax = YoungAdult }
}
rnd = math.random(10) -- global functions are available here
end
setmetatable(AgePresets, nil)
You can mix the two styles: table constructor for fields that don't need to reference variables that aren't in scope yet, followed by assignment statements for the rest.
I would do that unless the order of the fields in the code significantly enhanced comprehension.

Erlang:creating list of tuples within lists:foreach

I query the list of users from Mnesia Database in Chicagoboss. I'm getting the error when I try to add the Lists within lists:foreach with ++ operator. My aim is, based on userid I will do ets:lookup to my cache and create a List like - [{{<<"name">>,<<"Batman">>}, {<<"steps">>,2552}, {<<"distance">>,2050}}].
For each user I'll create this list and add with the previous List. So that ultimately I can can sort on <<"steps">> and convert the binary list by json encoding and send it to the client via Websockets.
I'm getting the error at this line:
Reading1 = Reading2 ++ Currentlist
as I've decalred Reading1 as an Empty list.
My question is how can I manipulate the lists within the lists:foreach and then send the result List via websocket?
BelugaUsers = boss_db:find(users, [{accesstoken, 'not_equals', ''}]),
Reading1 = [],
Reading2 = [],
lists:foreach(fun(X) ->
{_,_,BEmail,BName,_,_,BAccessToken,_} = X,
UserKey = BEmail ++ "-" ++ ?MYAPICALL1,
io:format("UserKey for Leader Board: ~n~p~n",[UserKey]),
[Reading] = ets:lookup(myapi_cache, list_to_binary(UserKey)),
{_,Result} = Reading,
ActivitySummary = proplists:get_value(<<"activitySummary">>, Result),
%Print ActivitySummary for the user ....printing fine
io:format("ActivitySummary ==========: ~n~p~n",[ActivitySummary]),
%Create a list of the format
%[{{<<"name">>,<<"Batman">>}, {<<"steps">>,2552}, {<<"distance">>,2050}}]
Currentlist = [{{<<"name">>, list_to_binary(BName)}, {<<"steps">>, proplists:get_value(<<"steps">>, ActivitySummary)}, {<<"distance">>, proplists:get_value(<<"distance">>, ActivitySummary)}}],
%% HERE I'M GETTING error%%
Reading1 = Reading2 ++ Currentlist
end, BelugaUsers),
%sort the list
Reading3 = lists:keysort(2, Reading1),
%reverse the list
Reading4 = lists:reverse(Reading3),
WebSocketId ! {text, jsx:encode(Reading4)},
Erlang variables are single-assignment; once bound to a value, they can't be re-bound to a different value.
The lists:foreach/2 function is not useful for this problem because it can't create a new value and return it to its caller. You should instead use lists:map/2, perhaps like this:
BelugaUsers = boss_db:find(users, [{accesstoken, 'not_equals', ''}]),
Reading = lists:map(
fun(X) ->
{_,_,BEmail,BName,_,_,BAccessToken,_} = X,
UserKey = BEmail ++ "-" ++ ?MYAPICALL1,
io:format("UserKey for Leader Board: ~n~p~n",[UserKey]),
{_,Result} = hd(ets:lookup(myapi_cache, list_to_binary(UserKey))),
ActivitySummary = proplists:get_value(<<"activitySummary">>, Result),
%%Print ActivitySummary for the user ....printing fine
io:format("ActivitySummary ==========: ~n~p~n",[ActivitySummary]),
%%Create a tuple of the format
%%{{<<"name">>,<<"Batman">>}, {<<"steps">>,2552}, {<<"distance">>,2050}}
{{<<"name">>, list_to_binary(BName)},
{<<"steps">>, proplists:get_value(<<"steps">>, ActivitySummary)},
{<<"distance">>, proplists:get_value(<<"distance">>, ActivitySummary)}}
end, BelugaUsers),
%%sort the list
Reading2 = lists:keysort(2, Reading),
%%reverse the list
Reading3 = lists:reverse(Reading2),
WebSocketId ! {text, jsx:encode(Reading3)}.
The lists:map/2 function applies a function to each value in a list to a produce a potentially different value and returns a new list consisting of those new values. This is essentially what you were trying to do with lists:foreach/2 and your attempt to use imperative assignment to add each element to an already-existing list.
You could alternatively use a list comprehension but I think lists:map/2 is clearer in this situation.

Rewrite variable in Erlang

I am playing with records and list. Please, I want to know how to use one variable twice. When I assign any values into variable _list and after that I try rewrite this variable then raising error:
** exception error: no match of right hand side value
-module(hello).
-author("anx00040").
-record(car, {evc, type, color}).
-record(person, {name, phone, addresa, rc}).
-record(driver, {rc, evc}).
-record(list, {cars = [], persons = [], drivers = []} ).
%% API
-export([helloIF/1, helloCase/1, helloResult/1, helloList/0, map/2, filter/2, helloListCaA/0, createCar/3, createPerson/4, createDriver/2, helloRecords/0, empty_list/0, any_data/0, del_Person/1, get_persons/1, do_it_hard/0, add_person/2]).
createCar(P_evc, P_type, P_color) -> _car = #car{evc = P_evc, type = P_type, color = P_color}, _car
.
createPerson(P_name, P_phone, P_addres, P_rc) -> _person= #person{name = P_name, phone = P_phone, addresa = P_addres, rc = P_rc}, _person
.
createDriver(P_evc, P_rc) -> _driver = #driver{rc = P_rc, evc = P_evc}, _driver
.
empty_list() ->
#list{}.
any_data() ->
_car1 = hello:createCar("BL 4", "Skoda octavia", "White"),
_person1 = hello:createPerson("Eduard B.","+421 917 111 711","Kr, 81107 Bratislava1", "8811235"),
_driver1 = hello:createDriver(_car1#car.evc, _person1#person.rc),
_car2 = hello:createCar("BL 111 HK", "BMW M1", "Red"),
_person2 = hello:createPerson("Lenka M","+421 917 111 111","Krizn0, 81107 Bratislava1", "8811167695"),
_driver2 = hello:createDriver(_car2#car.evc, _person2#person.rc),
_car3 = hello:createCar("BL 123 AB", "Audi A1 S", "Black"),
_person3 = hello:createPerson("Stela Ba.","+421 918 111 711","Azna 20, 81107 Bratislava1", "8811167695"),
_driver3 = hello:createDriver(_car3#car.evc, _person3#person.rc),
_list = #list{
cars = [_car1,_car2,_car3],
persons = [_person1, _person2, _person3],
drivers = [_driver1, _driver2, _driver3]},
_list.
add_person(List, Person) ->
List#list{persons = lists:append([Person], List#list.persons) }.
get_persons(#list{persons = P}) -> P.
do_it_hard()->
empty_list(),
_list = add_person(any_data(), #person{name = "Test",phone = "+421Test", addresa = "Testova 20 81101 Testovo", rc =88113545}),
io:fwrite("\n"),
get_persons(add_person(_list, #person{name = "Test2",phone = "+421Test2", addresa = "Testova 20 81101 Testovo2", rc =991135455}))
.
But it raising error when i use variable _list twice:
do_it_hard()->
empty_list(),
_list = add_person(any_data(), #person{name = "Test",phone = "+421Test", addresa = "Testova 20 81101 Testovo", rc =88113545}),
_list =add_person(_list, #person{name = "Test2",phone = "+421Test2", addresa = "Testova 20 81101 Testovo2", rc =991135455}),
get_persons(_list)
.
In the REPL, it can be convenient to experiment with things while re-using variable names. There, you can do f(A). to have Erlang "forget" the current assignment of A.
1> Result = connect("goooogle.com").
{error, "server not found"}
2> % oops! I misspelled the server name
2> f(Result).
ok
3> Result = connect("google.com").
{ok, <<"contents of the page">>}
Note that this is only a REPL convenience feature. You can't do this in actual code.
In actual code, variables can only be assigned once. In a procedural language (C, Java, Python, etc), the typical use-case for reassignment is loops:
for (int i = 0; i < max; i++) {
conn = connect(servers[i]);
reply = send_data(conn);
print(reply);
}
In the above, the variables i, conn, and reply are reassigned in each iteration of the loop.
Functional languages use recursion to perform their loops:
send_all(Max, Servers) ->
send_loop(1, Max, Servers).
send_loop(Current, Max, _Servers) when Current =:= Max->
ok;
send_loop(Current, Max, Servers) ->
Conn = connect(lists:nth(Current, Servers)),
Reply = send_data(Conn),
print(Reply).
This isn't very idiomatic Erlang; I'm trying to make it mirror the procedural code above.
As you can see, I'm getting the same effect, but my assignments within a function are fixed.
As a side note, you are using a lot of variable names beginning with underscore. In Erlang this is a way of hinting that you will not be using the value of these variables. (Like in the above example, when I've reached the end of my list, I don't care about the list of servers.) Using a leading underscore as in your code turns off some useful compiler warnings and will confuse any other developers who look at your code.
In some situations it is convenient to use use SeqBind:
SeqBind is a parse transformation that auto-numbers all occurrences of these bindings following the suffix # (creating L#0, L#1, Req#0, Req#1) and so on.
Simple example:
...
-compile({parse_transform,seqbind}).
...
List# = lists:seq(0, 100),
List# = lists:filter(fun (X) -> X rem 2 == 0 end, List#)
...
I used google...
Erlang is a single-assignment language. That is, once a variable has been given a value, it cannot be given a different value. In this sense it is like algebra rather than like most conventional programming languages.
http://www.cis.upenn.edu/~matuszek/General/ConciseGuides/concise-erlang.html

Resources