Related
I wanted to send a message to a process after a delay, and discovered erlang:send_after/4.
When looking at the docs it looked like this is exactly what I wanted:
erlang:send_after(Time, Dest, Msg, Options) -> TimerRef
Starts a timer. When the timer expires, the message Msg is sent to the
process identified by Dest.
However, it doesn't seem to work when the destination is running on another node - it tells me one of the arguments are bad.
1> P = spawn('node#host', module, function, [Arg]).
<10585.83.0>
2> erlang:send_after(1000, P, {123}).
** exception error: bad argument
in function erlang:send_after/3
called as erlang:send_after(1000,<10585.83.0>,{123})
Doing the same thing with timer:send_after/3 appears to work fine:
1> P = spawn('node#host', module, function, [Arg]).
<10101.10.0>
2> timer:send_after(1000, P, {123}).
{ok,{-576458842589535,#Ref<0.1843049418.1937244161.31646>}}
And, the docs for timer:send_after/3 state almost the same thing as the erlang version:
send_after(Time, Pid, Message) -> {ok, TRef} | {error, Reason}
Evaluates Pid ! Message after Time milliseconds.
So the question is, why do these two functions, which on the face of it do the same thing, behave differently? Is erlang:send_after broken, or mis-advertised? Or maybe timer:send_after isn't doing what I think it is?
TL;DR
Your assumption is correct: these are intended to do the same thing, but are implemented differently.
Discussion
Things in the timer module such as timer:send_after/2,3 work through the gen_server that defines that as a service. Like any other service, this one can get overloaded if you assign a really huge number of tasks (timers to track) to it.
erlang:send_after/3,4, on the other hand, is a BIF implemented directly within the runtime and therefore have access to system primitives like the hardware timer. If you have a ton of timers this is definitely the way to go. In most programs you won't notice the difference, though.
There is actually a note about this in the Erlang Efficiency Guide:
3.1 Timer Module
Creating timers using erlang:send_after/3 and erlang:start_timer/3 , is much more efficient than using the timers provided by the timer module in STDLIB. The timer module uses a separate process to manage the timers. That process can easily become overloaded if many processes create and cancel timers frequently (especially when using the SMP emulator).
The functions in the timer module that do not manage timers (such as timer:tc/3 or timer:sleep/1), do not call the timer-server process and are therefore harmless.
A workaround
A workaround to gain the efficiency of the BIF without the same-node restriction is to have a process of your own that does nothing but wait for a message to forward to another node:
-module(foo_forward).
-export([send_after/3, cancel/1]).
% Obviously this is an example only. You would want to write this to
% be compliant with proc_lib, write a proper init/N and integrate with
% OTP. Note that this snippet is missing the OTP service functions.
start() ->
spawn(fun() -> loop(self(), [], none) end).
send_after(Time, Dest, Message) ->
erlang:send_after(Time, self(), {forward, Dest, Message}).
loop(Parent, Debug, State) ->
receive
{forward, Dest, Message} ->
Dest ! Message,
loop(Parent, Debug, State);
{system, From, Request} ->
sys:handle_msg(Request, From, Parent, ?MODULE, Debug, State);
Unexpected ->
ok = log(warning, "Received message: ~tp", [Unexpected]),
loop(Parent, Debug, State)
end.
The above example is a bit shallow, but hopefully it expresses the point. It should be possible to get the efficiency of the BIF erlang:send_after/3,4 but still manage to send messages across nodes as well as give you the freedom to cancel a message using erlang:cancel_timer/1
But why?
The puzzle (and bug) is why erlang:send_after/3,4 does not want to work across nodes. The example you provided above looks a bit odd as the first assignment to P was the Pid <10101.10.0>, but the crashed call was reported as <10585.83.0> -- clearly not the same.
For the moment I do not know why erlang:send_after/3,4 doesn't work, but I can say with confidence that the mechanism of operation between the two is not the same. I'll look into it, but I imagine that the BIF version is actually doing some funny business within the runtime to gain efficiency and as a result signalling the target process by directly updating its mailbox instead of actually sending an Erlang message on the higher Erlang-to-Erlang level.
Maybe it is good that we have both, but this should definitely be clearly marked in the docs, and it evidently is not (I just checked).
There is some difference in timeout order if you have many timers.
The example below shows erlang:send_after does not guarantee order, but
timer:send_after does.
1> A = lists:seq(1,10).
[1,2,3,4,5,6,7,8,9,10]
2> [erlang:send_after(100, self(), X) || X <- A].
...
3> flush().
Shell got 2
Shell got 3
Shell got 4
Shell got 5
Shell got 6
Shell got 7
Shell got 8
Shell got 9
Shell got 10
Shell got 1
ok
4> [timer:send_after(100, self(), X) || X <- A].
...
5> flush().
Shell got 1
Shell got 2
Shell got 3
Shell got 4
Shell got 5
Shell got 6
Shell got 7
Shell got 8
Shell got 9
Shell got 10
ok
I have set up a dev network consisting of 4VPs using the pbft consensus.
I am trying to test the behaviour of the VPs when one of them is down.
Step one
While the 4 VPs are running , i have deployed a chain code (chaincode_example02).
Checking localhost:7050/chain -> return 2
Step two
I shutdown one of the VP using (docker stop containerID)
Now when i execute an Invoke transaction and recheck the chain length:
localhost:7050/chain -> it still returns 2.
Step three
I restart the VP (from step 2) , and the i see that the invoke transaction (from step 2) is executed automatically and the chain size is now 3
localhost:7050/chain -> now returns 3.
My understanding is that with 4VP using the pbft consensus, we have tolerance for 1 faulty VP .If that is the case, then the invoke transaction should have been executed in step2.
Can someone please advise if that is the expected result and why?
Thanks in advance
My project has blown through the max 1M atoms, we've cranked up the limit, but I need to apply some sanity to the code that people are submitting with regard to list_to_atom and its friends. I'd like to start by getting a list of all the registered atoms so I can see where the largest offenders are. Is there any way to do this. I'll have to be creative about how I do it so I don't end up trying to dump 1-2M lines in a live console.
You can get hold of all atoms by using an undocumented feature of the external term format.
TL;DR: Paste the following line into the Erlang shell of your running node. Read on for explanation and a non-terse version of the code.
(fun F(N)->try binary_to_term(<<131,75,N:24>>) of A->[A]++F(N+1) catch error:badarg->[]end end)(0).
Elixir version by Ivar Vong:
for i <- 0..:erlang.system_info(:atom_count)-1, do: :erlang.binary_to_term(<<131,75,i::24>>)
An Erlang term encoded in the external term format starts with the byte 131, then a byte identifying the type, and then the actual data. I found that EEP-43 mentions all the possible types, including ATOM_INTERNAL_REF3 with type byte 75, which isn't mentioned in the official documentation of the external term format.
For ATOM_INTERNAL_REF3, the data is an index into the atom table, encoded as a 24-bit integer. We can easily create such a binary: <<131,75,N:24>>
For example, in my Erlang VM, false seems to be the zeroth atom in the atom table:
> binary_to_term(<<131,75,0:24>>).
false
There's no simple way to find the number of atoms currently in the atom table*, but we can keep increasing the number until we get a badarg error.
So this little module gives you a list of all atoms:
-module(all_atoms).
-export([all_atoms/0]).
atom_by_number(N) ->
binary_to_term(<<131,75,N:24>>).
all_atoms() ->
atoms_starting_at(0).
atoms_starting_at(N) ->
try atom_by_number(N) of
Atom ->
[Atom] ++ atoms_starting_at(N + 1)
catch
error:badarg ->
[]
end.
The output looks like:
> all_atoms:all_atoms().
[false,true,'_',nonode#nohost,'$end_of_table','','fun',
infinity,timeout,normal,call,return,throw,error,exit,
undefined,nocatch,undefined_function,undefined_lambda,
'DOWN','UP','EXIT',aborted,abs_path,absoluteURI,ac,accessor,
active,all|...]
> length(v(-1)).
9821
* In Erlang/OTP 20.0, you can call erlang:system_info(atom_count):
> length(all_atoms:all_atoms()) == erlang:system_info(atom_count).
true
I'm not sure if there's a way to do it on a live system, but if you can run it in a test environment you should be able to get a list via crash dump. The atom table is near the end of the crash dump format. You can create a crash dump via erlang:halt/1, but that will bring down the whole runtime system.
I dare say that if you use more than 1M atoms, then you are doing something wrong. Atoms are intended to be static as soon as the application runs or at least upper bounded by some small number, 3000 or so for a medium sized application.
Be very careful when an enemy can generate atoms in your vm. especially calls like list_to_atom/1 is somewhat dangerous.
EDITED (wrong answer..)
You can adjust number of atoms with +t
http://www.erlang.org/doc/efficiency_guide/advanced.html
..but I know very few use cases when it is necessary.
You can track atom stats with erlang:memory()
DISCLAIMER: This question is only for those who have access to the econometrics toolbox in Matlab.
The Situation: I would like to use Matlab to simulate N observations from an ARIMA(p, d, q) model using the econometrics toolbox. What's the difficulty? I would like the innovations to be simulated with deterministic, time-varying variance.
Question 1) Can I do this using the in-built matlab simulate function without altering it myself? As near as I can tell, this is not possible. From my reading of the docs, the innovations can either be specified to have a constant variance (ie same variance for each innovation), or be specified to be stochastically time-varying (eg a GARCH model), but they cannot be deterministically time-varying, where I, the user, choose their values (except in the trivial constant case).
Question 2) If the answer to question 1 is "No", then does anyone see any reason why I can't edit the simulate function from the econometrics toolbox as follows:
a) Alter the preamble such that the function won't throw an error if the Variance field in the input model is set to a numeric vector instead of a numeric scalar.
b) Alter line 310 of simulate from:
E(:,(maxPQ + 1:end)) = Z * sqrt(variance);
to
E(:,(maxPQ + 1:end)) = (ones(NumPath, 1) * sqrt(variance)) .* Z;
where NumPath is the number of paths to be simulated, and it can be assumed that I've included an error trap to ensure that the (input) deterministic variance path stored in variance is of the right length (ie equal to the number of observations to be simulated per path).
Any help would be most appreciated. Apologies if the question seems basic, I just haven't ever edited one of Mathwork's own functions before and didn't want to do something foolish.
UPDATE (2012-10-18): I'm confident that the code edit I've suggested above is valid, and I'm mostly confident that it won't break anything else. However it turns out that implementing the solution is not trivial due to file permissions. I'm currently talking with Mathworks about the best way to achieve my goal. I'll post the results here once I have them.
It's been a week and a half with no answer, so I think I'm probably okay to post my own answer at this point.
In response to my question 1), no, I have not found anyway to do this with the built-in matlab functions.
In response to my question 2), yes, what I have posted will work. However, it was a little more involved than I imagined due to matlab file permissions. Here is a step-by-step guide:
i) Somewhere in your matlab path, create the directory #arima_Custom.
ii) In the command window, type edit arima. Copy the text of this file into a new m file and save it in the directory #arima_Custom with the filename arima_Custom.m.
iii) Locate the econometrics toolbox on your machine. Once found, look for the directory #arima in the toolbox. This directory will probably be located (on a Linux machine) at something like $MATLAB_ROOT/toolbox/econ/econ/#arima (on my machine, $MATLAB_ROOT is at /usr/local/Matlab/R2012b). Copy the contents of #arima to #arima_Custom, except do NOT copy the file arima.m.
iv) Open arima_Custom for editing, ie edit arima_Custom. In this file change line 1 from:
classdef (Sealed) arima < internal.econ.LagIndexableTimeSeries
to
classdef (Sealed) arima_Custom < internal.econ.LagIndexableTimeSeries
Next, change line 406 from:
function OBJ = arima(varargin)
to
function OBJ = arima_Custom(varargin)
Now, change line 993 from:
if isa(OBJ.Variance, 'double') && (OBJ.Variance <= 0)
to
if isa(OBJ.Variance, 'double') && (sum(OBJ.Variance <= 0) > 0)
v) Open the simulate.m located in #arima_Custom for editing (we copied it there in step iii). It is probably best to open this file by navigating to it manually in the Current Folder window, to ensure the correct simulate.m is opened. In this file, alter line 310 from:
E(:,(maxPQ + 1:end)) = Z * sqrt(variance);
to
%Check that the input variance is of the right length (if it isn't scalar)
if isscalar(variance) == 0
if size(variance, 2) ~= 1
error('Deterministic variance must be a column vector');
end
if size(variance, 1) ~= numObs
error('Deterministic variance vector is incorrect length relative to number of observations');
end
else
variance = variance(ones(numObs, 1));
end
%Scale innovations using deterministic variance
E(:,(maxPQ + 1:end)) = sqrt(ones(numPaths, 1) * variance') .* Z;
And we're done!
You should now be able to simulate with deterministically time-varying variance using the arima_Custom class, for example (for an ARIMA(0,1,0)):
ARIMAModel = arima_Custom('D', 1, 'Variance', ScalarVariance, 'Constant', 0);
ARIMAModel.Variance = TimeVaryingVarianceVector;
[X, e, VarianceVector] = simulate(ARIMAModel, NumObs, 'numPaths', NumPaths);
Further, you should also still be able to use matlab's original arima class, since we didn't alter it.
I am currently pursuing Masters in Embedded and for my thesis I have to study the effectiveness of Erlang for programming Robot. AFAIK Erlang's declarative nature and concurrency can be effective, so I made an Erlang code for "Adaptive cruise control" which takes sensor values from C program(because Erlang can not read sensors directly) then perform computation and send back control signal to C program. But the code looks quite big in size(lines). Why am I not able to use declarative nature or there is some other problem?
Here is my code snippets.
start() ->
spawn( cr, read_sensor, []),
spawn(cr, take_decision, []),
sleep_infinite().
% this will make it to run infinitely
sleep_infinite() ->
receive
after infinity ->
true
end.
read_sensor() ->
register(read, self()),
Port = open_port({spawn , "./cr_cpgm" }, [{packet, 2}]),
Port ! {self(),{command, [49]}},% for executing read sensor fun in C pgm
read_reply(Port).
read_reply(Port) ->
receive
read_sensor ->
Port ! { self(), { command, [49]}};
{Port, {data, Data}} ->
[Left,Center,Right,Distance] = Data, % stored values of sensors into variables for further computation
io:format("value of Left: ~w and Center: ~w and Right: ~w and Distance: ~w~n",[Left,Center,Right,Distance]),
if Distance =< 100 -> decision ! {1, out}; % Distance shows the value returned by front sharp sensor
((Left > 25) and (Center > 25) and (Right > 25)) -> decision ! {2, out}; % stop robot
Center < 25 -> decision ! {3, out}; % move forward
((Left > 25) and (Center > 25)) -> decision ! {4, out}; % turn right
((Right > 25) and (Center > 25)) -> decision ! {5, out}; % turn left
true -> decision ! {6, out} % no match stop robot
end
end,
read_reply(Port).
take_decision() ->
register(decision, self()),
Port = open_port({spawn , "./cr_cpgm" }, [{packet, 2}]),
decision_reply(Port).
decision_reply(Port) ->
receive
{A, out} ->
Port ! {self(), {command, [50,A]}};
{Port,{data, Data}} ->
if
Data == [102] -> read ! read_sensor %
end
end,
decision_reply(Port).
This code looks more like a C code.
Is my way of implementation wrong?(especially IF...end) or problem itself is small(only 2 processes)
Please suggest me how to show the effectiveness of Erlang in programming robots. All suggestions are welcome.
Thanks..
Well I am agree with #cthulahoops that this problem is not enough to show the effectiveness of Erlang. Can anybody suggest some Robotic application which I can implement in Erlang??
Well, firstly I'd say that this doesn't sound like a very good project for showing the effectiveness of Erlang.
The first thing that comes to mind to make the code more declarative is to split the if out into a separate function like this:
choice(Distance, _Left, _Center, _Right) when Distance =< 100 -> something_you_didnt_say_what;
choice(_Distance, Left, Center, Right) when Left > 25, Center > 25, Right > 25 -> stop;
choice(_Distance, Left, _Center, _Right) when Center < 25 -> forward;
choice(_Distance, Left, Center, _Right) when Center > 25, Left > 25 -> right;
choice(_Distance, _Left, Center, Right) when Center > 25, Right > 25 -> left.
Which separates the declaration of how to respond to sensors from the messy business of looping and sending messages, etc. Also, returning atoms rather than the cryptic integers avoids having to put that information into comments. (Following the philosophy of comments tell you where you need to clarify the code.)
example: If you had multiple robots which would interact with some way and each had their own logic controlled by a central erlang server.
Normally you'd make a big loop and put the logic of all the elements each cycle through, with ugly stuff like shared memory and mutexes if you utilise standard threads. In erlang you can code it more naturally and spawn functions which waste minimal space and have them communicate via message passing. With OTP you can make generic structures which handle the more annoying non functional aspects to common problems and help make it fault tolerant with supervision trees. You end up with much easier to read code and much more efficient and robust structure to develop in.
That's the power of erlang.
If you need to calculate some decisions basing on couple variables (Right, Left, etc..) you obviously will not avoid it. The question is how to benefit from using erlang.
Here, what comes to my mind, is implementing one of OTP behaviours - gen_fsm (finite state machine). So, the logic would be (maybe/probably?): Receive Left -> wait only for Right or Center and so on. This would make your code very clear and give you possibility to spawn lots of actions basing on current state, which would result in asynchronous system totally under your control.
It strikes me that Erlang is particularly well-suited for robotic swarms. Having each member of the swarm rpc:abcast messages to all the other members is a fantastic alternative to the usual UDP boilerplate crap that you'd have to deal with in a procedural language. There's no binding to ports, no specifying a binary format for your messages, no serializing of objects, etc.
As long as you can sort out the discovery of other nodes in your area, it seems like a decentralized/distributed Erlang swarm would be a great project.