Hot code replacement in erlang - erlang

I am working on my first real project in erlang, however, this code is simplified for brevity. I want to be able to load a newer version of a file into my project remotely while it's running. I've read about using a behavior like gen_server or gen_fsm which has this for free. While that might achieve the result, I want to use this to learn how to do it, not just get it done. I've read the docs about code replacement, and LYSE's bit about Hot Code Loving, among other things, but I haven't been able to find anything that works for what I'm doing, so here is the basic idea.
-module(reloading).
-export([loop/0]).
loop() ->
receive
upgrade ->
?MODULE:loop();
hello ->
io:format("This is a test~n"),
loop();
_ ->
loop()
end.
I am simply looping with the idea that I can send the message upgrade and it will load a newer version of the code.
$ erl
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.9.1 (abort with ^G)
1> c(reloading).
{ok,reloading}
2> Loop = spawn(reloading, loop, []).
<0.39.0>
3> Loop ! hello.
This is a test
hello
At this point I change 10 line to io:format("I have changed this!~n"),
4> Loop ! upgrade.
upgrade
5> Loop ! hello.
This is a test
hello
I am expecting this hello call to print I have changed this! not This is a test. I know I can simply call c(reloading). and have this work the way expected, but I'm looking to send the actual project a message rather than manually updating the code. So where is my disconnect? What am I doing wrong, that I should be doing in order to hot load this code? As mentioned before, I am looking for a non-OTP solution for the sake of education.

For the sake of having an explicit answer, I am posting this.
Using #rvirding's suggestion of using the code module, I've modified it to look like this:
-module(reloading).
-export([loop/0]).
loop() ->
receive
upgrade ->
code:purge(?MODULE),
compile:file(?MODULE),
code:load_file(?MODULE),
?MODULE:loop();
hello ->
io:format("This is a test~n"),
loop();
_ ->
loop()
end.
First code:purge the old ?MODULE, then compile:file the new file, and finally, code:load_file the new ?MODULE.
This works as I originally intended.
$ erl
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.9.1 (abort with ^G)
1> Loop = spawn(reloading, loop, []).
<0.34.0>
2> Loop ! hello.
This is a test
hello
Change line to io:format("I have changed this!~n"),
3> Loop ! upgrade.
upgrade
4> Loop ! hello.
I have changed this!
hello

While erlang can handle two versions of a module and calling a function with mod:func(...) will always call the latest version of a module (if the function is exported) you still have to load the new version of the module into Erlang system. You can't expect it automagically detect that you happen to have a new version of the module somewhere, find it, compile it and load it.
N.B. compiling and loading are two separate things. So c(mod). both compiles and loads the module, while l(mod). just loads the object code (.beam file) of the already compiled module. The Erlang compiler is called from the module compile and it just compiles and generates a .beam file while the code loading is handled by the module code.

In addition to the above I'd like to notice that some tools that reload code automatically for you exist.
You should have a look at sync or active projects.

Compile *.beam locally, then send it to your server and reload it as mentioned in man pages:
http://erlang.org/documentation/doc-1/reference_manual/code_loading.html#id86381
-module(m).
-export([loop/0]).
loop() ->
receive
code_switch ->
m:loop();
Msg ->
...
loop()
end.

Related

Accessing one erlang module by another when in different folders

I have a pretty simple doubt, but I can't seem to find a proper solution for it anywhere.
I have 2 erlang modules, module1.erl and module2.erl. As defined by the submission guidelines for my project, both modules belong to different parts and are hence in different folders part1 and part2 respectively under the same directory. This is how the structure looks:
src/
part1/
module1.erl
part2/
module2.erl
Now module2 is dependent on module1, and calls various methods of module1 as module1:method(). I'm able to achieve full functionality when both module1.erl and module2.erl are in the same folder, but now they're in different folders, and I try to run module2.erl from part2 folder, I can't figure out how to allow module2 to compile and call the methods of module1.
Since the emulator is being used, the path to module1 is not in the code path by default when we are launching the emulator in the path of module2 directly and with default options. This can be verified using ...
1> code:get_path().
code:get_path().
[".","/usr/local/lib/erlang/lib/kernel-8.2/ebin",
"/usr/local/lib/erlang/lib/stdlib-3.17/ebin",
"/usr/local/lib/erlang/lib/xmerl-1.3.28/ebin",
"/usr/local/lib/erlang/lib/wx-2.1.1/ebin",
"/usr/local/lib/erlang/lib/tools-3.5.2/ebin",
"/usr/local/lib/erlang/lib/tftp-1.0.3/ebin",
"/usr/local/lib/erlang/lib/syntax_tools-2.6/ebin",
"/usr/local/lib/erlang/lib/ssl-10.6/ebin",
"/usr/local/lib/erlang/lib/ssh-4.13/ebin",
"/usr/local/lib/erlang/lib/snmp-5.11/ebin",
"/usr/local/lib/erlang/lib/sasl-4.1.1/ebin",
"/usr/local/lib/erlang/lib/runtime_tools-1.17/ebin",
"/usr/local/lib/erlang/lib/reltool-0.9/ebin",
"/usr/local/lib/erlang/lib/public_key-1.11.3/ebin",
"/usr/local/lib/erlang/lib/parsetools-2.3.2/ebin",
"/usr/local/lib/erlang/lib/os_mon-2.7.1/ebin",
"/usr/local/lib/erlang/lib/odbc-2.13.5/ebin",
"/usr/local/lib/erlang/lib/observer-2.10.1/ebin",
"/usr/local/lib/erlang/lib/mnesia-4.20.1/ebin",
"/usr/local/lib/erlang/lib/megaco-4.2/ebin",
"/usr/local/lib/erlang/lib/inets-7.5/ebin",
"/usr/local/lib/erlang/lib/hipe-4.0.1/ebin",
"/usr/local/lib/erlang/lib/ftp-1.1/ebin",
"/usr/local/lib/erlang/lib/eunit-2.7/ebin",
"/usr/local/lib/erlang/lib/et-1.6.5/ebin",
"/usr/local/lib/erlang/lib/erts-12.2/ebin",
"/usr/local/lib/erlang/lib/erl_interface-5.1/ebin",
[...]|...]
This list has . in it, but not ../part1, so when when we are compiling, been in the part2 directory, it fails...
2> c(module1).
c(module1).
{error,non_existing}
There are several ways to work around this, few simple ones could be...
c("../part1/module1.erl").. As per the documentation of c
...Module can be either a module name or a source file path, with or without .erl extension...
And here, in the option above, we used the relative path to the source file of module1.
Invoke erl with an option -pa which adds the path of part1 in the code path for that session of the erlang emulator.
part2$ erl -pa "../part1"
Erlang/OTP 25 [DEVELOPMENT] [erts-12.2] [source-c1ab4b5424] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [jit:ns]
Eshell V12.2 (abort with ^G)
1> code:get_path().
["../part1",".","/usr/local/lib/erlang/lib/kernel-8.2/ebin",
"/usr/local/lib/erlang/lib/stdlib-3.17/ebin",
"/usr/local/lib/erlang/lib/xmerl-1.3.28/ebin",
"/usr/local/lib/erlang/lib/wx-2.1.1/ebin",
"/usr/local/lib/erlang/lib/tools-3.5.2/ebin",
"/usr/local/lib/erlang/lib/tftp-1.0.3/ebin",
"/usr/local/lib/erlang/lib/syntax_tools-2.6/ebin",
"/usr/local/lib/erlang/lib/ssl-10.6/ebin",
"/usr/local/lib/erlang/lib/ssh-4.13/ebin",
"/usr/local/lib/erlang/lib/snmp-5.11/ebin",
"/usr/local/lib/erlang/lib/sasl-4.1.1/ebin",
"/usr/local/lib/erlang/lib/runtime_tools-1.17/ebin",
"/usr/local/lib/erlang/lib/reltool-0.9/ebin",
"/usr/local/lib/erlang/lib/public_key-1.11.3/ebin",
"/usr/local/lib/erlang/lib/parsetools-2.3.2/ebin",
"/usr/local/lib/erlang/lib/os_mon-2.7.1/ebin",
"/usr/local/lib/erlang/lib/odbc-2.13.5/ebin",
"/usr/local/lib/erlang/lib/observer-2.10.1/ebin",
"/usr/local/lib/erlang/lib/mnesia-4.20.1/ebin",
"/usr/local/lib/erlang/lib/megaco-4.2/ebin",
"/usr/local/lib/erlang/lib/inets-7.5/ebin",
"/usr/local/lib/erlang/lib/hipe-4.0.1/ebin",
"/usr/local/lib/erlang/lib/ftp-1.1/ebin",
"/usr/local/lib/erlang/lib/eunit-2.7/ebin",
"/usr/local/lib/erlang/lib/et-1.6.5/ebin",
"/usr/local/lib/erlang/lib/erts-12.2/ebin",
[...]|...]
2> c(module2).
{ok,module2}
3> c(module1).
Recompiling /home/nalin/source/erlang/part2/../part1/module1.erl
{ok,module1}
4> module2:exec().
"Module2"
5> module1:exec().
"Module1"
6>
I hope this much should be sufficient to get you going. Also, you must take an opportunity to read through the Compilation and Code Loading to have some idea around what goes on.
WYSIWYG => WHAT YOU SHOW IS WHAT YOU GET

Erlang: programmatically get application version

I have an OTP application. The version of this application is found in two places: the src/application_name.src file (standard with OTP applications), and in my rebar.config.
Is there an "official" way for an application to get it's own version, or does this need to be hacked via sed/grep etc? I want to have an "info" endpoint in my application that prints it's own version. Of course I can always do something like grep the version out of my rebar.config, but it seems hacky.
According to the Erlang documentation you can use API of application module.
Example:
Erlang/OTP 19 [erts-8.2.2] [source-1ca84a4] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V8.2.2 (abort with ^G)
1> application:loaded_applications().
[{stdlib,"ERTS CXC 138 10","3.2"},
{kernel,"ERTS CXC 138 10","5.1.1"}]
2> GetVer =
fun(App) ->
case lists:keyfind(App, 1, application:loaded_applications()) of
{_, _, Ver} ->
Ver;
false ->
not_found
end
end.
#Fun<erl_eval.6.52032458>
3> GetVer(stdlib).
"3.2"
4> GetVer(eunit).
not_found
5> application:load(eunit).
ok
6> GetVer(eunit).
"2.3.2"
7>
If it is properly packaged as an application, you can get it using application:which_applications(). I have some sample code, but basically you'd do something like this:
vsn() -> vsn(your_application_atom_name).
vsn(Application) -> vsn(Application, application:which_applications()).
vsn(_Application, []) -> undefined;
vsn(Application, [{Application,_,Vsn}|_]) -> Vsn;
vsn(Application, [_|Rest]) -> vsn(Application, Rest).
The downside is that you do have to hardcode the name of your application (as an atom). I never found a way around this, but you might be able to cook something up.
application:get_key/1,2 can give you the version number of the application.
1> application:get_key(tfs, vsn).
{ok,"94a3d72"}
2> application:get_key(mnesia, vsn).
{ok,"4.16.1"}

Erlang: where does the unmatched message go?

When tried to send unmatched message to a spawned process in erlang shell, I was expecting the message should remain in the mailbox, but it seemed like the mailbox is empty, why?
Erlang R15B02 (erts-5.9.2) [smp:2:2] [async-threads:0]
Eshell V5.9.2 (abort with ^G)
1> Pid = spawn(fun()->receive stop->stop end end).
<0.33.0>
2> Pid ! msg.
msg
3> erlang:process_info(Pid, messages).
{messages,[]} %% where is the msg?
When the message can't be matched against a receive pattern it is moved from the mailbox to a save queue, see http://ndpar.blogspot.se/2010/11/erlang-explained-selective-receive.html for a detailed explanation of what happens.
The messages parameter to process_info/2 only shows the mailbox contents, AFAIK there is no way to inspect the contents of the save queue.
The message is of course there and it will be checked in subsequent receives. The fact that you can't see it with erlang:process_info(Pid, messages) is, in my opinion, weird.
(ppb2_bs6#esekilvxen245)1> self() ! a.
a
(ppb2_bs6#esekilvxen245)2> erlang:process_info(self(), messages).
{messages,[a]}

Erlang: How to write my outputs in a text file?

I need to write my outputs from shell into a text file to keeping some required records. Can anyone please tell me how can I do this?
Thanks
If you have the data you wish to store as a single term you can read how here. In the simplest case you can just do file:write_file(Path, Data). If your data is more complex you may want to use io_lib:fwrite/2 to format it in a more readable way. For example:
Data = [1,2,3,{car, "honda"}],
file:write_file("/tmp/foo", io_lib:fwrite("~p.\n", [Data])).
tee command can capture shell output and save it to a file:
$ erl | tee output.txt
Eshell V5.8 (abort with ^G)
1> A = 5.
5
2> 5 + A.
10
3> ^Ca
$ cat output.txt
Eshell V5.8 (abort with ^G)
1> 5
2> 10
3>
There are many possibilities. Here's the first one which came to my mind.
In Erlang, every process is part of a group. For each group, a process - named group leader gets all the output from the group-mates. The shell process is part of the group init. So, what you could do is to change the group leader for the shell process by doing:
{ok, Log} = file:open("log", [write]),
erlang:group_leader(Log, self()).
You might want to create a dedicated process acting as group leader who manages the output in a more clever way. For example, it could wrap the file after a while or when the file reaches a certain size.
The problem with this approach is that you have to execute these lines any time you start your shell. Doing it by using the -s flag:
erl -s shell_log
where shell_log is:
-module(shell_log).
-export([start/0]).
start() ->
{ok, Log} = file:open("log", [write]),
erlang:group_leader(Log, self()).
won't work, because it's still too early (init is still in {starting, started}, as reported by init:get_status()).
In the same way, doing it by using the .erlang file in your HOME directory (whose lines are automatically executed every time you start a shell) will fail for a similar reason (init is still in the {starting, applications_loaded} state).
Not sure which is the best way to circumvent this problem. I'll think about it.
Finally, this question seems a duplicate of this thread.

Query an Erlang process for its state?

A common pattern in Erlang is the recursive loop that maintains state:
loop(State) ->
receive
Msg ->
NewState = whatever(Msg),
loop(NewState)
end.
Is there any way to query the state of a running process with a bif or tracing or something? Since crash messages say "...when state was..." and show the crashed process's state, I thought this would be easy, but I was disappointed that I haven't been able to find a bif to do this.
So, then, I figured using the dbg module's tracing would do it. Unfortunately, I believe because these loops are tail call optimized, dbg will only capture the first call to the function.
Any solution?
If your process is using OTP, it is enough to do sys:get_status(Pid).
The error message you mentions is displayed by SASL. SASL is an error reporting daemon in OTP.
The state you are referring in your example code is just an argument of tail recursive function. There is no way to extract it using anything except for tracing BIFs. I guess this would be not a proper solution in production code, since tracing is intended to be used only for debug purposes.
Proper, and industry tested, solution would be make extensive use of OTP in your project. Then you can take full advantage of SASL error reporting, rb module to collect these reports, sys - to inspect the state of the running OTP-compatible process, proc_lib - to make short-lived processes OTP-compliant, etc.
It turns out there's a better answer than all of these, if you're using OTP:
sys:get_state/1
Probably it didn't exist at the time.
It looks like you're making the problem out of nothing. erlang:process_info/1 gives enough information for debugging purposes. If your REALLY need loop function arguments, why don't you give it back to caller in response to one of the special messages that you define yourself?
UPDATE:
Just to clarify terminology. The closest thing to the 'state of the process' on the language level is process dictionary, usage of which is highly discouraged. It can be queried by erlang:process_info/1 or erlang:process/2.
What you actually need is to trace process's local functions calls along with their arguments:
-module(ping).
-export([start/0, send/1, loop/1]).
start() ->
spawn(?MODULE, loop, [0]).
send(Pid) ->
Pid ! {self(), ping},
receive
pong ->
pong
end.
loop(S) ->
receive
{Pid, ping} ->
Pid ! pong,
loop(S + 1)
end.
Console:
Erlang (BEAM) emulator version 5.6.5 [source] [smp:2] [async-threads:0] [kernel-poll:false]
Eshell V5.6.5 (abort with ^G)
1> l(ping).
{module,ping}
2> erlang:trace(all, true, [call]).
23
3> erlang:trace_pattern({ping, '_', '_'}, true, [local]).
5
4> Pid = ping:start().
<0.36.0>
5> ping:send(Pid).
pong
6> flush().
Shell got {trace,<0.36.0>,call,{ping,loop,[0]}}
Shell got {trace,<0.36.0>,call,{ping,loop,[1]}}
ok
7>
{status,Pid,_,[_,_,_,_,[_,_,{data,[{_,State}]}]]} = sys:get_status(Pid).
That's what I use to get the state of a gen_server. (Tried to add it as a comment to the reply above, but couldn't get formatting right.)
As far as I know you cant get the arguments passed to a locally called function. I would love for someone to prove me wrong.
-module(loop).
-export([start/0, loop/1]).
start() ->
spawn_link(fun () -> loop([]) end).
loop(State) ->
receive
Msg ->
loop([Msg|State])
end.
If we want to trace this module you do the following in the shell.
dbg:tracer().
dbg:p(new,[c]).
dbg:tpl(loop, []).
Using this tracing setting you get to see local calls (the 'l' in tpl means that local calls will be traced as well, not only global ones).
5> Pid = loop:start().
(<0.39.0>) call loop:'-start/0-fun-0-'/0
(<0.39.0>) call loop:loop/1
<0.39.0>
6> Pid ! foo.
(<0.39.0>) call loop:loop/1
foo
As you see, just the calls are included. No arguments in sight.
My recommendation is to base correctness in debugging and testing on the messages sent rather than state kept in processes. I.e. if you send the process a bunch of messages, assert that it does the right thing, not that it has a certain set of values.
But of course, you could also sprinkle some erlang:display(State) calls in your code temporarily. Poor man's debugging.
This is a "oneliner" That can be used in the shell.
sys:get_status(list_to_pid("<0.1012.0>")).
It helps you convert a pid string into a Pid.

Resources