I need to write my outputs from shell into a text file to keeping some required records. Can anyone please tell me how can I do this?
Thanks
If you have the data you wish to store as a single term you can read how here. In the simplest case you can just do file:write_file(Path, Data). If your data is more complex you may want to use io_lib:fwrite/2 to format it in a more readable way. For example:
Data = [1,2,3,{car, "honda"}],
file:write_file("/tmp/foo", io_lib:fwrite("~p.\n", [Data])).
tee command can capture shell output and save it to a file:
$ erl | tee output.txt
Eshell V5.8 (abort with ^G)
1> A = 5.
5
2> 5 + A.
10
3> ^Ca
$ cat output.txt
Eshell V5.8 (abort with ^G)
1> 5
2> 10
3>
There are many possibilities. Here's the first one which came to my mind.
In Erlang, every process is part of a group. For each group, a process - named group leader gets all the output from the group-mates. The shell process is part of the group init. So, what you could do is to change the group leader for the shell process by doing:
{ok, Log} = file:open("log", [write]),
erlang:group_leader(Log, self()).
You might want to create a dedicated process acting as group leader who manages the output in a more clever way. For example, it could wrap the file after a while or when the file reaches a certain size.
The problem with this approach is that you have to execute these lines any time you start your shell. Doing it by using the -s flag:
erl -s shell_log
where shell_log is:
-module(shell_log).
-export([start/0]).
start() ->
{ok, Log} = file:open("log", [write]),
erlang:group_leader(Log, self()).
won't work, because it's still too early (init is still in {starting, started}, as reported by init:get_status()).
In the same way, doing it by using the .erlang file in your HOME directory (whose lines are automatically executed every time you start a shell) will fail for a similar reason (init is still in the {starting, applications_loaded} state).
Not sure which is the best way to circumvent this problem. I'll think about it.
Finally, this question seems a duplicate of this thread.
Related
I have a pretty simple doubt, but I can't seem to find a proper solution for it anywhere.
I have 2 erlang modules, module1.erl and module2.erl. As defined by the submission guidelines for my project, both modules belong to different parts and are hence in different folders part1 and part2 respectively under the same directory. This is how the structure looks:
src/
part1/
module1.erl
part2/
module2.erl
Now module2 is dependent on module1, and calls various methods of module1 as module1:method(). I'm able to achieve full functionality when both module1.erl and module2.erl are in the same folder, but now they're in different folders, and I try to run module2.erl from part2 folder, I can't figure out how to allow module2 to compile and call the methods of module1.
Since the emulator is being used, the path to module1 is not in the code path by default when we are launching the emulator in the path of module2 directly and with default options. This can be verified using ...
1> code:get_path().
code:get_path().
[".","/usr/local/lib/erlang/lib/kernel-8.2/ebin",
"/usr/local/lib/erlang/lib/stdlib-3.17/ebin",
"/usr/local/lib/erlang/lib/xmerl-1.3.28/ebin",
"/usr/local/lib/erlang/lib/wx-2.1.1/ebin",
"/usr/local/lib/erlang/lib/tools-3.5.2/ebin",
"/usr/local/lib/erlang/lib/tftp-1.0.3/ebin",
"/usr/local/lib/erlang/lib/syntax_tools-2.6/ebin",
"/usr/local/lib/erlang/lib/ssl-10.6/ebin",
"/usr/local/lib/erlang/lib/ssh-4.13/ebin",
"/usr/local/lib/erlang/lib/snmp-5.11/ebin",
"/usr/local/lib/erlang/lib/sasl-4.1.1/ebin",
"/usr/local/lib/erlang/lib/runtime_tools-1.17/ebin",
"/usr/local/lib/erlang/lib/reltool-0.9/ebin",
"/usr/local/lib/erlang/lib/public_key-1.11.3/ebin",
"/usr/local/lib/erlang/lib/parsetools-2.3.2/ebin",
"/usr/local/lib/erlang/lib/os_mon-2.7.1/ebin",
"/usr/local/lib/erlang/lib/odbc-2.13.5/ebin",
"/usr/local/lib/erlang/lib/observer-2.10.1/ebin",
"/usr/local/lib/erlang/lib/mnesia-4.20.1/ebin",
"/usr/local/lib/erlang/lib/megaco-4.2/ebin",
"/usr/local/lib/erlang/lib/inets-7.5/ebin",
"/usr/local/lib/erlang/lib/hipe-4.0.1/ebin",
"/usr/local/lib/erlang/lib/ftp-1.1/ebin",
"/usr/local/lib/erlang/lib/eunit-2.7/ebin",
"/usr/local/lib/erlang/lib/et-1.6.5/ebin",
"/usr/local/lib/erlang/lib/erts-12.2/ebin",
"/usr/local/lib/erlang/lib/erl_interface-5.1/ebin",
[...]|...]
This list has . in it, but not ../part1, so when when we are compiling, been in the part2 directory, it fails...
2> c(module1).
c(module1).
{error,non_existing}
There are several ways to work around this, few simple ones could be...
c("../part1/module1.erl").. As per the documentation of c
...Module can be either a module name or a source file path, with or without .erl extension...
And here, in the option above, we used the relative path to the source file of module1.
Invoke erl with an option -pa which adds the path of part1 in the code path for that session of the erlang emulator.
part2$ erl -pa "../part1"
Erlang/OTP 25 [DEVELOPMENT] [erts-12.2] [source-c1ab4b5424] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [jit:ns]
Eshell V12.2 (abort with ^G)
1> code:get_path().
["../part1",".","/usr/local/lib/erlang/lib/kernel-8.2/ebin",
"/usr/local/lib/erlang/lib/stdlib-3.17/ebin",
"/usr/local/lib/erlang/lib/xmerl-1.3.28/ebin",
"/usr/local/lib/erlang/lib/wx-2.1.1/ebin",
"/usr/local/lib/erlang/lib/tools-3.5.2/ebin",
"/usr/local/lib/erlang/lib/tftp-1.0.3/ebin",
"/usr/local/lib/erlang/lib/syntax_tools-2.6/ebin",
"/usr/local/lib/erlang/lib/ssl-10.6/ebin",
"/usr/local/lib/erlang/lib/ssh-4.13/ebin",
"/usr/local/lib/erlang/lib/snmp-5.11/ebin",
"/usr/local/lib/erlang/lib/sasl-4.1.1/ebin",
"/usr/local/lib/erlang/lib/runtime_tools-1.17/ebin",
"/usr/local/lib/erlang/lib/reltool-0.9/ebin",
"/usr/local/lib/erlang/lib/public_key-1.11.3/ebin",
"/usr/local/lib/erlang/lib/parsetools-2.3.2/ebin",
"/usr/local/lib/erlang/lib/os_mon-2.7.1/ebin",
"/usr/local/lib/erlang/lib/odbc-2.13.5/ebin",
"/usr/local/lib/erlang/lib/observer-2.10.1/ebin",
"/usr/local/lib/erlang/lib/mnesia-4.20.1/ebin",
"/usr/local/lib/erlang/lib/megaco-4.2/ebin",
"/usr/local/lib/erlang/lib/inets-7.5/ebin",
"/usr/local/lib/erlang/lib/hipe-4.0.1/ebin",
"/usr/local/lib/erlang/lib/ftp-1.1/ebin",
"/usr/local/lib/erlang/lib/eunit-2.7/ebin",
"/usr/local/lib/erlang/lib/et-1.6.5/ebin",
"/usr/local/lib/erlang/lib/erts-12.2/ebin",
[...]|...]
2> c(module2).
{ok,module2}
3> c(module1).
Recompiling /home/nalin/source/erlang/part2/../part1/module1.erl
{ok,module1}
4> module2:exec().
"Module2"
5> module1:exec().
"Module1"
6>
I hope this much should be sufficient to get you going. Also, you must take an opportunity to read through the Compilation and Code Loading to have some idea around what goes on.
WYSIWYG => WHAT YOU SHOW IS WHAT YOU GET
First a disclaimer I am learning erlang. Not an expert here at all.
While making some examples using ETS I came across something I am not understanding (even after searching).
I have a process where I create a public ETS with
TableID = ets:new(tablename, [public])}
I then pass TableID to other processes. When I do this running the modules form the shell, all is ok. When I run exactly the same modules with erl -noshell -s ... or even without the -noshell option, it fails.
I keep getting error:badarg as if the tabled does not exist. The ID is properly passes, but the table is actually behaving as private!
Is there a difference between running modules interactively from the shell or without?
Thanks
I am adding an example of the code I am using to try and debug the issue. As it is a piece of a larger software (and it is basically stripped to the bone to find the issue), it might be difficult to understand.
-record(loop_state, {
commands
}).
start() ->
LoopState = #loop_state{commands = ets:new(commands, [public])},
tcpserver_otp_backend:start(?MODULE, 7000, {?MODULE, loop}, LoopState).
loop(Socket, LoopState = #loop_state{commands = Commands}) ->
case gen_tcp:recv(Socket, 0) of
{ok, Data} ->
% the call below fails, no error generated, AND only in non interactive shell
A = newCommand(Commands, Data),
gen_tcp:send(Socket, A),
loop(Socket, LoopState);
{error, closed} ->
ok
end.
newCommand(CommandTableId, Command) ->
case ets:lookup(CommandTableId,Command) of
[] ->
_A = ets:insert(CommandTableId, {Command, 1}),
<<1, "new", "1">>; % used for testing
_ ->
<<1, "old", "1">> % used for testing
end.
When I remove the "offending command" ets:lookup, all works again as int he interactive shell.
The problem seems to be that you create the ets table in your start() function. An ets table has an owner (by default the creating process), and when the owner dies, the table gets deleted. When you run the start/0 function from the command line by passing -s to erl, that owner process will be some internal process in the Erlang kernel that is part of handling the startup sequence. Regardless of whether you pass -noshell or not, that process is probably transient and will die before the lookup function gets time to execute, so the table no longer exists when the lookup finally happens.
The proper place to create the ets table would be in the init() callback function of the gen_server that you start up. If it's supposed to be a public est table accessed by several processes, then you might want to create a separate server process whose task it is to own the table.
I am working on my first real project in erlang, however, this code is simplified for brevity. I want to be able to load a newer version of a file into my project remotely while it's running. I've read about using a behavior like gen_server or gen_fsm which has this for free. While that might achieve the result, I want to use this to learn how to do it, not just get it done. I've read the docs about code replacement, and LYSE's bit about Hot Code Loving, among other things, but I haven't been able to find anything that works for what I'm doing, so here is the basic idea.
-module(reloading).
-export([loop/0]).
loop() ->
receive
upgrade ->
?MODULE:loop();
hello ->
io:format("This is a test~n"),
loop();
_ ->
loop()
end.
I am simply looping with the idea that I can send the message upgrade and it will load a newer version of the code.
$ erl
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.9.1 (abort with ^G)
1> c(reloading).
{ok,reloading}
2> Loop = spawn(reloading, loop, []).
<0.39.0>
3> Loop ! hello.
This is a test
hello
At this point I change 10 line to io:format("I have changed this!~n"),
4> Loop ! upgrade.
upgrade
5> Loop ! hello.
This is a test
hello
I am expecting this hello call to print I have changed this! not This is a test. I know I can simply call c(reloading). and have this work the way expected, but I'm looking to send the actual project a message rather than manually updating the code. So where is my disconnect? What am I doing wrong, that I should be doing in order to hot load this code? As mentioned before, I am looking for a non-OTP solution for the sake of education.
For the sake of having an explicit answer, I am posting this.
Using #rvirding's suggestion of using the code module, I've modified it to look like this:
-module(reloading).
-export([loop/0]).
loop() ->
receive
upgrade ->
code:purge(?MODULE),
compile:file(?MODULE),
code:load_file(?MODULE),
?MODULE:loop();
hello ->
io:format("This is a test~n"),
loop();
_ ->
loop()
end.
First code:purge the old ?MODULE, then compile:file the new file, and finally, code:load_file the new ?MODULE.
This works as I originally intended.
$ erl
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.9.1 (abort with ^G)
1> Loop = spawn(reloading, loop, []).
<0.34.0>
2> Loop ! hello.
This is a test
hello
Change line to io:format("I have changed this!~n"),
3> Loop ! upgrade.
upgrade
4> Loop ! hello.
I have changed this!
hello
While erlang can handle two versions of a module and calling a function with mod:func(...) will always call the latest version of a module (if the function is exported) you still have to load the new version of the module into Erlang system. You can't expect it automagically detect that you happen to have a new version of the module somewhere, find it, compile it and load it.
N.B. compiling and loading are two separate things. So c(mod). both compiles and loads the module, while l(mod). just loads the object code (.beam file) of the already compiled module. The Erlang compiler is called from the module compile and it just compiles and generates a .beam file while the code loading is handled by the module code.
In addition to the above I'd like to notice that some tools that reload code automatically for you exist.
You should have a look at sync or active projects.
Compile *.beam locally, then send it to your server and reload it as mentioned in man pages:
http://erlang.org/documentation/doc-1/reference_manual/code_loading.html#id86381
-module(m).
-export([loop/0]).
loop() ->
receive
code_switch ->
m:loop();
Msg ->
...
loop()
end.
I want to run tail on a named pipe to facilitate some binary logfile processing. The problem is that mysterious data is being added to the beginning of the stream. I run my tests by starting the erlang process with the opened port (open_port) and then I use another shell to cat the bin into the named pipe.
Here is a simple function for getting data from the port:
bin_from_tail() ->
open_port({spawn,"/usr/bin/tail -F named_pipe"},
[binary,in,eof]),
receive
{_,{data,<<Data/binary>>}} -> Data
end.
So here are two ways for me to grab the same data...
Create the named pipe
mkfifo named_pipe
This command blocks until you run "cat log.bin > named_pipe" from another shell
{ok,TailBin} = file:read_file(log.bin).
Read the entire file into memory using the erlang file library
FileBin = file:read_file(log.in).
But TailBin and FileBin are not the same! TailBin has a mysterious 120-byte string at the beginning:
<<40,6,161,69,172,216,56,14,100,0,80,6,0,0,0>>
Thanks for the idea about the endlessly looping cat/restarting a dead port. It appears that named pipes buffer just a little bit, so if the port opens up fast enough the writer process (another program) won't crash! Definitely risky stuff, but as far as hacks go... it works.
Because all the mailing list posts just said do this, do that without examples, I'm going to post how mine works! If anyone wants to offer up improvements, please feel free to do so. My solution:
read() ->
Port = open_port({spawn,"/bin/cat /path/to/pipe"},
[binary,in,eof]),
do_read(Port).
do_read(Port) ->
receive
{Port,{data,<<Data/binary>>}} ->
case do_something:with(Data) of
ok ->
io:format("G") % Good
Any ->
io:format("B") % Bad
end;
{Port,eof} ->
read();
Any ->
io:format("No match fifo_client:do_read/1, ~p~n",[Any])
end,
do_read(Port).
I found the same thing happened outside erlang. The problem is that tail is trying to show you the end of the file, not the whole file. If you use it on a normal file, anything written would be new, and picked up by -f, but in this case it looks like tail is waiting until the end of the file (the eof that comes through the pipe) and then showing the last 10 lines (treating the binary as text).
tail -F -c 9999999
(assuming your log is 9999999 bytes or less) would probably work.
Maybe try using cat instead of tail -F, that seemed to work for me. Then you just need to avoid the fact that cat exits upon eof, which I assume you were trying to avoid by using tail.
So a shell script which loops cat endlessly, maybe?
Or get erlang to restart close and recreate the port when it dies, since you're getting the eof signal anyway. Or use the exit_status flag to open_port to be signalled when the process exits, incase you need to distinguish eof and process exit. (If you use both exit_status and eof, the eof never comes, a brief test with cat < /dev/null indicates)
A common pattern in Erlang is the recursive loop that maintains state:
loop(State) ->
receive
Msg ->
NewState = whatever(Msg),
loop(NewState)
end.
Is there any way to query the state of a running process with a bif or tracing or something? Since crash messages say "...when state was..." and show the crashed process's state, I thought this would be easy, but I was disappointed that I haven't been able to find a bif to do this.
So, then, I figured using the dbg module's tracing would do it. Unfortunately, I believe because these loops are tail call optimized, dbg will only capture the first call to the function.
Any solution?
If your process is using OTP, it is enough to do sys:get_status(Pid).
The error message you mentions is displayed by SASL. SASL is an error reporting daemon in OTP.
The state you are referring in your example code is just an argument of tail recursive function. There is no way to extract it using anything except for tracing BIFs. I guess this would be not a proper solution in production code, since tracing is intended to be used only for debug purposes.
Proper, and industry tested, solution would be make extensive use of OTP in your project. Then you can take full advantage of SASL error reporting, rb module to collect these reports, sys - to inspect the state of the running OTP-compatible process, proc_lib - to make short-lived processes OTP-compliant, etc.
It turns out there's a better answer than all of these, if you're using OTP:
sys:get_state/1
Probably it didn't exist at the time.
It looks like you're making the problem out of nothing. erlang:process_info/1 gives enough information for debugging purposes. If your REALLY need loop function arguments, why don't you give it back to caller in response to one of the special messages that you define yourself?
UPDATE:
Just to clarify terminology. The closest thing to the 'state of the process' on the language level is process dictionary, usage of which is highly discouraged. It can be queried by erlang:process_info/1 or erlang:process/2.
What you actually need is to trace process's local functions calls along with their arguments:
-module(ping).
-export([start/0, send/1, loop/1]).
start() ->
spawn(?MODULE, loop, [0]).
send(Pid) ->
Pid ! {self(), ping},
receive
pong ->
pong
end.
loop(S) ->
receive
{Pid, ping} ->
Pid ! pong,
loop(S + 1)
end.
Console:
Erlang (BEAM) emulator version 5.6.5 [source] [smp:2] [async-threads:0] [kernel-poll:false]
Eshell V5.6.5 (abort with ^G)
1> l(ping).
{module,ping}
2> erlang:trace(all, true, [call]).
23
3> erlang:trace_pattern({ping, '_', '_'}, true, [local]).
5
4> Pid = ping:start().
<0.36.0>
5> ping:send(Pid).
pong
6> flush().
Shell got {trace,<0.36.0>,call,{ping,loop,[0]}}
Shell got {trace,<0.36.0>,call,{ping,loop,[1]}}
ok
7>
{status,Pid,_,[_,_,_,_,[_,_,{data,[{_,State}]}]]} = sys:get_status(Pid).
That's what I use to get the state of a gen_server. (Tried to add it as a comment to the reply above, but couldn't get formatting right.)
As far as I know you cant get the arguments passed to a locally called function. I would love for someone to prove me wrong.
-module(loop).
-export([start/0, loop/1]).
start() ->
spawn_link(fun () -> loop([]) end).
loop(State) ->
receive
Msg ->
loop([Msg|State])
end.
If we want to trace this module you do the following in the shell.
dbg:tracer().
dbg:p(new,[c]).
dbg:tpl(loop, []).
Using this tracing setting you get to see local calls (the 'l' in tpl means that local calls will be traced as well, not only global ones).
5> Pid = loop:start().
(<0.39.0>) call loop:'-start/0-fun-0-'/0
(<0.39.0>) call loop:loop/1
<0.39.0>
6> Pid ! foo.
(<0.39.0>) call loop:loop/1
foo
As you see, just the calls are included. No arguments in sight.
My recommendation is to base correctness in debugging and testing on the messages sent rather than state kept in processes. I.e. if you send the process a bunch of messages, assert that it does the right thing, not that it has a certain set of values.
But of course, you could also sprinkle some erlang:display(State) calls in your code temporarily. Poor man's debugging.
This is a "oneliner" That can be used in the shell.
sys:get_status(list_to_pid("<0.1012.0>")).
It helps you convert a pid string into a Pid.