TLDR;
When starting erlang node (using just erl command for an instance), how could I force it not to use local OTP libraries and get code:get_path() empty?
Rationale.
I want to touch erl_boot_server. Don't doing something certain, just playing. I have built sample release and want to load it via network. Here is it.
[vkovalev#t30nix foobar]$ tree -L 2
.
|-- bin
| |-- foobar
| |-- foobar-0.0.0+build.1.ref307ae38
| |-- install_upgrade.escript
| |-- nodetool
| `-- start_clean.boot
|-- erts-6.1
| |-- bin
| |-- doc
| |-- include
| |-- lib
| |-- man
| `-- src
|-- lib
| |-- foobar-0.1.0
| |-- kernel-3.0.1
| |-- sasl-2.4
| `-- stdlib-2.1
`-- releases
|-- 0.0.0+build.1.ref307ae38
|-- RELEASES
`-- start_erl.data
First I start boot node.
[vkovalev#t30nix foobar]$ erl -sname boot -pa lib/*/ebin -pa releases/0.0.0+build.1.ref307ae38/ -s erl_boot_server start localhost
(boot#t30nix)1> {ok, _, _} = erl_prim_loader:get_file("foobar.boot").
(boot#t30nix)2> {ok, _, _} = erl_prim_loader:get_file("foobar_app.beam").
As you can see, all okay here. Then I start slave node:
[vkovalev#t30nix ~]$ erl -sname slave -loader inet -hosts 127.0.0.1 -boot foobar
{"init terminating in do_boot",{'cannot get bootfile','foobar.boot'}}
Crash dump was written to: erl_crash.dump
init terminating in do_boot ()
I dug into erl_prim_loader and found that stuff. One clause acts when Paths is empty (it just forward requested filename to boot server as is), another acts when Paths is non-empty. In this case (I wonder why) prim loader cripples requested file name with its own (clientside) paths and then ask SERVER to serve this path. In my understanding this is quite weird thing, but okay. Then I checked code:get_path() on slave node, and yes, it has paths to local otp installation.
So, returning to subject. How could I force slave node not to use any local OTP installation (if it already presents)?
UPD: Added more investigation results.
First thing -
https://github.com/erlang/otp/blob/maint/erts/preloaded/src/erl_prim_loader.erl#L669.
erl_prim_loader (in inet mode) for some (unclear for me) reasons tries
to cripple any requested module with local (clientside) paths.
It seems there is no way to force loader on slave node to keep its
paths empty: https://github.com/erlang/otp/blob/maint/erts/preloaded/src/init.erl#L697
Paths in my bootscript looks like
{path,["$ROOT/lib/kernel-4.0/ebin","$ROOT/lib/stdlib-2.5/ebin"]}, so
it seems, if I'll get bootscript loaded, anyway, I won't be able boot
system with it.
What's going on? Is erlang network boot feature broken? Or just my
brains? How could I get node successfully network-booted?
Do you think node of slave type is just named by "slave"?
The following code from the project tsung, file named "ts_os_mon_erlang.erl".
start_beam(Host) ->
Args = ts_utils:erl_system_args(),
?LOGF("Starting os_mon beam on host ~p ~n", [Host], ?NOTICE),
?LOGF("~p Args: ~p~n", [Host, Args], ?DEB),
slave:start(list_to_atom(Host), ?NODE, Args).
In additon, the slave module restriction as follows:
Slave nodes on other hosts than the current one are started with the program rsh. The user must be allowed to rsh to the remote hosts without being prompted for a password. This can be arranged in a number of ways (refer to the rsh documentation for details). A slave node started on the same host as the master inherits certain environment values from the master, such as the current directory and the environment variables. For what can be assumed about the environment when a slave is started on another host, read the documentation for the rsh program.
An alternative to the rsh program can be specified on the command line to erl as follows: -rsh Program.
The slave node should use the same file system at the master. At least, Erlang/OTP should be installed in the same place on both computers and the same version of Erlang should be used.
If you want to start a node with different path, I think you could do it by script with differnt environment variable, for master node, not slave node.
I think rebar project's can help for the similiar purpose. It include how to manipulate the path:
From rebar_core.erl file:
process_dir1(Dir, Command, DirSet, Config, CurrentCodePath,
{DirModules, ModuleSetFile}) ->
Config0 = rebar_config:set(Config, current_command, Command),
%% Get the list of modules for "any dir". This is a catch-all list
%% of modules that are processed in addition to modules associated
%% with this directory type. These any_dir modules are processed
%% FIRST.
{ok, AnyDirModules} = application:get_env(rebar, any_dir_modules),
Modules = AnyDirModules ++ DirModules,
%% Invoke 'preprocess' on the modules -- this yields a list of other
%% directories that should be processed _before_ the current one.
{Config1, Predirs} = acc_modules(Modules, preprocess, Config0,
ModuleSetFile),
%% Remember associated pre-dirs (used for plugin lookup)
PredirsAssoc = remember_cwd_predirs(Dir, Predirs),
%% Get the list of plug-in modules from rebar.config. These
%% modules may participate in preprocess and postprocess.
{ok, PluginModules} = plugin_modules(Config1, PredirsAssoc),
{Config2, PluginPredirs} = acc_modules(PluginModules, preprocess,
Config1, ModuleSetFile),
AllPredirs = Predirs ++ PluginPredirs,
?DEBUG("Predirs: ~p\n", [AllPredirs]),
{Config3, DirSet2} = process_each(AllPredirs, Command, Config2,
ModuleSetFile, DirSet),
%% Make sure the CWD is reset properly; processing the dirs may have
%% caused it to change
ok = file:set_cwd(Dir),
%% Check that this directory is not on the skip list
Config7 = case rebar_config:is_skip_dir(Config3, Dir) of
true ->
%% Do not execute the command on the directory, as some
%% module has requested a skip on it.
?INFO("Skipping ~s in ~s\n", [Command, Dir]),
Config3;
false ->
%% Check for and get command specific environments
{Config4, Env} = setup_envs(Config3, Modules),
%% Execute any before_command plugins on this directory
Config5 = execute_pre(Command, PluginModules,
Config4, ModuleSetFile, Env),
%% Execute the current command on this directory
Config6 = execute(Command, Modules ++ PluginModules,
Config5, ModuleSetFile, Env),
%% Execute any after_command plugins on this directory
execute_post(Command, PluginModules,
Config6, ModuleSetFile, Env)
end,
%% Mark the current directory as processed
DirSet3 = sets:add_element(Dir, DirSet2),
%% Invoke 'postprocess' on the modules. This yields a list of other
%% directories that should be processed _after_ the current one.
{Config8, Postdirs} = acc_modules(Modules ++ PluginModules, postprocess,
Config7, ModuleSetFile),
?DEBUG("Postdirs: ~p\n", [Postdirs]),
Res = process_each(Postdirs, Command, Config8,
ModuleSetFile, DirSet3),
%% Make sure the CWD is reset properly; processing the dirs may have
%% caused it to change
ok = file:set_cwd(Dir),
%% Once we're all done processing, reset the code path to whatever
%% the parent initialized it to
restore_code_path(CurrentCodePath),
%% Return the updated {config, dirset} as result
Res.
restore_code_path(no_change) ->
ok;
restore_code_path({added, Paths}) ->
%% Verify that all of the paths still exist -- some dynamically
%% added paths can get blown away during clean.
[code:del_path(F) || F <- Paths, erl_prim_loader_is_file(F)],
ok.
erl_prim_loader_is_file(File) ->
erl_prim_loader:read_file_info(File) =/= error.
Make sure you use the -setcookie option. From the erl -man erl page:
-loader Loader:
Specifies the method used by erl_prim_loader to
load Erlang modules in to the system. See
erl_prim_loader(3). Two Loader methods are supported,
efile and inet. efile means use the local file
system, this is the default. inet means use a boot
server on another machine, and the -id, -hosts and
-setcookie flags must be specified as well. If Loader
is something else, the user supplied Loader port
program is started.
Related
I booted the Erlang shell with sasl started and with the following configuration file, but it doesn't create an error log on disk:
%% rotating log and minimal tty
[{sasl, [
{sasl_error_logger, false},
%% define the parameters of the rotating log
%% the log file directory
{error_logger_mf_dir,"ErrorLogger/erLog"},
%% # bytes per logfile
{error_logger_mf_maxbytes,10485760}, % 10 MB
%% maximum number of logfiles
{error_logger_mf_maxfiles, 10}
]}].
However if I place "." as the directory value, it creates unnamed logs on disk: {error_logger_mf_dir,"."}
You must provide absolute or relative address for error_logger_mf_dir config.
Absolute:
%% the log file absolute directory
{error_logger_mf_dir,"/path/to/ErrorLogger/erLog"},
Relative:
%% the log file based on current directory
{error_logger_mf_dir,"./ErrorLogger/erLog"},
%% Also it must work
{error_logger_mf_dir,"ErrorLogger/erLog"},
Note: Make sure that your Erlang application has write access to ErrorLogger/erLog directory. If so sasl will create there one file called index, and report files 1, 2, 3 and so on which should be browsed by the Report Browser (rb) tool.
I am using "rebar shell" to test my app. This is documented as:
Start a shell with project and deps preloaded similar to
'erl -pa ebin -pa deps/*/ebin'.
How do I add extra args to the underlying invocation of 'erl'? For
example, I want to add application specific environment variables and
run a Module/Function. I want to invoke something like:
erl -pa ebin -pa deps/*/ebin -browser_spy browser_exe "/my/dir" -run bs_example test
(and I want code:priv_dir to work as it does when using rebar shell,
which the above 'erl' command does not do).
You cannot
rebar shell does not execute erl ... command actually, but only tries to replicate its behaviour.
Actually rebar just turns yourself into the shell along with mimicking -pa by adding paths with code:add_pathz
See here for implementation details:
shell(_Config, _AppFile) ->
true = code:add_pathz(rebar_utils:ebin_dir()),
%% scan all processes for any with references to the old user and save them to
%% update later
NeedsUpdate = [Pid || Pid <- erlang:processes(),
proplists:get_value(group_leader, erlang:process_info(Pid)) == whereis(user)
],
%% terminate the current user
ok = supervisor:terminate_child(kernel_sup, user),
%% start a new shell (this also starts a new user under the correct group)
_ = user_drv:start(),
%% wait until user_drv and user have been registered (max 3 seconds)
ok = wait_until_user_started(3000),
%% set any process that had a reference to the old user's group leader to the
%% new user process
_ = [erlang:group_leader(whereis(user), Pid) || Pid <- NeedsUpdate],
%% enable error_logger's tty output
ok = error_logger:swap_handler(tty),
%% disable the simple error_logger (which may have been added multiple
%% times). removes at most the error_logger added by init and the
%% error_logger added by the tty handler
ok = remove_error_handler(3),
%% this call never returns (until user quits shell)
timer:sleep(infinity).
I have the following Erlang code:
#!/usr/bin/env escript
%%! -pz ../deps/amqp_client ../deps/rabbit_common ../deps/amqp_client/ebin ../deps/rabbit_common/ebin
% RMQ module
-module(rmq).
-export([main/1, send/1, validate/0, test/0]).
-include_lib("../deps/amqp_client/include/amqp_client.hrl").
main(_) ->
%send(<<"test_esio">>),
%validate(),
Pid = spawn(rmq, test, []),
% Pid = spawn(fun() -> test() end), <= I've tried this way too
Pid ! s.
test() ->
receive
s ->
io:format("BAR ~n"),
send(<<"esio">>),
test();
get ->
validate(),
test();
_ ->
io:format("FOO"),
test()
end.
I run this with:
excript rmq.erl
This code doesn't work. Looks like spawn doesn't work.
Rest of my code works, function send and validate works correctly if I run it from main (I've commented its). What I'm doing wrong?
Sorry, maybe it's a dumb question but I'm a beginner with erlang. I've tried search answer in internet and books and I failed...
The problem is not actually in spawn, but in module/escript confusion.
In few words, escript file are not really modules, not from point of Erlang VM, even if you use -module() directive. They are interpreted, and not compiled at all, and definitely they can not be called by module like "rmq:test()", or in you case trough dynamic module call by spawn.
Easiest solution is separate script from actual modules. In your rmq.es you would just start some proper module:
#!/usr/bin/env escript
%%! -pz ../deps/amqp_client ../deps/rabbit_common ../deps/amqp_client/ebin ../deps/rabbit_common/ebin
main(_) ->
rmq:start().
And there in module rmq.erl:
-module(rmq).
-export([start/0, send/1, validate/0, test/0]).
-include_lib("../deps/amqp_client/include/amqp_client.hrl").
start() ->
Pid = spawn(rmq, test, []),
%% Pid = spawn(?MODULE, test, []), %% works with macro too
%% Pid = spawn(fun() -> test() end), <= I've tried this way too
Pid ! s.
test() ->
receive
s ->
io:format("BAR ~n"),
send(<<"esio">>),
test();
get ->
validate(),
test();
_ ->
io:format("FOO"),
test()
end.
Or you could just start this module without escript, with -run flag like this
erl -pz deps/*/ebin -run rmq start
EDIT regarding compilation problems
You compile your modules with erlc command. To just compile use erlc rmq.erl, which will produce rmq.beam file in current directory. But convention is to keep all your source files in src directory, all compiled files in ebin direcory, and things like run-scripts could be placed in the top directory. Something like that:
project
|-- ebin
| |-- rmq.beam
|
|-- src
| |-- rmq.erl
|
|-- rmq.es
Assuming that you run all your shell commands form project directory, to compile all file from src and place .beam binaries in ebin use erlc src/rmq.erl -o ebin, or erlc src/* -o ebin In documentation you can find you explanation of -o flag"
-o directory
The directory where the compiler should place the output files. If not specified, output files will be placed in the current working directory.
Then, after compilation you can run your code, either with erl Erlang VM or using escript (which kind-off uses erl.
erl to runs code from compiled modules, and to do that he needs to be able to locate those compiled *.ebin binaries. For this he uses code path, which is the list of directors in which he will search for those files. This list automatically consist standard library directories, and of course you can add to it directories with your own code with use of -pa flag.
-pa Dir1 Dir2 ...
Adds the specified directories to the beginning of the code path, similar to code:add_pathsa/1. See code(3). As an alternative to -pa, if several directories are to be prepended to the code and the directories have a common parent directory, that parent directory could be specified in the ERL_LIBS environment variable. See code(3).
In your case it would be erl -pa ebin, or to include binaries of all deps you can use erl -pa ebin -pa deps/*/ebin.
Exactly same options are used in second line of your escript. With exception of * character, which will not be expand like it would be in the shell. In escript you have to provide paths to each dependency separately. But the idea of including -pa ebin stays exactly the same.
To automate and standardize this process tools like rebar and erlang.mk where created (I would recommend the later). Using those should help you a little with your workflow.
Trying to run tutorial samples for RabbitMQ in Erlang under OSX but it fails with the following message:
./send.erl:20: can't find include lib "rabbit_common/include/rabbit.hrl"
./send.erl:21: can't find include lib "rabbit_common/include/rabbit_framing.hrl"
escript: There were compilation errors.
amqp_example.erl:
-module(amqp_example).
-include("amqp_client.hrl").
-compile([export_all]).
test() ->
%% Start a network connection
{ok, Connection} = amqp_connection:start(#amqp_params_network{}),
%% Open a channel on the connection
{ok, Channel} = amqp_connection:open_channel(Connection),
%% Declare a queue
#'queue.declare_ok'{queue = Q}
= amqp_channel:call(Channel, #'queue.declare'{}),
%% Publish a message
Payload = <<"foobar">>,
Publish = #'basic.publish'{exchange = <<>>, routing_key = Q},
amqp_channel:cast(Channel, Publish, #amqp_msg{payload = Payload}),
%% Get the message back from the queue
Get = #'basic.get'{queue = Q},
{#'basic.get_ok'{delivery_tag = Tag}, Content}
= amqp_channel:call(Channel, Get),
%% Do something with the message payload
%% (some work here)
%% Ack the message
amqp_channel:cast(Channel, #'basic.ack'{delivery_tag = Tag}),
%% Close the channel
amqp_channel:close(Channel),
%% Close the connection
amqp_connection:close(Connection),
ok.
Please help me to fix this problem. Thank!!!
Erlang has macro include_lib, which can search for libraries in path and is convenient, because you don't have to specify version of library - it automatically uses newest version. So, instead of
-include("rabbit_common-3.3.5/include/rabbit.hrl").
you can just write:
-include_lib("rabbit_common/include/rabbit.hrl").
So, in your case, you have to make sure, that file rabbit_common-[version]/include/rabbit.hrl is in ERL_LIBS path. In the tutorial, you are using, they want you to download those files from here and unpack them like this:
unzip -d deps deps/amqp_client.ez
unzip -d deps deps/rabbit_common.ez
Those unpacking commands do not work on OS X, because unzip works only with .zip files. So this might be your problem. Try using another application to unpack them and double check, that the file is there. Don't forget to add ERL_LIBS=deps before compiling and running the example:
ERL_LIBS=deps erlc -o ebin amqp_example.erl
ERL_LIBS=deps erl -pa ebin
how to save mnesia:info() output?
I use remote sh in unix screen and can't to scroll window
Here's a function that you can put in the user_default.erl module on the remote node:
out(Fun, File) ->
G = erlang:group_leader(),
{ok, FD} = file:open(File, [write]),
erlang:group_leader(FD, self()),
Fun(),
erlang:group_leader(G, self()),
file:close(FD).
Then, you can do the following (after recompiling and loading user_default):
1> out(fun () -> mnesia:info() end, "mnesia_info.txt").
Or, just cut-and paste the following into the shell:
F = fun (Fun, File) ->
G = erlang:group_leader(),
{ok, FD} = file:open(File, [write]),
erlang:group_leader(FD, self()),
Fun(),
erlang:group_leader(G, self()),
file:close(FD)
end,
F(fun () -> mnesia:info() end, "mnesia_info.txt").
In cases where you are situated at a terminal without scrolling (if you are on a xterm and see no scrollbar simply switch it on) a tool very useful is screen: it provides virtual vt100 termials, you can switch between terminals even detach from it and come back later (nice for long running programs on remote serversthat need the occasional interaction).
And you can log transcripts to a file and scroll in the output of the virtual terminal.
If you are on a Unix like System you will probably be able to just install a pre-built package, if all else fails you can always pick up the source and build it yourself.
Also look at this article for other solutions.
If you are not able to install screen on the system, a simple but not very comfortable hack that only uses Unix built-in stuff is:
Start erlang shell with tee(1) to redirect the output:
$ erl | tee output.log
Eshell V5.7.5 (abort with ^G)
1> mnesia:info().
===> System info in version {mnesia_not_loaded,nonode#nohost,
{1301,742014,571300}}, debug level = none <===
opt_disc. Directory "/usr/home/peer/Mnesia.nonode#nohost" is NOT used.
use fallback at restart = false
running db nodes = []
stopped db nodes = [nonode#nohost]
ok
2>
Its a bit hard to get out of the shell (you probably have to type ^D to end the input file) but then you have the tty output in the file:
$ cat output.log
Eshell V5.7.5 (abort with ^G)
1> ===> System info in version {mnesia_not_loaded,nonode#nohost,
{1301,742335,572797}}, debug level = none <===
...
I believe you cant. See system_info(all).
Convert to a string:
S = io_lib:format("~p~n", [mnesia:info()]).
Then write it to disk.