I am setting up an ejabbered + riak cluster, where i have to use basic riak (get,put,delete..)functions in the file ejabberd/src/ejabberd_riak.erl
The functions put, get, get_by_index etc. work great and using the usage of the module in the file I could figure out what is what.
I'm facing an issue with the function delete_by_index and also get_keys_by_index, which is called by delete_by_index, anyhow.
The error thrown when I do this ->
ejabberd_riak:get_keys_by_index(game <<"language">>,
term_to_binary("English")).
{error,<<"Phase 0: invalid module named in PhaseSpec function:\n must be a valid module name (failed to load ejabberd_r"...>>}
(ejabberd#172.43.12.133)57> 12:28:55.177 [error] database error:
** Function: get_keys_by_index
** Table: game
** Index = <<"language">>
** Key: <<131,107,0,7,69,110,103,108,105,115,104>>
** Error: Phase 0: invalid module named in PhaseSpec function:
must be a valid module name (failed to load ejabberd_riak: nofile)
You should probably load ejabberd_riak on the riak side
You are currently using riak as separate erlang application, communicating with the database by protobuf. In this configuration you have independent (from each other) module sets loaded in ejabbered and riak applications. ejabberd_riak module loaded in ejabberd application, but not in riak application.
However get_by_index uses mapred that demands ejabberd_riak loaded on the riak side
-spec get_keys_by_index(atom(), binary(),
any()) -> {ok, [any()]} | {error, any()}.
%% #doc Returns a list of primary keys of objects indexed by `Key'.
get_keys_by_index(Table, Index, Key) ->
{NewIndex, NewKey} = encode_index_key(Index, Key),
Bucket = make_bucket(Table),
case catch riakc_pb_socket:mapred(
get_random_pid(),
{index, Bucket, NewIndex, NewKey},
[{map, {modfun, ?MODULE, map_key}, none, true}]) of
%% ^^^^^^
%% here is the problem
{ok, [{_, Keys}]} ->
{ok, Keys};
{ok, []} ->
{ok, []};
{error, _} = Error ->
log_error(Error, get_keys_by_index, [{table, Table},
{index, Index},
{key, Key}]),
Error
end.
You can customize your riak and add ejabberd_riak to the riak application (you don't need to start the whole ejabberd application on the riak side, however)
With monckeypatching approach, you should copy ejabberd_riak.erl, ejabberd.hrl, logger.hrl to the riak/deps/riak_kv/src. Then rebuild riak. You should distribute your files over the whole cluster, as map phase is executed on each cluster node.
Related
Trust you all are doing well.
I am trying to make multiple sessions to SMSC using OSERL application.
Since to make a SMPP client you need to inherit gen_esme behaviour.
I was wondering whether it is possible to make multiple connections towards SMSC without writing multiple gen_esme modules?
There are two main strategies for starting multiple processes using the same gen_esme based module:
gen_esme:start_link/4 - named or reference based server
gen_esme:start_link/3 - pid based server
I'm going to be referencing the sample_esme file found under the examples for oserl.
Named Server
Most of the examples from oserl show usage of gen_esme:start_link/4 which in turn is calling gen_server:start_link/4. The ServerName variable for gen_server:start_link/4 has a typespec of {local, Name::atom()} | {global, GlobalName::term()} | {via, Module::atom(), ViaName::term()}.
That means if we change the sample_esme:start_link/0,1,2 functions to look like this:
start_link() ->
start_link(?MODULE).
start_link(SrvName) ->
start_link(SrvName, true).
start_link(SrvName, Silent) ->
Opts = [{rps, 1}, {queue_file, "./sample_esme.dqueue"}],
gen_esme:start_link({local, SrvName}, ?MODULE, [Silent], Opts).
We can start multiple servers using:
sample_esme:start_link(). %% SrvName = 'sample_esme'
sample_esme:start_link(my_client1). %% SrvName = 'my_client1'
sample_esme:start_link(my_client2). %% SrvName = 'my_client2'
To make our sample_esme module work properly with this named server strategy, most of its calling functions will need to be modified. Let's use sample_esme:rps/0,1 as an example:
rps() ->
rps(?MODULE).
rps(SrvName) ->
gen_esme:rps(SrvName).
Now we can call the gen_esme:rps/1 function on any of our running servers:
sample_esme:rps(). %% calls 'sample_esme'
sample_esme:rps(my_client1). %% 'my_client1'
sample_esme:rps(my_client2). %% 'my_client2'
This is similar to how projects like pooler reference members of pools it creates.
pid Server
This is essentially the same as the Named Server strategy, but we're just going to pass the pid of the server around instead of a registered atom.
That means if we change the sample_esme:start_link/0,1 functions to look like this:
start_link() ->
start_link(true).
start_link(Silent) ->
Opts = [{rps, 1}, {queue_file, "./sample_esme.dqueue"}],
gen_esme:start_link(?MODULE, [Silent], Opts).
Notice that all we did was drop the {local, SrvName} argument so it won't register the SrvName atom with the server's pid.
That means we need to capture the pid of each created server:
{ok, Pid0} = sample_esme:start_link().
{ok, Pid1} = sample_esme:start_link().
{ok, Pid2} = sample_esme:start_link().
Using the same sample_esme:rps/0,1 example from Named Server, we will need to remove sample_esme:rps/0 and add a sample_esme:rps/1 function which takes a pid:
rps(SrvPid) ->
gen_esme:rps(SrvPid).
Now we can call the gen_esme:rps/1 function on any of our running servers:
sample_esme:rps(Pid0).
sample_esme:rps(Pid1).
sample_esme:rps(Pid2).
This is similar to how projects like poolboy reference members of pools it creates.
Recommendations
If you are simply trying to pool connections, I would recommend using a library like pooler or poolboy.
If you have a finite number of specifically named connections that you want to reference by name, I would recommend just having a supervisor with a child spec like the following for each connection:
{my_client1,
{sample_esme, start_link, [my_client1]},
permanent, 5000, worker, [sample_esme]}
I am new in erlang and ejabbered/mongooseIM. I am trying to write a very simple mongoose module that will add an extra child element to the packets before sending. Below is my code:
-module(mod_test).
-behavior(gen_mod).
-export([start/2, stop/1]).
-export([add_child/1]).
-include("ejabberd.hrl").
start(Host, Opts) ->
ejabberd_hooks:add(filter_packet, Host, ?MODULE, add_child, 0),
?DEBUG(" MOD_TEST Started",[]),
ok.
stop(Host) ->
ejabberd_hooks:delete(filter_packet, Host, ?MODULE, add_child, 0),
ok.
add_child({From, To, XML} = Packet) ->
Tag = {"a","b"},
NewPacket = xml:append_subtags(Packet, [Tag]),
?INFO_MSG(" To party: ~p~n",[To]),
NewPacket.
I can compile the code with few warnings
mod_test.erl:3: Warning: behaviour gen_mod undefined
mod_test.erl:11: Warning: variable 'Opts' is unused
mod_test.erl:20: Warning: variable 'From' is unused
mod_test.erl:20: Warning: variable 'XML' is unused
Then when I add the module and run it it gets started but it doesnt make any changes to the packets and doesnt generate any logs either.
Another issue is, if I add a log within my start function, it gets compiled but I see errors while starting the module
2015-03-03 16:36:34.772 [critical] <0.200.0>#gen_mod:start_module:94 Problem starting the module mod_test for host <<"localhost">>
options: []
error: undef
[{lager,info,[" mod_test starting ...",[[]]],[]},
{mod_test,start,2,[{file,"mod_test.erl"},{line,13}]},
{gen_mod,start_module,3,[{file,"src/gen_mod.erl"},{line,83}]},
{lists,foreach,2,[{file,"lists.erl"},{line,1336}]},
{ejabberd_app,start,2,[{file,"src/ejabberd_app.erl"},{line,69}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,272}]}]
2015-03-03 16:36:34.773 [critical] <0.200.0>#gen_mod:start_module:99 ejabberd initialization was aborted because a module start failed.
The trace is [{lager,info,[" mod_test starting ...",[[]]],[]},{mod_test,start,2,[{file,"mod_test.erl"},{line,13}]},{gen_mod,start_module,3,[{file,"src/gen_mod.erl"},{line,83}]},{lists,foreach,2,[{file,"lists.erl"},{line,1336}]},{ejabberd_app,start,2,[{file,"src/ejabberd_app.erl"},{line,69}]},{application_master,start_it_old,4,[{file,"application_master.erl"},{line,272}]}].
Crash dump was written to: erl_crash.dump
Problem starting the module mod_test for host <<"localhost">>
options: []
error: undef
[{lager,info,[" mod_xyz starting ...",[[]]],[]},
{mod_test,start,2,[{file,"mod_timetagg
What wrong am I doing?
So your example is actually a bit tricky, because of how the filter_packet hook works. You picked the worst hook to register for on your first attempt ;)
If you look into ejabberd_router:do_route, you'll see that filter_packet is run without a Host parameter -- it is a global hook, so when you register your add_child function for a particular Host, it will essentially be ignored.
Try the following:
-module(mod_test).
-behavior(gen_mod).
-export([start/2, stop/1]).
-export([add_child/1]).
-include_lib("ejabberd/include/ejabberd.hrl").
-include_lib("exml/include/exml.hrl").
start(Host, Opts) ->
ejabberd_loglevel:set_custom(?MODULE, 5),
ejabberd_hooks:add(filter_local_packet, Host, ?MODULE, add_child, 1),
?DEBUG(" MOD_TEST Started",[]),
ok.
stop(Host) ->
ejabberd_hooks:delete(filter_local_packet, Host, ?MODULE, add_child, 1),
ok.
add_child({From, To, Element} = HookData) ->
?DEBUG("Filtering ~p~n", [HookData]),
case Element#xmlel.name of
<<"message">> ->
Tag = #xmlel{name = <<"added-tag">>, attrs = [], children = []},
NewElement = xml:append_subtags(Element, [Tag]),
?DEBUG("will return new el: ~p", [NewElement]),
{From, To, NewElement};
_ ->
?DEBUG("will pass old el: ~p", [Element]),
HookData
end.
Registering for filter_local_packet on your given Host will now work, and all incoming stanzas will be passed to your function. It is important to remember that adding spurious tags to all stanzas may break things, so the above code will only add a <added-tag> element to <message> stanzas.
Use the example above and work from there.
Good luck!
The error is undef, which means that a function was called that isn't defined/exported. The stacktrace shows that the function in question is lager:info/2.
Lager (the logging library that handles your ?INFO_MSG) has a special quirk in that your code would call non-existent functions, but the code gets transformed by a parse transform before being compiled. It looks like this didn't happen for you.
The rebar.config file in apps/ejabberd in the MongooseIM tree contains {parse_transform, lager_transform} in erl_opts, which asks the compiler to apply the parse transform. I'd suggest putting mod_test.erl into apps/ejabberd/src and build the entire MongooseIM tree; that would ensure that your file gets built with the correct options.
I am trying to write a first program in Erlang that effects message communication between a client and server. In theory the server exits when it receives no message from the client, but every time I edit the client code and run the server again, it executes the old code. I have to ^G>q>erl>[re-enter command] to get it to see the new code.
-module(srvEsOne).
%%
%% export functions
%%
-export([start/0]).
%%function definition
start()->
io:format("Server: Starting at pid: ~p \n",[self()]),
case lists:member(serverEsOne, registered()) of
true ->
unregister(serverEsOne); %if the token is present, remove it
false ->
ok
end,
register(serverEsOne,self()),
Pid = spawn(esOne, start,[self()]),
loop(false, false,Pid).
%
loop(Prec, Nrec,Pd)->
io:format("Server: I am waiting to hear from: ~p \n",[Pd]),
case Prec of
true ->
case Nrec of
true ->
io:format("Server: I reply to ~p \n",[Pd]),
Pd ! {reply, self()},
io:format("Server: I quit \n",[]),
ok;
false ->
receiveLoop(Prec,Nrec,Pd)
end;
false ->
receiveLoop(Prec,Nrec,Pd)
end.
receiveLoop(Prec,Nrec,Pid) ->
receive
{onPid, Pid}->
io:format("Server: I received a message to my pid from ~p \n",[Pid]),
loop(true, Nrec,Pid);
{onName,Pid}->
io:format("Server: I received a message to name from ~p \n",[Pid]),
loop(Prec,true,Pid)
after
5000->
io:format("Server: I received no messages, i quit\n",[]),
ok
end.
And the client code reads
-module(esOne).
-export([start/1, func/1]).
start(Par) ->
io:format("Client: I am ~p, i was spawned by the server: ~p \n",[self(),Par]),
spawn(esOne, func, [self()]),
io:format("Client: Now I will try to send a message to: ~p \n",[Par]),
Par ! {self(), hotbelgo},
serverEsOne ! {self(), hotbelgo},
ok.
func(Parent)->
io:format("Child: I am ~p, i was spawned from ~p \n",[self(),Parent]).
The server is failing to receive a message from the client, but I can't sensibly begin to debug that until I can try changes to the code in a more straightforward manner.
When you make modification to a module you need to compile it.
If you do it in an erlang shell using the command c(module) or c(module,[options]), the new compiled version of the module is automatically loaded in that shell. It will be used by all the new process you launch.
For the one that are alive and already use it is is more complex to explain and I think it is not what you are asking for.
If you have several erlang shells running, only the one where you compile the module loaded it. That means that in the other shell, if the module were previously loaded, basically if you already use the module in those shell, and even if the corresponding processes are terminated, the new version is ignored.
Same thing if you use the command erlc to compile.
In all these cases, you need to explicitly load the module with the command l(module) in the shell.
Your server loop contain only local function calls. Running code is changed only if there is remote (or external) function call. So you have to export your loop function first:
-export([loop/3]).
and then you have to change all loop/3 calls in function receiveLoop/3 to
?MODULE:loop(...)
Alternatively you can do same thing with receiveLoop/3 instead. Best practice for serious applications is doing hot code swapping by demand so you change loop/3 to remote/external only after receiving some special message.
I have some apps in my cluster, I need to start some of them sometimes on different hosts.
The story is that the Erlang cluster is already running, so even though I have my .app resource file per application stating which applications should be started before mine, this only works to create a startup script, not to start an app in a already running node.
At the moment I have a custom routine that uses application:get_key(Application,applications) to extract the dependencies and start them separately before starting the given application.
I was wondering if there isn't a better way of doing this.
Since Erlang R16B02, there is also application:ensure_all_started.
Frankly, the standard tools for doing this in Erlang are unnecessarily annoying right now. I tend to put the following boiler-plate in my application callback module:
-module(myapp_app).
-export([start/0]).
start() -> a_start(myapp, permanent).
a_start(App, Type) ->
start_ok(App, Type, application:start(App, Type)).
start_ok(_App, _Type, ok) -> ok;
start_ok(_App, _Type, {error, {already_started, _App}}) -> ok;
start_ok(App, Type, {error, {not_started, Dep}}) ->
ok = a_start(Dep, Type),
a_start(App, Type);
start_ok(App, _Type, {error, Reason}) ->
erlang:error({app_start_failed, App, Reason}).
You can then add -s myapp_app to your erlang command line and this will start the app and all its dependencies recursively. Why this function isn't in the application module I don't know :)
There is a working example of this custom erlang app startup code in my Erlang Factory 2012 SFBay example app.
When starting the app outside of the startup script you do need to start the dependencies first. You could build the smarts to do this into the app itself so that when the app starts it will start any required dependencies before it needs them.
One place I've seen this done is in Mochiweb apps. The default app templates include code for loading dependencies on startup:
-module(some_app).
-export([start/0, stop/0]).
ensure_started(App) ->
case application:start(App) of
ok ->
ok;
{error, {already_started, App}} ->
ok
end.
%% #spec start() -> ok
%% #doc Start the some_app server.
start() ->
some_app_deps:ensure(),
ensure_started(crypto),
application:start(some_app).
%% #spec stop() -> ok
%% #doc Stop the some_app server.
stop() ->
application:stop(some_app).
If you write your app under "OTP Design Principles", you will have to make yourappname.app file, which will contains `applications' section. This section defines which other applications you want to be started before yours. Here is stated:
applications
All applications which must be started before this
application is started. systools uses this list to generate correct
boot scripts. Defaults to [], but note that all applications have
dependencies to at least kernel and stdlib.
So if you use releases, this dependency resolution will be solved by systools.
I am a beginner with erlang, and i write a basic gen server program as follows, I want to know, how to test the server so i can know it works well.
-module(gen_server_test).
-behaviour(gen_server).
-export([start_link/0]).
-export([alloc/0, free/1]).
-export([init/1, handle_call/3, handle_cast/2]).
start_link() ->
gen_server:start_link({local, gen_server_test}, ch3, [], []).
alloc() ->
gen_server:call(gen_server_test, alloc).
free(Ch) ->
gen_server:cast(gen_server_test, {free, Ch}).
init(_Args) ->
{ok, channels()}.
handle_call(alloc, _From, Chs) ->
{Ch, Chs2} = alloc(Chs),
{reply, Ch, Chs2}.
handle_cast({free, Ch}, Chs) ->
io:format(Ch),
io:format(Chs),
Chs2 = free(),
{noreply, Chs2}.
free() ->
io:format("free").
channels() ->
io:format("channels").
alloc(chs) ->
io:format("alloc chs").
BTW: The program can be compiled, and it is not a good program, I just want to print something to make sure it works :)
The beauty of a gen_server implementing module is that it is just a callback module. One need not even have to spawn the underlying gen_server process to test it.
All you need to do is have your test framework (usually eunit) to invoke all the handle_call/cast/info functions by injecting it with different inputs (different gen_server states, different input messages) etc. and ensure it returns the correct response tuple (for eg. {reply, ok, NewState} or {noreply, NewState} etc.)
Ofcourse, this wouldnt work perfectly if your callback functions arn't pure functions. For eg., in your handle_call function, if you are sending a message to another process for eg., or if you are modifying an ets table. In which case, you have to ensure that all the required processes and tables are pre-created before running the test.
You could try one of the following:
Use the erlang shell and invoke the commands manually. Make sure the source or .beam file is in the Erlang path (parameter -pz, like this: erl -pz <path here>)
Write an EUnit test case
PS: I think you have an error in your code, because you seem to start the module ch3 as the server, not the gen_server_test module.