I have skimmed through the Mochiweb code, but have not found any sign of the State variable.
Does something similar to gen_server's State variable exist in Mochiweb?
I need to store some small amount of state-related server-side (not session-related) data on the server and I do not want to use ETS or Mnesia for that.
I think you have somewhat a misunderstanding of what gen_server state is.
First, let me explain briefly how mochiweb works.
Mochiweb doesn't produce a gen_server process per client. Instead, it just spawns a new process using proc_lib:spawn/3 and creates a parametrized module, which is, basically, a tuple of the following kind:
{mochiweb_request, #Port<0.623>, get, "/users", {1, 1}, []}
which is
{mochiweb_request, Socket, Method, RawPath, HTTPVersion, Headers}
This tuple is used as an argument to a function that you pass as a loop parameter to mochiweb_http:start/1. So, when this "loop" function is called, it will look like this:
handle_request(Req) ->
%% The pattern matching below just shows what Req really is
{mochiweb_request, _, _, _, _, _} = Req,
...
Now, to explanation of gen_server state.
Basically, gen_server is a process with approximately the following structure. Of course, IRL it's more complicated, but this should give you the general idea:
init(Options)
State = ...
loop(Module, State).
loop(Module, State)
NewState = receive
{call, Msg, From} -> Module:handle_call(Msg, From, State)
{cast, Msg} -> Module:handle_cast(Msg, State)
Info -> Module:handle_info(Info, State)
end,
loop(Module, NewState).
So, state is just an argument that you drag through all the function calls and change inside your loop. It doesn't actually matter if your process is a gen_server or not, it doesn't have what lifetime it has. In the following example the term [1, 2, 3] is a state too:
a() ->
b([1, 2, 3], now()).
b(State, Timestamp) ->
Result = do_something(Timestamp)
c(State, Result).
c(State, Payload) ->
exit({State, Payload}).
Now, back to mochiweb.
If you need to create a state of your own, you can just add an extra function argument:
handle_request(Req) ->
User = Req:get(path),
UserData = load_user_data(User),
handle_request(Req, UserData).
handle_request(Req, UserData) ->
...
Now UserData is a state too. You can loop this process, or let it respond and end right away – but you won't lose UserData as long as you pass it as an argument.
Finally, if you really want to make this process a gen_server (which is really unreasonable in most cases), you can use gen_server:enter_loop/3 function that will make your current process a gen_server. And The 3rd argument of this function will be your state that will be stored inside the started gen_server.
Related
I am implementing the Gossip Algorithm in which multiple actors spread a gossip at the same time in parallel. The system stops when each of the Actor has listened to the Gossip for 10 times.
Now, I have a scenario in which I am checking the listen count of the recipient actor before sending the gossip to it. If the listen count is already 10, then gossip will not be sent to the recipient actor. I am doing this using synchronous call to get the listen count.
def get_message(server, msg) do
GenServer.call(server, {:get_message, msg})
end
def handle_call({:get_message, msg}, _from, state) do
listen_count = hd(state)
{:reply, listen_count, state}
end
The program runs well in the starting but after some time the Genserver.call stops with a timeout error like following. After some debugging, I realized that the Genserver.call becomes dormant and couldn't initiate corresponding handle_call method. Is this behavior expected while using synchronous calls? Since all actors are independent, shouldn't the Genserver.call methods be running independently without waiting for each others response.
02:28:05.634 [error] GenServer #PID<0.81.0> terminating
** (stop) exited in: GenServer.call(#PID<0.79.0>, {:get_message, []}, 5000)
** (EXIT) time out
(elixir) lib/gen_server.ex:774: GenServer.call/3
Edit: The following code can reproduce the error when running in iex shell.
defmodule RumourActor do
use GenServer
def start_link(opts) do
{:ok, pid} = GenServer.start_link(__MODULE__,opts)
{pid}
end
def set_message(server, msg, recipient) do
GenServer.cast(server, {:set_message, msg, server, recipient})
end
def get_message(server, msg) do
GenServer.call(server, :get_message)
end
def init(opts) do
state=opts
{:ok,state}
end
def handle_cast({:set_message, msg, server, recipient},state) do
:timer.sleep(5000)
c = RumourActor.get_message(recipient, [])
IO.inspect c
{:noreply,state}
end
def handle_call(:get_message, _from, state) do
count = tl(state)
{:reply, count, state}
end
end
Open iex shell and load above module. Start two processes using:
a = RumourActor.start_link(["", 3])
b = RumourActor.start_link(["", 5])
Produce error by calling a deadlock condition as mentioned by Dogbert in comments. Run following without much time difference.
cb = RumourActor.set_message(elem(a,0), [], elem(b,0))
ca = RumourActor.set_message(elem(b,0), [], elem(a,0))
Wait for 5 seconds. Error will appear.
A gossip protocol is a way of dealing with asynchronous, unknown, unconfigured (random) networks that may be suffering intermittent outages and partitions and where no leader or default structure is present. (Note that this situation is somewhat unusual in the real world and out-of-band control is always imposed on systems in some way.)
With that in mind, let's change this to be an asynchronous system (using cast) so that we are following the spirit of the concept of chatty gossip style communication.
We need digest of messages that counts how many times a given message has been received, a digest of messages that have been received and are already over the magic number (so we don't re-send one if it is way late), and a list of processes enrolled in our system so we know to whom we are broadcasting:
(The following example is in Erlang because I just trip over Elixir syntax ever since I stopped using it...)
-module(rumor).
-record(s,
{peers = [] :: [pid()],
digest = #{} :: #{message_id(), non_neg_integer()},
dead = sets:new() :: sets:set(message_id())}).
-type message_id() :: zuuid:uuid().
Here I am using a UUID, but it could be whatever. An Erlang reference would be fine for a test case, but since gossip isn't useful within an Erlang cluster, and references are unsafe outside the originating system I'm just jumping to the assumption this is for a networked system.
We will need an interface function that allows us to tell a process to inject a new message into the system. We will also need an interface function that sends a message between two processes once it is already in the system. Then we will need an inner function that broadcasts messages to all the known (subscribed) peers. Ah, that means we need a greeting interface so that peer processes can notify each other of their presence.
We will also want a way to have a process tell itself to keep broadcasting over time. How long to set the interval on retransmission is not actually a simple decision -- it has everything to do with network topology, latency, variability, etc (you would actually probably occasionally ping peers and develop some heuristic based on the latency, drop peers that seem unresponsive, and so on -- but we're not going to get into that madness here). Here I'm just going to set it for 1 second because that is an easy to interpret interval for humans observing the system.
Note that everything below is asynchronous.
Interfaces...
insert(Pid, Message) ->
gen_server:cast(Pid, {insert, Message}).
relay(Pid, ID, Message) ->
gen_server:cast(Pid, {relay, ID, Message}).
greet(Pid) ->
gen_server:cast(Pid, {greet, self()}).
make_introduction(Pid, PeerPid) ->
gen_server:cast(Pid, {make_introduction, PeerPid}).
That last function is going to be our way as testers of the system to cause one of the processes to call greet/1 on some target Pid so they start to build a peer network. In the real world something slightly different usually goes on.
Inside our gen_server callback for receiving a cast we will get:
handle_cast({insert, Message}, State) ->
NewState = do_insert(Message, State);
{noreply, NewState};
handle_cast({relay, ID, Message}, State) ->
NewState = do_relay(ID, Message, State),
{noreply, NewState};
handle_cast({greet, Peer}, State) ->
NewState = do_greet(Peer, State),
{noreply, NewState};
handle_cast({make_introduction, Peer}, State) ->
NewState = do_make_introduction(Peer, State),
{noreply, NewState}.
Pretty simple stuff.
Above I mentioned that we would need a way for this thing to tell itself to resend after a delay. To do that we are going to send ourselves a naked message to "redo_relay" after a delay using erlang:send_after/3 so we are going to need a handle_info/2 to deal with it:
handle_info({redo_relay, ID, Message}, State) ->
NewState = do_relay(ID, Message, State),
{noreply, NewState}.
Implementation of the message bits is the fun part, but none of this is terribly tricky. Forgive the do_relay/3 below -- it could be more concise, but I'm writing this in a browser off the top of my head, so...
do_insert(Message, State = #s{peers = Peers, digest = Digest}) ->
MessageID = zuuid:v1(),
NewDigest = maps:put(MessageID, 1, Digest),
ok = broadcast(Message, Peers),
ok = schedule_resend(MessageID, Message),
State#s{digest = NewDigest}.
do_relay(ID,
Message,
State = #s{peers = Peers, digest = Digest, dead = Dead}) ->
case maps:find(ID, Digest) of
{ok, Count} when Count >= 10 ->
NewDigest = maps:remove(ID, Digest),
NewDead = sets:add_element(ID, Dead),
ok = broadcast(Message, Peers),
State#s{digest = NewDigest, dead = NewDead};
{ok, Count} ->
NewDigest = maps:put(ID, Count + 1),
ok = broadcast(ID, Message, Peers),
ok = schedule_resend(ID, Message),
State#s{digest = NewDigest};
error ->
case set:is_element(ID, Dead) of
true ->
State;
false ->
NewDigest = maps:put(ID, 1),
ok = broadcast(Message, Peers),
ok = schedule_resend(ID, Message),
State#s{digest = NewDigest}
end
end.
broadcast(ID, Message, Peers) ->
Forward = fun(P) -> relay(P, ID, Message),
lists:foreach(Forward, Peers).
schedule_resend(ID, Message) ->
_ = erlang:send_after(1000, self(), {redo_relay, ID, Message}),
ok.
And now we need the social bits...
do_greet(Peer, State = #s{peers = Peers}) ->
case lists:member(Peer, Peers) of
false -> State#s{peers = [Peer | Peers]};
true -> State
end.
do_make_introduction(Peer, State = #s{peers = Peers}) ->
ok = greet(Peer),
do_greet(Peer, State).
So what did all of the horribly untypespecced stuff up there do?
It avoided any possibility of a deadlock. The reason deadlocks are so, well, deadly in peer systems is that anytime you have two identical processes (or actors, or whatever) communicating synchronously, you have created a textbook case of a potential deadlock.
Any time A has a synchronous message headed toward B and B has a synchronous message headed toward A at the same time you now have a deadlock. There is no way to create to identical processes that call each other synchronously without creating a potential deadlock. In massively concurrent systems anything that might happen almost certainly will eventually, so you're going to run into this sooner or later.
Gossip is intended to be asynchronous for a reason: it is a sloppy, unreliable, inefficient way to deal with a sloppy, unreliable, inefficient network topology. Trying to make calls instead of casts not only defeats the purpose of gossip-style message relay, it also pushes you into impossible deadlock territory incident to changing the nature of the protocol from asynch to synch.
Genser.call has a default timeout of 5000 milliseconds. So what probably happening is, the message queue of the actor is filled with millions of messages and by the time it reaches to call, the calling actor has timed out.
You can handle timeout using a try...catch:
try do
c = RumourActor.get_message(recipient, [])
catch
:exit, reason ->
# handle timeout
Now, the called actor will finally get to the call message and respond, which will come as an unexpected message to the first process. This you'll need to handle using handle_info. So one way is to ignore the error in catch block and send it rumor from handle_info.
Also, this will significantly degrade the performance if there are many process waiting to be timed-out for 5 seconds before moving ahead. One could deliberately reduce the timeout and handle the reply in handle_info. This will reduce to using cast and handling reply from other process.
Your blocking call need to be broken into two non blocking calls. So if A is making a blocking call to B, instead of waiting for reply, A can ask B to send its state on a given address (A's address) and move on.
Then A will handle that message separately and reply if necessary.
A.fun1():
body of A before blocking call
result = blockingcall()
do things based on result
needs to be divided into:
A.send():
body of A before blocking call
nonblockingcall(A.receive) #A.receive is where B should send results
do other things
A.receive(result):
do things based on result
I am a little confused with gproc and Pub/Sub methods( https://github.com/uwiger/gproc#use-case-pubsub-patterns ).
I can't understand how to receive messages from another process.
Example:
-module(ws_handler).
-export([init/2]).
init(Req, Opts) ->
lager:info("WS: init ws handler"),
gproc:reg({p, l, {?MODULE, WSNewMsgKey}}),
{cowboy_websocket, Req, Opts}.
process_data(Data) ->
lager:info("WS: start processing of json data"),
gproc:send({p, l, WSNewMsgKey}, {self(), WSNewMsgKey, Data}).
There are 2 processes, both of them are registered as subscribers. They should share incoming data with each other. I guess that i have to implement some interface/function but docs don't tell which exactly.
I've never used gproc for this, but it certainly seems like two things are missing: a definition of WSNewMsgKey (its never in scope in your snippet above) and a receive clause somewhere to accept the messages sent:
-module(ws_handler).
-export([init/2]).
init(Req, Opts) ->
gproc:reg({p, l, {?MODULE, ws_event}}),
{some_state_blah, Req, Opts}.
notify_peers(Event) ->
gproc:send({p, l, ws_event}, {self(), ws_event, Event}).
...and elsewhere either
handle_info({From, ws_event, Event}, State) ->
ok = handle_ws_event(From, Event).
or in your loop (if you wrote your process by hand):
loop(State) ->
receive
{From, ws_event, Event} ->
ok = handle_ws_event(From, Event),
loop(State);
Whatever ->
% other stuff...
end.
I'm not sure if the message that is sent would be sent by as a call, a cast, or a normal message (I'm assuming either an OTP generic cast, or normal message) -- but it seems that this is what should happen. In all cases, though, you need a well-defined key to identify the category of message being sent, and here I've used the atom 'ws_event' to make this explicit.
As for the details of the snippet above... you appear to be broadcasting the same JSON message to a bunch of processes at once for some sort of processing? I'm not sure what this would do for you -- I can't think of any case where broadcasting raw JSON would be beneficial (unless maybe if the need is to broadcast the JSON outside of the system and you are broadcasting to a bunch of subscribed client socket handlers?). So I'm confused at the context (what are you trying to achieve?).
This appears to be the way the docs intend this to be used -- but I'd have to actually play with it to be certain.
E.g. suppose I have a list that looks something roughly like this:
Handlers = [{foo, FooHandler}, {bar, BarHandler} | Etc()]
The best that I can come up with is this:
receive
Message ->
Handler = find_matching_handler(Message, Handlers),
Handler(Message)
end
The problem with this is that if Message does not match anything in Handlers, it's too late: I've taken it out of the mailbox.
I guess if there's a way to put a message back into the mailbox (into the save queue) without reordering, then that would take care of it. Simply resending to self() would reorder. It would also not restart the receive, and even if it did, you might get stuck in a spin loop until a message of interest arrives. Is there a way to put a message into the mailbox's save queue?
Another near solution that I thought of was to use match guard, but IIUC, you can only use BIFs in guards, which seems to preclude using find_matching_handler (unless there is a BIF for that).
Another near solution: map matching:
receive
M when Handlers#{M := Handler} -> Handler(M) % booyah?
end
Alas, I have not found an incantation that satisfies Erlang...
Match on the message:
loop() ->
receive
{foo, Data} ->
handle_foo(Data),
loop();
{bar, Data} ->
handle_bar(Data),
loop()
end.
This is the basic way of distinguishing between message forms.
You can also be less direct and match in a function head you pass all messages to:
loop() ->
receive
Message ->
handle_message(Message),
loop()
end.
handle_message({foo, Data}) ->
foo(Data),
ok;
handle_message({bar, Data}) ->
bar(Data),
ok.
A combination of the first and second forms is sort of the way gen_server type callback modules are structured in OTP. The message handlers receive a slightly more complex set of arguments and exist in their own module (the part you write), and the actual receive occurs in the generic gen_server module.
You can use a selective receive pattern to periodcally scan the mailbox for handler messages. Something like this:
check_msg_handlers(Handlers) ->
[check_handler(X) || X <- Handlers],
timer:sleep(500),
check_msg_handlers(Handlers).
check_handler(Handler) ->
receive
{_Handler={M,F}, Msg} ->
M:F(Msg)
after
0 ->
no_msg
end.
Note the receive X -> Y after -> N no_msg end, this is the selective receive. When using a timeout of N=0 it effectively becomes a scan of the mailbox to see if the X message is present or not, i.e. it becomes a non-blocking receive. The order of the messages is preserved after the scan as required in your case.
The LYSE chapter More On Multiprocessing has a section on selective receives that is very good.
As an exercise to test my understanding of sequential Erlang I wrote a simple-minded html tag editor.
It maintains a current position in the text, e.g. edit state, in a record like this:
-record(ed_txt, {a=[], b=[], c=[]).
...where a holds text above the current paragraph, b holds the current paragraph, and c holds text below the current paragraph.
It is called like this:
tag(App, FileName) ->
{ok, File1} = get_file(App, FileName),
{ok, File2} = file_to_paragraphs(File1),
{ok, Record} = create_record(File2),
....
tag(Record) ->
{ok, InputTuple} = get_input(Record), % gets edit command
...
do(Command, Record)
The function do/2 is recursive function that looks like this:
do("h", Record) -> tag(help(Record));
do("down", Record) -> tag(down(Record));
do("up",Record) -> tag(up(Record));
do("h1", Record) -> tag(wrap("H1", Record));
...
do("quit", Record) -> {ok, quit(Record)};
do(_, Record) -> tag(Record).
Now I'd like to turn this module into an OTP gen-server, but realize that I don't really understand how to maintain state.
I created a state record:
-record(state, {app, filename, a=[], b=[], c=[]}).
My thought is to use a one-to-one supervisor so each user is working in his/her own process, ditch the edit loop, and use handle_call (or handle_cast) to call the respective
edit functions.
But what confuses me is this:
As long as the process is running, presumably user state is maintained in the process, but both handle_call and handle_cast take State as a parameter and return State.
Does this mean that I also have to maintain state for each user OUTSIDE of their respective processes, in ets perhaps?
Or is there a better way to do this altogether?
Many thanks,
LRP
I'm getting started with Erlang, and could use a little help understanding the different results when applying the PID returned from spawn/3 to the process_info/1 method.
Given this simple code where the a/0 function is exported, which simply invokes b/0, which waits for a message:
-module(tester).
-export([a/0]).
a() ->
b().
b() ->
receive {Pid, test} ->
Pid ! alrighty_then
end.
...please help me understand the reason for the different output from the shell:
Example 1:
Here, current_function of Pid is shown as being tester:b/0:
Pid = spawn(tester, a, []).
process_info( Pid ).
> [{current_function,{tester,b,0}},
{initial_call,{tester,a,0}},
...
Example 2:
Here, current_function of process_info/1 is shown as being tester:a/0:
process_info( spawn(tester, a, []) ).
> [{current_function,{tester,a,0}},
{initial_call,{tester,a,0}},
...
Example 3:
Here, current_function of process_info/1 is shown as being tester:a/0, but the current_function of Pid is tester:b/0:
process_info( Pid = spawn(tester, a, []) ).
> [{current_function,{tester,a,0}},
{initial_call,{tester,a,0}},
...
process_info( Pid ).
> [{current_function,{tester,b,0}},
{initial_call,{tester,a,0}},
...
I assume there's some asynchronous code happening in the background when spawn/3 is invoked, but how does variable assignment and argument passing work (especially in the last example) such that Pid gets one value, and process_info/1 gets another?
Is there something special in Erlang that binds variable assignment in such cases, but no such binding is offered to argument passing?
EDIT:
If I use a function like this:
TestFunc = fun( P ) -> P ! {self(), test}, flush() end.
TestFunc( spawn(tester,a,[]) ).
...the message is returned properly from tester:b/0:
Shell got alrighty_then
ok
But if I use a function like this:
TestFunc2 = fun( P ) -> process_info( P ) end.
TestFunc2( spawn(tester,a,[]) ).
...the process_info/1 still shows tester:a/0:
[{current_function,{tester,a,0}},
{initial_call,{tester,a,0}},
...
Not sure what to make of all this. Perhaps I just need to accept it as being above my pay grade!
If you look at the docs for spawn it says it returns the newly created Pid and places the new process in the system scheduler queue. In other words, the process gets started but the caller keeps on executing.
Erlang is different from some other languages in that you don't have to explicitly yield control, but rather you rely on the process scheduler to determine when to execute which process. In the cases where you were making an assignment to Pid, the scheduler had ample time to switch over to the spawned process, which subsequently made the call to b/0.
It's really quite simple. The execution of the spawned process starts with a call to a() which at some point shortly afterwards will call b() and then just sits there and waits until it receives a specific message. In the examples where you manage to immediately call process_info on the pid, you catch it while the process is still executing a(). In the other cases, when some delay is involved, you catch it after it has called b(). What about this is confusing?