As a newbie, writing a toy matching (trading) engine using gen_server.
Once a trade/match occurs there is need to notify both the clients.
Documentation says that :
reply(Client, Reply) -> Result
Types:
Client - see below
Reply = term()
Result = term()
This function can be used by a gen_server to explicitly send a reply
to a client that called call/2,3 or multi_call/2,3,4, when the reply
cannot be defined in the return value of Module:handle_call/3.
Client must be the From argument provided to the callback function. Reply is an arbitrary term, which will be given back to
the client as the return value of call/2,3 or multi_call/2,3,4.
The return value Result is not further defined, and should always be
ignored.
Given the above how is it possible to send notification to the other client.
SAMPLE SEQUENCE OF ACTIONS
C1 -> Place order IBM,BUY,100,10.55
Server -> Ack C1 for order
C2 -> Place order IBM,SELL,100,10.55
Server -> Ack C2 for order
-> Trade notification to C2
-> Trade notification to C1 %% Can I use gen_server:reply()
%% If yes - How ?
Well, you can't. Your ACK is already a reply. And only single reply is acceptable by gen_server:call contract. I mean, gen_server:call will only wait for one reply.
Generally gen_server:reply can be implemented like
reply({Pid, Ref}, Result) ->
Pid ! {Ref, Result}.
That means that if you try sending multiple replies, you just get some weired messages in the message box of the caller process.
Proposal
Instead, I believe, you should send associate every trade with some reference, and send message to the caller with that reference CX_Ref during the ACK procedure. Then, when you have to send a notification, you just emit message {C1_Ref, Payload} to C1 and {C2_Ref, Payload} to C2.
Also you may want to introduce some monitoring to handle broker crashes.
Related
There's variable in my module, and there's receive method to renew variable value. And multiple process are calling this method simultaneously. I need lock this variable when one process is modifying it. Sample as below
mytest.erl
%%%-------------------------------------------------------------------
-module(mytest).
%% API
-export([start_link/0,display/1,callDisplay/2]).
start_link()->
Pid=spawn(mytest,display,["Hello"]),
Pid.
display(Val) ->
io:format("It started: ~p",[Val]),
NextVal=
receive
{call,Msg}->
NewVal=Val++" "++Msg++" ",
NewVal;
stop->
true
end,
display(NextVal).
callDisplay(Pid,Val)->
Pid!{call,Val}.
Start it
Pid=mytest:start_link().
Two process are calling it in the same time
P1=spawn(mytest,callDisplay,[Pid,"Walter"]),
P2=spawn(mytest,callDisplay,[Pid,"Dave"]).
I hope it can add "Walter", "Dave" one by one like "Hello Walter Dave", however, when there're too many of them running together, some Names(Walter, Dave, etc) will be override.
Because when P1, P2 started the same time, Val both are "Hello". P1 add "Walter" to become "Hello Walter", P2 add "Dave" to become "Hello Dave". P1 saved it firstly to NextVal as "Hello Walter", then P2 saved it to NextVal as "Hello Dave", so result will be "Hello Dave". "Hello Walter" is replaced by "Hello Dave", and "Walter" lost forever.
Is there any way I can lock "Val", so when we add "Walter", "Dave" will waiting till Value setting is done?
Even though it's an old question but it's worth explaining.
From what you said and if I'm correct,
you expect to see
"Hello Walter", and "Hello Dave". However, you're seeing successive names been appended to the former as, "Hello Walter Dave.."
This behavior is normal and to see that let look briefly at Erlang memory model. Erlang process memory is divided into three main parts:
Process Control Block(PCB):
This hold the process pid, registered name,table,states and pointers to messages in the it's queue.
Stack:
This hold function parameters, local variables and function return address.
Private Heap: This hold incoming message compound data like tuple, list and binary(not larger than 64 bytes).
All data in these memory belong to and are private to the owning process.
Stage1:
When Pid=spawn(mytest,display,["Hello"]) is called, the server process is created, then the display function with "Hello" passed as argument is called. Since display/1 is executed in the serve process, the "Hello" argument lives in the server's process stack. Execution of display/1 continues until it reaches the receive clause then block and await message matching your format.
Stage 2:
Now P1 starts, it executes ServerPid ! {call, "Walter"}, then P2 executes ServerPid ! {call, "Dave"}. In both cases, erlang makes a copy of the message and send it to the server's process mailbox (Private Heap). This copied message in the mailbox belongs to the server process not the client's.
Now, when {call, "Walter"} is matched, Msg get bound to "Walter".
From stage1, we know Val is bounded to "Hello", Newval then get bounded to "Val ++ " " ++ Msg" = "Hello Walter".
At this point, P2's message, {call, "Dave"}, is still in the server's mailbox awaiting the next receive clause which will happen in the next recursive call to display/1. NextVal get bound to NewVal and the recursive call to dispaly/1 with "Hello Walter" passed as argument is made. This gives the first print "Hello Walter " which now also lives in the server's process stack.
Now when the receive clause is reach again, P2's message {call, "Dave"} is matched.
Now NewVal and NextVal get bound to "Hello Walter" ++ " " ++ "Dave" = "Hello Walter Dave". This get passed as argument to display/1 as the new Val to print Hello Walter Dave. In a nutshell, this variable is updated on every server loop. It serves the same purpose as the State term in gen_server behavior. In your case, successive client calls just appends the message to this serve state variable. Now to your question,
Is there any way I can lock Val, so when we add "Walter", "Dave" will waiting till Value setting is done?
No. Not by locking. Erlang does not work this way.
There are no process locking constructs as it does not need one.
Data(Variables) are always immutable and private(except large binaries which stays in the Shared Heap) to the process that created it.
Also, it's not the actual message you used in the Pid ! Msg construct that is process by the receiving process. It's it copy. The Val parameter in yourdisplay/1 function is private and belongs to the server process because it lives in it stack memory as every call to display/1 is made by the server process itself. So there is no way any other process can lock not even see that variable.
Yes. By sequential message processing
This is exactly what the server process is doing. Polling one message a time from it queue. When {call, "Walter"} was taken, {call, "Dave"} was waiting in the queue. The reason why you see unexpected greeting is because the you change the server state, the display/1 parameter for the next display/1 call which process {call, "Dave"}
I am trying to build a very simple barrier-synchronization server, where the server is initially fed a number of processes that will be communicating with it. When a process is done, it receives a message with that process' Pid, and it keeps a list of every process to do so. When the barrier reaches zero (all processes have sent messages), my server needs to send a message to each of these (I am using [Pid | ProcList] as my list of pids).
I have tried using a helper function to no avail, list comprehensions keep me in an infinite loop, and as such I am looking into how to use lists:foreach to take care of this.
I am fairly new to functional programming, but from what I understand, this foreach needs to take in the list as well as a lambda-calculus function to send a message to each node in the list. Due to the infix nature of "!", I have yet to find a way to do this without causing syntax errors.
How you've made infinite loop in list comprehension? I must say, that's quite challenging. Try this:
Message = % broadcast message goes here
ListOfPids = % list of recipients
[Pid ! Message || Pid <- ListOfPids].
If you want to use foreach, than it takes one argument function as first argument, so need to wrap send first, as it is two argument function.
Message = % broadcast message goes here
ListOfPids = % list of recipients
Fun = fun (Pid) -> Pid ! Message end,
lists:foreach(Fun, ListOfPids).
I am learning Erlang and trying to figure out how I can, and should, save state inside a process.
For example, I am trying to write a program that given a list of numbers in a file, tells me whether a number appears in that file. My approach is to uses two processes
cache which reads the content of the file into a set, then waits for numbers to check, and then replies whether they appear in the set.
is_member_loop(Data_file) ->
Numbers = read_numbers(Data_file),
receive
{From, Number} ->
From ! {self(), lists:member(Number, Numbers)},
is_member_loop(Data_file)
end.
client which sends numbers to cache and waits for the true or false response.
check_number(Number) ->
NumbersPid ! {self(), Number},
receive
{NumbersPid, Is_member} ->
Is_member
end.
This approach is obviously naive since the file is read for every request. However, I am quite new at Erlang and it is unclear to me what would be the preferred way of keeping state between different requests.
Should I be using the process dictionary? Is there a different mechanism I am not aware of for that sort of process state?
Update
The most obvious solution, as suggested by user601836, is to pass the set of numbers as a param to is_member_loop instead of the filename. It seems to be a common idiom in Erlang and there is a good example in the fantastic online book Learn you some Erlang.
I think, however, that the question still holds for more complex state that I'd want to preserve in my process.
Simple solution, you can pass to your function is_member_loop(Data_file) the list of numbers rather then the file name.
The best solution when you deal with a state consists in using a gen_server. To learn more you should take a look at records and gen_server behaviour (this may also be useful).
In practice:
1) start with a module (yourmodule.erl) based on gen_server behaviour
2) read your file in the init function of the gen_server and pass it as state field:
init([]) ->
Numbers = read_numbers(Data_file),
{ok, #state{numbers=Numbers}}.
3) write a function which will be used to trigger a call to the gen_server
check_number(Number) ->
gen_server:call(?MODULE, {check_number, Number}).
4) write the code in order to handle messages triggered from your function
handle_call({check_number, Number}, _From, #state{numbers=Numbers} = State) ->
Reply = lists:member(Number, Numbers)},
{reply, Reply, State};
handle_call(_Request, _From, State) ->
Reply = ok,
{reply, Reply, State}.
5) export from yourmodule.erl function check_number
-export([check_number/1]).
Two things to be explained about point 4:
a) we extract values inside the record State using pattern matching
b) As you may see I left the generic handle call, otherwise your gen_server will fail due to wrong pattern matching whenever a message different from {check_number, Number} is received
Note: if you are new to erlang, don't use process dictionary
Not sure how idiomatic this is, since I'm not exactly an Erlang pro yet, but I'd handle this by using ETS. Basically,
read_numbers_to_ets(DataFile) ->
Table = ets:new(numbers, [ordered_set]),
insert_numbers(Table, DataFile),
Table.
insert_numbers(Table, DataFile) ->
case read_next_number(DataFile) of
eof -> ok;
Num -> ets:insert(numbers, {Num})
end.
you could then define your is_member as
is_member(TableId, Number) ->
case ets:match(TableId, {Number}) of
[] -> false; %% no match from ets
[[]] -> true %% ets found the number you're looking for in that table
end.
Instead of taking a Data_file, your is_member_loop would take the id of the table to do a lookup on.
Can I Nest receive {tcp, Socket, Bin} -> calls? For example I have a top level loop called Loop, which upon receipt of tcp data calls a function, parse_header, to parse header data (an integer which indicates the kind of data to follow and thus its size), after that I need to receive the entire payload before moving on. I might only receive 4 bytes when I need a full 20 bytes and would like to call receive in a separate function called parse_payload. So the call chain would look like loop->parse_header->parse_payload and I would like parse_payload to call receive {tcp, Socket, Bin} ->. I don't know if this ok or if I'm completely going to mess things up and can only do it in the Loop function. Can someone enlighten me? If I am allowed to do this is am I violating some sort of best practice?
Maybe you can check the sample code for "erlang programming".
The download page is Erlang Programming Source Code
In file socket_examples.erl, please check "receive_data" function.
For perse message, I think you should determine how to seperate messages one by one (fixed length or with termination byte), then parse message's header, and payload.
receive_data(Socket, SoFar) ->
receive
{tcp,Socket,Bin} -> %% (3)
receive_data(Socket, [Bin|SoFar]);
{tcp_closed,Socket} -> %% (4)
list_to_binary(reverse(SoFar)) %% (5)
end.
You can also set a gen_tcp socket in passive mode. This way, the owning process won't receive the input by messages but has to fetch it using gen_tcp:recv(Socket, ByteCount) which returns either {ok, Input} or {error, Reason}. As this methods waits infinitely for the bytes you might want to add a timeout using gen_tcp:recv/3. (Erlang documentation of gen_tcp:recv)
While at first glance it might seem the process is now completely unable to react to messages sent to it, there is the following workaround improving the situation a bit:
f1(X) ->
receive
message1 ->
... do something ...,
f1(X);
message2 ->
... do something ...,
f1(X)
after 0 %timeout in ms
{ok, Input} = gen_tcp:recv(Socket, ByteCount, Timeout),
... do something ... % maybe call some times gen_tcp:recv again
f1(X)
end.
If you don't add a timeout to gen_tcp:recv here, other processes could wait ages for f1 to handle their messages.
My scenario is as follows -
I have a client C with function foo() which performs some computation.
I'd like a server S, which doesn't know about foo(), to execute this function instead, and send the result back to the client.
I am trying to determine the best way to perform this in Erlang. I am considering:
Hot code swapping - i.e. "upgrade" code in S such that it has the function foo(). Execute and send back to the client.
In a distributed manner where nodes are all appropriately registered, do something along the lines of S ! C:foo() - for the purpose of "sending" the function to process/node S
Are there other methods (or features of the language) that I am not thinking of?
Thanks for the help!
If the computation function is self contained i.e. does not depend on any other modules or functions on the client C, then what you need to do is a fun (Functional Objects). A fun can be sent across the network and applied by a remote machine and in side the fun, the sender has embedded their address and a way of getting the answer back. So the executor may only see a fun to which they may or may not give an argument, yet inside the fun, the sender has forced a method where by the answer will automatically be sent back. The fun is an abstraction of very many tasks within one thing, and it can be moved around as arguments.
At the client, you can have code like this:
%% somewhere in the client
%% client runs on node() == 'client#domain.com'
-module(client).
-compile(export_all).
-define(SERVER,{server,'server#domain.com'}).
give_a_server_a_job(Number)-> ?SERVER ! {build_fun(),Number}.
build_fun()->
FunObject = fun(Param)->
Answer = Param * 20/1000, %% computation here
rpc:call('client#domain.com',client,answer_ready,[Answer])
end,
FunObject.
answer_ready(Answer)->
%%% use Answer for all sorts of funny things....
io:format("\n\tAnswer is here: ~p~n",[Answer]).
The server then has code like this:
%%% somewhere on the server
%%% server runs on node() == 'server#domain.com'
-module(server).
-compile(export_all).
start()-> register(server,spawn(?MODULE,loop,[])).
loop()->
receive
{Fun,Arg} ->
Fun(Arg), %% server executes job
%% job automatically sends answer back
%% to client
loop();
stop -> exit(normal);
_ -> loop()
end.
In this way, the job executor needs not know about how to send back the reply, The job itself comes knowing how it will send back the answer to however sent the job!. I have used this method of sending functional objects across the network in several project, its so cool !!!
#### EDIT #####
If you have a recursive problem, You manipulate recursion using funs. However, you will need at least one library function at the client and/or the server to assist in recursive manipulations. Create a function which should be in the code path of the client as well as the server.
Another option is to dynamically send code from the server to the client and then using the library: Dynamic Compile erlang to load and execute erlang code at the server from the client. Using dynamic compile, here is an example:
1> String = "-module(add).\n -export([add/2]). \n add(A,B) -> A + B. \n".
"-module(add).\n -export([add/2]). \n add(A,B) -> A + B. \n"
2> dynamic_compile:load_from_string(String).
{module,add}
3> add:add(2,5).
7
4>
What we see above is a piece of module code that is compiled and loaded dynamically from a string. If the library enabling this is available at the server and client , then each entity can send code as a string and its loaded and executed dynamically at the other. This code can be unloaded after use. Lets look at the Fibonacci function and how it can be sent and executed at the server:
%% This is the normal Fibonacci code which we are to convert into a string:
-module(fib).
-export([fib/1]).
fib(N) when N == 0 -> 0;
fib(N) when (N < 3) and (N > 0) -> 1;
fib(N) when N > 0 -> fib(N-1) + fib(N-2).
%% In String format, this would now become this piece of code
StringCode = " -module(fib).\n -export([fib/1]). \nfib(N) when N == 0 -> 0;\n fib(N) when (N < 3) and (N > 0) -> 1;\n fib(N) when N > 0 -> fib(N-1) + fib(N-2). \n".
%% Then the client would send this string above to the server and the server would %% dynamically load the code and execute it
send_fib_code(Arg)->
{ServerRegName,ServerNode} ! {string,StringCode,fib,Arg},
ok.
get_answer({fib,of,This,is,That}) ->
io:format("Fibonacci (from server) of ~p is: ~p~n",[This,That]).
%%% At Server
loop(ServerState)->
receive
{string,StringCode,Fib,Arg} when Fib == fib ->
try dynamic_compile:load_from_string(StringCode) of
{module,AnyMod} ->
Answer = AnyMod:fib(Arg),
%%% send answer back to client
%%% should be asynchronously
%%% as the channels are different & not make
%% client wait
rpc:call('client#domain.com',client,get_answer,[{fib,of,Arg,is,Answer}])
catch
_:_ -> error_logger:error_report(["Failed to Dynamic Compile & Load Module from client"])
end,
loop(ServerState);
_ -> loop(ServerState)
end.
That piece of rough code can show you what am trying to say. However, you remember to unload all un-usable dynamic modules. Also you can a have a way in which the server tries to check wether such a module was loaded already before loading it again. I advise that you donot copy and paste the above code. Look at it and understand it and then write your own version that can do the job. success !!!
If you do S ! C:foo() it will compute on client side function foo/1 from module C and send its result to process S. It doesn't seem like what you want to do. You should do something like:
% In client
call(S, M, F, A) ->
S ! {do, {M, F, A}, self()},
receive
{ok, V} -> V
end.
% In server
loop() ->
receive
{do, {M, F, A}, C} ->
C ! {ok, apply(M, F, A)},
loop()
end.
But in real scenario you would have to do a lot more work e.g. mark your client message to perform selective receive (make_ref/0), catch error in server and send it back to client, monitor server from client to catch server down, add some timeout and so. Look how are gen_server:call/2 and rpc:call/4,5 implemented and it is reason why there is OTP to save you from most of gotcha.