I have a variable:
Data = [[<<>>,
[<<"10">>,<<"171">>],
[<<"112">>,<<"Gen20267">>],
[<<"52">>,<<"20100812-06:32:30.687">>]]
I am trying to pattern match for two specific cases..
One where anything that resembles the outside structure - simply []
Anything inside goes I have tried [ _ ] but no go?
The Second, for a specific pattern inside, like when I see a <<"10">> or <<"112">> or <<"52">> then I am going to take the right side which is the actual data into an atom.
Basically the <<"10">> or <<"112">> or <<"52">> are the fields, the right side the data.
I have tried statements like [<<"10">>, _ ] still no go
Here is the rest of the code:
dataReceived(Message) ->
receive
{start} ->
ok;
[ _ ] -> %%No go
io:format("Reply 1 = ~p~n", [Message]);
[<<"10">>, _ ] -> %%No go
io:format("Reply 1 = ~p~n", [Message])
end.
As a note the Message is not sent as a tuple it is exactly like Data =
Can anyone lead me in the right direction?
Thanks and Goodnight!
-B
UPDATE
Ok now I think Im getting warmer, I have to pattern match whatever comes in.
So if I had say
Message = = [[<<>>],
[<<"10">>,<<"171">>],
[<<"112">>,<<"Gen20267">>],
[<<"52">>,<<"20100812-06:32:30.687">>]]
And I was looking to pattern match the field <<"112">>
Such as the 112 is always going to say 112, but the Gen2067 can change whenever to whatever.. its the data, it will be stored in a variable.
loop() ->
receive
[_,[<<"112">>, Data], _] when is_list(X) -> %% Match a list inside another.
?DEBUG("Got a list ~p~n", [X]),
loop();
_Other ->
?DEBUG("I don't understand ~p~n", [_Other]),
loop()
end.
I feel im close, but not 100%
-B
Update OP is trying to pass an argument to the function and not send messages.
As the name indicates the receive block is used to receive and process messages sent to a process. When you call dataReceived with an argument it proceeds to wait for messages. As no messages are sent it will continue to wait endlessly. Given the current code if you want it to do something then you'll have to spawn the function, get the process ID and then send a message to the process ID.
You probably need a function where the argument is pattern matched and not messages.
Something like this:
dataReceived([Message]) when is_list(Message) ->
io:format("Got a list as arg ~p~n", [Message]);
dataReceived(_Other) ->
io:format("Unknown arg ~p~n", [_Other]).
On a side note your third pattern [X] when is_list(X) will never match as the second pattern is a superset of the third. Anything that matches [X] when is_list(X) will always match [X] and therefore your third match clause will never get triggered.
Original Answer
I am not sure I understand your question. Are you trying to send a message to the function or are you passing it an argument?
This is a partial answer about how to match a list of lists in case you are sending a message.
-module(mtest).
-export([run/0]).
-ifdef(debug).
-define(DEBUG(Format, Args), io:format(Format, Args)).
-else.
-define(DEBUG(Format, Args), void).
-endif.
loop() ->
receive
[X] when is_list(X) -> %% Match a list inside another.
?DEBUG("Got a list ~p~n", [X]),
loop();
_Other ->
?DEBUG("I don't understand ~p~n", [_Other]),
loop()
end.
Take a look at the first clause in the receive block. [X] when is_list(X) will bind the inner list to the name X. I tested it with the value of Data you provided and it worked.
%% From the shell.
1> c(mtest, {d, debug}).
{ok,mtest}
2> Pid = mtest:run().
<0.40.0>
3> Data = [[<<>>, [<<"10">>,<<"171">>], [<<"112">>,<<"Gen20267">>], [<<"52">>,<<"20100812-06:32:30.687">>]]].
[[<<>>,
[<<"10">>,<<"171">>],
[<<"112">>,<<"Gen20267">>],
[<<"52">>,<<"20100812-06:32:30.687">>]]]
4> Pid ! Data.
[[<<>>,
[<<"10">>,<<"171">>],
[<<"112">>,<<"Gen20267">>],
[<<"52">>,<<"20100812-06:32:30.687">>]]]
Got a list [<<>>,
[<<"10">>,<<"171">>],
[<<"112">>,<<"Gen20267">>],
[<<"52">>,<<"20100812-06:32:30.687">>]]
5>
Related
I am looking at how to code "map reduce" type scenarios directly in erlang. As a toy example, imagine I want to decide which of several files is the biggest. Those files might be anywhere on the internet, so getting each one might take some time; so I'd like to gather them in parallel. Once I have them all, I can compare their sizes.
My assumed approach is as follows:
A 'main' process to co-ordinate the work and determine which is biggest;
A 'worker' process for each file, which fetches the file and returns the size to the main process.
Here's a clunky but functioning example (using local files only, but it shows the intent):
-module(cmp).
-export([cmp/2]).
cmp(Fname1, Fname2) ->
Pid1 = fsize(Fname1),
Pid2 = fsize(Fname2),
{Size1, Size2} = collect(Pid1, Pid2),
if
Size1 > Size2 ->
io:format("The first file is bigger~n");
Size2 > Size1 ->
io:format("The second file is bigger~n");
true ->
io:format("The files are the same size~n")
end.
fsize(Fname) ->
Pid = spawn(?MODULE, fsize, [self(), Fname]),
Pid.
fsize(Sender, Fname) ->
Size = filelib:file_size(Fname),
Sender ! {self(), Fname, Size}.
collect(Pid1, Pid2) ->
receive
{Pida, Fnamea, Sizea} ->
io:format("Pid: ~p, Fname: ~p, Size: ~p~n", [Pida, Fnamea, Sizea])
end,
receive
{Pidb, Fnameb, Sizeb} ->
io:format("Pid: ~p, Fname: ~p, Size: ~p~n", [Pidb, Fnameb, Sizeb])
end,
if
Pida =:= Pid1 -> {Sizea, Sizeb};
Pida =:= Pid2 -> {Sizeb, Sizea}
end.
Specific Questions
Is the approach idiomatic? i.e. hiving off each 'long running' task into a separate process, then collecting results back in a 'master'?
Is there a library to handle the synchronisation mechanics? Specifically, the collect function in the example above?
Thanks.
--
Note: I know the collect function in particular is clunky; it could be generalised by e.g. storing the pids in a list, and looping until all had completed.
In my opinion it's best to learn from an example, so I had a look at how they do that in otp/rpc and based on that I implemented a bit shorter/simpler version of the parallel eval call.
call(M, F, ArgL, Timeout) ->
ReplyTo = self(),
Keys = [spawn(fun() -> ReplyTo ! {self(), promise_reply, M:F(A)} end) || A <- ArgL],
Yield = fun(Key) ->
receive
{Key, promise_reply, {error, _R} = E} -> E;
{Key, promise_reply, {'EXIT', {error, _R} = E}} -> E;
{Key, promise_reply, {'EXIT', R}} -> {error, R};
{Key, promise_reply, R} -> R
after Timeout -> {error, timeout}
end
end,
[Yield(Key) || Key <- Keys].
I am not a MapReduce expert but I did had some experience using this 3rd party mapreduce module. So I will try to answer your question based on my current knowledge.
First, your input should be arranged as pairs of keys and values in order to properly use the mapreduce model. In general, your master process should first start workers processes (or nodes). Each worker receives a map function and a pair of key and value, lets name it {K1,V1}. It then executes the map function with the key and value and emits a new pair of key and value {K2,V2}. The master process collects the results and waits for all workers to finish their jobs. After all workers are done, the master starts the reduce part on the pairs {K2,List[V2]} that were emited by the workers. This part can be executed in parallel or not, it used to combine all the results into a single output. Note that the List[V2] is because there can be more then one value that was emited by the workers for a single K2 key.
From the 3rd party module I mentioned above:
%% Input = [{K1, V1}]
%% Map(K1, V1, Emit) -> Emit a stream of {K2,V2} tuples
%% Reduce(K2, List[V2], Emit) -> Emit a stream of {K2,V2} tuples
%% Returns a Map[K2,List[V2]]
If we look into Erlangs' lists functions, the map part is actually equal for doing lists:map/2 and the reduce part is in some way similar to lists:foldl/3 or lists:foldr/3 and the combination between them are: lists:mapfoldl/3, lists:mapfoldr/3.
If you are using this pattern of mapreduce using sets of keys and values, there is no need for special synchronization if that is what you mean. You just need to wait for all workers to finish their job.
I suggest you to go over the 3rd party module I mentioned above. Take also a look at this example. As you can see, the only things you need to define are the Map and Reduce functions.
I just had a simple question and I can't seem to find an answer to it. Basically my question is if I have a function recording state (recursive) and I send multiple messages to it, will it keep going through the receive block until it no longer has messages in its "mailbox"?
state(Fridge) ->
receive
Pat1 ->
io:format("ok"),
state(S);
Pat2 ->
io:format("not ok"),
state(S)
end.
So if I'd send to this process 3 messages (Pat1, Pat2, Pat1) using "!" and its not able to go into its loop before receiving messages will it still print out the following?
1> "ok"
2> "not ok"
3> "ok"
Sorry if this isn't very clearly put, by simplifying the question it might make it hard to picture what I am asking.
Your question isn't clear but you seem to be asking whether the process will receive the three messages even if they're sent before the target process has called receive — if that's the question, the answer is yes. When you send a message to a process, it goes into that process's message queue until the process calls receive to remove it from the message queue and deal with it.
If you call erlang:process_info(Pid, messages) where Pid is the receiver's process id, you can see what messages are in its queue. You might try this from the Erlang shell.
As an extreme example of message queueing, under some heavy load conditions it can be a source of out-of-memory problems if a receiver can't keep up with a fast sender. Under these conditions, a receiver's message queue might grow without bound until the system runs out of memory.
Does this answer your question?
1> OUT = fun(X) -> io:format(">>> ~p~n", [X]) end.
#Fun<erl_eval.6.54118792>
2> F = fun X() -> receive foo -> OUT(foo), X(); bar -> OUT(bar), X() end end.
#Fun<erl_eval.44.54118792>
3> P = spawn(F).
<0.38.0>
4> [ P ! X || X <- [foo, bar, foo]].
>>> foo
[foo,bar,foo]
>>> bar
>>> foo
A message arrives in the mailbox and then receive patterns are applied to the message like the function clause or case statement. But unlike those, if none of them match, next message is processed and the previous one left in the message box untouched. Other receive clause starts always from beginning of message queue.
1> OUT = fun(X) -> io:format(">>> ~p~n", [X]) end.
#Fun<erl_eval.6.54118792>
2> F = fun() -> receive start -> (fun X() -> receive foo -> OUT(foo), X(); bar -> OUT(bar), X() end end)() end end.
#Fun<erl_eval.20.54118792>
3> P = spawn(F).
<0.38.0>
4> [ P ! X || X <- [foo, bar, foo]].
[foo,bar,foo]
5> P! start.
>>> foo
start
>>> bar
>>> foo
Note that foo, bar, foo is in the queue in the first receive but it is not processed. When start arrives (last in the queue) second receive starts process foo and bar messages.
-module(wy).
-compile(export_all).
state() ->
timer:sleep(2000),
io:format("~p~n", [erlang:process_info(self(), messages)]),
receive
pat1 ->
io:format("ok~n"),
state();
pat2 ->
io:format("not ok~n"),
state()
end.
main() ->
Pid = spawn(fun state/0),
[ Pid ! X || X <- [pat1, pat2, pat1]].
You can run this code, from the output you can understand everything.
**Note:**when you send message to a process, the sent messages will be stored in mail box of the process.
6> wy:main().
[pat1,pat2,pat1]
7> {messages,[pat1,pat2,pat1]}
7> ok
7> {messages,[pat2,pat1]}
7> not ok
7> {messages,[pat1]}
7> ok
7> {messages,[]}
7>
One comments for your code:
receive
Pat1 ->
io:format("ok"),
state(S);
Pat2 ->
io:format("not ok"),
state(S)
end.
the second clause will not match forever.
Following is the program in which i have tried to spawn 3 processes form a method called best. I want to receive response from all the processes and store them in a tuple but I am able to get only one response.
test() ->
receive
{From,N} -> From!{self(),N},
loop()
end.
best(N) ->
Aid=spawn(fun t:loop/0),
Aid ! {self(),N},
Bid=spawn(fun t:loop/0),
Bid ! {self(),N},
Cid=spawn(fun t:loop/0),
Cid ! {self(),N},
receive
{Pid,Response} ->{Response}
end.
Can someone please help me out with this probem
Your receive bloc, in the best/2 function exit as soon as it receives one message. If you launch this code in the shell, you can verify that the other message are still in the message queue with the function flush(). (The code you posted is missing the t:loop/0 function, I guess it will compute something based on N and return the answer via a message to the spawner)
To be able to receive more than one message, you must put the receive bloc in a "loop" that recursively calls itself until it got all answers. You will have to use a variable that allows the recursive loop to know when it is finished (number of answers expected, list of processes that should answer...) and collect the answers in a list variable for example.
-module(wy).
-compile(export_all).
loop() ->
Self = self(),
receive
{From, Ref, N} ->
From ! {Self, Ref, N * N}
end.
receive_result(Ref) ->
receive
{Pid, Ref, R} ->
io:format("process ~p: ~p~n", [Pid, R]),
receive_result(Ref)
after 10 ->
ok
end.
best() ->
APid = spawn(fun loop/0),
BPid = spawn(fun loop/0),
CPid = spawn(fun loop/0),
Self = self(),
Ref = make_ref(),
APid ! {Self, Ref, 2},
BPid ! {Self, Ref, 3},
CPid ! {Self, Ref, 4},
receive_result(Ref).
You can follow this small code. The result is:
9> wy:best().
process <0.77.0>: 4
process <0.78.0>: 9
process <0.79.0>: 16
ok
I've been making a chat application in Erlang for a school project, but I've run into an issue. I'm trying to make my program concurrent and in order to do so I start a new thread for each message a channel is sending. I do this using lists:foreach, but I want to make sure that I don't message the person who typed in the channel.
request(State, {message, {UserID,UserPID}, Token}) ->
case catch lists:member({UserID,UserPID}, State#channel.users) of
false ->
{{error, user_not_joined}, State};
true ->
spawn( fun() ->
ListOfUsers = State#channel.users,
UserPIDs = lists:map(fun ({_, V}) -> V end, ListOfUsers),
%spawn( fun() ->end)
lists:foreach(
fun(UserPID) -> ok end,
fun(PID) ->
spawn( fun() -> genserver:request(PID, {message_from_server, State#channel.name, UserID, Token}) end)
end, UserPIDs) end),
{ok, State}
end;
What I pretty much want to do is to match with the UserPID inside the foreach. As of now I only get warnings:
channel.erl:39: Warning: variable 'UserPID' is unused
channel.erl:39: Warning: variable 'UserPID' shadowed in 'fun'
Line 3 is fun(UserPID) -> ok end,
Cheers
The answer by legoscia is fine, but I'd add that often list comprehension is simpler to use and simpler to read than lists:foreach. Note that list comprehension is able to ignore values based on clauses. Consider the following example:
-module(filter).
-export([do/0]).
do() ->
Values = lists:seq(1,10),
IgnoreThisValue = 5,
%% print all values unequal to IgnoreThisValue
[io:format("Value: ~b~n", [Value]) ||
Value <- Values, Value =/= IgnoreThisValue],
ok.
Run it in the shell:
1> make:all([load]).
Recompile: filter
up_to_date
2> filter:do().
Value: 1
Value: 2
Value: 3
Value: 4
Value: 6
Value: 7
Value: 8
Value: 9
Value: 10
A side note on your code: Why do you spawn a thread per process? I assume that you are using the behaviour gen_server (correct me if I am wrong). If so, you should consider using the cast function to simply send a message. As you do not check the result of genserver:request/2, this might be a viable option which saves you a lot of processes.
Since the function argument shadows the existing variable, you need to use a guard for that:
fun(PID) when PID =:= UserPID -> ok end
From the Learn You Some Erlang for Great Good!
Another special case is when the timeout is at 0:
flush() ->
receive
_ -> flush()
after 0 ->
ok
end
.
When that happens, the Erlang VM will try and find a message that fits
one of the available patterns. In the case above, anything matches. As
long as there are messages, the flush/0 function will recursively call
itself until the mailbox is empty. Once this is done, the after 0 ->
ok part of the code is executed and the function returns.
I don't understand purpose of after 0. After reading above text I thought it was like after infinity (waiting forever) but I changed a little the flush function:
flush2() ->
receive
_ -> timer:sleep(1000), io:format("aa~n"), flush()
after 0 ->
okss
end
.
flush3() ->
receive
_ -> io:format("aa~n"), flush()
after 0 ->
okss
end
.
In the first function it waits 1 second and in the second function it doesn't wait.
In both cases it doesn't display a text (aa~n).
So it doesn't work as after infinity.
If block between the receive and the after are not executed then above 2 codes can be simplified to:
flush4() ->
okss
.
What I am missing?
ps. I am on the Erlang R16B03-1, and author of the book was, as fair I remember, was on the Erlang R13.
Every process has a 'mailbox' -- message queue. Messages can be fetched by receive. if there is no messages in the queue. after part specifies how much time 'receive will wait for them. So after 0 -- means process checking (by receive ) if any messages in the queue and if queue is empty immediately continue to next instructions.
It can be used for instance if we want periodically check if any messages here and to do something (hopefully helpful) if there is no messages.
Consider after 0 to be finally.
Consider the use of after 0 to process receives with a priority: http://learnyousomeerlang.com/more-on-multiprocessing#selective-receives
May this different look on things enlighten you.
You can play with the following shell command to understand the effect of the after command:
4> L = fun(G) ->
4> receive
4> stop -> ok;
4> M -> io:format("received ~p~n",[M]), G(G)
4> after 0 ->
4> io:format("no message~n")
4> end
4> end.
#Fun<erl_eval.6.80484245>
5> F = fun() -> timer:sleep(10000),
5> io:format("end of wait for messages, go to receive block~n"),
5> L(L)end.
#Fun<erl_eval.20.80484245>
6> spawn(F).
<0.46.0>
end of wait for messages, go to receive block
no message
7> P1 = spawn(F).
<0.52.0>
8> P1 ! hello.
hello
end of wait for messages, go to receive block
received hello
no message
9> P2 ! hello, P2 ! stop.
* 1: variable 'P2' is unbound
8> P2 = spawn(F).
<0.56.0>
9> P2 ! hello, P2 ! stop.
stop
end of wait for messages, go to receive block
received hello
10>
If you do not intend to use a nested receive, rather than using "after" part, I think a better approach is to use "Unexpected ->" variable to handle all unmatched messages.