How to use rpc:pmap correctly? - erlang

I've found in module rpc function pmap. And I've stuck when tried to use it.
The first question - does it requires connection with other nodes, or it would performs as lists:map if there is no connection to other nodes?
I've tried to execute pmap without connection to other nodes, but got exception:
7> rpc:pmap( { erlang, '+' }, [], [[1,1],[2,3]] ).
** exception exit: badrpc
in function rpc:check/2 (rpc.erl, line 745)
After that - I've launched another local node and connected it with current. But still get the same error.
Please provide me how to use rpc:pmap correctly.
Thanks
P.S.
Following code works expectedly (returns result 3):
70> erlang:apply( erlang, '+', [1,2] ).
3
71> erlang:'+'(1,2).
3

The rpc:map({Module, Function},ExtraArgs,List1) evaluates apply(Module, Function, [Elem|ExtraArgs]), for every element Elem in List1, in parallel.
So the right syntax using erlang:'+' is:
1> rpc:pmap( { erlang, '+' }, [2], [1,2,3] ).
[3,4,5]
2>
which evaluates [1+2,2+2,3+2]
the syntax rpc:pmap( { lists, sum }, [], [[1,2],[4,5]] ) will evaluate [1+2,4+5] and return
[3,9]
Pascal

Related

Why the output differs in these two erlang expression sequence in shell?

In Erlang shell why the following produces different result?
1> Total=15.
2> Calculate=fun(Number)-> Total=2*Number end.
3> Calculate(6).
exception error: no match of right hand side value 12
1> Calculate=fun(Number)-> Total=2*Number end.
2> Total=15.
3> Calculate(6).
12
In Erlang the = operator is both assignment and assertion.
If I do this:
A = 1,
A = 2,
my program will crash. I just told it that A = 1 which, when A is unbound (doesn't yet exist as a label) it now is assigned the value 1 forever and ever -- until the scope of execution changes. So then when I tell it that A = 2 it tries to assert that the value of A is 2, which it is not. So we get a crash on a bad match.
Scope in Erlang is defined by two things:
Definition of the current function. This scope is absolute for the duration of the function definition.
Definition of the current lambda or list comprehension. This scope is local to the lambda but also closes over whatever values from the outer scope are referenced.
These scopes are always superceded at the time they are declared by whatever is in the outer scope. That is how we make closures with anonymous functions. For example, let's say I had have a socket I want to send a list of data through. The socket is already bound to the variable name Socket in the head of the function, and we want to use a list operation to map the list of values to send to a side effect of being sent over that specific socket. I can close over the value of the socket within the body of a lambda, which has the effect of currying that value out of the more general operation of "sending some data":
send_stuff(Socket, ListOfMessages) ->
Send = fun(Message) -> ok = gen_tcp:send(Socket, Message) end,
lists:foreach(Send, ListOfMessages).
Each iteration of the list operation lists:foreach/2 can only accept a function of arity 1 as its first argument. We have created a closure that captures the value of Socket internally already (because that was already bound in the outer scope) and combines it with the unbound, inner variable Message. Note also that we are checking whether gen_tcp:send/2 worked each time within the lambda by asserting that the return value of gen_tcp:send/2 was really ok.
This is a super useful property.
So with that in mind, let's look at your code:
1> Total = 15.
2> Calculate = fun(Number)-> Total = 2 * Number end.
3> Calculate(6).
In the code above you've just assigned a value to Total, meaning you have created a label for that value (just like we had assigned Socket in the above example). Then later you are asserting that the value of Total is whatever the result of 2 * Number might be -- which can never be true since Total was an integer so 2 * 7.5 wouldn't cut it either, because the result would be 15.0, not 15.
1> Calculate = fun(Number)-> Total = 2 * Number end.
2> Total = 15.
3> Calculate(6).
In this example, though, you've got an inner variable called Total which does not close over any value declared in the outer scope. Later, you are declaring a label in the outer scope called Total, but by this time the lambda definition on the first line has been converted to an abstract function and the label Total as used there has been completely given over to the immutable space of the new function definition the assignment to Calculate represented. Thus, no conflict.
Consider what happens, for example, with trying to reference an inner value from a list comprehension:
1> A = 2.
2
2> [A * B || B <- lists:seq(1,3)].
[2,4,6]
3> A.
2
4> B.
* 1: variable 'B' is unbound
This is not what you would expect from, say, Python 2:
>>> a = 2
>>> a
2
>>> [a * b for b in range(1,4)]
[2, 4, 6]
>>> b
3
Incidentally, this has been fixed in Python 3:
>>> a = 2
>>> a
2
>>> [a * b for b in range(1,4)]
[2, 4, 6]
>>> b
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'b' is not defined
(And I would provide a JavaScript example for comparison as well, but the scoping rules there are just so absolutely insane it doesn't even matter...)
In the first case you have bound Total to 15. In Erlang, variable are unmutable, but in the shell when you write Total = 15. you do not really create the variable Total, the shell does its best to mimic the behavior you will have if you were running an application, and it stores in a table the couple {"Total",15}.
On the next line you define the fun Calculate. the Parser find the expression Total=2*Number, and it goes through its table to detect that Total was previously defined. The evaluation is turned into something equivalent to 15 = 2*Number.
So, in the third line, when you ask to evaluate Calculate(6), it goes to calculate and evaluates 15 = 2*6 and issues the error message
exception error: no match of right hand side value 12
In the second example, Total is not yet defined when you define the function. The function is stored without assignment (Total is not used anymore), at least no assignment to a global variable. So there is no conflict when you define Total, and no error when you evaluate Calculate(6).
The behavior would be exactly the same in a compiled module.
The variable 'Total' is already assigned a value 15, so you can NOT using the same variable name Total in the second line. You should change to the other name Total1 or Total2...

Erlang list of tuples jsx:encode error

I want to create a List of tuples in json format with jsx:encode, which is returning error:
io:format("Mylist After reverse ==========: ~n~p~n",[Mylist]),
FinalReading = lists:map(
fun(X) ->
jsx:encode([X])
end, Mylist),
Mylist is printed as follows:
[{{<<"name">>,<<"Batman">>},{<<"steps">>,1033},{<<"distance">>,830}},
{{<<"name">>,<<"Superman">>},{<<"steps">>,641},{<<"distance">>,520}}]
I'm getting this error:
09:49:24.048 [error] ** Boss Service Handler myproj_priv_mylib_websocket terminating in handle_incoming/4
for the reason error:badarg
ServiceUrl: "/websocket/mylib"
WebSocketId: <0.336.0>
SessionId : <<"a4f60118091309990af7c89c9a1acb49ee3bb08d">>
Message : <<"admin#mydomain.com:z3CMLU9jDTYdiAacNiJrMIkdp4lTf6sb">>
State : []
** Stacktrace: [{jsx_parser,value,4,[{file,"src/jsx_parser.erl"},{line,125}]},{lists,map,2,[{file,"lists.erl"},{line,1237}]},{myproj_priv_mylib_websocket,handle_incoming,5,[{file,"..."},{line,130}]},{boss_service_worker,handle_cast,2,[{file,"src/boss/boss_service_worker.erl"},{line,173}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,599}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]
It would be great if someone points out the correct code.
Thanking all in advance.
You'll have to convert the top level tuples to lists using erlang:tuple_to_list/1 and lists:map/2 so that jsx can correctly encode them to a JSON Array.
1> List = [{{<<"name">>,<<"Batman">>},{<<"steps">>,1033},{<<"distance">>,830}},
1> {{<<"name">>,<<"Superman">>},{<<"steps">>,641},{<<"distance">>,520}}].
[{{<<"name">>,<<"Batman">>},
{<<"steps">>,1033},
{<<"distance">>,830}},
{{<<"name">>,<<"Superman">>},
{<<"steps">>,641},
{<<"distance">>,520}}]
2> List2 = lists:map(fun erlang:tuple_to_list/1, List).
[[{<<"name">>,<<"Batman">>},
{<<"steps">>,1033},
{<<"distance">>,830}],
[{<<"name">>,<<"Superman">>},
{<<"steps">>,641},
{<<"distance">>,520}]]
3> io:format("~s~n", [jsx:encode(List2)]).
[{"name":"Batman","steps":1033,"distance":830},{"name":"Superman","steps":641,"distance":520}]
I'm very much a newbie to Erlang, so not sure if I've the right to judge, but my feeling is that it would be good to hide the need to convert the tuple to list in jsx:encode.
If
[{a,<<"a">>}, {b,<<"b">>}].
becomes
{
"a" : "a",
"b" : "b"
}
I believe the same should work as well for this syntax
{{a,<<"a">>}, {b,<<"b">>}}.
This last syntax brings more information (it reflects an ordering which I can count on, within the code, for pattern matching).
I usually try to convert my JSON into a tuple for which I know the ordering, so that I can play with pattern matching instead of using keyfind and friends. But because the above syntax does not work, I have to wrap jsx:encode(tuple_to_list()) to automatically manage "PropTuples".
Am I completely misunderstanding it? Is there a more decent way to represent/treat/manipulate JSON in Erlang?

what's the difference between list_to_binary and iolist_to_binary?

I only know list_to_binary in erlang otp, but today I see iolist_to_binary?
I tried it in erlang shell, but I can't find the difference.
(ppb1_bs6#esekilvxen263)59> list_to_binary([<<1>>, [1]]).
<<1,1>>
(ppb1_bs6#esekilvxen263)60> iolist_to_binary([<<1>>, [1]]).
<<1,1>>
(ppb1_bs6#esekilvxen263)61> iolist_to_binary([<<1>>, [1], 1999]).
** exception error: bad argument
in function iolist_to_binary/1
called as iolist_to_binary([<<1>>,[1],1999])
(ppb1_bs6#esekilvxen263)62> list_to_binary([<<1>>, [1], 1999]).
** exception error: bad argument
in function list_to_binary/1
called as list_to_binary([<<1>>,[1],1999])
(ppb1_bs6#esekilvxen263)63> list_to_binary([<<1>>, [1], <<1999>>]).
<<1,1,207>>
(ppb1_bs6#esekilvxen263)64> ioslist_to_binary([<<1>>, [1], <<1999>>]).
** exception error: undefined shell command ioslist_to_binary/1
(ppb1_bs6#esekilvxen263)65> iolist_to_binary([<<1>>, [1], <<1999>>]).
<<1,1,207>>
From the test, I think iolist_to_binary may be the same as list_to_binary.
In current versions of Erlang/OTP both functions will automatically flatten the list or iolist() you provide, leaving functionally only one difference, which is that list_to_binary/1 will not accept a binary parameter, but iolist_to_binary/1 will:
1> erlang:iolist_to_binary(<<1,2,3>>).
<<1,2,3>>
2> erlang:list_to_binary(<<1,2,3>>).
** exception error: bad argument
in function list_to_binary/1
called as list_to_binary(<<1,2,3>>)
This is clear from the type specification of iolist():
maybe_improper_list(byte() | binary() | iolist(), binary() | [])
As you can see, iolist() can be a binary, but clearly a list can never be a binary.
If you want to follow through the convolution of the improper list type you can find its definition, along with iolist() at (Erlang) Types and Function Specifications.
In terms of practical use iolist() is a messy or unflattened list. Think of it as a kind of lazy output, deliberately not flattened to be a proper list by the produver as an optimisation because not every consumer will need to flatten it. You can see what it looks like by checking the output of io_lib:format/2:
1> io_lib:format("Hello ~s ~b!~n", ["world", 834]).
[72,101,108,108,111,32,"world",32,"834",33,"\n"]
Or:
2> io:format("~w~n", [ io_lib:format("Hello ~s ~b!~n", ["world", 834]) ]).
[72,101,108,108,111,32,[119,111,114,108,100],32,[56,51,52],33,[10]]
I notice that many of your examples include the number 1999. Attempts to convert any list or iolist() which contain any number that is greater than 255 to a binary will always fail by the way:
8> erlang:list_to_binary([1,2,3,255]).
<<1,2,3,255>>
9> erlang:list_to_binary([1,2,3,256]).
** exception error: bad argument
in function list_to_binary/1
called as list_to_binary([1,2,3,256])
This is because it wants to create a list of bytes from your list, and a number above 255 is ambiguous in terms of its possible byte representation; it can't be known if you mean little endian or big endian, etc (see (Wikipedia) Endianness).
The differences are not large. Type checking is done. But in the end caused by one and the same function. If I correctly found.
Source list_to_binary (https://github.com/erlang/otp/blob/maint/erts/emulator/beam/binary.c#L896):
BIF_RETTYPE list_to_binary_1(BIF_ALIST_1)
{
return erts_list_to_binary_bif(BIF_P, BIF_ARG_1, bif_export[BIF_list_to_binary_1]);
}
Source iolist_to_binary (https://github.com/erlang/otp/blob/maint/erts/emulator/beam/binary.c#L903):
BIF_RETTYPE iolist_to_binary_1(BIF_ALIST_1)
{
if (is_binary(BIF_ARG_1)) {
BIF_RET(BIF_ARG_1);
}
return erts_list_to_binary_bif(BIF_P, BIF_ARG_1, bif_export[BIF_iolist_to_binary_1]);
}
iolist_to_binary - converts iolist to binary
for your ref:
http://www.erlangpatterns.org/iolist.html
http://www.erlang.org/doc/man/erlang.html

Erlang MapReduce on Riak database giving exception

I am going through the tutorials of riak and Erlang, I have stored data in riak by the riak-erlang-client and i did the following:
1> {ok,Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087).
{ok,<0.34.0>}
2> Val1 = [1,2,3].
[1,2,3]
3> MyBucket = <<"numbers">>.
<<"numbers">>
4> Obj1 = riakc_obj:new(MyBucket,<<"ott">>,Val1).
5> riakc_pb_socket:put(Pid,Obj1).
ok
6> {ok, Fetched} = riakc_pb_socket:get(Pid,MyBucket,<<"ott">>).
{ok,{riakc_obj,<<"numbers">>,<<"ott">>,
<<107,206,97,96,96,96,204,96,202,5,82,28,202,156,255,126,
6,251,159,184,148,193,148,...>>,
[{{dict,3,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],...},
{{[],[],[],[],[],[],[],[],[],[],...}}},
<<131,107,0,3,1,2,3>>}],
undefined,undefined}}
7> Val1 == binary_to_term(riakc_obj:get_value(Fetched)).
true
8> binary_to_term(riakc_obj:get_value(Fetched)).
[1,2,3]
Now everything was fine , i want to extract the values from the list [1,2,3] that where only divisible by 2 from a mapreduce through erlang.
for that i have done the following,
9>Res = fun(Fetched,none)-> [X || X <- binary_to_term(riakc_obj:get_value(Fetched)), X rem 2 == 0] end.
I don't know exactly about what should write in the function parameters,as i am new to erlang and riak, so please correct me if am wrong.
Now when i going for mapreduce i have written this,
10>{ok , [{X}]} = riakc_pb_socket:mapred(Pid,[{<<"numbers">>,<<"ott">>}],[{map,{qfun,Res},none,true}]).
It was throwing the below exception..please suggest me a good way to understand what's going on and how to run mapreduce.
** exception exit: {noproc,{gen_server,call,
[<0.34.0>,
{req,{rpbmapredreq,<<131,108,0,0,0,3,104,2,100,0,6,105,
110,112,117,116,115,108,...>>,
<<"application/x-erlang-binary">>},
60100,
{132578423,<0.49.0>}},
infinity]}}
in function gen_server:call/3 (gen_server.erl, line 188)
in call from riakc_pb_socket:mapred/5 (src/riakc_pb_socket.erl, line 643)
Please,help me and suggest me, i hjave followed the links but i am not getting clearly that how to write on my own for a mapreduce so please guide me..it will help me.
You are getting the noproc error because you are passing an arity-2 function. The map phase function should be arity-3. Also, specifying none in the function header will cause that clause to only be executed when the corresponding argument explicitly matches, use _ in place of arguments you don't care about, like:
Res = fun(Fetched,_,_)-> [X || X <- binary_to_term(riakc_obj:get_value(Fetched)), X rem 2 == 0] end.

Exception error in Erlang

So I've been using Erlang for the last eight hours, and I've spent two of those banging my head against the keyboard trying to figure out the exception error my console keeps returning.
I'm writing a dice program to learn erlang. I want it to be able to call from the console through the erlang interpreter. The program accepts a number of dice, and is supposed to generate a list of values. Each value is supposed to be between one and six.
I won't bore you with the dozens of individual micro-changes I made to try and fix the problem (random engineering) but I'll post my code and the error.
The Source:
-module(dice2).
-export([d6/1]).
d6(1) ->
random:uniform(6);
d6(Numdice) ->
Result = [],
d6(Numdice, [Result]).
d6(0, [Finalresult]) ->
{ok, [Finalresult]};
d6(Numdice, [Result]) ->
d6(Numdice - 1, [random:uniform(6) | Result]).
When I run the program from my console like so...
dice2:d6(1).
...I get a random number between one and six like expected.
However when I run the same function with any number higher than one as an argument I get the following exception...
**exception error: no function clause matching dice2:d6(1, [4|3])
... I know I I don't have a function with matching arguments but I don't know how to write a function with variable arguments, and a variable number of arguments.
I tried modifying the function in question like so....
d6(Numdice, [Result]) ->
Newresult = [random:uniform(6) | Result],
d6(Numdice - 1, Newresult).
... but I got essentially the same error. Anyone know what is going on here?
This is basically a type error. When Result is a list, [Result] is a list with one element. E.g., if your function worked, it would always return a list with one element: Finalresult.
This is what happens (using ==> for "reduces to"):
d6(2) ==> %% Result == []
d6(2, [[]]) ==> %% Result == [], let's say random:uniform(6) gives us 3
d6(1, [3]) ==> %% Result == 3, let's say random:uniform(6) gives us 4
d6(0, [4|3]) ==> %% fails, since [Result] can only match one-element lists
Presumably, you don't want [[]] in the first call, and you don't want Result to be 3 in the third call. So this should fix it:
d6(Numdice) -> Result = [], d6(Numdice, Result). %% or just d6(Numdice, []).
d6(0, Finalresult) -> {ok, Finalresult};
d6(Numdice, Result) -> d6(Numdice - 1, [random:uniform(6) | Result]).
Lesson: if a language is dynamically typed, this doesn't mean you can avoid getting the types correct. On the contrary, it means that the compiler won't help you in doing this as much as it could.

Resources