Related
I was reading Learn You Some Erlang and I came upon this example in the Recursion chapter.
tail_sublist(_, 0, SubList) -> SubList;
tail_sublist([], _, SubList) -> SubList;
tail_sublist([H|T], N, SubList) when N > 0 ->
tail_sublist(T, N-1, [H|SubList]).
As the author goes on to explain that there is a fatal flaw in our code. It being that the sub lists hence produced would be reverse and we would have to re-reverse them to get the correct output. In contrast, what I did was use the ++ operator to avoid reversing the list later.
sublist_tail([],_,Acc) -> Acc;
sublist_tail(_,0,Acc) -> Acc;
sublist_tail([H|T],N,Acc) -> sublist_tail(T,N-1,Acc++[H]).
My question is, is the ++ operator more expensive than the | operator? And if it is, would my solution (using ++ operator) still be slow compared to the author's solution (including reversing the list to get the correct output)?
You might want to read about this issue in the Erlang efficiency guide, since there it says that building the list via | and then reversing the result is more efficient than using the appending ++ operator. If you want to know the performance difference, use timer:tc:
1> timer:tc(fun() -> lists:reverse(lists:foldl(fun(V, Acc) -> [V|Acc] end, [], lists:seq(1,1000))) end).
{1627,
[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,
23,24,25,26,27|...]}
2> timer:tc(fun() -> lists:foldl(fun(V, Acc) -> Acc++[V] end, [], lists:seq(1,1000)) end).
{6216,
[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,
23,24,25,26,27|...]}
Both approaches create lists of 1000 integers, but these measurements based on Erlang/OTP 17.5 show the prepending/reversing version is roughly 4x faster than the appending version (YMMV of course).
is the ++ operator more expensive than the | operator?
That depends. If you use it correctly, then no. ++ is only dangerous when you have a big left-hand-side operand.
Each time a "++"-operator is invoked on a left-hand List (like: List1 ++ List2), you are creating a new List, that is a copy of your left-hand operand (List1). Each copy operation then has a runtime, that is dependent on the length of your List1 (which keeps growing with your iterations).
So, if you prepend your values 'head first', you don't have to perform a copy-operation over the whole list in each step. This also means, accumulation with ++ at the head of the List wouldn't be so bad either, since only the "H"-value is copied once in each iteration:
sublist_tail([H|T],N,Acc) -> sublist_tail(T,N-1,[H]++Acc).
But if you are already accumulating head-first (and thus have to reverse later anyhow), you can do it with the cons-operator (|)
sublist_tail([H|T],N,Acc) -> sublist_tail(T,N-1,[H|Acc]).
This is the 'proper' way, since (please correct me if I am wrong) ++ is only syntactic sugar and is implemented internally with a cons-operator (|).
I'm a completely new to erlang. As an exercise to learn the language, I'm trying to implement the function sublist using tail recursion and without using reverse. Here's the function that I took from this site http://learnyousomeerlang.com/recursion:
tail_sublist(L, N) -> reverse(tail_sublist(L, N, [])).
tail_sublist(_, 0, SubList) -> SubList;
tail_sublist([], _, SubList) -> SubList;
tail_sublist([H|T], N, SubList) when N > 0 ->
tail_sublist(T, N-1, [H|SubList]).
It seems the use of reverse in erlang is very frequent.
In Mozart/Oz, it's very easy to create such the function using unbound variables:
proc {Sublist Xs N R}
if N>0 then
case Xs
of nil then
R = nil
[] X|Xr then
Unbound
in
R = X|Unbound
{Sublist Xr N-1 Unbound}
end
else
R=nil
end
end
Is it possible to create a similar code in erlang? If not, why?
Edit:
I want to clarify something about the question. The function in Oz doesn't use any auxiliary function (no append, no reverse, no anything external or BIF). It's also built using tail recursion.
When I ask if it's possible to create something similar in erlang, I'm asking if it's possible to implement a function or set of functions in erlang using tail recursion, and iterating over the initial list only once.
At this point, after reading your comments and answers, I'm doubtful that it can be done, because erlang doesn't seem to support unbound variables. It seems that all variables need to be assigned to value.
Short Version
No, you can't have a similar code in Erlang. The reason is because in Erlang variables are Single assignment variables.
Unbound Variables are simply not allowed in Erlang.
Long Version
I can't imagine a tail recursive function similar to the one you presenting above due to differences at paradigm level of the two languages you are trying to compare.
But nevertheless it also depends of what you mean by similar code.
So, correct me if I am wrong, the following
R = X|Unbound
{Sublist Xr N-1 Unbound}
Means that the attribution (R=X|Unbound) will not be executed until the recursive call returns the value of Unbound.
This to me looks a lot like the following:
sublist(_,0) -> [];
sublist([],_) -> [];
sublist([H|T],N)
when is_integer(N) ->
NewTail = sublist(T,N-1),
[H|NewTail].
%% or
%%sublist([H|T],N)
%% when is_integer(N) -> [H|sublist(T,N-1)].
But this code isn't tail recursive.
Here's a version that uses appends along the way instead of a reverse at the end.
subl(L, N) -> subl(L, N, []).
subl(_, 0, Accumulator) ->
Accumulator;
subl([], _, Accumulator) ->
Accumulator;
subl([H|T], N, Accumulator) ->
subl(T, N-1, Accumulator ++ [H]).
I would not say that "the use of reverse in Erlang is very frequent". I would say that the use of reverse is very common in toy problems in functional languages where lists are a significant data type.
I'm not sure how close to your Oz code you're trying to get with your "is it possible to create a similar code in Erlang? If not, why?" They are two different languages and have made many different syntax choices.
I'm trying to create a list and print it out, counting down from N to 1. This is my attempt:
%% Create a list counting down from N to 1 %%
-module(list).
-export([create_list/1]).
create_list(N) when length(N)<hd(N) ->
lists:append([N],lists:last([N])-1),
create_list(lists:last([N])-1);
create_list(N) ->
N.
This works when N is 1, but otherwise I get this error:
172> list:create_list([2]).
** exception error: an error occurred when evaluating an arithmetic expression
in function list:create_list/1 (list.erl, line 6)
Any help would be appreciated.
You should generally avoid using append or ++, which is the same thing, when building lists. They both add elements to the end of a list which entails making a copy of the list every time. Sometimes it is practical but it is always faster to work at the front of the list.
It is a bit unclear in which order you wanted the list so here are two alternatives:
create_up(N) when N>=1 -> create_up(1, N). %Create the list
create_up(N, N) -> [N];
create_up(I, N) ->
[I|create_up(I+1, N)].
create_down(N) when N>1 -> %Add guard test for safety
[N|create_down(N-1)];
create_down(1) -> [1].
Neither of these are tail-recursive. While tail-recursion is nice it doesn't always give as much as you would think, especially when you need to call a reverse to get the list in the right order. See Erlang myths for more information.
The error is lists:last([N])-1. Since N is an array as your input, lists:last([N]) will return N itself. Not a number you expect. And if you see the warning when compiling your code, there is another bug: lists:append will not append the element into N itself, but in the return value. In functional programming, the value of a variable cannot be changed.
Here's my implementation:
create_list(N) ->
create_list_iter(N, []).
create_list_iter(N, Acc) ->
case N > 0 of
true -> NewAcc = lists:append(Acc, [N]),
create_list_iter(N-1, NewAcc);
false -> Acc
end.
If I correctly understand your question, here is what you'll need
create_list(N) when N > 0 ->
create_list(N, []).
create_list(1, Acc) ->
lists:reverse([1 | Acc]);
create_list(N, Acc) ->
create_list(N - 1, [N | Acc]).
If you work with lists, I'd suggest you to use tail recursion and lists construction syntax.
Also, to simplify your code - try to use pattern matching in function declarations, instead of case expressions
P.S.
The other, perhaps, most simple solution is:
create_list(N) when N > 0 ->
lists:reverse(lists:seq(1,N)).
It is easy to implement the algorithm using a single process, however, how can I use multiple processes to do the job?
Here is what I have done so far.
find_largest([H], _) -> H;
find_largest([H, Q | T], R) ->
if H > Q -> find_largest([H | T], [Q | R]);
true -> find_largest([Q | T], [H | R])
end.
Thanks
Given how Erlang represents lists, this is probably not a good idea to try and do in parallel. Partitioning the list implies a lot of copying (since they are linked lists) and so does sending these partitions to other processes. I expect the comparison to be far cheaper than copying everything twice and then combining the results.
The implementation is also not correct, you can find a good one in lists.erl as max/1
%% max(L) -> returns the maximum element of the list L
-spec max([T,...]) -> T.
max([H|T]) -> max(T, H).
max([H|T], Max) when H > Max -> max(T, H);
max([_|T], Max) -> max(T, Max);
max([], Max) -> Max.
If by some chance your data are already in separate processes, simply get the lists:max/1 or each of the lists and send them to a single place, and then get the lists:max/1 of the result list. You could also do the comparison as you receive the results to avoid building this intermediate list.
The single process version of your code should be replaced by lists:max/1. A useful function for parallelizing code is as follows:
pmap(Fun, List) ->
Parent = self(),
P = fun(Elem) ->
Ref = make_ref(),
spawn_link(fun() -> Parent ! {Ref, Fun(Elem)} end),
Ref
end,
Refs = [P(Elem) || Elem <- List],
lists:map(fun(Ref) -> receive {Ref, Elem} -> Elem end end, Refs).
pmap/2 applies Fun to each member of List in parallel and collects the results in input order. To use pmap with this problem, you would need to segment your original list into a list of lists and pass that to pmap. e.g. lists:max(pmap(fun lists:max/1, ListOfLists)). Of course, the act of segmenting the lists would be more expensive than simply calling lists:max/1, so this solution would require that the list be pre-segmented. Even then, it's likely that the overhead of copying the lists outweighs any benefit of parallelization - especially on a single node.
The inherent problem with your situation is that the computation of each sub-task is tiny when compared with the overhead of managing the data. Tasks which are more computationally intensive, (e.g. factoring a list of large numbers), are more easily parallelized.
This isn't to say that finding a max value can't be parallelized, but I believe it would require that your data be pre-segmented or segmented in a way that didn't require iterating over every value.
I am getting myself familiar to Sequential Erlang (and the functional programming thinking) now. So I want to implement the following two functionality without the help of BIF. One is left_rotate (which I have come up with the solution) and the other is right_rotate (which I am asking here)
-export(leftrotate/1, rightrotate/1).
%%(1) left rotate a lits
leftrotate(List, 0) ->
List;
leftrotate([Head | Tail], Times) ->
List = append(Tail, Head),
leftrotate(List, Times -1).
append([], Elem)->
[Elem];
append([H|T], Elem) ->
[H | append(T, Elem)].
%%right rotate a list, how?
%%
I don't want to use BIF in this exercise. How can I achieve the right rotation?
A related question and slightly more important question. How can I know one of my implementation is efficient or not (i.e., avoid unnecessary recursion if I implement the same thing with the help of a BIF, and etc.)
I think BIF is built to provide some functions to improve efficiency that functional programming is not good at (or if we do them in a 'functional way', the performance is not optimal).
The efficiency problem you mention has nothing to do with excessive recursion (function calls are cheap), and everything to do with walking and rebuilding the list. Every time you add something to the end of a list you have to walk and copy the entire list, as is obvious from your implementation of append. So, to rotate a list N steps requires us to copy the entire list out N times. We can use lists:split (as seen in one of the other answers) to do the entire rotate in one step, but what if we don't know in advance how many steps we need to rotate?
A list really isn't the ideal data structure for this task. Lets say that instead we use a pair of lists, one for the head and one for the tail, then we can rotate easily by moving elements from one list to the other.
So, carefully avoiding calling anything from the standard library, we have:
rotate_right(List, N) ->
to_list(n_times(N, fun rotate_right/1, from_list(List))).
rotate_left(List, N) ->
to_list(n_times(N, fun rotate_left/1, from_list(List))).
from_list(Lst) ->
{Lst, []}.
to_list({Left, Right}) ->
Left ++ reverse(Right).
n_times(0, _, X) -> X;
n_times(N, F, X) -> n_times(N - 1, F, F(X)).
rotate_right({[], []}) ->
{[], []};
rotate_right({[H|T], Right}) ->
{T, [H|Right]};
rotate_right({[], Right}) ->
rotate_right({reverse(Right), []}).
rotate_left({[], []}) ->
{[], []};
rotate_left({Left, [H|T]}) ->
{[H|Left], T};
rotate_left({Left, []}) ->
rotate_left({[], reverse(Left)}).
reverse(Lst) ->
reverse(Lst, []).
reverse([], Acc) ->
Acc;
reverse([H|T], Acc) ->
reverse(T, [H|Acc]).
The module queue provides a data structure something like this. I've written this without reference to that though, so theirs is probably more clever.
First, your implementation is a bit buggy (try it with the empty list...)
Second, I would suggest you something like:
-module(foo).
-export([left/2, right/2]).
left(List, Times) ->
left(List, Times, []).
left([], Times, Acc) when Times > 0 ->
left(reverse(Acc), Times, []);
left(List, 0, Acc) ->
List ++ reverse(Acc);
left([H|T], Times, Acc) ->
left(T, Times-1, [H|Acc]).
right(List, Times) ->
reverse(foo:left(reverse(List), Times)).
reverse(List) ->
reverse(List, []).
reverse([], Acc) ->
Acc;
reverse([H|T], Acc) ->
reverse(T, [H|Acc]).
Third, for benchmarking your functions, you can do something like:
test(Params) ->
{Time1, _} = timer:tc(?MODULE, function1, Params),
{Time2, _} = timer:tc(?MODULE, function2, Params),
{{solution1, Time1}, {solution2, Time2}}.
I didn't test the code, so look at it critically, just get the idea.
Moreover, you might want to implement your own "reverse" function. It will be trivial by using tail recursion. Why not to try?
If you're trying to think in functional terms then perhaps consider implementing right rotate in terms of your left rotate:
rightrotate( List, 0 ) ->
List;
rightrotate( List, Times ) ->
lists:reverse( leftrotate( lists:reverse( List ), Times ) ).
Not saying this is the best idea or anything :)
Your implementation will not be efficient since the list is not the correct representation to use if you need to change item order, as in a rotation. (Imagine a round-robin scheduler with many thousands of jobs, taking the front job and placing it at the end when done.)
So we're actually just asking ourself what would be the way with least overhead to do this on lists anyway. But then what qualifies as overhead that we want to get rid of? One can often save a bit of computation by consing (allocating) more objects, or the other way around. One can also often have a larger than needed live-set during the computation and save allocation that way.
first_last([First|Tail]) ->
put_last(First, Tail).
put_last(Item, []) ->
[Item];
put_last(Item, [H|Tl]) ->
[H|put_last(Item,Tl)].
Ignoring corner cases with empty lists and such; The above code would cons the final resulting list directly. Very little garbage allocated. The final list is built as the stack unwinds. The cost is that we need more memory for the entire input list and the list in construction during this operation, but it is a short transient thing. My damage from Java and Lisp makes me reach for optimizing down excess consing, but in Erlang you dont risk that global full GC that kills every dream of real time properties. Anyway, I like the above approach generally.
last_first(List) ->
last_first(List, []).
last_first([Last], Rev) ->
[Last|lists:reverse(Rev)];
last_first([H|Tl], Rev) ->
last_first(Tl, [H|Rev]).
This approach uses a temporary list called Rev that is disposed of after we have passed it to lists:reverse/1 (it calls the BIF lists:reverse/2, but it is not doing anything interesting). By creating this temporary reversed list, we avoid having to traverse the list two times. Once for building a list containing everything but the last item, and one more time to get the last item.
One quick comment to your code. I would change the name of the function you call append. In a functional context append usually means adding a new list to the end of a list, not just one element. No sense in adding confusion.
As mentioned lists:split is not a BIF, it is a library function written in erlang. What a BIF really is is not properly defined.
The split or split like solutions look quite nice. As someone has already pointed out a list is not really the best data structure for this type of operation. Depends of course on what you are using it for.
Left:
lrl([], _N) ->
[];
lrl(List, N) ->
lrl2(List, List, [], 0, N).
% no more rotation needed, return head + rotated list reversed
lrl2(_List, Head, Tail, _Len, 0) ->
Head ++ lists:reverse(Tail);
% list is apparenly shorter than N, start again with N rem Len
lrl2(List, [], _Tail, Len, N) ->
lrl2(List, List, [], 0, N rem Len);
% rotate one
lrl2(List, [H|Head], Tail, Len, N) ->
lrl2(List, Head, [H|Tail], Len+1, N-1).
Right:
lrr([], _N) ->
[];
lrr(List, N) ->
L = erlang:length(List),
R = N rem L, % check if rotation is more than length
{H, T} = lists:split(L - R, List), % cut off the tail of the list
T ++ H. % swap tail and head