Related
This is an example of my current code:
DataSet = [1,2,3,4,5,6,7,8,9].
Sequence = [3,4,5,6].
ReducedDataSet = lists:foldl( fun(SeqNumber, Output) ->
Row = lists:nth(SeqNumber, DataSet),
[Row|Output]
end,
[],
Sequence
).
ReducedDataSet ends up as [6,5,4,3] and if I change it to lists:foldr, ReducedDataSet would be [3,4,5,6].
I didn't expect this as when absorbed left to right, the 3rd value is 3 and should proceed to 6, but when absorbed right to left, the 3rd value would be 7, and proceed to 4.
Does this mean there's a hidden row number on my list, and foldl and foldr only differ in the sort order of the final list?
I think this is a more general fold question.
In general, fold performs the following: (new_element, acc) -> new_acc
If the operation new_element ° acc is commutative (e.g. the sum), foldl and foldr are the same.
If the operation is "append" there is a difference between appending the element to the left or to the right.
[3] ° 4 -> [3, 4] VS 4 ° [3] -> [4, 3]
I never remember which is foldl and foldr but I think left/right refers to the position of the accumulator ([3] ° 4 is foldl with this definition)
TL;DR
No, there is no hidden index or "row number" in an Erlang list.
Discussion
It may be helpful to explore the nature of list operations a bit more in the context of functional lists of the "lists are a bunch of conses" variety.
I wrote an explanation of folds a while back that might be useful to you: Explanation of lists:fold function
Keep in mind that functional lists only have pointers that go one-way. That is, they are singly linked lists. There is no concept of a "rownum" or "index" as it would be in a C style array. Each call to lists:nth/2 is actually traversing the list to the nth element before returning that element.
We could write lists:nth/2 like this if we want a version that crashes on bad input (and, looking it up, it turns out that it is written almost exactly like this):
nth(1, [Element | _]) ->
Element;
nth(N, [_ | Rest]) when N > 1 ->
lists:nth(N - 1, Rest).
(As a side note, consider not inlining funs that require you to write multi-line definitions as function arguments...)
I am trying to turn a tuple of the form:
{{A,B,{C,A,{neg,A}}},{A,B,{neg,A}}}
Into
{{A,B,C,A,{neg,A}},{A,B,{neg,A}}
I'm quite new to Erlang so I would appreciate any hints. It makes no difference if the final structure is a list or a tuple, as long as any letter preceded by neg stays as a tuple/list.
A simple solution:
convert({{A,B,{C,D,E}},F}) -> {{A,B,C,D,E},F}.
If why this works is puzzling, consider:
1> YourTuple = {{a, b, {c, a, {neg, a}}}, {a, b, {neg, a}}}.
{{a,b,{c,a,{neg,a}}},{a,b,{neg,a}}}
2> Convert = fun({{A,B,{C,D,E}},F}) -> {{A,B,C,D,E},F} end.
#Fun<erl_eval.6.54118792>
3> Convert(YourTuple).
{{a,b,c,a,{neg,a}},{a,b,{neg,a}}}
The reason this happens is because we are matching over entire values based on the shape of the data. That's the whole point of matching, and also why its super useful in so many cases (and also why we want to use tuples in more specific circumstances in a language with matching VS a language where "everything is an iterable"). We can substitute the details with anything and they will be matched and returned accordingly:
4> MyTuple = {{"foo", bar, {<<"baz">>, balls, {ugh, "HURR!"}}}, {"Fee", "fi", "fo", "fum"}}.
{{"foo",bar,{<<"baz">>,balls,{ugh,"HURR!"}}},
{"Fee","fi","fo","fum"}}
5> Convert(MyTuple).
{{"foo",bar,<<"baz">>,balls,{ugh,"HURR!"}},
{"Fee","fi","fo","fum"}}
Why did this work when the last element of the top-level pair was so different in shape than the first one? Because everything about that second element was bound to the symbol F in the function represented by Convert (note that in the shell I named an anonymous function for convenience, this would be exactly the same as using convert/1 that I wrote at the top of this answer). We don't care what that second element was -- in fact we don't want to have to care about the details of that. The freedom to selectively not care about the shape of a given element of data is one of the key abstractions we use in Erlang.
"But those were just atoms 'a', 'b', 'c' etc. I have different things in there!"
Just to make it look superficially like your example above (and reinforce what I was saying about not caring about exactly what we bound to a given variable):
6> A = 1.
1
7> B = 2.
2
8> C = 3.
3
9> AnotherTuple = {{A, B, {C, A, {neg, A}}}, {A, B, {neg, A}}}.
{{1,2,{3,1,{neg,1}}},{1,2,{neg,1}}}
10> Convert(AnotherTuple).
{{1,2,3,1,{neg,1}},{1,2,{neg,1}}}
Needing to do this is not usually optimal, though. Generally speaking the other parts of the program that are producing that data in the first place should be returning useful data types for you. If not you can certainly hide them behind a conversion function such as the one above (especially when you're dealing with APIs that are out of your control), but generally speaking the need for this is a code smell.
And moving on
The more general case of "needing to flatten a tuple" is a bit different.
Tuples are tuples because each location within it has a meaning. So you don't usually hear of people needing to "flatten a tuple" because that fundamentally changes the meaning of the data you are dealing with. If you have this problem, you should not be using tuples to begin with.
That said, we can convert a tuple to a list, and we can check the shape of a data element. With these two operations in hand we could write a procedure that moves through a tuplish structure, building a list out of whatever it finds inside as it goes. A naive implementation might look like this:
-module(tuplish).
-export([flatten/1]).
-spec flatten(list() | tuple()) -> list().
flatten(Thing) ->
lists:flatten(flatten(Thing, [])).
flatten(Thing, A) when is_tuple(Thing) ->
flatten(tuple_to_list(Thing), A);
flatten([], A) ->
lists:reverse(A);
flatten([H | T], A) when is_tuple(H) ->
flatten(T, [flatten(H) | A]);
flatten([H | T], A) when is_list(H) ->
flatten(T, [flatten(H) | A]);
flatten([H | T], A) ->
flatten(T, [H | A]).
Keep in mind that after several years of writing Erlang code I have never needed to actually do this. Remember: tuples mean something different than lists.
All that said, the problem you are facing is almost certainly handled better by using records.
In the Programming Erlang book, there is some example pseudo code that shows a pattern for efficiently adding elements to the head of a list:
some_function([H|T], ..., Result, ...) ->
H1 = ... H ...,
some_function(T, ..., [H1|Result], ...);
some_function([H|T], ..., Result, ...) ->
{..., Result, ...}.
I'm still getting used to functional programming so the above example is a little too abstract for me to understand at the moment.
I think it will be easier to understand if there is a concrete implementation of the pattern that I could dissect.
Question: Is there a simple concrete implementation of this pattern that someone can provide?
Let's say that we want a function which behaves a like the uniq command.
The function takes a list of elements and returns a list with all consecutive occurrences of an element substituted with a single occurrence of that element.
One of the possible approaches is presented below:
uniq(L) ->
uniq(L, []).
uniq([], Acc) ->
lists:reverse(Acc);
uniq([H, H | T], Acc) ->
uniq([H | T], Acc);
uniq([H | T], Acc) ->
uniq(T, [H | Acc]).
We build up an accumulator, by inserting new elements at the head of the Acc list (cheapest insertion cost) and once we're done, we reverse the whole list to get the initial order of elements back.
We "visit" some of the elements of the initial list twice, but the total cost is still linear, i.e. only dependent on the number of elements of the initial list.
This takes a factorized list, i.e.
[[],[2],[3],[2,2],[5],[2,3],[7],[2,2,2],etc...]
and removes all the primes.
remove_primes([HD|TL], Results) ->
case length(HD) of
0 -> % You're at 1
remove_primes (TL , Results);
1 -> % Its a prime, remove it, and keep going
remove_primes( TL , Results) ;
_ -> % its not prime, leave it in and keep going.
remove_primes(TL, [ HD | Results])
end;
remove_primes([], Result) ->
{Result}.
The structure Joe Armstrong was alluding too, is the standard structure of walking a list and applying a function to each element on the list. In this case, I desired to treat each element differently depending on its contents.
In practice, it is much easier to to use maps, filters and such, so I believe you will see that much more often - but as you seem to know, understanding the basics is vital to becoming a proficient functional programmer.
In hopes centralize information pertaining to 'building lists in natural order', does anyone know why pattern matching at the function level, works, 'but unpacking' a variable does not? (compare this)(it does not work)
remove_primes(Factorized_List, Results) ->
[HD|TL] = Factorized_List, % unpack the list <-------------
case length(HD) of
0 -> % You're at 1
remove_primes (TL , Results);
1 -> % Its a prime, remove it, and keep going
remove_primes( TL , Results) ;
_ -> % its not prime, leave it in and keep going.
remove_primes(TL, [HD|Results])
end;
remove_primes([], Result) ->
{Result}.
I believe this leads to more readable code, but it does not seem to work.
-rC
Here is the only way I can get your pattern to execute:
some_func([H|T], 4, Result, 4) ->
H1 = H * 2,
some_func(T, 3, [H1|Result], 4);
some_func([H|T], 3, Result, _) ->
{H, Result, T}.
--output:--
25> a:some_func([1, 2, 3], 4, [], 4).
{2,[2],[3]}
...which does nothing useful.
The pattern in the pseudo code makes no sense to me, so I'll join you in your confusion.
Here is another attempt:
some_func([H|T], [_|T2], Result, Y) ->
H1 = H * Y,
some_func(T, T2, [H1|Result], Y);
some_func([H|T], [], Result, _) ->
{H, Result, T}.
--output:--
34> a:some_func([1, 2, 3, 4], [one, two, three], [], 2).
{4,[6,4,2],[]}
I want to write a function to replace a specific atom with the given atom in an input list. But I want to do it using pattern matching and not using conditional statements. Any idea?
And also I want to write a function to return unique atoms in an expression.
e.g.
Input:
[a, b, c, a, b]
Output:
c
Input:
[b, b, b, r, t, y, y]
Output:
[t, r]
Assuming you want to replace all instances and keep the order of the list (works with all terms):
replace(Old, New, List) -> replace(Old, New, List, []).
replace(_Old, _New, [], Acc) -> lists:reverse(Acc);
replace(Old, New, [Old|List], Acc) -> replace(Old, New, List, [New|Acc]);
replace(Old, New, [Other|List], Acc) -> replace(Old, New, List, [Other|Acc]).
For the unique elements filter, you need to keep a state of which elements you have looked at already.
It would be really awkward to implement such a function using only pattern matching in the function headers and you would not really gain anything (performance) from it. The awkwardness would come from having to loop through both the list in question and the list(s) keeping your state of already parsed elements. You would also loose a lot of readability.
I would recommend going for something simpler (works with all terms, not just atoms):
unique(List) -> unique(List, []).
unique([], Counts) ->
lists:foldl(fun({E, 1}, Acc) -> [E|Acc];
(_, Acc) -> Acc
end, [], Counts);
unique([E|List], Counts) ->
unique(List, count(E, Counts).
count(E, []) -> [{E, 1}];
count(E, [{E, N}|Rest]) -> [{E, N + 1}|Rest];
count(E, [{X, N}|Rest]) -> [{X, N}|count(E, Rest)].
One way I'm looking for solving your first question would be to use guards, instead of if statements. Using only pattern matching doesn't seem possible (or desirable, even if you can do it).
So, for instance, you could do something like:
my_replace([H|T], ToReplace, Replacement, Accum) when H == ToReplace ->
my_replace(T, ToReplace, Replacement, [Replacement|Accum]);
my_replace([H|T], ToReplace, Replacement, Accum) ->
my_replace(T, ToReplace, Replacement, [H|Accum]);
my_replace([], ToReplace, Replacement, Accum) ->
lists:reverse(Accum).
EDIT: Edited for simplicity and style, thanks for the comments. :)
For the second part of your question, what do you consider an "expression"?
EDIT: Nevermind that, usort doesn't completely remove duplicates, sorry.
I am getting myself familiar to Sequential Erlang (and the functional programming thinking) now. So I want to implement the following two functionality without the help of BIF. One is left_rotate (which I have come up with the solution) and the other is right_rotate (which I am asking here)
-export(leftrotate/1, rightrotate/1).
%%(1) left rotate a lits
leftrotate(List, 0) ->
List;
leftrotate([Head | Tail], Times) ->
List = append(Tail, Head),
leftrotate(List, Times -1).
append([], Elem)->
[Elem];
append([H|T], Elem) ->
[H | append(T, Elem)].
%%right rotate a list, how?
%%
I don't want to use BIF in this exercise. How can I achieve the right rotation?
A related question and slightly more important question. How can I know one of my implementation is efficient or not (i.e., avoid unnecessary recursion if I implement the same thing with the help of a BIF, and etc.)
I think BIF is built to provide some functions to improve efficiency that functional programming is not good at (or if we do them in a 'functional way', the performance is not optimal).
The efficiency problem you mention has nothing to do with excessive recursion (function calls are cheap), and everything to do with walking and rebuilding the list. Every time you add something to the end of a list you have to walk and copy the entire list, as is obvious from your implementation of append. So, to rotate a list N steps requires us to copy the entire list out N times. We can use lists:split (as seen in one of the other answers) to do the entire rotate in one step, but what if we don't know in advance how many steps we need to rotate?
A list really isn't the ideal data structure for this task. Lets say that instead we use a pair of lists, one for the head and one for the tail, then we can rotate easily by moving elements from one list to the other.
So, carefully avoiding calling anything from the standard library, we have:
rotate_right(List, N) ->
to_list(n_times(N, fun rotate_right/1, from_list(List))).
rotate_left(List, N) ->
to_list(n_times(N, fun rotate_left/1, from_list(List))).
from_list(Lst) ->
{Lst, []}.
to_list({Left, Right}) ->
Left ++ reverse(Right).
n_times(0, _, X) -> X;
n_times(N, F, X) -> n_times(N - 1, F, F(X)).
rotate_right({[], []}) ->
{[], []};
rotate_right({[H|T], Right}) ->
{T, [H|Right]};
rotate_right({[], Right}) ->
rotate_right({reverse(Right), []}).
rotate_left({[], []}) ->
{[], []};
rotate_left({Left, [H|T]}) ->
{[H|Left], T};
rotate_left({Left, []}) ->
rotate_left({[], reverse(Left)}).
reverse(Lst) ->
reverse(Lst, []).
reverse([], Acc) ->
Acc;
reverse([H|T], Acc) ->
reverse(T, [H|Acc]).
The module queue provides a data structure something like this. I've written this without reference to that though, so theirs is probably more clever.
First, your implementation is a bit buggy (try it with the empty list...)
Second, I would suggest you something like:
-module(foo).
-export([left/2, right/2]).
left(List, Times) ->
left(List, Times, []).
left([], Times, Acc) when Times > 0 ->
left(reverse(Acc), Times, []);
left(List, 0, Acc) ->
List ++ reverse(Acc);
left([H|T], Times, Acc) ->
left(T, Times-1, [H|Acc]).
right(List, Times) ->
reverse(foo:left(reverse(List), Times)).
reverse(List) ->
reverse(List, []).
reverse([], Acc) ->
Acc;
reverse([H|T], Acc) ->
reverse(T, [H|Acc]).
Third, for benchmarking your functions, you can do something like:
test(Params) ->
{Time1, _} = timer:tc(?MODULE, function1, Params),
{Time2, _} = timer:tc(?MODULE, function2, Params),
{{solution1, Time1}, {solution2, Time2}}.
I didn't test the code, so look at it critically, just get the idea.
Moreover, you might want to implement your own "reverse" function. It will be trivial by using tail recursion. Why not to try?
If you're trying to think in functional terms then perhaps consider implementing right rotate in terms of your left rotate:
rightrotate( List, 0 ) ->
List;
rightrotate( List, Times ) ->
lists:reverse( leftrotate( lists:reverse( List ), Times ) ).
Not saying this is the best idea or anything :)
Your implementation will not be efficient since the list is not the correct representation to use if you need to change item order, as in a rotation. (Imagine a round-robin scheduler with many thousands of jobs, taking the front job and placing it at the end when done.)
So we're actually just asking ourself what would be the way with least overhead to do this on lists anyway. But then what qualifies as overhead that we want to get rid of? One can often save a bit of computation by consing (allocating) more objects, or the other way around. One can also often have a larger than needed live-set during the computation and save allocation that way.
first_last([First|Tail]) ->
put_last(First, Tail).
put_last(Item, []) ->
[Item];
put_last(Item, [H|Tl]) ->
[H|put_last(Item,Tl)].
Ignoring corner cases with empty lists and such; The above code would cons the final resulting list directly. Very little garbage allocated. The final list is built as the stack unwinds. The cost is that we need more memory for the entire input list and the list in construction during this operation, but it is a short transient thing. My damage from Java and Lisp makes me reach for optimizing down excess consing, but in Erlang you dont risk that global full GC that kills every dream of real time properties. Anyway, I like the above approach generally.
last_first(List) ->
last_first(List, []).
last_first([Last], Rev) ->
[Last|lists:reverse(Rev)];
last_first([H|Tl], Rev) ->
last_first(Tl, [H|Rev]).
This approach uses a temporary list called Rev that is disposed of after we have passed it to lists:reverse/1 (it calls the BIF lists:reverse/2, but it is not doing anything interesting). By creating this temporary reversed list, we avoid having to traverse the list two times. Once for building a list containing everything but the last item, and one more time to get the last item.
One quick comment to your code. I would change the name of the function you call append. In a functional context append usually means adding a new list to the end of a list, not just one element. No sense in adding confusion.
As mentioned lists:split is not a BIF, it is a library function written in erlang. What a BIF really is is not properly defined.
The split or split like solutions look quite nice. As someone has already pointed out a list is not really the best data structure for this type of operation. Depends of course on what you are using it for.
Left:
lrl([], _N) ->
[];
lrl(List, N) ->
lrl2(List, List, [], 0, N).
% no more rotation needed, return head + rotated list reversed
lrl2(_List, Head, Tail, _Len, 0) ->
Head ++ lists:reverse(Tail);
% list is apparenly shorter than N, start again with N rem Len
lrl2(List, [], _Tail, Len, N) ->
lrl2(List, List, [], 0, N rem Len);
% rotate one
lrl2(List, [H|Head], Tail, Len, N) ->
lrl2(List, Head, [H|Tail], Len+1, N-1).
Right:
lrr([], _N) ->
[];
lrr(List, N) ->
L = erlang:length(List),
R = N rem L, % check if rotation is more than length
{H, T} = lists:split(L - R, List), % cut off the tail of the list
T ++ H. % swap tail and head