Related
I accidentally did (the equivalent of) the following:
lists:foldl(fun(X, Acc) -> [X|Acc] end, 0, List).
Note the not-a-list initial value for the accumulator.
This resulted in an improper list. This means that length, etc., don't work on it.
Given that my "equivalent of" took an hour to run, and I don't want to run it again, how do I repair my improper list?
For a simpler example of an improper list and the problem that it causes:
1> L = [1|[2|[3|4]]].
[1,2,3|4]
2> length(L).
** exception error: bad argument
in function length/1
called as length([1,2,3|4])
If you want to preserve the "improper tail", this would be enough:
Fix = fun Fix([H | T]) -> [H | Fix(T)];
Fix(T) -> [T]
end.
Here is a possible approach:
Lister = fun L([], Acc) -> lists:reverse(Acc);
L([[_ | _] = H | T], Acc) -> L(T, [L(H, []) | Acc]);
L([[] | T], Acc) -> L(T, Acc);
L([H | T], Acc) -> L(T, [H | Acc]);
L(X, Acc) -> L([], [X | Acc])
end.
L = [[[1,[1|2]],1|2],1|[2|[3|4]]].
Lister(L, []).
% output [[[1,[1,2]],1,2],1,2,3,4]
For the simple case I had, with non-nested improper list, where I don't want the extra item (because it should have been an empty list and doesn't mean anything), this'll do it:
Fix = fun F([H|T], A) when is_list(T) -> F(T, [H|A]);
F([H|_], A) -> F([], [H|A]);
F([], A) -> lists:reverse(A)
end.
Fix(L, []).
you must use list for ACC
lists:foldl(fun(X, Acc) -> [X|Acc] end, 0, [1,2,3]).
result => [3,2,1|0]
but if you use [0] for ACC argument in lists:foldl/3 function like bellow
lists:foldl(fun(X, Acc) -> [X|Acc] end, [0], [1,2,3]).
result => [3,2,1,0]
The title^ is kinda confusing but I will illustrate what I want to achieve:
I have:
[{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538077790705827">>},
{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538078530667847">>},
{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538077778390908">>},
{<<"5b71d7e458c37fa04a7ce768">>,<<"5bad45b1e990057961313822">>,<<"1538082492283531">>
}]
I want to convert it to a list like this:
[
{<<"5b3f77502dfe0deeb8912b42">>,
[{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538077790705827">>},
{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538078530667847">>},
{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538077778390908">>}
]},
{<<"5bad45b1e990057961313822">>,
[{<<"5b71d7e458c37fa04a7ce768">>,<<"5bad45b1e990057961313822">>,<<"1538082492283531">>}
]}
]
List of tuples [{id, [<List>]}, {id2, [<List>]} ] where ids are the second item of the tuple of the original list
Example :
<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538077790705827">>
Erlang newbie here. I created a dict with the second members of the tuples as keys and lists of corresponding tuples as values, then used dict:fold to transform it into the expected output format.
-export([test/0, transform/1]).
transform([H|T]) ->
transform([H|T], dict:new()).
transform([], D) ->
lists:reverse(
dict:fold(fun (Key, Tuples, Acc) ->
lists:append(Acc,[{Key,Tuples}])
end,
[],
D));
transform([Tuple={_S1,S2,_S3}|T], D) ->
transform(T, dict:append_list(S2, [Tuple], D)).
test() ->
Input=[{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538077790705827">>},
{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538078530667847">>},
{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538077778390908">>},
{<<"5b71d7e458c37fa04a7ce768">>,<<"5bad45b1e990057961313822">>,<<"1538082492283531">>}
],
Output=transform(Input),
case Output of
[
{<<"5b3f77502dfe0deeb8912b42">>,
[{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538077790705827">>},
{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538078530667847">>},
{<<"5b71d7e458c37fa04a7ce768">>,<<"5b3f77502dfe0deeb8912b42">>,<<"1538077778390908">>}
]},
{<<"5bad45b1e990057961313822">>,
[{<<"5b71d7e458c37fa04a7ce768">>,<<"5bad45b1e990057961313822">>,<<"1538082492283531">>}
]}
] -> ok;
_Else -> error
end.
I think I see what you're after... Please correct me if I'm wrong.
There are a number of ways to do this, it really just depends on what sort of data structure you're interested in using to check the presence of like-keys. I'll show you two fundamentally different ways to do this and a third hybrid method that has become recently available:
Indexed data types (in this case a map)
List operations with matching
Hybrid matching over map keys
Since you're new I'll use the first case to demonstrate two ways of writing it: explicit recursion and using an actual list function from the lists module.
Indexy Data Types
The first way we'll do this is to use a hash table (aka "dict", "map", "hash", "K/V", etc.) and explicitly recurse through the elements, checking for the presence of the key encountered and adding it if it is missing, or appending to the list of values it points to if it does. We'll use an Erlang map for this. At the end of the function we'll convert the utility map back to a list:
explicit_convert(List) ->
Map = explicit_convert(List, maps:new()),
maps:to_list(Map).
explicit_convert([H | T], A) ->
K = element(2, H),
NewA =
case maps:is_key(K, A) of
true ->
V = maps:get(K, A),
maps:put(K, [H | V], A);
false ->
maps:put(K, [H], A)
end,
explicit_convert(T, NewA);
explicit_convert([], A) ->
A.
There is nothing wrong with explicit recursion (it is particularly good if you're new, because every part of it is left in the open to be examined), but this is a "left fold" and we already have a library function that abstracts a little bit of the plumbing out. So we really only need to write a function that checks for the presence of an element, and adds the key or appends the value:
fun_convert(List) ->
Map = lists:foldl(fun convert/2, maps:new(), List),
maps:to_list(Map).
convert(H, A) ->
K = element(2, H),
case maps:is_key(K, A) of
true ->
V = maps:get(K, A),
maps:put(K, [H | V], A);
false ->
maps:put(K, [H], A)
end.
Listy Conversion
The other major way we could have done this is with listy matching. To do that you need to first guarantee that your elements are sorted on the element you want to use as a key so that you can use it as a sort of "working element" and match on it. The code should be pretty easy to understand once you stare at it for a bit (maybe write out how it will step through your list by hand on paper once if you're totally perplexed):
listy_convert(List) ->
[T = {_, K, _} | Rest] = lists:keysort(2, List),
listy_convert(Rest, {K, [T]}, []).
listy_convert([T = {_, K, _} | Rest], {K, Ts}, Acc) ->
listy_convert(Rest, {K, [T | Ts]}, Acc);
listy_convert([T = {_, K, _} | Rest], Done, Acc) ->
listy_convert(Rest, {K, [T]}, [Done | Acc]);
listy_convert([], Done, Acc) ->
[Done | Acc].
Note that we split the list immediately after sorting it. The reason is that we have "prime the pump", so to speak, on the first call we make to listy_convert/3. This also means that this function will crash if you pass it an empty list. You can solve that by adding a clause to listy_convert/1 that matches on the empty list [].
A Final Bit of Magic
With those firmly in mind... consider that we also have a bit of a hybrid option available in newer versions of Erlang due to the magical syntax available to maps. We can match (most values) on map keys inside of a case clause (though we can't unify on a key value provided by other arguments within a function head):
map_convert(List) ->
maps:to_list(map_convert(List, #{})).
map_convert([T = {_, K, _} | Rest], Acc) ->
case Acc of
#{K := Ts} -> map_convert(Rest, Acc#{K := [T | Ts]});
_ -> map_convert(Rest, Acc#{K => [T]})
end;
map_convert([], Acc) ->
Acc.
Here is a one-liner that would produce your expected result:
[{K, [E || {_, K2, _} = E <- List, K =:= K2]} || {_, K, _} <- lists:ukeysort(2, List)].
What’s going on here? Let’s do it step by step…
This is your original list
List = […],
lists:ukeysort/2 leaves just one element per key in the list
OnePerKey = lists:ukeysort(2, List),
We then extract the keys with the first list comprehension
Keys = [K || {_, K, _} <- OnePerKey],
With the second list comprehension, we find the elements with the key…
fun Filter(K, List) ->
[E || {_, K2, _} = E <- List, K =:= K2]
end
Keep in mind that we can’t just pattern-match with K in the generator (i.e. [E || {_, K, _} = E <- List]) because generators in LCs introduce new scope for the variables.
Finally, putting all together…
[{K, Filter(K, List)} || K <- Keys]
It really depends on your dataset. For lager data sets using maps is a bit more efficient.
-module(test).
-export([test/3, v1/2, v2/2, v3/2, transform/1, do/2]).
test(N, Keys, Size) ->
List = [{<<"5b71d7e458c37fa04a7ce768">>,rand:uniform(Keys),<<"1538077790705827">>} || I <- lists:seq(1,Size)],
V1 = timer:tc(test, v1, [N, List]),
V2 = timer:tc(test, v2, [N, List]),
V3 = timer:tc(test, v3, [N, List]),
io:format("V1 took: ~p, V2 took: ~p V3 took: ~p ~n", [V1, V2, V3]).
v1(N, List) when N > 0 ->
[{K, [E || {_, K2, _} = E <- List, K =:= K2]} || {_, K, _} <- lists:ukeysort(2, List)],
v1(N-1, List);
v1(_,_) -> ok.
v2(N, List) when N > 0 ->
do(List,maps:new()),
v2(N-1, List);
v2(_,_) -> ok.
v3(N, List) when N > 0 ->
transform(List),
v3(N-1, List);
v3(_,_) -> ok.
do([], R) -> maps:to_list(R);
do([H={_,K,_}|T], R) ->
case maps:get(K,R,null) of
null -> NewR = maps:put(K, [H], R);
V -> NewR = maps:update(K, [H|V], R)
end,
do(T, NewR).
transform([H|T]) ->
transform([H|T], dict:new()).
transform([], D) ->
lists:reverse(
dict:fold(fun (Key, Tuples, Acc) ->
lists:append(Acc,[{Key,Tuples}])
end,
[],
D));
transform([Tuple={_S1,S2,_S3}|T], D) ->
transform(T, dict:append_list(S2, [Tuple], D)).
Running both with 100 unique keys and 100,000 records I get:
> test:test(1,100,100000).
V1 took: {75566,ok}, V2 took: {32087,ok} V3 took: {887362,ok}
ok
What's the Erlang equivalent to the following Python code:
for x in range(9):
for y in range(9):
for z in range(9):
foo(x, y, z)
I know I can generate the product first with C = [{X,Y,Z} || X<- lists:seq(1,9), Y<- lists:seq(1,9), Z<- lists:seq(1,9)] then foo([])->done; foo([H|T])->blah blah.
How do I do it without an auxiliary list, using recursion only?
You could do it with three recursive functions.
You might be able to do it with some complex pattern-matching in function head.
But easiest way to skip creation of auxiliary list is to call your function inside list comprehension
C = [foo(X, Y, Z) || X<- lists:seq(1,9),
Y<- lists:seq(1,9),
Z<- lists:seq(1,9)]
Where foo/3 process one element.
List comprehension still forces you to create auxiliary lists in memory.
In case of dealing with huge data sets you should avoid it. Writing recursive functions every time is also awkward so i came up with my own generic for function. It's a little bit slower in traversing than direct recursion or list comprehension but it's memory stable, generic and easy to use.
Usage:
(for({10}))(
fun (X) -> io:format("~p ",[X]) end).
> 1 2 3 4 5 6 7 8 9 10
(for({10, -10, -2}))(
fun (X) -> io:format("~p ",[X]) end).
> 10 8 6 4 2 0 -2 -4 -6 -8 -10
Works with lists too:
(for(lists:seq(10, -10, -2)))(
fun (X) -> io:format("~p ",[X]) end).
> 10 8 6 4 2 0 -2 -4 -6 -8 -10
It's also possible to define step or guard as a function:
(for({256, 1.1, fun (X) -> math:sqrt(X) end, fun (X, Range) -> X > Range end}))(
fun (X) -> io:format("~p ",[X]) end).
> 256 16.0 4.0 2.0 1.4142135623730951 1.189207115002721
If you pass to for a two parameter function, then you can use accumulator feature just like with lists:foldl/3. You also need to pass initial accumulator to for:
Fact = (for(1, {1, 5}))(
fun(X, Acc) ->
X * Acc
end),
io:format("~p", [Fact]).
> 120
e_fact(N) ->
{_, E} = (for({1, 1}, {1, N}))( % i assumed 1/0! equals 1
fun(X, {LastFact, Sum}) ->
Fact = LastFact * X,
{Fact, Sum + 1 / Fact}
end),
E.
io:format("e=~p", [e_fact(10)]).
> e=2.7182818011463845
Also step and guard functions can be dependent on accumulator. Just pass function with one more parameter.
Nested loops finding Pythagorean triples. Easy with closures:
pyth_lists(N) ->
[io:format("~p ", [{A, B, C}]) ||
A <- lists:seq(1, N),
B <- lists:seq(A + 1, N),
C <- lists:seq(B + 1, N),
A * A + B * B == C * C].
pyth_for(N) ->
(for({1, N}))(
fun(A) ->
(for({A + 1, N}))(
fun(B) ->
(for({B + 1, N}))(
fun(C) ->
case A * A + B * B == C * C of
true -> io:format("~p ", [{A, B, C}]);
false -> ok
end
end)
end)
end).
It's too small for external repository. I keep it in my utilities module.
If you find it helpful, here is code:
-export([for/1, for/2]).
for(Through) ->
for([], Through).
for(InitAcc, Opts) when is_tuple(Opts) ->
{Init, Range, Step, Guard} = for_apply_default_opts(Opts),
fun(Fun) ->
UpdFun = if
is_function(Fun, 1) ->
fun(I, _FAcc) -> Fun(I) end;
is_function(Fun, 2) ->
Fun
end,
for_iter(UpdFun, InitAcc, Init, Range, Step, Guard) end;
for(InitAcc, List) when is_list(List) ->
fun(Fun) -> for_list_eval(Fun, InitAcc, List) end.
for_iter(Fun, Acc, I, Range, Step, Guard) ->
case Guard(I, Range, Acc) of
false ->
Acc;
true ->
NewAcc = Fun(I, Acc),
for_iter(Fun, NewAcc, Step(I, NewAcc), Range, Step, Guard)
end.
for_list_eval(Fun, Acc, List) ->
if
is_function(Fun, 1) ->
lists:foreach(Fun, List);
is_function(Fun, 2) ->
lists:foldl(Fun, Acc, List)
end.
for_apply_default_opts({Range}) ->
DefaultInit = 1,
for_apply_default_opts({DefaultInit, Range});
for_apply_default_opts({Init, Range}) ->
DefaultStep = 1,
for_apply_default_opts({Init, Range, DefaultStep});
for_apply_default_opts({Init, Range, Step}) ->
DefaultGuard = case (Step > 0) or is_function(Step) of
true -> fun(I, IterRange, _Acc) -> I =< IterRange end;
false -> fun(I, IterRange, _Acc) -> I >= IterRange end
end,
for_apply_default_opts({Init, Range, Step, DefaultGuard});
for_apply_default_opts({Init, Range, Step, Guard}) when is_function(Guard, 2) ->
for_apply_default_opts({Init, Range, Step, fun(I, IterRange, _Acc) -> Guard(I, IterRange) end});
for_apply_default_opts({Init, Range, Step, DefaultGuard}) when is_number(Step) ->
for_apply_default_opts({Init, Range, fun(I, _Acc) -> I + Step end, DefaultGuard});
for_apply_default_opts({Init, Range, Step, DefaultGuard}) when is_function(Step, 1) ->
for_apply_default_opts({Init, Range, fun(I, _Acc) -> Step(I) end, DefaultGuard});
for_apply_default_opts({_Init, _Range, _Step, _DefaultGuard} = Opts) ->
Opts.
I want to split:
[1,2,3,4,5,6,7,8]
into:
[[1,2],[3,4],[5,6],[7,8]]
It generally works great with:
[ lists:sublist(List, X, 2) || X <- lists:seq(1,length(List),2) ] .
But it is really slow this way. 10000 Elements take amazing 2.5 seconds on my netbook. I have also written a really fast recursive function, but I am simply interested: Could this list comprehension also be written in a different way, so that it is faster?
Try this:
part(List) ->
part(List, []).
part([], Acc) ->
lists:reverse(Acc);
part([H], Acc) ->
lists:reverse([[H]|Acc]);
part([H1,H2|T], Acc) ->
part(T, [[H1,H2]|Acc]).
Test in erlang-shell (I've declared this function in module part):
2> part:part([1,2,3,4,5,6,7,8]).
[[1,2],[3,4],[5,6],[7,8]]
3>
3> timer:tc(part, part, [lists:seq(1,10000)]).
{774,
[[1,2],
[3,4],
[5,6],
[7,8],
"\t\n","\v\f",
[13,14],
[15,16],
[17,18],
[19,20],
[21,22],
[23,24],
[25,26],
[27,28],
[29,30],
[31,32],
"!\"","#$","%&","'(",")*","+,","-.","/0","12","34",
[...]|...]}
Just 774 microseconds (which is ~0,8 milliseconds)
Here are two quick solutions for you that are both flexible. One is easy to read, but only slightly faster than your proposed solution. The other is quite fast, but is a bit cryptic to read. And note that both of my proposed algorithms will work for lists of anything, not just numeric ordered lists.
Here is the "easy-to-read" one. Call by n_length_chunks(List,Chunksize). For example, to get a list of chunks 2 long, call n_length_chunks(List,2). This works for chunks of any size, ie, you could call n_length_chunks(List,4) to get [[1,2,3,4],[5,6,7,8],...]
n_length_chunks([],_) -> [];
n_length_chunks(List,Len) when Len > length(List) ->
[List];
n_length_chunks(List,Len) ->
{Head,Tail} = lists:split(Len,List),
[Head | n_length_chunks(Tail,Len)].
The much faster one is here, but is definitely harder to read, and is called in the same way: n_length_chunks_fast(List,2) (I've made one change to this compared with the one above, in that it pads the end of the list with undefined if the length of the list isn't cleanly divisible by the desired chunk length.
n_length_chunks_fast(List,Len) ->
LeaderLength = case length(List) rem Len of
0 -> 0;
N -> Len - N
end,
Leader = lists:duplicate(LeaderLength,undefined),
n_length_chunks_fast(Leader ++ lists:reverse(List),[],0,Len).
n_length_chunks_fast([],Acc,_,_) -> Acc;
n_length_chunks_fast([H|T],Acc,Pos,Max) when Pos==Max ->
n_length_chunks_fast(T,[[H] | Acc],1,Max);
n_length_chunks_fast([H|T],[HAcc | TAcc],Pos,Max) ->
n_length_chunks_fast(T,[[H | HAcc] | TAcc],Pos+1,Max);
n_length_chunks_fast([H|T],[],Pos,Max) ->
n_length_chunks_fast(T,[[H]],Pos+1,Max).
Tested on my (really old) laptop:
Your proposed solution took about 3 seconds.
My slow-but-readable one was slightly faster and takes about 1.5 seconds (still quite slow)
My fast version takes about 5 milliseconds.
For completeness, Isac's solution took about 180 milliseconds on my same machine.
Edit: wow, I need to read the complete question first. Oh well I'll keep here for posterity if it helps. As far as I can tell, there's not a good way to do this using list comprehensions. Your original version is slow because each iteration of sublist needs to traverse the list each time to get to each successive X, resulting in complexity just under O(N^2).
Or with a fold:
lists:foldr(fun(E, []) -> [[E]];
(E, [H|RAcc]) when length(H) < 2 -> [[E|H]|RAcc] ;
(E, [H|RAcc]) -> [[E],H|RAcc]
end, [], List).
I want to submit slightly complicated but more flexible (and mostly faster) solution of one proposed by #Tilman
split_list(List, Max) ->
element(1, lists:foldl(fun
(E, {[Buff|Acc], C}) when C < Max ->
{[[E|Buff]|Acc], C+1};
(E, {[Buff|Acc], _}) ->
{[[E],Buff|Acc], 1};
(E, {[], _}) ->
{[[E]], 1}
end, {[], 0}, List)).
so function part can be implemented as
part(List) ->
RevList = split_list(List, 2),
lists:foldl(fun(E, Acc) ->
[lists:reverse(E)|Acc]
end, [], RevList).
update
I've added reverse in case if you want to preserve order, but as I can see it adds no more than 20% of processing time.
You could do it like this:
1> {List1, List2} = lists:partition(fun(X) -> (X rem 2) == 1 end, List).
{[1,3,5|...],[2,4,6|...]}
2> lists:zipwith(fun(X, Y) -> [X, Y] end, List1, List2).
[[1,2],[3,4],[5,6]|...]
This takes ~73 milliseconds with a 10000 elements List on my computer. The original solution takes ~900 miliseconds.
But I would go with the recursive function anyway.
I was looking for a partition function which can split a large list to small amount of workers. With lkuty's partition you might get that one worker gets almost double work than all the others. If that's not what you want, here is a version which sublist lengths differ by at most 1.
Uses PropEr for testing.
%% #doc Split List into sub-lists so sub-lists lengths differ most by 1.
%% Does not preserve order.
-spec split_many(pos_integer(), [T]) -> [[T]] when T :: term().
split_many(N, List) ->
PieceLen = length(List) div N,
lists:reverse(split_many(PieceLen, N, List, [])).
-spec split_many(pos_integer(), pos_integer(), [T], [[T]]) ->
[[T]] when T :: term().
split_many(PieceLen, N, List, Acc) when length(Acc) < N ->
{Head, Tail} = lists:split(PieceLen, List),
split_many(PieceLen, N, Tail, [Head|Acc]);
split_many(_PieceLen, _N, List, Acc) ->
% Add an Elem to each list in Acc
{Appendable, LeaveAlone} = lists:split(length(List), Acc),
Appended = [[Elem|XS] || {Elem, XS} <- lists:zip(List, Appendable)],
lists:append(Appended, LeaveAlone).
Tests:
split_many_test_() ->
[
?_assertEqual([[1,2]], elibs_lists:split_many(1, [1,2])),
?_assertEqual([[1], [2]], elibs_lists:split_many(2, [1,2])),
?_assertEqual([[1], [3,2]], elibs_lists:split_many(2, [1,2,3])),
?_assertEqual([[1], [2], [4,3]], elibs_lists:split_many(3, [1,2,3,4])),
?_assertEqual([[1,2], [5,3,4]], elibs_lists:split_many(2, [1,2,3,4,5])),
?_assert(proper:quickcheck(split_many_proper1())),
?_assert(proper:quickcheck(split_many_proper2()))
].
%% #doc Verify all elements are preserved, number of groups is correct,
%% all groups have same number of elements (+-1)
split_many_proper1() ->
?FORALL({List, Groups},
{list(), pos_integer()},
begin
Split = elibs_lists:split_many(Groups, List),
% Lengths of sub-lists
Lengths = lists:usort(lists:map(fun erlang:length/1, Split)),
length(Split) =:= Groups andalso
lists:sort(lists:append(Split)) == lists:sort(List) andalso
length(Lengths) =< 2 andalso
case Lengths of
[Min, Max] -> Max == Min + 1;
[_] -> true
end
end
).
%% #doc If number of groups is divisable by number of elements, ordering must
%% stay the same
split_many_proper2() ->
?FORALL({Groups, List},
?LET({A, B},
{integer(1, 20), integer(1, 10)},
{A, vector(A*B, term())}),
List =:= lists:append(elibs_lists:split_many(Groups, List))
).
Here is a more general answer that works with any sublist size.
1> lists:foreach(fun(N) -> io:format("~2.10.0B -> ~w~n",[N, test:partition([1,2,3,4,5,6,7,8,9,10],N)] ) end, [1,2,3,4,5,6,7,8,9,10]).
01 -> [[1],[2],[3],[4],[5],[6],[7],[8],[9],[10]]
02 -> [[1,2],[3,4],[5,6],[7,8],[9,10]]
03 -> [[1,2,3],[4,5,6],[7,8,9],[10]]
04 -> [[1,2,3,4],[5,6,7,8],[10,9]]
05 -> [[1,2,3,4,5],[6,7,8,9,10]]
06 -> [[1,2,3,4,5,6],[10,9,8,7]]
07 -> [[1,2,3,4,5,6,7],[10,9,8]]
08 -> [[1,2,3,4,5,6,7,8],[10,9]]
09 -> [[1,2,3,4,5,6,7,8,9],[10]]
10 -> [[1,2,3,4,5,6,7,8,9,10]]
And the code to achieve this is stored inside a file called test.erl:
-module(test).
-compile(export_all).
partition(List, N) ->
partition(List, 1, N, []).
partition([], _C, _N, Acc) ->
lists:reverse(Acc) ;
partition([H|T], 1, N, Acc) ->
partition(T, 2, N, [[H]|Acc]) ;
partition([H|T], C, N, [HAcc|TAcc]) when C < N ->
partition(T, C+1, N, [[H|HAcc]|TAcc]) ;
partition([H|T], C, N, [HAcc|TAcc]) when C == N ->
partition(T, 1, N, [lists:reverse([H|HAcc])|TAcc]) ;
partition(L, C, N, Acc) when C > N ->
partition(L, 1, N, Acc).
It could probably be more elegant regarding the special case where C > N. Note that C is the size of the current sublist being constructed. At start, it is 1. And then it increments until it reaches the partition size of N.
We could also use a modified version of #chops code to let the last list contains the remaining items even if its size < N :
-module(n_length_chunks_fast).
-export([n_length_chunks_fast/2]).
n_length_chunks_fast(List,Len) ->
SkipLength = case length(List) rem Len of
0 -> 0;
N -> Len - N
end,
n_length_chunks_fast(lists:reverse(List),[],SkipLength,Len).
n_length_chunks_fast([],Acc,_Pos,_Max) -> Acc;
n_length_chunks_fast([H|T],Acc,Pos,Max) when Pos==Max ->
n_length_chunks_fast(T,[[H] | Acc],1,Max);
n_length_chunks_fast([H|T],[HAcc | TAcc],Pos,Max) ->
n_length_chunks_fast(T,[[H | HAcc] | TAcc],Pos+1,Max);
n_length_chunks_fast([H|T],[],Pos,Max) ->
n_length_chunks_fast(T,[[H]],Pos+1,Max).
I've slightly altered the implementation from #JLarky to remove the guard expression, which should be slightly faster:
split_list(List, Max) ->
element(1, lists:foldl(fun
(E, {[Buff|Acc], 1}) ->
{[[E],Buff|Acc], Max};
(E, {[Buff|Acc], C}) ->
{[[E|Buff]|Acc], C-1};
(E, {[], _}) ->
{[[E]], Max}
end, {[], Max}, List)).
I have a simple record structure consisting of a header (H) and a list of the data lines (D) 1:N. All header lines must start with a digit. All data lines have a leading whitespace. There also might be some empty lines (E) in between that must be ignored.
L = [H, D, D, E, H, D, E, H, D, D, D].
I would like to create a list of records:
-record(posting,{header,data}).
using list comprehension. Whats the best way to do it?
You must use lists:foldl/3 instead of list comprehensions in this case. With foldl/3 you can accumulate values of header and data through whole list L.
You should do something like this:
make_records(L) when is_list(L) ->
F = fun([32|_]=D,{#posting{}=H,Acc}) -> {H,[H#posting{data=D}|Acc]};
([], Acc) -> Acc;
([F|_]=H, {_,Acc}) when F=<$0, F>=$9 -> {#posting{header=>H}, Acc}
end,
{_, R} = lists:foldl(F, {undefined, []}, L),
R.
Anyway I think that straightforward Erlang version doesn't seems too complicated and should be little bit faster.
make_records2(L) when is_list(L) ->
make_records2(L, undefined, []).
make_records2([], _, R) -> R;
make_records2([[32|_]=D|T], H, Acc) when is_list(H) ->
make_records2(T, H, [#posting{header=H,data=D}|Acc]);
make_records2([[]|T], H, Acc) ->
make_records2(T, H, Acc);
make_records2([[F|_]=H|T], _, Acc) when F>=$0, F=<$9 ->
make_records2(T, H, Acc).
Edit: If you have to add better row classification or parsing, adding new function is better because it improves readability.
parse_row([Digit|_]=R) when Digit >= $0, Digit =< $9 -> {header, R};
parse_row(R) -> try_spaces(R).
try_spaces([]) -> empty;
try_spaces([Sp|R]) when Sp=:=$\s; Sp=:=$\t; Sp=:=$\n ->
try_spaces(R); % skip all white spaces from Data field
try_spaces(Data) -> {data, Data}.
You can use it like this:
make_records(L) when is_list(L) ->
F = fun(Row, {H, Acc}) ->
case parse_row(Row) of
{data, D} when is_record(H, posting) -> {H,[H#posting{data=D}|Acc]};
empty -> Acc;
{header, H} -> {#posting{header=>H}, Acc}
end,
{_, R} = lists:foldl(F, {undefined, []}, L),
R.
Tail recursive native Erlang solution:
make_records2(L) when is_list(L) ->
make_records2([parse_row(R) || R<-L], undefined, []).
make_records2([], _, R) -> R;
make_records2([{data, D}|T], H, Acc) when is_list(H) ->
make_records2(T, H, [#posting{header=H,data=D}|Acc]);
make_records2([empty|T], H, Acc) ->
make_records2(T, H, Acc);
make_records2([{header,H}|T], _, Acc) ->
make_records2(T, H, Acc).
I think that there is no reason use tail recursion from performance point of view:
make_records3(L) when is_list(L) ->
make_records3(L, undefined).
make_records3([], _) -> [];
make_records3([R|T], H) ->
case parse_row(R) of
{data, D} when is_list(H) -> [#posting{head=H,data=D}|make_records3(T, H)];
empty -> make_records3(T, H);
{header, H2} -> make_records3(T, H2)
end.
... and many many other variants.
I needed to collapse all Data lines beneath the header - so for the moment here is what I have:
sanitize(S) -> trim:trim(S).
make_records(L) when is_list(L) -> make_records(L, undefined, []).
make_records([], _, R) -> lists:reverse(R);
make_records([[32|_]=D|T], H, Acc) when is_tuple(H) ->
make_records(T, {element(1,H),[sanitize(D)|element(2,H)]},Acc);
make_records([[$\n|_]=D|T], H, Acc) when is_tuple(H) ->
make_records(T, H, Acc);
make_records([[F|_]=H|T], B, Acc) when F>=$0, F=<$9 ->
if is_tuple(B) ->
make_records(T, {sanitize(H),[]}, [#posting{header=element(1,B),
data=lists:reverse(element(2,B))}|Acc]);
true ->
make_records(T, {sanitize(H),[]}, Acc)
end.