This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Linked list partition function and reversed results
Actually I don't care about the input type or the output type, any of seq, array, list will do. (It doesn't have to be generic) Currently my code takes list as input and (list * list) as output
let takeWhile predicator list =
let rec takeWhileRec newList remain =
match remain with
| [] -> (newList |> List.rev, remain)
| x::xs -> if predicator x then
takeWhileRec (x::newList) xs
else
(newList |> List.rev, remain)
takeWhileRec [] list
However, there is a pitfall. As fas as I see, List.rev is O(n^2), which would likely to dominate the overall speed? I think it is even slower than the ugly solution: Seq.takeWhile, then count, and then take tail n times... which is still O(n)
(If there is a C# List, then i would use that without having to reverse it...)
A side question, what's difference between Array.ofList and List.toArray , or more generally, A.ofB and B.ofA in List, Seq, Array?
is seq myList identical to List.toSeq myList?
Another side question, is nested Seq.append have same complexity as Seq.concat?
e.g.
Seq.append (Seq.append (Seq.append a b) c) d // looks aweful
Seq.concat [a;b;c;d]
1)The relevant implementation of List.rev is in local.fs in the compiler - it is
// optimized mutation-based implementation. This code is only valid in fslib, where mutation of private
// tail cons cells is permitted in carefully written library code.
let rec revAcc xs acc =
match xs with
| [] -> acc
| h::t -> revAcc t (h::acc)
let rev xs =
match xs with
| [] -> xs
| [_] -> xs
| h1::h2::t -> revAcc t [h2;h1]
The comment does seem odd as there is no obvious mutation. Note that this is in fact O(n) not O(n^2)
2) As pad said there is no difference - I prefer to use the to.. as I think
A
|> List.map ...
|> List.toArray
looks nicer than
A
|> List.map ...
|> Array.ofList
but that is just me.
3)
Append (compiler source):
[<CompiledName("Append")>]
let append (source1: seq<'T>) (source2: seq<'T>) =
checkNonNull "source1" source1
checkNonNull "source2" source2
fromGenerator(fun () -> Generator.bindG (toGenerator source1) (fun () -> toGenerator source2))
Note that for each append we get an extra generator that has to be walked through. In comparison, the concat implementation will just have 1 single extra function rather than n so using concat is probably better.
To answer your questions:
1) Time complexity of List.rev is O(n) and worst-case complexity of takeWhile is also O(n). So using List.rev doesn't increase complexity of the function. Using ResizeArray could help you avoid List.rev, but you have to tolerate a bit of mutation.
let takeWhile predicate list =
let rec loop (acc: ResizeArray<_>) rest =
match rest with
| x::xs when predicate x -> acc.Add(x); loop acc xs
| _ -> (acc |> Seq.toList, rest)
loop (ResizeArray()) list
2) There is no difference. Array.ofList and List.toArray uses the same function internally (see here and here).
3). I think Seq.concat has the same complexity with a bunch of Seq.append. In the context of List andArray, concat is more efficient than append because you have more information to pre-allocate space for outputs.
how about this:
let takeWhile pred =
let cont = ref true
List.partition (pred >> fun r -> !cont && (cont := r; r))
It uses a single library function, List.partition, which is efficiently implemented.
Hope this is what you meant :)
Related
I'm going through a mutable ConcurrentDictionary to remove old entries.
let private cache = ConcurrentDictionary<Instrument * DateTimeOffset, SmallSet>()
and since I can't remove entries while iterating through the keys, I was wondering if this would be a good use for Seq.cache:
let old = DateTimeOffset.UtcNow.AddHours(-1.)
cache.Keys
|> Seq.filter (fun x -> snd x <= old)
|> Seq.cache
|> Seq.iter (fun x -> cache.TryRemove x |> ignore)
I have never used Seq.cache, and I assume it creates a separation between the two loops. Am I understanding how it works correctly?
In the scenario you described I don't see any reason why you need to iterate the collection multiple times. You can just go over the KeyValuePairs inside the dictionary and analyze each KeyValuePair if it matches your condition or no.
So, something like this should do the job:
cache |> Seq.iter(function
| x when snd x.Key <= old -> cache.TryRemove(x.Key) |> ignore
| _ -> ())
I am trying to convert the following normal-recursive code to tail-recursive in F#, but I am failing miserably.
let rec insert elem lst =
match lst with
| [] -> [elem]
| hd::tl -> if hd > elem then
elem::lst
else
hd::(insert elem tl)
let lst1 = []
let lst2 = [1;2;3;5]
printfn "\nInserting 4 in an empty list: %A" (insert 4 lst1)
printfn "\nInserting 4 in a sorted list: %A" (insert 4 lst2)
Can you guys help? Unfortunately I am a beginner in f#. Also, can anyone point me to a good tutorial to understand tail-recursion?
The point of tail recursion is the following: the last operation before returning from a function is a call to itself; this is called a tail call, and is where tail recursion gets its name from (the recursive call is in last, i.e. tail position).
Your function is not tail recursive because at least one of its branches has an operation after the recursive call (the list cons operator).
The usual way of converting a recursive function into a tail-recursive function is to add an argument to accumulate intermediate results (the accumulator). When it comes to lists, and when you realize that the only elementary list operation is prepending an element, this also means that after you are through with processing your list, it will be reversed, and thus the resulting accumulator will usually have to be reversed again.
With all these points in mind, and given that we do not want to change the function's public interface by adding a parameter that is superfluous from the caller's point of view, we move the real work to an internal subfunction. This particular function is slightly more complicated because after the element has been inserted, there is nothing else to do but concatenate the two partial lists again, one of which is now in reverse order while the other is not. We create a second internal function to handle that part, and so the whole function looks as follows:
let insert elm lst =
let rec iter acc = function
| [] -> List.rev (elm :: acc)
| (h :: t) as ls ->
if h > elm then finish (elm :: ls) acc
else iter (h :: acc) t
and finish acc = function
| [] -> acc
| h :: t -> finish (h :: acc) t
iter [] lst
For further studying, Scott Wlaschin's F# for Fun and Profit is a great resource, and tail recursion is handled in a larger chapter about recursive types and more: https://fsharpforfunandprofit.com/posts/recursive-types-and-folds
I have a list of lists
LL = [[1;2;3];[4;5;6];[7;8;9]]
And I would like it to look like this
LSimple= [1;2;3;4;5;6;7;8;9]
That's as simple as I can ask it, but maybe rewording helps. How can I parse this lists of lists and create a simple list from it?
List.concat LL
Will do what you want. The X.concat family of functions concatenate any sequence of the collection X to a single X where X may be a List, Array, Seq or even a String with a given separator.
Depending on the implementation of List.concat it could be highly inefficient. if it is implemented with the naive approach it will use the naive way for a single appending.
for the simple case: assume that we got two list xs ys we wish to append together.
then the implementation would be
let rec append =
function
| [], ys -> ys
| xs, [] -> xs
| x::xs, y::ys -> append(x :: append (xs,[y]), ys)
This is the most inefficient implementation there is, another implementation will simple replace the last case return with
x::append(xs,y::ys)
Both has O(n) running time. but the last will have a smaller konstant k (k*n).
The most effecient way with 2*k*n run time are given below. where n is the number of elements, and k is the kost of concatenating a single element to a list.
let collapse lst =
let rec help acc =
function
| [] -> // conversion point
List.rev acc
| (x :: []) :: xss -> // singleton case
help (x :: acc) xss
| (x :: xs) :: xss -> // general case
help (x :: acc) (xs :: xss)
help [] lst // return
In general, don't trust libraries when it comes to efficiency, and be aware that those who make them are just humans as you, so there can be a lot of bugs.
Hey guys, I'm trying to get cozy with functional programming (particularly with F#), and I've hit a wall when it comes to building tail-recursive functions. I'm pretty good with turning basic recursion (where the function basically calls itself once per invocation), into tail recursion, but I now have a slightly more complicated situation.
In my case, the function must accept a single list as a parameter. When the function is called, I have to remove the first element from the list, and then recur using the remainder of the list. Then I need to apply the first element which I removed in some way to the result of the recursion. Next, I remove the second element and do the same thing (Note: when I say "remove the seond element", that is from the original list, so the list passed at the recursion includes the first element as well). I do the same for the third, fourth, etc. elements of the list.
Is there a way to convert the above situation into a tail-recursive function? Maybe nested tail-recursive functions??? Thank you for any answers.
Okay, so here's my basic code. This particular one is a permutation generator (I'm not too concern with the permutation part, though - it's the recursion I'd like to focusing on):
let permutationsOther str =
match str with
| value :: [] ->
[[value]]
| _ ->
let list = (List.map (fun a -> // This applies the remove part for every element a
let lst = (List.filter (fun b -> b <> a) str) // This part removes element a from the list
let permutedLst = permutations lst // recursive call
consToAll a permutedLst // constToAll this is my own function which performs "cons" operation with a and every element in the list permutedLst
) str)
List.reduce (fun acc elem -> elem # acc) list // flatten list of lists produce by map into a single list
I hope this is clear enough - I'll be happy to provide clarifications if needed.
By the way, I have found just a way to rewrite this particular function so that it only uses a single recursion, but it was a fluke more than an informed decision. However, this has encouraged me that there may be a general method of turning multiple recursion into single recursion, but I have not yet found it.
Conversion to CPS should do the trick:
NOTE 1: Source of the sample is typed directly in browser, so may contain errors :(. But I hope it can demonstrate the general idea.
NOTE 2: consToAll function should be converted to CPS too: consToAll: 'T -> 'T list list -> ('T list list -> 'R) -> 'R
let remove x l = List.filter ((<>) x) l // from original post: should duplicates also be removed ???
let permute l =
let rec loop k l =
match l with
| [] -> k []
| [value] -> k [[value]]
| _ -> filter l [] l (fun r -> r |> List.reduce (fun acc elem -> elem # acc) |> k )
and filter l acc orig fk =
match l with
| [] -> fk acc
| x::xs ->
remove x orig
|> loop (fun res ->
consToAll x res (fun rs -> filter xs (rs::acc) orig fk)
)
loop id l
The :: operator in F# always prepends elements to the list. Is there an operator that appends to the list? I'm guessing that using # operator
[1; 2; 3] # [4]
would be less efficient, than appending one element.
As others said, there is no such operator, because it wouldn't make much sense. I actually think that this is a good thing, because it makes it easier to realize that the operation will not be efficient. In practice, you shouldn't need the operator - there is usually a better way to write the same thing.
Typical scenario: I think that the typical scenario where you could think that you need to append elements to the end is so common that it may be useful to describe it.
Adding elements to the end seems necessary when you're writing a tail-recursive version of a function using the accumulator parameter. For example a (inefficient) implementation of filter function for lists would look like this:
let filter f l =
let rec filterUtil acc l =
match l with
| [] -> acc
| x::xs when f x -> filterUtil (acc # [x]) xs
| x::xs -> filterUtil acc xs
filterUtil [] l
In each step, we need to append one element to the accumulator (which stores elements to be returned as the result). This code can be easily modified to use the :: operator instead of appending elements to the end of the acc list:
let filter f l =
let rec filterUtil acc l =
match l with
| [] -> List.rev acc // (1)
| x::xs when f x -> filterUtil (x::acc) xs // (2)
| x::xs -> filterUtil acc xs
filterUtil [] l
In (2), we're now adding elements to the front of the accumulator and when the function is about to return the result, we reverse the list (1), which is a lot more efficient than appending elements one by one.
Lists in F# are singly-linked and immutable. This means consing onto the front is O(1) (create an element and have it point to an existing list), whereas snocing onto the back is O(N) (as the entire list must be replicated; you can't change the existing final pointer, you must create a whole new list).
If you do need to "append one element to the back", then e.g.
l # [42]
is the way to do it, but this is a code smell.
The cost of appending two standard lists is proportional to the length of the list on the left. In particular, the cost of
xs # [x]
is proportional to the length of xs—it is not a constant cost.
If you want a list-like abstraction with a constant-time append, you can use John Hughes's function representation, which I'll call hlist. I'll try to use OCaml syntax, which I hope is close enough to F#:
type 'a hlist = 'a list -> 'a list (* a John Hughes list *)
let empty : 'a hlist = let id xs = xs in id
let append xs ys = fun tail -> xs (ys tail)
let singleton x = fun tail -> x :: tail
let cons x xs = append (singleton x) xs
let snoc xs x = append xs (singleton x)
let to_list : 'a hlist -> 'a list = fun xs -> xs []
The idea is that you represent a list functionally as a function from "the rest of the elements" to "the final list". This works great if you are going to build up the whole list before you look at any of the elements. Otherwise you'll have to deal with the linear cost of append or use another data structure entirely.
I'm guessing that using # operator [...] would be less efficient, than appending one element.
If it is, it will be a negligible difference. Both appending a single item and concatenating a list to the end are O(n) operations. As a matter of fact I can't think of a single thing that # has to do, which a single-item append function wouldn't.
Maybe you want to use another data structure. We have double-ended queues (or short "Deques") in fsharpx. You can read more about them at http://jackfoxy.com/double-ended-queues-for-fsharp
The efficiency (or lack of) comes from iterating through the list to find the final element. So declaring a new list with [4] is going to be negligible for all but the most trivial scenarios.
Try using a double-ended queue instead of list. I recently added 4 versions of deques (Okasaki's spelling) to FSharpx.Core (Available through NuGet. Source code at FSharpx.Core.Datastructures). See my article about using dequeus Double-ended queues for F#
I've suggested to the F# team the cons operator, ::, and the active pattern discriminator be made available for other data structures with a head/tail signature.3