Break the iteration and got the values and states? - f#

I need to call a function on each item in the list; and quit immediately if the function returns -1. I need to return the sum of results of the function and a string of "Done" or "Error".
let input = seq { 0..4 } // fake input
let calc1 x = // mimic failing after input 3. It's a very expensive function and should stop running after failing
if x >= 3 then -1 else x * 2
let run input calc =
input
|> Seq.map(fun x ->
let v = calc x
if v = -1 then .... // Error occurred, stop the execution if gets -1. Calc will not work anymore
v)
|> Seq.sum, if hasError then "Error" else "Done"
run input calc // should return (6, "Error")
run input id // should return (20, "Done")

The simplest way to effectively achieve exactly what is asked in idiomatic manner would be to use of an inner recursive function for traversing the sequence:
let run input calc =
let rec inner unprocessed sum =
match unprocessed with
| [] -> (sum, "Done")
| x::xs -> let res = calc x
if res < 0 then (sum, "Error") else inner xs (sum + res)
inner (input |> Seq.toList) 0
Then run (seq {0..4}) (fun x -> if x >=3 then -1 else x * 2) returns (6,"Error") while
run (seq [0;1;2;1;0;0;1;1;2;2]) (fun x -> if x >=3 then -1 else x * 2) returns (20, "Done")

More efficient version of the same thing shown below. This means it is now essentially a copy of #GeneBelitski's answer.
let run input calc =
let inputList = Seq.toList input
let rec subrun inp acc =
match inp with
| [] -> (acc, "Done")
| (x :: xs) ->
let res = calc x
match res with
| Some(y) -> subrun xs (acc + y)
| None -> (acc, "Error")
subrun inputList 0
Note that this function below is EXTREMELY slow, likely because it uses Seq.tail (I had thought that would be the same as List.tail). I leave it in for posterity.
The easiest way I can think of for doing this in F# would be to use a tail-recursive function. Something like
let run input calc =
let rec subrun inp acc =
if Seq.isEmpty inp then
(acc, "Done")
else
let res = calc (Seq.head inp)
match res with
| Some(x) -> subrun (Seq.tail inp) (acc + x)
| None -> (acc, "Error")
subrun input 0
I'm not 100% sure just how efficient that would be. In my experience, sometimes for some reason, my own tail-recursive functions seem to be considerably slower than using the built-in higher-order functions. This should at least get you to the right result.
The below, while apparently not answering the actual question, is left in just in case it is useful to someone.
The typical way to handle this would be to make your calc function return either an Option or Result type, e.g.
let calc1 x = if x = 3 then None else Some(x*2)
and then map that to your input. Afterwards, you can fairly easily do something like
|> Seq.exists Option.isNone
to make see if there are Nones in the resulting seq (you can pipe it to not if you want the opposite result).
If you just need to eliminate Nones from the list, you can use
Seq.choose id
which will eliminate all Nones while leaving the Options intact.
For summing the list, assuming that you have used choose to be left with just the Somes, then you can do
Seq.sumBy Option.get

Here is a monadic way of doing it using the Result monad.
First we create function calcR that if calc returns -1 returns Error otherwise returns Ok with the value:
let calcR f x =
let r = f x
if r = -1 then Error "result was = -1" else
Ok r
Then, we create function sumWhileOk that uses Seq.fold over the input, adding up the results as long as they are Ok.
let sumWhileOk fR =
Seq.fold(fun totalR v ->
totalR
|> Result.bind(fun total ->
fR v
|> Result.map (fun r -> total + r)
|> Result.mapError (fun _ -> total )
)
) (Ok 0)
Result.bind and Result.map only invoke their lambda function if the supplied value is Ok if it is Error it gets bypassed. Result.mapError is used to replace the error message from calcR with the current total as an error.
It is called this way:
input |> sumWhileOk (calcR id)
// returns: Ok 10
input |> sumWhileOk (calcR calc1)
// return: Error 6

Related

Combine memoization and tail-recursion

Is it possible to combine memoization and tail-recursion somehow? I'm learning F# at the moment and understand both concepts but can't seem to combine them.
Suppose I have the following memoize function (from Real-World Functional Programming):
let memoize f = let cache = new Dictionary<_, _>()
(fun x -> match cache.TryGetValue(x) with
| true, y -> y
| _ -> let v = f(x)
cache.Add(x, v)
v)
and the following factorial function:
let rec factorial(x) = if (x = 0) then 1 else x * factorial(x - 1)
Memoizing factorial isn't too difficult and making it tail-recursive isn't either:
let rec memoizedFactorial =
memoize (fun x -> if (x = 0) then 1 else x * memoizedFactorial(x - 1))
let tailRecursiveFactorial(x) =
let rec factorialUtil(x, res) = if (x = 0)
then res
else let newRes = x * res
factorialUtil(x - 1, newRes)
factorialUtil(x, 1)
But can you combine memoization and tail-recursion? I made some attempts but can't seem to get it working. Or is this simply not possible?
As always, continuations yield an elegant tailcall solution:
open System.Collections.Generic
let cache = Dictionary<_,_>() // TODO move inside
let memoizedTRFactorial =
let rec fac n k = // must make tailcalls to k
match cache.TryGetValue(n) with
| true, r -> k r
| _ ->
if n=0 then
k 1
else
fac (n-1) (fun r1 ->
printfn "multiplying by %d" n //***
let r = r1 * n
cache.Add(n,r)
k r)
fun n -> fac n id
printfn "---"
let r = memoizedTRFactorial 4
printfn "%d" r
for KeyValue(k,v) in cache do
printfn "%d: %d" k v
printfn "---"
let r2 = memoizedTRFactorial 5
printfn "%d" r2
printfn "---"
// comment out *** line, then run this
//let r3 = memoizedTRFactorial 100000
//printfn "%d" r3
There are two kinds of tests. First, this demos that calling F(4) caches F(4), F(3), F(2), F(1) as you would like.
Then, comment out the *** printf and uncomment the final test (and compile in Release mode) to show that it does not StackOverflow (it uses tailcalls correctly).
Perhaps I'll generalize out 'memoize' and demonstrate it on 'fib' next...
EDIT
Ok, here's the next step, I think, decoupling memoization from factorial:
open System.Collections.Generic
let cache = Dictionary<_,_>() // TODO move inside
let memoize fGuts n =
let rec newFunc n k = // must make tailcalls to k
match cache.TryGetValue(n) with
| true, r -> k r
| _ ->
fGuts n (fun r ->
cache.Add(n,r)
k r) newFunc
newFunc n id
let TRFactorialGuts n k memoGuts =
if n=0 then
k 1
else
memoGuts (n-1) (fun r1 ->
printfn "multiplying by %d" n //***
let r = r1 * n
k r)
let memoizedTRFactorial = memoize TRFactorialGuts
printfn "---"
let r = memoizedTRFactorial 4
printfn "%d" r
for KeyValue(k,v) in cache do
printfn "%d: %d" k v
printfn "---"
let r2 = memoizedTRFactorial 5
printfn "%d" r2
printfn "---"
// comment out *** line, then run this
//let r3 = memoizedTRFactorial 100000
//printfn "%d" r3
EDIT
Ok, here's a fully generalized version that seems to work.
open System.Collections.Generic
let memoize fGuts =
let cache = Dictionary<_,_>()
let rec newFunc n k = // must make tailcalls to k
match cache.TryGetValue(n) with
| true, r -> k r
| _ ->
fGuts n (fun r ->
cache.Add(n,r)
k r) newFunc
cache, (fun n -> newFunc n id)
let TRFactorialGuts n k memoGuts =
if n=0 then
k 1
else
memoGuts (n-1) (fun r1 ->
printfn "multiplying by %d" n //***
let r = r1 * n
k r)
let facCache,memoizedTRFactorial = memoize TRFactorialGuts
printfn "---"
let r = memoizedTRFactorial 4
printfn "%d" r
for KeyValue(k,v) in facCache do
printfn "%d: %d" k v
printfn "---"
let r2 = memoizedTRFactorial 5
printfn "%d" r2
printfn "---"
// comment out *** line, then run this
//let r3 = memoizedTRFactorial 100000
//printfn "%d" r3
let TRFibGuts n k memoGuts =
if n=0 || n=1 then
k 1
else
memoGuts (n-1) (fun r1 ->
memoGuts (n-2) (fun r2 ->
printfn "adding %d+%d" r1 r2 //%%%
let r = r1+r2
k r))
let fibCache, memoizedTRFib = memoize TRFibGuts
printfn "---"
let r5 = memoizedTRFib 4
printfn "%d" r5
for KeyValue(k,v) in fibCache do
printfn "%d: %d" k v
printfn "---"
let r6 = memoizedTRFib 5
printfn "%d" r6
printfn "---"
// comment out %%% line, then run this
//let r7 = memoizedTRFib 100000
//printfn "%d" r7
The predicament of memoizing tail-recursive functions is, of course, that when tail-recursive function
let f x =
......
f x1
calls itself, it is not allowed to do anything with a result of the recursive call, including putting it into cache. Tricky; so what can we do?
The critical insight here is that since the recursive function is not allowed to do anything with a result of recursive call, the result for all arguments to recursive calls will be the same! Therefore if recursion call trace is this
f x0 -> f x1 -> f x2 -> f x3 -> ... -> f xN -> res
then for all x in x0,x1,...,xN the result of f x will be the same, namely res. So the last invocation of a recursive function, the non-recursive call, knows the results for all the previous values - it is in a position to cache them. The only thing you need to do is to pass a list of visited values to it. Here is what it might look for factorial:
let cache = Dictionary<_,_>()
let rec fact0 l ((n,res) as arg) =
let commitToCache r =
l |> List.iter (fun a -> cache.Add(a,r))
match cache.TryGetValue(arg) with
| true, cachedResult -> commitToCache cachedResult; cachedResult
| false, _ ->
if n = 1 then
commitToCache res
cache.Add(arg, res)
res
else
fact0 (arg::l) (n-1, n*res)
let fact n = fact0 [] (n,1)
But wait! Look - l parameter of fact0 contains all the arguments to recursive calls to fact0 - just like the stack would in a non-tail-recursive version! That is exactly right. Any non-tail recursive algorithm can be converted to a tail-recursive one by moving the "list of stack frames" from stack to heap and converting the "postprocessing" of recursive call result into a walk over that data structure.
Pragmatic note: The factorial example above illustrates a general technique. It is quite useless as is - for factorial function it is quite enough to cache the top-level fact n result, because calculation of fact n for a particular n only hits a unique series of (n,res) pairs of arguments to fact0 - if (n,1) is not cached yet, then none of the pairs fact0 is going to be called on are.
Note that in this example, when we went from non-tail-recursive factorial to a tail-recursive factorial, we exploited the fact that multiplication is associative and commutative - tail-recursive factorial execute a different set of multiplications than a non-tail-recursive one.
In fact, a general technique exists for going from non-tail-recursive to tail-recursive algorithm, which yields an algorithm equivalent to a tee. This technique is called "continuatuion-passing transformation". Going that route, you can take a non-tail-recursive memoizing factorial and get a tail-recursive memoizing factorial by pretty much a mechanical transformation. See Brian's answer for exposition of this method.
I'm not sure if there's a simpler way to do this, but one approach would be to create a memoizing y-combinator:
let memoY f =
let cache = Dictionary<_,_>()
let rec fn x =
match cache.TryGetValue(x) with
| true,y -> y
| _ -> let v = f fn x
cache.Add(x,v)
v
fn
Then, you can use this combinator in lieu of "let rec", with the first argument representing the function to call recursively:
let tailRecFact =
let factHelper fact (x, res) =
printfn "%i,%i" x res
if x = 0 then res
else fact (x-1, x*res)
let memoized = memoY factHelper
fun x -> memoized (x,1)
EDIT
As Mitya pointed out, memoY doesn't preserve the tail recursive properties of the memoee. Here's a revised combinator which uses exceptions and mutable state to memoize any recursive function without overflowing the stack (even if the original function is not itself tail recursive!):
let memoY f =
let cache = Dictionary<_,_>()
fun x ->
let l = ResizeArray([x])
while l.Count <> 0 do
let v = l.[l.Count - 1]
if cache.ContainsKey(v) then l.RemoveAt(l.Count - 1)
else
try
cache.[v] <- f (fun x ->
if cache.ContainsKey(x) then cache.[x]
else
l.Add(x)
failwith "Need to recurse") v
with _ -> ()
cache.[x]
Unfortunately, the machinery which is inserted into each recursive call is somewhat heavy, so performance on un-memoized inputs requiring deep recursion can be a bit slow. However, compared to some other solutions, this has the benefit that it requires fairly minimal changes to the natural expression of recursive functions:
let fib = memoY (fun fib n ->
printfn "%i" n;
if n <= 1 then n
else (fib (n-1)) + (fib (n-2)))
let _ = fib 5000
EDIT
I'll expand a bit on how this compares to other solutions. This technique takes advantage of the fact that exceptions provide a side channel: a function of type 'a -> 'b doesn't actually need to return a value of type 'b, but can instead exit via an exception. We wouldn't need to use exceptions if the return type explicitly contained an additional value indicating failure. Of course, we could use the 'b option as the return type of the function for this purpose. This would lead to the following memoizing combinator:
let memoO f =
let cache = Dictionary<_,_>()
fun x ->
let l = ResizeArray([x])
while l.Count <> 0 do
let v = l.[l.Count - 1]
if cache.ContainsKey v then l.RemoveAt(l.Count - 1)
else
match f(fun x -> if cache.ContainsKey x then Some(cache.[x]) else l.Add(x); None) v with
| Some(r) -> cache.[v] <- r;
| None -> ()
cache.[x]
Previously, our memoization process looked like:
fun fib n ->
printfn "%i" n;
if n <= 1 then n
else (fib (n-1)) + (fib (n-2))
|> memoY
Now, we need to incorporate the fact that fib should return an int option instead of an int. Given a suitable workflow for option types, this could be written as follows:
fun fib n -> option {
printfn "%i" n
if n <= 1 then return n
else
let! x = fib (n-1)
let! y = fib (n-2)
return x + y
} |> memoO
However, if we're willing to change the return type of the first parameter (from int to int option in this case), we may as well go all the way and just use continuations in the return type instead, as in Brian's solution. Here's a variation on his definitions:
let memoC f =
let cache = Dictionary<_,_>()
let rec fn n k =
match cache.TryGetValue(n) with
| true, r -> k r
| _ ->
f fn n (fun r ->
cache.Add(n,r)
k r)
fun n -> fn n id
And again, if we have a suitable computation expression for building CPS functions, we can define our recursive function like this:
fun fib n -> cps {
printfn "%i" n
if n <= 1 then return n
else
let! x = fib (n-1)
let! y = fib (n-2)
return x + y
} |> memoC
This is exactly the same as what Brian has done, but I find the syntax here is easier to follow. To make this work, all we need are the following two definitions:
type CpsBuilder() =
member this.Return x k = k x
member this.Bind(m,f) k = m (fun a -> f a k)
let cps = CpsBuilder()
I wrote a test to visualize the memoization. Each dot is a recursive call.
......720 // factorial 6
......720 // factorial 6
.....120 // factorial 5
......720 // memoizedFactorial 6
720 // memoizedFactorial 6
120 // memoizedFactorial 5
......720 // tailRecFact 6
720 // tailRecFact 6
.....120 // tailRecFact 5
......720 // tailRecursiveMemoizedFactorial 6
720 // tailRecursiveMemoizedFactorial 6
.....120 // tailRecursiveMemoizedFactorial 5
kvb's solution returns the same results are straight memoization like this function.
let tailRecursiveMemoizedFactorial =
memoize
(fun x ->
let rec factorialUtil x res =
if x = 0 then
res
else
printf "."
let newRes = x * res
factorialUtil (x - 1) newRes
factorialUtil x 1
)
Test source code.
open System.Collections.Generic
let memoize f =
let cache = new Dictionary<_, _>()
(fun x ->
match cache.TryGetValue(x) with
| true, y -> y
| _ ->
let v = f(x)
cache.Add(x, v)
v)
let rec factorial(x) =
if (x = 0) then
1
else
printf "."
x * factorial(x - 1)
let rec memoizedFactorial =
memoize (
fun x ->
if (x = 0) then
1
else
printf "."
x * memoizedFactorial(x - 1))
let memoY f =
let cache = Dictionary<_,_>()
let rec fn x =
match cache.TryGetValue(x) with
| true,y -> y
| _ -> let v = f fn x
cache.Add(x,v)
v
fn
let tailRecFact =
let factHelper fact (x, res) =
if x = 0 then
res
else
printf "."
fact (x-1, x*res)
let memoized = memoY factHelper
fun x -> memoized (x,1)
let tailRecursiveMemoizedFactorial =
memoize
(fun x ->
let rec factorialUtil x res =
if x = 0 then
res
else
printf "."
let newRes = x * res
factorialUtil (x - 1) newRes
factorialUtil x 1
)
factorial 6 |> printfn "%A"
factorial 6 |> printfn "%A"
factorial 5 |> printfn "%A\n"
memoizedFactorial 6 |> printfn "%A"
memoizedFactorial 6 |> printfn "%A"
memoizedFactorial 5 |> printfn "%A\n"
tailRecFact 6 |> printfn "%A"
tailRecFact 6 |> printfn "%A"
tailRecFact 5 |> printfn "%A\n"
tailRecursiveMemoizedFactorial 6 |> printfn "%A"
tailRecursiveMemoizedFactorial 6 |> printfn "%A"
tailRecursiveMemoizedFactorial 5 |> printfn "%A\n"
System.Console.ReadLine() |> ignore
That should work if mutual tail recursion through y are not creating stack frames:
let rec y f x = f (y f) x
let memoize (d:System.Collections.Generic.Dictionary<_,_>) f n =
if d.ContainsKey n then d.[n]
else d.Add(n, f n);d.[n]
let rec factorialucps factorial' n cont =
if n = 0I then cont(1I) else factorial' (n-1I) (fun k -> cont (n*k))
let factorialdpcps =
let d = System.Collections.Generic.Dictionary<_, _>()
fun n -> y (factorialucps >> fun f n -> memoize d f n ) n id
factorialdpcps 15I //1307674368000

Seq.fold and boolean accumulator

I can never find the source code of the F# core libraries. I know it is supposedly open but google is not kind to me in helping me locate it, if so I would have looked up the impl of Seq.fold - but here goes the question.
Does anybody see any issue with the following snippet:
let success = myList |>
Seq.fold
(fun acc item -> evaluation item)
false
Logically it doesn't seem to hold water and I can and will experiment to test it. But I wanted to ask the community. If any single evaluation inside of myList retruns false, I want the success variable to be false...
So the test:
let myList = [true; true]
let success = List.fold (fun acc item -> acc && item) true myList
and
let myList = [true; false; true]
let success = List.fold (fun acc item -> acc && item) true myList
do return the proper results - I just would be more comfy seeing the source...
I think what you're looking for is something like this:
let success = myList |>
Seq.fold
(fun acc item -> acc && evaluation item)
true
This also offers "short-circut" evaluation so that if acc is false from a previous evaluation, evaluation item won't run and the expression will simply return false.
MSDN documentation for fold operator
Seq.exists will short circuit:
let success =
[1;2;3;40;5;2]
|> Seq.exists (fun item->(item>30))
|> not
I get that this is an old question, but the following may be relevant to those who have a similar question.
About the specific question here
There already exists a function that returns false as soon as one element in a Sequence is false: Seq.forAll.
So the easiest answer to the question is in fact:
let success = Seq.forAll evaluation myList
which is slightly easier to grasp than TechNeilogy’s (rewritten) answer
let success = not (Seq.exists evaluation myList)
Both in the accepted answer by Wesley Wiser and in this answer, the evaluation function is not evaluated on the items after the first item that evaluates to fold.
But, as Pascal Cuoq correctly remarked, in the accepted answer all the elements of the remainder of the list are still iterated over, which is useless.
In contrast, Seq.forAll really stops iterating when there is no use to continue. So do Seq.exists, Seq.takeWhile, …
About short-circuiting a folding in general
There are other cases where one wants to short-circuit a folding. It can be done.
Step 1: Define a folder with some kind of indication that the state won’t change during the traversal the rest of the source sequence, and the folding should be short-circuited.
Step 2: Use Seq.scan instead of Seq.fold.
Seq.scan is like Seq.fold, takes the same arguments, but computes on-demand, and returns not just the final state, but the sequence of all intermediate states and the final state.
It follows that (for finite mySequence): Seq.last (Seq.scan folder initialState mySequence) = Seq.fold folder initialState mySequence
Step 3: Use a short-circuiting function on the output of Seq.scan. Take your pick: Seq.takeWhile, Seq.forall, Seq.exists, …
In the following example, the state becomes None when a duplicate element is found, which means that the scanning may be short-circuited.
let allDistinct mySequence =
let folder state element =
match state with
| Some elementsSoFar when not (Set.contains element elementsSoFar) ->
Some (Set.add element elementsSoFar)
| _ ->
None
let initialState = Some Set.empty
let scanning = Seq.scan folder initialState mySequence
Seq.forall Option.isSome scanning
Hmmmm, I upgraded my Visual Studio and F# recently, and can't seem to locate the directory containing the F# library code. But, for what its worth, Seq.fold is equivalent to the following:
let fold f seed items =
let mutable res = seed
for item in items do
res <- f res item
res
If any single evaluation inside of
myList retruns false, I want the
success variable to be false...
It depends on how your evaluation function is implemented. If you want to return false when any of your items are false, use Seq.forall instead.
something like this
let l = [true; true; true; false; true]
let eval x = x
let x = (true, l) ||> Seq.fold(fun acc item -> acc && (eval item))
or you want to stop evaluation on first false result?
let l = [true; false; true]
l |> Seq.forall id
As for the original source, here are the fold functions from the August 10, 2010 release.
Shouldn't really need to concern yourself over the implementation, but seeing it can often be educational.
// Seq module
let fold<'T,'State> f (x:'State) (source : seq<'T>) =
checkNonNull "source" source
use e = source.GetEnumerator()
let mutable state = x
while e.MoveNext() do
state <- f state e.Current;
state
// Array module
let fold<'T,'State> (f : 'State -> 'T -> 'State) (acc: 'State) (array:'T[]) = //'
checkNonNull "array" array
let f = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)
let mutable state = acc
let len = array.Length
for i = 0 to len - 1 do
state <- f.Invoke(state,array.[i])
state
// List module
let fold<'T,'State> f (s:'State) (list: 'T list) =
match list with
| [] -> s
| _ ->
let f = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)
let rec loop s xs =
match xs with
| [] -> s
| h::t -> loop (f.Invoke(s,h)) t
loop s list
// MapTree module (Used by Map module)
let rec fold (f:OptimizedClosures.FSharpFunc<_,_,_,_>) x m =
match m with
| MapEmpty -> x
| MapOne(k,v) -> f.Invoke(x,k,v)
| MapNode(k,v,l,r,_) ->
let x = fold f x l
let x = f.Invoke(x,k,v)
fold f x r
// Map module
let fold<'Key,'T,'State when 'Key : comparison> f (z:'State) (m:Map<'Key,'T>) = //'
let f = OptimizedClosures.FSharpFunc<_,_,_,_>.Adapt(f)
MapTree.fold f z m.Tree
// SetTree module (Used by Set module)
let rec fold f x m =
match m with
| SetNode(k,l,r,_) ->
let x = fold f x l in
let x = f x k
fold f x r
| SetOne(k) -> f x k
| SetEmpty -> x
// Set module
let fold<'T,'State when 'T : comparison> f (z:'State) (s : Set<'T>) = //'
SetTree.fold f z s.Tree

Rfactor this F# code to tail recursion

I write some code to learning F#.
Here is a example:
let nextPrime list=
let rec loop n=
match n with
| _ when (list |> List.filter (fun x -> x <= ( n |> double |> sqrt |> int)) |> List.forall (fun x -> n % x <> 0)) -> n
| _ -> loop (n+1)
loop (List.max list + 1)
let rec findPrimes num=
match num with
| 1 -> [2]
| n ->
let temp = findPrimes <| n-1
(nextPrime temp ) :: temp
//find 10 primes
findPrimes 10 |> printfn "%A"
I'm very happy that it just works!
I'm totally beginner to recursion
Recursion is a wonderful thing.
I think findPrimes is not efficient.
Someone help me to refactor findPrimes to tail recursion if possible?
BTW, is there some more efficient way to find first n primes?
Regarding the first part of your question, if you want to write a recursive list building function tail-recursively you should pass the list of intermediate results as an extra parameter to the function. In your case this would be something like
let findPrimesTailRecursive num =
let rec aux acc num =
match num with
| 1 -> acc
| n -> aux ((nextPrime acc)::acc) (n-1)
aux [2] num
The recursive function aux gathers its results in an extra parameter conveniently called acc (as in acc-umulator). When you reach your ending condition, just spit out the accumulated result. I've wrapped the tail-recursive helper function in another function, so the function signature remains the same.
As you can see, the call to aux is the only, and therefore last, call to happen in the n <> 1 case. It's now tail-recursive and will compile into a while loop.
I've timed your version and mine, generating 2000 primes. My version is 16% faster, but still rather slow. For generating primes, I like to use an imperative array sieve. Not very functional, but very (very) fast.
An alternative is to use an extra continuation argument to make findPrimes tail recursive. This technique always works. It will avoid stack overflows, but probably won't make your code faster.
Also, I put your nextPrime function a little closer to the style I'd use.
let nextPrime list=
let rec loop n = if list |> List.filter (fun x -> x*x <= n)
|> List.forall (fun x -> n % x <> 0)
then n
else loop (n+1)
loop (1 + List.head list)
let rec findPrimesC num cont =
match num with
| 1 -> cont [2]
| n -> findPrimesC (n-1) (fun temp -> nextPrime temp :: temp |> cont)
let findPrimes num = findPrimesC num (fun res -> res)
findPrimes 10
As others have said, there's faster ways to generate primes.
Why not simply write:
let isPrime n =
if n<=1 then false
else
let m = int(sqrt (float(n)))
{2..m} |> Seq.forall (fun i->n%i<>0)
let findPrimes n =
{2..n} |> Seq.filter isPrime |> Seq.toList
or sieve (very fast):
let generatePrimes max=
let p = Array.create (max+1) true
let rec filter i step =
if i <= max then
p.[i] <- false
filter (i+step) step
{2..int (sqrt (float max))} |> Seq.iter (fun i->filter (i+i) i)
{2..max} |> Seq.filter (fun i->p.[i]) |> Seq.toArray
BTW, is there some more efficient way to find first n primes?
I described a fast arbitrary-size Sieve of Eratosthenes in F# here that accumulated its results into an ever-growing ResizeArray:
> let primes =
let a = ResizeArray[2]
let grow() =
let p0 = a.[a.Count-1]+1
let b = Array.create p0 true
for di in a do
let rec loop i =
if i<b.Length then
b.[i] <- false
loop(i+di)
let i0 = p0/di*di
loop(if i0<p0 then i0+di-p0 else i0-p0)
for i=0 to b.Length-1 do
if b.[i] then a.Add(p0+i)
fun n ->
while n >= a.Count do
grow()
a.[n];;
val primes : (int -> int)
I know that this is a bit late, and an answer was already accepted. However, I believe that a good step by step guide to making something tail recursive may be of interest to the OP or anyone else for that matter. Here are some tips that have certainly helped me out. I'm going to use a strait-forward example other than prime generation because, as others have stated, there are better ways to generate primes.
Consider a naive implementation of a count function that will create a list of integers counting down from some n. This version is not tail recursive so for long lists you will encounter a stack overflow exception:
let rec countDown = function
| 0 -> []
| n -> n :: countDown (n - 1)
(* ^
|... the cons operator is in the tail position
as such it is evaluated last. this drags
stack frames through subsequent recursive
calls *)
One way to fix this is to apply continuation passing style with a parameterized function:
let countDown' n =
let rec countDown n k =
match n with
| 0 -> k [] (* v--- this is continuation passing style *)
| n -> countDown (n - 1) (fun ns -> n :: k ns)
(* ^
|... the recursive call is now in tail position *)
countDown n (fun ns -> ns)
(* ^
|... and we initialize k with the identity function *)
Then, refactor this parameterized function into a specialized representation. Notice that the function countDown' is not actually counting down. This is an artifact of the way the continuation is built up when n > 0 and then evaluated when n = 0. If you have something like the first example and you can't figure out how to make it tail recursive, what I'm suggesting is that you write the second one and then try to optimize it to eliminate the function parameter k. That will certainly improve the readability. This is an optimization of the second example:
let countDown'' n =
let rec countDown n ns =
match n with
| 0 -> List.rev ns (* reverse so we are actually counting down again *)
| n -> countDown (n - 1) (n :: ns)
countDown n []

Remove a single non-unique value from a sequence in F#

I have a sequence of integers representing dice in F#.
In the game in question, the player has a pool of dice and can choose to play one (governed by certain rules) and keep the rest.
If, for example, a player rolls a 6, 6 and a 4 and decides to play one the sixes, is there a simple way to return a sequence with only one 6 removed?
Seq.filter (fun x -> x != 6) dice
removes all of the sixes, not just one.
Non-trivial operations on sequences are painful to work with, since they don't support pattern matching. I think the simplest solution is as follows:
let filterFirst f s =
seq {
let filtered = ref false
for a in s do
if filtered.Value = false && f a then
filtered := true
else yield a
}
So long as the mutable implementation is hidden from the client, it's still functional style ;)
If you're going to store data I would use ResizeArray instead of a Sequence. It has a wealth of functions built in such as the function you asked about. It's simply called Remove. Note: ResizeArray is an abbreviation for the CLI type List.
let test = seq [1; 2; 6; 6; 1; 0]
let a = new ResizeArray<int>(test)
a.Remove 6 |> ignore
Seq.toList a |> printf "%A"
// output
> [1; 2; 6; 1; 0]
Other data type options could be Array
let removeOneFromArray v a =
let i = Array.findIndex ((=)v) a
Array.append a.[..(i-1)] a.[(i+1)..]
or List
let removeOneFromList v l =
let rec remove acc = function
| x::xs when x = v -> List.rev acc # xs
| x::xs -> remove (x::acc) xs
| [] -> acc
remove [] l
the below code will work for a list (so not any seq but it sounds like the sequence your using could be a List)
let rec removeOne value list =
match list with
| head::tail when head = value -> tail
| head::tail -> head::(removeOne value tail)
| _ -> [] //you might wanna fail here since it didn't find value in
//the list
EDIT: code updated based on correct comment below. Thanks P
EDIT: After reading a different answer I thought that a warning would be in order. Don't use the above code for infite sequences but since I guess your players don't have infite dice that should not be a problem but for but for completeness here's an implementation that would work for (almost) any
finite sequence
let rec removeOne value seq acc =
match seq.Any() with
| true when s.First() = value -> seq.Skip(1)
| true -> seq.First()::(removeOne value seq.Skip(1))
| _ -> List.rev acc //you might wanna fail here since it didn't find value in
//the list
However I recommend using the first solution which Im confident will perform better than the latter even if you have to turn a sequence into a list first (at least for small sequences or large sequences with the soughtfor value in the end)
I don't think there is any function that would allow you to directly represent the idea that you want to remove just the first element matching the specified criteria from the list (e.g. something like Seq.removeOne).
You can implement the function in a relatively readable way using Seq.fold (if the sequence of numbers is finite):
let removeOne f l =
Seq.fold (fun (removed, res) v ->
if removed then true, v::res
elif f v then true, res
else false, v::res) (false, []) l
|> snd |> List.rev
> removeOne (fun x -> x = 6) [ 1; 2; 6; 6; 1 ];
val it : int list = [1; 2; 6; 1]
The fold function keeps some state - in this case of type bool * list<'a>. The Boolean flag represents whether we already removed some element and the list is used to accumulate the result (which has to be reversed at the end of processing).
If you need to do this for (possibly) infinite seq<int>, then you'll need to use GetEnumerator directly and implement the code as a recursive sequence expression. This is a bit uglier and it would look like this:
let removeOne f (s:seq<_>) =
// Get enumerator of the input sequence
let en = s.GetEnumerator()
let rec loop() = seq {
// Move to the next element
if en.MoveNext() then
// Is this the element to skip?
if f en.Current then
// Yes - return all remaining elements without filtering
while en.MoveNext() do
yield en.Current
else
// No - return this element and continue looping
yield en.Current
yield! loop() }
loop()
You can try this:
let rec removeFirstOccurrence item screened items =
items |> function
| h::tail -> if h = item
then screened # tail
else tail |> removeFirstOccurrence item (screened # [h])
| _ -> []
Usage:
let updated = products |> removeFirstOccurrence product []

F#: How do i split up a sequence into a sequence of sequences

Background:
I have a sequence of contiguous, time-stamped data. The data-sequence has gaps in it where the data is not contiguous. I want create a method to split the sequence up into a sequence of sequences so that each subsequence contains contiguous data (split the input-sequence at the gaps).
Constraints:
The return value must be a sequence of sequences to ensure that elements are only produced as needed (cannot use list/array/cacheing)
The solution must NOT be O(n^2), probably ruling out a Seq.take - Seq.skip pattern (cf. Brian's post)
Bonus points for a functionally idiomatic approach (since I want to become more proficient at functional programming), but it's not a requirement.
Method signature
let groupContiguousDataPoints (timeBetweenContiguousDataPoints : TimeSpan) (dataPointsWithHoles : seq<DateTime * float>) : (seq<seq< DateTime * float >>)= ...
On the face of it the problem looked trivial to me, but even employing Seq.pairwise, IEnumerator<_>, sequence comprehensions and yield statements, the solution eludes me. I am sure that this is because I still lack experience with combining F#-idioms, or possibly because there are some language-constructs that I have not yet been exposed to.
// Test data
let numbers = {1.0..1000.0}
let baseTime = DateTime.Now
let contiguousTimeStamps = seq { for n in numbers ->baseTime.AddMinutes(n)}
let dataWithOccationalHoles = Seq.zip contiguousTimeStamps numbers |> Seq.filter (fun (dateTime, num) -> num % 77.0 <> 0.0) // Has a gap in the data every 77 items
let timeBetweenContiguousValues = (new TimeSpan(0,1,0))
dataWithOccationalHoles |> groupContiguousDataPoints timeBetweenContiguousValues |> Seq.iteri (fun i sequence -> printfn "Group %d has %d data-points: Head: %f" i (Seq.length sequence) (snd(Seq.hd sequence)))
I think this does what you want
dataWithOccationalHoles
|> Seq.pairwise
|> Seq.map(fun ((time1,elem1),(time2,elem2)) -> if time2-time1 = timeBetweenContiguousValues then 0, ((time1,elem1),(time2,elem2)) else 1, ((time1,elem1),(time2,elem2)) )
|> Seq.scan(fun (indexres,(t1,e1),(t2,e2)) (index,((time1,elem1),(time2,elem2))) -> (index+indexres,(time1,elem1),(time2,elem2)) ) (0,(baseTime,-1.0),(baseTime,-1.0))
|> Seq.map( fun (index,(time1,elem1),(time2,elem2)) -> index,(time2,elem2) )
|> Seq.filter( fun (_,(_,elem)) -> elem <> -1.0)
|> PSeq.groupBy(fst)
|> Seq.map(snd>>Seq.map(snd))
Thanks for asking this cool question
I translated Alexey's Haskell to F#, but it's not pretty in F#, and still one element too eager.
I expect there is a better way, but I'll have to try again later.
let N = 20
let data = // produce some arbitrary data with holes
seq {
for x in 1..N do
if x % 4 <> 0 && x % 7 <> 0 then
printfn "producing %d" x
yield x
}
let rec GroupBy comp (input:LazyList<'a>) : LazyList<LazyList<'a>> =
LazyList.delayed (fun () ->
match input with
| LazyList.Nil -> LazyList.cons (LazyList.empty()) (LazyList.empty())
| LazyList.Cons(x,LazyList.Nil) ->
LazyList.cons (LazyList.cons x (LazyList.empty())) (LazyList.empty())
| LazyList.Cons(x,(LazyList.Cons(y,_) as xs)) ->
let groups = GroupBy comp xs
if comp x y then
LazyList.consf
(LazyList.consf x (fun () ->
let (LazyList.Cons(firstGroup,_)) = groups
firstGroup))
(fun () ->
let (LazyList.Cons(_,otherGroups)) = groups
otherGroups)
else
LazyList.cons (LazyList.cons x (LazyList.empty())) groups)
let result = data |> LazyList.of_seq |> GroupBy (fun x y -> y = x + 1)
printfn "Consuming..."
for group in result do
printfn "about to do a group"
for x in group do
printfn " %d" x
You seem to want a function that has signature
(`a -> bool) -> seq<'a> -> seq<seq<'a>>
I.e. a function and a sequence, then break up the input sequence into a sequence of sequences based on the result of the function.
Caching the values into a collection that implements IEnumerable would likely be simplest (albeit not exactly purist, but avoiding iterating the input multiple times. It will lose much of the laziness of the input):
let groupBy (fun: 'a -> bool) (input: seq) =
seq {
let cache = ref (new System.Collections.Generic.List())
for e in input do
(!cache).Add(e)
if not (fun e) then
yield !cache
cache := new System.Collections.Generic.List()
if cache.Length > 0 then
yield !cache
}
An alternative implementation could pass cache collection (as seq<'a>) to the function so it can see multiple elements to chose the break points.
A Haskell solution, because I don't know F# syntax well, but it should be easy enough to translate:
type TimeStamp = Integer -- ticks
type TimeSpan = Integer -- difference between TimeStamps
groupContiguousDataPoints :: TimeSpan -> [(TimeStamp, a)] -> [[(TimeStamp, a)]]
There is a function groupBy :: (a -> a -> Bool) -> [a] -> [[a]] in the Prelude:
The group function takes a list and returns a list of lists such that the concatenation of the result is equal to the argument. Moreover, each sublist in the result contains only equal elements. For example,
group "Mississippi" = ["M","i","ss","i","ss","i","pp","i"]
It is a special case of groupBy, which allows the programmer to supply their own equality test.
It isn't quite what we want, because it compares each element in the list with the first element of the current group, and we need to compare consecutive elements. If we had such a function groupBy1, we could write groupContiguousDataPoints easily:
groupContiguousDataPoints maxTimeDiff list = groupBy1 (\(t1, _) (t2, _) -> t2 - t1 <= maxTimeDiff) list
So let's write it!
groupBy1 :: (a -> a -> Bool) -> [a] -> [[a]]
groupBy1 _ [] = [[]]
groupBy1 _ [x] = [[x]]
groupBy1 comp (x : xs#(y : _))
| comp x y = (x : firstGroup) : otherGroups
| otherwise = [x] : groups
where groups#(firstGroup : otherGroups) = groupBy1 comp xs
UPDATE: it looks like F# doesn't let you pattern match on seq, so it isn't too easy to translate after all. However, this thread on HubFS shows a way to pattern match sequences by converting them to LazyList when needed.
UPDATE2: Haskell lists are lazy and generated as needed, so they correspond to F#'s LazyList (not to seq, because the generated data is cached (and garbage collected, of course, if you no longer hold a reference to it)).
(EDIT: This suffers from a similar problem to Brian's solution, in that iterating the outer sequence without iterating over each inner sequence will mess things up badly!)
Here's a solution that nests sequence expressions. The imperitave nature of .NET's IEnumerable<T> is pretty apparent here, which makes it a bit harder to write idiomatic F# code for this problem, but hopefully it's still clear what's going on.
let groupBy cmp (sq:seq<_>) =
let en = sq.GetEnumerator()
let rec partitions (first:option<_>) =
seq {
match first with
| Some first' -> //'
(* The following value is always overwritten;
it represents the first element of the next subsequence to output, if any *)
let next = ref None
(* This function generates a subsequence to output,
setting next appropriately as it goes *)
let rec iter item =
seq {
yield item
if (en.MoveNext()) then
let curr = en.Current
if (cmp item curr) then
yield! iter curr
else // consumed one too many - pass it on as the start of the next sequence
next := Some curr
else
next := None
}
yield iter first' (* ' generate the first sequence *)
yield! partitions !next (* recursively generate all remaining sequences *)
| None -> () // return an empty sequence if there are no more values
}
let first = if en.MoveNext() then Some en.Current else None
partitions first
let groupContiguousDataPoints (time:TimeSpan) : (seq<DateTime*_> -> _) =
groupBy (fun (t,_) (t',_) -> t' - t <= time)
Okay, trying again. Achieving the optimal amount of laziness turns out to be a bit difficult in F#... On the bright side, this is somewhat more functional than my last attempt, in that it doesn't use any ref cells.
let groupBy cmp (sq:seq<_>) =
let en = sq.GetEnumerator()
let next() = if en.MoveNext() then Some en.Current else None
(* this function returns a pair containing the first sequence and a lazy option indicating the first element in the next sequence (if any) *)
let rec seqStartingWith start =
match next() with
| Some y when cmp start y ->
let rest_next = lazy seqStartingWith y // delay evaluation until forced - stores the rest of this sequence and the start of the next one as a pair
seq { yield start; yield! fst (Lazy.force rest_next) },
lazy Lazy.force (snd (Lazy.force rest_next))
| next -> seq { yield start }, lazy next
let rec iter start =
seq {
match (Lazy.force start) with
| None -> ()
| Some start ->
let (first,next) = seqStartingWith start
yield first
yield! iter next
}
Seq.cache (iter (lazy next()))
Below is some code that does what I think you want. It is not idiomatic F#.
(It may be similar to Brian's answer, though I can't tell because I'm not familiar with the LazyList semantics.)
But it doesn't exactly match your test specification: Seq.length enumerates its entire input. Your "test code" calls Seq.length and then calls Seq.hd. That will generate an enumerator twice, and since there is no caching, things get messed up. I'm not sure if there is any clean way to allow multiple enumerators without caching. Frankly, seq<seq<'a>> may not be the best data structure for this problem.
Anyway, here's the code:
type State<'a> = Unstarted | InnerOkay of 'a | NeedNewInner of 'a | Finished
// f() = true means the neighbors should be kept together
// f() = false means they should be split
let split_up (f : 'a -> 'a -> bool) (input : seq<'a>) =
// simple unfold that assumes f captured a mutable variable
let iter f = Seq.unfold (fun _ ->
match f() with
| Some(x) -> Some(x,())
| None -> None) ()
seq {
let state = ref (Unstarted)
use ie = input.GetEnumerator()
let innerMoveNext() =
match !state with
| Unstarted ->
if ie.MoveNext()
then let cur = ie.Current
state := InnerOkay(cur); Some(cur)
else state := Finished; None
| InnerOkay(last) ->
if ie.MoveNext()
then let cur = ie.Current
if f last cur
then state := InnerOkay(cur); Some(cur)
else state := NeedNewInner(cur); None
else state := Finished; None
| NeedNewInner(last) -> state := InnerOkay(last); Some(last)
| Finished -> None
let outerMoveNext() =
match !state with
| Unstarted | NeedNewInner(_) -> Some(iter innerMoveNext)
| InnerOkay(_) -> failwith "Move to next inner seq when current is active: undefined behavior."
| Finished -> None
yield! iter outerMoveNext }
open System
let groupContigs (contigTime : TimeSpan) (holey : seq<DateTime * int>) =
split_up (fun (t1,_) (t2,_) -> (t2 - t1) <= contigTime) holey
// Test data
let numbers = {1 .. 15}
let contiguousTimeStamps =
let baseTime = DateTime.Now
seq { for n in numbers -> baseTime.AddMinutes(float n)}
let holeyData =
Seq.zip contiguousTimeStamps numbers
|> Seq.filter (fun (dateTime, num) -> num % 7 <> 0)
let grouped_data = groupContigs (new TimeSpan(0,1,0)) holeyData
printfn "Consuming..."
for group in grouped_data do
printfn "about to do a group"
for x in group do
printfn " %A" x
Ok, here's an answer I'm not unhappy with.
(EDIT: I am unhappy - it's wrong! No time to try to fix right now though.)
It uses a bit of imperative state, but it is not too difficult to follow (provided you recall that '!' is the F# dereference operator, and not 'not'). It is as lazy as possible, and takes a seq as input and returns a seq of seqs as output.
let N = 20
let data = // produce some arbitrary data with holes
seq {
for x in 1..N do
if x % 4 <> 0 && x % 7 <> 0 then
printfn "producing %d" x
yield x
}
let rec GroupBy comp (input:seq<_>) = seq {
let doneWithThisGroup = ref false
let areMore = ref true
use e = input.GetEnumerator()
let Next() = areMore := e.MoveNext(); !areMore
// deal with length 0 or 1, seed 'prev'
if not(e.MoveNext()) then () else
let prev = ref e.Current
while !areMore do
yield seq {
while not(!doneWithThisGroup) do
if Next() then
let next = e.Current
doneWithThisGroup := not(comp !prev next)
yield !prev
prev := next
else
// end of list, yield final value
yield !prev
doneWithThisGroup := true }
doneWithThisGroup := false }
let result = data |> GroupBy (fun x y -> y = x + 1)
printfn "Consuming..."
for group in result do
printfn "about to do a group"
for x in group do
printfn " %d" x

Resources