I have a naive implementation of a gameloop
let gameLoop gamestate =
let rec innerLoop prev gamestate =
let now = getTicks()
let delta = now - prev
gamestate
|> readInput delta
|> update delta
|> render delta
|> innerLoop delta
innerLoop 0L gamestate
This implementation throws a stackoverflowexception. In my mind this should be tail recursive. I could make a work around like this
let gameLoop gamestate =
let rec innerLoop prev gamestate =
let now = getTicks()
let delta = now - prev
let newState = gamestate
|> readInput delta
|> update delta
|> render delta
innerLoop now newState
innerLoop 0L gamestate
So my question is why the first code example throws a stackoverflow exception.
I think the answer is the same as in the thread Vandroiy links: when you have
a
|> f b
then in debug mode the compiler may compile this like a very literal interpretation of
(f b) a
and explicitly calculate f b in one step and apply it to a in a second step. The call with argument a is still a tail call, but if the compiler doesn't emit the tail. opcode prefix (because tailcalls are turned off, as they are by default in debug mode), then you'll grow the stack with the explicit call and eventually get a stack overflow.
On the other hand, if you write
f b a
directly then this doesn't happen: the compiler does not partially apply f, and instead will recognize that this is a direct recursive call and optimize it into a loop (even in debug mode).
I think this is the explanation, though I encourage F# compiler experts to weigh in if I'm off-base:
The first example is not tail-recursive because the expression in tail position is a call to |>, not a call to innerLoop.
Recalling that |> is defined as
let (|>) x f = f x
if we desugar the pipeline syntax a little bit, when you call
gamestate
|> readInput delta
|> update delta
|> render delta
|> innerLoop delta
you're effectively calling:
|> (innerLoop delta) (|> (render delta) (|> (update delta) (|> (readInput delta) gamestate)))
as your body expression in the recursive function.
The infix notation obscures this a bit, making it look like innerLoop is in tail position.
Related
I implemented newton's method to find roots of a function. I'm wondering if i can optimize the code as make it more time and space efficient and visually enlightening. Here I used a mutable variable, but I'm wondering if we can do it without. Here is the question and the code that I wrote:
open System
let newton (f:(float->float),x0: float,tol:float, dx:float)=
let mutable x=x0
while Math.Abs (f x) >= tol do
//compute derivative
let fderivative = (f x+dx-f x)/dx
x<-x-(f x)/fderivative
x
You can use recursion to avoid mutation:
let newton (f: float -> float,x0,tol,dx)=
let rec loop x =
if abs (f x) < tol then x
else
let f' = (f x+dx-f x)/dx
x - f x / f' |> loop
loop x0
What is the most elegant way to implement dynamic programming algorithms that solve problems with overlapping subproblems? In imperative programming one would usually create an array indexed (at least in one dimension) by the size of the problem, and then the algorithm would start from the simplest problems and work towards more complicated once, using the results already computed.
The simplest example I can think of is computing the Nth Fibonacci number:
int Fibonacci(int N)
{
var F = new int[N+1];
F[0]=1;
F[1]=1;
for(int i=2; i<=N; i++)
{
F[i]=F[i-1]+F[i-2];
}
return F[N];
}
I know you can implement the same thing in F#, but I am looking for a nice functional solution (which is O(N) as well obviously).
One technique that is quite useful for dynamic programming is called memoization. For more details, see for example blog post by Don Syme or introduction by Matthew Podwysocki.
The idea is that you write (a naive) recursive function and then add cache that stores previous results. This lets you write the function in a usual functional style, but get the performance of algorithm implemented using dynamic programming.
For example, a naive (inefficient) function for calculating Fibonacci number looks like this:
let rec fibs n =
if n < 1 then 1 else
(fibs (n - 1)) + (fibs (n - 2))
This is inefficient, because when you call fibs 3, it will call fibs 1 three times (and many more times if you call, for example, fibs 6). The idea behind memoization is that we write a cache that stores the result of fib 1 and fib 2, and so on, so repeated calls will just pick the pre-calculated value from the cache.
A generic function that does the memoization can be written like this:
open System.Collections.Generic
let memoize(f) =
// Create (mutable) cache that is used for storing results of
// for function arguments that were already calculated.
let cache = new Dictionary<_, _>()
(fun x ->
// The returned function first performs a cache lookup
let succ, v = cache.TryGetValue(x)
if succ then v else
// If value was not found, calculate & cache it
let v = f(x)
cache.Add(x, v)
v)
To write more efficient Fibonacci function, we can now call memoize and give it the function that performs the calculation as an argument:
let rec fibs = memoize (fun n ->
if n < 1 then 1 else
(fibs (n - 1)) + (fibs (n - 2)))
Note that this is a recursive value - the body of the function calls the memoized fibs function.
Tomas's answer is a good general approach. In more specific circumstances, there may be other techniques that work well - for example, in your Fibonacci case you really only need a finite amount of state (the previous 2 numbers), not all of the previously calculated values. Therefore you can do something like this:
let fibs = Seq.unfold (fun (i,j) -> Some(i,(j,i+j))) (1,1)
let fib n = Seq.nth n fibs
You could also do this more directly (without using Seq.unfold):
let fib =
let rec loop i j = function
| 0 -> i
| n -> loop j (i+j) (n-1)
loop 1 1
let fibs =
(1I,1I)
|> Seq.unfold (fun (n0, n1) -> Some (n0 , (n1, n0 + n1)))
|> Seq.cache
Taking inspiration from Tomas' answer here, and in an attempt to resolve the warning in my comment on said answer, I propose the following updated solution.
open System.Collections.Generic
let fib n =
let cache = new Dictionary<_, _>()
let memoize f c =
let succ, v = cache.TryGetValue c
if succ then v else
let v = f c
cache.Add(c, v)
v
let rec inner n =
match n with
| 1
| 2 -> bigint n
| n ->
memoize inner (n - 1) + memoize inner (n - 2)
inner n
This solution internalizes the memoization, and while doing so, allows the definitions of fib and inner to be functions, instead of fib being a recursive object, which allows the compiler to (I think) properly reason about the viability of the function calls.
I also return a bigint instead of an int, as int quickly overflows with even a small of n.
Edit: I should mention, however, that this solution still runs into stack overflow exceptions with sufficiently large values of n.
I'm trying to come up with an Rx Builder to use Reactive Extension within the F# Computation Expression syntax. How do I fix it so that it doesnt blow the stack? Like the Seq example below.
And is there any plans to provide an implementation of the RxBuilder as part of the Reactive Extensions or as part of future versions of the .NET Framework ?
open System
open System.Linq
open System.Reactive.Linq
type rxBuilder() =
member this.Delay f = Observable.Defer f
member this.Combine (xs: IObservable<_>, ys : IObservable<_>) =
Observable.merge xs ys
member this.Yield x = Observable.Return x
member this.YieldFrom (xs:IObservable<_>) = xs
let rx = rxBuilder()
let rec f x = seq { yield x
yield! f (x + 1) }
let rec g x = rx { yield x
yield! g (x + 1) }
//do f 5 |> Seq.iter (printfn "%A")
do g 5 |> Observable.subscribe (printfn "%A") |> ignore
do System.Console.ReadLine() |> ignore
A short answer is that Rx Framework doesn't support generating observables using a recursive pattern like this, so it cannot be easily done. The Combine operation that is used for F# sequences needs some special handling that observables do not provide. The Rx Framework probably expects that you'll generate observables using Observable.Generate and then use LINQ queries/F# computation builder to process them.
Anyway, here are some thoughts -
First of all, you need to replace Observable.merge with Observable.Concat. The first one runs both observables in parallel, while the second first yields all values from the first observable and then produces values from the second observable. After this change, the snippet will at least print ~800 numbers before the stack overflow.
The reason for the stack overflow is that Concat creates an observable that calls Concat to create another observable that calls Concat etc. One way to solve this is to add some synchronization. If you're using Windows Forms, then you can modify Delay so that it schedules the observable on the GUI thread (which discards the current stack). Here is a sketch:
type RxBuilder() =
member this.Delay f =
let sync = System.Threading.SynchronizationContext.Current
let res = Observable.Defer f
{ new IObservable<_> with
member x.Subscribe(a) =
sync.Post( (fun _ -> res.Subscribe(a) |> ignore), null)
// Note: This is wrong, but we cannot easily get the IDisposable here
null }
member this.Combine (xs, ys) = Observable.Concat(xs, ys)
member this.Yield x = Observable.Return x
member this.YieldFrom (xs:IObservable<_>) = xs
To implement this properly, you would have to write your own Concat method, which is quite complicated. The idea would be that:
Concat returns some special type e.g. IConcatenatedObservable
When the method is called recursively you'll create a chain of IConcatenatedObservable that reference each other
The Concat method will look for this chain and when there are e.g. three objects, it will drop the middle one (to always keep chain of length at most 2).
That's a bit too complex for a StackOverflow answer, but it may be a useful feedback for the Rx team.
Notice this has been fixed in Rx v2.0 (as mentioned here already), more generally for all of the sequencing operators (Concat, Catch, OnErrorResumeNext), as well as the imperative operators (If, While, etc.).
Basically, you can think of this class of operators as doing a subscribe to another sequence in a terminal observer message (e.g. Concat subscribes to the next sequence upon receiving the current one's OnCompleted message), which is where the tail recursion analogy comes in.
In Rx v2.0, all of the tail-recursive subscriptions are flattened into a queue-like data structure for processing one at a time, talking to the downstream observer. This avoids the unbounded growth of observers talking to each other for successive sequence subscriptions.
This has been fixed in Rx 2.0 Beta. And here's a test.
What about something like this?
type rxBuilder() =
member this.Delay (f : unit -> 'a IObservable) =
{ new IObservable<_> with
member this.Subscribe obv = (f()).Subscribe obv }
member this.Combine (xs:'a IObservable, ys: 'a IObservable) =
{ new IObservable<_> with
member this.Subscribe obv = xs.Subscribe obv ;
ys.Subscribe obv }
member this.Yield x = Observable.Return x
member this.YieldFrom xs = xs
let rx = rxBuilder()
let rec f x = rx { yield x
yield! f (x + 1) }
do f 5 |> Observable.subscribe (fun x -> Console.WriteLine x) |> ignore
do System.Console.ReadLine() |> ignore
http://rxbuilder.codeplex.com/ (created for the purpose of experimenting with RxBuilder)
The xs disposable is not wired up. As soon as I try to wire up the disposable it goes back to blowing up the stack.
If we remove the syntactic sugar from this computation expression (aka Monad) we will have:
let rec g x = Observable.Defer (fun () -> Observable.merge(Observable.Return x, g (x + 1) )
Or in C#:
public static IObservable<int> g(int x)
{
return Observable.Defer<int>(() =>
{
return Observable.Merge(Observable.Return(x), g(x + 1));
});
}
Which is definitely not tail recursive. I think if you can make it tail recursive then it would probably solve your problem
I am trying to learn F# so I paid a visit to Project Euler and I am currently working on Problem 3.
The prime factors of 13195 are 5, 7,
13 and 29.
What is the largest prime
factor of the number 600851475143?
Some things to consider:
My first priority is to learn good functional habits.
My second priority is I would like it to be fast and efficient.
Within the following code I have marked the section this question is regarding.
let isPrime(n:int64) =
let rec check(i:int64) =
i > n / 2L or (n % i <> 0L && check(i + 1L))
check(2L)
let greatestPrimeFactor(n:int64) =
let nextPrime(prime:int64):int64 =
seq { for i = prime + 1L to System.Int64.MaxValue do if isPrime(i) then yield i }
|> Seq.skipWhile(fun v -> n % v <> 0L)
|> Seq.hd
let rec findNextPrimeFactor(number:int64, prime:int64):int64 =
if number = 1L then prime else
//************* No variable
(fun p -> findNextPrimeFactor(number / p, p))(nextPrime(prime))
//*************
//************* Variable
let p = nextPrime(prime)
findNextPrimeFactor(number / p, p)
//*************
findNextPrimeFactor(n, 2L)
Update
Based off some of the feedback I have refactored the code to be 10 times faster.
module Problem3
module private Internal =
let execute(number:int64):int64 =
let rec isPrime(value:int64, current:int64) =
current > value / 2L or (value % current <> 0L && isPrime(value, current + 1L))
let rec nextPrime(prime:int64):int64 =
if number % prime = 0L && isPrime(prime, 2L) then prime else nextPrime(prime + 1L)
let rec greatestPrimeFactor(current:int64, prime:int64):int64 =
if current = 1L then prime else nextPrime(prime + 1L) |> fun p -> greatestPrimeFactor(current / p, p)
greatestPrimeFactor(number, 2L)
let execute() = Internal.execute(600851475143L)
Update
I would like to thank everyone for there advice. This latest version is a compilation of all the advice I received.
module Problem3
module private Internal =
let largestPrimeFactor number =
let rec isPrime value current =
current > value / 2L || (value % current <> 0L && isPrime value (current + 1L))
let rec nextPrime value =
if number % value = 0L && isPrime value 2L then value else nextPrime (value + 1L)
let rec find current prime =
match current / prime with
| 1L -> prime
| current -> nextPrime (prime + 1L) |> find current
find number (nextPrime 2L)
let execute() = Internal.largestPrimeFactor 600851475143L
Functional programming becomes easier and more automatic with practice, so don't sweat it if you don't get it absolutely right on the first try.
With that in mind, let's take your sample code:
let rec findNextPrimeFactor(number:int64, prime:int64):int64 =
if number = 1L then prime else
//************* No variable
(fun p -> findNextPrimeFactor(number / p, p))(nextPrime(prime))
//*************
//************* Variable
let p = nextPrime(prime)
findNextPrimeFactor(number / p, p)
//*************
Your no variable version is just weird, don't use it. I like your version with the explicit let binding.
Another way to write it would be:
nextPrime(prime) |> fun p -> findNextPrimeFactor(number / p, p)
Its ok and occasionally useful to write it like this, but still comes across as a little weird. Most of the time, we use |> to curry values without needing to name our variables (in "pointfree" style). Try to anticipate how your function will be used, and if possible, re-write it so you can use it with the pipe operator without explicit declared variables. For example:
let rec findNextPrimeFactor number prime =
match number / prime with
| 1L -> prime
| number' -> nextPrime(prime) |> findNextPrimeFactor number'
No more named args :)
Ok, now that we have that out of the way, let's look at your isPrime function:
let isPrime(n:int64) =
let rec check(i:int64) =
i > n / 2L or (n % i <> 0L && check(i + 1L))
check(2L)
You've probably heard to use recursion instead of loops, and that much is right. But, wherever possible, you should abstract away recursion with folds, maps, or higher order functions. Two reasons for this:
its a little more readable, and
improperly written recursion will result in a stack overflow. For example, your function is not tail recursive, so it'll blow up on large values of n.
I'd rewrite isPrime like this:
let isPrime n = seq { 2L .. n / 2L } |> Seq.exists (fun i -> n % i = 0L) |> not
Most of the time, if you can abstract away your explicit looping, then you're just applying transformations to your input sequence until you get your results:
let maxFactor n =
seq { 2L .. n - 1L } // test inputs
|> Seq.filter isPrime // primes
|> Seq.filter (fun x -> n % x = 0L) // factors
|> Seq.max // result
We don't even have intermediate variables in this version. Coolness!
My second priority is I would like it
to be fast and efficient.
Most of the time, F# is going to be pretty comparable with C# in terms of speed, or it's going to be "fast enough". If you find your code takes a long time to execute, it probably means you're using the wrong data structure or a bad algorithm. For a concrete example, read the comments on this question.
So, the code I've written is "elegant" in the sense that its concise, gives the correct results, and doesn't rely on any trickery. Unfortunately, its not very fast. For start:
it uses trial division to create a sequence of primes, when the Sieve of Eratosthenes would be much faster. [Edit: I wrote a somewhat naive version of this sieve which didn't work for numbers larger than Int32.MaxValue, so I've removed the code.]
read Wikipedia's article on the prime counting function, it'll give you pointers on calculating the first n primes as well as estimating the upper and lower bounds for the nth prime.
[Edit: I included some code with a somewhat naive implementation of a sieve of eratosthenes. It only works for inputs less than int32.MaxValue, so it probably isn't suitable for project euler.]
Concerning "good functional habit" or rather good practice I see three minor things. Using the yield in your sequence is a little harder to read than just filter. Unnecessary type annotations in a type inferred language leads to difficult refactoring and makes the code harder to read. Don't go overboard and try to remove every type annotation though if you're finding it difficult. Lastly making a lambda function which only takes a value to use as a temp variable reduces readability.
As far as personal style goes I prefer more spaces and only using tupled arguments when the data makes sense being grouped together.
I'd write your original code like this.
let isPrime n =
let rec check i =
i > n / 2L || (n % i <> 0L && check (i + 1L))
check 2L
let greatestPrimeFactor n =
let nextPrime prime =
seq {prime + 1L .. System.Int64.MaxValue}
|> Seq.filter isPrime
|> Seq.skipWhile (fun v -> n % v <> 0L)
|> Seq.head
let rec findNextPrimeFactor number prime =
if number = 1L then
prime
else
let p = nextPrime(prime)
findNextPrimeFactor (number / p) p
findNextPrimeFactor n 2L
Your updated code is optimal for your approach. You would have to use a different algorithm like Yin Zhu answer to go faster. I wrote a test to check to see if F# makes the "check" function tail recursive and it does.
the variable p is actually a name binding, not a variable. Using name binding is not a bad style. And it is more readable. The lazy style of nextPrime is good, and it actually prime-test each number only once during the whole program.
My Solution
let problem3 =
let num = 600851475143L
let rec findMax (n:int64) (i:int64) =
if n=i || n<i then
n
elif n%i=0L then
findMax (n/i) i
else
findMax n (i+1L)
findMax num 2L
I basically divides num from 2, 3, 4.. and don't consider any prime numbers. Because if we divides all 2 from num, then we won't be able to divide it by 4,8, etc.
on this number, my solution is quicker:
> greatestPrimeFactor 600851475143L;;
Real: 00:00:01.110, CPU: 00:00:00.702, GC gen0: 1, gen1: 1, gen2: 0
val it : int64 = 6857L
>
Real: 00:00:00.001, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0
val problem3 : int64 = 6857L
I think that the code with the temporary binding is significantly easier to read. It's pretty unusual to create an anonymous function and then immediately apply it to a value as you do in the other case. If you really want to avoid using a temporary value, I think that the most idiomatic way to do that in F# would be to use the (|>) operator to pipe the value into the anonymous function, but I still think that this isn't quite as readable.
I have this "learning code" I wrote for the morris seq in f# that suffers from stack overflow that I don't know how to avoid. "morris" returns an infinite sequence of "see and say" sequences (i.e., {{1}, {1,1}, {2,1}, {1,2,1,1}, {1,1,1,2,2,1}, {3,1,2,2,1,1},...}).
let printList l =
Seq.iter (fun n -> printf "%i" n) l
printfn ""
let rec morris s =
let next str = seq {
let cnt = ref 1 // Stack overflow is below when enumerating
for cur in [|0|] |> Seq.append str |> Seq.windowed 2 do
if cur.[0] <> cur.[1] then
yield!( [!cnt ; cur.[0]] )
cnt := 0
incr cnt
}
seq {
yield s
yield! morris (next s) // tail recursion, no stack overflow
}
// "main"
// Print the nth iteration
let _ = [1] |> morris |> Seq.nth 3125 |> printList
You can pick off the nth iteration using Seq.nth but you can only get so far before you hit a stack overflow. The one bit of recursion I have is tail recursion and it in essence builds a linked set of enumerators. That's not where the problem is. It's when "enum" is called on the say the 4000th sequence. Note that's with F# 1.9.6.16, the previous version topped out above 14000). It's because the way the linked sequences are resolved. The sequences are lazy and so the "recursion" is lazy. That is, seq n calls seq n-1 which calls seq n-2 and so forth to get the first item (the very first # is the worst case).
I understand that [|0|] |> Seq.append str |> Seq.windowed 2, is making my problem worse and I could triple the # I could generate if I eliminated that. Practically speaking the code works well enough. The 3125th iteration of morris would be over 10^359 characters in length.
The problem I'm really trying to solve is how to retain the lazy eval and have a no limit based on stack size for the iteration I can pick off. I'm looking for the proper F# idiom to make the limit based on memory size.
Update Oct '10
After learning F# a bit better, a tiny bit of Haskell, thinking & investigating this problem for over year, I finally can answer my own question. But as always with difficult problems, the problem starts with it being the wrong question. The problem isn't sequences of sequences - it's really because of a recursively defined sequence. My functional programming skills are a little better now and so it's easier to see what's going on with the version below, which still gets a stackoverflow
let next str =
Seq.append str [0]
|> Seq.pairwise
|> Seq.scan (fun (n,_) (c,v) ->
if (c = v) then (n+1,Seq.empty)
else (1,Seq.ofList [n;c]) ) (1,Seq.empty)
|> Seq.collect snd
let morris = Seq.unfold(fun sq -> Some(sq,next sq))
That basicially creates a really long chain of Seq processing function calls to generate the sequnces. The Seq module that comes with F# is what can't follow the chain without using the stack. There's an optimization it uses for append and recursively defined sequences, but that optimization only works if the recursion is implementing an append.
So this will work
let rec ints n = seq { yield n; yield! ints (n+1) }
printf "%A" (ints 0 |> Seq.nth 100000);;
And this one will get a stackoverflow.
let rec ints n = seq { yield n; yield! (ints (n+1)|> Seq.map id) }
printf "%A" (ints 0 |> Seq.nth 100000);;
To prove the F# libary was the issue, I wrote my own Seq module that implemented append, pairwise, scan and collect using continutions and now I can begin generating and printing out the 50,000 seq without a problem (it'll never finish since it's over 10^5697 digits long).
Some additional notes:
Continuations were the idiom I was looking for, but in this case, they had to go into the F# library, not my code. I learned about continuations in F# from Tomas Petricek's Real-World Functional Programming book.
The lazy list answer that I accepted held the other idiom; lazy evaluation. In my rewritten library, I also had to leverage the lazy type to avoid stackoverflow.
The lazy list version sorta of works by luck (maybe by design but that's beyond my current ability to determine) - the active-pattern matching it uses while it's constructing and iterating causes the lists to calculate values before the required recursion gets too deep, so it's lazy, but not so lazy it needs continuations to avoid stackoverflow. For example, by the time the 2nd sequence needs a digit from the 1st sequence, it's already been calculated. In other words, the LL version is not strictly JIT lazy for sequence generation, only list management.
You should definitely check out
http://research.microsoft.com/en-us/um/cambridge/projects/fsharp/manual/FSharp.PowerPack/Microsoft.FSharp.Collections.LazyList.html
but I will try to post a more comprehensive answer later.
UPDATE
Ok, a solution is below. It represents the Morris sequence as a LazyList of LazyLists of int, since I presume you want it to be lazy in 'both directions'.
The F# LazyList (in the FSharp.PowerPack.dll) has three useful properties:
it is lazy (evaluation of the nth element will not happen until it is first demanded)
it does not recompute (re-evaluation of the nth element on the same object instance will not recompute it - it caches each element after it's first computed)
you can 'forget' prefixes (as you 'tail' into the list, the no-longer-referenced prefix is available for garbage collection)
The first property is common with seq (IEnumerable), but the other two are unique to LazyList and very useful for computational problems such as the one posed in this question.
Without further ado, the code:
// print a lazy list up to some max depth
let rec PrintList n ll =
match n with
| 0 -> printfn ""
| _ -> match ll with
| LazyList.Nil -> printfn ""
| LazyList.Cons(x,xs) ->
printf "%d" x
PrintList (n-1) xs
// NextMorris : LazyList<int> -> LazyList<int>
let rec NextMorris (LazyList.Cons(cur,rest)) =
let count = ref 1
let ll = ref rest
while LazyList.nonempty !ll && (LazyList.hd !ll) = cur do
ll := LazyList.tl !ll
incr count
LazyList.cons !count
(LazyList.consf cur (fun() ->
if LazyList.nonempty !ll then
NextMorris !ll
else
LazyList.empty()))
// Morris : LazyList<int> -> LazyList<LazyList<int>>
let Morris s =
let rec MakeMorris ll =
LazyList.consf ll (fun () ->
let next = NextMorris ll
MakeMorris next
)
MakeMorris s
// "main"
// Print the nth iteration, up to a certain depth
[1] |> LazyList.of_list |> Morris |> Seq.nth 3125 |> PrintList 10
[1] |> LazyList.of_list |> Morris |> Seq.nth 3126 |> PrintList 10
[1] |> LazyList.of_list |> Morris |> Seq.nth 100000 |> PrintList 35
[1] |> LazyList.of_list |> Morris |> Seq.nth 100001 |> PrintList 35
UPDATE2
If you just want to count, that's fine too:
let LLLength ll =
let rec Loop ll acc =
match ll with
| LazyList.Cons(_,rest) -> Loop rest (acc+1N)
| _ -> acc
Loop ll 0N
let Main() =
// don't do line below, it leaks
//let hundredth = [1] |> LazyList.of_list |> Morris |> Seq.nth 100
// if we only want to count length, make sure we throw away the only
// copy as we traverse it to count
[1] |> LazyList.of_list |> Morris |> Seq.nth 100
|> LLLength |> printfn "%A"
Main()
The memory usage stays flat (under 16M on my box)... hasn't finished running yet, but I computed the 55th length fast, even on my slow box, so I think this should work just fine. Note also that I used 'bignum's for the length, since I think this will overflow an 'int'.
I believe there are two main problems here:
Laziness is very inefficient so you can expect a lazy functional implementation to run orders of magnitude slower. For example, the Haskell implementation described here is 2,400× slower than the F# I give below. If you want a workaround, your best bet is probably to amortize the computations by bunching them together into eager batches where the batches are produced on-demand.
The Seq.append function is actually calling into C# code from IEnumerable and, consequently, its tail call doesn't get eliminated and you leak a bit more stack space every time you go through it. This shows up when you come to enumerate over the sequence.
The following is over 80× faster than your implementation at computing the length of the 50th subsequence but perhaps it is not lazy enough for you:
let next (xs: ResizeArray<_>) =
let ys = ResizeArray()
let add n x =
if n > 0 then
ys.Add n
ys.Add x
let mutable n = 0
let mutable x = 0
for i=0 to xs.Count-1 do
let x' = xs.[i]
if x=x' then
n <- n + 1
else
add n x
n <- 1
x <- x'
add n x
ys
let morris =
Seq.unfold (fun xs -> Some(xs, next xs)) (ResizeArray [1])
The core of this function is a fold over a ResizeArray that could be factored out and used functionally without too much performance degradation if you used a struct as the accumulator.
Just save the previous element that you looked for.
let morris2 data = seq {
let cnt = ref 0
let prev = ref (data |> Seq.nth 0)
for cur in data do
if cur <> !prev then
yield! [!cnt; !prev]
cnt := 1
prev := cur
else
cnt := !cnt + 1
yield! [!cnt; !prev]
}
let rec morrisSeq2 cur = seq {
yield cur
yield! morrisSeq2 (morris2 cur)
}