I have some code in F# that works fine under .net but overflows the stack under Mono. A related issue is that it seems to do so long before the stack space supposedly available to it runs out (it is started with System.Threading.Thread (ts, 1000000000)). As far as I can tell, the fold it dies in is tail-recursive and the stack trace looks as if tail-optimization is not being done. I am running 3.2.1 with --optimize=tailc.
Does somebody please know exactly what kinds of tail calls remove the calling stack and which do not? Or alternatively how to allocate more stack? Many thanks.
I am aware of Tailcall elimination in Mono
EDIT: here is an outline of the code as requested in the comments. It is a part of a fold over a large data structure, but the failing stacktrace has just mapk and myfold on it.
let rec myfold f x k =
let rec mapk xs k =
match xs with
[] -> k []
| x::xs -> mapk xs (fun xs' -> myfold f x (fun x' -> (x' :: xs') |> k))
...
mapk (...) ( ... >> k)
As far as I know, --optimize=tailc isn't a supported F# compiler flag.
I don't think there's a way to enable/disable tailcall-optimization support in Mono (from the command-line, anyway); the F# compiler flag to enable tail-call optimizations is --tailcalls+, but according to Compiler Options (F#) that's on by default.
I think your best choices to get this resolved are:
File a bug report with Xamarin
Go on the #monodev IRC channel (on irc.gnome.org) and see if one of the developers/contributors there can help you out.
Related
I was trying to write a generic mapFoldWhile function, which is just mapFold but requires the state to be an option and stops as soon as it encounters a None state.
I don't want to use mapFold because it will transform the entire list, but I want it to stop as soon as an invalid state (i.e. None) is found.
This was myfirst attempt:
let mapFoldWhile (f : 'State option -> 'T -> 'Result * 'State option) (state : 'State option) (list : 'T list) =
let rec mapRec f state list results =
match list with
| [] -> (List.rev results, state)
| item :: tail ->
let (result, newState) = f state item
match newState with
| Some x -> mapRec f newState tail (result :: results)
| None -> ([], None)
mapRec f state list []
The List.rev irked me, since the point of the exercise was to exit early and constructing a new list ought to be even slower.
So I looked up what F#'s very own map does, which was:
let map f list = Microsoft.FSharp.Primitives.Basics.List.map f list
The ominous Microsoft.FSharp.Primitives.Basics.List.map can be found here and looks like this:
let map f x =
match x with
| [] -> []
| [h] -> [f h]
| (h::t) ->
let cons = freshConsNoTail (f h)
mapToFreshConsTail cons f t
cons
The consNoTail stuff is also in this file:
// optimized mutation-based implementation. This code is only valid in fslib, where mutation of private
// tail cons cells is permitted in carefully written library code.
let inline setFreshConsTail cons t = cons.(::).1 <- t
let inline freshConsNoTail h = h :: (# "ldnull" : 'T list #)
So I guess it turns out that F#'s immutable lists are actually mutable because performance? I'm a bit worried about this, having used the prepend-then-reverse list approach as I thought it was the "way to go" in F#.
I'm not very experienced with F# or functional programming in general, so maybe (probably) the whole idea of creating a new mapFoldWhile function is the wrong thing to do, but then what am I to do instead?
I often find myself in situations where I need to "exit early" because a collection item is "invalid" and I know that I don't have to look at the rest. I'm using List.pick or Seq.takeWhile in some cases, but in other instances I need to do more (mapFold).
Is there an efficient solution to this kind of problem (mapFoldWhile in particular and "exit early" in general) with functional programming concepts, or do I have to switch to an imperative solution / use a Collections.Generics.List?
In most cases, using List.rev is a perfectly sufficient solution.
You are right that the F# core library uses mutation and other dirty hacks to squeeze some more performance out of the F# list operations, but I think the micro-optimizations done there are not particularly good example. F# list functions are used almost everywhere so it might be a good trade-off, but I would not follow it in most situations.
Running your function with the following:
let l = [ 1 .. 1000000 ]
#time
mapFoldWhile (fun s v -> 0, s) (Some 1) l
I get ~240ms on the second line when I run the function without changes. When I just drop List.rev (so that it returns the data in the other order), I get around ~190ms. If you are really calling the function frequently enough that this matters, then you'd have to use mutation (actually, your own mutable list type), but I think that is rarely worth it.
For general "exit early" problems, you can often write the code as a composition of Seq.scan and Seq.takeWhile. For example, say you want to sum numbers from a sequence until you reach 1000. You can write:
input
|> Seq.scan (fun sum v -> v + sum) 0
|> Seq.takeWhile (fun sum -> sum < 1000)
Using Seq.scan generates a sequence of sums that is over the whole input, but since this is lazily generated, using Seq.takeWhile stops the computation as soon as the exit condition happens.
I'm currently trying to learn f# using the book Real-World Functional Programming by Petricek and Skeet (2010) but have been encountering problems when using continuations to avoid stack overflow.
The problem that I have been encountering is that my code using continuations works perfectly when launched in the f# interactive, but still causes stack overflow when placing the code in the program.fs file and then launching it through the debugger in Visual Studio.
It is unclear to me why this happens, and would very much appreciate if anyone could give me an explanation to why this happens.
In case the version of Visual Studio is relevant, I am using:
Visual Studio Ultimate 2012
Version 11.0.61030.00 Update 4
The .Net framework used is:
Version. 4.5.51641
The code presented in the book that is causing this problem is presented below:
open System
let rand = new Random()
//A tree is either a leaf with a value or a node that contains two children
type IntTree =
| Leaf of int
| Node of IntTree * IntTree
//A list of that will decide all leaf values
let numbers2 = List.init 1000000 (fun _ -> rand.Next(-50,51))
///Creates an imbalanced tree with 1000001 leafs.
let imbalancedTree2 =
numbers2 |> List.fold (fun currentTree num ->
Node(Leaf(num), currentTree)) (Leaf(0))
//Sums all leafs in a tree by continuation and will execute the inserted function with the total
//sum as argument once all values have been summed.
let rec sumTreeCont tree cont =
match tree with
| Leaf(num) -> cont(num)
| Node(left, right) ->
sumTreeCont left (fun leftSum ->
sumTreeCont right (fun rightSum ->
cont(leftSum + rightSum)))
//Sums the imbalanced tree using the sumTreeCont function
let sumOfTree = sumTreeCont imbalancedTree2 (fun x -> x)
Thanks in advance!
//Tigerstrom
If you are running the program in Debug mode, then the default Visual Studio project setting disables tail calls. The main reason is that, with tail calls enabled, you do not get very useful information in the call stack (which makes debugging harder).
To fix this, you can go to your project options and check "Generate tail calls" on the "Build" page. In release mode, this is enabled by default.
I am not sure why this happens
//fine, type of xs is Set<int>
let e = Seq.scan (fun xs x -> Set.add x xs) Set.empty [1..2]
//bad, type of xs is Set<obj>, no type check no good
let e = Seq.scan (fun(xs:Set<_>) x -> Set.add x xs) Set.empty [1..2]
Must be my eyes but what gives ?
As precised in the comment, I was opening the namespace of some other library further up in my file.
Being a C# library, it redefines its own Set (poor folks)
Doing so, writing Set<_> induced the compiler in picking up their version, whilst Set.add still refered to the trusted FSharp Set<_>.
Eventually, the two came in brutal head to head conflict with each other.
...or in FSharpx?
let tee sideEffect =
fun x ->
do sideEffect x
x
The usage could be something like
f >> tee (printfn "F returned: %A") >> g >> h
Or is there another simple way to do this?
thanks!
The closest I've seen is actually in WebSharper. The definition is:
let inline ( |>! ) x sideEffect =
do sideEffect x
x
Usage:
(x |>! printf "%A") |> nextFunc
ExtCore includes a function called tap which does exactly what you want. I use it for primarily for inspecting intermediate values within an F# "pipeline" (hence the name).
For example:
[| 1;2;3 |]
|> Array.map (fun x -> x * 2)
|> tap (fun arr ->
printfn "The mapped array values are: %A" arr)
|> doOtherStuffWithArray
As far as I know, a function like this isn't defined anywhere in the F# core library - though the library is missing many standard functions that are quite easy to define yourself, so my recommendation would be just to add it somewhere in your project - your tee seems like the best way to go.
That said, I'd probably prefer using less declarative style if I need side-effects and write something like:
let fResult = f fInput
printfn "F returned: %A" fResult
fResult |> g |> h
This is just a matter of style, but I prefer declarative style for fully declarative code and imperative style when there are side-effects involved. As a bonus, using local variables makes debugging easier. But using a function like tee is an equally good alternative that many people in the F# community would prefer.
I have an issues concerning the F# on mono. Im doing this course in functional programming at my university. In the course we are using F#, and I uses Xamarin as my editor.
The thing is that we had a lesson on tail recursion, as a tool for getting efficiency. But when you are not able to write your function tail recursive, we had to use continuous, such that we using the heap and not the stack.
This seems not to work on mono 3.10.0 with F# 3.1, I get an System.StackOverflowException. This should be impossible to get, due the continuous should use the heap.
let rec fibC n c =
match n with
|0 -> c 0
|1 -> c 1
|n -> fibC (n-1) (fun v1 -> fibC (n-2) (fun v2 -> c(v1+v2)))
I tested a Fibonacci implementation passing an accumulator instead of a function (continuation) like this:
let fib n =
let rec _fib i (a,b) =
match i with
| 0 -> a
| _ -> _fib (i-1) (b, a+b)
_fib n (0,1)
which worked fine on Mono, i.e. no stack overflow.
So I guess it's only an issue with TCO when using continuations. There's a Xamarin ticket from June 2013 addressing this.