I'm trying to make test for this function
let extract_one_rule (rule:Rule.t<'a,'b>) =
let rec expand = function
|PAlt (a,b) -> expand a # expand b
|PSeq (a,b) -> let wrap = List.map (fun x -> (x.rule, fun r -> {x with rule = r})) a
|> List.unzip
in
let rec gen = function
| hd::tl -> [for x in hd -> x :: ( gen tl |> List.concat)]
| [] -> []
in
fst wrap |> List.map expand |> gen
|> List.map (fun x -> PSeq ((List.map2 ( |> ) x (snd wrap)),b))
|PRef _
|PLiteral _
|PToken _ as t -> [t]
| _ -> (System.Console.WriteLine("incorrect tree for alternative expanding!")
; failwith "incorrect tree for alternative expanding!")
in
expand rule.body |> List.map (fun x -> {rule with body = x})
using FsCheck
so i have this
let ExpandAlterTest(t : Rule.t<Source.t,Source.t> ) = convertToMeta t |> List.forall (fun x -> ruleIsAfterEBNF x)
but i'l see exception "incorrect tree for alternative expanding!"
but when i use smth like that
let ExpandAlterTest(t : Rule.t<Source.t,Source.t> ) = (correctForAlExp t.body) ==> lazy ( convertToMeta t |> List.forall (fun x -> ruleIsAfterEBNF x))
NUnit doesn't stop working
Why it can be?
It could be that the precondition you added is very restrictive, so that it takes a long time before a good value (one that actually passes the precondition) is found. FsCheck is hardened against this - by default, it tries to find 100 values but when it has rejected 1000 it gives up and you should see a "Arguments exhausted after x tests" output. But this might take a long time, if generating and checking the value takes a long time.
Could also be that you actually have a bug somewhere, like an infinite loop.
Try changing the FsCheck config to run less tests, doing a verbose run (verboseCheck), and breaking in the debugger when it seems to hang.
Related
I am trying to create a recursive function that is conditionally calls itself and so far is is defined as follows:
let rec crawlPage (page : String, nestingLevel : int) =
HtmlDocument.Load(page)
|> fun m -> m.CssSelect("a")
|> List.map(fun a -> a.AttributeValue("href"))
|> Seq.distinctBy id
|> Seq.map (fun x -> baseUrl + x)
|> Seq.map (fun x ->
match nestingLevel with
// Compiler says it is expecting a but given seq<a> in reference to the recursive call
| _ when (nestingLevel > 0) -> crawlPage(x, (nestingLevel - 1))
| _ when (nestingLevel <= 0) -> ignore
| _ -> (* To silence warnigs.*) ignore
)
It is that the Seq.map (fun x -> ...) cannot handle the return sequence or can the match condition not handle the returned sequence? Given that the crawlPage is underlined by the compiler it seems that the match statement cannot handle the seq returned so how can this be done?
The rule is that all the matching branches must return the same type, so you have to:
Replace ignore with Seq.singleton x to indicate that this branch yields nothing more except the x itself.
At the end, concat (flat map) the seq<seq<string>> to transform it to a seq<string>.
The code would be:
|> Seq.map (fun x ->
match nestingLevel with
| _ when (nestingLevel > 0) -> crawlPage(x, (nestingLevel - 1))
| _ -> Seq.singleton x)
|> Seq.concat
The existing post answers your specific question, but I think it is worth noting that there are a few other changes that could be done to your code snippet. Some of those are a matter of personal preference, but I believe they make your code simpler:
You can use sequence comprehension, which lets you handle recursive calls nicely using yield! (and non-recursive using yield)
You do not actually need match, because you have just two branches that are more easily tested using ordinary if
I would also avoid the |> fun m -> m.Xyz pattern, because it's not necessary here.
With all those tweaks, my preferred version of the code snippet would be:
let rec crawlPage (page : String, nestingLevel : int) = seq {
let urls =
HtmlDocument.Load(page).CssSelect("a")
|> List.map(fun a -> a.AttributeValue("href"))
|> Seq.distinctBy id
|> Seq.map (fun x -> baseUrl + x)
for x in urls do
if nestingLevel > 0 then
yield! crawlPage(x, (nestingLevel - 1))
else
yield x }
I'm quite a newbie at F# and want to find how many times a value x has occurred in a list ys
So for example multiplicity (2, [1;2;3;4;2]) returns 2. The code I've written below returns 4 on the above example. What am I missing?
let rec multiplicity (x, ys) =
match ys with
| [] -> 0
| y::tail when x=y -> x + multiplicity(x, tail)
| y::tail -> multiplicity(x, tail)
If you're not writing this as a recursive function as a learning exercise, it's probably more idiomatic to use the built-in collection functions:
[1;2;3;4;2] |> Seq.filter ((=) 2) |> Seq.length
[1;2;3;4;2] |> List.sumBy (fun x -> if x = 2 then 1 else 0)
Alright, this is a good example of why it's always a good idea to write down a problem/question.
I figured out, I should do this instead:
let rec multiplicity (x, ys) =
match ys with
| [] -> 0
| y::tail when x=y -> 1 + multiplicity(x, tail)
| y::tail -> multiplicity(x, tail)
It should be 1 and not x, which is added to the recursive call, doh.
I find it's a good idea to replace recursion with fold, so here is another version:
let xs = [1;2;3;4;2]
(0,xs) ||> List.fold (fun acc elem -> match elem with
| 2 -> acc + 1
| _ -> acc)
You can also use countBy, which will return a list of tuples with true and false.
xs |> List.countBy (fun x -> x = 2)
I'm trying to explore the dynamic capabilities of F# for situations where I can't express some function with the static type system. As such, I'm trying to create a mapN function for (say) Option types, but I'm having trouble creating a function with a dynamic number of arguments. I've tried:
let mapN<'output> (f : obj) args =
let rec mapN' (state:obj) (args' : (obj option) list) =
match args' with
| Some x :: xs -> mapN' ((state :?> obj -> obj) x) xs
| None _ :: _ -> None
| [] -> state :?> 'output option
mapN' f args
let toObjOption (x : #obj option) =
Option.map (fun x -> x :> obj) x
let a = Some 5
let b = Some "hi"
let c = Some true
let ans = mapN<string> (fun x y z -> sprintf "%i %s %A" x y z) [a |> toObjOption; b |> toObjOption; c |> toObjOption]
(which takes the function passed in and applies one argument at a time) which compiles, but then at runtime I get the following:
System.InvalidCastException: Unable to cast object of type 'ans#47' to type
'Microsoft.FSharp.Core.FSharpFunc`2[System.Object,System.Object]'.
I realize that it would be more idiomatic to either create a computation expression for options, or to define map2 through map5 or so, but I specifically want to explore the dynamic capabilities of F# to see whether something like this would be possible.
Is this just a concept that can't be done in F#, or is there an approach that I'm missing?
I think you would only be able to take that approach with reflection.
However, there are other ways to solve the overall problem without having to go dynamic or use the other static options you mentioned. You can get a lot of the same convenience using Option.apply, which you need to define yourself (or take from a library). This code is stolen and adapted from F# for fun and profit:
module Option =
let apply fOpt xOpt =
match fOpt,xOpt with
| Some f, Some x -> Some (f x)
| _ -> None
let resultOption =
let (<*>) = Option.apply
Some (fun x y z -> sprintf "%i %s %A" x y z)
<*> Some 5
<*> Some "hi"
<*> Some true
To explain why your approach does not work, the problem is that you cannot cast a function of type int -> int (represented as FSharpFunc<int, int>) to a value of type obj -> obj (represented as FSharpFunc<obj, obj>). The types are the same generic types, but the cast fails because the generic parameters are different.
If you insert a lot of boxing and unboxing, then your function actually works, but this is probably not something you want to write:
let ans = mapN<string> (fun (x:obj) -> box (fun (y:obj) -> box (fun (z:obj) ->
box (Some(sprintf "%i %s %A" (unbox x) (unbox y) (unbox z))))))
[a |> toObjOption; b |> toObjOption; c |> toObjOption]
If you wanted to explore more options possible thanks to dynamic hacks - then you can probably do more using F# reflection. I would not typically use this in production (simple is better - I'd just define multiple map functions by hand or something like that), but the following runs:
let rec mapN<'R> f args =
match args with
| [] -> unbox<'R> f
| x::xs ->
let m = f.GetType().GetMethods() |> Seq.find (fun m ->
m.Name = "Invoke" && m.GetParameters().Length = 1)
mapN<'R> (m.Invoke(f, [| x |])) xs
mapN<obj> (fun a b c -> sprintf "%d %s %A" a b c) [box 1; box "hi"; box true]
This is a reference question to this: StackOverflow in continuation monad
with whom I played a little and would need a few clarifications.
1) I suppose this:
member this.Delay(mk) = fun c -> mk () c
makes the behavior in computational workflow do the diffrence as showed by toyvo between these:
cBind (map xs) (fun xs -> cReturn (f x :: xs))
cBind (fun c -> map xs c) (fun xs -> cReturn (f x :: xs))
So I don't exactly understand what is the trick, when
(fun c -> map xs c) is only different notation of (map xs)
2) Inference issue. - In OP's second map example I found out it doesn't compile due to inference problem with v value, because it infers f as a -> b list, instead of desired a -> b. Why it infers in this way? In case let v = f x it would infer well.
3) It seems to me that VS shows inaccurate type signatures in the tooltips:
return type of the monad's Return is: ('e->'f)->f, while the return type of the Bind is only 'c->'b. -It seems it simplify ('e->'f) to only c in the Bind case, or am I missing something here?
Thanks for the clarification,
tomas
Edit - testing dump:
let cReturn x = fun k -> k x
let cBind m f =
printfn "cBind %A" <| m id
fun c -> m (fun a -> f a c)
let map_fixed f xs =
let rec map xs =
printfn "map %A" xs
match xs with
| [] -> cReturn []
| x :: xs -> cBind (fun c -> map xs c) (fun xs -> cReturn (f x :: xs))
map xs (fun x -> x)
let map f xs =
let rec map xs =
printfn "map %A" xs
match xs with
| [] -> cReturn []
| x :: xs -> cBind (map xs) (fun xs -> cReturn (f x :: xs))
map xs (fun x -> x)
[1..2] |> map_fixed ((+) 1) |> printfn "%A"
[1..2] |> map ((+) 1) |> printfn "%A"
map_fixed:
map [1; 2]
map [2]
map []
cBind []
map []
cBind [3]
map [2]
map []
cBind []
map []
[2; 3]
map:
map [1; 2]
map [2]
map []
cBind []
cBind [3]
[2; 3]
Edit to question 2:
let map f xs =
let rec map xs =
cont {
match xs with
| [] -> return []
| x :: xs ->
let v = f x // Inference ok
//let! v = cont { return f x } // ! Inference issue - question 2
let! xs = map xs
return v :: xs
}
map xs id
The issue is exactly that fun c -> map xs c is not the same as map xs. They have the same "meaning" in some sense, but their runtime semantics are different. In the latter case, evaluating the expression results in an immediate call to the map function with xs as an argument (returning another function as the result). On the other hand, evaluating fun c -> map xs c does not result in an immediate call to map! The call to map is delayed until the resulting function is actually applied. This is the critical difference that prevents a stack overflow.
Regarding your other questions, I can't quite make out what you're asking in your second question. For your third question, the compiler has inferred the most general type possible for Bind. You're right that the traditional type that you might expect is more specific than this, but it's not really a problem that you can call Bind in a wider set of contexts than is strictly necessary. And if you really want a more specific type, you can always add annotations to constrain the signature.
Disclosure: this came up in FsCheck, an F# random testing framework I maintain. I have a solution, but I do not like it. Moreover, I do not understand the problem - it was merely circumvented.
A fairly standard implementation of (monadic, if we're going to use big words) sequence is:
let sequence l =
let k m m' = gen { let! x = m
let! xs = m'
return (x::xs) }
List.foldBack k l (gen { return [] })
Where gen can be replaced by a computation builder of choice. Unfortunately, that implementation consumes stack space, and so eventually stack overflows if the list is long enough.The question is: why? I know in principle foldBack is not tail recursive, but the clever bunnies of the F# team have circumvented that in the foldBack implementation. Is there a problem in the computation builder implementation?
If I change the implementation to the below, everything is fine:
let sequence l =
let rec go gs acc size r0 =
match gs with
| [] -> List.rev acc
| (Gen g)::gs' ->
let r1,r2 = split r0
let y = g size r1
go gs' (y::acc) size r2
Gen(fun n r -> go l [] n r)
For completeness, the Gen type and computation builder can be found in the FsCheck source
Building on Tomas's answer, let's define two modules:
module Kurt =
type Gen<'a> = Gen of (int -> 'a)
let unit x = Gen (fun _ -> x)
let bind k (Gen m) =
Gen (fun n ->
let (Gen m') = k (m n)
m' n)
type GenBuilder() =
member x.Return(v) = unit v
member x.Bind(v,f) = bind f v
let gen = GenBuilder()
module Tomas =
type Gen<'a> = Gen of (int -> ('a -> unit) -> unit)
let unit x = Gen (fun _ f -> f x)
let bind k (Gen m) =
Gen (fun n f ->
m n (fun r ->
let (Gen m') = k r
m' n f))
type GenBuilder() =
member x.Return v = unit v
member x.Bind(v,f) = bind f v
let gen = GenBuilder()
To simplify things a bit, let's rewrite your original sequence function as
let rec sequence = function
| [] -> gen { return [] }
| m::ms -> gen {
let! x = m
let! xs = sequence ms
return x::xs }
Now, sequence [for i in 1 .. 100000 -> unit i] will run to completion regardless of whether sequence is defined in terms of Kurt.gen or Tomas.gen. The issue is not that sequence causes a stack overflow when using your definitions, it's that the function returned from the call to sequence causes a stack overflow when it is called.
To see why this is so, let's expand the definition of sequence in terms of the underlying monadic operations:
let rec sequence = function
| [] -> unit []
| m::ms ->
bind (fun x -> bind (fun xs -> unit (x::xs)) (sequence ms)) m
Inlining the Kurt.unit and Kurt.bind values and simplifying like crazy, we get
let rec sequence = function
| [] -> Kurt.Gen(fun _ -> [])
| (Kurt.Gen m)::ms ->
Kurt.Gen(fun n ->
let (Kurt.Gen ms') = sequence ms
(m n)::(ms' n))
Now it's hopefully clear why calling let (Kurt.Gen f) = sequence [for i in 1 .. 1000000 -> unit i] in f 0 overflows the stack: f requires a non-tail-recursive call to sequence and evaluation of the resulting function, so there will be one stack frame for each recursive call.
Inlining Tomas.unit and Tomas.bind into the definition of sequence instead, we get the following simplified version:
let rec sequence = function
| [] -> Tomas.Gen (fun _ f -> f [])
| (Tomas.Gen m)::ms ->
Tomas.Gen(fun n f ->
m n (fun r ->
let (Tomas.Gen ms') = sequence ms
ms' n (fun rs -> f (r::rs))))
Reasoning about this variant is tricky. You can empirically verify that it won't blow the stack for some arbitrarily large inputs (as Tomas shows in his answer), and you can step through the evaluation to convince yourself of this fact. However, the stack consumption depends on the Gen instances in the list that's passed in, and it is possible to blow the stack for inputs that aren't themselves tail recursive:
// ok
let (Tomas.Gen f) = sequence [for i in 1 .. 1000000 -> unit i]
f 0 (fun list -> printfn "%i" list.Length)
// not ok...
let (Tomas.Gen f) = sequence [for i in 1 .. 1000000 -> Gen(fun _ f -> f i; printfn "%i" i)]
f 0 (fun list -> printfn "%i" list.Length)
You're correct - the reason why you're getting a stack overflow is that the bind operation of the monad needs to be tail-recursive (because it is used to aggregate values during folding).
The monad used in FsCheck is essentially a state monad (it keeps the current generator and some number). I simplified it a bit and got something like:
type Gen<'a> = Gen of (int -> 'a)
let unit x = Gen (fun n -> x)
let bind k (Gen m) =
Gen (fun n ->
let (Gen m') = k (m n)
m' n)
Here, the bind function is not tail-recursive because it calls k and then does some more work. You can change the monad to be a continuation monad. It is implemented as a function that takes the state and a continuation - a function that is called with the result as an argument. For this monad, you can make bind tail recursive:
type Gen<'a> = Gen of (int -> ('a -> unit) -> unit)
let unit x = Gen (fun n f -> f x)
let bind k (Gen m) =
Gen (fun n f ->
m n (fun r ->
let (Gen m') = k r
m' n f))
The following example will not stack overflow (and it did with the original implementation):
let sequence l =
let k m m' =
m |> bind (fun x ->
m' |> bind (fun xs ->
unit (x::xs)))
List.foldBack k l (unit [])
let (Gen f) = sequence [ for i in 1 .. 100000 -> unit i ]
f 0 (fun list -> printfn "%d" list.Length)