Expecto FsCheck getting stack overflow exception when generating string - f#

I'm trying to learn how to use FsCheck properly, and integrating it with Expecto at the moment. I can get property tests to run if I'm using the default FsCheck config, but when I try to use my own Generator, it causes a stack overflow exception.
Here is my generator
type NameGen() =
static member Name() =
Arb.generate<string * string>
|> Gen.where (fun (firstName, lastName) ->
firstName.Length > 0 && lastName.Length > 0
)
|> Gen.map (fun (first, last) -> sprintf "%s %s" first last)
|> Arb.fromGen
|> Arb.convert string id
And I'm trying to use it like this:
let config = { FsCheckConfig.defaultConfig with arbitrary = [typeof<NameGen>] }
let propertyTests input =
let output = toInitials input
output.EndsWith(".")
testPropertyWithConfig config "Must end with period" propertyTests
The exception is thrown before it even gets into the Gen.where function
What am I doing wrong? Thanks

You are trying to use FsCheck's generator of strings to redefine how its generator of strings work, but when you do that, it'll recursively call itself until it runs out of stack space. It's a known issue: https://github.com/fscheck/FsCheck/issues/109
Does this alternative work?
type NameGen =
static member Name () =
Arb.Default.NonEmptyString().Generator
|> Gen.map (fun (NonEmptyString s) -> s)
|> Gen.two
|> Gen.map (fun (first, last) -> sprintf "%s %s" first last)
|> Arb.fromGen

You are defining a new generator for the type string, but within that you're using the generator for string * string, which uses the generator for string. FsCheck unfortunately seems to store generators in global mutable state (perhaps with good reason?) and I think this means that the generator keeps calling itself until stack overflow.
You could solve this by defining the generator for a custom wrapper type instead of a plain string (shown below).
The next problem you'll hit will be null reference exceptions. The initial generated string could be null and you try to access the .Length property. This can be resolved using the String.length function instead, which returns 0 for null.
With these changes your generator looks like this:
type Name = Name of string
type NameGen() =
static member Name() =
Arb.generate<string * string>
|> Gen.where (fun (firstName, lastName) ->
String.length firstName > 0 && String.length lastName > 0
)
|> Gen.map (fun (first, last) -> sprintf "%s %s" first last)
|> Arb.fromGen
|> Arb.convert Name (fun (Name n) -> n)
And your property needs a slight modification:
let propertyTests (Name input) =
let output = toInitials input
output.EndsWith(".")

Related

A better way to use Seq.fold in F#

I wrote a small console app that update a Type record without using any mutable variable. If that looks simple for seasoned functional programmers, it was quite a hard work for me.
It works, but there is one thing I am not happy with. But before that, let's start with the code:
open System
//------------------------------------------------------------------------------------
// Type, no data validation to keep it simple
//------------------------------------------------------------------------------------
[<StructuredFormatDisplay("{FirstName} {LastName} is a {Age} year old {Sex}")>]
type Student = {
FirstName: string
LastName : string
Sex : char
Age: int
}
//------------------------------------------------------------------------------------
// I/O functions
//------------------------------------------------------------------------------------
let getConsoleChar message =
printf "\n%s" message
Console.ReadKey().KeyChar
let getConsoleString message =
printf "\n%s" message
Console.ReadLine()
let getConsoleInt = getConsoleString >> Int32.Parse //no tryparse to keep it simple, I'm sure you can type an integer
let isValidCommand command = [ 'f'; 'l'; 's'; 'a'; 'x'] |> List.contains command
let isStopCommand = (=) 'x'
let processCommand student command =
match command with
| 'f' -> { student with FirstName = (getConsoleString "First Name: ")}
| 'l' -> { student with LastName = (getConsoleString "Last Name: ")}
| 's' -> { student with Sex = (getConsoleChar "Sex: ")}
| 'a' -> { student with Age = (getConsoleInt "Age: ")}
| 'x' -> student
| _ -> failwith "You've just broken the Internet, theorically you cannot be here"
//------------------------------------------------------------------------------------
// Program
//------------------------------------------------------------------------------------
let initialStudent = {
FirstName = String.Empty
LastName = String.Empty
Sex = Char.MinValue
Age = 0
}
let commands = seq {
while true do
yield getConsoleChar "Update [f]irst name, [l]ast name, [s]ex, [a]ge or e[x]it: " }
let finalStudent =
commands
|> Seq.filter isValidCommand
|> Seq.takeWhile (not << isStopCommand)
|> Seq.map (fun cmd -> (initialStudent, cmd))
|> Seq.fold (fun student studentAndCommand -> processCommand student (snd studentAndCommand)) initialStudent
printfn "\n<<<< %A >>>>\n" finalStudent
My problem is with
|> Seq.map (fun cmd -> (initialStudent, cmd))
|> Seq.fold (fun student studentAndCommand -> processCommand student (snd studentAndCommand)) initialStudent
It looks bizarre to transform a sequence of char into a Student*char to be able to plug it with aSeq.fold. Also, if using the initialStudent as a starting point for Seq.fold is logical, it feel weird to use it in the mapping transformation (I'm not sure anyone would understand the logic if this code was pushed in prod).
Is there a better way to treat the sequence of commands or is this code standard and acceptable in the functional world?
You can get rid of the map and simplify the fold considerably:
commands
|> Seq.filter isValidCommand
|> Seq.takeWhile (not << isStopCommand)
|> Seq.fold processCommand initialStudent
I'm not sure why you thought you had to map the seq<char> into a seq<Student * char> to begin with. Since you immediately use snd to extract the char from the tuple, undoing the map, the tuples' first elements are totally ignored. Much cleaner to simply avoid creating tuples in the first place

Reading text file, iterating over lines to find a match, and return the value with FSharp

I have a text file that contains the following and I need to retrieve the value assigned to taskId, which in this case is AWc34YBAp0N7ZCmVka2u.
projectKey=ProjectName
serverUrl=http://localhost:9090
serverVersion=10.5.32.3
strong text**interfaceUrl=http://localhost:9090/interface?id=ProjectName
taskId=AWc34YBAp0N7ZCmVka2u
taskUrl=http://localhost:9090/api/ce/task?id=AWc34YBAp0N7ZCmVka2u
I have two different ways of reading the file that I've wrote.
let readLines (filePath:string) = seq {
use sr = new StreamReader (filePath)
while not sr.EndOfStream do
yield sr.ReadLine ()
}
readLines (FindFile currentDirectory "../**/sample.txt")
|> Seq.iter (fun line ->
printfn "%s" line
)
and
let readLines (filePath:string) =
(File.ReadAllLines filePath)
readLines (FindFile currentDirectory "../**/sample.txt")
|> Seq.iter (fun line ->
printfn "%s" line
)
At this point, I don't know how to approach getting the value I need. Options that, I think, are on the table are:
use Contains()
Regex
Record type
Active Pattern
How can I get this value returned and fail if it doesn't exist?
I think all the options would be reasonable - it depends on how complex the file will actually be. If there is no escaping then you can probably just look for = in the line and use that to split the line into a key value pair. If the syntax is more complex, this might not always work though.
My preferred method would be to use Split on string - you can then filter to find values with your required key, map to get the value and use Seq.head to get the value:
["foo=bar"]
|> Seq.map (fun line -> line.Split('='))
|> Seq.filter (fun kvp -> kvp.[0] = "foo")
|> Seq.map (fun kvp -> kvp.[1])
|> Seq.head
Using active patterns, you could define a pattern that takes a string and splits it using = into a list:
let (|Split|) (s:string) = s.Split('=') |> List.ofSeq
This then lets you get the value using Seq.pick with a pattern matching that looks for strings where the substring before = is e.g. foo:
["foo=bar"] |> Seq.pick (function
| Split ["foo"; value] -> Some value
| _ -> None)
The active pattern trick is quite neat, but it might be unnecessarily complicating the code if you only need this in one place.

parse log files with f#

I'm trying to parse data from iis log files.
Each row has a date that I need like this:
u_ex15090503.log:3040:2015-09-05 03:57:45
And a name and email address I need in here:
&actor=%7B%22name%22%3A%5B%22James%2C%20Smith%22%5D%2C%22mbox%22%3A%5B%22mailto%3AJames.Smith%40student.colled.edu%22%5D%7D&
I start off by getting the correct column like this. This part works fine.
//get the correct column
let getCol =
let line = fileReader inputFile
line
|> Seq.filter (fun line -> not (line.StartsWith("#")))
|> Seq.map (fun line -> line.Split())
|> Seq.map (fun line -> line.[7],1)
|> Seq.toArray
getCol
Now I need to parse the above and get the date, name, and email, but I'm having a hard time figuring out how to do that.
So far I have this, which gives me 2 errors(below):
//split the above column at every "&"
let getDataInCol =
let line = getCol
line
|> Seq.map (fun line -> line.Split('&'))
|> Seq.map (fun line -> line.[5], 1)
|> Seq.toArray
getDataInCol
Seq.map (fun line -> line.Split('&'))
the field constructor 'Split' is not defined
The errors:
Seq.map (fun line -> line.[5], 1)
the operator 'expr.[idx]' has been used on an object of indeterminate type based on information prior to this program point.
Maybe I'm going about this all wrong. I'm very new to f# so I apologize for the sloppy code.
Something like this would get the name and email. You'll still need to parse the date.
#r "Newtonsoft.Json.dll"
open System
open System.Text.RegularExpressions
open Newtonsoft.Json.Linq
let (|Regex|_|) pattern input =
let m = Regex.Match(input, pattern)
if m.Success then Some(List.tail [ for g in m.Groups -> g.Value ])
else None
type ActorDetails =
{
Date: DateTime
Name: string
Email: string
}
let parseActorDetails queryString =
match queryString with
| Regex #"[\?|&]actor=([^&]+)" [json] ->
let jsonValue = JValue.Parse(Uri.UnescapeDataString(json))
{
Date = DateTime.UtcNow (* replace with parsed date *)
Name = jsonValue.Value<JArray>("name").[0].Value<string>()
Email = jsonValue.Value<JArray>("mbox").[0].Value<string>().[7..]
}
| _ -> invalidArg "queryString" "Invalid format"
parseActorDetails "&actor=%7B%22name%22%3A%5B%22James%2C%20Smith%22%5D%2C%22mbox%22%3A%5B%22mailto%3AJames.Smith%40student.colled.edu%22%5D%7D&"
val it : ActorDetails = {Date = 11/10/2015 9:14:25 PM;
Name = "James, Smith";
Email = "James.Smith#student.colled.edu";}

F# how to list functions with custom attribute?

I'm trying to create some kind of interface, but i cannot find how to use custom attributes in F# as MSDN only shows usage of CLR attributes. This is what i want to achieve:
open System
type Command (name : string) =
inherit Attribute()
member this.Name = name
[<Command("something")>]
let doSomething () =
Console.Write("I'm doing something")
[<Command("somethingElse")>]
let doSomethingElse () =
Console.Write("I'm doing something else")
[<EntryPoint>]
let main args =
let command = Console.ReadLine()
// find function where Command.Name = command and call it
Console.Read()
0
To extend on your answer, a more generic approach would be to get all the types and then filter the functions that have the attribute you're looking for (as your approach would break down once your application grows and no longer has everything "packed" into the Program class):
let getCommands () =
let types = Assembly.GetExecutingAssembly().GetTypes()
let commands =
types
|> Array.collect (fun typ -> typ.GetMethods())
|> Array.choose (fun mi ->
mi.CustomAttributes
|> Seq.tryFind (fun attr -> attr.AttributeType = typeof<Command>)
|> Option.map (fun attr -> attr, mi))
let commandsMap =
commands
|> Seq.map (fun (attr, mi) ->
let name =
let arg = attr.ConstructorArguments.[0]
unbox<string> arg.Value
name, mi)
|> Map.ofSeq
commandsMap
This gets all the functions from all the types in the executing assembly, then filters out everything that doesn't have command attribute. Then it builds a map where the key is the attribute argument and the value is the MethodInfo of the function.
Ok, found it.
Reflection.Assembly.GetExecutingAssembly().GetType("Program").GetMethods()
Program typename is not viable in code so it cannot be used in typeof<Program>, but this type exists and can be taken from assembly.

Can I call a function by name in f#?

Is there any way to call a function by name in F#? Given a string, I want to pluck a function value from the global namespace (or, in general, a given module), and call it. I know the type of the function already.
Why would I want to do this? I'm trying to work around fsi not having an --eval option. I have a script file that defines many int->() functions, and I want to execute one of them. Like so:
fsianycpu --use:script_with_many_funcs.fsx --eval "analyzeDataSet 1"
My thought was to write a trampoline script, like:
fsianycpu --use:script_with_many_funcs.fsx trampoline.fsx analyzeDataSet 1
In order to write "trampoline.fsx", I'd need to look up the function by name.
There is no built-in function for this, but you can implement it using .NET reflection. The idea is to search through all types available in the current assembly (this is where the current code is compiled) and dynamically invoke the method with the matching name. If you had this in a module, you'd have to check the type name too.
// Some sample functions that we might want to call
let hello() =
printfn "Hello world"
let bye() =
printfn "Bye"
// Loader script that calls function by name
open System
open System.Reflection
let callFunction name =
let asm = Assembly.GetExecutingAssembly()
for t in asm.GetTypes() do
for m in t.GetMethods() do
if m.IsStatic && m.Name = name then
m.Invoke(null, [||]) |> ignore
// Use the first command line argument (after -- in the fsi call below)
callFunction fsi.CommandLineArgs.[1]
This runs hello world when called by:
fsi --use:C:\temp\test.fsx --exec -- "hello"
You can use reflection to get the functions as MethodInfo's by FSharp function name
open System
open System.Reflection
let rec fsharpName (mi:MemberInfo) =
if mi.DeclaringType.IsNestedPublic then
sprintf "%s.%s" (fsharpName mi.DeclaringType) mi.Name
else
mi.Name
let functionsByName =
Assembly.GetExecutingAssembly().GetTypes()
|> Seq.filter (fun t -> t.IsPublic || t.IsNestedPublic)
|> Seq.collect (fun t -> t.GetMethods(BindingFlags.Static ||| BindingFlags.Public))
|> Seq.filter (fun m -> not m.IsSpecialName)
|> Seq.groupBy (fun m -> fsharpName m)
|> Map.ofSeq
|> Map.map (fun k v -> Seq.exactlyOne v)
You can then invoke the MethodInfo
functionsByName.[fsharpFunctionNameString].Invoke(null, objectArrayOfArguments)
But you probably need to do more work to parse your string arguments using the MethodInfo.GetParameters() types as a hint.
You could also use FSharp.Compiler.Service to make your own fsi.exe with an eval flag
open System
open Microsoft.FSharp.Compiler.Interactive.Shell
open System.Text.RegularExpressions
[<EntryPoint>]
let main(argv) =
let argAll = Array.append [| "C:\\fsi.exe" |] argv
let argFix = argAll |> Array.map (fun a -> if a.StartsWith("--eval:") then "--noninteractive" else a)
let optFind = argv |> Seq.tryFind (fun a -> a.StartsWith "--eval:")
let evalData = if optFind.IsSome then
optFind.Value.Replace("--eval:",String.Empty)
else
String.Empty
let fsiConfig = FsiEvaluationSession.GetDefaultConfiguration()
let fsiSession = FsiEvaluationSession(fsiConfig, argFix, Console.In, Console.Out, Console.Error)
if String.IsNullOrWhiteSpace(evalData) then
fsiSession.Run()
else
fsiSession.EvalInteraction(evalData)
0
If the above was compiled into fsieval.exe it could be used as so
fsieval.exe --load:script_with_many_funcs.fsx --eval:analyzeDataSet` 1

Resources