Running only certain canopy tests as specified on command line - f#

Let's say I have the following simple Canopy test program:
open System
open canopy.runner.classic
open canopy.configuration
open canopy.classic
canopy.configuration.chromeDir <- System.AppContext.BaseDirectory
start chrome
"test 1" &&& fun _ ->
...
"test 2" &&& fun _ ->
...
"test 3" &&& fun _ ->
...
[<EntryPoint>]
let main args =
run()
quit()
0
When I run this program:
dotnet run
All three tests are run.
Let's say that by default, I want to run test 1 and test 2. But sometimes, I'd like to only run test 3.
What's the recommended way to set this up in Canopy?
Something like passing command line options would be fine:
dotnet run # Run the default tests
dotnet run -- test-3 # Run test 3
Alternatively, I guess I could have test 3 be in a whole separate project. But that seems like alot of overhead just to have a separate test.
Thanks for any suggestions!

I don't think there's any built-in way to do this, but it seems pretty easy to manage manually:
let defaultTests () =
"test 1" &&& fun _ -> ()
"test 2" &&& fun _ -> ()
let test3 () =
"test 3" &&& fun _ -> ()
[<EntryPoint>]
let main (args : _[]) =
let arg =
if args.Length = 0 then "default"
else args.[0]
match arg with
| "default" -> defaultTests ()
| "test-3" -> test3 ()
| _ -> failwith $"Unknown arg: {arg}"
run()
quit()
0
The trick is to make sure that only the tests you want to run actually get defined at run-time.

Related

FsCheck with Setup and Teardown

Summary
Are there any events that can be run before every property case so that I can run setup and teardown for each run of a property?
Full version
I want to be able to test paired behaviors like "I can always fetch written records" or "output of readAllLines equals input to writeAllLines" with properties. I also want the property to not care how the operation sets are implemented (i.e. if any resources need to be cleaned up).
Each run of the property should
be independent from other runs
maintain state between calls of the operations within this single run
not know about how how the operations maintain state
not be a resource leak
I'm using FsCheck and Expecto. Examples will be in Expecto, but the problem isn't specific to the framework.
It is super easy to write this sort of setup and teardown with example-based tests. They take a predictable argument set, so I can run them in a wrapper that adds before and after events.
let testWithEnv setup cleanup name test =
let testWrap () =
let (api, env) = setup ()
test api
cleanup env
testCase name testWrap
The same can't be done with property tests. They have an unknown number of arguments that will largely be filled with random data.
I can apply the set of paired behaviors pretty easily, but any created resources like streams are left undisposed.
let testPropertyWithEnv setup cleanup name test =
let testWrap () =
let (api, env) = setup () // this is actually run once, but immutable so the individual runs don't leak state
test api // have to return this to pass along unapplied parameters
testProperty name testWrap
I've looked into
Runner events
Looking at how to run FsCheck tests the closest hooks appear to be
OnStartFixture which is only run once per test class
OnArguments is run after every pass and would potentially work to run cleanup
Model-based features
There is also the experimental Model-based testing features that could work. However, it seems really heavy considering I'm only concerned with the external consistency of operations. I do not want access to the backing state.
Giving up and inlining
I can always write
testProperty "name" (fun arg1 arg2 ->
let (api,env) = setup ()
//test code here
cleanup env
)
but I'd like to avoid the boilerplate in every property and the exposure of the backing state.
IDisposables
Disposable objects also don't address the lack of setup hook.
More hands-on run loop
I looked into ways of running the property tests in my wrapper, but the smallest runner Check.one is for a single property, with no hooks between runs of the property.
Lazy wrapper
Making the wrapper lazy also doesn't work testProperty name lazy(testWithSetup)
There isn't really anything in FsCheck to help you, and even if there was I don't think it'd be straightforwardly exposed in Expecto. I also don't think it's that straightforward to add on the FsCheck side - that said happy to discuss that further if you open an issue in the FsCheck repo.
In any case, with some clever use of partial application and at the cost of some slight boilerplate, it is in fact possible to wrap "variadic" functions, which I think is essentially what you're asking.
Behold, the code:
// these types are here to make the signatures look nicer
type Api = Api
type Env = Env
let testProperty k =
// call the property with "random" arguments
for i in 0..2 do
k i (char i) (string i)
let setup() =
printfn "setup ran"
(Api, Env)
let teardown Env =
printfn "teardown ran"
let test0 Api arg1 =
printfn "test0 %A" arg1
let test1 Api (arg1:int) (arg2:char) =
printfn "test1 %A %A" arg1 arg2
let test2 Api arg1 arg2 arg3 =
printfn "testFun %A %A %A" arg1 arg2 arg3
let testWithEnv (setup:unit -> Api*Env) (teardown: Env -> unit) (test: Api -> 'a) (k: 'a -> unit) :unit =
let (api, env) = setup()
k (test api)
teardown env
let (<*>) (f,k) arg =
f, (fun c -> k c arg)
let (<!>) f arg =
f, (fun k -> k arg)
let run (f, k) = f k
testProperty (fun arg1 arg2 arg3 ->
testWithEnv setup teardown test2 <!> arg1 <*> arg2 <*> arg3 |> run
)
The idea here is that you use the operators <!> and <*> to string together any number of arguments of any type, pass that to the testWithEnv function and then call run on the result. The operators and the run are basically necessary to build up and then apply the variadic list of arguments.
It's all type safe i.e. if you forget to pass an argument or it's of the wrong type, you'll get a type error, though admittedly it might not be as clear as normal function application.
I recommend pasting this in an IDE and checking out the types, that will help greatly with understanding what is going on.
There are a few other styles you can write this in. E.g. with a slightly different definition and a the function you can write something like
let (<+>) k arg =
fun c -> k c arg
let the arg =
fun k -> k arg
testWithEnv setup teardown test2 (the arg1 <+> arg2 <+> arg3)
This replaces the run function with the and needs only one operator <+>. There's probably other ways to cut it too, pick your poison.

Matching command line args

Starting out learning F#. Want to make a simple program that just tells me what it found in the command line args. I have:
[<EntryPoint>]
let main argv =
printfn "%A" argv
match argv with
| [] -> 42
| _ -> 43
But this gives errors. If I hover over argv I see:
val argv : string[]
which is what I would have expected (a list of strings). However the first match expression has an error:
Error 1 This expression was expected to have type
string [] but here has type
'a list
Basically I just want to match on an empty argument list (an empty list of strings). What's the right way to do that?
I should add: I don't just want a solution (though that would be nice). I also want to understand what the compiler is looking for here that I'm not giving it.
It might be confusing since [] literal is used to signify an empty list, but type string [] is an array of strings rather than a list.
You can pattern match against an array like this:
[<EntryPoint>]
let main argv =
printfn "%A" argv
match argv with
| [||] -> 42
| _ -> 43
Like many seemingly inconsistent things in F#, this is a result of its dual heritage.
In OCaml, you'd use int list and int array for the types, [1;2;3] and [|1;2;3|] for the values respectively. But in C#/.NET, square brackets as in int[] are the way to indicate you're dealing with an array.
Probably in an attempt to be more approachable for the .NET crowd, in type names F# uses [] as an alias for array, so both forms are usable. It's rather unfortunate that this coincides with the empty list literal, but leaving that 'as is' was another constraint - one of early goals in F# design was to make it compatible with OCaml code, so that porting from that language to F# is as friction-less as possible.
If you want to implement some command-line utility I suggest using a wonderful library https://github.com/JordanMarr/FSharp.SystemCommandLine or https://github.com/cannorin/FSharp.CommandLine is also good.
But if you want to do that by hand, try
[<EntryPoint>]
let main (argv : string list) =
printfn "%A" argv
match argv with
| [] -> 42
| _ -> 43
or
[<EntryPoint>]
let main argv =
printfn "%A" argv
match argv |> List.ofArray with
| [] -> 42
| _ -> 43

F#: Is it an idiomatic way to call functions with pipe against parentheses?

If one have function like this:
let test arg func =
func arg
Is it an idiomatic way to call it using pipe,
test 1 <| fun n -> n = 2
rather then to call it using parentheses?
test 1 (fun n -> n = 2)
And when one should not use pipe in a such way?
tl;dr
In my opinion, this is fine:
test 1 (fun n -> n = 2)
It may not compose well because of the order of the arguments, so further details follow below. These are options you can use if you need them.
Details
I think it'd be more idiomatic to flip the arguments for the test function around:
let test' func arg = func arg
because this would enable you to write this expression instead:
1 |> test' (fun n -> n = 2)
If you can't change the signature of test, you can introduce a flip function:
let flip f x y = f y x
This function flips the arguments for a function, so that you can write this instead:
1 |> (flip test) (fun n -> n = 2)
In the example given here, that's making things more complicated than they have to be, so I'd most likely prefer one of the original suggestions:
test 1 (fun n -> n = 2)
Personally, I find that easier to understand than the backwards pipe, which always confuses me.
BTW, you can reduce the function in this particular example:
test 1 ((=) 2)
This snippet by Stephen Swensen demonstrates a high precedence, right associative backward pipe, (^<|).
let inline (^<|) f a = f a
This single-liner from the linked page demonstrates how to use it:
{1..10} |> Seq.map ^<| fun x -> x + 3
And here is an example how to use it for multi-line functions. I find it most useful for real-world multi-liners as you no longer need to keep closing parenthesis at the end:
myCommands
|> List.map ^<| fun command ->
let returnValue = manipulate command
doSomethingElse()
returnValue
Yet another benefit is keeping the natural order of arguments which often helps you avoid extra type annotations. In the snippet above, the type for command is inferred automatically.
I don't think there's a strong preference for one over the other.
Personally I'd use parens for a one-liner like your example:
test 1 (fun n -> n = 2)
and pipeline if func has multiple lines - this way you don't need the awkward closing paren that you have to move around as you change the function's body:
test 1 <| fun n ->
let x = n + 1
x = 3

Why are multiple branches of a match expressions evaluated?

I am using a match expression and getting a much different result than I expected. In my case the value of wi.State = "Done" so I expected the wi.Close() call to be executed and that's it. However it runs that and then runs the statements that come after the _ -> catch-all match as well. Clearly I misunderstand how these expressions are supposed so work so I will appreciate any pointers.
workItemCollection.Cast<WorkItem>()
|> Seq.iter(fun wi ->
wi.Open()
match wi.State with
| "Done"
| "Removed" ->
wi.Close()
| _ ->
printfn "Closing %s -> %i" iccmCallNumber wi.Id
wi.State <- "Done"
wi.History <- wi.History + "Closed by Iccm to Tfs integration"
wi.Save())

Encode result type in a discriminated union

I have multiple code blocks which can either end with an unrecoverable error, be cancelled by user input or otherwise provide an input value for the next code block. How then to deal with the final step, that is Success (all computations finished successfully).
type Res<'T> =
| Next of 'T
| Cancel
| Fail of string
module Res =
let bind arg f =
match arg with
| Next x -> f x
| Cancel -> Cancel
| Fail x -> Fail x
I'm now able to string these steps together like this, alas, the type Res<unit> takes on a special meaning.
let (|?>) = Res.bind
firstFunc()
|?> fun intermediateResult ->
secondFunc intermediateResult
...
|?> fun otherResult ->
lastFunc otherResult
|> function
| Fail msg -> printfn "Error: %s" msg
| Cancel -> printfn "Cancelled"
| Next() -> printfn "Succeeded"
If I had encoded Success as a discrete alternative, I would be stuck with an extra case Next _ in the final clause. What's the preferred way around this?
I think that your Next of 'T and Success of 'T represent almost the same thing - they mean that a computation block finished and returned some value. For composability reasons, it should not matter if the computation block is the last one in the entire chain or if it is not - in principle, the behavior of computation block should be the same regardless of whether it is at the beginning or at the end of your pipeline.
But if you want to distinguish between the two cases (and if this does not affect the properties when composing the blocks), then I would probably have Success case with an additional flag:
type ResultKind = Success | Next
type Result<'T> =
| Completed of 'T * ResultKind
| Cancel
| Fail of string
Now you can use Completed(42, Success) to create a final value (and Completed(42, Next) to create an immediate result).
In pattern matching, you can ignore the flag and just write | Completed(result, _) -> ...

Resources