I am fitting a bunch of models using a drake plan. Some of them are failing due to problems in the initialization. I am running `make(plan, keep_going = T) in order to finish the plan in anyway, but what I would really like is to be able to skip the failed targets and treat them as missing values in the rest of the plan.
Is there anyway to replace failed targets with, let's say a constant NA symbol?
Edit
Here is a better example than the one I originally provided. All you need is to wrap your models in a custom function that turns failures into NAs.
library(drake)
fail_na <- function(code) {
tryCatch(code, error = error_na)
}
error_na <- function(e) {
NA
}
plan <- drake_plan(
failure = fail_na(stop()),
success = fail_na("success")
)
make(plan, keep_going = TRUE)
#> In drake, consider r_make() instead of make(). r_make() runs make() in a fresh R session for enhanced robustness and reproducibility.
#> target success
#> target failure
readd(failure)
#> [1] NA
readd(success)
#> [1] "success"
Created on 2019-11-14 by the reprex package (v0.3.0)
Original answer
It is possible, but it requires some custom code. Below, we need to check that x is NULL or missing.
library(drake)
`%||%` <- function(x, y) {
if (is.null(x)) {
y
} else {
x
}
}
na_fallback <- function(x) {
out <- tryCatch(
x %||% NA,
error = function(e) {
NA
}
)
out
}
plan <- drake_plan(
x = stop(),
y = na_fallback(x)
)
make(plan, keep_going = TRUE)
#> In drake, consider r_make() instead of make(). r_make() runs make() in a fresh R session for enhanced robustness and reproducibility.
#> target x
#> fail x
#> target y
readd(y)
#> [1] NA
Created on 2019-11-14 by the reprex package (v0.3.0)
Related
FsCheck offers the Gen.filter / Gen.where functions. Example:
type Status = Pending | Activated | Deactivated
type Item = { Id: int; Status: Status }
let nonPendingItemGen =
Arb.generate
|> Gen.filter (fun item -> item.Id > 0 && item.Status <> Pending }
Is it possible to refactor this code to use gen computation expression with the "filter" inside?
A quick attempt using an if statement failed because of the type discrepancies between the if and the else branches.
I found a solution by generating a sequence and then filter this sequence but I find it less elegant.
You can do this by generating a valid ID and status within the computation expression and then constructing the item from those values. Instead of thinking about what values you don't want, think about the values you do want and generate them:
let nonPendingItemGen =
gen {
let! id = Arb.generate<PositiveInt>
let! status = Gen.elements [ Activated; Deactivated ]
return { Id = id.Get; Status = status }
}
This approach is more efficient in general, because you don't waste time generating (and then rejecting) invalid items.
Note that I've used FsCheck's built-in PositiveInt type. Alternatively, you could generate a positive integer directly via:
Gen.choose (1, Int32.MaxValue)
I've created some basic types with the intention that invalid states cannot be created. For example:
type PositiveDecimal = private PositiveDecimal of decimal
[<RequireQualifiedAccess>]
module PositiveDecimal =
let create num =
if num >= 0m then
num |> PositiveDecimal |> Ok
else
Error.validation None (sprintf "'%f' should be positive." num)
This returns a Result with either the successfully created object or a custom error type.
I'm finding it quite awkward to unit test (using MSTest):
[<TestMethod>]
member __.Create() =
let actual =
PositiveDecimal.create 1m
|> Result.map PositiveDecimal.value
// This explicit type is unfortunately required because otherwise
// they will have different error types :(.
let expected : Result<_, Error.Error> = Ok 1m
Assert.AreEqual(expected, actual)
As you can see, I have to map it back to a decimal because I can't create a PositiveDecimal through another source.
Of course, I could use make the constructor internal and use InternalsVisibleTo, but that circumvents the entire point of safety if the entire assembly is allowed to ignore the factory. Is there a better way to test this?
I wouldn't probably go so far to write a test for a smart constructor like the one you have, but if I were writing one, I'd probably just avoid the problem altogether and have a test that checks for a success given a positive argument, i.e.
member __.Create() =
let res = PositiveDecimal.create 1m
Assert.True(match res with Ok pd -> PositiveDecimal.value pd = 1M | _ -> false)
and another one with negative argument for a failure case (and arguably one for zero).
Or combine the two in an FsCheck property:
open FsCheck
let prop x =
let res = PositiveDecimal.create x
match res with
| Ok pd -> x >= 0M && PositiveDecimal.value pd = x
| Error _ -> x < 0M
Check.Quick prop
I'm trying to wrap my head around mon-, err, workflows in F# and while I think that I have a pretty solid understanding of the basic "Maybe" workflow, trying to implement a state workflow to generate random numbers has really got me stumped.
My non-completed attempt can be seen here:
let randomInt state =
let random = System.Random(state)
// Generate random number and a new state as well
random.Next(0,1000), random.Next()
type RandomWF (initState) =
member this.Bind(rnd,rest) =
let value, newState = rnd initState
// How to feed "newState" into "rest"??
value |> rest
member this.Return a = a // Should I maybe feed "initState" into the computation here?
RandomWF(0) {
let! a = randomInt
let! b = randomInt
let! c = randomInt
return [a; b; c]
} |> printfn "%A"
Edit: Actually got it to work! Not exactly sure how it works though, so if anyone wants to lay it out in a good answer, it's still up for grabs. Here's my working code:
type RandomWF (initState) =
member this.Bind(rnd,rest) =
fun state ->
let value, nextState = rnd state
rest value nextState
member this.Return a = fun _ -> a
member this.Run x = x initState
There are two things that make it harder to see what your workflow is doing:
You're using a function type for the type of your monad,
Your workflow not only builds up the computation, it also runs it.
I think it's clearer to follow once you see how it would look without those two impediments. Here's the workflow defined using a DU wrapper type:
type Random<'a> =
Comp of (int -> 'a * int)
let run init (Comp f) = f init
type Random<'a> with
member this.Run(state) = fst <| run state this
type RandomBuilder() =
member this.Bind(Comp m, f: 'a -> Random<_>) =
Comp <| fun state ->
let value, nextState = m state
let comp = f value
run nextState comp
member this.Return(a) = Comp (fun s -> a, s)
let random = RandomBuilder()
And here is how you use it:
let randomInt =
Comp <| fun state ->
let rnd = System.Random(state)
rnd.Next(0,1000), rnd.Next()
let rand =
random {
let! a = randomInt
let! b = randomInt
let! c = randomInt
return [a; b; c ]
}
rand.Run(0)
|> printfn "%A"
In this version you separately build up the computation (and store it inside the Random type), and then you run it passing in the initial state. Look at how types on the builder methods are inferred and compare them to what MSDN documentation describes.
Edit: Constructing a builder object once and using the binding as an alias of sorts is mostly convention, but it's well justified in that it makes sense for the builders to be stateless. I can see why having parameterized builders seems like a useful feature, but I can't honestly imagine a convincing use case for it.
The key selling point of monads is the separation of definition and execution of a computation.
In your case - what you want to be able to do is to take a representation of your computation and be able to run it with some state - perhaps 0, perhaps 42. You don't need to know the initial state to define a computation that will use it. By passing in the state to the builder, you end up blurring the line between definition and execution, and this simply makes the workflow less useful.
Compare that with async workflow - when you write an async block, you don't make the code run asynchronously. You only create an Async<'a> object representing a computation that will produce an object of 'a when you run it - but how you do it, is up to you. The builder doesn't need to know.
I'm trying to work through an example of FsCheck right now for a type with discriminated unions in order to establish best practices for our larger project. Right now I am getting null from my generator and I am not sure why. In the following code, DataGen.containerGenerator is null.
namespace Container
open System
open Xunit
open FsCheck
module ContainerLibrary =
type [<Measure>] oz
type Container =
| Cup of Common
| Bowl of Common
and Common =
{ Volume :decimal<oz>
Weight :decimal}
module DataGen =
type Generators =
static member arbVolume =
FsCheck.Gen.choose (1, 16)
|> FsCheck.Gen.map(fun x -> (decimal x / 8.0M) * 1.0M<ContainerLibrary.oz>)
|> FsCheck.Arb.fromGen
FsCheck.Arb.register<Generators>() |> ignore
let bowlGenerator =
FsCheck.Gen.map2 (fun a b -> ContainerLibrary.Bowl( { Volume = a
Weight = b}))
(Generators.arbVolume.Generator)
(FsCheck.Arb.generate<decimal>)
let cupGenerator =
FsCheck.Gen.map2 (fun a b -> ContainerLibrary.Cup( { Volume = a
Weight = b}))
(Generators.arbVolume.Generator)
(FsCheck.Arb.generate<decimal>)
let containerGenerator =
Gen.oneof [bowlGenerator; cupGenerator]
module Tests =
[<Fact;>]
let ``01 : Containers must be no more than 20 oz`` () =
//Is this the best way to get one of something?
let c = FsCheck.Gen.sample 0 1 DataGen.containerGenerator |> Seq.head
Assert.NotNull (c)
It doesn't seem to be null when I run it, even when I get more values. Which version of FsCheck are you using?
[<Fact;>]
let ``01 : Containers must be no more than 20 oz`` () =
//Is this the best way to get one of something?
Gen.sample 0 100 DataGen.containerGenerator |> Seq.iter(fun c -> printf "%A" c; Assert.NotNull (c))
In any case, there are several things to note about what you're doing.
FsCheck uses reflection to register generators; and type of measure type parameters cannot be seen by reflection. So the Arb.register would actually override the decimal generator, for all decimals.
Somehow the FsCheck. qualifications you used confused intellisense no end.
Gen.sample is a reasonable way to test generators but I mostly use it in interactive settings; if you've gone through the trouble of setting up a test I tend to use FsCheck's built in test-case observation capabilities. See "Observing test case distribution" here: https://fsharp.github.io/FsCheck/Properties.html
Using Arb.register in the module init like you're doing is a bit brittle, depending on module initialization rules in F# to register generators. If you're using Xunit, it's much better to use the built-in integration to reduce inevitable frustration in this area.
I've rewritten your example a bit taking some of these things into account:
module DataGen =
open ContainerLibrary
//can't really register this one because of the measure, would override all decimal generatos
let volumeGenerator =
Gen.choose (1, 16)
|> Gen.map(fun x -> (decimal x / 8.0M) * 1.0M<ContainerLibrary.oz>)
let commonGenerator =
Gen.map2 (fun a b -> { Volume = a
Weight = b})
(volumeGenerator)
(Arb.generate<decimal>)
//in case you like applicative style, otherwise completely equivalent
let commonGeneratorAlternative =
(fun a b -> { Volume = a; Weight = b}) <!> volumeGenerator <*> Arb.generate<decimal>
let bowlGenerator = Gen.map Bowl commonGenerator
let cupGenerator = Gen.map Cup commonGenerator
let containerGenerator =
Gen.oneof [bowlGenerator; cupGenerator]
type Generators =
static member Container() = containerGenerator |> Arb.fromGen
module Tests =
open FsCheck.Xunit
open ContainerLibrary
//use PropertyAttribute from FsCheck.Xunit
//use the defined container generator - can also move this to module level
//other ways to parametrize
[<Property(Arbitrary=[|typeof<DataGen.Generators>|])>]
//thanks to PropertyAttribute can now just take container as argument
let ``01 : Containers must be no more than 20 oz`` (container:Container) =
match container with
| Cup common
| Bowl common -> common.Volume <= 20.0M<oz>
|> Prop.collect container //see the generated values in the output
This outputs something like:
Ok, passed 100 tests.
1% Cup {Volume = 2.0M;
Weight = -0.0000221360928858744815609M;}.
1% Cup {Volume = 1.8750M;
Weight = 922337.20325598085121M;}.
etc
I'm looking for a clean set of ways to manage Test Specific Equality in F# unit tests. 90% of the time, the standard Structural Equality fits the bill and I can leverage it with unquote to express the relation between my result and my expected.
TL;DR "I can't find a clean way to having a custom Equality function for one or two properties in a value which 90% of is well served by Structural Equality, does F# have a way to match an arbitrary record with custom Equality for just one or two of its fields?"
Example of a general technique that works for me
When verifying a function that performs a 1:1 mapping of a datatype to another, I'll often extract matching tuples from both sides of in some cases and compare the input and output sets. For example, I have an operator:-
let (====) x y = (x |> Set.ofSeq) = (y |> Set.ofSeq)
So I can do:
let inputs = ["KeyA",DateTime.Today; "KeyB",DateTime.Today.AddDays(1); "KeyC",DateTime.Today.AddDays(2)]
let trivialFun (a:string,b) = a.ToLower(),b
let expected = inputs |> Seq.map trivialFun
let result = inputs |> MyMagicMapper
test <# expected ==== actual #>
This enables me to Assert that each of my inputs has been mapped to an output, without any superfluous outputs.
The problem
The problem is when I want to have a custom comparison for one or two of the fields.
For example, if my DateTime is being passed through a slightly lossy serialization layer by the SUT, I need a test-specific tolerant DateTime comparison. Or maybe I want to do a case-insensitive verification for a string field
Normally, I'd use Mark Seemann's SemanticComparison library's Likeness<Source,Destination> to define a Test Specific equality, but I've run into some roadblocks:
tuples: F# hides .ItemX on Tuple so I can't define the property via a .With strongly typed field name Expression<T>
record types: TTBOMK these are sealed by F# with no opt-out so SemanticComparison can't proxy them to override Object.Equals
My ideas
All I can think of is to create a generic Resemblance proxy type that I can include in a tuple or record.
Or maybe using pattern matching (Is there a way I can use that to generate an IEqualityComparer and then do a set comparison using that?)
Alternate failing test
I'm also open to using some other function to verify the full mapping (i.e. not abusing F# Set or involving too much third party code. i.e. something to make this pass:
let sut (a:string,b:DateTime) = a.ToLower(),b + TimeSpan.FromTicks(1L)
let inputs = ["KeyA",DateTime.Today; "KeyB",DateTime.Today.AddDays(1.0); "KeyC",DateTime.Today.AddDays(2.0)]
let toResemblance (a,b) = TODO generate Resemblance which will case insensitively compare fst and tolerantly compare snd
let expected = inputs |> List.map toResemblance
let result = inputs |> List.map sut
test <# expected = result #>
Firstly, thanks to all for the inputs. I was largely unaware of SemanticComparer<'T> and it definitely provides a good set of building blocks for building generalized facilities in this space. Nikos' post gives excellent food for thought in the area too. I shouldn't have been surprised Fil exists too - #ptrelford really does have a lib for everything (the FSharpValue point is also v valuable)!
We've thankfully arrived at a conclusion to this. Unfortunately it's not a single all-encompassing tool or technique, but even better, a set of techniques that can be used as necessary in a given context.
Firstly, the issue of ensuring a mapping is complete is really an orthogonal concern. The question refers to an ==== operator:-
let (====) x y = (x |> Set.ofSeq) = (y |> Set.ofSeq)
This is definitely the best default approach - lean on Structural Equality. One thing to note is that, being reliant on F# persistent sets, it requires your type to support : comparison (as opposed to just : equality).
When doing set comparisons off the proven Structural Equality path, a useful technique is to use HashSet<T> with a custom IEqualityComparer:-
[<AutoOpen>]
module UnorderedSeqComparisons =
let seqSetEquals ec x y =
HashSet<_>( x, ec).SetEquals( y)
let (==|==) x y equals =
let funEqualityComparer = {
new IEqualityComparer<_> with
member this.GetHashCode(obj) = 0
member this.Equals(x,y) =
equals x y }
seqSetEquals funEqualityComparer x y
the equals parameter of ==|== is 'a -> 'a -> bool which allows one to use pattern matching to destructure args for the purposes of comparison. This works well if either the input or the result side are naturally already tuples. Example:
sut.Store( inputs)
let results = sut.Read()
let expecteds = seq { for x in inputs -> x.Name,x.ValidUntil }
test <# expecteds ==|== results
<| fun (xN,xD) (yN,yD) ->
xF=yF
&& xD |> equalsWithinASecond <| yD #>
While SemanticComparer<'T> can do a job, it's simply not worth bothering for tuples with when you have the power of pattern matching. e.g. Using SemanticComparer<'T>, the above test can be expressed as:
test <# expecteds ==~== results
<| [ funNamedMemberComparer "Item2" equalsWithinASecond ] #>
using the helper:
[<AutoOpen>]
module MemberComparerHelpers =
let funNamedMemberComparer<'T> name equals = {
new IMemberComparer with
member this.IsSatisfiedBy(request: PropertyInfo) =
request.PropertyType = typedefof<'T>
&& request.Name = name
member this.IsSatisfiedBy(request: FieldInfo) =
request.FieldType = typedefof<'T>
&& request.Name = name
member this.GetHashCode(obj) = 0
member this.Equals(x, y) =
equals (x :?> 'T) (y :?> 'T) }
let valueObjectMemberComparer() = {
new IMemberComparer with
member this.IsSatisfiedBy(request: PropertyInfo) = true
member this.IsSatisfiedBy(request: FieldInfo) = true
member this.GetHashCode(obj) = hash obj
member this.Equals(x, y) =
x.Equals( y) }
let (==~==) x y mcs =
let ec = SemanticComparer<'T>( seq {
yield valueObjectMemberComparer()
yield! mcs } )
seqSetEquals ec x y
All of the above is best understood by reading Nikos Baxevanis' post NOW!
For types or records, the ==|== technique can work (except critically you lose Likeness<'T>s verifying coverage of fields). However the succinctness can make it a valuable tool for certain sorts of tests :-
sut.Save( inputs)
let expected = inputs |> Seq.map (fun x -> Mapped( base + x.ttl, x.Name))
let likeExpected x = expected ==|== x <| (fun x y -> x.Name = y.Name && x.ValidUntil = y.ValidUntil)
verify <# repo.Store( is( likeExpected)) #> once