I am experimenting with F# and want to do some constraint programming where I use Or-Tools. I have previously used the package together with Python, but I can’t get it to work with F#.
I follow the C# example: https://developers.google.com/optimization/cp/cp_solver#c_5
But get an error when try to add a constraint:
So this is kind of annoying, but it's how consuming operators that were overloaded in C# from F# works.
The reason why you couldn't use != like this is because:
The operator != is (and this is unusual) overloaded in C# as a static operator on the LinearExpr class.
The operator != compiles down to op_Inequality, but op_Inequality in F# is <>, not !=
F# already defines <> as a generic operator that accepts any member that satisfies the equality constraint, which LinearExpr does
The defined operator <> resolves correctly, and produces a bool, which is incompatible with model.Add because it doesn't expect a bool
The solution is to explicitly qualify your access to the operator like so:
LinearExpr.(<>) (x, y)
Note that because it takes tupled arguments in its definition, you must also tuple your arguments and you can't use it like a "normal" operator.
Here is the full F# solution, with a few small tweaks to make it idiomatic:
#r "nuget: Google.OrTools"
open Google.OrTools.Sat
let model = CpModel()
// Creates the variables.
let num_vals = 3L;
let x = model.NewIntVar(0L, num_vals - 1L, "x")
let y = model.NewIntVar(0L, num_vals - 1L, "y")
let z = model.NewIntVar(0L, num_vals - 1L, "z")
// Creates the constraints.
model.Add(LinearExpr.(<>) (x, y))
// Creates a solver and solves the model.
let solver = CpSolver();
let status = solver.Solve(model)
if status = CpSolverStatus.Optimal then
printfn $"x = {solver.Value(x)}"
printfn $"y = {solver.Value(y)}"
printfn $"z = {solver.Value(z)}"
A way to make this a bit nicer from F# is to define a module of operators that map to the LinearExpr operators like so:
module LinearExprOperators =
let ( ^<> ) (x: LinearExpr) (y: LinearExpr) = LinearExpr.(<>) (x, y)
let ( ^= ) (x: LinearExpr) (y: LinearExpr) = LinearExpr.(=) (x, y)
Then you can use those operators instead. Another annoyance is that it appears that + and - and * work just fine, because the F# type doesn't produce a different type like bool.
So in short, this particular API is a little annoying to use from F#.
Related
For a while F# has supported the ability to auto-quote using [<ReflectedDefinitionAttribute>]. Is there anything similar for laziness?
e.g.
member __.Quoted ([<ReflectedDefinitionAttribute>] quotation:Expr<'T>) = ...
member __.Thunked ([<LazyAttribute>] thunk:Lazy<'T>) = ...
I suppose I could use something like
member __.Quoted ([<ReflectedDefinitionAttribute>] quotation:Expr<'T>) =
Lazy (evaluate (<# fun () -> %quotation #>)) // evaluate using Unquote or similar
But wouldn't this be costly?
UPDATE:
I found a hack, it's not exactly what I would like but it give the correct behavior.
type Signal = Signal with
member __.Return x = x
member __.Delay (f:unit -> _) = f
let a = Signal { return randint }
let b = Signal { return randint }
let c = Signal { return a() + b() }
There is nothing like the ReflectedDefinition attribute for automatically turning things into delayed Lazy<'T> computations.
You are right that automatically quoting the argument achieves something like this. You could use the (very limited) LeafExpressionConverter.EvaluateQuotation to do this for some limited kinds of expressions, but as you note, this would be inefficient. The following is a proof of concept though (but you cannot call custom functions in the branches as this uses LINQ expressions):
open Microsoft.FSharp.Quotations
open Microsoft.FSharp.Linq.RuntimeHelpers
type A =
static member If<'T>(c:bool,
[<ReflectedDefinition>] t:Expr<'T>,
[<ReflectedDefinition>] f:Expr<'T>) =
if c then LeafExpressionConverter.EvaluateQuotation t :?> 'T
else LeafExpressionConverter.EvaluateQuotation f :?> 'T
A.If(1 = 2, 0, 1)
In practice, I think a more reasonable approach is to just use the built-in Lazy<'T> values. F# has a (not widely known) lazy keyword that gives you a bit nicer syntax for creating those:
let iff c (t:Lazy<_>) (f:Lazy<_>) =
if c then t.Value else f.Value
iff (1 = 2)
(lazy (printfn "true"; 41))
(lazy (printfn "false"; 42))
Why are parentheses needed on read_rest_of_csv below?
let read_rest_of_csv() =
csv_data.Add(csv_fileH.ReadFields()) |> ignore
not csv_fileH.EndOfData
while read_rest_of_csv() do ignore None
Without the parentheses, the loop will not terminate.
open System
open System.Threading
open System.Collections.Generic
open System.Linq
open System.Text
open System.Threading.Tasks
open System.IO
open Microsoft.VisualBasic.FileIO
[<EntryPoint>]
let main argv =
let csv_fileH = new TextFieldParser("test1.csv")
csv_fileH.TextFieldType = FieldType.Delimited |> ignore
let x = csv_fileH.SetDelimiters(",")
let csv_data = new List<string[]>()
let eod = csv_fileH.EndOfData
if not eod then
let column_headings = csv_fileH.ReadFields()
csv_data.Add(column_headings) |> ignore
let read_rest_of_csv =
csv_data.Add(csv_fileH.ReadFields()) |> ignore
not csv_fileH.EndOfData
while read_rest_of_csv do ignore None
0
I apologize that I cannot remember where I saw this. I think it was in SO. It's a nice example.
Could this be that without parens I'm dealing with a function object of sorts?
I am indeed coming from not only a C, C++, and C# background, but also an intermediate Clojure background as well. In my case with F# syntax, reading my Haskell manual in a little more detail might have helped, because the syntaxes seem similar.
It seems that people coming from C-family languages (C#, Java, C, C++, JavaScript) are having problems understanding the use of brackets in F#. I certainly had, and it took me some years learning how things work.
In a nutshell, the most basic building block in F# is a value. Values can be let-bound:
let foo = bar
This means that foo is a value, which happens to be equal to bar.
Functions are also values:
// 'a -> 'a * 'a
let f = fun x -> x, x
Here, f is a function that takes some value (x) and returns a tuple with x as both the first and the second element.
That's a bit cumbersome to write, so there's a shorthand for that:
// 'a -> 'a * 'a
let f x = x, x
Notice that there are no brackets in these expressions.
Sometimes you need to adjust the precedence of operators. Just like in maths, 1 + 2 * 3 (which is equivalent to 1 + (2 * 3)) isn't the same as (1 + 2) * 3. In F#, you also use brackets to override precedence. Thus
// 'a -> string * 'a
let f x = someOtherFunction x, x
isn't the same as
// x:'a -> string
let f x = someOtherFunction (x, x)
(in this case, someOtherFunction is a function that returns a string.)
Notice that the brackets don't denote a function call; they're only there to control order of evaluation.
Sometimes, you want to define a function that doesn't take any input. You can't, however, define it like this:
let f = whatever
because that would make it a value that's immediately let-bound to whatever. Instead, you can let the function take a value of the built-in type unit. This type only has a single value, which is written ():
let f () = whatever
This means that f is a function that pattern matches its input against the only known value of unit.
Whenever you invoke f with (), the expression whatever is evaluated and returned.
Without the parentheses, the content executes once and never again. read_rest_of_csv has a type of bool: You are basically saying while true do ignore None.
The parentheses indicate that read_rest_of_csv has type unit -> bool, so every time you invoke it, it reads a row and moves the cursor. Otherwise, it will only do this once.
The answer to your question is that:
let read_rest_of_csv =
csv_data.Add(csv_fileH.ReadFields()) |> ignore
not csv_fileH.EndOfData
is not a function at all. This is no different from:
> let i = 1;;
val i : int = 1
This declares a binding with an integer value. If you want to declare a binding with a function value which takes no parameters, that looks like this:
> let i () = 1;;
val i : unit -> int
The exact same reasoning applies to read_rest_of_csv. Without the parenthesis, you are declaring a binding with type bool. With the parenthesis, you are declaring a binding with type unit->bool i.e. a binding with a function value where the function takes no inputs and returns a bool value.
say I've got a tuple:
let t = (1,'a')
and I want to create a triple/3-tuple:
let createTriple (a,b) c =
(a, b, c)
is there a syntax in F# where I can create an n-tuple from an n-1-tuple and a singular value?
let createNTuple (a..z) q =
(a..z, q)
You're asking about stepping outside the type system, so sure you can do it through reflection, but not in a type-safe way.
let createNTuple tuple x =
let fields = [|yield! FSharpValue.GetTupleFields(tuple); yield box x|]
let fieldsTy = [|yield! FSharpType.GetTupleElements(tuple.GetType()); yield x.GetType()|]
let ty = FSharpType.MakeTupleType(fieldsTy)
FSharpValue.MakeTuple(fields, ty) |> unbox
let (a: int, b: char, c: string) = createNTuple (1, 'a') "foo"
What I would suggest is defining a set of functions (perhaps as static members on a class, so that you can make use of overloading), that would handle appending to tuples in a type-safe way. Realistically, you should only need a 5- or 7-tuple at most, anything more feels like pushing it too far (though you can still try to handle those cases through reflection if you care).
Something like this:
type TupleAppender =
static member Append((a, b), x) = (a, b, x)
static member Append((a, b, c), x) = (a, b, c, x)
(* and so on *)
This is boilerplate, but at least you're within the type system.
No, what you're asking is impossible. What type signature would such a function have? As opposed to lists, tuples can have differently typed elements, but they are of fixed length - precisely because you need to know the type for each element. You can't specify a type for any n-element tuple. If you need variable-length structures, use a list.
I'm looking for a clean set of ways to manage Test Specific Equality in F# unit tests. 90% of the time, the standard Structural Equality fits the bill and I can leverage it with unquote to express the relation between my result and my expected.
TL;DR "I can't find a clean way to having a custom Equality function for one or two properties in a value which 90% of is well served by Structural Equality, does F# have a way to match an arbitrary record with custom Equality for just one or two of its fields?"
Example of a general technique that works for me
When verifying a function that performs a 1:1 mapping of a datatype to another, I'll often extract matching tuples from both sides of in some cases and compare the input and output sets. For example, I have an operator:-
let (====) x y = (x |> Set.ofSeq) = (y |> Set.ofSeq)
So I can do:
let inputs = ["KeyA",DateTime.Today; "KeyB",DateTime.Today.AddDays(1); "KeyC",DateTime.Today.AddDays(2)]
let trivialFun (a:string,b) = a.ToLower(),b
let expected = inputs |> Seq.map trivialFun
let result = inputs |> MyMagicMapper
test <# expected ==== actual #>
This enables me to Assert that each of my inputs has been mapped to an output, without any superfluous outputs.
The problem
The problem is when I want to have a custom comparison for one or two of the fields.
For example, if my DateTime is being passed through a slightly lossy serialization layer by the SUT, I need a test-specific tolerant DateTime comparison. Or maybe I want to do a case-insensitive verification for a string field
Normally, I'd use Mark Seemann's SemanticComparison library's Likeness<Source,Destination> to define a Test Specific equality, but I've run into some roadblocks:
tuples: F# hides .ItemX on Tuple so I can't define the property via a .With strongly typed field name Expression<T>
record types: TTBOMK these are sealed by F# with no opt-out so SemanticComparison can't proxy them to override Object.Equals
My ideas
All I can think of is to create a generic Resemblance proxy type that I can include in a tuple or record.
Or maybe using pattern matching (Is there a way I can use that to generate an IEqualityComparer and then do a set comparison using that?)
Alternate failing test
I'm also open to using some other function to verify the full mapping (i.e. not abusing F# Set or involving too much third party code. i.e. something to make this pass:
let sut (a:string,b:DateTime) = a.ToLower(),b + TimeSpan.FromTicks(1L)
let inputs = ["KeyA",DateTime.Today; "KeyB",DateTime.Today.AddDays(1.0); "KeyC",DateTime.Today.AddDays(2.0)]
let toResemblance (a,b) = TODO generate Resemblance which will case insensitively compare fst and tolerantly compare snd
let expected = inputs |> List.map toResemblance
let result = inputs |> List.map sut
test <# expected = result #>
Firstly, thanks to all for the inputs. I was largely unaware of SemanticComparer<'T> and it definitely provides a good set of building blocks for building generalized facilities in this space. Nikos' post gives excellent food for thought in the area too. I shouldn't have been surprised Fil exists too - #ptrelford really does have a lib for everything (the FSharpValue point is also v valuable)!
We've thankfully arrived at a conclusion to this. Unfortunately it's not a single all-encompassing tool or technique, but even better, a set of techniques that can be used as necessary in a given context.
Firstly, the issue of ensuring a mapping is complete is really an orthogonal concern. The question refers to an ==== operator:-
let (====) x y = (x |> Set.ofSeq) = (y |> Set.ofSeq)
This is definitely the best default approach - lean on Structural Equality. One thing to note is that, being reliant on F# persistent sets, it requires your type to support : comparison (as opposed to just : equality).
When doing set comparisons off the proven Structural Equality path, a useful technique is to use HashSet<T> with a custom IEqualityComparer:-
[<AutoOpen>]
module UnorderedSeqComparisons =
let seqSetEquals ec x y =
HashSet<_>( x, ec).SetEquals( y)
let (==|==) x y equals =
let funEqualityComparer = {
new IEqualityComparer<_> with
member this.GetHashCode(obj) = 0
member this.Equals(x,y) =
equals x y }
seqSetEquals funEqualityComparer x y
the equals parameter of ==|== is 'a -> 'a -> bool which allows one to use pattern matching to destructure args for the purposes of comparison. This works well if either the input or the result side are naturally already tuples. Example:
sut.Store( inputs)
let results = sut.Read()
let expecteds = seq { for x in inputs -> x.Name,x.ValidUntil }
test <# expecteds ==|== results
<| fun (xN,xD) (yN,yD) ->
xF=yF
&& xD |> equalsWithinASecond <| yD #>
While SemanticComparer<'T> can do a job, it's simply not worth bothering for tuples with when you have the power of pattern matching. e.g. Using SemanticComparer<'T>, the above test can be expressed as:
test <# expecteds ==~== results
<| [ funNamedMemberComparer "Item2" equalsWithinASecond ] #>
using the helper:
[<AutoOpen>]
module MemberComparerHelpers =
let funNamedMemberComparer<'T> name equals = {
new IMemberComparer with
member this.IsSatisfiedBy(request: PropertyInfo) =
request.PropertyType = typedefof<'T>
&& request.Name = name
member this.IsSatisfiedBy(request: FieldInfo) =
request.FieldType = typedefof<'T>
&& request.Name = name
member this.GetHashCode(obj) = 0
member this.Equals(x, y) =
equals (x :?> 'T) (y :?> 'T) }
let valueObjectMemberComparer() = {
new IMemberComparer with
member this.IsSatisfiedBy(request: PropertyInfo) = true
member this.IsSatisfiedBy(request: FieldInfo) = true
member this.GetHashCode(obj) = hash obj
member this.Equals(x, y) =
x.Equals( y) }
let (==~==) x y mcs =
let ec = SemanticComparer<'T>( seq {
yield valueObjectMemberComparer()
yield! mcs } )
seqSetEquals ec x y
All of the above is best understood by reading Nikos Baxevanis' post NOW!
For types or records, the ==|== technique can work (except critically you lose Likeness<'T>s verifying coverage of fields). However the succinctness can make it a valuable tool for certain sorts of tests :-
sut.Save( inputs)
let expected = inputs |> Seq.map (fun x -> Mapped( base + x.ttl, x.Name))
let likeExpected x = expected ==|== x <| (fun x y -> x.Name = y.Name && x.ValidUntil = y.ValidUntil)
verify <# repo.Store( is( likeExpected)) #> once
I am creating Linq expression trees from F# that operates on a custom datatype I have. The type is a very simple discriminated union that has the usual arithmetic operators overloaded. But for some reason I cannot create arithmetic linq expression nodes due to the fact that it can't find the correct overload. Thing is, I swear I had this working some time ago but I can't figure out what I changed to make it break.
I'll attach a small code sample showing the problem. The datatype below has the Addition operator overloaded. Using the overloaded operator works like a charm, but when I try to create an addition expression tree node using Expression.Add(lhs, rhs) the system throws an exception complaining that it can't find the overload for the Add operation.
Does anyone have an idea of what I am doing wrong?
Thank you,
Rickard
open System.Linq.Expressions
module DataType =
exception NotImplementedYet of string
type DataCarrier =
| ScalarCarrier of float
| VectorCarrier of float array
member this.Add(other) =
match (this, other) with
| ScalarCarrier(x), ScalarCarrier(y) -> ScalarCarrier(x + y)
| VectorCarrier(u), VectorCarrier(v) ->
VectorCarrier(Array.map2 (fun x y -> x + y) u v)
| _,_ -> raise (NotImplementedYet("No go!"))
static member (+) (lhs:DataCarrier, rhs) =
lhs.Add(rhs)
module Main =
let createAddOp (lhs:DataType.DataCarrier) (rhs:DataType.DataCarrier) =
let clhs = Expression.Constant(lhs)
let crhs = Expression.Constant(rhs)
Expression.Add(clhs, crhs)
(* no problems with this one *)
printf "Testing operator overloading: %A" (DataType.ScalarCarrier(1.0)
+ DataType.ScalarCarrier(2.0))
(* this throws an exception *)
printf "Testing expr construction %A" (Main.createAddOp
(DataType.ScalarCarrier(1.0))
(DataType.ScalarCarrier(2.0)))
One solution is to explicitly type the Expression operands (giving them the static type DataType.DataCarrier instead of their runtime type DataType.DataCarrier.ScalarCarrier):
module Main =
let createAddOp (lhs:DataType.DataCarrier) (rhs:DataType.DataCarrier) =
let clhs = Expression.Constant(lhs, typeof<DataType.DataCarrier>)
let crhs = Expression.Constant(rhs, typeof<DataType.DataCarrier>)
Expression.Add(clhs, crhs)
Another option would be to explicitly pass the addition operator to use:
module Main =
let createAddOp (lhs:DataType.DataCarrier) (rhs:DataType.DataCarrier) =
let clhs = Expression.Constant(lhs)
let crhs = Expression.Constant(rhs)
Expression.Add(clhs, crhs, typeof<DataType.DataCarrier>.GetMethod("op_Addition"))
I am surprised that your original code doesn't work, though. It appears to be a limitation in how expression trees find relevant add operators (that is, it appears that Linq only looks for add operators on the runtime types of operands).