Premise: I thought that the piping operator is nothing but syntactic sugar, thus x |> f should be exactly the same as f(x).
Analogously, I thought that f (fun x -> foo) is equivalent to let g = fun x -> foo; f g
But apparently there are differences that I do not understand.
Example 1:
static member contents =
let files = Directory.EnumerateFiles (__SOURCE_DIRECTORY__+ #"\foo\bar.txt")
let fileList = List.ofSeq files
fileList |> List.map (fun f -> TestCaseData(f).SetName(""))
This works fine: TestCaseData expects an arg:obj which is matched by f which in turn in inferred to be a string since fileList is a list of file names.
However the following does not work
static member contents =
let files = Directory.EnumerateFiles (__SOURCE_DIRECTORY__+ #"\foo\bar.txt")
let fileList = List.ofSeq files
List.map (fun f -> TestCaseData(f).SetName("")) fileList
Nothing but the last line has changed. Suddenly f is inferred as obj [] and TestCaseData requires an argument of type obj [] and hence I get an error
Error 1 Type mismatch. Expecting a obj [] list but given a string list
The type 'obj []' does not match the type 'string'
I would have thought that both snippets are equivalent in produce correct code but only the first does?!
Example 2:
[<TestCase("nonsense", TestName="Nonsense")>]
member x.InvalidInputs str =
let lexbuf = Microsoft.FSharp.Text.Lexing.LexBuffer<char>.FromString(str)
Assert.Throws<FatalError> (fun () -> ParsePkg.parse "Dummy path" lexbuf |> ignore)
|> ignore
Above everything works fine.
[<TestCase("nonsense", TestName="Nonsense")>]
member x.InvalidInputs str =
let lexbuf = Microsoft.FSharp.Text.Lexing.LexBuffer<char>.FromString(str)
let ff = fun () -> ParsePkg.parse "Dummy path" lexbuf |> ignore
Assert.Throws<FatalError> (ff)
|> ignore
As you see, all I did was pull out the argument of the assertion by first defining let ff = ... (for readability reasons, say) and suddenly the compiler points to the (ff) argument and complains:
Error 2 This expression was expected to have type TestDelegate but here has type unit -> unit
TestDelegate is a type of NUnit that I am using here and it happens to coincide with unit->unit so I'd assume it would be unified anyway but that does not even matter. Why at all is it possible that the type changes, since again I believe to have done a purely syntactic replacement?!
type inferences is done sequential top to bottom. So in the first case fileList is the first lexical argument.
The information that the fileList is a list of strings is then used in the piping expression. To know whether stringis a legal type for f the signature of TestCaseData is used. As commented, based on the error message TestCaseData probably accepts [<Params>] obj [] making a single string argument valid.
In the second version there's no information to use when determining the type of f other than the signature of TestCaseData and therefor f is inferred to be of type obj []
Similar is true in the other example. Just the other way around. Pulling out the function removes the information that it's supposed to be of type TestDelegate.
At the lexical point the only information available is that it's a function of type unit->unit.
When the function is used at a program point where a TestDelegate is required. The type inference tests whether the function can be used as a TestDelegate and if so, infers the type to be TestDelegate
Related
I am trying to develop F# type provider.
It provides some DTOs (with the structure described in some external document) and a set of methods for processing them. The processing algorithm is based on reflection, and I want to have a single quotation representing it.
Generally, this algorithm must pass all method call arguments to the already written function serialize: obj -> MySerializationFormat, storing all results in a list, so I getting a value of MySerializationFormat list.
Code sample below shows, how I tried to do that for first time:
let serialize (value: obj) = ...
let processingCode: Expr list -> Expr =
fun args ->
let serializeArgExpr (arg: Expr) = <# serialize %%arg} #>
let argsExprs = List.map serializeArgExpr args
let serializedArgList =
List.foldBack (fun head tail -> <# (%head) :: (%tail)#>) argsExprs <# [] #>
// futher processing
At that point I faced with exception: In function serializeArgExpr the actual type of value in arg: Expr may vary, it can be some primitive type (e.g string, int, float), or some provided type. The problem is %% operator treats that arg as an expression of the obj type. Type check is performed on that line in Microsoft.FSharp.Quotations.Patterns module, in function fillHolesInRawExpr.
So, as the actual type of my term not matched the treated type for "hole" in the quotation, it throws invalidArg.
I have tried several technics to avoid these exceptions with casting operations in my quotation, but they don't work. Then I found Expr.Coerce(source, target) function, which looks like solving my problem. I have changed the code of serializeArgExpr to something like that:
let serializeArgExpr (arg: Expr) =
let value' = Expr.Coerce(value, typeof<obj>)
<# serialize %%value' } #>
Then faced a new strange exception:
The design-time type (point to a code line that uses my processingCode) utilized by a type provider was not found in the target reference assembly set
For me, it seems that my problem is to cast the type of value in any input Expr to an obj type. Thank you for diving in and trying to help.
In this snippet, the current (4.1) compiler complains that it can't determine which overload to Convert to use (a type annotation may be needed), but it recognizes s as a string. Why?
let wtf (list:string List) =
List.map (fun s -> System.Convert.ToInt32(s)) list
As mentioned in the comments, the compiler uses single pass over the code to determine the types. This means that it will report an error (e.g. when it cannot determine which overload to use) as it passes over the problematic code. If it later finds more information, the additional information is not used.
This is exactly what is happening here. Consider:
let wtf (list:string List) =
List.map (fun s -> s) list
Here, the type of s is string and everything works fine. However, when checking fun s -> s, the compiler does not yet know that the type of s is string and it treats it as a value of type variable 'a - it checks the type of function as 'a -> 'a and later unifies 'a with string. Going back to your case:
let wtf (list:string List) =
List.map (fun s -> System.Convert.ToInt32(s)) list
When checking the lambda, the compiler knows s is of type 'a and it fails to resolve the overload of ToInt32. It reports the error, skips the sub-expression and continues (to get as useful IntelliSense as it can) and it later figures out that s was actually string. This appears in the IntelliSense, but it does not undo the error.
Just for the record, the best way to avoid this is to use |> and put the list input first:
let awesome (list:string List) =
list |> List.map (fun s -> System.Convert.ToInt32(s))
given
[
1,"test2"
3,"test"
]
|> dict
// turn it into keyvaluepair sequence
|> Seq.map id
|> fun x -> x.ToDictionary<_,_,_>((fun x -> x.Key), fun x -> x.Value)
which fails to compile if I don't explicitly use the <_,_,_> after ToDictionary.
Intellisense works just fine, but compilation fails with the error: Lookup on object of indeterminate type based on information prior to this program point
So, it seems, Intellisense knows how to resolve the method call.
This seems to be a clue
|> fun x -> x.ToDictionary<_,_>((fun x -> x.Key), fun x -> x.Value)
fails with
Type constraint mismatch.
The type 'b -> 'c is not compatible with type IEqualityComparer<'a>
The type 'b -> 'c' is not compatible with the type 'IEqualityComparer<'a>'
(using external F# compiler)
x.ToDictionary((fun x -> x.Key), id)
works as expected as does
let vMap (item:KeyValuePair<_,_>) = item.Value
x.ToDictionary((fun x -> x.Key), vMap)
I've replicated the behavior in FSI and LinqPad.
As a big fan of and avid reader of Eric Lippert I really want to know
what overload resolution, (or possibly extension methods from different places) are conflicting here that the compiler is confused by?
Even though the types are known ahead, the compiler's getting confused between the overload which takes an element selector and a comparer. The lambda compiles to FSharpFunc rather than the standard delegate types in C# like Action or Func, and issues do come up translating from one to the other. To make it work, you can :
Supply a type annotation for the offending Func
fun x -> x.ToDictionary((fun pair -> pair.Key), (fun (pair : KeyValuePair<_, _>) -> pair.Value)) //compiles
or name the argument as a hint
fun x -> x.ToDictionary((fun pair -> pair.Key), elementSelector = (fun (pair) -> pair.Value))
or force it to pick the 3 argument version:
x.ToLookup((fun pair -> pair.Key), (fun (pair) -> pair.Value), EqualityComparer.Default)
Aside
In your example,
let vMap (item:KeyValuePair<_,_>) = item.Value
x.ToDictionary((fun x -> x.Key), vMap)
you would explicitly need to annotate vMap because the compiler cannot find out what type the property exists on without another pass. For example,
List.map (fun x -> x.Length) ["one"; "two"] // this fails to compile
This is one of the reasons why the pipe operator is so useful, because it allows you to avoid type annotations:
["one"; "two"] |> List.map (fun x -> x.Length) // works
List.map (fun (x:string) -> x.Length) ["one"; "two"] //also works
The short answer:
The extension method ToDictionary is defined like this:
static member ToDictionary<'TSource,_,_>(source,_,_)
but is called like this:
source.ToDictionary<'TSource,_,_>(_,_)
The long answer:
This is the F# type signature of the function you are calling from msdn.
static member ToDictionary<'TSource, 'TKey, 'TElement> :
source:IEnumerable<'TSource> *
keySelector:Func<'TSource, 'TKey> *
elementSelector:Func<'TSource, 'TElement> -> Dictionary<'TKey, 'TElement>
But I only specified two regular parameters: keySelector and elementSelector. How come this has a source parameter?!
The source parameter is actually not put in the parenthesis, but is passed in by saying x.ToDictionary, where x is the source parameter. This is actually an example of a type extension. These kinds of methods are very natural in a functional programming language like F#, but more uncommon in an object oriented language like C#, so if you're coming from the C# world, it will be pretty confusing. Anyway, if we look at the C# header, it is a little easier to understand what is going on:
public static Dictionary<TKey, TElement> ToDictionary<TSource, TKey, TElement>(
this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector,
Func<TSource, TElement> elementSelector
)
So the method is defined with a "this" prefix on a first parameter even though it is technically static. It basically allows you to add methods to already defined classes without re-compiling or extending them. This is called prototyping. It's kinda rare if you're a C# programmer, but languages like python and javascript force you to be aware of this. Take this example from https://docs.python.org/3/tutorial/classes.html:
class Dog:
tricks = [] # mistaken use of a class variable
def __init__(self, name):
self.name = name
def add_trick(self, trick):
self.tricks.append(trick)
>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.add_trick('roll over')
>>> e.add_trick('play dead')
>>> d.tricks # unexpectedly shared by all dogs
['roll over', 'play dead']
The method add_trick is defined with self as a first parameter, but the function is called as d.add_trick('roll over'). F# actually does this naturally as well, but in a way that mimics the way the function is called. When you declare:
member x.doSomething() = ...
or
member this.doSomething() = ...
Here, you are adding function doSomething to the prototype (or class definition) of "x"/"this". Thus, in your example, you actually have three type parameters, and three regular parameters, but one of them is not used in the call. All you have left is to declare the key selector function, and the element selector function, which you did. That's why it looks weird.
I thought that conversions between F# functions and System.Func had to be done manually, but there appears to be a case where the compiler (sometimes) does it for you. And when it goes wrong the error message isn't accurate:
module Foo =
let dict = new System.Collections.Generic.Dictionary<string, System.Func<obj,obj>>()
let f (x:obj) = x
do
// Question 1: why does this compile without explicit type conversion?
dict.["foo"] <- fun (x:obj) -> x
// Question 2: given that the above line compiles, why does this fail?
dict.["bar"] <- f
The last line fails to compile, and the error is:
This expression was expected to have type
System.Func<obj,obj>
but here has type
'a -> obj
Clearly the function f doesn't have a signature of 'a > obj. If the F# 3.1 compiler is happy with the first dictionary assignment, then why not the second?
The part of the spec that should explain this is 8.13.7 Type Directed Conversions at Member Invocations. In short, when invoking a member, an automatic conversion from an F# function to a delegate will be applied. Unfortunately, the spec is a bit unclear; from the wording it seems that this conversion might apply to any function expression, but in practice it only appears to apply to anonymous function expressions.
The spec is also a bit out of date; in F# 3.0 type directed conversions also enable a conversion to a System.Linq.Expressions.Expression<SomeDelegateType>.
EDIT
In looking at some past correspondence with the F# team, I think I've tracked down how a conversion could get applied to a non-syntactic function expression. I'll include it here for completeness, but it's a bit of a strange corner case, so for most purposes you should probably consider the rule to be that only syntactic functions will have the type directed conversion applied.
The exception is that overload resolution can result in converting an arbitrary expression of function type; this is partly explained by section 14.4 Method Application Resolution, although it's pretty dense and still not entirely clear. Basically, the argument expressions are only elaborated when there are multiple overloads; when there's just a single candidate method, the argument types are asserted against the unelaborated arguments (note: it's not obvious that this should actually matter in terms of whether the conversion is applicable, but it does matter empirically). Here's an example demonstrating this exception:
type T =
static member M(i:int) = "first overload"
static member M(f:System.Func<int,int>) = "second overload"
let f i = i + 1
T.M f |> printfn "%s"
EDIT: This answer explains only the mysterious promotion to 'a -> obj. #kvb points out that replacing obj with int in OPs example still doesn't work, so that promotion is in itself insufficient explanation for the observed behaviour.
To increase flexibility, the F# type elaborator may under certain conditions promote a named function from f : SomeType -> OtherType to f<'a where 'a :> SomeType> : 'a -> OtherType. This is to reduce the need for upcasts. (See spec. 14.4.2.)
Question 2 first:
dict["bar"] <- f (* Why does this fail? *)
Because f is a "named function", its type is promoted from f : obj -> obj following sec. 14.4.2 to the seemingly less restrictive f<'a where 'a :> obj> : 'a -> obj. But this type is incompatible with System.Func<obj, obj>.
Question 1:
dict["foo"] <- fun (x:obj) -> x (* Why doesn't this, then? *)
This is fine because the anonymous function is not named, and so sec. 14.4.2 does not apply. The type is never promoted from obj -> obj and so fits.
We can observe the interpreter exhibit behaviour following 14.4.2:
> let f = id : obj -> obj
val f : (obj -> obj) (* Ok, f has type obj -> obj *)
> f
val it : ('a -> obj) = <fun:it#135-31> (* f promoted when used. *)
(The interpreter doesn't output constraints to obj.)
As I know, explicit type parameters in value definitions is a one way to overcome "value restriction" problem.
Is there another cases when I need to use them?
Upd: I mean "explicitly generic constructs", where type parameter is enclosed in angle brackets, i.e.
let f<'T> x = x
Polymorphic recursion is another case. That is, if you want to use a different generic instantiation within the function body, then you need to use explicit parameters on the definition:
// perfectly balanced tree
type 'a PerfectTree =
| Single of 'a
| Node of ('a*'a) PerfectTree
// need type parameters here
let rec fold<'a,'b> (f:'a -> 'b) (g:'b->'b->'b) : 'a PerfectTree -> 'b = function
| Single a -> f a
| Node t -> t |> fold (fun (a,b) -> g (f a) (f b)) g
let sum = fold id (+)
let ten = sum (Node(Node(Single((1,2),(3,4)))))
This would likely be rare, but when you want to prevent further generalization (§14.6.7):
Explicit type parameter definitions on value and member definitions can affect the process of type inference and generalization. In particular, a declaration that includes explicit generic parameters will not be generalized beyond those generic parameters. For example, consider this function:
let f<'T> (x : 'T) y = x
During type inference, this will result in a function of the following type, where '_b is a type inference variable that is yet to be resolved.
f<'T> : 'T -> '_b -> '_b
To permit generalization at these definitions, either remove the explicit generic parameters (if they can be inferred), or use the required number of parameters, as the following example shows:
let throw<'T,'U> (x:'T) (y:'U) = x
Of course, you could also accomplish this with type annotations.
Most obvious example: write a function to calculate the length of a string.
You have to write:
let f (a:string) = a.Length
and you need the annotation. Without the annotation, the compiler can't determine the type of a. Other similar examples exist - particularly when using libraries designed to be used from C#.
Dealing with updated answer:
The same problem applies - string becomes A<string> which has a method get that returns a string
let f (a:A<string>) = a.get().Length