Can somebody help me with article of Tomas Petricek: http://tomasp.net/blog/fsharp-dynamic-lookup.aspx/#dynfslinks?
The problem is that it is severely outdated. I understand that namespaces
open Microsoft.FSharp.Quotations.Typed
open Microsoft.FSharp.Quotations.Raw
are gone. So I removed the openings. But there are still errors. "Typed" is not defined. "RecdGet" is not defined. And I suspect they are not the last. I'm trying to prove to my boss that F# is good to use for database normalization. Dynamic lookup of fields would really helped me to deal with similarly named fields having different prefixes.
There is also post of Tomas on fpish: https://fpish.net/topic/None/57493, which I understand predates the article
Here's a rough equivalent:
open Microsoft.FSharp.Quotations
open Microsoft.FSharp.Quotations.Patterns
type DynamicMember<'t,'u> = Expr<'t -> 'u>
let getValueReader (expr:DynamicMember<'recdT, 'fieldT>) =
// Match the quotation representing the symbol
match expr with
| Lambda(v, PropertyGet (Some (Var v'), pi, [])) when v = v' ->
// It represents reading of the F# record field..
// .. get a function that reads the record field using F# reflection
let rdr = Reflection.FSharpValue.PreComputeRecordFieldReader pi
// we're not adding any additional processing, so we just
// simply add type conversion to the correct types & return it
((box >> rdr >> unbox) : 'recdT -> 'fieldT)
| _ ->
// Quotation doesn't represent symbol - this is an error
failwith "Invalid expression - not reading record field!"
type SampleRec = { Str : string; Num : int }
let readStrField = getValueReader <# fun (r : SampleRec) -> r.Str #>
let readNumField = getValueReader <# fun (r : SampleRec) -> r.Num #>
let rc = { Str = "Hello world!"; Num = 42 }
let s, n = readStrField rc, readNumField rc
printfn "Extracted: %s, %d" s n
Related
Is there any particular reason why piping to int works here but the system convert doesn't? Which method should be used?
printfn "%i" ("1_2" |> int)
printfn "%i" (System.Int32.Parse("1_2"))
I am using .NET core 2.2
It looks like the first method calls
FSharp.Core.dll!Microsoft.FSharp.Core.LanguagePrimitives.ParseInt32(string s)
and the second calls
System.Private.CoreLib.dll!int.Parse(string s)
So if anyone is curious, I looked at the difference. the .NET core dll doesnt strip underscores
System.Private.CoreLib.dll uses
private static unsafe void StringToNumber(ReadOnlySpan<char> str, NumberStyles options, ref NumberBuffer number, NumberFormatInfo info, bool parseDecimal)
{
Debug.Assert(info != null);
fixed (char* stringPointer = &MemoryMarshal.GetReference(str))
{
char* p = stringPointer;
if (!ParseNumber(ref p, p + str.Length, options, ref number, info, parseDecimal)
|| (p - stringPointer < str.Length && !TrailingZeros(str, (int)(p - stringPointer))))
{
throw new FormatException(SR.Format_InvalidString);
}
}
}
And FSharp.Core.dll uses
let ParseInt32 (s:string) =
if System.Object.ReferenceEquals(s,null) then
raise( new System.ArgumentNullException("s") )
let s = removeUnderscores (s.Trim())
let l = s.Length
let mutable p = 0
let sign = getSign32 s &p l
let specifier = get0OXB s &p l
if p >= l then formatError() else
match Char.ToLowerInvariant(specifier) with
| 'x' -> sign * (int32OfUInt32 (Convert.ToUInt32(UInt64.Parse(s.Substring(p), NumberStyles.AllowHexSpecifier,CultureInfo.InvariantCulture))))
| 'b' -> sign * (int32OfUInt32 (Convert.ToUInt32(parseBinaryUInt64 (s.Substring(p)))))
| 'o' -> sign * (int32OfUInt32 (Convert.ToUInt32(parseOctalUInt64 (s.Substring(p)))))
| _ -> Int32.Parse(s, NumberStyles.AllowLeadingSign, CultureInfo.InvariantCulture)
As others said two different implementations of parsing an integer is used and doesn't necessarily produce the same result. However, one might wonder why F# allows 1_2 as a valid int?
I browsed the history of the source code and found that it was implemented in this commit: implement Underscore Literals
It was made to support literals in F# like this:
let x = 1_000_000
let y = 1000000
x = y // true
x and y is equal but 1_000_000 is thanks to the underscores somewhat easier to read as 1 million.
Because how it was implemented it also leaked into runtime as now int "1_000_000" parse successfully.
In the first case ("1_2" |> int) you are using int which is an F# primitive.
In the second case (System.Int32.Parse("1_2")) you are using System.Int32 which is a .NET CLR and not specifically an F# type.
The two use different rules for parsing integers as you discovered in the implementations.
Learning F# these days, I've noticed that in some libraries like this one or that one
there are some similar functions which seem to be common in F# but can't really decipher them, what are they doing, what are they for?
let ap x f =
match f, x with
| Ok f , Ok x -> Ok (f x)
| Error e , _ -> Error e
| _ , Error e -> Error e
let inline (<*>) f x = ap x f
let inline (<!>) f x = Result.map f x
let inline lift2 f a b = f <!> a <*> b
Even aggregating comments with them does not really help in my understanding:
/// Sequential application
/// If the wrapped function is a success and the given result is a success the function is applied on the value.
/// Otherwise the exisiting error messages are propagated.
let ap x f =
match f,x with
| Ok f , Ok x -> Ok (f x)
| Error e , _ -> Error e
| _ , Error e -> Error e
/// Sequential application
/// If the wrapped function is a success and the given result is a success the function is applied on the value.
/// Otherwise the exisiting error messages are propagated.
let inline (<*>) f x = ap x f
/// Infix map, lifts a function into a Result and applies it on the given result.
let inline (<!>) f x = Result.map f x
/// Promote a function to a monad/applicative, scanning the monadic/applicative arguments from left to right.
let inline lift2 f a b = f <!> a <*> b
I don't even see an example of how they could be used, not sure also why inline has been used.
If there is somebody who could hint about how useful those functions are, I would greatly appreciate.
These are called "applicative functors" (sometimes just "applicatives"). Their purpose is to combine data from multiple Something<'T> using a function. Basically, "lifting" a function of type 'Arg1 -> 'Arg2 -> ... -> 'Result into a function of type Something<'Arg1> -> Something<'Arg2> -> ... -> Something<'Result>.
For example, given the standard Result type:
type Result<'T, 'Err> = Ok of 'T | Error of 'Err
you may have several Result values that you want to combine together. For example, say you have a form with inputs firstName, lastName and age. You also have a result type Person:
type Person = { firstName: string; lastName: string; age: int }
// string -> string -> int -> Person
let makePerson firstName lastName age =
{ firstName = firstName; lastName = lastName; age = age }
The values coming from your actual form may have type Result<string, InputError> or Result<int, InputError>, which can be Error if eg. the user hasn't entered a value.
type InputError =
| FieldMissing of fieldName: string
// Other error cases...
You want to combine them into a Result<Person, InputError>, which is Ok if all inputs are Ok, or Error if any input is Error. Using the applicative, you can do it like this:
// Result<string, InputError> -> Result<string, InputError> -> Result<int, InputError> -> Result<Person, InputError>
let makePersonResult firstName lastName age =
makePerson <!> firstName <*> lastName <*> age
// Example uses:
makePersonResult (Ok "John") (Ok "Doe") (Ok 42)
// --> Ok { firstName = "John"; lastName = "Doe"; age = 42 }
makePersonResult (Error (FieldMissing "firstName")) (Ok "Doe") (Ok 42)
// --> Error (FieldMissing "firstName")
A similar concept can be applied to many other types than Result, which is why it was given a name. For example, an applicative on Async<'T> could run all the argument Asyncs in parallel, and when they're finished, combine their results into an Async<'Result>. Another example, an applicative on 'T list would be equivalent to the standard library's List.map2 or List.map3 but generalizable to any number of argument lists.
Side note: if you look up "applicative functor", most of the results you'll find will be in Haskell, where the map operator, usually written <!> in F#, is written <$> instead.
Scott Wlaschin's F# for fun and profit (https://fsharpforfunandprofit.com) has a series Map and Bind and Apply, Oh my! (https://fsharpforfunandprofit.com/posts/elevated-world-7) which should be able to shed more light on this. Regarding your particular question:
<!> is the map operator which applies a function f and a parameter x to elements of the data structure you are mapping over, or in other words, lifts the function into the realm of the data structure, in this case the Result type.
<*> is the ap (apply) operator which unpacks a function wrapped inside a elevated value into a lifted function.
lift2 is basically the map operator for a two-parameter function.
Please have a look at the blog, it really helps!
I processed some HTML to extract various information from a website (no proper API exists there), and generated a list of tokens using an F# discriminated union. I have simplified my code to the essence:
type tokens =
| A of string
| B of int
| C of string
let input = [A "1"; B 2; C "2.1"; C "2.2"; B 3; C "3.1"]
// how to transform the input to the following ???
let desiredOutput = [A "1", [[ B 2, [ C "2.1"; C "2.2" ]]; [B 3, [ C "3.1" ]]]]
This roughly corresponds to parsing the grammar: g -> A b* ; b -> B c* ; c-> C
The key thing is my token list is flat, but I want to work with the hierarchy implied by the grammar.
Perhaps there is another representation of my desiredOutput which would be better; what I really want to do is process exactly one A followed by a zero or more sequence of Bs, which happen to contain zero or more Cs.
I've looked at parser combinators articles, e.g. about FParsec, but I couldn't find a good solution that allows me to start from a list of tokens rather than a stream of characters. I'm familiar with imperative techniques for parsing, but I don't know what is idiomatic F#.
Progress made due to Answer
Thanks to the answer from Vandroiy, I was able to write the following to move forward a hobby project I am working on to learn idiomatic F# (and also to scrape quiz websites).
// transform flat data scraped from a Quiz website into a hierarchical data structure
type ScrapedQuiz =
| Title of string
| Description of string
| Blurb of string * picture: string
| QuizId of string
| Question of num:string * text:string * picture : string
| Answer of text:string
| Error of exn
let input =
[Title "Example Quiz Scraped from a website";
Description "What the Quiz is about";
Blurb ("more details","and a URL for a picture");
Question ("#1", "How good is F#", "URL to picture of new F# logo");
Answer ("we likes it");
Answer ("we very likes it");
Question ("#2", "How useful is Stack Overflow", "URL to picture of Stack Overflow logo");
Answer ("very good today");
Answer ("lobsters");
]
type Quiz =
{ Title : string
Description : string
Blurb : string * PictureURL
Questions : Quest list }
and Quest =
{ Number : string
Text : string
Pic : PictureURL
Answers : string list}
and PictureURL = string
let errorMessage = "unexpected input format"
let parseList reader input =
let rec run acc inp =
match reader inp with
| Some(o, inp') -> run (o :: acc) inp'
| None -> List.rev acc, inp
run [] input
let readAnswer = function Answer(a) :: t -> Some(a, t) | _ -> None
let readDescription =
function Description(a) :: t -> (a, t) | _ -> failwith errorMessage
let readBlurb = function Blurb(a,b) :: t -> ((a,b),t) | _ -> failwith errorMessage
let readQuests = function
| Question(n,txt,pic) :: t ->
let answers, input' = parseList readAnswer t
Some( { Number=n; Text=txt; Pic=pic; Answers = answers}, input')
| _ -> None
let readQuiz = function
| Title(s) :: t ->
let d, input' = readDescription t
let b, input'' = readBlurb input'
let qs, input''' = parseList readQuests input''
Some( { Title = s; Description = d; Blurb = b; Questions = qs}, input''')
| _ -> None
match readQuiz input with
| Some(a, []) -> a
| _ -> failwith errorMessage
I could not have written this yesterday; neither the target data type, nor the parsing code. I see room for improvement, but I think I have started to meet my goal of not writing C# in F#.
Indeed, it might help to first find a good representation.
Original output format
I presume the suggested output form, in standard printing, would be:
[(A "1", [(B 2, [C "2.1"; C "2.2"]); (B 3, [C "3.1"])])]
(This differs from the one in the question in the amount of list levels.) The code I used to get there is ugly. In part, this is because it abstracts at an awkward position, constraining input and output types very far without giving them a well-defined type. I'm posting it for the sake of completeness, but I recommend to skip over it.
let rec readBranch checkOne readInner acc = function
| h :: t when checkOne h ->
let dat, inp' = readInner t
readBranch checkOne readInner ((h, dat) :: acc) inp'
| l -> List.rev acc, l
let rec readCs acc = function
| C(s) :: t -> readCs (C(s) :: acc) t
| l -> List.rev acc, l
let readBs = readBranch (function B _ -> true | _ -> false) (readCs []) []
let readAs = readBranch (function A _ -> true | _ -> false) readBs []
input |> readAs |> fst
Surely, other people can do this more sensibly, but I doubt it would tackle the main problem: we're just projecting one weird data structure to the next. If it is difficult to read or formulate a parser's output format, there is probably something going wrong.
Strongly typed output
Rather than focus on how we are parsing, I prefer to first pay attention to what we are parsing into. These A B C things don't mean anything to me. Let's say they represent objects:
type Bravo =
{ ID : int
Charlies : string list }
type Alpha =
{ Name : string
Bravos : Bravo list }
There are two places where sequences of objects of the same type are parsed. Let's create a helper that repeatedly uses a specific parser to read a list of objects:
/// Parses objects into a list. reader takes an input and returns either
/// Some(parsed item, new input state), or None if the list is finished.
/// Returns a list of parsed objects and the remaining input.
let parseList reader input =
let rec run acc inp =
match reader inp with
| Some(o, inp') -> run (o :: acc) inp'
| None -> List.rev acc, inp
run [] input
Note that this is quite generic in the type of input. This helper could be used with strings, sequences, or whatever.
Now, we add concrete parsers. The following functions have the signature used in reader in the helper; they either return the parsed object and the remaining input, or None if parsing wasn't possible.
let readC = function C(s) :: t -> Some(s, t) | _ -> None
let readB = function
| B(i) :: t ->
let charlies, input' = parseList readC t
Some( { ID = i; Charlies = charlies }, input' )
| _ -> None
let readA = function
| A(s) :: t ->
let bravos, input' = parseList readB t
Some( { Name = s; Bravos = bravos }, input' )
| _ -> None
The code for reading Alphas and Bravos is practically a duplicate. If that happens in production code, I would recommend again to check whether the data structure is optimal, and only look at improving the algorithm afterwards.
We request to read one A into one Alpha, which was the goal after all:
match readA input with
| Some(a, []) -> a
| _ -> failwith "Unexpected input format"
There may be many better ways to do the parsing, especially when knowing more about the exact problem. The important fact is not how the parser works, but what the output looks like, which will be the focus when actual work is done in the program. The second version's output should be much easier to navigate in both code and debugger:
val it : Alpha =
{ Name = "1";
Bravos = [ { ID = 2; Charlies = ["2.1"; "2.2"] }
{ ID = 3; Charlies = ["3.1"] } ] }
One could take this a step further and replace the tokenized data structure with DOM (Document Object Model). Then, the first step would be to read HTML into DOM using a standard parsing library. In a second step, the concrete parsers would construct objects, using the DOM representation as input, calling one another top-down.
To work with structured hierarchy, you have to create matching structure of types. Something like
type
RootType = Level1 list
and
Level1 =
| A of string
| B of Level2 list
| C of string
and
Level2 =
{ b: int; c: string list }
In C# I could create a string representation of an object graph fairly easily with expression trees.
public static string GetGraph<TModel, T>(TModel model, Expression<Func<TModel, T>> action) where TModel : class
{
var method = action.Body as MethodCallExpression;
var body = method != null ? method.Object != null ? method.Object as MemberExpression : method.Arguments.Any() ? method.Arguments.First() as MemberExpression : null : action.Body as MemberExpression;
if (body != null)
{
string graph = GetObjectGraph(body, typeof(TModel))
return graph;
}
throw new Exception("Could not create object graph");
}
In F# I've been looking at Quotations to attempt to do the same thing, and can't quite figure it out. I've attempted converting the quotation into an Expression using the PowerPack libraries, but have had no luck so far, and the information on the internet seems fairly sparse on this topic.
If the input is:
let result = getGraph myObject <# myObject.MyProperty #>
the output should be "myobject.MyProperty"
You can see what you get from quotation expression in fsi session:
> let v = "abc"
> <# v.Length #>;;
val it : Expr<int>
= PropGet (Some (PropGet (None, System.String v, [])), Int32 Length, [])
> <# "abc".Length #>;;
val it : Expr<int>
= PropGet (Some (Value ("abc")), Int32 Length, [])
You can find description of all active patterns available to parse qoutations into
manual\FSharp.Core\Microsoft.FSharp.Quotations.Patterns.html
under your F# installation directory or at msdn site
There is nice Chris Smith's book "Programming F#" with chapter named "Quotations" :)
So, after all, just try to write simple parser:
open Microsoft.FSharp.Quotations
open Microsoft.FSharp.Quotations.Patterns
open Microsoft.FSharp.Quotations.DerivedPatterns
let rec getGraph (expr: Expr) =
let parse args =
List.fold_left (fun acc v -> acc ^ (if acc.Length > 0 then "," else "") ^ getGraph v) "" args
let descr s = function
| Some v -> "(* instance " ^ s ^ "*) " ^ getGraph v
| _ -> "(* static " ^ s ^ "*)"
match expr with
| Int32 i -> string i
| String s -> sprintf "\"%s\"" s
| Value (o,t) -> sprintf "%A" o
| Call (e, methodInfo, av) ->
sprintf "%s.%s(%s)" (descr "method" e) methodInfo.Name (parse av)
| PropGet(e, methodInfo, av) ->
sprintf "%s.%s(%s)" (descr "property" e) methodInfo.Name (parse av)
| _ -> failwithf "I'm don't understand such expression's form yet: %A" expr
P.S. And of course you will need some code to translate AST to human readable format.
I'm unsure what the state of things was back when you asked this question, but today you can convert an F# Quotation to an Expression using the PowerPack like so:
<# "asdf".Length #>.ToLinqExpression()
Also, I've been developing a library Unquote which is able to decompile many F# Quotations into F# single-line non-light syntax code. It can easily handle simple instance PropertyGet expressions like your required input / output:
> decompile <# "asdf".Length #>;;
val it : string = ""asdf".Length"
See my answer to a similar question for more information or just visit Unquote's home page.
I'm wondering what others have come up with for dealing with Nullable<'T> in F#. I want to use Nullable<'T> on data types so that serialization works properly (i.e., doesn't write out F# option type to XML). But, I don't want my code stuck dealing with the ugliness of dealing with Nullable<'T> directly. Any suggestions?
Is it better to use active patterns to match directly on Nullable, or just a converter to option and use Some/None matching?
Additionally, I'd love to hear ideas on dealing with nullable references in a nice manner too. If I use, say "string option", then I end up with the F# option type wrapping things. If I don't then I can't distinguish between truly optional strings and strings that shouldn't be null.
Any chance .NET 4 will take on an Option<'T> to help out? (If it's part of the BCL, then we might see better support for it...)
As active patterns as options plays nicely with pattern matching, but is seems by using active patterns (i.e. typeof and ??) your code will eat more ticks.
The base question is how you will deal with your nullable references?
In case your code is long chained computations it's nice to use monadic syntax:
type Maybe<'a> = (unit -> 'a option)
let succeed x : Maybe<'a> = fun () -> Some(x)
let fail : Maybe<'a> = fun () -> None
let run (a: Maybe<'a>) = a()
let bind p rest = match run p with None -> fail | Some r -> (rest r)
let delay f = fun () -> run (f ())
type MaybeBuilder() =
member this.Return(x) = succeed x
member this.Let(p,rest) = rest p
member this.Bind(p,rest) = bind p rest
member this.Delay(f) = delay f
let maybe = new MaybeBuilder()
let add (a:'a) (b:'a) =
maybe {
match TryGetNumericAssociation<'a>() with
| Some v -> return (v.Add(a,b))
| _ -> return! fail
}
let add3 (a:'a) (b:'a) (c:'a) =
maybe {
let! ab = add a b
let! abc = add ab c
return abc
}
> let r1 = add 1 2;;
val r1 : (unit -> int option)
> r1();;
val it : int option = Some 3
> let r2 = add "1" "2";;
val r2 : (unit -> string option)
> r2();;
val it : string option = None
> let r3 = add3 "one" "two" "three";;
val r3 : (unit -> string option)
> r3();;
val it : string option = None