1- I'm really confusing on applying F# Quotation & Pattern on Meta Programming, please suggest some way to approach this concept in F#.
2- Can you show me some real application of F# Quotations and Pattern in Meta Programming ?
3- Some guys said that he can even make another language like IronScheme by F#,is that right ?
Thanks.
1- I'm really confusing on applying F# Quotation & Pattern on Meta Programming, please suggest some way to approach this concept in F#.
A quotation mechanism lets you embed code in your code and have the compiler transform that code from the source you provide into a data structure that represents it. For example, the following gives you a data structure representing the F# expression 1+2:
> <# 1+2 #>;;
val it : Quotations.Expr<int> =
Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),
[Value (1), Value (2)])
{CustomAttributes = [NewTuple (Value ("DebugRange"),
NewTuple (Value ("stdin"), Value (3), Value (3), Value (3), Value (6)))];
Raw = ...;
Type = System.Int32;}
You can then hack on this data structure in order to apply transformations to your code, such as translating it from F# to Javascript in order to run it client side on almost any browser.
2- Can you show me some real application of F# Quotations and Pattern in Meta Programming ?
The F# quotation mechanism is extremely limited in functionality compared to the quotation mechanisms of languages like OCaml and Lisp, to the point where I wonder why it was ever added. Moreover, although the .NET Framework and F# compiler provide everything required to compile and execute quoted code at full speed, the evaluation mechanism for quoted code is orders of magnitude slower than real F# code which, again, renders it virtually useless. Consequently, I am not familiar with any real applications of it beyond Websharper.
For example, you can only quote certain kinds of expressions in F# and not other code such as type definitions:
> <# type t = Int of int #>;;
<# type t = Int of int #>;;
---^^^^
C:\Users\Jon\AppData\Local\Temp\stdin(4,4): error FS0010: Unexpected keyword 'type' in quotation literal
Most quotation mechanisms let you quote any valid code at all. For example, OCaml's quotation mechanism can quote the type definition that F# just barfed on:
$ ledit ocaml dynlink.cma camlp4oof.cma
Objective Caml version 3.12.0
Camlp4 Parsing version 3.12.0
# open Camlp4.PreCast;;
# let _loc = Loc.ghost;;
val _loc : Camlp4.PreCast.Loc.t = <abstr>
# <:expr< 1+2 >>;;
- : Camlp4.PreCast.Ast.expr =
Camlp4.PreCast.Ast.ExApp (<abstr>,
Camlp4.PreCast.Ast.ExApp (<abstr>,
Camlp4.PreCast.Ast.ExId (<abstr>, Camlp4.PreCast.Ast.IdLid (<abstr>, "+")),
Camlp4.PreCast.Ast.ExInt (<abstr>, "1")),
Camlp4.PreCast.Ast.ExInt (<abstr>, "2"))
# <:str_item< type t = Int of int >>;;
- : Camlp4.PreCast.Ast.str_item =
Camlp4.PreCast.Ast.StSem (<abstr>,
Camlp4.PreCast.Ast.StTyp (<abstr>,
Camlp4.PreCast.Ast.TyDcl (<abstr>, "t", [],
Camlp4.PreCast.Ast.TySum (<abstr>,
Camlp4.PreCast.Ast.TyOf (<abstr>,
Camlp4.PreCast.Ast.TyId (<abstr>,
Camlp4.PreCast.Ast.IdUid (<abstr>, "Int")),
Camlp4.PreCast.Ast.TyId (<abstr>,
Camlp4.PreCast.Ast.IdLid (<abstr>, "int")))),
[])),
Camlp4.PreCast.Ast.StNil <abstr>)
FWIW, here is an example in Common Lisp:
$ sbcl
This is SBCL 1.0.29.11.debian, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* '(+ 1 2)
(+ 1 2)
Metaprogramming is one application where pattern matching can be extremely useful but pattern matching is a general-purpose language feature. You may appreciate my article from the Benefits of OCaml about a minimal interpreter. In particular, note how easy pattern matching makes it to act upon each of the different kinds of expression:
> let rec eval vars = function
| EApply(func, arg) ->
match eval vars func, eval vars arg with
| VClosure(var, vars, body), arg -> eval ((var, arg) :: vars) body
| _ -> invalid_arg "Attempt to apply a non-function value"
| EAdd(e1, e2) -> VInt (int(eval vars e1) + int(eval vars e2))
| EMul(e1, e2) -> VInt (int(eval vars e1) * int(eval vars e2))
| EEqual(e1, e2) -> VBool (eval vars e1 = eval vars e2)
| EIf(p, t, f) -> eval vars (if bool (eval vars p) then t else f)
| EInt i -> VInt i
| ELetRec(var, arg, body, rest) ->
let rec vars = (var, VClosure(arg, vars, body)) :: vars in
eval vars rest
| EVar s -> List.assoc s vars;;
val eval : (string * value) list -> expr -> value = <fun>
That OCaml article was used as the basis of the F#.NET Journal article "Language-oriented programming: The Term-level Interpreter" (31st December 2007).
3- Some guys said that he can even make another language like IronScheme by F#,is that right ?
Yes, you can write compilers in F#. In fact, F# is derived from a family of languages that were specifically designed for metaprogramming, the so-called MetaLanguages (ML) family.
The article "Run-time code generation using System.Reflection.Emit" (31st August 2008) from the F#.NET Journal described the design and implementation of a simple compiler for a minimal language called Brainf*ck. You can extend this to implement more sophisticated languages like Scheme. Indeed, the F# compiler is mostly written in F# itself.
On a related note, I just completed a project writing high-performance serialization code that used reflection to consume F# types in a project and then spit out F# code to serialize and deserialize values of those types
F# quotations allow you to mark some piece of F# code and get the representation of the source code. This is ued in WebSharper (see for example this tutorial) to translate F# code to JavaScript. Another example is F# support for LINQ where code marked as <# ... #> is translated to SQL:
let res = <# for p in db.Products
if p.IsVisible then yield p.Name #> |> query
Pattern matching is simply a very powerful language construct, but it is nothing more mysterious than for example if. The idea is that you can match value against patterns and program will choose the first matching branch. This is powerful because patterns can be nested and so you can use it to process various complex data structures or implement symbolc processing:
match expr with
| Multiply(Constant 0, _) | Multiply(_, Constant 0) -> 0
| Multiply(expr1, expr2) -> (eval expr1) * (eval expr2)
// (other patterns)
For example, here we're using pattern matching to evaluate some representation of numerical expression. The first pattern is an optimization that deals with cases where one argument of multiplication is 0.
Writing languages You can use F# (just like any other general purpose language) to write compilers and tools for other languages. In F#, this is easy because it comes with tools for generating lexers and parsers. See for example this introduction.
Related
Often I hear that using a symbol table optimizes look ups of symbols in a programming language. Currently, my language is implemented only as an interpreter, not as a compiler. I do not yet want to allocate the time to build a compiler, so I'm attempting to optimize the interpreter. The language is based on Scheme semantics and syntax for the most part, and is statically-scoped. I use the AST for executing code at run-time (in my interpreter, implemented as discriminated unions just like the AST in Write Yourself a Scheme in 48 Hours.
Unfortunately, symbol look-up in my interpreter is slow due to the use of an F# Map to contain and look up symbols by name. (Well, in truth, it uses a Trie, but the performance is similarly problematic). I would like to instead use a symbol tree to achieve faster symbol lookup. However, I don't know if or how one can implement symbols tables in an interpreter. I hear about them only in the context of a compiler.
Is this possible? If the implementation strategy or performance differs from a symbol table in a compiler, could you describe the differences? Finally, is there an existing reference implementation of a symbol tree in an interpreter I might look at?
Thank you!
A symbol table associates some information with every symbol. In an interpreter, you would perhaps associate values with symbols. Map is one implementation particularly suitable for functional interpreters.
If you want to optimize your interpreter, get rid of the need for a symbol table at runtime. One way to to go is De Bruijn idexing.
There is also nice literature on mechanically deriving optimized interpreters, VMs and compilers from a functional interpreter, for example:
http://www.brics.dk/RS/03/14/BRICS-RS-03-14.pdf
For a simple example, consider lambda calculus with constants encoded with De Bruijn indices. Notice that the evaluator gets by without a symbol table, because it can use integers for lookup.
type exp =
| App of exp * exp
| Const of int
| Fn of exp
| Var of int
type value =
| Closure of exp * env
| Number of int
and env = value []
let lookup env i = Array.get env i
let extend value env = Array.append [| value |] env
let empty () : env = Array.empty
let eval exp =
let rec eval env exp =
match exp with
| App (f, x) ->
match eval env f with
| Closure (bodyF, envF) ->
let vx = eval env x
eval (extend vx envF) bodyF
| _ -> failwith "?"
| Const x -> Number x
| Fn e -> Closure (e, env)
| Var x -> lookup env x
eval (empty ()) exp
I'm at the moment doing some very basic pattern matching with quotations.
My code:
let rec test e =
match e with
| Patterns.Lambda(v,e) -> test e
| Patterns.Call(_, mi, [P.Value(value, _); P.Value(value2, _)]) ->
printfn "Value1: %A | Value2 : %A" value value2
| Patterns.Call(_, mi, [P.Value(value, _); P.PropertyGet(_, pi, exprs)]) ->
printfn "Value1: %A | Value2 : %A" value (pi.GetValue(pi, null))
| _ -> failwith "Expression not supported"
let quot1 = <# "Name" = "MyName" #>
(* Call (None, Boolean op_Equality[String](System.String, System.String),
[Value ("Name"), Value ("lol")]) *)
let quot2 = <# "Name" = getNameById 5 #>
(* Call (None, Boolean op_Equality[String](System.String, System.String),
[Value ("Name"),
Call (None, System.String getNameById[Int32](Int32), [Value (5)])]) *)
test quot1 // Works!
test quot2 // Fails.. Dosent match any of the patterns.
Is it possible to somehow evaluate the result of the getNameById function first, so that it will match one of the patterns, or am I doomed to assign a let binding with the result of the function outside the quotation?
I've tried playing with the ExprShape patterns, but without luck..
You can use PowerPack's Eval to evaluate only the arguments to the Call expression:
match e with
| Call(_,mi,[arg1;arg2]) ->
let arg1Value, arg2Value = arg1.Eval(), arg2.Eval()
...
And similarly for Lambda expressions, etc. Noticed this frees you from enumerating permutations of Value, Property, and other argument expressions.
Update
Since you want to avoid using Eval (for good reason if you are implementing a performance conscious application), you'll need to implement your own eval function using reflection (which is still not lightening fast, but should be faster than PowerPack's Eval which involves an intermediate translation of F# Quotations to Linq Expressions). You can get started by supporting a basic set of expressions, and expand from there as needed. Recursion is the key, the following can help you get started:
open Microsoft.FSharp.Quotations
open System.Reflection
let rec eval expr =
match expr with
| Patterns.Value(value,_) -> value //value
| Patterns.PropertyGet(Some(instance), pi, args) -> //instance property get
pi.GetValue(eval instance, evalAll args) //notice recursive eval of instance expression and arg expressions
| Patterns.PropertyGet(None, pi, args) -> //static property get
pi.GetValue(null, evalAll args)
| Patterns.Call(Some(instance), mi, args) -> //instance call
mi.Invoke(eval instance, evalAll args)
| Patterns.Call(None, mi, args) -> //static call
mi.Invoke(null, evalAll args)
| _ -> failwith "invalid expression"
and evalAll exprs =
exprs |> Seq.map eval |> Seq.toArray
And then wrapping this in an Active Pattern will improve syntax:
let (|Eval|) expr =
eval expr
match e with
| Patterns.Call(_, mi, [Eval(arg1Value); Eval(arg2Value)]) -> ...
Update 2
OK, this thread got me motivated to try and implement a robust reflection based solution, and I've done so with good results which are now part of Unquote as of version 2.0.0.
It turned out not to be as difficult as I thought it would be, currently I am supporting all quotation expressions except for AddressGet, AddressSet, and NewDelegate. This is already better than PowerPack's eval, which doesn't support PropertySet, VarSet, FieldSet, WhileLoop, ForIntegerRangeLoop, and Quote for example.
Some noteworthy implementation details are with VarSet and VarGet, where I need to pass around an environment name / variable lookup list to each recursive call. It is really an excellent example of the beauty of functional programming with immutable data-structures.
Also noteworthy is special care taken with issues surrounding exceptions: striping the TargetInvokationExceptions thrown by reflection when it catches exceptions coming from methods it is invoking (this is very important for handling TryWith evaluation properly, and also makes for better user handling of exceptions which fly out of the quotation evaluation.
Perhaps the most "difficult" implementation detail, or really the most grueling, was the need to implement all of the core operators (well, as most I could discover: the numeric and conversion operators, checked versions as well) since most of them are not given dynamic implementations in the F# library (they are implemented using static type tests with no fallback dynamic implementations), but also means a serious performance increase when using these functions.
Some informal benchmarking I observe performance increases of up to 50 times over PowerPack's (not pre-compiled) eval.
I am also confident that my reflection-based solution will be less bug prone then PowerPack's, simply because it is less complicated than the PowerPack's approach (not to mention I've backed it up with about 150 unit tests, duly fortified by Unquotes additional 200+ unit tests which now is driven by this eval implementation).
If you want to peek at the source code, the main modules are Evaluation.fs and DynamicOperators.fs (I've locked the links into revision 257). Feel free to grab and use the source code for your own purposes, it licensed under Apache License 2.0! Or you could wait a week or so, when I release Unquote 2.0.0 which will include evaluation operators and extensions publicly.
You can write an interpreter that will evaluate the quotation and call the getNameById function using Reflection. However, that would be quite a lot of work. The ExprShape isn't going to help you much - it is useful for simple traversing of quotations, but to write an interpreter, you'll need to cover all patterns.
I think the easiest option is to evaluate quotations using the PowerPack support:
#r "FSharp.PowerPack.Linq.dll"
open Microsoft.FSharp.Linq.QuotationEvaluation
let getNameById n =
if n = 5 then "Name" else "Foo"
let quot1 = <# "Name" = "MyName" #>
let quot2 = <# "Name" = getNameById 5 #>
quot1.Eval()
quot2.Eval()
This has some limitations, but it is really the easiest option. However, I'm not really sure what are you trying to achieve. If you could clarify that, then you may get a better answer.
I only know F#. I haven't learned the other functional programming languages. All the examples that I have seen for monads only describe the bind and unit methods. F# has lots of keywords (e.g. let!, do!, etc.) that allow you to do different things within the same computational expression. This seemingly gives you more power than your basic bind and unit methods. Is this unique to F# or is it common across functional programming languages?
Yes, I think that the F# syntax for computation expressions is unique in that it provides direct syntactic support for different types of computations. It can be used for working with monoids, usual monads and also MonadPlus computations from Haskell.
I wrote about these in the introduction of my Master thesis. I believe it is quite readable part, so you can go to page 27 to read it. Anyway, I'll copy the examples here:
Monoid is used just for concatenating values using some "+" operation (Combine). You can use it for example for building strings (this is inefficient, but it demonstrates the idea):
type StringMonoid() =
member x.Combine(s1, s2) = String.Concat(s1, s2)
member x.Zero() = ""
member x.Yield(s) = s
let str = new StringMonoid()
let hello = str { yield "Hello "
yield "world!" };;
Monads are the familiar example that uses bind and return operations of comptuation expressions. For example maybe monad represents computations that can fail at any point:
type MaybeMonad() =
member x.Bind(m, f) =
match m with Some(v) -> f v | None -> None
member x.Return(v) = Some(v)
let maybe = new MaybeMonad()
let rec productNameByID() = maybe {
let! id = tryReadNumber()
let! prod = db.TryFindProduct(id)
return prod.Name }
Additive monads (aka MonadPlus in Haskell) is a combination of the two. It is a bit like monadic computation that can produce multiple values. A common example is list (or sequence), which can implement both bind and combine:
type ListMonadPlus() =
member x.Zero() = []
member x.Yield(v) = [v]
member x.Combine(a, b) = a # b
member x.Bind(l, f) = l |> List.map f |> List.concat
let list = new ListMonadPlus()
let cities = list {
yield "York"
yield "Orleans" }
let moreCities = list {
let! n = cities
yield n
yield "New " + n }
// Creates: [ "York"; "New York"; "Orleans"; "New Orleans" ]
There are some additional keywords that do not directly correspond to any theoretical idea. The use keyword deals with resources and for and while can be used to implement looping. The sequence/list comprehension actually use for instead of let!, because that makes much more sense from the syntactic point of view (and for usually takes some sequence - although it may be e.g. asynchronous).
Monads are defined in terms of bind and unit operations (only). There are other structures which are defined by other operations (e.g. in Haskell, the MonadPlus typeclass has zero and plus operations - these correspond to Zero and Combine in F# computation expressions). As far as I know, F#'s computation builders are unique in terms of providing nice syntax for the wide range of operations that they support, but most of the operations are unrelated to monads.
F# binding forms ending in ! denote computation expressions, including let! use! do! yield! return!.
let! pat = expr in comp-expr -- binding computation
do! expr in comp-expr -- sequential computation
use! pat = expr in comp-expr -- auto cleanup computation
yield! expr -- yield computation
return! expr -- return computation
Computation expressions are used "for sequences and other non-standard interpretations of the F# expression syntax". These syntax forms offer ways to overload that syntax, for example, to encode monadic computations, or monoidal computations, and appear to be similar to e.g. the do-notation of Haskell, and corresponding (non-magic) bindings forms in that language.
So I would say that they support some overloading of syntax to support other interpretations of the expression syntax of the language, and this they have in common with many languages, including Haskell and OCaml. It is certainly a powerful and useful language feature.
References: The F# 2.0 Language Specification.
(Recall from memory, I may be off.)
While I think unit and bind are the typical basis for monads, I think maybe map and join for a different basis that I've seen in academic papers. This is kinda like how LINQ works in C# and VB, where the various from syntax desugars into Select or SelectMany which are similar to map and join. LINQ also has some 'extra' keywords, a little like F# though more ad-hoc (and mostly suited to querying enumerations/databases).
I don't know offhand of other functional languages like F# that effectively "lift" most of the control flow and other syntax into monads (well, "computation expressions", which may or may not be monads).
I have a function of the form
'a -> ('a * int) list -> int
let rec getValue identifier bindings =
match bindings with
| (identifier, value)::tail -> value
| (_, _)::tail -> getValue identifier tail
| [] -> -1
I can tell that identifier is not being bound the way I would like it to and is acting as a new variable within the match expression. How to I get identifier to be what is passed into the function?
Ok! I fixed it with a pattern guard, i.e. | (i, value)::tail when i = indentifier -> value
but I find this ugly compared to the way I originally wanted to do it (I'm only using these languages because they are pretty...). Any thoughts?
You can use F# active patterns to create a pattern that will do exactly what you need. F# supports parameterized active patterns that take the value that you're matching, but also take an additional parameter.
Here is a pretty stupid example that fails when the value is zero and otherwise succeeds and returns the addition of the value and the specified parameter:
let (|Test|_|) arg value =
if value = 0 then None else Some(value + arg)
You can specify the parameter in pattern matching like this:
match 1 with
| Test 100 res -> res // 'res' will be 101
Now, we can easily define an active pattern that will compare the matched value with the input argument of the active pattern. The active pattern returns unit option, which means that it doesn't bind any new value (in the example above, it returned some value that we assigned to a symbol res):
let (|Equals|_|) arg x =
if (arg = x) then Some() else None
let foo x y =
match x with
| Equals y -> "equal"
| _ -> "not equal"
You can use this as a nested pattern, so you should be able to rewrite your example using the Equals active pattern.
One of the beauties of functional languages is higher order functions. Using those functions we take the recursion out and just focus on what you really want to do. Which is to get the value of the first tuple that matches your identifier otherwise return -1:
let getValue identifier list =
match List.tryFind (fun (x,y) -> x = identifier) list with
| None -> -1
| Some(x,y) -> y
//val getValue : 'a -> (('a * int) list -> int) when 'a : equality
This paper by Graham Hutton is a great introduction to what you can do with higher order functions.
This is not directly an answer to the question: how to pattern-match the value of a variable. But it's not completely unrelated either.
If you want to see how powerful pattern-matching could be in a ML-like language similar to F# or OCaml, take a look at Moca.
You can also take a look at the code generated by Moca :) (not that there's anything wrong with the compiler doing a lot of things for you in your back. In some cases, it's desirable, even, but many programmers like to feel they know what the operations they are writing will cost).
What you're trying to do is called an equality pattern, and it's not provided by Objective Caml. Objective Caml's patterns are static and purely structural. That is, whether a value matches the pattern depends solely on the value's structure, and in a way that is determined at compile time. For example, (_, _)::tail is a pattern that matches any non-empty list whose head is a pair. (identifier, value)::tail matches exactly the same values; the only difference is that the latter binds two more names identifier and value.
Although some languages have equality patterns, there are non-trivial practical considerations that make them troublesome. Which equality? Physical equality (== in Ocaml), structural equality (= in Ocaml), or some type-dependent custom equality? Furthermore, in Ocaml, there is a clear syntactic indication of which names are binders and which names are reference to previously bound values: any lowercase identifier in a pattern is a binder. These two reasons explain why Ocaml does not have equality patterns baked in. The idiomatic way to express an equality pattern in Ocaml is in a guard. That way, it's immediately clear that the matching is not structural, that identifier is not bound by this pattern matching, and which equality is in use. As for ugly, that's in the eye of the beholder — as a habitual Ocaml programmer, I find equality patterns ugly (for the reasons above).
match bindings with
| (id, value)::tail when id = identifier -> value
| (_, _)::tail -> getValue identifier tail
| [] -> -1
In F#, you have another possibility: active patterns, which let you pre-define guards that concern a single site in a pattern.
This is a common complaint, but I don't think that there's a good workaround in general; a pattern guard is usually the best compromise. In certain specific cases there are alternatives, though, such as marking literals with the [<Literal>] attribute in F# so that they can be matched against.
When quoting
<# 1 + 1 #>
I want "1 + 1"
instead of
"Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),
[Value (1), Value (1)])"
You'll have to write it yourself. See the F# quotations visualizer code as a guide for transforming the quotations abstract syntax tree.
I have implemented a quotation decompiler as part of a larger open source project Unquote. It can decompile many simple F# quoted expressions as single-line non-light syntax strings (see the project's home page for a list of decompiler features). For example,
> decompile <# (11 + 3) / 2 = String.length ("hello world".Substring(4, 5)) #>;;
val it : string =
"(11 + 3) / 2 = String.length ("hello world".Substring(4, 5))"
#Kurt Schelfthout is correct about the many challenges faced when decompiling F# Quotations into human readable form. But from my work so far, I believe that it is possible to write a quotation decompiler which can generate correct F# code. Take match expressions and computation expressions for example, the Unquote decompiler can produce correct F# code in the following simple cases:
> decompile <# match true with | true -> "hi" | _ -> "bye" #>;;
val it : string =
"let matchValue = true in if matchValue then "hi" else "bye""
> decompile <# seq {yield 1; yield 2} #>;;
val it : string =
"seq (Seq.delay (fun unitVar -> Seq.append (Seq.singleton 1) (Seq.delay (fun unitVar -> Seq.singleton 2))))"
Infix and prefix operators are not too hard (as you can see in the first example), but source structure such as new lines and indentation is an interesting topic (though not terribly difficult, I think). However, single-line non-light syntax is sufficient for Unquote's requirements.
There is none, and it's not quite that easy, except in very simple cases. One of the main problems, for example, is the match construct. It is syntactic sugar for a whole bunch of if and switch statements (try printing a quotation with a match in, you'll see). Another one of those biggies are computation expressions, but I guess you could skip those at first.
Then there is a the rabbit hole of ambiguities you'll have to resolve, with conventions like the pipe operator starts a new line, let starts a new line, indentation, infix, prefix, special cases like the (::) operator and so forth.
All in all, doable, but not trivial. Sort of like decompiling.