When quoting
<# 1 + 1 #>
I want "1 + 1"
instead of
"Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),
[Value (1), Value (1)])"
You'll have to write it yourself. See the F# quotations visualizer code as a guide for transforming the quotations abstract syntax tree.
I have implemented a quotation decompiler as part of a larger open source project Unquote. It can decompile many simple F# quoted expressions as single-line non-light syntax strings (see the project's home page for a list of decompiler features). For example,
> decompile <# (11 + 3) / 2 = String.length ("hello world".Substring(4, 5)) #>;;
val it : string =
"(11 + 3) / 2 = String.length ("hello world".Substring(4, 5))"
#Kurt Schelfthout is correct about the many challenges faced when decompiling F# Quotations into human readable form. But from my work so far, I believe that it is possible to write a quotation decompiler which can generate correct F# code. Take match expressions and computation expressions for example, the Unquote decompiler can produce correct F# code in the following simple cases:
> decompile <# match true with | true -> "hi" | _ -> "bye" #>;;
val it : string =
"let matchValue = true in if matchValue then "hi" else "bye""
> decompile <# seq {yield 1; yield 2} #>;;
val it : string =
"seq (Seq.delay (fun unitVar -> Seq.append (Seq.singleton 1) (Seq.delay (fun unitVar -> Seq.singleton 2))))"
Infix and prefix operators are not too hard (as you can see in the first example), but source structure such as new lines and indentation is an interesting topic (though not terribly difficult, I think). However, single-line non-light syntax is sufficient for Unquote's requirements.
There is none, and it's not quite that easy, except in very simple cases. One of the main problems, for example, is the match construct. It is syntactic sugar for a whole bunch of if and switch statements (try printing a quotation with a match in, you'll see). Another one of those biggies are computation expressions, but I guess you could skip those at first.
Then there is a the rabbit hole of ambiguities you'll have to resolve, with conventions like the pipe operator starts a new line, let starts a new line, indentation, infix, prefix, special cases like the (::) operator and so forth.
All in all, doable, but not trivial. Sort of like decompiling.
Related
It turns out that it is really hard to google the word comment.
Is is possible to represent "comment" expressions in quotations abstract syntax tree?
<# //this gets ignored by the compiler and don't inject the quotation
#>
If not, can you suggest a workaround to represent the comments?
As Ganesh points out, the Expr type has no way of representing comments - the F# quotations really represent just the AST of the expression, rather than full information about the source code (although you can get a file name & a location of a quoted expression).
To somehow embed comments in quotations, you'd need to come up with a way of embedding comments as valid F# code that means something - so you could e.g. define a dummy function comment and do something like this:
let comment (s:string) = ()
let sample =
<# comment "this is not ignored"
1 + ( comment "this is also not ignored"
4 ) #>
Then you could write an active pattern that looks for an expression of the form comment "..."; <expr> and extract the string and the following <expr>:
open Microsoft.FSharp.Quotations
let (|Comment|_|) = function
| Patterns.Sequential(DerivedPatterns.SpecificCall <## comment ##> (None, [], [Patterns.Value(comment, _)]), body) ->
Some(unbox<string> comment, body)
| _ -> None
Using the pattern, we can now write an (incomplete) pattern matching that succeeds when the top-level expression is some commented body expression:
match sample with
| Comment(comment, body) ->
printfn "// %s\n%A" comment body
This is not a very nice way of doing it, but I guess it is as good as it can get if you want to embed some annotations in a hand-written quotation code.
The Expr type that quotations return doesn't contain any way to represent a comment, so this is very unlikely to be possible.
I'm at the moment doing some very basic pattern matching with quotations.
My code:
let rec test e =
match e with
| Patterns.Lambda(v,e) -> test e
| Patterns.Call(_, mi, [P.Value(value, _); P.Value(value2, _)]) ->
printfn "Value1: %A | Value2 : %A" value value2
| Patterns.Call(_, mi, [P.Value(value, _); P.PropertyGet(_, pi, exprs)]) ->
printfn "Value1: %A | Value2 : %A" value (pi.GetValue(pi, null))
| _ -> failwith "Expression not supported"
let quot1 = <# "Name" = "MyName" #>
(* Call (None, Boolean op_Equality[String](System.String, System.String),
[Value ("Name"), Value ("lol")]) *)
let quot2 = <# "Name" = getNameById 5 #>
(* Call (None, Boolean op_Equality[String](System.String, System.String),
[Value ("Name"),
Call (None, System.String getNameById[Int32](Int32), [Value (5)])]) *)
test quot1 // Works!
test quot2 // Fails.. Dosent match any of the patterns.
Is it possible to somehow evaluate the result of the getNameById function first, so that it will match one of the patterns, or am I doomed to assign a let binding with the result of the function outside the quotation?
I've tried playing with the ExprShape patterns, but without luck..
You can use PowerPack's Eval to evaluate only the arguments to the Call expression:
match e with
| Call(_,mi,[arg1;arg2]) ->
let arg1Value, arg2Value = arg1.Eval(), arg2.Eval()
...
And similarly for Lambda expressions, etc. Noticed this frees you from enumerating permutations of Value, Property, and other argument expressions.
Update
Since you want to avoid using Eval (for good reason if you are implementing a performance conscious application), you'll need to implement your own eval function using reflection (which is still not lightening fast, but should be faster than PowerPack's Eval which involves an intermediate translation of F# Quotations to Linq Expressions). You can get started by supporting a basic set of expressions, and expand from there as needed. Recursion is the key, the following can help you get started:
open Microsoft.FSharp.Quotations
open System.Reflection
let rec eval expr =
match expr with
| Patterns.Value(value,_) -> value //value
| Patterns.PropertyGet(Some(instance), pi, args) -> //instance property get
pi.GetValue(eval instance, evalAll args) //notice recursive eval of instance expression and arg expressions
| Patterns.PropertyGet(None, pi, args) -> //static property get
pi.GetValue(null, evalAll args)
| Patterns.Call(Some(instance), mi, args) -> //instance call
mi.Invoke(eval instance, evalAll args)
| Patterns.Call(None, mi, args) -> //static call
mi.Invoke(null, evalAll args)
| _ -> failwith "invalid expression"
and evalAll exprs =
exprs |> Seq.map eval |> Seq.toArray
And then wrapping this in an Active Pattern will improve syntax:
let (|Eval|) expr =
eval expr
match e with
| Patterns.Call(_, mi, [Eval(arg1Value); Eval(arg2Value)]) -> ...
Update 2
OK, this thread got me motivated to try and implement a robust reflection based solution, and I've done so with good results which are now part of Unquote as of version 2.0.0.
It turned out not to be as difficult as I thought it would be, currently I am supporting all quotation expressions except for AddressGet, AddressSet, and NewDelegate. This is already better than PowerPack's eval, which doesn't support PropertySet, VarSet, FieldSet, WhileLoop, ForIntegerRangeLoop, and Quote for example.
Some noteworthy implementation details are with VarSet and VarGet, where I need to pass around an environment name / variable lookup list to each recursive call. It is really an excellent example of the beauty of functional programming with immutable data-structures.
Also noteworthy is special care taken with issues surrounding exceptions: striping the TargetInvokationExceptions thrown by reflection when it catches exceptions coming from methods it is invoking (this is very important for handling TryWith evaluation properly, and also makes for better user handling of exceptions which fly out of the quotation evaluation.
Perhaps the most "difficult" implementation detail, or really the most grueling, was the need to implement all of the core operators (well, as most I could discover: the numeric and conversion operators, checked versions as well) since most of them are not given dynamic implementations in the F# library (they are implemented using static type tests with no fallback dynamic implementations), but also means a serious performance increase when using these functions.
Some informal benchmarking I observe performance increases of up to 50 times over PowerPack's (not pre-compiled) eval.
I am also confident that my reflection-based solution will be less bug prone then PowerPack's, simply because it is less complicated than the PowerPack's approach (not to mention I've backed it up with about 150 unit tests, duly fortified by Unquotes additional 200+ unit tests which now is driven by this eval implementation).
If you want to peek at the source code, the main modules are Evaluation.fs and DynamicOperators.fs (I've locked the links into revision 257). Feel free to grab and use the source code for your own purposes, it licensed under Apache License 2.0! Or you could wait a week or so, when I release Unquote 2.0.0 which will include evaluation operators and extensions publicly.
You can write an interpreter that will evaluate the quotation and call the getNameById function using Reflection. However, that would be quite a lot of work. The ExprShape isn't going to help you much - it is useful for simple traversing of quotations, but to write an interpreter, you'll need to cover all patterns.
I think the easiest option is to evaluate quotations using the PowerPack support:
#r "FSharp.PowerPack.Linq.dll"
open Microsoft.FSharp.Linq.QuotationEvaluation
let getNameById n =
if n = 5 then "Name" else "Foo"
let quot1 = <# "Name" = "MyName" #>
let quot2 = <# "Name" = getNameById 5 #>
quot1.Eval()
quot2.Eval()
This has some limitations, but it is really the easiest option. However, I'm not really sure what are you trying to achieve. If you could clarify that, then you may get a better answer.
1- I'm really confusing on applying F# Quotation & Pattern on Meta Programming, please suggest some way to approach this concept in F#.
2- Can you show me some real application of F# Quotations and Pattern in Meta Programming ?
3- Some guys said that he can even make another language like IronScheme by F#,is that right ?
Thanks.
1- I'm really confusing on applying F# Quotation & Pattern on Meta Programming, please suggest some way to approach this concept in F#.
A quotation mechanism lets you embed code in your code and have the compiler transform that code from the source you provide into a data structure that represents it. For example, the following gives you a data structure representing the F# expression 1+2:
> <# 1+2 #>;;
val it : Quotations.Expr<int> =
Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),
[Value (1), Value (2)])
{CustomAttributes = [NewTuple (Value ("DebugRange"),
NewTuple (Value ("stdin"), Value (3), Value (3), Value (3), Value (6)))];
Raw = ...;
Type = System.Int32;}
You can then hack on this data structure in order to apply transformations to your code, such as translating it from F# to Javascript in order to run it client side on almost any browser.
2- Can you show me some real application of F# Quotations and Pattern in Meta Programming ?
The F# quotation mechanism is extremely limited in functionality compared to the quotation mechanisms of languages like OCaml and Lisp, to the point where I wonder why it was ever added. Moreover, although the .NET Framework and F# compiler provide everything required to compile and execute quoted code at full speed, the evaluation mechanism for quoted code is orders of magnitude slower than real F# code which, again, renders it virtually useless. Consequently, I am not familiar with any real applications of it beyond Websharper.
For example, you can only quote certain kinds of expressions in F# and not other code such as type definitions:
> <# type t = Int of int #>;;
<# type t = Int of int #>;;
---^^^^
C:\Users\Jon\AppData\Local\Temp\stdin(4,4): error FS0010: Unexpected keyword 'type' in quotation literal
Most quotation mechanisms let you quote any valid code at all. For example, OCaml's quotation mechanism can quote the type definition that F# just barfed on:
$ ledit ocaml dynlink.cma camlp4oof.cma
Objective Caml version 3.12.0
Camlp4 Parsing version 3.12.0
# open Camlp4.PreCast;;
# let _loc = Loc.ghost;;
val _loc : Camlp4.PreCast.Loc.t = <abstr>
# <:expr< 1+2 >>;;
- : Camlp4.PreCast.Ast.expr =
Camlp4.PreCast.Ast.ExApp (<abstr>,
Camlp4.PreCast.Ast.ExApp (<abstr>,
Camlp4.PreCast.Ast.ExId (<abstr>, Camlp4.PreCast.Ast.IdLid (<abstr>, "+")),
Camlp4.PreCast.Ast.ExInt (<abstr>, "1")),
Camlp4.PreCast.Ast.ExInt (<abstr>, "2"))
# <:str_item< type t = Int of int >>;;
- : Camlp4.PreCast.Ast.str_item =
Camlp4.PreCast.Ast.StSem (<abstr>,
Camlp4.PreCast.Ast.StTyp (<abstr>,
Camlp4.PreCast.Ast.TyDcl (<abstr>, "t", [],
Camlp4.PreCast.Ast.TySum (<abstr>,
Camlp4.PreCast.Ast.TyOf (<abstr>,
Camlp4.PreCast.Ast.TyId (<abstr>,
Camlp4.PreCast.Ast.IdUid (<abstr>, "Int")),
Camlp4.PreCast.Ast.TyId (<abstr>,
Camlp4.PreCast.Ast.IdLid (<abstr>, "int")))),
[])),
Camlp4.PreCast.Ast.StNil <abstr>)
FWIW, here is an example in Common Lisp:
$ sbcl
This is SBCL 1.0.29.11.debian, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* '(+ 1 2)
(+ 1 2)
Metaprogramming is one application where pattern matching can be extremely useful but pattern matching is a general-purpose language feature. You may appreciate my article from the Benefits of OCaml about a minimal interpreter. In particular, note how easy pattern matching makes it to act upon each of the different kinds of expression:
> let rec eval vars = function
| EApply(func, arg) ->
match eval vars func, eval vars arg with
| VClosure(var, vars, body), arg -> eval ((var, arg) :: vars) body
| _ -> invalid_arg "Attempt to apply a non-function value"
| EAdd(e1, e2) -> VInt (int(eval vars e1) + int(eval vars e2))
| EMul(e1, e2) -> VInt (int(eval vars e1) * int(eval vars e2))
| EEqual(e1, e2) -> VBool (eval vars e1 = eval vars e2)
| EIf(p, t, f) -> eval vars (if bool (eval vars p) then t else f)
| EInt i -> VInt i
| ELetRec(var, arg, body, rest) ->
let rec vars = (var, VClosure(arg, vars, body)) :: vars in
eval vars rest
| EVar s -> List.assoc s vars;;
val eval : (string * value) list -> expr -> value = <fun>
That OCaml article was used as the basis of the F#.NET Journal article "Language-oriented programming: The Term-level Interpreter" (31st December 2007).
3- Some guys said that he can even make another language like IronScheme by F#,is that right ?
Yes, you can write compilers in F#. In fact, F# is derived from a family of languages that were specifically designed for metaprogramming, the so-called MetaLanguages (ML) family.
The article "Run-time code generation using System.Reflection.Emit" (31st August 2008) from the F#.NET Journal described the design and implementation of a simple compiler for a minimal language called Brainf*ck. You can extend this to implement more sophisticated languages like Scheme. Indeed, the F# compiler is mostly written in F# itself.
On a related note, I just completed a project writing high-performance serialization code that used reflection to consume F# types in a project and then spit out F# code to serialize and deserialize values of those types
F# quotations allow you to mark some piece of F# code and get the representation of the source code. This is ued in WebSharper (see for example this tutorial) to translate F# code to JavaScript. Another example is F# support for LINQ where code marked as <# ... #> is translated to SQL:
let res = <# for p in db.Products
if p.IsVisible then yield p.Name #> |> query
Pattern matching is simply a very powerful language construct, but it is nothing more mysterious than for example if. The idea is that you can match value against patterns and program will choose the first matching branch. This is powerful because patterns can be nested and so you can use it to process various complex data structures or implement symbolc processing:
match expr with
| Multiply(Constant 0, _) | Multiply(_, Constant 0) -> 0
| Multiply(expr1, expr2) -> (eval expr1) * (eval expr2)
// (other patterns)
For example, here we're using pattern matching to evaluate some representation of numerical expression. The first pattern is an optimization that deals with cases where one argument of multiplication is 0.
Writing languages You can use F# (just like any other general purpose language) to write compilers and tools for other languages. In F#, this is easy because it comes with tools for generating lexers and parsers. See for example this introduction.
Sometimes in books I see this syntax for list and sequence comprehensions in F#:
seq { for i = 0 to System.Int32.MaxValue -> i }
This is from Programming F# by Chris Smith, page 80. In the F# which comes with VS2010, this doesn't compile. I believe -> has been deprecated. (See Alternative List Comprehension Syntax). However, -> can still be used in comprehensions which involve ranges:
seq { for c in 'A' .. 'Z' -> c }
According to Expert F# 2.0, page 58, this is because -> is shorthand for Seq.map over a range.
Why was the first usage of -> above deprecated?
The current use of -> seems inconsistent. Can anyone reconcile this for me?
The -> construct is supported only in the "simple" sequence expression syntax where you're doing a projection (Seq.map) over some data source using the following structure:
seq { for <binding> in <input> -> <projection> }
The first example you mentioned is using for .. to .. which is a different syntactical construct than for .. in, but you can rewrite it using the second one (In fact, I almost always use for .. in when writing sequence expressions):
seq { for i in 0 .. System.Int32.MaxValue -> i }
In all other forms of sequence expressions you'll have to use yield. In earlier versions of F#, the -> syntax was equivalent to yield (and there was also ->> which was equivalent to yield!). So for example, you was able to write:
seq { -> 10 // You need 'yield' here
->> [ 1; 2; 3 ] } // You need 'yield!' here
This syntax looks quite odd, so I think that the main reason for making these two deprecated is to keep the language consistent. The same computation expression syntax is used in sequence expressions (where -> makes some sense), but also for other computation types (and you can define your own), where yield feels more appropriate (and it also corresponds to return in asynchronous workflows or other computation expressions).
The "simple" sequence-specific syntax is still useful, because it saves you some typing (you replace do yield with just ->), but in more complicated cases, you don't save that many characters and I think that the syntax using -> & ->> can look a bit cryptic.
I wrote the follwing function:
let str2lst str =
let rec f s acc =
match s with
| "" -> acc
| _ -> f (s.Substring 1) (s.[0]::acc)
f str []
How can I know if the F# compiler turned it into a loop? Is there a way to find out without using Reflector (I have no experience with Reflector and I Don't know C#)?
Edit: Also, is it possible to write a tail recursive function without using an inner function, or is it necessary for the loop to reside in?
Also, Is there a function in F# std lib to run a given function a number of times, each time giving it the last output as input? Lets say I have a string, I want to run a function over the string then run it again over the resultant string and so on...
Unfortunately there is no trivial way.
It is not too hard to read the source code and use the types and determine whether something is a tail call by inspection (is it 'the last thing', and not in a 'try' block), but people second-guess themselves and make mistakes. There's no simple automated way (other than e.g. inspecting the generated code).
Of course, you can just try your function on a large piece of test data and see if it blows up or not.
The F# compiler will generate .tail IL instructions for all tail calls (unless the compiler flags to turn them off is used - used for when you want to keep stack frames for debugging), with the exception that directly tail-recursive functions will be optimized into loops. (EDIT: I think nowadays the F# compiler also fails to emit .tail in cases where it can prove there are no recursive loops through this call site; this is an optimization given that the .tail opcode is a little slower on many platforms.)
'tailcall' is a reserved keyword, with the idea that a future version of F# may allow you to write e.g.
tailcall func args
and then get a warning/error if it's not a tail call.
Only functions that are not naturally tail-recursive (and thus need an extra accumulator parameter) will 'force' you into the 'inner function' idiom.
Here's a code sample of what you asked:
let rec nTimes n f x =
if n = 0 then
x
else
nTimes (n-1) f (f x)
let r = nTimes 3 (fun s -> s ^ " is a rose") "A rose"
printfn "%s" r
I like the rule of thumb Paul Graham formulates in On Lisp: if there is work left to do, e.g. manipulating the recursive call output, then the call is not tail recursive.