What is Core Erlang's `letrec` for? - erlang

What is Core Erlang's letrec for?
Richard Carlsson writes, in "Introduction to Core Erlang":
Furthermore letrec expressions allow local (recursive) function definitions, which ERLANG itself does not have, but which are often useful in transformations.
What transformations is letrec useful for?
Does erlc actually generate letrec when translating from Erlang? Or would letrecs only be generated when translating from a non-Erlang source language?

Nowadays, list comprehensions are translated to letrecs
(which are available in Core Erlang), and the letrecs are in
turn translated to ordinary recursive functions.
Note that "Nowadays" is 2010. Also, there are some examples of its use in EEP 52: Allow key and size expressions in map and binary matching:
In OTP 23, all variables used in a segment size expression must be already
bound in the enclosing environment. The previous example must be rewritten
like this using nested cases:
'foo'/1 =
fun (_0) ->
case _0 of
<#{#<Sz>(16,1,'integer',['unsigned'|['big']]),
#<_2>('all',1,'binary',['unsigned'|['big']])}#> when 'true' ->
case _2 of
<#{#<X>(Sz,1,'integer',['unsigned'|['big']])}#> when 'true' ->
X
<_3> when 'true' ->
%% Raise function_clause exception.
.
.
.
end
<_4> when 'true' ->
%% Raise function_clause exception.
.
.
.
end
However, as can be seen from the example, the code for raising the function_clause
exception has been duplicated. The code duplication is no big deal in this simple
example, but it would be in a function where the binary matching clause was followed
by many other clauses. To avoid the code duplication, we must use letrec with
the letrec_goto annotation:
'foo'/1 =
fun (_0) ->
( letrec
'label^0'/0 =
fun () ->
case _0 of
<_1> when 'true' ->
%% Raise function_clause exception.
.
.
.
end
in case _0 of
<#{#<Sz>(16,1,'integer',['unsigned'|['big']]),
#<_2>('all',1,'binary',['unsigned'|['big']])}#> when 'true' ->
case _2 of
<#{#<X>(Sz,1,'integer',['unsigned'|['big']])}#> when 'true' ->
X
<_3> when 'true' ->
apply 'label^0'/0()
end
<_4> when 'true' ->
apply 'label^0'/0()
end
-| ['letrec_goto'] )

Related

Why does Erlang allow putting parentheses after a fun?

This question is about some syntax a partner came across today and though we understand how it works, we don't understand why is it allowed (what is its use?).
Look at this snippet:
fun() -> ok end().
Without the last pair of parentheses this will produce something like:
#Fun<erl_eval.20.82930912>
But with them, the function is evaluated producing:
ok
My question is, why is that syntax allowed in Erlang ? why would I want to create a function just to call it immediately instead of just writing out its contents? is there any practical use to it ?
The only thing we could think about was introducing local variables inside the fun's body (but that would look ugly and unclear to me).
Please note that this other syntax is not allowed in Erlang, even though it follows the same concept of the former:
fun() -> fun() -> ok end end()().
(It would mean: a function A that returns a function B. And I'm evaluating A (thus producing B) and then evaluating B to get 'ok').
The syntax you mentioned is a natural outcome of Erlang's being functional.
In Erlang, functions are values (stored as closures).
The value of fun() -> ok end is a function, which takes nothing and returns ok. When we put parentheses after it, we are calling that function. Another way to demonstrate this is:
> F = fun() -> ok end.
#Fun<erl_eval.20.80484245>
> F().
ok
The functions in the second example of yours need to be grouped properly in order for the parser to make sense of them.
As for your question -- "why this syntax is allowed", I'd have to say it's a natural outcome of functions being values in Erlang. This ability enables the functional style of programming. Here is an example:
> lists:map(fun(X) -> X * 2 end, [1,2,3]).
[2,4,6]
The above code is in essence this:
> [fun(X) -> X * 2 end(1), fun(X) -> X * 2 end(2), fun(X) -> X * 2 end(3)].
[2,4,6]
A "natural outcome" is just a natural outcome, it really doesn't have to be of any practical use. So, you will probably never see code like (fun() -> fun() -> ok end end())(). being used:)
You typically wont't have much use for the syntax fun() -> ok end (). But it can be useful to do something like (find_right_fun()) (), which is basically the same thing - an expression that evaluates to a function.
Note that the Erlang parser requires you to specify the precedence using () to sort out the meaning of ()(), i.e. your second example should be (fun() -> fun() -> ok end end()) ().

Evaluate function inside quotation

I'm at the moment doing some very basic pattern matching with quotations.
My code:
let rec test e =
match e with
| Patterns.Lambda(v,e) -> test e
| Patterns.Call(_, mi, [P.Value(value, _); P.Value(value2, _)]) ->
printfn "Value1: %A | Value2 : %A" value value2
| Patterns.Call(_, mi, [P.Value(value, _); P.PropertyGet(_, pi, exprs)]) ->
printfn "Value1: %A | Value2 : %A" value (pi.GetValue(pi, null))
| _ -> failwith "Expression not supported"
let quot1 = <# "Name" = "MyName" #>
(* Call (None, Boolean op_Equality[String](System.String, System.String),
[Value ("Name"), Value ("lol")]) *)
let quot2 = <# "Name" = getNameById 5 #>
(* Call (None, Boolean op_Equality[String](System.String, System.String),
[Value ("Name"),
Call (None, System.String getNameById[Int32](Int32), [Value (5)])]) *)
test quot1 // Works!
test quot2 // Fails.. Dosent match any of the patterns.
Is it possible to somehow evaluate the result of the getNameById function first, so that it will match one of the patterns, or am I doomed to assign a let binding with the result of the function outside the quotation?
I've tried playing with the ExprShape patterns, but without luck..
You can use PowerPack's Eval to evaluate only the arguments to the Call expression:
match e with
| Call(_,mi,[arg1;arg2]) ->
let arg1Value, arg2Value = arg1.Eval(), arg2.Eval()
...
And similarly for Lambda expressions, etc. Noticed this frees you from enumerating permutations of Value, Property, and other argument expressions.
Update
Since you want to avoid using Eval (for good reason if you are implementing a performance conscious application), you'll need to implement your own eval function using reflection (which is still not lightening fast, but should be faster than PowerPack's Eval which involves an intermediate translation of F# Quotations to Linq Expressions). You can get started by supporting a basic set of expressions, and expand from there as needed. Recursion is the key, the following can help you get started:
open Microsoft.FSharp.Quotations
open System.Reflection
let rec eval expr =
match expr with
| Patterns.Value(value,_) -> value //value
| Patterns.PropertyGet(Some(instance), pi, args) -> //instance property get
pi.GetValue(eval instance, evalAll args) //notice recursive eval of instance expression and arg expressions
| Patterns.PropertyGet(None, pi, args) -> //static property get
pi.GetValue(null, evalAll args)
| Patterns.Call(Some(instance), mi, args) -> //instance call
mi.Invoke(eval instance, evalAll args)
| Patterns.Call(None, mi, args) -> //static call
mi.Invoke(null, evalAll args)
| _ -> failwith "invalid expression"
and evalAll exprs =
exprs |> Seq.map eval |> Seq.toArray
And then wrapping this in an Active Pattern will improve syntax:
let (|Eval|) expr =
eval expr
match e with
| Patterns.Call(_, mi, [Eval(arg1Value); Eval(arg2Value)]) -> ...
Update 2
OK, this thread got me motivated to try and implement a robust reflection based solution, and I've done so with good results which are now part of Unquote as of version 2.0.0.
It turned out not to be as difficult as I thought it would be, currently I am supporting all quotation expressions except for AddressGet, AddressSet, and NewDelegate. This is already better than PowerPack's eval, which doesn't support PropertySet, VarSet, FieldSet, WhileLoop, ForIntegerRangeLoop, and Quote for example.
Some noteworthy implementation details are with VarSet and VarGet, where I need to pass around an environment name / variable lookup list to each recursive call. It is really an excellent example of the beauty of functional programming with immutable data-structures.
Also noteworthy is special care taken with issues surrounding exceptions: striping the TargetInvokationExceptions thrown by reflection when it catches exceptions coming from methods it is invoking (this is very important for handling TryWith evaluation properly, and also makes for better user handling of exceptions which fly out of the quotation evaluation.
Perhaps the most "difficult" implementation detail, or really the most grueling, was the need to implement all of the core operators (well, as most I could discover: the numeric and conversion operators, checked versions as well) since most of them are not given dynamic implementations in the F# library (they are implemented using static type tests with no fallback dynamic implementations), but also means a serious performance increase when using these functions.
Some informal benchmarking I observe performance increases of up to 50 times over PowerPack's (not pre-compiled) eval.
I am also confident that my reflection-based solution will be less bug prone then PowerPack's, simply because it is less complicated than the PowerPack's approach (not to mention I've backed it up with about 150 unit tests, duly fortified by Unquotes additional 200+ unit tests which now is driven by this eval implementation).
If you want to peek at the source code, the main modules are Evaluation.fs and DynamicOperators.fs (I've locked the links into revision 257). Feel free to grab and use the source code for your own purposes, it licensed under Apache License 2.0! Or you could wait a week or so, when I release Unquote 2.0.0 which will include evaluation operators and extensions publicly.
You can write an interpreter that will evaluate the quotation and call the getNameById function using Reflection. However, that would be quite a lot of work. The ExprShape isn't going to help you much - it is useful for simple traversing of quotations, but to write an interpreter, you'll need to cover all patterns.
I think the easiest option is to evaluate quotations using the PowerPack support:
#r "FSharp.PowerPack.Linq.dll"
open Microsoft.FSharp.Linq.QuotationEvaluation
let getNameById n =
if n = 5 then "Name" else "Foo"
let quot1 = <# "Name" = "MyName" #>
let quot2 = <# "Name" = getNameById 5 #>
quot1.Eval()
quot2.Eval()
This has some limitations, but it is really the easiest option. However, I'm not really sure what are you trying to achieve. If you could clarify that, then you may get a better answer.

Confusing about F# Quotations and Pattern Matching in Meta-programming

1- I'm really confusing on applying F# Quotation & Pattern on Meta Programming, please suggest some way to approach this concept in F#.
2- Can you show me some real application of F# Quotations and Pattern in Meta Programming ?
3- Some guys said that he can even make another language like IronScheme by F#,is that right ?
Thanks.
1- I'm really confusing on applying F# Quotation & Pattern on Meta Programming, please suggest some way to approach this concept in F#.
A quotation mechanism lets you embed code in your code and have the compiler transform that code from the source you provide into a data structure that represents it. For example, the following gives you a data structure representing the F# expression 1+2:
> <# 1+2 #>;;
val it : Quotations.Expr<int> =
Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),
[Value (1), Value (2)])
{CustomAttributes = [NewTuple (Value ("DebugRange"),
NewTuple (Value ("stdin"), Value (3), Value (3), Value (3), Value (6)))];
Raw = ...;
Type = System.Int32;}
You can then hack on this data structure in order to apply transformations to your code, such as translating it from F# to Javascript in order to run it client side on almost any browser.
2- Can you show me some real application of F# Quotations and Pattern in Meta Programming ?
The F# quotation mechanism is extremely limited in functionality compared to the quotation mechanisms of languages like OCaml and Lisp, to the point where I wonder why it was ever added. Moreover, although the .NET Framework and F# compiler provide everything required to compile and execute quoted code at full speed, the evaluation mechanism for quoted code is orders of magnitude slower than real F# code which, again, renders it virtually useless. Consequently, I am not familiar with any real applications of it beyond Websharper.
For example, you can only quote certain kinds of expressions in F# and not other code such as type definitions:
> <# type t = Int of int #>;;
<# type t = Int of int #>;;
---^^^^
C:\Users\Jon\AppData\Local\Temp\stdin(4,4): error FS0010: Unexpected keyword 'type' in quotation literal
Most quotation mechanisms let you quote any valid code at all. For example, OCaml's quotation mechanism can quote the type definition that F# just barfed on:
$ ledit ocaml dynlink.cma camlp4oof.cma
Objective Caml version 3.12.0
Camlp4 Parsing version 3.12.0
# open Camlp4.PreCast;;
# let _loc = Loc.ghost;;
val _loc : Camlp4.PreCast.Loc.t = <abstr>
# <:expr< 1+2 >>;;
- : Camlp4.PreCast.Ast.expr =
Camlp4.PreCast.Ast.ExApp (<abstr>,
Camlp4.PreCast.Ast.ExApp (<abstr>,
Camlp4.PreCast.Ast.ExId (<abstr>, Camlp4.PreCast.Ast.IdLid (<abstr>, "+")),
Camlp4.PreCast.Ast.ExInt (<abstr>, "1")),
Camlp4.PreCast.Ast.ExInt (<abstr>, "2"))
# <:str_item< type t = Int of int >>;;
- : Camlp4.PreCast.Ast.str_item =
Camlp4.PreCast.Ast.StSem (<abstr>,
Camlp4.PreCast.Ast.StTyp (<abstr>,
Camlp4.PreCast.Ast.TyDcl (<abstr>, "t", [],
Camlp4.PreCast.Ast.TySum (<abstr>,
Camlp4.PreCast.Ast.TyOf (<abstr>,
Camlp4.PreCast.Ast.TyId (<abstr>,
Camlp4.PreCast.Ast.IdUid (<abstr>, "Int")),
Camlp4.PreCast.Ast.TyId (<abstr>,
Camlp4.PreCast.Ast.IdLid (<abstr>, "int")))),
[])),
Camlp4.PreCast.Ast.StNil <abstr>)
FWIW, here is an example in Common Lisp:
$ sbcl
This is SBCL 1.0.29.11.debian, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.
SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses. See the CREDITS and COPYING files in the
distribution for more information.
* '(+ 1 2)
(+ 1 2)
Metaprogramming is one application where pattern matching can be extremely useful but pattern matching is a general-purpose language feature. You may appreciate my article from the Benefits of OCaml about a minimal interpreter. In particular, note how easy pattern matching makes it to act upon each of the different kinds of expression:
> let rec eval vars = function
| EApply(func, arg) ->
match eval vars func, eval vars arg with
| VClosure(var, vars, body), arg -> eval ((var, arg) :: vars) body
| _ -> invalid_arg "Attempt to apply a non-function value"
| EAdd(e1, e2) -> VInt (int(eval vars e1) + int(eval vars e2))
| EMul(e1, e2) -> VInt (int(eval vars e1) * int(eval vars e2))
| EEqual(e1, e2) -> VBool (eval vars e1 = eval vars e2)
| EIf(p, t, f) -> eval vars (if bool (eval vars p) then t else f)
| EInt i -> VInt i
| ELetRec(var, arg, body, rest) ->
let rec vars = (var, VClosure(arg, vars, body)) :: vars in
eval vars rest
| EVar s -> List.assoc s vars;;
val eval : (string * value) list -> expr -> value = <fun>
That OCaml article was used as the basis of the F#.NET Journal article "Language-oriented programming: The Term-level Interpreter" (31st December 2007).
3- Some guys said that he can even make another language like IronScheme by F#,is that right ?
Yes, you can write compilers in F#. In fact, F# is derived from a family of languages that were specifically designed for metaprogramming, the so-called MetaLanguages (ML) family.
The article "Run-time code generation using System.Reflection.Emit" (31st August 2008) from the F#.NET Journal described the design and implementation of a simple compiler for a minimal language called Brainf*ck. You can extend this to implement more sophisticated languages like Scheme. Indeed, the F# compiler is mostly written in F# itself.
On a related note, I just completed a project writing high-performance serialization code that used reflection to consume F# types in a project and then spit out F# code to serialize and deserialize values of those types
F# quotations allow you to mark some piece of F# code and get the representation of the source code. This is ued in WebSharper (see for example this tutorial) to translate F# code to JavaScript. Another example is F# support for LINQ where code marked as <# ... #> is translated to SQL:
let res = <# for p in db.Products
if p.IsVisible then yield p.Name #> |> query
Pattern matching is simply a very powerful language construct, but it is nothing more mysterious than for example if. The idea is that you can match value against patterns and program will choose the first matching branch. This is powerful because patterns can be nested and so you can use it to process various complex data structures or implement symbolc processing:
match expr with
| Multiply(Constant 0, _) | Multiply(_, Constant 0) -> 0
| Multiply(expr1, expr2) -> (eval expr1) * (eval expr2)
// (other patterns)
For example, here we're using pattern matching to evaluate some representation of numerical expression. The first pattern is an optimization that deals with cases where one argument of multiplication is 0.
Writing languages You can use F# (just like any other general purpose language) to write compilers and tools for other languages. In F#, this is easy because it comes with tools for generating lexers and parsers. See for example this introduction.

Why was "->" deprecated in F# comprehensions?

Sometimes in books I see this syntax for list and sequence comprehensions in F#:
seq { for i = 0 to System.Int32.MaxValue -> i }
This is from Programming F# by Chris Smith, page 80. In the F# which comes with VS2010, this doesn't compile. I believe -> has been deprecated. (See Alternative List Comprehension Syntax). However, -> can still be used in comprehensions which involve ranges:
seq { for c in 'A' .. 'Z' -> c }
According to Expert F# 2.0, page 58, this is because -> is shorthand for Seq.map over a range.
Why was the first usage of -> above deprecated?
The current use of -> seems inconsistent. Can anyone reconcile this for me?
The -> construct is supported only in the "simple" sequence expression syntax where you're doing a projection (Seq.map) over some data source using the following structure:
seq { for <binding> in <input> -> <projection> }
The first example you mentioned is using for .. to .. which is a different syntactical construct than for .. in, but you can rewrite it using the second one (In fact, I almost always use for .. in when writing sequence expressions):
seq { for i in 0 .. System.Int32.MaxValue -> i }
In all other forms of sequence expressions you'll have to use yield. In earlier versions of F#, the -> syntax was equivalent to yield (and there was also ->> which was equivalent to yield!). So for example, you was able to write:
seq { -> 10 // You need 'yield' here
->> [ 1; 2; 3 ] } // You need 'yield!' here
This syntax looks quite odd, so I think that the main reason for making these two deprecated is to keep the language consistent. The same computation expression syntax is used in sequence expressions (where -> makes some sense), but also for other computation types (and you can define your own), where yield feels more appropriate (and it also corresponds to return in asynchronous workflows or other computation expressions).
The "simple" sequence-specific syntax is still useful, because it saves you some typing (you replace do yield with just ->), but in more complicated cases, you don't save that many characters and I think that the syntax using -> & ->> can look a bit cryptic.

Dynamic pattern matching

How can I do dynamic pattern matching in Erlang?
Supose I have the function filter/2 :
filter(Pattern, Array)
where Pattern is a string with the pattern I want to match (e.g "{book, _ }" or "{ebook, _ }") typed by an user and Array is an array of heterogenous elements (e.g {dvd, "The Godfather" } , {book, "The Hitchhiker's Guide to the Galaxy" }, {dvd, "The Lord of Rings"}, etc) Then I would like filter/2 above to return the array of elements in Array that match Pattern.
I've tried some ideas with erl_eval without any sucess...
tks in advance.
With little bit documentation study:
Eval = fun(S) -> {ok, T, _} = erl_scan:string(S), {ok,[A]} = erl_parse:parse_exprs(T), {value, V, _} = erl_eval:expr(A,[]), V end,
FilterGen = fun(X) -> Eval(lists:flatten(["fun(",X,")->true;(_)->false end."])) end,
filter(FilterGen("{book, _}"), [{dvd, "The Godfather" } , {book, "The Hitchhiker's Guide to the Galaxy" }, {dvd, "The Lord of Rings"}]).
[{book,"The Hitchhiker's Guide to the Galaxy"}]
Is there any special reason why you want the pattern in a string?
Patterns as such don't exist in Erlang, they can really only occur in code. An alternative is to use the same conventions as with ETS match and select and write your own match function. It is really quite simple. The ETS convention uses a term to represent a pattern where the atoms '$1', '$2', etc are used as variables which can be bound and tested, and '_' is the don't care variable. So your example patterns would become:
{book,'_'}
{ebook,'_'}
{dvd,"The Godfather"}
This is probably the most efficient way of doing it. There is the possibility of using match specifications here but it would complicate the code. It depends on how complicated matching you need.
EDIT:
I add without comment code for part of the matcher:
%% match(Pattern, Value) -> {yes,Bindings} | no.
match(Pat, Val) ->
match(Pat, Val, orddict:new()).
match([H|T], [V|Vs], Bs0) ->
case match(H, V, Bs0) of
{yes,Bs1} -> match(T, Vs, Bs1);
no -> no
end;
match('_', _, Bs) -> {yes,Bs}; %Don't care variable
match(P, V, Bs) when is_atom(P) ->
case is_variable(P) of
true -> match_var(P, V, Bs); %Variable atom like '$1'
false ->
%% P just an atom.
if P =:= V -> {yes,Bs};
true -> no
end
end.
match_var(P, V, Bs) ->
case orddict:find(P, Bs) of
{ok,B} when B =:= V -> {yes,Bs};
{ok,_} -> no;
error -> {yes,orddict:store(P, V, Bs)}
end.
You can use lists:filter/2 to do the filtering part. Converting the string to code is a different matter. Are all the patterns in the form of {atom, _}? If so, you might be able to store the atom and pass that into the closure argument of lists:filter.
Several possibilities come to the mind, depending on how dynamic the patterns are and what features you need in your patterns:
If you need exactly the syntax of erlang patterns and the pattern doesnt't change very often. You could create the matching source code and write it to a file. Use compile:file to create a binary and load this with code:load_binary.
Advantage: Very fast matching
Disadvantage: overhead when pattern changes
Stuff the data from Array into ETS and use match specifications to get out the data
You might use fun2ms to help create the match specification. But fun2ms normally is used as a parse transfor during compile time. There is also a mode used by the shell that can be made to work from strings with the help of the parser probably. For details see ms_transform
There might also be some way to use qlc but I didn't look into this in detail.
In any case be careful to sanitize your matching data if it comes from untrusted sources!

Resources