I need to parse simple DSL language like the following:
import "library.txt"
def <int, bool, byte> main(int param1, bool param2)
{
var a = f4(param1); // or var d = f1(f2(f3(f4(param1))));
var b = f3(a);
var c = f2(b);
var d = f1(c);
return <d, param2, b0>;
}
What is the most suitable tool to parse such kind of language?
Lex/Yacc are usually better for complete languages with complicated grammars. Parsec is faster to work with when you have short semi-simple tasks. I think that for your case, Lex/Yacc would be much more suitable.
You might find this bullet-point comparison of FParsec with parser generator tools (e.g. fslex & fsyacc) and "hand‐written" recursive descent parsers useful for choosing between the available alternatives.
What is the most suitable tool to parse such kind of language?
I would use active patterns.
Related
So it is well known that the top-down parsing paradigm can not deal with left-recursion. The grammar must either be refactored to get rid of left-recursion or some other paradigm must be used. I've been working on a parser combinator library and since I'm doing this in a language that allows for global side-effects it struck me that I can use some global store for tracking which rules have fired and which ones have not. This scheme of guarding on certain conditions lets me deal with very simple cases of left-recursion at the cost of some extra annotation on the combinators. Here's an example grammar in TypeScript
var expression = Parser.delay(_ => TestGrammar.expression);
var in_plus = false;
class TestGrammar {
static terminal = Parser.m(x => 'a' === x);
static op = Parser.m(x => '+' === x);
static plus = expression.on_success(_ => {
in_plus = false;
}).then(TestGrammar.op).then(expression).on_enter(_ => {
in_plus = true;
}).guard(_ => !in_plus);
static expression = TestGrammar.plus.or(TestGrammar.terminal);
}
console.log(TestGrammar.expression.parse_input('a+a+a+a'));
The idea is pretty simple. In cases where we might get stuck in a loop the rules are amended with guards like in the case of plus in the above example. The rule fails if we hit a looping condition and the guard is lifted as soon as we make progress.
What I'd like to know is if this idea has been explored and analysed. I'd rather not go down this rabbit hole and try to figure stuff out if this is a dead end.
Have a look at GLL algorithms
(e.g. https://github.com/djspiewak/gll-combinators).
They can handle ambiguous and left-recursive grammars efficiently.
They do not directly call the parser function of the sub-parser, but keep a 'todo'-list of (Parser,Position) tupels (called the Trampoline).
This way the endless loop (recursing into self) is avoided (no tupel is added twice).
While working through Expert F# again, I decided to implement the application for manipulating algebraic expressions. This went well and now I've decided as a next exercise to expand on that by building a more advanced application.
My first idea was to have a setup that allows for a more extendible way of creating functions without having to recompile. To that end I have something like:
type IFunction =
member x.Name : string with get
/// additional members omitted
type Expr =
| Num of decimal
| Var of string
///... omitting some types here that don't matter
| FunctionApplication of IFunction * Expr list
So that say a Sin(x) could be represented a:
let sin = { new IFunction() with member x.Name = "SIN" }
let sinExpr = FunctionApplication(sin,Var("x"))
So far all good, but the next idea that I would like to implement is having additional interfaces to represent function of properties. E.g.
type IDifferentiable =
member Derivative : int -> IFunction // Get the derivative w.r.t a variable index
One of the ideas the things I'm trying to achieve here is that I implement some functions and all the logic for them and then move on to the next part of the logic I would like to implement. However, as it currently stands, that means that with every interface I add, I have to revisit all the IFunctions that I've implemented. Instead, I'd rather have a function:
let makeDifferentiable (f : IFunction) (deriv : int -> IFunction) =
{ f with
interface IDifferentiable with
member x.Derivative = deriv }
but as discussed in this question, that is not possible. The alternative that is possible, doesn't meet my extensibility requirement. My question is what alternatives would work well?
[EDIT] I was asked to expand on the "doesn't meet my extenibility requirement" comment. The way this function would work is by doing something like:
let makeDifferentiable (deriv : int -> IFunction) (f : IFunction)=
{ new IFunction with
member x.Name = f.Name
interface IDifferentiable with
member x.Derivative = deriv }
However, ideally I would keep on adding additional interfaces to an object as I add them. So if I now wanted to add an interface that tell whether on function is even:
type IsEven =
abstract member IsEven : bool with get
then I would like to be able to (but not obliged, as in, if I don't make this change everything should still compile) to change my definition of a sine from
let sin = { new IFunction with ... } >> (makeDifferentiable ...)
to
let sin = { new IFunction with ... } >> (makeDifferentiable ...) >> (makeEven false)
The result of which would be that I could create an object that implements the IFunction interface as well as potentially, but not necessarily a lot of different other interfaces as well; the operations I'd then define on them, would potentially be able to optimize what they are doing based on whether or not a certain function implements an interface. This will also allow me to add additional features/interfaces/operations first without having to change the functions I've defined (though they wouldn't take advantage of the additional features, things wouldn't be broken either.[/EDIT]
The only thing I can think of right now is to create a dictionary for each feature that I'd like to implement, with function names as keys and the details to build an interface on the fly, e.g. along the lines:
let derivative (f : IFunction) =
match derivativeDictionary.TryGetValue(f.Name) with
| false, _ -> None
| true, d -> d.Derivative
This would require me to create one such function per feature that I add in addition to one dictionary per feature. Especially if implemented asynchronously with agents, this might be not that slow, but it still feels a little clunky.
I think the problem that you're trying to solve here is what is called The Expression Problem. You're essentially trying to write code that would be extensible in two directions. Discriminated unions and object-oriented model give you one or the other:
Discriminated union makes it easy to add new operations (just write a function with pattern matching), but it is hard to add a new kind of expression (you have to extend the DU and modify all code
that uses it).
Interfaces make it easy to add new kinds of expressions (just implement the interface), but it is hard to add new operations (you have to modify the interface and change all code that creates it.
In general, I don't think it is all that useful to try to come up with solutions that let you do both (they end up being terribly complicated), so my advice is to pick the one that you'll need more often.
Going back to your problem, I'd probably represent the function just as a function name together with the parameters:
type Expr =
| Num of decimal
| Var of string
| Application of string * Expr list
Really - an expression is just this. The fact that you can take derivatives is another part of the problem you're solving. Now, to make the derivative extensible, you can just keep a dictionary of the derivatives:
let derrivatives =
dict [ "sin", (fun [arg] -> Application("cos", [arg]))
... ]
This way, you have an Expr type that really models just what an expression is and you can write differentiation function that will look for the derivatives in the dictionary.
I thought that I might be able to do this with quotations - but I can't see how.
Should I just use a table of the functions with their names - or is their a way of doing this?
Thanks.
For more info......
I'm calling a lot of f# functions from excel and I wondered if I could write a f# function
let fs_wrapper (f_name:string) (f_params:list double) =
this bit calls fname with f_params
and then use
=fs_wrapper("my_func", 3.14, 2.71)
in the sheet rather than wrap all the functions separately.
You'll need to use standard .NET Reflection to do this. Quotations aren't going to help, because they represent function calls using standard .NET MethodInfo, so you'll need to use reflection anyway. The only benefit of quotations (compared to naive reflection) is that you can compile them, which could give you better performance (but the compilation isn't perfect).
Depending on your specific scenario (e.g. where are the functions located), you'd have to do something like:
module Functions =
let sin x = sin(x)
let sqrt y = sqrt(y)
open System.Reflection
let moduleInfo =
Assembly.GetExecutingAssembly().GetTypes()
|> Seq.find (fun t -> t.Name = "Functions")
let name = "sin"
moduleInfo.GetMethod(name).Invoke(null, [| box 3.1415 |])
Unless you need some extensibility or have a large number of functions, using a dictionary containing string as a key and function value as the value may be an easier option:
let funcs =
dict [ "sin", Functions.sin;
"sqrt", Functions.sqrt ]
funcs.[name](3.1415)
There are many methods but one way is to use Reflection, for instance:
typeof<int>.GetMethod("ToString", System.Type.EmptyTypes).Invoke(1, null)
typeof<int>.GetMethod("Parse", [|typeof<string>|]).Invoke(null, [|"112"|])
GetMethod optionally takes an array of types that define the signature, but you can skip that if your method is unambiguous.
Following up on what Thomas alluded to, have a look at Using and Abusing the F# Dynamic Lookup Operator by Matthew Podwysocki. It offers a syntactically clean way for doing dynamic lookup in F#.
How do F# immutable types interface with C#. I'm just starting to learn F# and I'd like to mix it in with some C# code I have, but I want my F# classes to be immutable.
Let's say we're making a Vector class in F#. Vector.X and Vector.Y should be re-assignable, but only be returning a new Vector class. In C# this would take allot of legwork to make .WithX(float x) clone the existing object and return a new one. Is there an easy way to do this in F#?
I've been searching for some time and I can't seem to find any docs on this. So any help would be great.
And finally, if I imported this class into C# what would its interface look like? Will the F# code restrict me from doing something stupid like Vector.X = 10?
This will look similar regardless of whether it's C# or F#.
You say "in C# it will take legwork", but cmon, I think
Vector WithX(float x) { return new Vector(x, this.Y); }
is it, right?
In both C# and F#, to prevent assignment to the X property, you author a property with a 'getter' but no 'setter'.
I think you're making all of this out to be harder than it is, or maybe I'm misunderstanding what you're asking.
EDIT
For the (I think rare) case of where there are 20 field and you may want to change just a small arbitrary subset of them, I found a cute hack to use F# and C# optional parameters together nicely.
F# Code:
namespace global
open System.Runtime.InteropServices
type Util =
static member Some<'T>(x:'T) = Some x
type MyClass(x:int, y:int, z:string) =
new (toClone:MyClass,
[<Optional>] ?x,
[<Optional>] ?y,
[<Optional>] ?z) =
MyClass(defaultArg x toClone.X,
defaultArg y toClone.Y,
defaultArg z toClone.Z)
member this.X = x
member this.Y = y
member this.Z = z
F# client code:
let a = new MyClass(3,4,"five")
let b = new MyClass(a, y=44) // clone a but change y
C# client code:
var m = new MyClass(3, 4, "five");
var m2 = new MyClass(m, y:Util.Some(44)); // clone m but change y
That is, optional parameters are a nice way to do this, and while C# optional parameters have some limitations, you can expose F# optional parameters in a way that works ok with C#, as suggested above.
F# Record types have a built in way of doing exactly what you're asking:
type Vector = {X:float; Y:float}
let v1 = {X=1.; Y=2.}
let v2 = {v1 with X=3.}
How that interops with C#, I'm not sure (edit: see Brian's comment).
Vector will be immutable from any .NET language, since X and Y are implemented as getters without setters.
I am playing about with Erlang and I am trying to write a simple arithmetic parser.
I want to try and parse the following expression:
((12+3)-4)
I want to parse the expression into a stack of AST nodes. When parsing this expression, I would first of all create a binary expression for the (12+3) expression which would look something like this in C#:
var binaryStructure = new BinaryStructure();
binaryStructure.Left = IntegerLiteralExpression(12);
binaryStructure.Right = IntegerLiteralExpression(4);
binaryStructure.Operator = binaryExpression.Operator != BinaryOperatorType.Addition;
I am quite new to Erlang and I am wondering how I would go about creating a structure like this in Erlang that I can place on a List that I would use as the stack of expressions.
Can anyone suggest how to create such a tree like structure? Would a function be a good fit?
In functional language like Erlang it is far simpler. Just make it
{'+', 12, 3}
In more abstract way
A = 12,
B = 3,
OP = '+',
{OP, A, B}.
Also, have a look to the erl_parse.erl module in the stdlib application.
Reading from to the mkop function:
mkop(L, {Op,Pos}, R) -> {op,Pos,Op,L,R}.
mkop({Op,Pos}, A) -> {op,Pos,Op,A}.