I have the following grammar:
Model: prog+=Program*;
Program: g=Greeting de+=DataEntry* s+=Statement*;
Greeting: 'Hello' t=ProgPara '!';
ProgPara: 'PROGRAM' pname=Progname ';';
Progname : name=ID;
DataEntry: a=INT v=Varname ';';
Varname : name = ID;
Statement: (c=CopyStmt|m=MoveStmt) ';';
CopyStmt: 'COPY' 'TO' qname=[IndexVarname|ID] ;
IndexVarname : (Indexname|Varname);
Indexname : '(' name = ID ')';
MoveStmt: 'MOVE' 'TO' p=[PrVarName|ID];
PrVarName : (Varname|Progname);
But it throws error for:
PrVarName : (Varname|Progname);
So i modified the grammar to have as below:
PrVarName : (v=Varname|Progname);
I updated the Scope provider as below:
override getScope(EObject context, EReference reference) {
if (context instanceof CopyStmt) {
if (reference.featureID == TestDslPackage.COPY_STMT__QNAME) {
val rootElement = EcoreUtil2.getRootContainer(context);
val candidates1 = EcoreUtil2.getAllContentsOfType(rootElement, Indexname);
val candidates2 = EcoreUtil2.getAllContentsOfType(rootElement, Varname);
val candidates = candidates1 + candidates2;
return Scopes.scopeFor(candidates);
}
} else if (context instanceof MoveStmt) {
if (reference.featureID == TestDslPackage.MOVE_STMT__P) {
val rootElement = EcoreUtil2.getRootContainer(context);
val candidates1 = EcoreUtil2.getAllContentsOfType(rootElement, Progname);
val candidates2 = EcoreUtil2.getAllContentsOfType(rootElement, Varname);
val candidates = candidates1 + candidates2;
return Scopes.scopeFor(candidates);
}
}
}
Once the grammar was built and i ran the below test case, it is throwing error in the MOVE statement saying "Couldn't resolve reference to PrVarName 'test1'."
Hello PROGRAM test;!
1 test1;
2 test2;
3 test3;
COPY TO test2;
MOVE TO test1;
Looks like i cannot use Varname in two different cross references. But there is a very valid need for it. How do I achieve this?
Thanks in advance.
PrVarName : p=(Progname|Varname);
is bad cause it changes the type hierarchy. Progname and Varname are no longer subtypes of PrVarName. By reverting the change and introducing a common Named supertype you can resolve this
Model:
prog+=Program*;
Program:
g=Greeting de+=DataEntry* s+=Statement*;
Greeting:
'Hello' t=ProgPara '!';
ProgPara:
'PROGRAM' pname=Progname ';';
DataEntry:
a=INT (v=Varname | in=Indexname) ';';
Statement:
(c=CopyStmt | m=MoveStmt) ';';
CopyStmt:
'COPY' 'TO' qname=[IndexVarname|ID];
MoveStmt:
'MOVE' 'TO' p=[PrVarName|ID];
PrVarName:
Progname | Varname;
IndexVarname:
(Indexname | Varname);
Named:Progname|Indexname|Varname;
Progname:
{Progname} name=ID;
Indexname:
{Indexname}'(' name=ID ')';
Varname:
{Varame}name=ID;
Related
Let's say I have logical expressions in this form
((a OR b) AND c) OR (d AND e = (f OR g))
I'm interested in getting a, b, c, d and ignore e, f, g (everything that has to do with assignments).
My attempt was something like this
infixOp : OR | AND ;
assignment : Identifier EQUALS expression ;
expression :
Identifier |
assignment |
expression infixOp expression |
LEFTBRACKET expression RIGHTBRACKET ;
Then in my listener in enterExpression() I just printed expressionContext.getTokens(identifierType). Of course, this is too much and I got all of them.
How could "skip" over an assignment expression? Could this be done in the grammar? Or it can only be done programatically?
What you can do is create a small listener and keep track of the fact when you enter- and exit an assignment. Then inside the enterExpression method, check if you're NOT inside an assignment AND the Identifier token has a value.
A quick demo for the grammar:
grammar T;
parse
: expression EOF
;
expression
: '(' expression ')'
| expression 'AND' expression
| expression 'OR' expression
| assignment
| Identifier
;
assignment
: Identifier '=' expression
;
Identifier : [a-z]+;
Space : [ \t\r\n] -> skip;
and Java class:
public class Main {
public static void main(String[] args) {
TLexer lexer = new TLexer(CharStreams.fromString("((a OR b) AND c) OR (d AND e = (f OR g))"));
TParser parser = new TParser(new CommonTokenStream(lexer));
MyListener listener = new MyListener();
ParseTreeWalker.DEFAULT.walk(listener, parser.parse());
System.out.println(listener.ids);
}
}
class MyListener extends TBaseListener {
public List<String> ids = new ArrayList<String>();
private boolean inAssignment = false;
#Override
public void enterExpression(TParser.ExpressionContext ctx) {
if (!this.inAssignment && ctx.Identifier() != null) {
this.ids.add(ctx.Identifier().getText());
}
}
#Override
public void enterAssignment(TParser.AssignmentContext ctx) {
this.inAssignment = true;
}
#Override
public void exitAssignment(TParser.AssignmentContext ctx) {
this.inAssignment = false;
}
}
will print:
[a, b, c, d]
Maybe something like:
if (!(expressionContext.parent() instanceof AssignmentContext)) {
expressionContext.getTokens(identifierType);
}
You can fine that by walking the parse tree structure and check the different members in the expression context.
Can the "memory" be avoided in the following Xtext scope provider for the field dot-notation of the following grammar, which is stock-standard except for the field dot-notation of the LocalRelation.
grammar test.path.Path with org.eclipse.xtext.common.Terminals
generate path "http://www.path.test/Path"
Model:
(elements+=AbstractElement)*;
PackageDeclaration:
'package' name=QualifiedName '{'
(elements+=AbstractElement)*
'}';
AbstractElement:
PackageDeclaration | Entity | Import;
Import:
'import' importedNamespace=QualifiedNameWithWildcard;
Entity:
'entity' name=ID '{'
relations+=Relation*
'}';
Relation:
GlobalRelation | LocalRelation;
GlobalRelation:
('id')? name=ID ':' ('*' | '+' | '?')? ref=[Entity|QualifiedName];
LocalRelation:
('id')? name=ID ':' ('*' | '+' | '?')? 'this' '.' head=[Relation|ID] ('.' tail+=[Relation|ID])*;
QualifiedNameWithWildcard:
QualifiedName '.*'?;
QualifiedName:
ID ('.' ID)*;
The follow grammar instance demonstrates the LocalRelation dot-notion.
entity A {
b : B
}
entity B {
a : A
}
entity C {
b : B
x1 : this.b
x2 : this.x1
// x3 : this.x3 No self ref
x4 : this.b.a
x5 : this.x1.a
x6 : this.x1.a.b.a.b.a.b.a
}
entity D {
c : C
x1 : this.c.b.a
}
The scoping resolution for GlobalRelations works out of the box, but of course the scoping for LocalRelations does not. I have come up with the following working scope provider, but it uses a global map to keep track of dot-depth, together with a special head to set the counter to zero, since one cannot sample the value of a reference before it is defined as that causes an infinite loop.
class PathScopeProvider extends AbstractPathScopeProvider {
#Inject extension IQualifiedNameProvider
override getScope(EObject context, EReference reference) {
if (context instanceof LocalRelation) {
return if (reference == PathPackage.Literals.LOCAL_RELATION__HEAD)
getScopeLocalRelation_HEAD(context as LocalRelation, context.eContainer as Entity)
else if (reference == PathPackage.Literals.LOCAL_RELATION__TAIL)
getScopeLocalRelation_TAIL(context as LocalRelation, context.eContainer as Entity)
}
return super.getScope(context, reference);
}
def IScope getScopeLocalRelation_HEAD(LocalRelation contextLocalRelation,
Entity contextLocalRelationContainingEntity) {
// Don't touch contextLocalRelation.head not contextLocalRelation.tail!
val result = newArrayList
contextLocalRelationContainingEntity.relations.filter(
target |
target != contextLocalRelation
).forEach [ target |
{
result.add(EObjectDescription.create(QualifiedName.create(target.name), target))
resetDepth(contextLocalRelation)
}
]
return new SimpleScope(IScope.NULLSCOPE, result)
}
def IScope getScopeLocalRelation_TAIL(LocalRelation contextLocalRelation,
Entity contextLocalRelationContainingEntity) {
// Note that head is well-defined, while tail is well-defined up to depth
val head = contextLocalRelation.head
val result = newArrayList
val depthSoFar = getDepth(contextLocalRelation)
incDepth(contextLocalRelation)
val targetSoFar = if(depthSoFar === 0) head else contextLocalRelation.tail.get(depthSoFar - 1)
if (targetSoFar instanceof GlobalRelation) {
val targetSoFar_Global = targetSoFar as GlobalRelation
targetSoFar_Global.ref.relations.forEach [ t |
result.add(EObjectDescription.create(QualifiedName.create(t.name), t))
]
} else if (targetSoFar instanceof LocalRelation) {
var Relation i = targetSoFar as LocalRelation
while (i instanceof LocalRelation) {
i = if(i.tail.empty) i.head else i.tail.last
}
(i as GlobalRelation).ref.relations.forEach [ t |
result.add(EObjectDescription.create(QualifiedName.create(t.name), t))
]
}
return new SimpleScope(IScope.NULLSCOPE, result)
}
// DEPTH MEMORY
val enity_relation__depthSoFar = new HashMap<String, Integer>()
private def void resetDepth(LocalRelation r) {
enity_relation__depthSoFar.put(r.fullyQualifiedName.toString, 0)
}
private def int getDepth(LocalRelation r) {
enity_relation__depthSoFar.get(r.fullyQualifiedName.toString)
}
private def int incDepth(LocalRelation r) {
enity_relation__depthSoFar.put(r.fullyQualifiedName.toString, getDepth(r) + 1)
}
}
Can this additional depth memory be avoided in any way?
Is there some internal way of detecting the depth of the tail so far scoped?
I have experimented with a try-catch block but that doesn't work, and would be sloppy anyway.
The following solution is derived from Xtext and Dot/Path-Expressions, as suggested by Christian Dietrich, and that is the example to use if you are looking to implement standard field dot path scoping.
The grammar above requires a small change.
LocalRelation:
('id')? name=ID ':' ('*' | '+' | '?')? 'this' '.' path=Path;
Path: head=[Relation|ID] ({Path.curr=current} "." tail+=[Relation|ID])*;
The only change is {Path.curr=current} and this is the "magic ingredient" that makes all the difference. The scope provider is now almost trivial, especially compared to the monster in the OP.
override getScope(EObject context, EReference reference) {
if (context instanceof Path) {
if (reference == PathPackage.Literals.PATH__HEAD) {
// Filter self reference
return new FilteringScope(super.getScope(context, reference), [ e |
!Objects.equals(e.getEObjectOrProxy(), context)
]);
} else { // DOT_EXPRESSION__TAIL
val target = context.curr.followTheDots
if (target === null) {
return IScope::NULLSCOPE
}
return new FilteringScope(super.getScope(target, reference), [ e |
!Objects.equals(e.getEObjectOrProxy(), context)
]);
}
}
return super.getScope(context, reference);
}
def Entity followTheDots(Path path) {
var targetRelation = if(path.tail === null || path.tail.empty) path.head else path.tail.last;
return if (targetRelation instanceof GlobalRelation)
targetRelation.ref
else if (targetRelation instanceof LocalRelation)
targetRelation.path.followTheDots
else null // happens when dot path is nonsense
}
def GlobalRelation followTheDots(LocalRelation exp) {
var targetRelation = if(exp.tail === null || exp.tail.empty) exp.head else exp.tail.last;
return if (targetRelation instanceof GlobalRelation)
targetRelation
else if (targetRelation instanceof LocalRelation)
targetRelation.followTheDots
else null // happens when dot path is nonsense
}
While we cannot "touch" tail in the scope provider, the "magic ingredient" {LocalRelation.curr=current} allows the iterative scope provider full access to the previously fully resolved and well-defined reference via curr. Post scoping, the full resolved and well-defined trail is available via head and tail.
[... Edited because original was wrong! ...]
I have the grammar
Model:
vars+=Vars*
funcs+=Funcs*;
Name:
name=ID;
VarName:
Name;
FuncName:
Name;
Funcs:
'func' left=FuncName (bracket?='(' ')')? '=' right=[Name]';';
Vars:
'var' VarName ';';
where the right hand size of the Func rule can be either of type VarName or FuncName depending is the brackets on the left hand size appear.
Must I modify the xtext grammar or do a type of validation/scoping?
Update 1
the scope function:
override getScope(EObject context, EReference reference) {
if (context instanceof Funcs) {
val func = context as Funcs
if (reference == MultiNameDslPackage.Literals.FUNCS__RIGHT) {
if (func.bracket) {
val rootElement = EcoreUtil2.getRootContainer(context)
val candidates = EcoreUtil2.getAllContentsOfType(rootElement, VarName)
return Scopes.scopeFor(candidates)
} else {
val rootElement = EcoreUtil2.getRootContainer(context)
val candidates = EcoreUtil2.getAllContentsOfType(rootElement, FuncName)
return Scopes.scopeFor(candidates)
}
}
return super.getScope(context, reference);
}
}
The left hand size is independent of the presence of the brackets in the editor.
Update 2
Using validation
#Check
def checkFuncContext(Funcs func) {
if (func.bracket) {
if (!(func.right instanceof VarName)) {
warning("Right hand size must be of Var type",
MultiNameDslPackage.Literals.FUNCS__RIGHT
)
}
} else {
if (!(func.right instanceof FuncName)) {
warning("Right hand size must be of Function type",
MultiNameDslPackage.Literals.FUNCS__RIGHT
)
}
}
}
The warning statements are not executed. The statement func.right instanceof FuncName) does not behave as expected.
How can I test for the correct instance?
Update 3
Using a modified grammar
VarName:
name=ID;
FuncName:
name=ID;
Funcs:
'func' left=FuncName (bracket?='(' ')')? '=' (right=[FuncName] | r1=[VarName]) ';';
does not compile: Decision can match input such as "RULE_ID" using multiple alternatives: 1, 2
You need to change your grammar to get the inheritance order for Name, FuncName and VarName right (Name super type of both)
Either use a parser fragment
fragment Name: name=ID;
Or use
Name:VarName|FuncName;
VarName: name=ID;
FuncName:name=ID;
I have a simple grammar like this:
grammar RuleDSL with org.eclipse.xtext.common.Terminals
generate ruledsl "http://www.ruledsl.org"
Model:
rules += Rule*;
Rule:
rule = 'Rule' '{' subrules += Subrule* '}'
;
Subrule: fieldA=ID opName=Operator fieldC=INT ';';
Operator:
'<' | '<=' | '=' | '>=' | '>'
;
And I wonder why this input is accepted:
Rule {
field1 > 5
field3 = a
}
This is parsed by
#RunWith(XtextRunner)
#InjectWith(RuledslInjectorProvider)
class RuledslParsingTest{
#Inject
ParseHelper<Model> parseHelper
#Test
def void loadModel() {
val result = parseHelper.parse('''
Rule {
field1 > 5
field3 = a
}
''')
println(result)
}
}
and yields - after some iterating through the tree and printing the Subrule -:
field1>5
field3=0
anull0
I can't figure out why this happens. In the generated Eclipse PlugIn, this input is labeled incorrect as it should be.
You an ask the result for its eResource and the Resource for its errors. you may have a look at ValidationTestHelper as well.
I have run into a problem, when i tried to parse a stacked arithmetic comparison expression:
"1<2<3<4<5"
into a logical Tree of Conjunctions:
CONJUNCTION(COMPARISON(1,2,<) COMPARISON(2,3,<) COMPARISON(3,4,<) COMPARISON(4,5,<))
Is there a way in Antlr3 Tree Rewrite rules to iterate through matched tokens and create the result Tree from them in the target language (I'm using java)? So i could make COMPARISON nodes from element x, x-1 of matched 'addition' tokens. I know i can reference the last result of a rule but that way i'd only get nested COMPARISON rules, that's not what i wish for.
/This is how i approached the problem, sadly it doesn't do what i would like to do yet of course.
fragment COMPARISON:;
operator
:
('<'|'>'|'<='|'>='|'=='|'!=')
;
comparison
#init{boolean secondpart = false;}
:
e=addition (operator {secondpart=true;} k=addition)*
-> {secondpart}? ^(COMPARISON ^(VALUES addition*) ^(OPERATORS operator*))
-> $e
;
//Right now what this does is:
tree=(COMPARISON (VALUES (INTEGERVALUE (VALUE 1)) (INTEGERVALUE (VALUE 2)) (INTEGERVALUE (VALUE 3)) (INTEGERVALUE (VALUE 4)) (INTEGERVALUE (VALUE 5))) (OPERATORS < < < <))
//The label for the CONJUNCTION TreeNode that i would like to use:
fragment CONJUNCTION:;
I came up with a nasty solution to this problem by writing actual tree building java code:
grammar testgrammarforcomparison;
options {
language = Java;
output = AST;
}
tokens
{
CONJUNCTION;
COMPARISON;
OPERATOR;
ADDITION;
}
WS
:
('\t' | '\f' | ' ' | '\r' | '\n' )+
{$channel = HIDDEN;}
;
comparison
#init
{
List<Object> additions = new ArrayList<Object>();
List<Object> operators = new ArrayList<Object>();
boolean secondpart = false;
}
:
(( e=addition {additions.add(e.getTree());} ) ( op=operator k=addition {additions.add(k.getTree()); operators.add(op.getTree()); secondpart = true;} )*)
{
if(secondpart)
{
root_0 = (Object)adaptor.nil();
Object root_1 = (Object)adaptor.nil();
root_1 = (Object)adaptor.becomeRoot(
(Object)adaptor.create(CONJUNCTION, "CONJUNCTION")
, root_1);
Object lastaddition = additions.get(0);
for(int i=1;i<additions.size();i++)
{
Object root_2 = (Object)adaptor.nil();
root_2 = (Object)adaptor.becomeRoot(
(Object)adaptor.create(COMPARISON, "COMPARISON")
, root_2);
adaptor.addChild(root_2, additions.get(i-1));
adaptor.addChild(root_2, operators.get(i-1));
adaptor.addChild(root_2, additions.get(i));
adaptor.addChild(root_1, root_2);
}
adaptor.addChild(root_0, root_1);
}
else
{
root_0 = (Object)adaptor.nil();
adaptor.addChild(root_0, e.getTree());
}
}
;
/** lowercase letters */
fragment LOWCHAR
: 'a'..'z';
/** uppercase letters */
fragment HIGHCHAR
: 'A'..'Z';
/** numbers */
fragment DIGIT
: '0'..'9';
fragment LETTER
: LOWCHAR
| HIGHCHAR
;
IDENTIFIER
:
LETTER (LETTER|DIGIT)*
;
addition
:
IDENTIFIER ->^(ADDITION IDENTIFIER)
;
operator
:
('<'|'>') ->^(OPERATOR '<'* '>'*)
;
parse
:
comparison EOF
;
For input
"DATA1 < DATA2 > DATA3"
This outputs tree such as:
If you guys know any better solutions, please tell me about them