I'd like to run through a simple Rascal MPL parsing example, and am trying to follow Listing 1 from the Rascal Language Workbench (18531D.pdf) of May 3rd 2011. I've downloaded the current Rascal MPL version 0.5.1, and notice that a few module paths have changed. The following shows the content of my Entities.rsc:
module tut1::Entities
extend lang::std::Layout;
extend lang::std::Id;
extend Type;
start syntax Entities
= entities: Entity* entities;
syntax Entity
= #Foldable entity: "entity" Id name "{" Field* "}";
syntax Field
= field: Symbol Id name;
I'm assuming here that what was Name and Ident is now Id; and what was Type is now Symbol. I then continue as follows:
rascal>import tut1::Entities;
ok
rascal>import ParseTree;
ok
However, when I attempt to execute the crucial parse function, I receive the errors listed below. Where am I going wrong? (Despite the message I note that I can declare a Symbol variable at the Rascal prompt.)
rascal>parse(#Entities, "entity Person { string name integer age }");
Extending again?? ParseTree
Extending again?? Type
expanding parameterized symbols
generating stubs for regular
generating literals
establishing production set
generating item allocations
computing priority and associativity filter
printing the source code of the parser class
|prompt:///|(22,43,<1,22>,<1,65>): Java("Undeclared non-terminal: Symbol, in class: class org.rascalmpl.java.parser.object.$shell$")
org.rascalmpl.parser.gtd.SGTDBF.invokeExpects(SGTDBF.java:139)
org.rascalmpl.parser.gtd.SGTDBF.expandStack(SGTDBF.java:864)
org.rascalmpl.parser.gtd.SGTDBF.expand(SGTDBF.java:971)
org.rascalmpl.parser.gtd.SGTDBF.parse(SGTDBF.java:1032)
org.rascalmpl.parser.gtd.SGTDBF.parse(SGTDBF.java:1089)
org.rascalmpl.parser.gtd.SGTDBF.parse(SGTDBF.java:1082)
org.rascalmpl.interpreter.Evaluator.parseObject(Evaluator.java:493)
org.rascalmpl.interpreter.Evaluator.parseObject(Evaluator.java:544)
org.rascalmpl.library.Prelude.parse(Prelude.java:1644)
org.rascalmpl.library.Prelude.parse(Prelude.java:1637)
sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-2)
somewhere in: $shell$
The example is out-of-date. Something like this would work better:
module tut1::Entities
extend lang::std::Layout; // for spaces and such
extend lang::std::Id; // for the Id non-terminal
start syntax Entities
= entities: Entity* entities;
syntax Entity
= #Foldable entity: "entity" Id name "{" Field* "}";
syntax Field
= field: Id symbol Id name; // now Id is used instead of Symbol and "symbol" is just the name of a slot in the rule
Some explanation:
the imports/extends have gone, they were unnecessary and might be confusing
there was a missing definition for the Symbol non-terminal. I don't know what it was supposed to do, but it should have been defined syntax Symbol = ..., but it did not make sense to me and instead I reused Id to define the type of a field.
the type checker (under development) would have warned you before using the parse function.
Related
I would like to understand how can I analyze methods / functions body to find types that are explicitly referenced from it. I have success analyzing method declaration (return type, parameter types, etc..), however I have no idea how to do that for body.
Assuming following function:
String someFunction(int param) {
final list = <String>['a', 'b', 'c']; // -> DartTypes: String, List<String>
final myClass = MyClass<Arg>(); // -> DartTypes: Arg, MyClass<Arg>
final functionCall = anotherFunction<FunctionArg<Arg>>(); // -> DartTypes: Arg, FunctionArg<Arg>
return 'result';
}
// At is point I would like to know that my function depends on
// String, List<String>, Arg, MyClass<Arg>, FunctionArg<Arg>
// in term of DartType instances with proper typeArguments.
I tried getting AstNode for method element described here: https://stackoverflow.com/a/57043177/2033394
However I could not get elements from nodes to figure out their types. Their declaredElement values are always null. So I can not get back to Element API from AST API.
If you've used the exact snippet from the answer you've referenced, the problem is likely in getParsedLibraryByElement(). This method only parses the referenced library - meaning that you'll get an AST that doesn't necessarily have semantic references (like the declaredElement of AST nodes) set.
Instead, you'll want to use getResolvedLibraryByElement. The AST returned by that method will have its types and references fully resolved.
With the resolved AST, you could visit the body of the method with a custom visitor to find type references. Your definition of "referenced types" isn't really exact - but perhaps you can collect types in visitNamedType for type references and visitVariableDeclaration to collect the types of variables.
I have a matcher that works perfectly for matching operator() calls on instances of a class or classes derived from that class. For example, it matches the final line of:
class MyBase { void operator()(...) {} };
MyBase b;
b(parameters);
using a matcher like:
const auto MyBaseExpr =
expr(hasType(cxxRecordDecl(isSameOrDerivedFrom("::MyBase"))));
Finder->addMatcher(traverse(
TK_AsIs, cxxOperatorCallExpr(
hasOverloadedOperatorName("()"),
hasArgument(0, anyOf(MyBaseExpr, MyOtherBaseExpr)),
hasAnyArgument(...),
this);
But I'd also like to be able to match such calls on instances of typedefs for the base or derived types like in the last line below:
typedef MyBase MyTypedef;
MyTypedef t;
t(parameters);
and I can't seem to fathom the correct way to specify this match. Attempting to use hasUnqualifiedDesugaredType rather than hasType doesn't work since it works on a type rather than a Decl and if I try to do more matching with the type then I can't use isSameOrDerived which returns a Matcher<CXXRecordDecl>. A similar problem occurs when trying to use hasCanonicalType:
.../RedundantStringCStrCheck.cpp:193:40: error: invalid initialization of reference of type ‘const clang::ast_matchers:
:internal::Matcher<clang::QualType>&’ from expression of type ‘clang::ast_matchers::internal::BindableMatcher<clang::Decl>’
193 | expr(hasCanonicalType(cxxRecordDecl(isSameOrDerivedFrom("::MyBase"))));
MyTypedef is defined from MyBase so its Canonical Type should be MyBase. More information about canonical type: https://clang.llvm.org/docs/InternalsManual.html#canonical-types
This is the example from LibASTMatchersReference , it uses hasType().
Thien Tran provided the pointer which led me to the right answer. Here's my original expression
const auto MyBaseExpr =
expr(hasType(cxxRecordDecl(isSameOrDerivedFrom("::MyBase"))));
I was trying to use:
const auto MyBaseExpr =
expr(hasCanonicalType(cxxRecordDecl(isSameOrDerivedFrom("::MyBase"))));
but the description of hasCanonicalType in LibASTMatchersReference shows that it takes and returns Matcher<QualType> yet cxxRecordDecl has type Matcher<Decl>, so this did not compile.
The mismatch of types can be corrected by inserting a call to hasDeclaration. It's then also necessary to keep the call to hasType in order to turn the Matcher<QualType> result of hasCanonicalType back into something that can be passed to expr.
After all that I ended up with:
const auto MyBaseExpr =
expr(hasType(hasCanonicalType(hasDeclaration(cxxRecordDecl(isSameOrDerivedFrom("::MyBase"))))));
which seems to work perfectly.
Is there any way of storing line numbers in the created parse tree, using ANTLR 4? I came across this article, which does it but I think it's for older ANTLR version, because
parser.setASTFactory(factory);
It does not seem to be applicable to ANTLR 4.
I am thinking of having something like
treenode.getLine()
, like we can have
treenode.getChild()
With Antlr4, you normally implement either a listener or a visitor.
Both give you a context where you find the location of the tokens.
For example (with a visitor), I want to keep the location of an assignment defined by a Uppercase identifier (UCASE_ID in my token definition).
The bit you're interested in is ...
ctx.UCASE_ID().getSymbol().getLine()
The visitor looks like ...
static class TypeAssignmentVisitor extends ASNBaseVisitor<TypeAssignment> {
#Override
public TypeAssignment visitTypeAssignment(TypeAssignmentContext ctx) {
String reference = ctx.UCASE_ID().getText();
int line = ctx.UCASE_ID().getSymbol().getLine();
int column = ctx.UCASE_ID().getSymbol().getCharPositionInLine()+1;
Type type = ctx.type().accept(new TypeVisitor());
TypeAssignment typeAssignment = new TypeAssignment();
typeAssignment.setReference(reference);
typeAssignment.setReferenceToken(new Token(ctx.UCASE_ID().getSymbol().getLine(), ctx.UCASE_ID().getSymbol().getCharPositionInLine()+1));
typeAssignment.setType(type);
return typeAssignment;
}
}
I was new to Antlr4 and found this useful to get started with listeners and visitors ...
https://github.com/JakubDziworski/AntlrListenerVisitorComparison/
I have a question regarding how to access fields from another file in ada.
I have this one type that goes :
type Token_Type is (INT_LIT_TOK, IDENT_TOK, ASSIGN_OP_TOK, ADD_OP_TOK,
SUB_OP_TOK, MULT_OP_TOK, DIV_OP_TOK, LE_TOK, LT_TOK,
GE_TOK, EQ_TOK, NE_TOK, GT_TOK, EOS_TOK, IF_TOK,
PRINT_TOK, DO_TOK, WHILE_TOK, THEN_TOK, ELSE_TOK,
LOOP_TOK, LEFT_PAREN_TOK, RIGHT_PAREN_TOK, MAIN_TOK,
SUB_TOK, END_TOK);
Now I have another file where I want to do a comparaison to see if whatever I have is one of those types, so I did
tok = Token_Type.INT_LIT;
but it doesn't seem to like the second part of this line. Did I miss something on how to access fields from a different file?
If your declaration of the type Token_Type is declared in Some_Package, and given that tok is of type Some_Package.Token_Type, you should type
tok = Some_Package.INT_LIT
Some additional comments: You don't need to use the package name if there's a USE clause that mentions the package that will apply to your statement:
procedure Something is
use Some_Package;
begin
if tok = INT_LIT then ...
end Something;
Or, in Ada 2012:
procedure Something is
use all type Some_Package.Token_Type;
begin
if tok = INT_LIT then ...
end Something;
If you really want to include the type name, to make it clear to the reader what you're doing, you could also say
if tok = Some_Package.Token_Type'(Some_Package.INT_LIT) then ...
and you can eliminate the Some_Package. prefixes depending on what USE clauses apply. Sometimes you may have to write your code this way, if the name INT_LIT is ambiguous and there isn't enough context for the compiler to determine that one or the other meaning of INT_LIT is impossible.
For our master project at university we created multiple DSLs using Xtext. One of the DSLs is a model entity DSL which allows the user to create a class with properties and methods.
We reuse Xbase because, of course, we want the methods to have a real programming language without reinventing the wheel:
grammar … with org.eclipse.xtext.xbase.Xbase
generate …
EntityModel:
'package' importedNamespace=QualifiedName
…
implementation=Entity;
Entity:
'entity' name=ValidID '{'
features+=Feature*
'}';
Feature:
LocalVariable | …;
LocalVariable:
'var' (isStatic?='static')? name=ValidID ':' type=JvmTypeReference;
For some reason, even though the type of the LocalVariable is set to JvmTypeReference, when using String (in an actual implementation), it will always show the error
Xtext: Couldn't resolve reference to JvmType 'String'
package com.example
Entity foo {
var bar: String
}
We already tried using a ImportedNamespaceAwareLocalScopeProvider which in getImportedNamespaceResolvers adds java.lang.* like that:
List<ImportNormalizer> implicitImports = super.getImportedNamespaceResolvers(context, ignoreCase);
List<ImportNormalizer> javaLangImports = singletonList(new ImportNormalizer(QualifiedName.create("java", "lang"), true, ignoreCase));
implicitImports.addAll(javaLangImports);
return implicitImports;
Even thought that method is called a lot of times, the imports still don't work. When inspecting the EObject context parameter, it sometimes returns java.lang.String (which I guess is for the JvmTypeReference but it still displays an error.
In the RuntimeModule the new scope provider is configured like that:
public void configureIScopeProviderDelegate(com.google.inject.Binder binder) {
binder.bind(org.eclipse.xtext.scoping.IScopeProvider.class).annotatedWith(com.google.inject.name.Names.named(org.eclipse.xtext.scoping.impl.AbstractDeclarativeScopeProvider.NAMED_DELEGATE)).to(MasterDSLImportedNamespaceAwareLocalScopeProvider.class);
}
In the Workflow we configured
fragment = scoping.ImportNamespacesScopingFragment {}
fragment = exporting.QualifiedNamesFragment {}
fragment = builder.BuilderIntegrationFragment {}
The rest of the project is quite complex already (4 Xtext DSLs in one project and multiple generators). But except for completely different DSLs, they use almost the same workflow and RuntimeModule configuration. Another DSL also uses JvmTypeReference and also doesn't find e.g. boolean or anything else.
The question of course is: are we doing something wrong or is there something else we have to do. It used to work when we had a significantly smaller project but after some major changes suddenly this stopped working.