I need to insert new nodes to AST. for instance, adding a namespace to a function:
Turning this -
void foo();
into this -
namespace bar {
void foo();
}
I read How to clone or create an AST Stmt node of clang? but I prefer not using source-to-source compilation
Tnx
The answer can be found here
http://clang-developers.42468.n3.nabble.com/Adding-nodes-to-Clang-s-AST-td4054800.html
However, the nodes are added to the compiled AST - for instance, in case one wants to inject a namespace to the code, he can't use the namespace directly via the code because the lexer don't know about it.
Related
I am looking for a library or tools which offer functionality to build up a data structure which I can use to find variable or method declarations that are used in a class in another scope.
example code:
class A
{
public void methodA()
{
B external = new B();
external.methodB(); // I would like to know the name/location where this method is declared. something like: classB.java ... line 3
}
}
class B
{
public void methodB()
{
}
}`
Would Rascal be a good candidate to retrieve this kind of information? I have been using the tool before. As far as I know, I can create an AST but this will not have enough information to determine the scope of where certain variables/methods are declared.
If this would not be the right candidate, any ideas on alternatives? My list of candidates I am currently looking into are: Antlr/symtab; JavaParser/JavaSymbolSolver; Spoon; Rascal; JDT
Yes, there are certainly several options to find out where something is declared.
The first answer is to retrieve the decl information from any node in the AST like so, for example:
rascal>myTree.decl
loc: |java+parameter:///myClass/fac(int)/n|
From the location value you see that this is the parameter named n in the method named fac in the class myClass.
Now you can disect the location to go find the parent, using the access fields and function for locations, like myTree.decl.path etc, or you can find your information further in an M3 model which the AST builder can also construct:
model = createM3FromEclipseProject(|project://myProject);
This model contains relations like this: rel[loc, loc] containment, all defined here:
Core language model: https://github.com/usethesource/rascal/blob/master/src/org/rascalmpl/library/analysis/m3/Core.rsc
Java extensions: https://github.com/usethesource/rascal/blob/master/src/org/rascalmpl/library/lang/java/m3/Core.rsc
To find the parents in the containment relation you could try this:
model.containment[myTree.decl]
or this for the reverse lookup:
invert(model.containment)[myTree.decl]
For your particular question the rel[loc src, loc name] uses relation is also quite interesting which maps fully qualified declaration names to the source location uses, and rel[loc name, loc src] declarations which maps the qualified names to where they are declared.
I have two connected problems that I kind of know how to solve "by foot" (using custom validators and scope providers). But I'm surprised they don't work out of the box in xtext so I'll ask anyway (in case I missed something).
I have this DSL:
Model:
'package' name=QualifiedName
imports+=Import*
entities+=Entity*;
Import:
'import' importedNamespace=QualifiedName;
Entity:
name=ID '{'
references += Reference*
'}';
Reference:
name=ID':'entitiy=[Entity]
;
QualifiedName:
ID('.'ID)*
;
Apart from defining multiple entities in the same file I want to be able to define multiple files that "live" in the same package. And I have 2 problems here:
1) I found no obvious way to easily auto-import all elements from the current package. So If I have one file
package com.test
EntityA {}
and a second one
package com.test
import com.test.EntityA // fails if I remove this line
EntityB{
refToA:EntityA // I could make qualified names work here.
// But that is not the goal.
}
the reference to EntityA can't be resolved unless I explicitly import it (which is something that I want to avoid) although they are in the same namespace.
So: is there a way to easily enable outimports for the "current" package?
2) I have enabled the org.eclipse.xtext.validation.NamesAreUniqueValidator which works fine for entities defined in the same file. But if I redefine an imported entity like this
package com.test
import com.test.EntityA
EntityB{
refToA:EntityA
}
EntityA {}
I don't get the expected validation error. Is there something I can do about that without having to write my own validator?
Thx.
To make auto-imports work, you need to use Xbase and
Model:
'package' name=QualifiedName
importSection=XImportSection?
To use Xbase, append this to your grammar: with org.eclipse.xtext.xbase.Xbase
See http://www.lorenzobettini.it/2013/01/the_ximportsection_in_xbase_2_4/ for details.
The second one can only be solved with your own validator AFAIK. The reason is that you can have a grammar which allows:
import com.test.EntityA as X
private EntityA extends X {}
I want to modify the AST by clang LibTooling. How can I clone an AST node or add a new one, e.g. I'd like to create a BinaryOperator with ADD opcode
Some of the clang's AST nodes (classes) have a static Create method that is used to allocate an instance of that node whose memory is managed by the ASTContext instance passed to it. For these classes you may use this method for instantiation purposes. Check clang::DeclRefExpr class for instance.
Other classes miss this method but have public constructor that you may be used to instantiate the object. However, the vanilla new and delete operators are purposely hidden so you cannot use them to instantiate the objects on the heap. Instead, you must use placement new/delete operators providing ASTContext instance as an argument.
Personally, I prefer to allocate all clang related objects using ASTContext instance and let it manage the memory internally so I don't have to bother with it (all memory will get released when ASTContext instance gets destroyed).
Here is a simple class that allocates the memory for the clang object using the placement new operator and ASTContext instance:
#ifndef CLANG_ALLOCATOR_H
#define CLANG_ALLOCATOR_H
#include <clang/AST/ASTContext.h>
/// Allocator that relies on clang's AST context for actual memory
/// allocation. Any class that wishes to allocated an AST node may
/// create an instance of this class for that purpose
class ClangAllocator
{
public:
explicit ClangAllocator(clang::ASTContext& ast_context)
: m_ast_context(ast_context)
{
}
template<class ClassType, class ... Args>
inline ClassType* Alloc(Args&& ... args)
{
return new (m_ast_context) ClassType(std::forward<Args&&>(args)...);
}
private:
clang::ASTContext& m_ast_context;
};
#endif /// CLANG_ALLOCATOR_H
Regarding the AST modifications, probably the best way to accomplish this is to inherit the TreeTransform class and override its Rebuild methods that are invoked to produce new statements for various AST nodes.
If all you require is to replace one AST node with another, very simple way to achieve this is to find its immediate parent statement and then use std::replace on its children. For example:
/// immediate_parent is immediate parent of the old_stmt
std::replace(
immediate_parent->child_begin()
, immediate_parent->child_end()
, old_stmt
, new_stmt);
Creating new AST nodes is quite cumbersome in Clang, and it's not the recommended way to use libTooling. Rather, you should "read" the AST and emit back code, or code changes (rewritings, replacements, etc).
See this article and other articles (and code samples) linked from it for more information on the right way to do this.
While trying to prove to a colleague that it's possible to use C++ classes from F#, I came up with the following proof of concept. The first snippet is the code he provided for the challenge, and the code snippet below is my implementation in F#.
namespace testapp {
struct trivial_foo {
int bar;
__declspec(dllexport) void set(int n) { bar = n; }
__declspec(dllexport) int get() { return bar; }
}
}
open System.Runtime.InteropServices
type TrivialFoo =
struct
val bar: int
new(_bar: int) = { bar = _bar }
end
[<DllImport("Win32Project2.dll", EntryPoint="?get#trivial_foo#testapp##QAEHXZ", CallingConvention = CallingConvention.ThisCall)>]
extern int trivial_foo_get(TrivialFoo& trivial_foo)
[<DllImport("Win32Project2.dll", EntryPoint="?set#trivial_foo#testapp##QAEXH#Z", CallingConvention = CallingConvention.ThisCall)>]
extern void trivial_foo_set(TrivialFoo& trivial_foo, int bar)
type TrivialFoo with
member this.Get() = trivial_foo_get(&this)
member this.Set(bar) = trivial_foo_set(&this, bar)
When debugged in Visual Studio or run as a standalone program, this works predictably: TrivialFoo.Get returns the value of bar and TrivialFoo.Set assigns to it. When run from F# Interactive however, TrivialFoo.Set will not set the field. I suspect it might have something to do with accessing managed memory from unmanaged code, but that doesn't explain why it only happens when using F# Interactive. Does anyone know what's going on here?
I don't think this proof of concept is a good proof of interoperability. You may be better off creating DLL export definitions from your C++ project and use the de-decorated names instead.
As a PoC: F# creates MSIL that fits in the CLI, so it can interoperate with any other CLI language out there. If that is not enough and you want native-to-net interop, consider using COM, or as mentioned above, DLL export definitions on your C++. I personally wouldn't try to interop with C++ class definitions the way you suggest here, there are way easier ways to do that.
Alternatively, just change your C++ project into a .NET C++ project and you can access the classes directly from F#, while still having the power of C++.
Naturally, you may still be wondering why the example doesn't run in FSI. You can see a hint of an answer by running the following:
> System.IO.Directory.GetCurrentDirectory();;
val it : string = "R:\TMP"
To fix this, you have a myriad of options:
copy Win32Project2.dll to that directory
add whatever path it is in to PATH
use an absolute path
use a compile-time constant
or use an environment variable (the path will be expanded)
dynamically locate the dll and dynamically bind to it (complex)
Copying is probably the easiest of these solutions.
Since FSI is meant to be a REPL, it may not be best tailored for this kind of tasks that require multiple projects, libraries or otherwise complex configurations. You may consider voting on this FSI request for support for #package to import NuGet packages, which could be used to ease such tasks.
The counterpart of a C++ struct in F# is not necessarily a struct. In C++, the only difference between classes and structs resides in their default access restrictions.
In F#, structs are used for value types, classes are used for reference types. One problem with value types is that they are meant to be used as immutable values, and temporary copies are often created silently.
The problem you are observing is consistent with that scenario. For some reason, F# interactive creates a copy of your struct and passes a reference to that. The C++ code then modifies the copy, leaving the original untouched.
If you switch to using a class, make sure you pin the instance before letting native code use it, or you can end up in a situation where the garbage collector moves it after the native code gets a reference to it.
I'm using an interpreter for my domain specific language rather than a compiler (despite the performance). I'm struggling to understand some of the concepts though:
Suppose I have a DSL (in XML style) for a game so that developers can make building objects easily:
<building>
<name> hotel </name>
<capacity> 10 </capacity>
</building>
The DSL script is parsed, then what happens?
Does it execute an existing method for creating a new building? As I understand it does not simply transform the DSL into a lower level language (as this would then need to be compiled).
Could someone please describe what an interpreter would do with the resulting parsed tree?
Thank you for your help.
Much depends on your specific application details. For example, are name and capacity required? I'm going to give a fairly generic answer, which might be a bit overkill.
Assumptions:
All nested properties are optional
There are many nested properties, possibly of varying depth
This invites 2 ideas: structuring your interpreter as a recursive descent parser and using some sort of builder for your objects. In your specific example, you'd have a BuildingBuilder that looks something like (in Java):
public class BuildingBuilder {
public BuildingBuilder() { ... }
public BuildingBuilder setName(String name) { ... return this; }
public BuildingBuilder setCapacity(int capacity) { ... return this; }
...
public Building build() { ... }
}
Now, when your parser encounters a building element, use the BuildingBuilder to build a building. Then add that object to whatever context the DSL applies to (city.addBuilding(building)).
Note that if the name and capacity are exhaustive and are always required, you can just create a building by passing the two parameters directly. You can also construct the building and set the properties directly as encountered instead of using the builder (the Builder Pattern is nice when you have many properties and said properties are both immutable and optional).
If this is in a non-object-oriented context, you'll end up implementing some sort of buildBuilding function that takes the current context and the inner xml of the building element. Effectively, you are building a recursive descent parser by hand with an xml library providing the actual parsing of individual elements.
However you implement it, you will probably appreciate having a direct semantic mapping between xml elements in your DSL and methods/objects in your interpreter.