In Xtext, how to tweak certain function calls - xtext

I am using Xtext 2.15 to generate a language that, among other things, processes asynchronous calls in a way they look synchronous.
For instance, the following code in my language:
int a = 1;
int b = 2;
boolean sleepSuccess = doSleep(2000); // sleep two seconds
int c = 3;
int d = 4;
would generate the following Java code:
int a = 1;
int b = 2;
doSleep(2000, new DoSleepCallback() {
public void onTrigger(boolean rc) {
boolean sleepSuccess = rc;
int c = 3;
int d = 4;
}
});
To achieve it, I defined the grammar this way:
grammar org.qedlang.qed.QED with jbase.Jbase // Jbase inherits Xbase
...
FunctionDeclaration return XExpression:
=>({FunctionDeclaration} type=JvmTypeReference name=ValidID '(')
(params+=FullJvmFormalParameter (',' params+=FullJvmFormalParameter)*)?
')' block=XBlockExpression
;
The FunctionDeclaration rule is used to define asynchronous calls. In my language library, I would have as system call:
boolean doSleep(int millis) {} // async FunctionDeclaration element stub
The underlying Java implementation would be:
public abstract class DoSleepCallback {
public abstract void onTrigger(boolean rc);
}
public void doSleep(int millis, DoSleepCallback callback) {
<perform sleep and call callback.onTrigger(<success>)>
}
So, using the inferrer, type computer and compiler, how to identify calls to FunctionDeclaration elements, add a callback parameter and process the rest of the body in an inner class?
I could, for instance, override appendFeatureCall in the language compiler, would it work? There is still a part I don't know how to do...
override appendFeatureCall(XAbstractFeatureCall call, ITreeAppendable b) {
...
val feature = call.feature
...
if (feature instanceof JvmExecutable) {
b.append('(')
val arguments = call.actualArguments
if (!arguments.isEmpty) {
...
arguments.appendArguments(b, shouldBreakFirstArgument)
// HERE IS THE PART I DON'T KNOW HOW TO DO
<IF feature IS A FunctionDeclaration>
<argument.appendArgument(NEW GENERATED CALLBACK PARAMETER)>
<INSERT REST OF XBlockExpression body INSIDE CALLBACK INSTANCE>
<ENDIF>
}
b.append(');')
}
}
So basically, how to tell if "feature" points to FunctionDeclaration? The rest, I may be able to do it...
Related to another StackOverflow entry, I had the idea of implementing FunctionDeclaration in the inferrer as a class instead of as a method:
def void inferExpressions(JvmDeclaredType it, FunctionDeclaration function) {
// now let's go over the features
for ( f : (function.block as XBlockExpression).expressions ) {
if (f instanceof FunctionDeclaration) {
members += f.toClass(f.fullyQualifiedName) [
inferVariables(f)
superTypes += typeRef(FunctionDeclarationObject)
// let's add a default constructor
members += f.toConstructor [
for (p : f.params)
parameters += p.toParameter(p.name, p.parameterType)
body = f.block
]
inferExpressions(f)
]
}
}
}
The generated class would extend FunctionDeclarationObject, so I thought there was a way to identify FunctionDeclaration as FunctionDeclarationObject subclasses. But then, I would need to extend the XFeatureCall default scoping to include classes in order to making it work...
I fully realize the question is not obvious, sorry...
Thanks,
Martin
EDIT: modified DoSleepCallback declaration from static to abstract (was erroneous)

I don't think you can generate what you need using the jvm model inferrer.
You should provide your own subclass of the XbaseCompiler (or JBaseCompiler, if any... and don't forget to register with guice in your runtime module), and override doInternalToJavaStatement(XExpression expr, ITreeAppendable it, boolean isReferenced) to manage how your FunctionDeclaration should be generated.

Related

type safe create Lua tables in Haxe without runtime overhead and without boilerplate

I am trying to write some externs to some Lua libraries that require to pass dictionary tables and I want to make them type safe.
So far, I have been declaring abstract classes with public inline constructors, but this gets tedious really fast:
abstract JobOpts(Table<String, Dynamic>) {
public inline function new(command:String, args:Array<String>) {
this = Table.create(null, {
command: command,
arguments: Table.create(args)
});
}
}
Is there a better way that allows me to keep things properly typed but that does not require that much boilerplate?
Please note that typedefs and anonymous structures are not valid options, because they introduce nasty fields in the created table and also do a function execution to assign a metatable to them:
--typedef X = {cmd: String}
_hx_o({__fields__={cmd=true},cmd="Yo"})
My abstract code example compiles to a clean lua table, but it is a lot of boilerplate
Some targets support #:nativeGen to strip Haxe-specific metadata from objects, but this does not seem to be the case for typedefs on Lua target. Fortunately, Haxe has a robust macro system so you can make the code write itself. Say,
Test.hx:
import lua.Table;
class Test {
public static function main() {
var q = new JobOpts("cmd", ["a", "b"]);
Sys.println(q);
}
}
#:build(TableBuilder.build())
abstract JobOpts(Table<String, Dynamic>) {
extern public inline function new(command:String, args:Array<String>) this = throw "no macro!";
}
TableBuilder.hx:
import haxe.macro.Context;
import haxe.macro.Expr;
class TableBuilder {
public static macro function build():Array<Field> {
var fields = Context.getBuildFields();
for (field in fields) {
if (field.name != "_new") continue; // look for new()
var f = switch (field.kind) { // ... that's a function
case FFun(_f): _f;
default: continue;
}
// abstract "constructors" transform `this = val;`
// into `{ var this; this = val; return this; }`
var val = switch (f.expr.expr) {
case EBlock([_decl, macro this = $x, _ret]): x;
default: continue;
}
//
var objFields:Array<ObjectField> = [];
for (arg in f.args) {
var expr = macro $i{arg.name};
if (arg.type.match(TPath({ name: "Array", pack: [] } ))) {
// if the argument's an array, make an unwrapper for it
expr = macro lua.Table.create($expr, null);
}
objFields.push({ field: arg.name, expr: expr });
}
var objExpr:Expr = { expr: EObjectDecl(objFields), pos: Context.currentPos() };
val.expr = (macro lua.Table.create(null, $objExpr)).expr;
}
return fields;
}
}
And thus...
Test.main = function()
local this1 = ({command = "cmd", args = ({"a","b"})});
local q = this1;
_G.print(Std.string(q));
end
Do note, however, that Table.create is a bit of a risky function - you will only be able to pass in array literals, not variables containing arrays. This can be remedied by making a separate "constructor" function with the same logic but without array➜Table.create unwrapping.

store a lambda that captures this

Using C++ 17, I'm looking for a way to store a lambda that captures the this pointer, without using std::function<>. The reason to not using std::function<> is that I need the guaranty that no dynamic memory allocations are used. The purpose of this, is to be able to define some asynchronous program flow. Example:
class foo {
public:
void start() {
timer(1ms, [this](){
set_pin(1,2);
timer(1ms, [this](){
set_pin(2,1);
}
}
}
private:
template < class Timeout, class Callback >
void timer( Timeout to, Callback&& cb ) {
cb_ = cb;
// setup timer and call cb_ one timeout reached
...
}
??? cb_;
};
Edit: Maybe it's not really clear: std::function<void()> would do the job, but I need / like to have the guaranty, that no dynamic allocations happens as the project is in the embedded field. In practice std::function<void()> seems to not require dynamic memory allocation, if the lambda just captures this. I guess this is due to some small object optimizations, but I would like to not rely on that.
You can write your own function_lite to store the lambda, then you can use static_assert to check the size and alignment requirements are satisfied:
#include <cstddef>
#include <new>
#include <type_traits>
class function_lite {
static constexpr unsigned buffer_size = 16;
using trampoline_type = void (function_lite::*)() const;
trampoline_type trampoline;
trampoline_type cleanup;
alignas(std::max_align_t) char buffer[buffer_size];
template <typename T>
void trampoline_func() const {
auto const obj =
std::launder(static_cast<const T*>(static_cast<const void*>(buffer)));
(*obj)();
}
template <typename T>
void cleanup_func() const {
auto const obj =
std::launder(static_cast<const T*>(static_cast<const void*>(buffer)));
obj->~T();
}
public:
template <typename T>
function_lite(T t)
: trampoline(&function_lite::trampoline_func<T>),
cleanup(&function_lite::cleanup_func<T>) {
static_assert(sizeof(T) <= buffer_size);
static_assert(alignof(T) <= alignof(std::max_align_t));
new (static_cast<void*>(buffer)) T(t);
}
~function_lite() { (this->*cleanup)(); }
function_lite(function_lite const&) = delete;
function_lite& operator=(function_lite const&) = delete;
void operator()() const { (this->*trampoline)(); }
};
int main() {
int x = 0;
function_lite f([x] {});
}
Note: this is not copyable; to add copy or move semantics you will need to add new members like trampoline and cleanup which can properly copy the stored object.
There is no drop in replacement in the language or the standard library.
Every lambda is a unique type in the typesystem. Technically you may have a lambda as a member, but then its type is fixed. You may not assign other lambdas to it.
If you really want to have an owning function wrapper like std::function, you need to write your own. Actually you want a std::function with a big enough small-buffer-optimization buffer.
Another approach would be to omit the this capture and pass it to the function when doing the call. So you have a captureless lambda, which is convertible to a function pointer which you can easily store. I would take this route and adapter complexer ways if really nessessary.
it would look like this (i trimmed down the code a bit):
class foo
{
public:
void start()
{
timer(1, [](foo* instance)
{
instance->set_pin(1,2);
});
}
private:
template < class Timeout, class Callback >
void timer( Timeout to, Callback&& cb )
{
cb_ = cb;
cb_(this); // call the callback like this
}
void set_pin(int, int)
{
std::cout << "pin set\n";
}
void(*cb_)(foo*);
};

Replacement + Side effect

While visiting a compilation unit--- and given a certain condition- I would like to apply a transformation (using the => operator) and count the number of times the same transformation was applied for a given compilation unit.
I was able to perform that using a kind of "global module variable", but I am quite sure that it is possible to combine both replacements and actions within a single visit expression. Is that possible?
module MultiCatch
import lang::java::\syntax::Java18;
import ParseTree;
import IO;
import Map;
import Type;
import List;
// sure, I don't like global variables.
//
// However I could not find a way to perform both
// a replacement and count the number of times
// it was applied in the same compilation unit.
int numberOfOccurences = 0;
/**
* Refactor a try-catch statement to use the
* MultiCatch construct of Java 7.
*/
public tuple[int, CompilationUnit] refactorMultiCatch(CompilationUnit unit) {
numberOfOccurences = 0;
CompilationUnit cu = visit(unit) {
case (TryStatement)`try <Block b1> <Catches c1>` => (TryStatement)`try <Block b1> <Catches mc>`
when mc := computeMultiCatches(c1)
};
return <numberOfOccurences, cu>;
}
/*
* Based on a simple notion of similarity,
* this function calculates the possible
* occurences of MultiCatch.
*/
private Catches computeMultiCatches(cs){
map [Block, tuple[list[CatchType], VariableDeclaratorId, Block] ] mCatches =();
visit(cs){
case(CatchClause)`catch (<CatchType t> <VariableDeclaratorId vId>) <Block b>` :{
if (b in mCatches){
<ts, vId, blk> = mCatches[b];
ts += t;
mCatches[b] = <ts, vId, blk>;
numberOfOccurences += 1;
}
else{
mCatches[b] = <[t], vId, b>;
}
}
}
return generateMultiCatches([mCatches[b] | b <- mCatches]);
}
/*
* Creates a syntactic catch clause (either a simple one or
* a multicatch).
*
* This is a recursive definition. The base case expects only
* one tuple, and than it returns a single catch clause. In the
* recursive definition, at least two tuples must be passed as
* arguments, and thus it returns at least two catches clauses
* (actually, one catch clause for each element in the list)
*/
private Catches generateMultiCatches([<ts, vId, b>]) = {
types = parse(#CatchType, intercalate("| ", ts));
return (Catches)`catch(<CatchType types> <VariableDeclaratorId vId>) <Block b>`;
};
private Catches generateMultiCatches([<ts, vId, b>, C*]) = {
catches = generateMultiCatches(C);
types = parse(#CatchType, intercalate("| ", ts));
return (Catches)`catch(<CatchType types> <VariableDeclaratorId vId>) <Block b> <CatchClause+ catches>`;
};
One way to do it is using a local variable and a block with an insert:
module MultiCatch
import lang::java::\syntax::Java18;
import ParseTree;
import IO;
import Map;
import Type;
import List;
/**
* Refactor a try-catch statement to use the
* MultiCatch construct of Java 7.
*/
public tuple[int, CompilationUnit] refactorMultiCatch(CompilationUnit unit) {
int numberOfOccurences = 0; /* the type is superfluous */
CompilationUnit cu = visit(unit) {
case (TryStatement)`try <Block b1> <Catches c1>` : {
numberOfOccurences += 1;
mc = computeMultiCatches(c1)
insert (TryStatement)`try <Block b1> <Catches mc>`;
}
};
return <numberOfOccurences, cu>;
}
The {...} block allows multiple statements to be executed after the match;
The local variable is now only present in the frame of refactorMultiCatch;
The insert statement has the same effect as the previous => arrow did;
Since the match := always succeeds, I changed the when clause into a simple assigment
There is also other more complex ways to share state in Rascal, but I personally prefer to have state not escape the lexical scope of a function.

Code substitution for DSL using ANTLR

The DSL I'm working on allows users to define a 'complete text substitution' variable. When parsing the code, we then need to look up the value of the variable and start parsing again from that code.
The substitution can be very simple (single constants) or entire statements or code blocks.
This is a mock grammar which I hope illustrates my point.
grammar a;
entry
: (set_variable
| print_line)*
;
set_variable
: 'SET' ID '=' STRING_CONSTANT ';'
;
print_line
: 'PRINT' ID ';'
;
STRING_CONSTANT: '\'' ('\'\'' | ~('\''))* '\'' ;
ID: [a-z][a-zA-Z0-9_]* ;
VARIABLE: '&' ID;
BLANK: [ \t\n\r]+ -> channel(HIDDEN) ;
Then the following statements executed consecutively should be valid;
SET foo = 'Hello world!';
PRINT foo;
SET bar = 'foo;'
PRINT &bar // should be interpreted as 'PRINT foo;'
SET baz = 'PRINT foo; PRINT'; // one complete statement and one incomplete statement
&baz foo; // should be interpreted as 'PRINT foo; PRINT foo;'
Any time the & variable token is discovered, we immediately switch to interpreting the value of that variable instead. As above, this can mean that you set up the code in such a way that is is invalid, full of half-statements that are only completed when the value is just right. The variables can be redefined at any point in the text.
Strictly speaking the current language definition doesn't disallow nesting &vars inside each other, but the current parsing doesn't handle this and I would not be upset if it wasn't allowed.
Currently I'm building an interpreter using a visitor, but this one I'm stuck on.
How can I build a lexer/parser/interpreter which will allow me to do this? Thanks for any help!
So I have found one solution to the issue. I think it could be better - as it potentially does a lot of array copying - but at least it works for now.
EDIT: I was wrong before, and my solution would consume ANY & that it found, including those in valid locations such as inside string constants. This seems like a better solution:
First, I extended the InputStream so that it is able to rewrite the input steam when a & is encountered. This unfortunately involves copying the array, which I can maybe resolve in the future:
MacroInputStream.java
package preprocessor;
import org.antlr.v4.runtime.ANTLRInputStream;
public class MacroInputStream extends ANTLRInputStream {
private HashMap<String, String> map;
public MacroInputStream(String s, HashMap<String, String> map) {
super(s);
this.map = map;
}
public void rewrite(int startIndex, int stopIndex, String replaceText) {
int length = stopIndex-startIndex+1;
char[] replData = replaceText.toCharArray();
if (replData.length == length) {
for (int i = 0; i < length; i++) data[startIndex+i] = replData[i];
} else {
char[] newData = new char[data.length+replData.length-length];
System.arraycopy(data, 0, newData, 0, startIndex);
System.arraycopy(replData, 0, newData, startIndex, replData.length);
System.arraycopy(data, stopIndex+1, newData, startIndex+replData.length, data.length-(stopIndex+1));
data = newData;
n = data.length;
}
}
}
Secondly, I extended the Lexer so that when a VARIABLE token is encountered, the rewrite method above is called:
MacroGrammarLexer.java
package language;
import language.DSL_GrammarLexer;
import org.antlr.v4.runtime.Token;
import java.util.HashMap;
public class MacroGrammarLexer extends MacroGrammarLexer{
private HashMap<String, String> map;
public DSL_GrammarLexerPre(MacroInputStream input, HashMap<String, String> map) {
super(input);
this.map = map;
// TODO Auto-generated constructor stub
}
private MacroInputStream getInput() {
return (MacroInputStream) _input;
}
#Override
public Token nextToken() {
Token t = super.nextToken();
if (t.getType() == VARIABLE) {
System.out.println("Encountered token " + t.getText()+" ===> rewriting!!!");
getInput().rewrite(t.getStartIndex(), t.getStopIndex(),
map.get(t.getText().substring(1)));
getInput().seek(t.getStartIndex()); // reset input stream to previous
return super.nextToken();
}
return t;
}
}
Lastly, I modified the generated parser to set the variables at the time of parsing:
DSL_GrammarParser.java
...
...
HashMap<String, String> map; // same map as before, passed as a new argument.
...
...
public final SetContext set() throws RecognitionException {
SetContext _localctx = new SetContext(_ctx, getState());
enterRule(_localctx, 130, RULE_set);
try {
enterOuterAlt(_localctx, 1);
{
String vname = null; String vval = null; // set up variables
setState(1215); match(SET);
setState(1216); vname = variable_name().getText(); // set vname
setState(1217); match(EQUALS);
setState(1218); vval = string_constant().getText(); // set vval
System.out.println("Found SET " + vname +" = " + vval+";");
map.put(vname, vval);
}
}
catch (RecognitionException re) {
_localctx.exception = re;
_errHandler.reportError(this, re);
_errHandler.recover(this, re);
}
finally {
exitRule();
}
return _localctx;
}
...
...
Unfortunately this method is final so this will make maintenance a bit more difficult, but it works for now.
The standard pattern to handling your requirements is to implement a symbol table. The simplest form is as a key:value store. In your visitor, add var declarations as encountered, and read out the values as var references are encountered.
As described, your DSL does not define a scoping requirement on the variables declared. If you do require scoped variables, then use a stack of key:value stores, pushing and popping on scope entry and exit.
See this related StackOverflow answer.
Separately, since your strings may contain commands, you can simply parse the contents as part of your initial parse. That is, expand your grammar with a rule that includes the full set of valid contents:
set_variable
: 'SET' ID '=' stringLiteral ';'
;
stringLiteral:
Quote Quote? (
( set_variable
| print_line
| VARIABLE
| ID
)
| STRING_CONSTANT // redefine without the quotes
)
Quote
;

Porting javascript to dart

I'd like to port this javascript code to dart.
function Beagle() {
this.argv_ = null;
this.io = null;
};
Beagle.prototype.run = function() {
this.io = this.argv_.io.push();
};
runCommandClass(Beagle);
the probleme is
How to create object Beagle
How to create prototype object Beagle.prototype.run
This kind of Js code (function definition and prototype changes) can be ported to a Dart class. You can follow these main rules :
function Xxxx(){/* js code to init */} (pseudo Js class) translates to :
class Xxxx {
/// constructor
Xxxx() {
/* Dart code to init */
}
}
when you have contructor parameters like in function Xxxx(param1, param2){/* js code to init */} you have to create an other constructor with those parameters :
class Xxxx {
/// constructor with parameters
Xxxx(param1, param2) {
/* Dart code to init with param1, param2 */
}
}
Xxxx.prototype.method1 = function(p1, p2, p3){/* js code for method */} are like methods that have to be translated to :
class Xxxx {
// .... other code
/// method
method1(p1, p2, p3) {
/* Dart code */
}
}
To make your code more clear you can also add type annotations on methods and constructors. This is recommanded by the Dart Style Guide.
Type annotations are important documentation for how a library should be used. Annotating the parameter and return types of public methods and functions helps users understand what the API expects and what it provides.
For instance :
class Xxxx {
/// constructor
Xxxx(String param1, int param2) {
/* Dart code to init with param1, param2 */
}
/// method
void method1(num p1, String p2, DateTime p3) {
/* Dart code */
}
}
class Beagle { //
Map argv_;
int io;
Map portInfo;
// could make sense to make this a constructor, that depends how the Terminal class uses it (didn't look)
void run(this.argv_) {
this.portInfo_ = JSON.parse(this.argv_['argString']); // not tested
io = argv_['io'].length;
}
void sendString_(String s) { // no idea what the underlines at the end of argv_, sendString_, ... are for
// ...
}
void onRead_(String s) {}
void onTerminalResize_(int width, int height) {}
void exit(code) {
// ...
}
void close() {
// ...
}
}
var b = new Beagle(); // not translated from the JS source - just added to show how to create a new object from the Beagle class
b.run(argvFromSomewhere);
This includes a some guessing about what the intention of the JavaScript code might be.
I prefer using specific types when porting from JavaScript. It helped me a lot finding bugs and understanding the intention. When I guessed the wrong type I get an exception at runtime, then I can reason about why I got an unexpected type and which of my assumptions were wrong.

Resources