Usage of go/parser across packages - parsing

I have used go/parser to parse a golang file and examine it's AST. I have a specific problem for which I want to use go/parser but I hit a roadblock.
Consider that the following files are present in GOPATH/src
$GOPATH/src/
example.go
example_package/
example_package.go
The following are the contents of the files above
example.go
package main
import (
"example_package"
)
type MyObject struct {
base *example_package.BaseObject
}
func DoMyThing(arg *example_package.FirstArg) {
arg.Write(10)
}
func DoMyAnotherThing() {
}
func main() {
example_package.GetItStarted(&MyObject{})
}
example_package.go
package example_package
func GetItStarted(obj interface{}) {
}
type FirstArg interface {
Read() int
Write(x int)
}
type BaseObject struct {
}
func (p *BaseObject) DoSomething(arg *FirstArg, a int) {
arg.Write(arg.Read() + a)
}
My intention is to write a go program called gen_structure that is used like this
$ gen_structure example.go
The output would be
> MyObject
- DoMyThing(arg)
- base
- DoSomething(arg, a)
What did gen_structure do?
It parses example.go and
Extracts "MyObject" from the line example_package.GetItStarted(&MyObject{}) from inside the main() function.
Looks for methods on MyObject that have atleast one argument with the first one being of type *package_example.FirstArg. It finds DoMyThing (and ignored DoMyAnotherThing).
Identifies the member base and peeks inside (by opening the example_package).
Applies the same process to find methods as above and finds DoSomething
Using the collected information, it prints the required output.
I understand I can parse a single file or a bunch of files in the same directory using the functionality within go/parser. However, I am unable to figure out how to resolve symbols across packages (In this case, example_package).
How do I do this?

Call ast.NewPackage to resolve a package names. You will need to supply an importer that returns an *ast.Object for the given import path. If all you want to do is resolve the name to a path, the importer can simply return an *ast.Object with the Kind set to ast.Pkg and the Name set to name of the package. Most of the heavy lifting in the importer can be done with the go/build package. If want to resolve do the AST for the target package, you will need to parse the package and return the ast.Object for the package. To prevent loading the same package multiple times, use the map argument to the importer as a cache of previously loaded packages.
Here's some untested code for finding the resolved package path from the *ast.SelectorExpr se:
if x, _ := se.X.(*ast.Ident); x != nil {
if obj := x.Obj; obj != nil && obj.Kind == ast.Pkg {
if spec, _ := obj.Decl.(*ast.ImportSpec); spec != nil {
if path, err := strconv.Unquote(spec.Path.Value); err == nil {
// path is resolved path for selector expression se.
}
}
}
}
The go/types package can also be used to get this information and more. I recommend using go/types instead of using go/ast directly.

Related

Null Assertions in null-safe mode and How to Avoid If Possible

Learning Dart and using dart_code_metrics to ensure that I write code that meets expectations. One of the rules that is active is avoid-non-null-assertion.
Note, the code below was created to recreate the problem encountered in a larger code base where the value of unitString is taken from a JSON file. As such the program cannot control what is specified in the JSON file.
From pubspec.yaml
environment:
sdk: '>=2.15.0 <3.0.0'
// ignore_for_file: avoid_print
import 'package:qty/qty.dart';
void main() {
const String unitString = 'in';
// unit.Width returns null if unitString is not a unit of Length.
if (Length().unitWith(symbol: unitString) == null) {
print('units $unitString not supported.');
} else {
// The following line triggers avoid-non-null-assertion with the use of !.
final Unit<Length> units = Length().unitWith(symbol: unitString)!;
final qty = Quantity(amount: 0.0, unit: units);
print('Qty = $qty');
}
}
If I don't use ! then I get the following type error:
A value of type 'Unit<Length>?' can't be assigned to a variable of type 'Unit<Length>'.
Try changing the type of the variable, or casting the right-hand type to 'Unit<Length>'.
Casting the right-hand side to
Unit<Length>
fixes the above error but cause a new error when instantiating Quantity() since the constructor expects
Unit<Length>
and not
Unit<Length>?
I assume there is an solution but I'm new to Dart and cannot formulate the correct search query to find the answer.
How can I modify the sample code to make Dart and dart_code_metrics happy?
Your idea of checking for null before using a value is good, it's just not implemented correctly. Dart does automatically promote nullable types to non-null ones when you check for null with an if, but in this case you need to use a temporary variable.
void main() {
const String unitString = 'in';
//Use a temp variable, you could specify the type instead of using just using final
final temp = Length().unitWith(symbol: unitString);
if (temp == null) {
print('units $unitString not supported.');
} else {
final Unit<Length> units = temp;
final qty = Quantity(amount: 0.0, unit: units);
print('Qty = $qty');
}
}
The basic reason for that when you call your unitWith function and see that it's not null the first time, there's no guarantee that the when you call it again that it will still return a non-null value. I think there's another SO question that details this better, but I can't seem to find.

mql4 } not all control paths return a value

I have taken this function from another mql4 script. The other script compiles absolutely fine with no error. Strangely, now that I have copied this function into my script I get the error } not all control paths return a value
I understand the concept of return a value but not sure when there is a compile difference between the scripts
int ModifyOrder(int ord_ticket,double op, double price,double tp, color mColor)
{
int CloseCnt, err;
CloseCnt=0;
while (CloseCnt < 3)
{
if (OrderModify(ord_ticket,op,price,tp,0,mColor))
{
CloseCnt = 3;
}
else
{
err=GetLastError();
Print(CloseCnt," Error modifying order : (", err , ") " + ErrorDescription(err));
if (err>0) CloseCnt++;
}
}
}
Most likely the difference is in #property strict. if using strict mode, you have to redeclare local variables, return value from every function (except void, of course) and some other differences.
In your example the function has to be ended with return CloseCnt; or maybe something else.
No way to declare non-strict mode - simply do not declare the strict one.
Once you declared it, it is applied to that file, and included into other files if importing.

Context dependent ANTLR4 ParseTreeVisitor implementation

I am working on a project where we migrate massive number (more than 12000) views to Hadoop/Impala from Oracle. I have written a small Java utility to extract view DDL from Oracle and would like to use ANTLR4 to traverse the AST and generate an Impala-compatible view DDL statement.
The most of the work is relatively simple, only involves re-writing some Oracle specific syntax quirks to Impala style. However, I am facing an issue, where I am not sure I have the best answer yet: we have a number of special cases, where values from a date field are extracted in multiple nested function calls. For example, the following extracts the day from a Date field:
TO_NUMBER(TO_CHAR(d.R_DATE , 'DD' ))
I have an ANTLR4 grammar declared for Oracle SQL and hence get the visitor callback when it reaches TO_NUMBER and TO_CHAR as well, but I would like to have special handling for this special case.
Is not there any other way than implementing the handler method for the outer function and then resorting to manual traversal of the nested structure to see
I have something like in the generated Visitor class:
#Override
public String visitNumber_function(PlSqlParser.Number_functionContext ctx) {
// FIXME: seems to be dodgy code, can it be improved?
String functionName = ctx.name.getText();
if (functionName.equalsIgnoreCase("TO_NUMBER")) {
final int childCount = ctx.getChildCount();
if (childCount == 4) {
final int functionNameIndex = 0;
final int openRoundBracketIndex = 1;
final int encapsulatedValueIndex = 2;
final int closeRoundBracketIndex = 3;
ParseTree encapsulated = ctx.getChild(encapsulatedValueIndex);
if (encapsulated instanceof TerminalNode) {
throw new IllegalStateException("TerminalNode is found at: " + encapsulatedValueIndex);
}
String customDateConversionOrNullOnOtherType =
customDateConversionFromToNumberAndNestedToChar(encapsulated);
if (customDateConversionOrNullOnOtherType != null) {
// the child node contained our expected child element, so return the converted value
return customDateConversionOrNullOnOtherType;
}
// otherwise the child was something unexpected, signalled by null
// so simply fall-back to the default handler
}
}
// some other numeric function, default handling
return super.visitNumber_function(ctx);
}
private String customDateConversionFromToNumberAndNestedToChar(ParseTree parseTree) {
// ...
}
For anyone hitting the same issue, the way to go seems to be:
changing the grammar definition and introducing custom sub-types for
the encapsulated expression of the nested function.
Then, I it is possible to hook into the processing at precisely the desired location of the Parse tree.
Using a second custom ParseTreeVisitor that captures the values of function call and delegates back the processing of the rest of the sub-tree to the main, "outer" ParseTreeVisitor.
Once the second custom ParseTreeVisitor has finished visiting all the sub-ParseTrees I had the context information I required and all the sub-tree visited properly.

I have a groovy utils file with a few methods in it how can one method call another

Is it possible to have a single .groovy file that has some utility functions defined and have one of those functions call another in that file?
note: for context, this is being used for Jenkins Pipeline library under vars folder. I wanted to have a function used for param validation call another function in the same groovy script file.
i.e. have the someFunction make use of the doSomething function, some pseudo code below.
//utils.groovy
def doSomething(def a) {
def aPrime = a
if (a == 'somethingSpecial') {
//handle it
//some logic goes here
aPrime = b
}
return aPrime
}
def someFunction(def x) {
y = doSomething(x);
more stuff.. using y
return someResult
}
def dodad() {
...
}
def whatsIt(){
...
}
In my actual code I get error like No signature of method: groovysh_evaluate.myCommonFunct() is applicable for argument types: () values: []
Nevermind this does work.
I got the error when I tried to run the contents of the file locally in groovysh. But no errors when it ran in the Jenkins pipeline

using llvm RecursiveASTVisitor for Objective C and iOS

I would like to use clang to preprocess objective C files from an iOS app. I looked over the source code and am trying to implement a pre-processor based on the RecursiveASTVisitor class. However, I seem to be running into many issues that I cannot resolve. I developed a preprocessor to add a "Enter" call at the beginning of each method and an "Exit" call at the end. I also added an "Exit" call before each return statement. I am using the following code to do the instrumentation:
class ExampleVisitor : public RecursiveASTVisitor<ExampleVisitor> {
private:
ASTContext *astContext; // used for getting additional AST info
std::string funcName;
public:
explicit ExampleVisitor(CompilerInstance *CI)
: astContext(&(CI->getASTContext())) // initialize private members
{
rewriter.setSourceMgr(astContext->getSourceManager(), astContext->getLangOpts());
}
virtual bool VisitObjCMethodDecl(ObjCMethodDecl *ND) {
funcName = ND->getDeclName().getAsString();
errs() << "Testing function: " << funcName << "\n";
if (ND->hasBody()) {
rewriter.InsertText(ND->getBody()->getSourceRange().getBegin().getLocWithOffset(1), "\nEnter(\""+funcName+"\");\n");
rewriter.InsertText(ND->getBody()->getSourceRange().getEnd(),"Exit(\""+funcName+"\");\n");
}
return true;
}
virtual bool VisitReturnStmt(ReturnStmt *ret) {
rewriter.InsertText(ret->getSourceRange().getBegin(), "\nExit(\""+funcName+"\");\n");
errs() << "** Rewrote ReturnStmt\n";
return true;
}
virtual ~ExampleVisitor() {}
};
class ExampleASTConsumer : public ASTConsumer {
private:
ExampleVisitor *visitor; // doesn't have to be private
public:
// override the constructor in order to pass CI
explicit ExampleASTConsumer(CompilerInstance *CI)
: visitor(new ExampleVisitor(CI)) // initialize the visitor
{ }
// override this to call our ExampleVisitor on the entire source file
virtual void HandleTranslationUnit(ASTContext &Context) {
/* we can use ASTContext to get the TranslationUnitDecl, which is
a single Decl that collectively represents the entire source file */
visitor->TraverseDecl(Context.getTranslationUnitDecl());
}
};
The code compiles. I created a command line executable "instrument". I then used the following command to run this on a simple Objective C program generated by Xcode:
instrument AppDelegate.m --
I run into two problems. First, I get the error: 'UIKit/UIKit.h' file not found. This is one of the includes generated by Xcode. Second, I'm seeing only some of the return statements being processed in the file. Can someone give me some insights into what is happening?
I'm using the 3.7.0 version of llvm.

Resources