Parsing comments with clang - clang

I am trying to utilize clang tooling library for the purpose of my future tool.
What I would like to do with this tool is:
1. parse all the source code (with includes) and detect any of my keywords in the comments (comments will be some kind of interface between the programmer and my tool, which will do various things with the rest of the source code according to commands placed in the comments).
2. according to commands from the source code, do some refactoring of it
The refactoring itself will be done using clang AST, like from example below:
http://eli.thegreenplace.net/2014/07/29/ast-matchers-and-clang-refactoring-tools
The thing I am looking for currently is how to parse the comments, within the same run of clang tooling procedures. I do not want to make separate step just for parsing the source code, because it have to be already done in tooling library.
Do you know how to somehow get the information about comments included in the source code I am parsing by tooling library?

Try the options -Wdocumentation and associated options (as -fparse-all-comments). If U use some tools (as clang-check or clang-tidy, adds these options in the compile commands db.

Related

Building Rascal from source

I'm thinking it might be easiest if I modify the Java syntax used in Rascal to better fit our Java-like language.
Is there a way I can build Rascal from the source? I've cloned the repo from Github and imported it as a project into Eclipse but there are some compilation errors regarding org.eclipse.imp. Before I head down the rabbit-hole of trying to get this all to work in Eclipse I thought I would post here to see if there is an easy way to handle this.
Thanks!
Sure you could build Rascal from scratch; following the developer instructions at https://github.com/cwi-swat/rascal/wiki/Rascal-Developers-Setup---Step-by-Step
On the other hand, if you wish to simply adapt the Java syntax definition it would be better to clone it into your own files. Grammars may look modular, but in reality there are complex interactions between different parts of the grammar. Better to clone and manage the whole thing as your own than depend on two co-evolving definitions.
If you clone the Java grammar Rascal will generate new parsers for you on-the-fly. If this generation becomes cumbersome, a "cached parser" can help you to optimize the deployment of your tools. Please contact us if you need help with that.

How to get type info from Go compiled packages in language x?

I want to write a simple editor with basic autocomplete functionality for the Go language as a pet project. How would one go about doing it? I took a look at the Go plugins for Eclipse and IntelliJ, but they were too big for me to comprehend (not to mention getting one to compile).
The Go standard library offers the building blocks for a Go parser which
you can use to parse the source files and look for function definitions and the like.
There's also the godoc command which
already does what you want: extracting method definitions and it's documentation. You may look in the
source code to see how godoc is
working or use godoc directly.
This editor written in Go projects has a manageable amount of code,
you may look into it.
The de facto standard approach to this problem is to use nsf's gocode. I have tried it only in Vim - it works very well.
Even though there's ready made support for specific editors, gocode is not editor specific. It's a daemon with a communication protocol. It is thus usable from any program.

Java: How to open a library?

I want to open libraries, because currently I want to see the algorithms used for drawing, modify them and implement them in my program. For example: I have tried to create an algorithm on my own for lines. But I failed. And even if I had succeeded, I fear that it might not give the same result as the algorithm in the libraries. And I don't want this to happen. That's why I want to copy the algorithms used for the methods in libraries. And I really hope that this will help me create the application I'm currently working on and with other applications in the future.
I tried to open the libraries with a code editor. But I had troubles finding the libraries- I don't really know where are they placed nor in what files are their codes stored.
How to open a Java library? Or is there a place in the Internet where the code is uploaded?
It sounds like what you want is to get inside the standard Java libraries (so you can see the code for methods like Graphics.drawLine()).
You can download the source files from the same place you got the JDK, if you are on Windows or Linux. For the Mac, see this question. You can even set up Eclipse so that you can debug into that source as if it were your own code.
However, you will probably not find line-drawing code in Java in these libraries - the Graphics implementation will almost certainly use native methods, and may just call existing methods in the OS.
If you are specifically looking for line drawing algorithms, another option would be to look at the Wikipedia page for the Bresenham (aliased) or Wu (antialiased) algorithm.
Edit:
The part of a Graphics2D call that actually puts pixels on the screen is probably inside a system call and therefore the source would not be available.
A java vector graphics library like Batik might have source for some of these algorithms, but probably relies on the Graphics2D calls for most of them. So, you might look for a comprehensive vector graphics library written in a language other than Java, where those graphics calls do not already exist by default.
Alternately, checking the table of contents for a computer graphics book might point you at a variety of algorithms that you could look up on Wikipedia.
For any given library:
Make sure to obey all licenses when using another's code
If you are referring to the Java SDK source code, you can find it here: http://grepcode.com/
If the project is open source, you can usually just get the source from the project website. No problem, though make sure to obey their license.
If the project is NOT open source, well, then you're in a pickle licensing wise, so I do NOT endorse this, however, you would need to use a Java Decompiler such as JD-Gui
As far as what drawing algorithms to use, there are so many different ones (obviously, people have been trying to draw quickly for many many years), your best bet is to figure out exactly what you need to do and then search for that specific need separately. There isn't really a good repository of ALL of them, except maybe wikipedia.
If you are using the libraries they are on your classpath. Check out how to figure out your classpath in whichever IDE you are using and you can find the JARs you depend on. If they are packaged with sources all you need to do it unjar them and look at the sources.
If you don't have access to the sources you can get the code using a Java Decompiler.
If you are trying to look at a standard Java library, see the other answers about getting the source to the JDK.
If you are interested in an open source library (such as something maintained by the Apache project), look on the site of the project for a 'source jar' which you can open with a standard zip utility.
If the library you want is not open source or you cannot find the source for it, you can try to decompile it. If you are using Eclipse, try this decompiler.

How do I implement an F# Read Eval Print Loop (REPL)?

I would like to add an F# REPL to my application for live debugging purposes. I am looking at the source code for fsi.exe (F# Interactive) and there is a ton of stuff going on that looks like it pokes around with F# compiler internals. I cannot get this same code to compile in the context of our application because of this.
Is there a nice simple example of implementing an F# REPL somewhere? I would have hoped for this to be fairly easy.
The short answer is that F# (unfortunatelly) doesn't currently provide any API for hosting F# Interactive in your applications. There are a lot of people asking for this - see for example this SO question.
There are essentially two things you could do about that:
You can modify the open-source release and compile fsi.exe as some DLL library that would provide the API you need. This isn't simple task - F# Interactive is tightly bound with the compiler (it compiles code you enter on the fly), but it should be doable to encapsulate the types implementing REPL into some type you could call (But you cannot just take some file out of it - you need to compile entire F# to get this working).
You can run fsi.exe as a separate process as Visual Studio does and send commands to it using standard input/output. You can get some more flexibility by loading your library when fsi.exe starts. The library could use .NET Remoting to connect back to your application and expose some data.
Unfortunatelly, these two options are probably the only things you can do at the moment.
EDIT I was thinking that I already answered this question somewhere (perhaps in email), but couldn't
find it! Thanks to Mauricio who found the exact duplicate (even with my duplicate answer... Doh!)
I've written a series of blog posts about using the open source F# interactive executable inside and WPF application.
The code base is available on github - https://github.com/oriches/Simple.Wpf.FSharp.Repl
The series of blog posts are:
http://awkwardcoder.blogspot.co.uk/2013/12/simple-f-repl-in-wpf-part-1.html
http://awkwardcoder.blogspot.co.uk/2013/12/simple-f-repl-in-wpf-part-2.html
http://awkwardcoder.blogspot.co.uk/2013/12/simple-f-repl-in-wpf-part-3.html
The final post is to follow soon.

llvm-clang: incremental or online parser?

Is there anyway to use the llvm-clang parser in an incremental/online manner?
Say I'm writing an editor and I want to be able to parse the C++ code I have in front of me.
I don't want to write my own hacked up parser.
I'd like to use something full featured, like llvm-clang.
Is there an easy way to hijack the llvm-clang parser? (And is it fast enough to run it continuously in the background)?
Thanks!
I don't think clang can incrementally parse C++ files, but it's one of this project goals: http://clang.llvm.org/features.html
I've written something similar for my final year project. It wasn't C++ editor, but a Visual Studio plugin, which main task was improving C++ intellisense (like Visual Assist X).
When I was writing this project I've been also thinking about C++ incremental parser, but I haven't found any suitable solution. To solve the C++ intellisense problem I used normal C++ parser from GCC. However it was to slow, to parse file after each code completion request (ctrl+space), just try including boost::spirit. To make this project work properly I parsed files in the background and after each code completion request I compared current file with it's previous version (via diff) to detect changes made from last parsing. Having those changes I updated syntax tree, mostly by adding or removing variables.
Except incremental parsing, there is also another problem with projects like this. Mostly you'll be parsing C++ code which is being edited so it's invalid code. Given the complex C++ grammar, sometimes parser won't be able to recover from syntax errors, so it won't detect correctly some symbols in code.
Another issue are C++ parsers / compilers differences. Let's say I'm using working in Visual Studio and I have used some VC++ compiler specific contruction in my code. Clang parser won't be able to parse it correctly.
For writing something similair to IntelliSense, I would advise you to write your own parser using the LALR parsing algorithm. Since you can save its state in each line so you don't have to reparse the whole file when a file has been editted, which is very fast!
Note that C++ can't be fully expressed in BNF, but I think you could get pretty far with some adjustments. It's ofcourse a lot more work than using Clang's frontend, but you could still use Clang for analysing header files in coöperation with you own written parser.

Resources