We have an application in F# with a deep function call dependency graph. It performs financial calculations in a hierarchical fashion.
How could we extract the graph of dependency calls? We are not interested in the full AST, just the dependency between modules/functions.
The purpose is to have a simplified version of the code which can be used in conversations with domain experts.
Scott Wlaschin has done similar things in his 'Cycles and modularity in the wild' article. His approach uses Mono.Cecil as a metadata reader and spit out dependency graphs in SVG. A recent article, 'Analysis of Roslyn vs. the F# compiler' uses the same method.
You might be able to modify these scripts for your purpose. The downside is that the approach isn't very precise; some F#-specific information might be lost when the code is compiled down to IL.
Another approach is to use FSharp.Compiler.Service as a metadata reader; its advantage is flexibility, however, it might take some time to get used to FSharp.Compiler.Service API. You could get started by traversing the whole hierarchy starting from an assembly signature. Here is a relevant example and documentation.
In the same spirit, if functions are annotated with ReflectedDefinitionAttribute, then Reflection, Quotations and Pattern modules can be used to retrieve the dependency graph. This approach is explained and detailed in FSharp Journal.
Related
I have been reading about libadalang 1 2 and I am very impressed by it. However, I was wondering if this technique has already been used and another language supports a library for syntactically and semantically analyzing its code. Is this a unique approach?
C and C++: libclang "The C Interface to Clang provides a relatively small API that exposes facilities for parsing source code into an abstract syntax tree (AST), loading already-parsed ASTs, traversing the AST, associating physical source locations with elements within the AST, and other facilities that support Clang-based development tools." (See libtooling for a C++ API)
Python: See the ast module in the Python Language Services section of the Python Library manual. (The other modules can be useful, as well.)
Javascript: The ongoing ESTree effort is attempting to standardize parsing services over different Javascript engines.
C# and Visual Basic: See the .NET Compiler Platform ("Roslyn").
I'm sure there are lots more; those ones just came off the top of my head.
For a practical and theoretical grounding, you should definitely (re)visit the classical textbook Structure and Interpretation of Computer Programs by Abelson & Sussman (1st edition 1985, 2nd edition 1996), which helped popularise the idea of Metacircular Interpretation -- that is, interpreting a computer program as a formal datastructure which can be interpreted (or otherwise analysed) programmatically.
You can see "libadalang" as ASIS Mark II. AdaCore seems to be attempting to rethink ASIS in a way that will support both what ASIS already can do, and more lightweight operations, where you don't require the source to compile, to provide an analysis of it.
Hopefully the final API will be nicer than that of ASIS.
So no, it is not a unique approach. It has already been done for Ada. (But I'm not aware of similar libraries for other languages.)
When writing object-oriented software, I use dependency injection a lot:
to compose together high-level functionality from lower-level capabilities: my account management service uses repositories and validation services rather than implementing them itself.
to isolate components from their dependencies: my account management service uses its dependencies through interfaces, so that I can swap implementations, mock for unit testing and so on.
What patterns exist in functional programming languages to achieve these goals?
edit: a commenter rightly asks: "what about just passing round functions?". I think that the following comment about function grouping hits the nail on the head - a service is a collection of functions with a shared set of dependencies that I can handle as an atomic group.
In Clojure it seems like protocols solve this nicely, but I was really wondering how the problem is solved more generally...
Some time ago I've read a post describing how dependency injection can be seen as currying in functional programming. I think it's very interesting, and it gives a good perspective on the topic.
At the small scale, things like currying and functions-as-parameters cut down the need for module dependencies. At a larger scale, things like Standard ML functors are very useful for this purpose. Racket has a system called units that does a good job on this too.
I developed a little library which I found helpful for DI in a functional-inspired (JavaScript) environment, it's nothing special, just a bit method I like.
I am currently working on an F# project that contains many parallel calculations. As being bound to the trimmed .Net 4 Silverlight Framework (because of the required Silverlight compatibility) I cannot use the available .Net implmenetations and may only use the Monitor object and simple locking by using the lock Keyword.
Do you have any idea how a Shared-Exclusive lock implementation for F# might be desigend best?
I did some functional programming before but haven't concentrated on doing that parallel stuff (yet).
I'm not quite sure what exactly you need - if you need standard mutual exclusion, then the lock function is available in the Silverlight version of F# runtime.
If you need something more complex (such as multiple readers, single writer), then you can rewrite your code to use F# agents and solve the problem more elegantly. If you can add more details about the higher-level structure of your code, then someone can post an example how to solve your particular problem.
Anyway, the following SO answer shows how to write a reusable agent for multiple readers/single writer:
Implement CCR Interleave Arbiter in F#
As mentioned in the comment, you should probably try to avoid writing locks and low-level synchronization primitives explicitly, as this is a source of infinite number of bugs. F# agents give you a higher-level abstraction that is easier to use.
Theres an excellent chapter on this in Expert F# 2.0, Chapter 13 Reactive, Asynchronous, and Parallel Programming.
See example 13.13 shows a nice Request gate, something similar may be of use.
I have an idea for a hobby project which performs some code analysis and manipulation. This project will require both the concrete and abstract syntax trees of a given source file. Additionally, bi-directional references between the two trees would be helpful. I would like to avoid the work of transcribing a grammar to construct my own lexer and parser.
Is there a standard format for describing either concrete or abstract syntax trees?
Do any widely-used tool chains support outputting to these formats?
I don't have a particular target programming language in mind. Any popular one will do for a prototype, but I'd prefer one I know well: Python, C#, Javascript, or C/C++.
I'd like the ability to run a source file through a tool or library and get back both trees. In an ideal world, it would be practical to run this tool on code as it is being edited by a user and be tolerant of errors. Again, I am simply trying to develop a prototype, so these requirements are pretty lax.
Thanks!
The research community decided that graph exchange was the right thing to do when moving information from one program analysis tool to another.
See http://www.gupro.de/GXL
More recently, the OMG has defined a standard for interchanging Abstract Syntax Trees.
See http://www.omg.org/spec/ASTM/1.0/Beta1/
This problem seems to get solved over and over again.
There's half a dozen "tool bus" proposals made over the years
that all solved it, with no one ever overtaking the industry.
The problem is that a) it is easy to represent ASTs using
any kind of nestable notation [parentheses like LISP,
like XML, ...] so people roll their own solution easily,
and b) for one tool to exchange an AST with another, they
both have to agree essentially on what the AST nodes mean;
but most ASTs are rather accidentally derived from the particular
grammar/parsing technology used by each tool, and there's
almost always disagreement about that between tools.
So, I've seen very few tools that exchange ASTs meaningfully.
If you're doing a hobby thing, I'd stick with a lisp-like
encoding of trees, where each node has the following format:
( ... )
Its easy to generate, and easy to read.
I work on a professional tool to manipulate programs. If we
have print out the AST, we do the above. Mostly individual
ASTs are far too complicated to look at in practice,
so we hardly ever print out the entire AST, at best only
a node and a few children deep. Our tool doesn't exchange
ASTs with anybody (see above reasons :) but does just
fine building it in memory, doing whizzy things with it
for analysis reasons or transformation reasons, and then
either just deleteing it (no need to send it anywhere)
or regenerating the original language text from the tree.
[The latter means you need anti-parsing or "prettyprinting"
technology]
In our project we defined the AST metamodel in UML and use ANTLR (Java) to populate the model. We also maintain the token information from ANTLR after parsing, but we have not yet tried to update the underlying text-file with modifications made on the model.
This has a hideous overhead (in infrastructure, such as Eclipse UML2/EMF), but our goal is to use high-level tools for Model-based/driven Development (MDD, MDA) anyway, so we decided to use it on each level.
I think one of our students once played with OpenArchitectureWare and managed to get changes from the Eclipse-based, generated editor back into the syntax tree (not related to the UML model above) automatically, but I don't know the details about this.
You might also want to look at ANTLR's tree grammars.
Specific standards are an expectation, while more general purpose standards may also be appropriate. Ira Baxter already mentioned GXL, and RDF may be added too, just that it would require an appropriate ontology and is more oriented toward semantic than syntax. Still may be an option to investigate.
For specific standards, Ira Baxter already mentioned ASTM, another one, although it rather targets a specific kind of programming language (logic languages), is a standard for semantic/conceptual graph, known as ISO‑IEC 24707 2007.
Not a standard on its own, but a paper about that matter: Towards Portable Source Code Representations Using XML
.
I don't know any effectively used standard (in this area, that's always house‑made cooking everywhere), I'm just interested too in this topic.
Is it possible to do Aspect Oriented Programming in Delphi? I would be interested in native support as well as third party solutions.
I don't have a specific problem I want to solve with AOP, but am simply interested in studying AOP.
AOP depends on two things:
The ability to inject additional code into an existing unit of code
A mechanism to place conditions on where code should be injected.
This is commonly referred to as code weaving. It is a specialization within the larger study of program transformation.
JIT compiled languages have more options for implementing code weaving than statically compiled programs because more information is retained in the bytecode/IL. They also support reflection, which offers the ability to manipulate code at runtime.
Delphi.NET and Prism have the same access to these capabilities as any other .NET language.
There are two AOP frameworks for Delphi Win32 that I'm aware of. The first is MeAOP, which has already been mentioned. The second is Infra. Both projects take a similar approach to AOP. They use a combination of RTTI and clever pointer manipulation to intercept method calls so you can run additional code before or after the method call. You define your cross-cutting feature as a subclass of the framework's AOP class. You register the methods you want intercepted by passing the method name as a string argument to the AOP framework.
Both frameworks are still actively developed and are actually larger in scope than just AOP. Unfortunately documentation is somewhat sparse (and in Infra's case mostly in Portuguese)
Another project attempted AOP through source code weaving back in 2004 with some success. Basically they built an aspect weaver on top of a general purpose program transformation tool called DMS and used it to inject code into delphi source files prior to compilation. Their aspect oriented language was primarily influenced by AspectJ.
http://www.gray-area.org/Research/GenAWeave/ has links to the original paper and presentation as well as some videos of the transformation process.
It may also be possible to use runtime code instrumentation to accomplish this as well. Its a technique used by some profilers to inject counters and stack traces into running code without modifying the original source. A similar technique could be used to inject crosscutting concerns into a statically compiled executable. The PinTool project is a good example of this.
ClassHelpers in the later versions of Delphi allow some very limited level of AOP type behavior. You can use ClassHelpers to inject behavior into other classes without descending from them. It allows overriding existing methods and then optionally calling that existing method.
The limitation is you must declare a ClassHelper for a specific class and it descendants. Additionally a class can only have one ClassHelper.
These are similar to Extension methods in C#.
The DSharp library features AOP:
https://bitbucket.org/sglienke/dsharp
More info can be found at: https://bitbucket.org/sglienke/dsharp
Also have a look at TVirtualMethodInterceptor.
It's in the RTL since Delphi 2010 and allows you to do OnBefore, OnAfter, etc. calls on all virtual methods on a class.
This call alone should cover must of what you need using Rtti, not weaving which is much faster than run-time weaving.