I have been reading about libadalang 1 2 and I am very impressed by it. However, I was wondering if this technique has already been used and another language supports a library for syntactically and semantically analyzing its code. Is this a unique approach?
C and C++: libclang "The C Interface to Clang provides a relatively small API that exposes facilities for parsing source code into an abstract syntax tree (AST), loading already-parsed ASTs, traversing the AST, associating physical source locations with elements within the AST, and other facilities that support Clang-based development tools." (See libtooling for a C++ API)
Python: See the ast module in the Python Language Services section of the Python Library manual. (The other modules can be useful, as well.)
Javascript: The ongoing ESTree effort is attempting to standardize parsing services over different Javascript engines.
C# and Visual Basic: See the .NET Compiler Platform ("Roslyn").
I'm sure there are lots more; those ones just came off the top of my head.
For a practical and theoretical grounding, you should definitely (re)visit the classical textbook Structure and Interpretation of Computer Programs by Abelson & Sussman (1st edition 1985, 2nd edition 1996), which helped popularise the idea of Metacircular Interpretation -- that is, interpreting a computer program as a formal datastructure which can be interpreted (or otherwise analysed) programmatically.
You can see "libadalang" as ASIS Mark II. AdaCore seems to be attempting to rethink ASIS in a way that will support both what ASIS already can do, and more lightweight operations, where you don't require the source to compile, to provide an analysis of it.
Hopefully the final API will be nicer than that of ASIS.
So no, it is not a unique approach. It has already been done for Ada. (But I'm not aware of similar libraries for other languages.)
I'm potentially interested in exploring a stack-based language like Forth (or Factor). What I'd like to see is how an application might be built from the ground up, step by step. The tutorials I've found are rudimentary and are not helping me to understand the bigger picture. It's confusing to think of how one might manage the stack when dealing with lots of parts.
I've always thought (maybe wrongly) that a good way to learn a language would be to use it to write a Roguelike game. I'm having trouble trying figure out how one would juggle a lot of things on a stack: a maze, dozens of creatures, treasures, character stats, etc.
In some sense, all languages are equivalent; you write a program by breaking a problem down into smaller pieces, then you code up those pieces and make them all work together. Forth has unusual syntax features, but it's still a programming language.
In fact, Forth places a great deal of power at your fingertips, much as Lisp does. In Lisp, using "macros", you can write your own control structures that are just as good as anything built-in; in Forth, you can do the same.
If you are interested in finding out more about Forth, I suggest you get the classic books Starting Forth and Thinking Forth by Leo Brodie.
Oh! Google just told me that both books are available free online now:
http://www.forth.com/starting-forth/
http://thinking-forth.sourceforge.net/
I'd point you to Factor rather than pure Forth; there are plenty of sample apps, GUIs, web apps, etc. If you're specifically interested in a web framework, look at Furnace.
Ultimately I don't understand the question; what does being stack-based have to do with getting anything done? Back in the old days it was the language of choice for embedded systems; I wrote everything from cereal boxing robots to calculators to... well, mostly everything.
I attempt to directly answer all of your questions at the bottom of this answer. But before that, here are 7 points or aspects of learning stack-based Forth, or generally about stack-based-languages, or ways to proceed with your goals:
1: Here is a key way to think about this: The stack is not the place where you keep your data -- just the way that you pass parameters from one function (or method) to another.
2: There is a "Basic Language" interpreter written in Forth. This would be one way to write an application in Forth -- write your own Basic interpreter (or type in one you find), then write the Roguelike game in the Basic language you just created.
3: If you go through the old "Forth Dimensions" magazine, several different object-oriented extensions to Forth were created and presented (including source code). Type one of those in, and use it to write your rogue-like game in one of those object-oriented ways. If you're still stuck, write it in your favorite C language -- C++, C#, or Java, and here's the key... write it as primitively as you can. Then take your own application apart -- each method, enum, constant, etc. constitutes part of a Domain Specific Language that you created in order to implement the game. Write down those parts, and figure out how to do it in Forth, ideally using only the idioms of Forth. You may have to work in Forth a while to get to that stage.
4: Learn how to program in assembly language, an assembly language for which a Forth exists that has an embedded assembler for the same microcontroller or CPU. This is where Forth really crushed the competition when computers were just becoming more widely available -- bringing a completely new microcontroller up, creating the BIOS and the OS. Once you've done this in pure assembly language, then go into the Forth that has the same assembly language and program the same application, but using the Forth assembler to build the building blocks (the DSL's) and then writing the BIOS and the OS in those DSL's. Forths happen to be both low-level and high-level at the same time, and the best-written applications are written in multiple DSL's, not in stack-based primitives. Once you've written in assembly language, when you try to learn Forth, you will find that it is easy -- the basic mechanisms are simple, but the combination of basic mechanisms is simple, powerful and elegant.
5: Look at either C# or Java, and realize that they are both based on Virtual Machines, which is a specific kind of DSL. But also realize that both Virtual Machines are stack machines, and the language that powers both of them are stack based languages. But if you program in Java, or if you program in C#, you really don't think at all about the stack foundation upon which you are running -- and that's a really good thing! You want to follow the same pattern. A. Realize what basic mechanisms / tools you need, B. Build the tools you need, C. Build your application with the tools you built. You may have to translate your own infix pseudocode to the postfix VM you've created. Do you need Object Orientation?--You can build it with Forth. Do you need multiprocessing?--You can build it. Do you need an Erlang kind of light multithreading?--You can build it. Do you need a Basic, a Pascal, a C#, a Java, a Lua?--You can build it. Build your Roguelike game in your favorite language, then perhaps a different language, then assembly language. Then build it in Forth.
6: At this link, there is a way to make a microprocessor that natively runs CIL (Common Intermediate Language -- the stack based language of both .NET and Mono, which used to be MSIL Microsoft Intermediate Language, before Microsoft Open Sourced the specification). You buy the IP logic block and then program it into an FPGA. Here is the Abstract of a paper that describes the object-oriented stack-based language and implementation of CIL:
Embedded systems and their applications are becoming ubiquitous and
transparent. Nowadays, the designers need to implement both hardware
and software as fast as they can to face the competition. Hence tools
and IPs became an important factor of the equation. In this paper, we
present a synthesizable core similar to the microarchitecture of
Tanenbaum's (2006) JVM processor. The core is an implementation of a
subset of Microsoft's CIL (Common Intermediate Language). We seek to
accelerate the development of embedded software by providing a
platform onto which the whole .NET framework (C#, Visual Basic.NET...)
(along with its object-oriented approach) could execute. We used a
Xilinx Virtex II Pro as the prototyping platform.
7: Go to the current work-place of Charles Moore, the father of Forth, GreenArrays, Inc. and look under Features for "Implements the colorForth instruction set", -- and you will find that he, too, has taken his virtual machine, having optimized Forth his entire life, and reduced it to an insanely fast 144 core microchip with a claimed capability of 100 BIPS. Here, you should be able to look at the "colorForth instruction set" and the description of it -- he has removed any fluff and bloat, and reduced it to an extremely spartan instruction set. We should learn from this genius before it is too late (he is advanced in years, and Covid19 currently threatens many of our patriarchs).
Below, I try to more directly handle each aspect of your question:
How would one code an application in Forth (or Factor)?
Same way as you would with any other application, learn the building blocks, and build the simplest thing that could possibly work, then grow it from there.
I'm potentially interested in exploring a stack-based language like Forth (or Factor).
There are a multitude of stack-based languages around. They seem to be the foundation, not only of most processors, but also of most virtual machines, as in Java and the JVM, and C# and the CIL.
What I'd like to see is how an application might be built from the ground up, step by step.
That's a great idea for a Forth book... Anybody game? I know there are a lot of Forth people out there, and because of Covid19, a lot of you are at home, too. What kink of application would you like to see? Just looking at FPC teaches a lot.
The tutorials I've found are rudimentary and are not helping me to understand the bigger picture.
There are higher level tutorials out there. But widen your search -- include JVM, CIL, and even LUA (because I think it has a VM, too). Write an app in Lisp. Then take the primitives you built in Lisp, and implement them in Forth, or Factor, or CIL, or JVM instructions.
It's confusing to think of how one might manage the stack when dealing with lots of parts.
That's why I said that you should learn an assembly language -- because most stack-based languages are a micro-step up from that. You have to learn the basic levers, springs, latches, and other primitive mechanisms out of which all of our software is ultimately made. But you only think in terms of stacks to make the primitive mechanisms -- once you have those made, in general, don't even touch the stack stuff -- think only in terms of the basic mechanisms, then build progressively higher layers, even progressing to object orientation, as exemplified in the CIL and JVM. The better applications are layers of implicit DSL's, each DSL targeted at a step up from the previous one. The layers are: Assembly-Language, CIL or other VM, BIOS, Operating-System, Library, Business-Language, Application. Most people only think of the Business-Language layer as a DSL, but every single layer is its own DSL, and really, every class, method, object, config-file, and specialized language, like HTML, CSS, JavaScript, etc. -- they are ALL DSL's, either explicitly (formally specified) or implicitly (created by the programmer as things are built.) Understanding gained from book "Code Complete" second edition -- read over and over is my suggestion.
I've always thought (maybe wrongly) that a good way to learn
a language would be to use it to write a Roguelike game.
Sure... but pick anything you like, really. But perhaps try getting an Arm-based Arduino, programming it, disassembling that, then writing your own assembly language -- then get a Forth for that Arm chip, and program it in that. Then implement your Roguelike game in the Arm-Forth.
I'm having trouble trying figure out how one would juggle a lot of
things on a stack: a maze, dozens of creatures, treasures, character
stats, etc.
Build your "Rogue" Domain-Specific-Language, make that into a VM, then implement the VM instructions first in a high level stack language, like CIL, then try to figure out how to do it in one of the low-level stack-languages. There are many flavors of object-orientation implemented in Forth -- spelunk the old Forth Dimensions articles
and some of the old Forth conference presentations to find them. Google, and the "Internet Archive", or Way-Back Machine, are your friends for this.
I'm new to Vala. I'm not familiar with GObject. As I understand it, GObject was spun off from the GLib project from GNOME. Correct me if I'm wrong.
I do like the syntax and implementation of Vala very much, yet it is not in my intentions to write desktop applications for GNOME.
I also know (think I know) that Vala does not have a standard library other than GObject itself.
So my question is: Can Vala be used without GObject and if it can, is it usable (are there optimal and maintained base libraries for common things like type conversions, math, string manipulation, buffers, etc... available)?
There is some other Vala profiles like Dova and Posix.
TLDR: I recommend using Vala with GLib/GObject, because it was designed on top of them.
While there may be alternative profiles for valac they are either unfinished or deprecated.
The whole point of Vala is to reduce the amount of boilerplate required to write GLib and Gtk+ applications in C.
It also adds some nice other improvements over C, like string and array being simple data types instead of error prone pointers.
It mostly wraps all the concepts present in GObject like:
classes
properties
inheritance
delegates
async methods
reference counting (which is manual in C + GObject, and automatic aka ARC in Vala)
type safety of objects
generics
probably much more ...
All of these concepts can be implemented without using GObject/GLib/Gio, but that would mean to basically rewrite GObject/GLib/Gio which doesn't make much sense.
If you don't want to write GUI applications GLib can be used to write console applications as well, using GIO or GTK+ is optional in Vala, applications work on a headless server as well.
I think that there is even some effort in Qt to eventually switch to the GLib main loop, which would make interoperability of Qt and GLib much easier.
A good example of a framework that uses GLib is GStreamer which is used across different desktop environments as well.
In summary:
GLib is a basic cross platform application framework
GObject is the object system used by the GLib ecosystem
GIO is an I/O abstraction (network, filesystem, etc.) based on GLib + GObject
GTK+ is a graphic UI toolkit based on GLib + GObject + GIO + others
GNOME is a desktop environment based on all the "G" technologies
Vala is a high level programming language designed to reduce the boiler plate neded to use the "G" libraries from the C language.
GTK+ originally came from GIMP and was since split into the different "G" libraries that are the basis for GNOME today.
Vala also has very powerful binding mechanisms to make it easy to write so called "VAPI" files for any kind of C library out there.
With the correct VAPI bindings you don't have to worry about manual memory management, pointers, zero termination of strings and arrays and some other tedious things that make writing correct C code so difficult.
Here is another profile that you can use Aroop. (Note it is still under heavy development). What I hope is it is good if you need high performance. Please check the features here.
Does anyone know if there are bindings for the gobject family of librares (glib, Gnome, clutter, etc) available for the D language that use the GIR introspection mechanism?
I would like to write an app targeting GTK 3 and Clutter. Unfortunately, I can't seem to find them. Given that GIR should allow bindings to be generated automatically for all libraries that have introspection information available, this surprises me. Does such a thing actually exist? (One big drawback of D is that it's practically impossible to search for on Google...)
(Incidentally, I do know about gtkD, but that only targets GTK 2 and doesn't do Clutter.)
So it would appear that there aren't any.
I am trying to build out a useful 3d game engine out of the Ogre3d rendering engine for mocking up some of the ideas i have come up with and have come to a bit of a crossroads. There are a number of scripting languages that are available and i was wondering if there were one or two that were vetted and had a proper following.
LUA and Squirrel seem to be the more vetted, but im open to any and all.
Optimally it would be best if there were a compiled form for the language for distribution and ease of loading.
One interesting option is stackless-python. This was used in the Eve-Online game.
The syntax is a matter of taste, Lua is like Javascript but with curly braces replaced with Pascal-like keywords. It has the nice syntactic feature that semicolons are never required but whitespace is still not significant, so you can even remove all line breaks and have it still work. As someone who started with C I'd say Python is the one with esoteric syntax compared to all the other languages.
LuaJIT is also around 10 times as fast as Python and the Lua interpreter is much much smaller (150kb or around 15k lines of C which you can actually read through and understand). You can let the user script your game without having to embed a massive language. On the other hand if you rip the parser part out of Lua it becomes even smaller.
The Python/C API manual is longer than the whole Lua manual (including the Lua/C API).
Another reason for Lua is the built-in support for coroutines (co-operative multitasking within the one OS thread). It allows one to have like 1000's of seemingly individual scripts running very fast alongside each other. Like one script per monster/weapon or so.
( Why do people write Lua in upper case so much on SO? It's "Lua" (see here). )
One more vote for Lua. Small, fast, easy to integrate, what's important for modern consoles - you can easily control its memory operations.
I'd go with Lua since writing bindings is extremely easy, the license is very friendly (MIT) and existing libraries also tend to be under said license. Scheme is also nice and easy to bind which is why it was chosen for the Gimp image editor for example. But Lua is simply great. World of Warcraft uses it, as a very high profile example. LuaJIT gives you native-compiled performance. It's less than an order of magnitude from pure C.
I wouldn't recommend LUA, it has a peculiar syntax so takes some time to get used to. Depending on who will be doing the scripting, this may not be a problem, but I would try to use something fairly accessible.
I would probably choose python. It normally compiles to bytecode, so you would need to embed the interpreter. However, if you must you can use PyPy to for example translate the code to C, and then compile it.
Embedding the interpreter is no issue. I am more interested in features and performance at this point in time. LUA and Squirrel are both interpreted, which is nice because one of the games i am building out is to include modifiable code, which has an editor in game.
I would love to hear about python, as i have seen its use within the battlefield series i believe.
python is also nice because it has actual OGRE bindings, just in case you need to modify something lower-level on the fly. I don't know of any equivalent bindings for Lua.
Since it's a C++ library, I would suggest either JavaScript or Squirrel, the latter being my personal favorite of the two for being even closer to C++, in particular to how it handles tables/structs and classes. It would be the easiest to get used to for a C++ coder because of all the similarities.
However, if you go with JavaScript and find an HTML5 version of Ogre3D, you should be able to port your game code directly into the web version with minimal (if any) changes necessary.
Both of these are a good choice, and both have their pros and cons, but both would definitely be the easiest to learn since you're likely already working in C++. If you're working with Java, the same may hold true, and if it's Game Maker, you wouldn't need either one unless you're trying to make an executable that people wouldn't need Game Maker itself to run, in which case, good luck finding an extension to run either of these.