What are the typical use cases of using an embeddable programming language? Do I understand it correctly that such language should be embedded into some program environment and should be able to be executed from there?
Since you tagged the question as "Lua", I'll give you an answer in the context of this language.
Introduction
Lua is written in C (almost completely compatible with C89 standard; the incompatible features can be easily disabled, if needed, using compile-time switches) and has been designed to be easily integrated with C code. In the the context of Lua, "integrated" means two different, but related, things:
You can easily write C code that can be used as a library by Lua code. The integration is achieved either by static or dynamic linking your C code to Lua engine's code. The linked library can then be referred to in your Lua code using the Lua require function.
Lua engine can be easily embedded in a C application, i.e. linked (again either statically or dynamically) to the C application code. Then the C application can interact with the Lua code using Lua's C application programming interface (Lua C-API).
Note: this can be done, with a little more effort, also with a C++ application.
Advantages of embedding a Lua engine
If your C application embeds Lua many, if not most, operations can be delegated to the Lua engine, i.e. either to code written using the C-API functions or, better yet, Lua code. Lua code could be embedded as C strings inside your C code or be stored as external Lua scripts.
Having part of your code logic implemented using Lua code has several advantages:
Lua is simpler (less tricky) to learn and use than C, and it is much more high-level. It supports powerful abstractions, such as function closures and object orientation (in a peculiar way, using Lua tables and metamethods).
Lua is a dynamic language: it requires no "off-line" compilation. You can modify the text of your Lua script and that's all you need to modify your application behavior (no additional compilation+linking steps needed). This simplifies application development and debugging.
Lua is a safer language than C: it is really difficult to write Lua code that exhibits undefined behavior, as intended in the context of C/C++. If a Lua script fails, it fails "loudly". Moreover Lua supports an exception mechanism (although with a different syntax than C++) which can be employed to implement error management in a much easier way compared to C.
Lua, as most dynamic languages, is garbage collected. This means that the programmer is spared the pain of manually managing dynamic memory, which is a major cause of bugs, leaks, instability and security loopholes in languages that lack garbage collection.
Lua can "eat its own dog food", i.e. you can build a string at runtime (even in Lua itself) and if it is valid Lua code, your program can execute it on the fly. This is something not frequently seen even in other dynamic languages (still it is not LISP, but it gets closer, and with much more readable syntax). This enables Lua scripts to:
employ powerful text-based metaprogramming techniques, where Lua code can generate other Lua code and execute it on the fly;
implement domain specific languages (DSLs) in an easy way; Lua code can load at runtime other Lua code that is crafted so as to reflect the specific problem domain in which it is used (Lua syntax is simple, yet flexible enough to allow such things);
be used as a configuration language with ease: your application (written in a mix of C and Lua) can use some lua files as configuration files without the need to craft an ad-hoc parser for a specific configuration file format. Therefore you don't need to parse *.properties, *.csv, *.ini, or whichever other format you would choose if you hadn't the option of using Lua files for that purpose.
Lua engine has a very small memory footprint (some hundreds kBs), packing powerful capabilities. With very few C code lines and a bunch of Lua files you could create a complete application that would require thousands of C code lines otherwise. The standard Lua standalone interpreter can be seen as just an example of embedding Lua in a C application!
Lua has a very liberal open-source license, which enables its use even in commercial applications without much hassle. This also allows the modification of its source code to adapt it to special needs.
Small memory footprint and easily tweakable C sources make Lua a perfect candidate for porting it on embedded systems or small microcomputer systems (microcontrollers, etc.). Many parts of the standard Lua distributions can be stripped off, reducing the core Lua engine in the ~100kB range. As an example, take the eLua project, a modified distribution of Lua designed for embedded devices.
Lua, and other scripting languages, provide various benefits that are dependant on your needs.
Provide rapid iteration of development.
Allow run-time code changes, such as reloading your UI in World of Warcraft which re-loads all scripts without stopping the game engine itself or logging you out.
Provide a distinct API for your application for users to extend, without exposing critical parts of your system to the public. Such as text editors providing a macro language to allow you to integrate custom behaviour without giving you unfettered access to the internals of the editor itself.
The uses are really quite extensive and depends on the developer.
Related
Hey so I was thinking about learning Lua and this question struck to me, that can I program a game which only has Lua code no C++,C, Or side-libraries of C, C++?
Lua is an embedded scripting language. Like most scripting languages, it runs inside of some application that actually executes what the language says for it to do. So if you want to get technical, it is impossible to run Lua without first running something involving "C", because that "something" is what makes Lua code possible to execute at all.
Because Lua is an embedded scripting language, what you can do with it is ultimately defined by the environment into which it is embedded. lua.exe is one embedded environment, but there are others.
So this question is probably intended to be this: Can I make a game in Lua written against the lua.exe environment, without adding supplemental, non-Lua modules?
Sure: so long as that game works within the limitations of the text console that lua.exe executes within. The standard Lua environment provides access to all of the Lua standard library, but that library is actually quite tiny. The only IO it provides is access to standard in, standard out, standard error, and files.
That's it.
You can make a text adventure game with that. With some knowledge of how to control the text console more directly (which is platform-specific), you can use the console to display ASCII-art graphics. So you could make an old-school Rogue-like, so long as timing isn't a factor, since the Lua standard library has no mechanism for doing that.
Yes, you can make a game of some kind. But the kinds of games you can make with it are probably not what you're thinking of when you said "game". The Lua standard library offers nothing that would allow you to do audio of any kind or anything visual except for sending text to standard out.
Because Lua standard library is intended for any application that needs to embed Lua, it is quite tiny, providing a small baseline of functionality and nothing more. It isn't intended for you to create graphics-intensive applications.
For that, you would need to explore a Lua environment that is designed for that, which provides functionality specific to those needs.
Of course, there are many games made only in Lua and others where you can program mods or servers inside them with Lua only like Roblox or FiveM. If you want to make an entire game using only Lua scripts, you can try using a proper engine for this as for example GameGuru
You can also try some Lua libraries made to program games like Love2D and CoronaSDK
I have been reading about libadalang 1 2 and I am very impressed by it. However, I was wondering if this technique has already been used and another language supports a library for syntactically and semantically analyzing its code. Is this a unique approach?
C and C++: libclang "The C Interface to Clang provides a relatively small API that exposes facilities for parsing source code into an abstract syntax tree (AST), loading already-parsed ASTs, traversing the AST, associating physical source locations with elements within the AST, and other facilities that support Clang-based development tools." (See libtooling for a C++ API)
Python: See the ast module in the Python Language Services section of the Python Library manual. (The other modules can be useful, as well.)
Javascript: The ongoing ESTree effort is attempting to standardize parsing services over different Javascript engines.
C# and Visual Basic: See the .NET Compiler Platform ("Roslyn").
I'm sure there are lots more; those ones just came off the top of my head.
For a practical and theoretical grounding, you should definitely (re)visit the classical textbook Structure and Interpretation of Computer Programs by Abelson & Sussman (1st edition 1985, 2nd edition 1996), which helped popularise the idea of Metacircular Interpretation -- that is, interpreting a computer program as a formal datastructure which can be interpreted (or otherwise analysed) programmatically.
You can see "libadalang" as ASIS Mark II. AdaCore seems to be attempting to rethink ASIS in a way that will support both what ASIS already can do, and more lightweight operations, where you don't require the source to compile, to provide an analysis of it.
Hopefully the final API will be nicer than that of ASIS.
So no, it is not a unique approach. It has already been done for Ada. (But I'm not aware of similar libraries for other languages.)
The fslex and fsyacc tools currently require 2-stage compilation, generating files that are then compiled by fsc. It seems to me that these tools would be much easier to use if the source files were embedded resources, fed to fslex and fsyacc programmatically and the generated code compiled on-the-fly using the CodeDom.
Is this feasible and, if so, what would be required to implement this?
Jon, this is a great question; in fact, one of the design goals I have for fsharp-tools (new lexer- and parser-generator implementations for F#) is for them to be embeddable, specifically to enable scenarios like this.
As of now, I haven't implemented (yet) the functionality which would let you do this easily in fsharplex, but don't let that deter you; I've written fsharplex (and the other tools in fsharp-tools) in a more-or-less purely-functional style, so there shouldn't be any issues with global state or anything like that. It should be relatively straightforward to hack up the compiler code so you can build a regex AST using some combinators, run the compiler to get a compiled DFA, then emit IL for your state machine into a dynamic assembly (which you could then "bake" and execute).
fsharpyacc currently uses an approach where I've put the bulk of the compilation logic into a purely-functional library, Graham; the idea there is that the grammar analysis/manipulation and parser DFA compilation algorithms should be generic, reusable, and easy to test, so anyone else wanting to build language tools with F# will have a common framework on which to build them. Likewise, contributions/improvements to Graham can easily flow back to fsharpyacc. Eventually, I will modify fsharplex to use this same approach, which will allow you to embed the regex compiler in your own code simply by referencing the NuGet package (you'd just need to write the code to generate IL from the DFA).
fsharplex and fsharpyacc use MEF to allow various backends to be plugged in; for now, they're only targetting fslex and fsyacc for compatibility reasons, but I'd like to implement code-based backends (as opposed to the current table-based backends) to get better performance in the future.
Update -- I just re-read your question and noticed you want to embed the *.fsl and *.fsy files themselves and invoke the respective compilers at run-time. You could accomplish this by compiling the tools and referencing the assemblies from your own projects. IIRC, I exposed an entry point in both compilers so they could be called from outside code; the main entry points (e.g., what gets executed when you invoke the tools from a console) simply parse the command-line arguments then pass them into this "external" entry point.
There is one problem with directly embedding the *.fsl and *.fsy files though; if you embed them, then run them through fsharplex and fsharpyacc at run-time, your user-defined actions (e.g., the code executed when a lexer or parser rule is matched) will still be specified as F# source code -- you'd need to decide how you want to compile them into executable code.
It should be feasible to provide a parser combinator-like interface with a backend that uses expression trees (the LISP "eval" of F#) or something similar, for full integration with the language. Or else a TypeProvider. There are many options. If table generation is an expensive computation, it could be cached by providing a Cache, for example a disk cache.
I think nothing except lack of time, dedication and expertise, prevents us from having tools with (non-monadic) parser combinator-like interface, yet efficient compiled implementation.
Sometimes I get back to this pet project of mine, playing with an algebraic approach to optimizing regular expressions (and lexers) specified in source using combinators and then compiled to a state machine. It still lacks a few key pieces for efficiency, but there it is:
https://github.com/toyvo/ocaml-regex-algebraic
For a research project i'm looking for an interpreter or even a parser for a programming language (doesn't matter what programming language) that has been ported to a number of languages. This probably means the code is small enough to do so.
I know Lisp-ish languages have been ported to a lot of environments, because Lisp is so easy to parse, however I haven't found a single implementation that has been multi targetted. For instance; it is very hard to find a version which works in PHP where the same code (the Lisp which runs on top / is parsed that is) would also work in Python.
Hope someone here can help...
What do I want to do with it? For a tool I'm making, the user group will write tiny pieces of logic; however the system underneath differs while the logic is the same. We don't want to force our users to learn Java, PHP, C#, etc just to write that logic.
The Lua (www.lua.org) scripting language can run from within C and has bindings to Python, php, Java, C#, probably some other languages too. It's a very small interpreter (something like 200k when compiled) because it comes "without batteries" - no builtin functions for some common operations like copying arrays. It should be pretty trivial to add support for embedding in another language, compared to other scripting languages, if need be.
I am trying to build out a useful 3d game engine out of the Ogre3d rendering engine for mocking up some of the ideas i have come up with and have come to a bit of a crossroads. There are a number of scripting languages that are available and i was wondering if there were one or two that were vetted and had a proper following.
LUA and Squirrel seem to be the more vetted, but im open to any and all.
Optimally it would be best if there were a compiled form for the language for distribution and ease of loading.
One interesting option is stackless-python. This was used in the Eve-Online game.
The syntax is a matter of taste, Lua is like Javascript but with curly braces replaced with Pascal-like keywords. It has the nice syntactic feature that semicolons are never required but whitespace is still not significant, so you can even remove all line breaks and have it still work. As someone who started with C I'd say Python is the one with esoteric syntax compared to all the other languages.
LuaJIT is also around 10 times as fast as Python and the Lua interpreter is much much smaller (150kb or around 15k lines of C which you can actually read through and understand). You can let the user script your game without having to embed a massive language. On the other hand if you rip the parser part out of Lua it becomes even smaller.
The Python/C API manual is longer than the whole Lua manual (including the Lua/C API).
Another reason for Lua is the built-in support for coroutines (co-operative multitasking within the one OS thread). It allows one to have like 1000's of seemingly individual scripts running very fast alongside each other. Like one script per monster/weapon or so.
( Why do people write Lua in upper case so much on SO? It's "Lua" (see here). )
One more vote for Lua. Small, fast, easy to integrate, what's important for modern consoles - you can easily control its memory operations.
I'd go with Lua since writing bindings is extremely easy, the license is very friendly (MIT) and existing libraries also tend to be under said license. Scheme is also nice and easy to bind which is why it was chosen for the Gimp image editor for example. But Lua is simply great. World of Warcraft uses it, as a very high profile example. LuaJIT gives you native-compiled performance. It's less than an order of magnitude from pure C.
I wouldn't recommend LUA, it has a peculiar syntax so takes some time to get used to. Depending on who will be doing the scripting, this may not be a problem, but I would try to use something fairly accessible.
I would probably choose python. It normally compiles to bytecode, so you would need to embed the interpreter. However, if you must you can use PyPy to for example translate the code to C, and then compile it.
Embedding the interpreter is no issue. I am more interested in features and performance at this point in time. LUA and Squirrel are both interpreted, which is nice because one of the games i am building out is to include modifiable code, which has an editor in game.
I would love to hear about python, as i have seen its use within the battlefield series i believe.
python is also nice because it has actual OGRE bindings, just in case you need to modify something lower-level on the fly. I don't know of any equivalent bindings for Lua.
Since it's a C++ library, I would suggest either JavaScript or Squirrel, the latter being my personal favorite of the two for being even closer to C++, in particular to how it handles tables/structs and classes. It would be the easiest to get used to for a C++ coder because of all the similarities.
However, if you go with JavaScript and find an HTML5 version of Ogre3D, you should be able to port your game code directly into the web version with minimal (if any) changes necessary.
Both of these are a good choice, and both have their pros and cons, but both would definitely be the easiest to learn since you're likely already working in C++. If you're working with Java, the same may hold true, and if it's Game Maker, you wouldn't need either one unless you're trying to make an executable that people wouldn't need Game Maker itself to run, in which case, good luck finding an extension to run either of these.