Folks, is there a way to clone a Lua state?
In my game application the initialization procedure of the Lua virtual machine is pretty heavy(about 1 sec, since many scripts are loaded at once). I have a separate Lua VM for each autonomous agent and once the agent is created its Lua initialization affects FPS pretty badly.
I'm thinking about the following schema: what about keeping "preforked" Lua state which is then simply cloned for each agent? Is it possible?
You want to consider using Lua's coroutines for each autonomous agent, instead of a completely separate VM. Coroutines are a more lightweight solution, but may or may not be suitable for your application.
If you can't change the architecture, you might try LuaJIT. It might make the initialization fast enough for your purposes.
More options:
Rings: "Rings is a library which provides a way to create new Lua states from within Lua. It also offers a simple way to communicate between the creator (master) and the created (slave) states."
Pluto: "Pluto is a library which allows users to write arbitrarily large portions of the "Lua universe" into a flat file, and later read them back into the same or a different Lua universe."
There's also Lanes (download, docs) and within the comparison to all similar products I know.
About Rings the comparison sheet says:
Rings offers separate Lua states, but
no multithreading. This makes it
simple, but it won't use more than one
CPU core.
Note: The comparison sheet says Lanes would only marshal 'non-cyclic tables'. It does do cycles, and does marshall functions, upvalues etc. And it does the copies between Lua states as direct copies, not needing to stringify the contents in the middle. This makes it fast.
If you're on Linux, you may try lper, LPSM-based experimental library by one of Lua authors.
Notice, works with Lua 5.2 and above
You can just restrict access to this VM. Create one instance with all functions required, that will not depend on _G (global Lua state) and then create separate table for each client. That they will use as their global namespace. Setting a table as current _G is possible via _ENV. That's quite difficult topic to explain in one post. In short you prepare "virtual" _G for your new client and then just replace _G for the client's code. There is where I advice you to start.
Here's the point.
local _ENV = t -- change the environment. without the local, this would change the environment for the entire chunk
Just remove local and you'll change _ENV for all further code. Good luck with experiments!
P. S. don't forget that you may set metatable for _ENV and _G tables and forbid changing that metatable. Lua is really flexible here.
Related
I'm working on a crowd simulator. The idea is people walking around a city in 2D. Think gray rectangles for the buildings and colored dots for the people. Now I want these people to be programmable by other people, without giving them access to the core back end.
I also don't want them to be able to use anything other than the methods I provide for them. Meaning no file access, internet access, RNG, nothing.
They will receive get events like "You have just been instructed to go to X" or "You have arrived at P" and such.
The script should then allow them to do things like move_forward or how_many_people_are_in_front_of me and such.
Now I have found out that Lua and python are both thousands of times slower than compiled languages (I figured it would be in order of magnitude of 10s times slower), which is way to slow for my simulation.
So heres my question: Is there a programming language that is FOSS, allows me to restrict system access (sandboxing) the entire language to limit the amount of information the script has by only allowing it to use my provided functions, that is reasonably fast, something like <10x slower than Java, where I can send events to objects inside that language with which I can load in new Classes/Objects on the fly.
Don't you think that if there was a scripting language faster than lua and python, then it'd be talked about at least as much as they are?
The speed of a scripting language is rather vague term. Scripting languages essentially are converted to a series of calls to functions written in fast compiled languages. But the functions are usually written to be general with lots of checks and fail-safes, rather than to be fast. For some problems, not a lot of redundant actions stacks up and the script translation results in essentially same machine code as the compiled program would have. For other problems, a person, knowledgeable about the language, might coerce it to translate to essentially same machine code. For other problems the price of convenience stay forever with the script.
If you look at the timings of benchmark tasks, you'll find that there's no consistent winner across them. For one task the language is fastest, for the other it is way behind.
It would make sense to gauge language speed at your task by looking at similar tasks in benchmarks. So, which of those problem maps the closest to yours? My guess would be: none.
Now, onto the question of user programs inside your program.
That's how script languages came to existence in the first place. You can read up on why such a language may be slow for example in SICP.
If you evaluate what you expect people to write in their programs, you might decide, that you don't need to give them whole programming language. Then you may give them a simple set of instructions they can use to describe a few branching decisions and value lookups. Then your own very performant program will construct an object that encompasses the described logic. This tric is described here and there.
However if you keep adding more and more complex commands for users to invoke, you'll just end up inventing your own language. At that point you'll likely wish you'd went with Lua from the very beginning.
That being said, I don't think the snippet below will run significantly different in compiled code, your own interpreter object, or any embedded scripting language:
if event = "You have just been instructed to go to X":
set_front_of_me(X) # call your function
n = how_many_people_are_in_front_of_me() #call to your function
if n > 3:
move_to_side() #call to function provided by you
else:
move_forward() #call to function provided by you
Now, if the users would need to do complex computer-sciency stuff, solve np-problems, do machine learning or other matrix multiplications, then yes, that would be slow, provided someone would actually trouble themselves with implementing that.
If you get to that point, it seem that there are at least some possibilities to sandbox the compiled dlls (at least in some languages). Or you could do compilation of users' code yourself to control the functionality they invoke and then plug it in as a library.
I am new to Lua and want to ask whether it is possible to restrict lua syntax in config file? I know that config loading have to be performed in jail, but how we can cope with while 1 do end in config file we want to load? Is there a way to allow only strings, assignments and tables in config and if not, then what is the best way to check that lua file doesn't contain undesirable constructs? Is manual pre-parsing the only solution?
You seem to already know about "sandboxing" in Lua. So what's left is as you say malicious constructs like infinite loops. And to solve that you need to solve the Halting Problem. Which is not practical.
Instead of "manually" parsing and hoping you find all the malicious content (you won't), how about just running your Lua interpreter with a timer set so that the script will be interrupted if it takes longer than N seconds?
If you want to explicitly forbid certain constructs in Lua, you have to actually scan the file yourself. Note that there are valid uses for those constructs, even in config files, so you are restricting what the user can do.
It wouldn't be too hard to write a simple Lua lexer that ignores the contents of strings and comments, but errors on any of the Lua keywords other than return. Given proper sandboxing (ie: no functions are available to be called), that should be sufficient to weed out anything malicious.
Also, note that Lua 5.1 doesn't make it easy to keep the parser from parsing non-text data (ie: compiled Lua bytecode). 5.2 offers specific API support for forcing the loader to only recognize text and therefore reject bytecode.
Many games these days make available some lua scripting, but this is universally undocumented.
So let's say I can get a game to run my lua script (it's lua 5.1) - and the script can write what it finds to text files on disk. How much can I discover about environment the script is executing in?
For example it seems I can list keys in tables, and find out what's a function and what's some other type of object, but there's no obvious way to guess how many arguments function takes (and a mistake usually results in crash to desktop).
Most languages provide some reflection functionality that could be used here - how much is possible in embedded lua environment?
"debug" standard library has some functions, which you may find useful:
debug.getfenv - Returns the environment of object.
debug.getinfo - Returns a table with information about a function.
... and more
Lua Reference Manual also states:
several of these functions violate some assumptions about Lua code (e.g., that variables local to a function cannot be accessed from outside or that userdata metatables cannot be changed by Lua code) and therefore can compromise otherwise secure code.
So with debug library, you may access more.
Unfortunately, there is not much you can learn about functions in Lua - they by design accept any number of parameters. Without the ability to look at the sources, your only resort is the documentation and/or other samples.
The most you can do in this case is traverse the entire _G table recursively and dump every table/function, printing the results to a file.
"A mistake usually results in crash to desktop" is a sign of a really bad design - good API should tell you, that it expects A, and you passed B. For example in Lqt, a Qt binding to Lua, we check every parameter against the original Qt API, so that the programmer is notified of mistakes:
> QApplication.setFont(1, 2)
QApplication::setFont(number, number): incorrect or extra arguments, expecting: QFont*,string,.
I have a c++ library which has functionality exposed to Lua, and am seeking opinions on the best ways to organise my lua code.
The library is a game engine, with a component based Game Object system. I want to be able to write some of these components as classes in Lua. I am using LuaBind, so I can do this but there are some implementation choices I must make, and would like to know how others have done it.
Should I have just one global lua_State, or one per object, one per scene, etc?
This sounds like a lot of memory overhead, but will keep everything nice and separate.
Should I have one GLOBALS table, or one per object, which can be put in place before a call to a member? This would seem to minimize the chances of some class deciding to use globals, and another accidentally overwriting it, with less memory overhead than having many lua_States.
Or should I just bung everything in the one globals table?
Another question involves the lua code ittself. Two strategies occur... Firstly shoving all class definitions in one place, loading them when the application launches, Secondly putting one class definition per file, and simply making sure that file is loaded when I need to instance it.
I'd appreciate anyone's thoughts on this, thanks.
While LuaBind is certainly very nifty and convenient, as your engine grows, so will your compile times, drastically.
If you already have, or are planning to add, a messaging system (which I heavily recommend, specially for networking), then it simplifies problems significantly. In this case, what you will need to do is simply bind a few key functions to interface with the messaging system. This will keep your compile times down, and give you a very flexible system.
Since you are doing a component based engine (Good choice BTW), It makes more sense to integrate scripting as an object component. This way, it usually makes more sense to make each scripting component a new coroutine that runs behavior for each particular object. You need not to worry about memory too much, Lua states are very light, and can be made really fast if you interface your memory manager with Lua.
If you implement scripting as a component, it is still a good idea to have global or per-level scripts loaded, (to coordinate event triggers by other objects, or maybe enemy spawning timers).
As far as loading scripts go, it would not be bad practice, to just load the scripts you will need for a level all at once, and keep them in a global table for fas accessing, loading of lua scripts is pretty fast, specially if you pre-compiled them.
One consideration is how you're planning to thread things. If you want to run the code for two Game Objects in parallel, for example, then they really ought to have their own separate lua_States so that they can both be running at the same time. (Of course, that also means that they can't really share any state, except via the C code where you'd need to be conscious of thread-safety.)
As to the Lua code, I'd recommend loading everything when the app launches (unless you really need to do "lazy" loading of your core classes on demand). It typically simplifies maintenance and debugging. And in the case of code being loaded that's no longer needed, the garbage collector will clean that up with a quickness. :-)
I am working on a game in C++. I've been told, though, that I should also use an embeddable scripting language like Lua or Angelscript, but to be honest, I have no idea how or why. What advantages would this bring me, over storing all of my data in some sort of text file? How do I get started? I tried to read some Lua examples, but I don't see how it works, or how exactly I am supposed to use it.
First the "why" question:
If you've made reasonable progress so far, you have game scenery where the action happens, and then a kind of GUI with your visible game controls: Maps, compass, hotkeys, chat box, whatever.
If you make the GUI (positions, sizes, settings, defaults, etc) configurable through a configuration file, that's OK for starters. But if you make it controllable by code then you can do many very cool things. Example: Minimize the map when entering a city. Show other player's portraits when in group. Update the map. Display different hot keys in combat. That kinda thing.
Now you can do your code-controlling of your GUI in C/C++ code, but one problem is that whenever you want to change the behavior, even if only a little, you need to recompile the whole dang game client. If you have a billion players, you have to ship them all a new game client. That's a turn-off. Another problem is that there's no way on earth that a player can customize the GUI.
A simple embedded language solves both problem. You can put that kind of code in separate files that get loaded at runtime and can be fiddled with to anyone's heart's content. If you want to update the GUI in some minor way, you can deliver updates of the GUI code separately from the game proper.
As for the how:
The simplest thing to do is to call a (e.g.) Lua "main" routine once for every frame, perhaps passing a bunch of parameters with the latest updatable information, and let that main routine call other functions to do whatever's needed. The thing to be aware of is that your embedded code only gets control for a short time, namely the time between two screen refreshes; so it does a little updating and painting, then it exits again and returns control to your C/C++ main program loop.
Technically, embedding a Lua interpreter in your program is pretty easy. The Lua interpreter has C source code, or there's pre-compiled libraries (DLLs) for Windows. Just link them into your program, initialize once, call the entry point on every iteration of the main frame loop, done.
Scripts are more powerful than storing all of your data in text files. You can assign arbitrary behavior, construct data from other data (e.g., orc captains are orcs with a bit more), and so on.
Scripts allow for faster development and easier maintenance than C++. No compile / edit / link cycle, you can even tweak the scripts while the game is running, and they're easier to update on end users' machines.
As far as the how, one suggestion would be to see how other games do it. For example, TOME, a Roguelike RPG written in C, uses Lua extensively.
For some inspiration, check out the Alternate Hard and Soft Layers pattern described on the C2 wiki.
As for my two cents, why embed a scripting language? Some reasons that I've experienced include,
REPL
easy string manipulation tools
leverage the power of loops, macros, and recursion within your data set
create dynamically generated content
wrappers to fetch content from the web
logic to provide default values if data is missing
unit tests written at the data set level