What I am trying to do to learn some lua/moonscript is migrate my awesome configuration file (rc.lua) to moonscript and unit-test a few things along the way. For this I set up rc.lua to require the moonscript config file like this
package.path = pathsToAdd .. package.path
-- a bit of a hassle to amend the lua require paths
-- correctly; I boldly assume for now that these are not the
-- cause of the problem
require('moonscript')
require('config')
For a first unit test to check if my config calls a specific function of the module 'gears' all went reasonably well. I ended up mocking the gears module of every subsequent call to
require('gears')
by setting up the unit test like so
package.loaded.gears = myMockVersion
fast forward to when my config file under test needs to require the 'awful' module:
its init.lua is called, immediately executing
return
{
client = require("awful.client");
...
}
which leads to client.lua doing
...
local tag = require("awful.tag")
...
local client = {}
-- define lots of functions, register some signal handlers
return client
and now, for everyone still reading, the problem in tag.lua:
...
local capi =
{
...
client = client,
...
}
...
capi.client.connect_signal(...)
That last call throws a good old
attempt to index a nil value (field 'client')
Which I assume is because client.lua did not yet run past the first few require calls and therefore is not available globally at all or, at least, did not define its functionality yet.
Which leads me, at last, to the question:
Why does this even run during your everyday awesome startup (awful is pretty much the core module) in the first place and what do I miss when trying to replicate the environment in which it does.
Thank your very much in advance.
Yours truly
The C core of awesome exports some objects for lua to use. Awful (and lots of others) use these directly. These are in awesome 3.5 (see https://awesome.naquadah.org/doc/api/):
tag
timer
drawin
keygrabber
drawable
root
mouse
client
screen
awesome
mousegrabber
selection
key
dbus
button
Most of these have wrappers in awful which add useful stuff (e.g. key vs awful.key, same for tag, keygrabber, button). Other stuff is completely hidden from "the average user" (e.g. drawin, drawable).
You should be able to mock these as well, but you will have to set global variables with the same name.
Edit: by the way, this is why you cannot require("awful") in a normal lua promt. The same built-in objects are missing.
Related
In python, there is a common construction of the form if __name__ == "__main__": to detect whether the file was imported or executed directly. Usually, the only action taken in this conditional is to execute some "sensible, top level" function. This allows the same file to be used as a basic script and as a library module (and also as something an interactive user can import and use).
I was wondering if there is a clean and reliable way to do this in lua. I thought I could use the _REQUIREDNAME global variable, but it turns out that this was changed in Lua 5.1. Currently, the lua require passes arguments (in the variadic ...), so in principle, these can be examined. However, this is either not reliable, not clean, or probably both, because obviously when a script is executed arguments can be passed. So to do this safely, you would have to examine the arguments.
FWIW, require passes the module name as argument 1 (the string you called require on), and the path to the file it eventually found as argument 2. So there is a obviously some examination that can be done to try to detect this, which if not nearly as nice as if __name__ == "__main__": and can always be bypassed by a user by passing two suitably constructed arguments to the script. Not exactly a security threat, but I would hope there is a better solution.
I also experimented with another method, which I found very ugly but promising. This was to use debug.traceack(). If the script is executed directly, the traceback is very predictable, in fact, it only has 3 lines. I thought this might be it, although, like I said, an ugly hack for sure.
Do any more frequent lua users have advice? In effect, if I am writing module X, I want to either return X.main_func() in script mode or return X in import mode.
EDIT:
I took out an item which was actually incorrect (and makes my traceback solution workable). Additionally, the link provided in the comment by Egor Skriptunoff did provide another trick from the debug library which is even cleaner than using the traceback. Other than that, it seems that everyone ran into the same issues as me and the lua team has been disinterested in providing an official means to support this.
Based on the links provided by Egor, the current cleanest and safest way to do this seems to be as outlined here:
How to determine whether my code is running in a lua module?
which I repeat for ease of reference:
if pcall(debug.getlocal, 4, 1) then
print("in package")
else
print("in main script")
end
There is a whole thread about it here:
http://lua.2524044.n2.nabble.com/Modules-with-standalone-main-program-td7681497.html
Like I said, it seems this is a popular feature which is going to remain unsupported for the time being, but the debug.getlocal method seems to be what common lua developers have settled on for now.
A require() returning in package.loaded.
So simply check package.loaded for what you want.
Also as a good start for more experience in that methodic i suggest to write package.preload functions for require() stuff without manipulating the path.
A simple example for showing this...
# /usr/local/bin/lua -i
Lua 5.4.3 Copyright (C) 1994-2021 Lua.org, PUC-Rio
> package.preload.mymod=function() return {dump=function(...)
>> local args={...}
>> local test,dump=pcall(assert,args[1])
>> if test then
>> for key,value in pairs(dump) do
>> io.write(string.format("%s=%s\n",key,value)):flush()
>> end
>> return true
>> else
>> return test,dump
>> end
>> end} end
> mymod=require('mymod')
> mymod.dump(package.loaded)
string=table: 0x56693590
mymod=table: 0x566aa890
utf8=table: 0x56694d80
package=table: 0x56691ed0
math=table: 0x56693cb0
table=table: 0x566920d0
_G=table: 0x56690730
debug=table: 0x566950e0
coroutine=table: 0x56692310
os=table: 0x56692e60
io=table: 0x566921b0
true
Take a look at this code:
local urgent = {}
local capi =
{
client = client,
}
local client
do
client = setmetatable({}, {
__index = function(_, k)
client = require("awful.client")
return client[k]
end,
__newindex = error -- Just to be sure in case anything ever does this
})
end
I'm having trouble understanding what it does. It's from the awesome-wm project. These are the things I'm having trouble understanding:
client = client in the declaration of capi
setmetatable stuff inside do-end
client = client in the declaration of capi
This is defining what portion of the capi is available in this file's scope, If you look at the client.lua file you will see that the capi defined in it has client, mouse, screen, and awesome.
For each item defined in the capi table there is a corresponding .c file. These files define objects such as client. urgent.lua has visibility of that object, likely it is a global variable, that is how we can set client = client the second client refers to the global variable.
Here is an example of 2 files:
main.lua
bar = "Hello World!"
local foo = require('foo')
print(foo.bar)
foo.lua
local foo = {
bar = bar
}
return foo
The print function in main.lua will result in Hello World!
setmetatable stuff inside do-end
Here by warping the setmetatable in a do-end block the code is executing in a restricted scope. This is normally done to contain the block's local variables so that they do not persist after the code's execution.
That said that is not the purpose of this block as the block has no local variables. As I see it, the blocking is simply to show that the object being modified is the local variable of client and not the global variable of client.
Additionally the metatable here is used to prevent circular dependency loops, this is mentioned comments in some of the places where similar code appears in the project, such as client.lua where local screen is defined.
#Nifim answer is excellent. I just want to add more context on why this code exist in its proper historical context. Before Lua 5.2, the module system was different. There was a magic module() function defined in the core Lua library. When you made a module, you had to first make local version of all global variables before calling module() because otherwise it would run in its own global environment. "capi" stands for "Core API" or "C (language) API" depending on the weather. If Awesome was written today with all the knowledge we now have, there would not be a public "C language" API and they would always be hidden in the private section to increase flexibility. Right now setting "c.my_own_property" do a couple round trips between capi.client and awful.client just to accommodate all the legacy constraints.
Now, the metatable magic is a Lua pattern called meta-lazy-loading. Because the urgent is a submodule of awful.client, it cannot directly import awful.client without causing a circular dependency. Over time, as Awesome APIs became better defined, more and more refactoring were made and they often introduced weird dependencies to maintain some degree of backward compatibility. In the best universe, we would have disregarded all users config and just re-engineered the whole code to avoid these circular dependencies. However every time we do that all users of the said APIs wake up one morning and they cannot login into their computer anymore. So this kind of workaround exist to prevent such events in return for some weird code and maintenance burden.
I'm using Rhino's context.evaluateString() to run some simple JavaScript from inside of Java. It's textbook right out of the Embedding Javascript guide:
String script = // simple logic
Context c = new ContextFactory().enterContext();
ScriptableObject scope = context.initStandardObjects();
Object o = context.evaluateString(scope, script, "myScript", 1, null);
ScriptableObject result = Context.jsToJava(o, ScriptableObject.class);
I'm not sure this is the current best-practice, because the main Rhino docs appear to be down, but it's working so far.
I'd like to be able to refer to a library in the working directory -- I see that Rhino shell supports load but I don't think this works in the embedding engine.
Is this possible? Is it documented anywhere? Ideally, I'd like to be able to just call something like load('other.js') and have it search directories I specify as a global property.
I have a sort-of answer that I don't really like, not least because it exposes what I'm pretty sure is a Rhino bug that drove me crazy for the last half hour:
eval("" + Packages.org.apache.commons.io.FileUtils.readFileToString(
new java.io.File("/the/local/root", "script.js");
));
{ That "" + ... is how I work around the bug -- if you eval() a Java String (such as is returned from the readFileToString call) without manually coercing it to a JavaScript native string, nothing appears to happen. The call just silently fails. }
This blindly reads an arbitrary file and evals it -- of course, this is what you do when you eval() from the Java side, so I don't worry about it too much.
Anyway, it's not elegant for a number of reasons, but it works. I'd love to hear a better answer!
I've heard mumblings around the internets about being able to hot-swap code in Lua similar to how it's done in Java, Erlang, Lisp, etc. However, 30 minutes of googling for it has turned up nothing. Has anyone read anything substantial about this? Anyone have any experience doing it? Does it work in LuaJIT or only in the reference VM?
I'm more interested in the technique as a shortcut in development/debugging than an upgrade path in a live environment.
Lua, and most scripting languages for that matter, do not support the most generalized form of "hot swapping" as you define it. That is, you cannot guaranteeably change a file on disk and have any changes in it propagate itself into an executing program.
However, Lua, and most scripting languages for that matter, are perfectly capable of controlled forms of hot swapping. Global functions are global functions. Modules simply load global functions (if you use them that way). So if a module loads global functions, you can reload the module again if it is changed, and those global function references will change to the newly loaded functions.
However, Lua, and most scripting languages for that matter, makes no guarantees about this. All that's happening is the changing of global state data. If someone copied an old function into a local variable, they can still access it. If your module uses local state data, the new version of the module cannot access the old module's state. If a module creates some kind of object that has member functions, unless those members are fetched from globals, these objects will always refer to the old functions, not the new ones. And so forth.
Also, Lua is not thread safe; you can't just interrupt a lua_State at some point and try to load a module again. So you would have to set up some specific point in time for it to check stuff out and reload changed files.
So you can do it, but it isn't "supported" in the sense that it can just happen. You have to work for it, and you have to be careful about how you write things and what you put in local vs. global functions.
As Nicol said, the language itself doesn't do it for you.
If you want to implement something like this yourself though, it's not that hard, the only thing "preventing" you is any "leftover" references (which will still point to the old code) and the fact require caches its return value in package.loaded.
The way I'd do it is by dividing your code into 3 modules:
the reloading logic at entry point (main.lua)
any data you want to preserve across reloads (data.lua)
the actual code to reload (payload.lua), making sure you don't keep any references to that (which is sometimes not possible when you e.g. have to give callbacks to some library; see below).
-- main.lua:
local PL = require("payload")
local D = require("data")
function reload(module)
package.loaded[module]=nil -- this makes `require` forget about its cache
return require(module)
end
PL.setX(5)
PL.setY(10)
PL.printX()
PL.printY()
-- .... somehow detect you want to reload:
print "reloading"
PL = reload("payload") -- make sure you don't keep references to PL elsewhere, e.g. as a function upvalue!
PL.printX()
PL.printY()
-- data.lua:
return {} -- this is a pretty dumb module, it's literally just a table stored in `package.loaded.data` to make sure everyone gets the same instance when requiring it.
-- payload.lua:
local D = require("data")
local y = 0
return {
setX = function(nx) D.x = nx end, -- using the data module is preserved
setY = function(ny) y = ny end, -- using a local is reset upon reload
printX = function() print("x:",D.x) end,
printY = function() print("y:", y) end
}
output:
x: 5
y: 10
reloading
x: 5
y: 0
you could flesh out that logic a bit better by having a "registry module" that keeps track of all the requiring/reloading for you and abstracts away any access into modules (thus allowing you to replace the references), and, using the __index metatable on that registry you could make it pretty much transparent without having to call ugly getters all over the place. this also means you can supply "one liner" callbacks that then actually just tail-call through the registry, if any 3rd party library needs that.
I'm lead dev for Bitfighter, and we're using Lua as a scripting language to allow players to program their own custom robot ships.
In Lua, you need not declare variables, and all variables default to global scope, unless declared otherwise. This leads to some problems. Take the following snippet, for example:
loc = bot:getLoc()
items = bot:findItems(ShipType) -- Find a Ship
minDist = 999999
found = false
for indx, item in ipairs(items) do
local d = loc:distSquared(item:getLoc())
if(d < minDist) then
closestItem = item
minDist = d
end
end
if(closestItem != nil) then
firingAngle = getFiringSolution(closestItem)
end
In this snippet, if findItems() returns no candidates, then closestItem will still refer to whatever ship it found the last time around, and in the intervening time, that ship could have been killed. If the ship is killed, it no longer exists, and getFiringSolution() will fail.
Did you spot the problem? Well, neither will my users. It's subtle, but with dramatic effect.
One solution would be to require that all variables be declared, and for all variables to default to local scope. While that change would not make it impossible for programmers to refer to objects that no longer exist, it would make it more difficult to do so inadvertently.
Is there any way to tell Lua to default all vars to local scope, and/or to require that they be declared? I know some other languages (e.g. Perl) have this option available.
Thanks!
Lots of good answers here, thanks!
I've decided to go with a slightly modified version of the Lua 'strict' module. This seems to get me where I want to go, and I'll hack it a little to improve the messages and make them more appropriate for my particular context.
There is no option to set this behavior, but there is a module 'strict' provided with the standard installation, which does exactly that (by modifying the meta-tables).
Usage:
require 'strict'
For more in-depth info and other solutions: http://lua-users.org/wiki/DetectingUndefinedVariables, but I recommend 'strict'.
Sorta.
In Lua, globals notionally live in the globals table _G (the reality is a bit more complex, but from the Lua side there's no way to tell AFAIK). As with all other Lua tables, it's possible to attach a __newindex metatable to _G that controls how variables are added to it. Let this __newindex handler do whatever you want to do when someone creates a global: throw an error, permit it but print a warning, etc.
To meddle with _G, it's simplest and cleanest to use setfenv. See the documentation.
"Local by default is wrong". Please see
http://lua-users.org/wiki/LocalByDefault
http://lua-users.org/wiki/LuaScopingDiscussion
You need to use some kind of global environment protection. There are some static tools to do that (not too mature), but the most common solution is to use runtime protection, based on __index and __newindex in _G's metatable.
Shameles plug: this page may also be useful:
http://code.google.com/p/lua-alchemy/wiki/LuaGlobalEnvironmentProtection
Note that while it discusses Lua embedded into swf, the described technique (see sources) do work for generic Lua. We use something along these lines in our production code at work.
Actually, the extra global variable with the stale reference to the ship will be sufficient to keep the GC from discarding the object. So it could be detected at run time by noticing that the ship is now "dead" and refusing to do anything with it. It still isn't the right ship, but at least you don't crash.
One thing you can do is to keep user scripts in a sandbox, probably a sandbox per script. With the right manipulation of either the sandbox's environment table or its metatable, you can arrange to discard all or most global variables from the sandbox before (or just after) calling the user's code.
Cleaning up the sandbox after calls would have the advantage of discarding extra references to things that shouldn't hang around. This could be done by keeping a whitelist of fields that are allowed to remain in the environment, and deleting all the rest.
For example, the following implements a sandboxed call to a user-supplied function with an environment containing only white-listed names behind a fresh scratch table supplied for each call.
-- table of globals that will available to user scripts
local user_G = {
print=_G.print,
math=_G.math,
-- ...
}
-- metatable for user sandbox
local env_mt = { __index=user_G }
-- call the function in a sandbox with an environment in which new global
-- variables can be created and modified but they will be discarded when the
-- user code completes.
function doUserCode(user_code, ...)
local env = setmetatable({}, env_mt) -- create a fresh user environment with RO globals
setfenv(user_code, env) -- hang it on the user code
local results = {pcall(user_code, ...)}
setfenv(user_code,{})
return unpack(results)
end
This could be extended to make the global table read-only by pushing it back behind one more metatable access if you wanted.
Note that a complete sandbox solution would also consider what to do about user code that accidentally (or maliciously) executes an infinite (or merely very long) loop or other operation. General solutions for this are an occasional topic of discussion on the Lua list, but good solutions are difficult.