I'm writing some logic for redis inside lua and almost each of my scripts have something common, it would be really handy to move out this to the shared function but
redis can't use lua's require statement
officially you can't call other redis function(see: https://stackoverflow.com/a/22599862/1812225)
For example I have this snippet literally everywhere
local prefix = "/" .. type
if typeId then
prefix = prefix .. "(" .. typeId .. ")"
end
I'm thinking about some post-processing before feeding scripts to redis but this seems like an over-kill...
What is the best practice to solve/reduce this problem?
Updated:
local registryKey = "/counters/set-" .. type
local updatedKey = "/counters/updated/set-" .. type
if typeId then
redis.call("SAdd", updatedKey, name .. ":" .. typeId)
redis.call("SAdd", registryKey, name .. ":" .. typeId)
else
redis.call("SAdd", updatedKey, name)
redis.call("SAdd", registryKey, name)
end
is another code sample and it can't be trivially moved to client-side as it invokes redis commands, and works as a part of transaction
Thanks!
"Hack" #1
After you SCRIPT LOAD something, you get back a sha1 hash that you can use with EVALSHA. The same sha1 value can be used to call that script from inside another script - simply call the function f_<sha1>. That said, there are some differences in how you pass the KEYS/ARGV structures when used that way.
Note that this is undocumented behavior, which means the behavior could change in a future version of Redis.
Credit for teaching me this goes to Dr. Josiah Carlson who, in turn, credits someone else (IIRC Fritzy). For more information check out his lua-call Python wrapper: https://github.com/josiahcarlson/lua-call
"Hack" #2
Redis sandboxes Lua and puts several restrictions on it in order to maintain sanity. You could go around some of these, e.g. access _G and define your utility function there so it will be available to all scripts (like I did with https://github.com/redislabs/redis-lua-debugger).
However, this is also pretty risky - besides potential replication issues, this usage is untested and could therefore lead to undefined behavior (I managed to crash quite a few instances with my little script ;)).
P.S.
Both hacks require additional administrative work to ensure that these "global" scripts are actually loaded before any other script calls them.
Related
I am trying to redocument the mod library for a game called 'harvest massive encounter'
Their documentation that I was able to find:
http://www.oxeyegames.com/wiki/index.php/Harvest_Library
Redocumenting everything they have documented isn't an issue, I've also found a way to discover the hooks they did not document. But I am unable to figure out a way to discover their undocumented functions.
For example: harvest.defineActionButton seems like something that I would really want to discover to make cool actions on buildings available. Where defineUpgradeButton is also a button on a building, but also replaces it.
Sadly, this button is not documented but it does exist. If I do harvest.print(type(harvest.defineActionButton)) I get "function"
Sadly this game came out in 2007 and only supports Lua 5.1, so debug.getinfo does not give me nparams as I've read online that might have helped:
code:
local function onDebug()
harvest.print(_VERSION)
harvest.print(type(harvest.defineActionButton))
local temp = debug.getinfo(harvest.defineActionButton)
for key, value in pairs(temp) do
harvest.print("key: " .. tostring(key))
harvest.print("value: " .. tostring(value))
end
end
hook.add("textInput", onDebug)
Any tips on how I can get the number of parameters on this function and maybe their name? (I would assume that expected type is impossible)
I have also attempted solutions found in:
How to get name of the argument in lua?
lua - get the list of parameter names of a function, from outside the function
But I am unable to make those solutions working
If the function was written in Lua, you can dump and analyze its bytecode with string.dump(f), which should include parameter info. If it's a C function, that's not possible (aside from statically analyzing the binary itself, which is a different category of question).
If other attempts fail, you could try redefining the function with a fake one, like this...
--...
local defineActionButton_real = harvest.defineActionButton
harvest.defineActionButton = function(...)
print(...)
return defineActionButton_real(...)
end
...then observing the output when the game calls the fake function.
I've read on the Lua wiki / here / etc. on how to sandbox lua code generally. But I haven't been able to find something that disallows function creation. For example, the example here provides a sample code as:
assert(run [[function f(x) return x^2 end; t={2}; t[1]=f(t[1])]])
And that's with an empty environment. But I want to eliminate the ability to create a function (the 1st part of the code) - e.g., just allow expressions. Any idea on how to do that? Does it have to be in C somehow? Thanks in advance!
If you want to evaluate expressions only, you could try this:
function run(s) return loadstring("return "..s)() end
(error handling omitted)
This simple solution will prevent most `attacks', but not eliminate them because one can say
(function () f=function(x) print"hello" end end)()
which defines a new function named f.
Your best bet is to use a sandbox and not worry about what the user does to the environment, because it'll not be your environment.
You can try detecting the creation of functions by looking for the string "function" before allowing the execution of the lua script. For example from your C/C++ backend.
If "function" appears throw a "you are not allowed to create functions" error and don't execute the code.
A couple notes:
You might want to try to customize the detection a bit more - only throw errors if you detect function followed by blanks and an opening parenthesis, for example. I'm leaving that as an exercise.
You should be aware that there are some standard lua functions that kindof expect the users to be able to create functions - for example, the string table has several of those. Without creating functions, it'll be very difficult for your users to work with strings (it is already difficult enough with functions...)
I'm lead dev for Bitfighter, and we're using Lua as a scripting language to allow players to program their own custom robot ships.
In Lua, you need not declare variables, and all variables default to global scope, unless declared otherwise. This leads to some problems. Take the following snippet, for example:
loc = bot:getLoc()
items = bot:findItems(ShipType) -- Find a Ship
minDist = 999999
found = false
for indx, item in ipairs(items) do
local d = loc:distSquared(item:getLoc())
if(d < minDist) then
closestItem = item
minDist = d
end
end
if(closestItem != nil) then
firingAngle = getFiringSolution(closestItem)
end
In this snippet, if findItems() returns no candidates, then closestItem will still refer to whatever ship it found the last time around, and in the intervening time, that ship could have been killed. If the ship is killed, it no longer exists, and getFiringSolution() will fail.
Did you spot the problem? Well, neither will my users. It's subtle, but with dramatic effect.
One solution would be to require that all variables be declared, and for all variables to default to local scope. While that change would not make it impossible for programmers to refer to objects that no longer exist, it would make it more difficult to do so inadvertently.
Is there any way to tell Lua to default all vars to local scope, and/or to require that they be declared? I know some other languages (e.g. Perl) have this option available.
Thanks!
Lots of good answers here, thanks!
I've decided to go with a slightly modified version of the Lua 'strict' module. This seems to get me where I want to go, and I'll hack it a little to improve the messages and make them more appropriate for my particular context.
There is no option to set this behavior, but there is a module 'strict' provided with the standard installation, which does exactly that (by modifying the meta-tables).
Usage:
require 'strict'
For more in-depth info and other solutions: http://lua-users.org/wiki/DetectingUndefinedVariables, but I recommend 'strict'.
Sorta.
In Lua, globals notionally live in the globals table _G (the reality is a bit more complex, but from the Lua side there's no way to tell AFAIK). As with all other Lua tables, it's possible to attach a __newindex metatable to _G that controls how variables are added to it. Let this __newindex handler do whatever you want to do when someone creates a global: throw an error, permit it but print a warning, etc.
To meddle with _G, it's simplest and cleanest to use setfenv. See the documentation.
"Local by default is wrong". Please see
http://lua-users.org/wiki/LocalByDefault
http://lua-users.org/wiki/LuaScopingDiscussion
You need to use some kind of global environment protection. There are some static tools to do that (not too mature), but the most common solution is to use runtime protection, based on __index and __newindex in _G's metatable.
Shameles plug: this page may also be useful:
http://code.google.com/p/lua-alchemy/wiki/LuaGlobalEnvironmentProtection
Note that while it discusses Lua embedded into swf, the described technique (see sources) do work for generic Lua. We use something along these lines in our production code at work.
Actually, the extra global variable with the stale reference to the ship will be sufficient to keep the GC from discarding the object. So it could be detected at run time by noticing that the ship is now "dead" and refusing to do anything with it. It still isn't the right ship, but at least you don't crash.
One thing you can do is to keep user scripts in a sandbox, probably a sandbox per script. With the right manipulation of either the sandbox's environment table or its metatable, you can arrange to discard all or most global variables from the sandbox before (or just after) calling the user's code.
Cleaning up the sandbox after calls would have the advantage of discarding extra references to things that shouldn't hang around. This could be done by keeping a whitelist of fields that are allowed to remain in the environment, and deleting all the rest.
For example, the following implements a sandboxed call to a user-supplied function with an environment containing only white-listed names behind a fresh scratch table supplied for each call.
-- table of globals that will available to user scripts
local user_G = {
print=_G.print,
math=_G.math,
-- ...
}
-- metatable for user sandbox
local env_mt = { __index=user_G }
-- call the function in a sandbox with an environment in which new global
-- variables can be created and modified but they will be discarded when the
-- user code completes.
function doUserCode(user_code, ...)
local env = setmetatable({}, env_mt) -- create a fresh user environment with RO globals
setfenv(user_code, env) -- hang it on the user code
local results = {pcall(user_code, ...)}
setfenv(user_code,{})
return unpack(results)
end
This could be extended to make the global table read-only by pushing it back behind one more metatable access if you wanted.
Note that a complete sandbox solution would also consider what to do about user code that accidentally (or maliciously) executes an infinite (or merely very long) loop or other operation. General solutions for this are an occasional topic of discussion on the Lua list, but good solutions are difficult.
I have an application that uses lua files for some of its more obscure configuration options. As such it mostly contains calls into the app to create things and alter properties; most C functions don't have a return value but some do.
I now have a need to read these same configuration files into a different application, and perform significantly different things when the functions are called (so I can't use common code). In addition, I'm only interested in a subset of the possible functions, and I think I can get away with by default ignoring (and/or returning nil) any other function call.
So I'm wondering what the best approach is here. How (from a C++ app), can I load and execute a lua script such that expressions etc are evaluated as normal but I can intercept and process certain app-defined C functions while simply ignoring (returning nil if required) calls to any other C functions?
(Note: I do have access to the vocabulary of the original app, which mostly uses luabind; I could just use the same definitions and change the implementation, but that's too fragile since the original app can have more functions added to it later. I would like something more generic.)
The goal is to get a bit of C code which I can use as a generic placeholder; the end result being "anything that's defined (standard library routines, functions defined in Lua, and C functions explicitly registered), call it as normal; for anything else, call one specific routine that simply does nothing, instead of raising an error". And preferably something compatible with luabind.
The whole process is initiated by a bit of C code that sets up the Lua environment, loads a set of files, calls one function, and then destroys the environment. There won't be anything ongoing.
Set a __call metamethod for nil:
debug.setmetatable(nil, { __call=function () end })
The _index metamethod for _G or other tables does not work because the name is resolved before the call.
(You don't need to use the debug API or library if you're setting this from C.)
How about using setfenv and a metatable? You can replace global environment table of certain function with an empty table. And then, set the placeholder function to ignore C-defined function.
local env = {} -- empty environment
local metatbl = {}
function metatbl.__index (tbl, key) -- provides placeholder function
return function()
print(key .. " called")
return(nil)
end
end
setmetatable(env, metatbl)
function samplefunc() -- your Lua code goes here
globalfunction "xyz" -- calls placeholder function
end
setfenv(samplefunc, env)
samplefunc()
If you want to use build-in function such as print, you can push it into env talbe like:
local env = {print = print}
If you have some way of differentiating the C functions from the ordinary Lua functions in your script, you can iterate over all functions defined in your Lua system, and for each C function that isn't on the list of functions you care about, replace that function's implementation with a simple nil-returning one. This is really easy if all the c-binding functions are defined tidily together in a table; if they're in _G you have a bit of a job ahead of you. The functions you do care about get bound via luabind as normal.
What I am trying to do to learn some lua/moonscript is migrate my awesome configuration file (rc.lua) to moonscript and unit-test a few things along the way. For this I set up rc.lua to require the moonscript config file like this
package.path = pathsToAdd .. package.path
-- a bit of a hassle to amend the lua require paths
-- correctly; I boldly assume for now that these are not the
-- cause of the problem
require('moonscript')
require('config')
For a first unit test to check if my config calls a specific function of the module 'gears' all went reasonably well. I ended up mocking the gears module of every subsequent call to
require('gears')
by setting up the unit test like so
package.loaded.gears = myMockVersion
fast forward to when my config file under test needs to require the 'awful' module:
its init.lua is called, immediately executing
return
{
client = require("awful.client");
...
}
which leads to client.lua doing
...
local tag = require("awful.tag")
...
local client = {}
-- define lots of functions, register some signal handlers
return client
and now, for everyone still reading, the problem in tag.lua:
...
local capi =
{
...
client = client,
...
}
...
capi.client.connect_signal(...)
That last call throws a good old
attempt to index a nil value (field 'client')
Which I assume is because client.lua did not yet run past the first few require calls and therefore is not available globally at all or, at least, did not define its functionality yet.
Which leads me, at last, to the question:
Why does this even run during your everyday awesome startup (awful is pretty much the core module) in the first place and what do I miss when trying to replicate the environment in which it does.
Thank your very much in advance.
Yours truly
The C core of awesome exports some objects for lua to use. Awful (and lots of others) use these directly. These are in awesome 3.5 (see https://awesome.naquadah.org/doc/api/):
tag
timer
drawin
keygrabber
drawable
root
mouse
client
screen
awesome
mousegrabber
selection
key
dbus
button
Most of these have wrappers in awful which add useful stuff (e.g. key vs awful.key, same for tag, keygrabber, button). Other stuff is completely hidden from "the average user" (e.g. drawin, drawable).
You should be able to mock these as well, but you will have to set global variables with the same name.
Edit: by the way, this is why you cannot require("awful") in a normal lua promt. The same built-in objects are missing.