I'm working in LuaJIT and have all my libraries and whatnot stored inside "foo", like this:
foo = {}; -- The only global variable
foo.print = {};
foo.print.say = function(msg) print(msg) end;
foo.print.say("test")
Now I was wondering, would using metatables and keeping all libraries local help at all? Or would it not matter. What I thought of is this:
foo = {};
local libraries = {};
setmetatable(foo, {
__index = function(t, key)
return libraries[key];
end
});
-- A function to create a new library.
function foo.NewLibrary(name)
libraries[name] = {};
return libraries[name];
end;
local printLib = foo.NewLibrary("print");
printLib.say = function(msg) print(msg) end;
-- Other file:
foo.print.say("test")
I don't really have the tools to benchmark this right now, but would keeping the actual contents of the libraries in a local table increase performance at all? Even the slightest?
I hope I made mysef clear on this, basically all I want to know is: Performance-wise is the second method better?
If someone could link/give a detailed explaination on how global variables are processed in Lua which could explain this that would be great too.
don't really have the tools to benchmark this right now
Sure you do.
local start = os.clock()
for i=1,100000 do -- adjust iterations to taste
-- the thing you want to test
end
print(os.clock() - start)
With performance, you pretty much always want to benchmark.
would keeping the actual contents of the libraries in a local table increase performance at all?
Compared to the first version of the code? Theoretically no.
Your first example (stripping out the unnecessary cruft):
foo = {}
foo.print = {}
function foo.print.say(msg)
print(msg)
end
To get to your print function requires three table lookups:
index _ENV with "foo"
index foo table with "print"
index foo.print table with "say".
Your second example:
local libraries = {}
libraries.print = {}
function libraries.print.say(msg)
print(msg)
end
foo = {}
setmetatable(foo, {
__index = function(t, key)
return libraries[key];
end
});
To get to your print function now requires five table lookups along with other additional work:
index _ENV with "foo"
index foo table with "print"
Lua finds the result is nil, checks to see if foo has a metatable, finds one
index metatable with "__index"
check to see if the result is is is table or function, Lua find it's a function so it calls it with the key
index libraries with "print"
index the print table with "say"
Some of this extra work is done in the C code, so it's going to be faster than if this was all implemented in Lua, but it's definitely going to take more time.
Benchmarking using the loop I showed above, the first version is roughly twice as fast as the second in vanilla Lua. In LuaJIT, both are exactly the same speed. Obviously the difference gets optimized away at runtime in LuaJIT (which is pretty impressive). Just goes to show how important benchmarking is.
Side note: Lua allows you to supply a table for __index, which will result in a lookup equivalent to your code:
setmetatable(foo, { __index = function(t, key) return libraries[key] end } )
So you could just write:
setmetatable(foo, { __index = libraries })
This also happens to be a lot faster.
Here is how I write my modules:
-- foo.lua
local MyLib = {}
function MyLib.foo()
...
end
return MyLib
-- bar.lua
local MyLib = require("foo.lua")
MyLib.foo()
Note that the return MyLib is not in a function. require captures this return value and uses it as the library. This way, there are no globals.
Related
I'm working on a lua obfuscator, and I want it to be able to switch from obfuscating code in the file if there are no arguments, or obfuscating the string argument from the lua cli. If that made absolutely no sense here's what I mean.
local Code = function()
print("test")
end
Dumped = string.dump(Code)
if arg[1] then
local function Temp()
loadstring(arg[1])()
end
Dumped = string.dump(Temp)
end
The problem with this is that in the dump, it's not actually whatever the argument is, rather the variable. So how'd I get around that and change my string to code? I'm sorry if I make no sense.
Here's what I've tried and the problem:
local Code = function()
print("test")
end
Dumped = string.dump(Code)
if arg[1] then
local function Temp()
loadstring(arg[1])()
end
Dumped = string.dump(Temp)
end
print(Dumped)
-- Say if we did lua file.lua print("hello world")
-- Expected result: LuaQ print hello world (with a bunch of whitespace in between)
-- Actual result: LuaQ arg[1] (also with a bunch of whitespace in between)
Right now you are dumping a function that loads the string and runs it. So you are getting the dump of the code that loads a string and runs it. No surprise there.
If you want to dump some other code, then you need to actually dump that code:
local code, codeError = loadstring(arg[1])
if code == nil then
print("Error:",codeError)
else
Dumped = string.dump(code)
end
Lets take a look at this pseudo code example:
-- Wraps a function with given parameters inside a callback
-- Usefull for sending class methods as callbacks.
-- E.g. callback(object.method, object)
local function callback(func, ...)
return function(...)
func(..., ...)
end
end
How would I go about this?
I know there is unpack, but it would get swallowed by the second vararg:
local function callback(func, ...)
local args = { ... }
return function(...)
func(table.unpack(args), ...)
end
end
callback(print, "First", "Second")("Third") --> prints First Third
The only option I found so far, is to concat those together and then unpack it:
local function callback(func, ...)
local args1 = { ... }
return function(...)
local args2 = { ... }
local args = { }
for _, value in ipairs(args1) do
table.insert(args, value)
end
for _, value in ipairs(args2) do
table.insert(args, value)
end
func(table.unpack(args))
end
end
Is this the only solution, or could I do better?
What I would like to have is either a function, that concats two arrays together (which should be faster than those two for loops) and then use table.unpack on it, or make those varargs concat.
From the Lua 5.4 Reference Manual: 3.4 Expressions
If an expression is used as the last (or the only) element of a list
of expressions, then no adjustment is made (unless the expression is
enclosed in parentheses). In all other contexts, Lua adjusts the
result list to one element, either discarding all values except the
first one or adding a single nil if there are no values.
So the only way is to manually combine both lists to a single one befor you use them.
There are several ways to do this.
for i,v in ipairs(list2) do
table.insert(list1, v)
end
for i,v in ipairs(list2) do
list1[#list1+i] = v
end
table.move(list2, 1, #list2, #list1 + 1, list1)
I'm not sure what kind of problem you're actually trying to solve here.
If you want to access your object from its methods use self.
-- Wraps a function with given parameters inside a callback
-- Usefull for sending class methods as callbacks.
-- E.g. callback(object.method, object)
Usually you would do something like this:
local callback = function(params) object:method(params) end
callback(params)
Instead of
callback(print, "First", "Second")("Third")
You could do this
local callback = function (...) print("First", "Second", ...) end
callback("Third")
Edit ( opinonated )
My main goal is to use member functions as callbacks. However, I'd
like to keep it general. The advantage of a callback function is to
omit the function and end keyword. It's shorter and looks closer to
what it would look like, if there would be no need for a self
argument. So this: object.event = function(...) return
self:method(...) end would become to this: object.event =
callback(self.method, self) what would be even nicer, is this (sadly
not possible in lua) object.event = self:method
So instead of
RegisterCallback(function() obj:method() end)
You say it is easier to do this, so you don't have to write function end?
local function callback(func, ...)
local args1 = { ... }
return function(...)
local args2 = { ... }
local args = { }
for _, value in ipairs(args1) do
table.insert(args, value)
end
for _, value in ipairs(args2) do
table.insert(args, value)
end
func(table.unpack(args))
end
end
RegisterCallback(callback(obj.method, obj)())
Your approach is not going to work if any of the arguments is nil to begin with. And there is not a single advantage. You're just typing other words while increasing the chance of running into errors.
You can control the param order to optimize your function.
local function callback(func, ...)
local args = {...}
return function(...)
func(..., table.unpack(args))
end
end
callback(print, "second", "third")("first") -- prints first second third
In Lua I can write a simple module like so
local database = require 'database'
local M = {}
function M:GetData()
return database:GetData()
end
return M
Which when required, will load once, and all future versions will load the same copy.
If I wanted to take an object-oriented approach I could do something like:
local M = {}
M.__index = M
function M:GetData()
return self.database:GetData()
end
return function(database)
local newM = setmetatable({}, M)
newM.database = database
return newM
end
Where M is only loaded once, and each copy of newM just holds its own data and uses the methods of the original M.
When it comes to testing, with the OO approach I can just pass in a fake version of 'database' and check it gets called, but with the first approach I can't.
So my question is how can I make the first approach support DI/testing without making it class-like?
My thought was to wrap it in a closure something like this:
local mClosure = function(database)
local M = {}
function M:GetData()
return database:GetData()
end
return M
end
return mClosure
but then every time it is called it will create a new copy of M, so it will lose the benefits of both of the previous approaches.
That's clearly a use case for the Lua debug library. With that you can just modify the upvalues of your function and inject dependancies. Also consider that you can use require for this; just require your database module once, create small table that collects data and then redirects to the original module and put it in package.loaded so the next time you require it, the require call returns the modified version of the module. The OO approach is how you would do this kind of thing in a language like Ruby, but in Lua we have way nicer ways of tapping into a module or function without it being specifically designed for that purpose.
local real_db = require 'db'
local fake_db = setmetatable({}, {__index=db})
function fake_db.exec(query) print('running query: '..query) end -- dummy function
function fake_db.something(...) print('doing something'); real_db.something(...) end
package.loaded.db = fake_db
require 'my_tests' -- this in turn requires 'db', but gets the fake one
package.loaded.db = real_db
-- After this point, `require 'db'` will return the original module
I've read about metatables (here and here) and I asked myself if I could create a function call proxy by just adding a function with the same name to the __index table, so that it calls the index function (mine), and then I can call the normal one.
This is what I've tried:
local test = {}
test.static = function () print("static") end -- The normal function
local old = getmetatable(test) or {} -- Get the orginal meta
old.__index = {
static = function () print("hook") end -- Adding my function
}
test.static() -- call just for test ( prints -> static )
test = setmetatable( test , old ) -- setting the metatable
test.static() -- call just for test ( prints -> static, but should be hook )
Try giving the official source a read, specifically §2.4 – Metatables and Metamethods, and the description of the __index methamethod, which reads:
__index: The indexing access table[key]. This event happens when table is not a table or when key is not present in table. The metamethod is looked up in table.
Despite the name, the metamethod for this event can be either a function or a table. If it is a function, it is called with table and key as arguments, and the result of the call (adjusted to one value) is the result of the operation. If it is a table, the final result is the result of indexing this table with key. (This indexing is regular, not raw, and therefore can trigger another metamethod.)
We can see that your thinking is backwards. Properties on a table referenced by the __index metamethod are only looked up if the original table does not contain the key.
If you want to 'hook' a function you'll need to overwrite it, possibly saving the original to restore it later. If you want to tack functionally on to an existing function, you can write a neat little hooking function, which simply creates a closure around the functions, calling them in turn.
local function hook (original_fn, new_fn)
return function (...)
original_fn(...)
new_fn(...)
end
end
local test = {}
test.foo = function () print('hello') end
test.foo = hook(test.foo, function () print('world!') end)
test.foo() --> prints 'hello' then 'world!'
Alternatively, you can switch between metatables, assuming the original table never overrides the keys of interest, to get different results:
local my_table, default, extra = {}, {}, {}
function default.foo () print('hello') end
function extra.foo() print('world!') end
local function set_mt (t, mt)
mt.__index = mt
return setmetatable(t, mt)
end
set_mt(my_table, default)
my_table.foo() --> prints 'hello'
set_mt(my_table, extra)
my_table.foo() --> prints 'world!'
How can I recreate the functionality of setfenv in Lua 5.2? I'm having some trouble understanding exactly how you are supposed to use the new _ENV environment variable.
In Lua 5.1 you can use setfenv to sandbox any function quite easily.
--# Lua 5.1
print('_G', _G) -- address of _G
local foo = function()
print('env', _G) -- address of sandbox _G
bar = 1
end
-- create a simple sandbox
local env = { print = print }
env._G = env
-- set the environment and call the function
setfenv(foo, env)
foo()
-- we should have global in our environment table but not in _G
print(bar, env.bar)
Running this example shows an output:
_G table: 0x62d6b0
env table: 0x635d00
nil 1
I would like to recreate this simple example in Lua 5.2. Below is my attempt, but it does not work like the above example.
--# Lua 5.2
local function setfenv(f, env)
local _ENV = env or {} -- create the _ENV upvalue
return function(...)
print('upvalue', _ENV) -- address of _ENV upvalue
return f(...)
end
end
local foo = function()
print('_ENV', _ENV) -- address of function _ENV
bar = 1
end
-- create a simple sandbox
local env = { print = print }
env._G = env
-- set the environment and call the function
foo_env = setfenv(foo, env)
foo_env()
-- we should have global in our envoirnment table but not in _G
print(bar, env.bar)
Running this example shows the output:
upvalue table: 0x637e90
_ENV table: 0x6305f0
1 nil
I am aware of several other questions on this subject, but they mostly seem to be dealing with loading dynamic code (files or string) which work quite well using the new load function provided in Lua 5.2. Here I am specifically asking for a solution to run arbitrary functions in a sandbox. I would like to do this without using the debug library. According to the Lua documentation we should not have to rely on it.
You cannot change the environment of a function without using the debug library from Lua in Lua 5.2. Once a function has been created, that is the environment it has. The only way to modify this environment is by modifying its first upvalue, which requires the debug library.
The general idea with environments in Lua 5.2 is that the environment should be considered immutable outside of trickery (ie: the debug library). You create a function in an environment; once created there, that's the environment it has. Forever.
This is how environments were often used in Lua 5.1, but it was easy and sanctioned to modify the environment of anything with a casual function call. And if your Lua interpreter removed setfenv (to prevent users from breaking the sandbox), then the user code can't set the environment for their own functions internally. So the outside world gets a sandbox, but the inside world can't have a sandbox within the sandbox.
The Lua 5.2 mechanism makes it harder to modify the environment post function-creation, but it does allow you to set the environment during creation. Which lets you sandbox inside the sandbox.
So what you really want is to just rearrange your code like this:
local foo;
do
local _ENV = { print = print }
function foo()
print('env', _ENV)
bar = 1
end
end
foo is now sandboxed. And now, it's much harder for someone to break the sandbox.
As you can imagine, this has caused some contention among Lua developers.
It's a bit expensive, but if it's that important to you...
Why not use string.dump, and re-load the function into the right environment?
function setfenv(f, env)
return load(string.dump(f), nil, nil, env)
end
function foo()
herp(derp)
end
setfenv(foo, {herp = print, derp = "Hello, world!"})()
To recreate setfenv/getfenv in Lua 5.2 you can do the following:
if not setfenv then -- Lua 5.2
-- based on http://lua-users.org/lists/lua-l/2010-06/msg00314.html
-- this assumes f is a function
local function findenv(f)
local level = 1
repeat
local name, value = debug.getupvalue(f, level)
if name == '_ENV' then return level, value end
level = level + 1
until name == nil
return nil end
getfenv = function (f) return(select(2, findenv(f)) or _G) end
setfenv = function (f, t)
local level = findenv(f)
if level then debug.setupvalue(f, level, t) end
return f end
end
RPFeltz's answer (load(string.dump(f)...)) is a clever one and may work for you, but it doesn't deal with functions that have upvalues (other than _ENV).
There is also compat-env module that implements Lua 5.1 functions in Lua 5.2 and vice versa.
In Lua5.2 a sandboxeable function needs to specify that itself. One simple pattern you can use is have it receive _ENV as an argument
function(_ENV)
...
end
Or wrap it inside something that defines the env
local mk_func(_ENV)
return function()
...
end
end
local f = mk_func({print = print})
However, this explicit use of _ENV is less useful for sandboxing, since you can't always assume the other function will cooperate by having an _ENV variable. In that case, it depends on what you do. If you just want to load code from some other file then functions such as load and loadfile usually receive an optional environment parameter that you can use for sandboxing. Additionally, if the code you are trying to load is in string format you can use string manipulation to add _ENV variables yourself (say, by wrapping a function with an env parameter around it)
local code = 'return function(_ENV) return ' .. their_code .. 'end'
Finally, if you really need dynamic function environment manipulation, you can use the debug library to change the function's internal upvalue for _ENV. While using the debug library is not usually encouraged, I think it is acceptable if all the other alternatives didn't apply (I feel that in this case changing the function's environment is deep voodoo magic already so using the debug library is not much worse)