Lua will write the code of a function out as bytes using string.dump, but warns that this does not work if there are any upvalues. Various snippets online describe hacking around this with debug. It looks like 'closed over variables' are called 'upvalues', which seems clear enough. Code is not data etc.
I'd like to serialise functions and don't need them to have any upvalues. The serialised function can take a table as an argument that gets serialised separately.
How do I detect attempts to pass closures to string.dump, before calling the broken result later?
Current thought is debug.getupvalue at index 1 and treat nil as meaning function, as opposed to closure, but I'd rather not call into the debug interface if there's an alternative.
Thanks!
Even with debug library it's very difficult to say whether a function has a non-trivial upvalue.
"Non-trivial" means "upvalue except _ENV".
When debug info is stripped from your program, all upvalues look almost the same :-)
local function test()
local function f1()
-- usual function without upvalues (except _ENV for accessing globals)
print("Hello")
end
local upv = {}
local function f2()
-- this function does have an upvalue
upv[#upv+1] = ""
end
-- display first upvalues
print(debug.getupvalue (f1, 1))
print(debug.getupvalue (f2, 1))
end
test()
local test_stripped = load(string.dump(test, true))
test_stripped()
Output:
_ENV table: 00000242bf521a80 -- test f1
upv table: 00000242bf529490 -- test f2
(no name) table: 00000242bf521a80 -- test_stripped f1
(no name) table: 00000242bf528e90 -- test_stripped f2
The first two lines of the output are printed by test(), the last two lines by test_stripped().
As you see, inside test_stripped functions f1 and f2 are almost undistinguishable.
I am writing a program to find the standard deviation of a data set. I don't have a proper Lua editor so I am testing everything from the interactive interpreter.
In the code below, everything seems to work until I get to the diffsqrd function. After I call this function, the interpreter stops letting me type anything in. I have to close it and start over again. I have tested this function by itself, without the code before it and it works fine.
I tried saving the whole thing as a .lua file and running it with dofile but it did the same thing.
I get nothing, and then I can no longer type into the interpreter. What is going on?
--a function to see if a file exists
function file_exists(file)
local f=io.open(file, "r")
if f then f:close() end
return f ~= nil
end
--get all lines from a file, returns an empty
--list/table if the file does not exist
function lines_from(file)
if not file_exists(file) then return {} end
lines = {}
for line in io.lines(file) do
lines[#lines + 1] = line
end
return lines
end
--Put the .rec file into an array
y_positions=lines_from([[Z:\Octupole stuff\programming\y_p_test.rec]])
--functions to find the standard deviation of an array.
--total, average, difference squared. I stop here because this is the
--minimum code required to cause the problem.
function total(a)
local sum=0
for i,v in ipairs(a) do sum = sum + v
end
return sum
end
function average(a)
if #a==0 then mean=0
else mean=total(a)/#a
end
return mean
end
function diffsqrd(a)
local diff={}
for i in ipairs(a) do
diff[i]=(a[i]-average(a))^2
end
return diff
end
--Use the diffsqrd function on the .rec file.
yd=diffsqrd(y_positions)
print(yd[1])
I am confused as to how function is declared in lua. What i want to know is the order the function definition is in the file. In my example, sample 1 works where as sample 2 wouldn't compile.
Sample 1
--This works, sample 1
function finddir(lpath)
local localfs = require "luci.fs"
if localfs.isdirectory(lpath) then
print "we have directory"
else
print "Directory not found"
end
end
**local ltest = finddir("/proc/net/")**
-- END --
Sample 2
--This Sample fails to compile, Sample 2
**local ltest = finddir("/proc/net/")**
function finddir(lpath)
local localfs = require "luci.fs"
if localfs.isdirectory(lpath) then
print "we have directory"
else
print "Directory not found"
end
end
-- END --
Functions in Lua are first-class values.
In the first example, the function is defined, in another word, the variable finddir has a value of type function. So you can call it.
In the second example, the function has not been defined when you call it, in another word, the varialbe finddir has a value nil, thus you can't call it.
It's not that different with other types, e.g:
n = 42
local a = n + 3 --fine
vs
local a = n + 3 --error, n is nil
n = 42
I'm trying to serialize and deserialize a Lua closure
my basic understanding is that the below factory should generate closures (and that Lua doesn't much distinguish between functions and closures -- i.e. there is no type 'closure')
> function ffactory(x) return function() return x end end
> f1 = ffactory(5)
> print(f1())
5 <-- so far so good
> s = string.dump(f1)
> f2 = load(s)
> print(f2())
table: 00000000002F7BA0 <-- expected the integer 5
> print(f2()==_ENV)
true <-- definitely didn't expect this!
I expected the integer 5 to be serialized with f1. Or, if string.dump can't handle closures, i expected an error.
I get quite different (but more what I expected) results with a mild change. It looks like f2 is indeed a closure, but string.dump didn't attempt to serialize the value of x at the time it was serialized.
The docs don't help me much. (what do they mean by "...with new upvalues"?)
> function ffactory(x) return function() return x+1 end end
> f1 = ffactory(5)
> print(f1())
6 <-- good
> s = string.dump(f1)
> f2 = load(s)
> print(f2())
stdin:1: attempt to perform arithmetic on upvalue 'x' (a table value)
stack traceback:
stdin:1: in function 'f2'
stdin:1: in main chunk
[C]: in ?
You can do something like this to save/restore those upvalues (note it doesn't handle upvalues shared between different functions):
local function capture(func)
local vars = {}
local i = 1
while true do
local name, value = debug.getupvalue(func, i)
if not name then break end
vars[i] = value
i = i + 1
end
return vars
end
local function restore(func, vars)
for i, value in ipairs(vars) do
debug.setupvalue(func, i, value)
end
end
function ffactory(x) return function() return x end end
local f1 = ffactory(5)
local f2 = (loadstring or load)(string.dump(f1))
restore(f2, capture(f1)) --<-- this restored upvalues from f1 for f2
print(f1(), f2())
This works under both Lua 5.1 and Lua 5.2.
Note an interesting result if you change ffactory slightly (added math.abs(0); anything that uses global table in any way will do):
function ffactory(x) return function() math.abs(0) return x end end
Now if you restore upvalues you get the same result, but if you don't restore upvalues you get a run-time error under Lua 5.2:
lua.exe: upvalues.lua:19: attempt to index upvalue '_ENV' (a nil value)
stack traceback:
upvalues.lua:19: in function 'f2'
upvalues.lua:24: in main chunk
[C]: in ?
the docs are pretty clear. string.dump doesn't handle closures using upvalues. this is because the upvalues could be anything (including userdata, and how would Lua know how to serialize that?)
upvalues are the external variables that are local to a function due to scoping/closures. since x in your example is an upvalue to the function returned by ffactory, it does not get serialized.
if you want to support this somehow, you would have to store the upvalues yourself and set them again after you've deserialized the function, like this:
function ffactory(x)
return function() return x+1 end
end
local f1 = ffactory(5)
print(f1())
local s = string.dump(f1)
f2 = loadstring(s)
debug.setupvalue(f2, 1, 5)
print(f2())
How can I implement bitwise operators in Lua language?
Specifically, I need a XOR operator/method.
In Lua 5.2, you can use functions in bit32 library.
In Lua 5.3, bit32 library is obsoleted because there are now native bitwise operators.
print(3 & 5) -- bitwise and
print(3 | 5) -- bitwise or
print(3 ~ 5) -- bitwise xor
print(7 >> 1) -- bitwise right shift
print(7 << 1) -- bitwise left shift
print(~7) -- bitwise not
Output:
1
7
6
3
14
-8
In Lua 5.2, you can use the bit32.bxor function.
Since you're referencing the floor function 3 times, using an excessive number of loops for most operations (numbers less than 2^31 don't need all 31 loops), are using the ^ operator, and aren't capitalizing on the fact that a and b might be wildly different numbers with different magnitudes, you're losing a lot of efficiency. The function also isn't localized, and you're doing two more division operations than you need to. I wrote this to be reasonably fast.
In general, you're going to see improvements of about 3 to 20 times.
local function BitXOR(a,b)--Bitwise xor
local p,c=1,0
while a>0 and b>0 do
local ra,rb=a%2,b%2
if ra~=rb then c=c+p end
a,b,p=(a-ra)/2,(b-rb)/2,p*2
end
if a<b then a=b end
while a>0 do
local ra=a%2
if ra>0 then c=c+p end
a,p=(a-ra)/2,p*2
end
return c
end
If you need more than this, say AND, OR, and NOT, then I've got you covered there, too.
local function BitOR(a,b)--Bitwise or
local p,c=1,0
while a+b>0 do
local ra,rb=a%2,b%2
if ra+rb>0 then c=c+p end
a,b,p=(a-ra)/2,(b-rb)/2,p*2
end
return c
end
local function BitNOT(n)
local p,c=1,0
while n>0 do
local r=n%2
if r<1 then c=c+p end
n,p=(n-r)/2,p*2
end
return c
end
local function BitAND(a,b)--Bitwise and
local p,c=1,0
while a>0 and b>0 do
local ra,rb=a%2,b%2
if ra+rb>1 then c=c+p end
a,b,p=(a-ra)/2,(b-rb)/2,p*2
end
return c
end
Don't worry, you won't need to change anything.
If you're needing an efficient way to do bitwise shifts, I wrote an article about that a while ago. Here's some functions which wrap the technique:
function lshift(x, by)
return x * 2 ^ by
end
function rshift(x, by)
return math.floor(x / 2 ^ by)
end
Try:
function xor(a,b)
return (a or b) and not (a and b)
end
From the OP; moved from question into this answer.
This is how I implemented XOR in Lua:
local floor = math.floor
function bxor (a,b)
local r = 0
for i = 0, 31 do
local x = a / 2 + b / 2
if x ~= floor (x) then
r = r + 2^i
end
a = floor (a / 2)
b = floor (b / 2)
end
return r
end
This is very simple. use NAND logic.
https://en.wikipedia.org/wiki/NAND_logic
function xor(a,b)
return not( not( a and not( a and b ) ) and not( b and not( a and b ) ) )
end
if you also need 1,0 inputs insert the following to the function
a = a==1 or a == true -- to accept nil, 1, 0, true or false
b = b==1 or b == true -- to accept nil, 1, 0, true or false
Hope this helps someone.