Is it possible to set an __index method for torch classes? I have tried to implement a simple dataset class as outlined in the Deep Learning with Torch tutoral: (ipynb here)
trainset = {
inputs = {0, 1, 1, 0},
targets = {1, 1, 1, 0}
}
index = function(t, i)
return {t.inputs[i], t.targets[i]}
end
setmetatable(trainset, {
__index = index
)
Which allows you to do trainset[1]] which returns {0, 1}.
However, implementing this as torch class does not work.
local torch = require("torch")
do
Dataset = torch.class("Dataset")
function Dataset:__init(i, t)
self.inputs = i
self.targets = t
end
function Dataset.__index(t, v)
print("inside index")
return {
rawget(t, inputs)[v],
rawget(t, targets)[v]
}
end
end
Dataset({0, 1, 1, 0}, {1, 1, 1, 0}) -- fails
It seems that, upon object creation, __index() is called and fails since index and targets are not yet created. If rawget is not used, then it causes a stack overflow.
My understanding of Lua is limited, but I'm surprised to see __index() being called during object creation: I think there's stuff going on behind the scenes I don't fully understand.
Torch classes all implement __index, which will look for __index__ in the metatable, which is for overloading.
From the docs:
If one wants to provide index or newindex in the metaclass,
these operators must follow a particular scheme:
index must either return a value and true or return false only. In the first case, it means index was able to handle the given
argument (for e.g., the type was correct). The second case means it
was not able to do anything, so __index in the root metatable can then
try to see if the metaclass contains the required value.
Which means for the example, the __index__ (not __index !) method must check if type(v) == "number" and if not, return false so that __index can look for the value in the object metatable.
local torch = require("torch")
do
Dataset = torch.class("Dataset")
function Dataset:__init(i, t)
self.inputs = i
self.targets = t
end
function Dataset.__index__(t, v)
if type(v) == "number" then
local tbl = {
t.inputs[v],
t.targets[v]
}
return tbl, true
else
return false
end
end
local dset = Dataset({0, 1, 1, 0}, {1, 1, 1, 0})
dset[1] --> {0, 1}
Related
Assume I have a function that returns multiple values. I happen to be working with LÖVE's (Image):getDimensions. This returns two values, which I know to be width,height. I want to assign them to a new table, as an array. I would like named (string) keys. So for instance, I would like to assign the return values of the getDimensions() function to a new table with keys width and height, respectively.
I know the following works...
image = {}
image.data = love.graphics.newImage('myimage.png')
image.size = {}
image.size.width, image.size.height = image.data:getDimensions()
I'm wondering if there is any sort of syntactic sugar I can use, or any use of standard library functions that will allow a syntax more along the lines of...
image.size = { width, height = image.data:getDimensions() }
I know the above line does not work, along with many variations I've tried, including various attempts to use unpack(). I'm new to Lua (~2 days in now), so maybe there is another standard function or best practice that I'm unaware of that will associate a table of keys to an array-like table. Thanks!
You can write your own functions:
local function set_fields(tab, fields, ...)
-- fields is an array of field names
-- (use empty string to skip value at corresponging position)
local values = {...}
for k, field in ipairs(fields) do
if field ~= "" then
tab[field] = values[k]
end
end
return tab
end
local function get_fields(tab, fields)
local values = {}
for k, field in ipairs(fields) do
values[k] = tab[field]
end
return (table.unpack or unpack)(values, 1, #fields)
end
Usage example #1:
image.size = set_fields({}, {"width", "height"}, image.data:getDimensions())
Usage example #2:
Swap the values on-the-fly!
local function get_weekdays()
return "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"
end
-- we want to save returned values in different order
local weekdays = set_fields({}, {7,1,2,3,4,5,6}, get_weekdays())
-- now weekdays contains {"Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"}
Usage example #3:
local function get_coords_x_y_z()
return 111, 222, 333 -- x, y, z of the point
end
-- we want to get the projection of the point on the ground plane
local projection = {y = 0}
-- projection.y will be preserved, projection.x and projection.z will be modified
set_fields(projection, {"x", "", "z"}, get_coords_x_y_z())
-- now projection contains {x = 111, y = 0, z = 333}
Usage example #4:
If require("some_module") returns a module with plenty of functions inside, but you need only a few of them:
local bnot, band, bor = get_fields(require("bit"), {"bnot", "band", "bor"})
Using a class construct, I have come up with the following...
Size = {}
Size.mt = {}
Size.prototype = { width = 0, height = 0 }
function Size.new (dimensions)
local size = setmetatable({}, Size.mt)
if dimensions ~= nil and type(dimensions) == 'table' then
for k,v in pairs(dimensions) do
size[k] = v
end
end
return size
end
Size.mt.__index = function (t, k)
if k == 'width' or k == 'height' then
rawset(t, k, Size.prototype[k])
return t[k]
else
return nil
end
end
Size.mt.__newindex = function (t, k, v)
if k == 1 or k == 'width' then
rawset(t, 'width', v)
elseif k == 2 or k == 'height' then
rawset(t, 'height', v)
end
end
Then I can initialize a Size object in a number of ways
Using multiple return values:
image.size = Size.new{image.data:getDimensions()}
image.size = Size.new(table.pack(image.data:getDimensions())
Using default values:
image.size = Size.new()
image.size = Size.new{}
image.size = Size.new({})
Using mixed array and hash tables:
image.size = Size.new({height=20, width=30})
image.size = Size.new({height=20, 30})
There are pros and cons to this approach vs. Egor's (utility function), which is what I was considering doing if there wasn't a simple syntax trick or an existing function that I was unaware of.
Pros:
(personal) learning experience with OO constructs in Lua
I can limit the number of actual keys on the table, while allowing 'synonyms' for those keys to be added by expanding the accepted values in the if/else logic of __index and __newindex
Explicit definition of fields in the table, without needing to worry about syncing a table of keys with a table of values (as with a general purpose utility function)
Cons
would need to repeat this pattern for each data structure where I wanted this behavior
costly, a lot of overhead for what amounts to a very small difference to the consumer
I'm sure I can make this approach more robust in time, but I would appreciate any feedback.
What is the most efficient way to convert number to table? Or is it possible to make a table without loops?
local t = 10 -- given number
{1, 2, 3, 4, 5, 6, 7, 8, 9, 10} -- result
Update: the t variable is mutable number and I want to for each the value.
t = 3
function foreach(f, t)
for i, v in ipairs(t) do
f(v)
end
end
foreach(print, t)
1
2
3
I need a just the quickest way of new Array(n) in Lua. Or doesn't make any sense?
Maybe you don't know how to answer #Sebastian's question. Here are a few alternatives to get you thinking.
Since your table has only computed elements, you could omit the storage and just perform the calculation on every read access (index operation).
local function newArray(size)
local t = {}
setmetatable(t, {
__index = function (_, i)
return i >= 1 and i <= size and i or nil end})
return t
end
t10 = newArray(10)
for i = 0, 11 do -- ipairs won't work as expected with such a table
print(i, t10[i])
end
t10[2] = "stored values override __index"
print(t10[2])
Of course, you could also replace the table with just an identity function that returns the value, or even just an identity expression. But, maybe you have an unexpressed requirement for a table or you need ipairs to iterate over the sequence.
Speaking of iterators,
local function seq_itor(first, last)
local i = first - 1
return function ()
i = i + 1
if i <= last then return i end
end
end
for i in seq_itor(1, 10) do
print(i)
end
The simplest way to do that would be to define a function:
function newArray(size)
local t = {}
for i = 1, size do
t[i] = i
end
return t
end
In chapter 13.2 of Programming in Lua it's stated that
Unlike arithmetic metamethods, relational metamethods do not support mixed types.
and at the same time
Lua calls the equality metamethod only when the two objects being compared share this metamethod
So I'm implementing my library in C and want to be able to support behavior like
a = A()
b = B()
a == b
by providing
static const struct luaL_Reg mylib_A[] =
{
{ "__eq", my_equal }
, <more stuff>
, { NULL, NULL }
};
and
static const struct luaL_Reg mylib_B[] =
{
{ "__eq", my_equal }
, <more stuff>
, { NULL, NULL }
};
Which doesn't seem to work, is there a workaround for this?
Note: my_equal is able to handle both userdata of type A and type B in any of it's arguments
UPDATE:
Metatables registration:
luaL_newmetatable(lua, "B");
lua_pushvalue(lua, -1);
lua_setfield(lua, -2, "__index");
luaL_register(lua, NULL, mylib_B);
luaL_newmetatable(lua, "A");
lua_pushvalue(lua, -1);
lua_setfield(lua, -2, "__index");
luaL_register(lua, NULL, mylib_A);
luaL_register(lua, "mylib", mylib); -- where mylib is a bunch of static functions
Application code:
require 'mylib'
a = mylib.new_A()
b = mylib.new_B()
a == b -- __eq is not called
EDIT: Also see whoever's answer which has a particular caveat with regards to implementing __eq in the C API.
The __eq metamethod belongs in your metatable, not in the __index table.
In lua:
function my_equal(x,y)
return x.value == y.value
end
A = {} -- luaL_newmetatable(lua, "A");
A.__eq = my_equal
function new_A(value)
local a = { value = value }
return setmetatable(a, A)
end
B = {} -- luaL_newmetatable(lua, "B");
B.__eq = my_equal
function new_B(value)
local b = { value = value }
return setmetatable(b, B)
end
a = new_A()
b = new_B()
print(a == b) -- __eq is called, result is true
a.value = 5
print(a == b) -- __eq is called, result is false
What you have done is this:
myLib_A = {}
myLib_A.__eq = my_equal
A = {} -- luaL_newmetatable(lua, "A");
A.__index = myLib_A
Note that __eq is not in A's metatable, it's on a totally separate table that you just be happen to be using in a different, unrelated metamethod (__index). Lua is not going to look there when trying to resolve the equality operator for a.
The Lua manual explains this in detail:
"eq": the == operation. The function getcomphandler defines how Lua chooses a metamethod for comparison operators. A metamethod only is selected when both objects being compared have the same type and the same metamethod for the selected operation.
function getcomphandler (op1, op2, event)
if type(op1) ~= type(op2) then return nil end
local mm1 = metatable(op1)[event]
local mm2 = metatable(op2)[event]
if mm1 == mm2 then return mm1 else return nil end
end
The "eq" event is defined as follows:
function eq_event (op1, op2)
if type(op1) ~= type(op2) then -- different types?
return false -- different objects
end
if op1 == op2 then -- primitive equal?
return true -- objects are equal
end
-- try metamethod
local h = getcomphandler(op1, op2, "__eq")
if h then
return (h(op1, op2))
else
return false
end
end
So when Lua encounters result = a == b, it's going to do the following (this is done in C, Lua used as pseudocode here):
-- Are the operands are the same type? In our case they are both tables:
if type(a) ~= type(b) then
return false
end
-- Are the operands the same object? This comparison is done in C code, so
-- it's not going to reinvoke the equality operator.
if a ~= b then
return false
end
-- Do the operands have the same `__eq` metamethod?
local mm1 = getmetatable(a).__eq
local mm2 = getmetatable(b).__eq
if mm1 ~= mm2 then
return false
end
-- Call the `__eq` metamethod for the left operand (same as the right, doesn't really matter)
return mm1(a,b)
You can see there's no path here that results in resolve a.__eq, which would resolve to myLib_A through your __index metamethod.
For all others who would be facing the same problem:
This was the only way how I made Lua aware of my_equal being the exact same function from the point of Lua in both cases and consequently returning correct operator from getcomphandler. Registering it in any other way (including separate luaL_Reg) doesn't work due to my_equal being saved under different closures upon luaL_register, which I here avoid by creating the closure only once.
// we'll copy it further to ensure lua knows that it's the same function
lua_pushcfunction(lua, my_equal);
luaL_newmetatable(lua, "B");
// removed __index for clarity
luaL_register(lua, NULL, mylib_B);
// Now we register __eq separately
lua_pushstring(lua, "__eq");
lua_pushvalue(lua, -3); // Copy my_equal on top
lua_settable(lua, -3); // Register it under B metatable
lua_pop(lua, 1);
luaL_newmetatable(lua, "A");
// removed __index for clarity
luaL_register(lua, NULL, mylib_A);
lua_pushstring(lua, "__eq");
lua_pushvalue(lua, -3); // Copy my_equal on top
lua_settable(lua, -3); // Register it under A metatable
luaL_register(lua, "mylib", mylib); // where mylib is a bunch of static functions
I'm trying to call a function in Lua that accepts multiple 'number' arguments
function addShape(x1, y1, x2, y2 ... xn, yn)
and I have a table of values which I'd like to pass as arguments
values = {1, 1, 2, 2, 3, 3}
Is it possible to dynamically 'unpack' (I'm not sure if this is the right term) these values in the function call? Something like..
object:addShape(table.unpack(values))
Equivalent to calling:
object:addShape(1, 1, 2, 2, 3, 3)
Apologies if this is a totally obvious Lua question, but I can't for the life of me find anything on the topic.
UPDATE
unpack(values) doesn't work either (delving into the method addShape(...) and checking the type of the value passed reveals that unpackis resulting in a single string.
You want this:
object:addShape(unpack(values))
See also: http://www.lua.org/pil/5.1.html
Here's a complete example:
shape = {
addShape = function(self, a, b, c)
print(a)
print(b)
print(c)
end
}
values = {1, 2, 3}
shape:addShape(unpack(values))
Whoever comes here and has Lua version > 5.1 unpack is moved into the table library so you can use: table.unpack
For more info: https://www.lua.org/manual/5.2/manual.html
This is not an answer about unpack, but a suggestion to use a different technique. Instead, do
function object:addShape(values)
for i,v in ipairs(values) do
x,y = v.x, v.y
...
end
end
function getPairs(values)
xyPairs = {}
for i=1,#values,2 do
v = {x=values[i], y=values[i+i] }
table.insert(xyPair, v)
end
return xyPairs
end
values = {1, 1, 2, 2, 3, 3}
object:addShape(getPairs(values))
The amount of work to be done should be similar as unpacking and the additional processing you will have to do in addShape() to support variable number of named arguments.
ORIGINAL POST
Given that there is no built in function in Lua, I am in search of a function that allows me to append tables together. I have googled quite a bit and have tried every solutions I stumbled across but none seem to work properly.
The scenario goes like this: I am using Lua embeded in an application. An internal command of the application returns a list of values in the form of a table.
What I am trying to do is call that command recursively in a loop and append the returned values, again in the form of a table, to the table from previous iterations.
EDIT
For those who come across this post in the future, please note what #gimf posted. Since Tables in Lua are as much like arrays than anything else (even in a list context), there is no real correct way to append one table to another. The closest concept is merging of tables. Please see the post, "Lua - merge tables?" for help in that regard.
Overcomplicated answers much?
Here is my implementation:
function TableConcat(t1,t2)
for i=1,#t2 do
t1[#t1+1] = t2[i]
end
return t1
end
If you want to concatenate an existing table to a new one, this is the most concise way to do it:
local t = {3, 4, 5}
local concatenation = {1, 2, table.unpack(t)}
Although I'm not sure how good this is performance-wise.
And one more way:
for _,v in ipairs(t2) do
table.insert(t1, v)
end
It seems to me the most readable one - it iterates over the 2nd table and appends its values to the 1st one, end of story. Curious how it fares in speed to the explicit indexing [] above
A simple way to do what you want:
local t1 = {1, 2, 3, 4, 5}
local t2 = {6, 7, 8, 9, 10}
local t3 = {unpack(t1)}
for I = 1,#t2 do
t3[#t1+I] = t2[I]
end
To add two tables together do this
ii=0
for i=#firsttable, #secondtable+#firsttable do
ii=ii+1
firsttable[i]=secondtable[ii]
end
use the first table as the variable you wanted to add as code adds the second one on to the end of the first table in order.
i is the start number of the table or list.
#secondtable+#firsttable is what to end at.
It starts at the end of the first table you want to add to, and ends at the end of the second table in a for loop so it works with any size table or list.
In general the notion of concatenating arbitrary tables does not make sense in Lua because a single key can only have one value.
There are special cases in which concatenation does make sense. One such is for tables containing simple arrays, which might be the natural result of a function intended to return a list of results.
In that case, you can write:
-- return a new array containing the concatenation of all of its
-- parameters. Scaler parameters are included in place, and array
-- parameters have their values shallow-copied to the final array.
-- Note that userdata and function values are treated as scalar.
function array_concat(...)
local t = {}
for n = 1,select("#",...) do
local arg = select(n,...)
if type(arg)=="table" then
for _,v in ipairs(arg) do
t[#t+1] = v
end
else
t[#t+1] = arg
end
end
return t
end
This is a shallow copy, and makes no attempt to find out if a userdata or function value is a container or object of some kind that might need different treatment.
An alternative implementation might modify the first argument rather than creating a new table. This would save the cost of copying, and make array_concat different from the .. operator on strings.
Edit: As observed in a comment by Joseph Kingry, I failed to properly extract the actual value of each argument from .... I also failed to return the merged table from the function at all. That's what I get for coding in the answer box and not testing the code at all.
If you want to merge two tables, but need a deep copy of the result table, for whatever reason, use the merge from another SO question on merging tables plus some deep copy code from lua-users.
(edit
Well, maybe you can edit your question to provide a minimal example... If you mean that a table
{ a = 1, b = 2 }
concatenated with another table
{ a = 5, b = 10 }
should result in
{ a = 1, b = 2, a = 5, b = 10 }
then you're out of luck. Keys are unique.
It seems you want to have a list of pairs, like { { a, 1 }, { b, 2 }, { a, 5 }, { b, 10 } }. You could also use a final structure like { a = { 1, 5 }, b = { 2, 10 } }, depending on your application.
But the simple of notion of "concatenating" tables does not make sense with Lua tables.
)
Here is an implementation I've done similar to RBerteig's above, but using the hidden parameter arg which is available when a function receives a variable number of arguments. Personally, I think this is more readable vs the select syntax.
function array_concat(...)
local t = {}
for i = 1, arg.n do
local array = arg[i]
if (type(array) == "table") then
for j = 1, #array do
t[#t+1] = array[j]
end
else
t[#t+1] = array
end
end
return t
end
Here is my implementation to concatenate a set of pure-integer-indexing tables, FYI.
define a function to concatenate two tables, concat_2tables
another recursive function concatenateTables: split the table list by unpack, and call concat_2tables to concatenate table1 and restTableList
t1 = {1, 2, 3}
t2 = {4, 5}
t3 = {6}
concat_2tables = function(table1, table2)
len = table.getn(table1)
for key, val in pairs(table2)do
table1[key+len] = val
end
return table1
end
concatenateTables = function( tableList )
if tableList==nil then
return nil
elseif table.getn(tableList) == 1 then
return tableList[1]
else
table1 = tableList[1]
restTableList = {unpack(tableList, 2)}
return concat_2tables(table1, concatenateTables(restTableList))
end
end
tt = {t1, t2, t3}
t = concatenateTables(tt)
-- Lua 5.1+
function TableAppend(t1, t2)
-- A numeric for loop is faster than pairs, but it only gets the sequential part of t2
for i = 1, #t2 do
t1[#t1 + 1] = t2[i] -- this is slightly faster than table.insert
end
-- This loop gets the non-sequential part (e.g. ['a'] = 1), if it exists
local k, v = next(t2, #t2 ~= 0 and #t2 or nil)
while k do
t1[k] = v -- if index k already exists in t1 then it will be overwritten
k, v = next(t2, k)
end
end
EDIT
Here's a better solution, the other one tended to overwrite numeric keys, the usage is still the same:
function merge(...)
local temp = {}
local index = 1
local result = {}
math.randomseed(os.time())
for i, tbl in ipairs({ ... }) do
for k, v in pairs(tbl) do
if type(k) == 'number' then
-- randomize numeric keys
k = math.random() * i * k
end
temp[k] = v
end
end
for k, v in pairs(temp) do
if type(k) == "number" then
-- Sort numeric keys into order
if result[index] then
index = index + 1
end
k = index
end
result[k] = v
end
return result
end
ORIGINAL
A wee bit late to the game, but this seems to work for me:
function concat(...)
local result = {}
for i, tbl in ipairs({...}) do
for k, v in pairs(tbl) do
if type(k) ~= "number" then
result[k] = v
else
result[i] = v
end
end
end
return result
end
It might be a bit overcomplicated, but it takes an infinite amount of arguments, and works for both key-value pairs and regular "arrays" (numbers as keys). Here's an example
I like the simplicity in #Weeve Ferrelaine answer, but mutations may cause many issues and in general, are not desirable.
Version with NO MUTATION.
---#param t1 {}
---#param t2 {}
function TableConcat(t1,t2)
local tOut = {}
for i = 1, #t1 do
tOut[i] = t1[i]
end
for i = #t1, #t1 + #t2 do
tOut[i] = t2[i]
end
return tOut
end
Original implementation, that's mutating t1.
function TableConcat(t1,t2)
for i=1,#t2 do
t1[#t1+1] = t2[i]
end
return t1
end
Use table.concat:
http://lua-users.org/wiki/TableLibraryTutorial
> = table.concat({ 1, 2, "three", 4, "five" })
12three4five
> = table.concat({ 1, 2, "three", 4, "five" }, ", ")
1, 2, three, 4, five
> = table.concat({ 1, 2, "three", 4, "five" }, ", ", 2)
2, three, 4, five
> = table.concat({ 1, 2, "three", 4, "five" }, ", ", 2, 4)
2, three, 4