I want to create a 0-memory lua array that actually jumps to my custom function when I use operators like # [] on it
Any ideas on how to do this?
I want the user using this fake array to not perceive it as fake, it is worse than a normal array in terms of access speed, but has better memory performance
Lua implements something called metamethods (documentation)
Metamethods are functions which exist subsequently to a table and fire on certain operations such as indexing the array, reading missing indices, gathering the length of the array, or even math operations such as + - * /
-- Start by creating your array, and another for demonstration purposes
local object = {}
local demo = {1, 2, 3, 4}
-- Create a your metamethods contained in a table
local metamethods = {
__index = function(self, index) return demo[index] end;
__newindex = function(self, index, value) demo[index] = value end;
}
-- Lets print out what is in the object table for demonstration
print(object[1]) -- nil
print(object[2]) -- nil
print(object[3]) -- nil
print(object[4]) -- nil
-- Use the setmetatable(table a, table b) function to set the metamethods
-- stored in 'b' to 'a'
setmetatable(object, metamethods);
-- Lets print out what is in the object table for demonstration
print(object[1]) -- 1
print(object[2]) -- 2
print(object[3]) -- 3
print(object[4]) -- 4
Why does the above code work? When a metatable is set with the index __index (metamethods.__index), if the attached table's (object) is indexed and the key isn't present (nil), then it will call the specified function. In the __index function, it returns the demo's table with the index passed straight to it. So its as if: when you do object[1], you actually do demo[1] but with a metamethod's help of course. This effectively creates a proxy of sorts.
One cool and quick usage of setmetatable() is that it returns the value you pass as the first parameter (a table).
local object1 = setmetatable({}, { __index = function(self, i) return 1 end })
print(object1["a"]) -- 1
print(object2[321]) -- 1
Related
I have the following code which Roblox Developer and the Lua.org manual both say should work to remove an instance from the table so I can store as a local, but the local is only holding a nil value.
The table is there. It shows up on the print function. It just will not store to be useful in the app.
I have tried multiple versions of this code including going with just the pairs function, just the table.remove function, and going with and without the position for the table remove, and it all generates nil variable.
response = HttpService:GetAsync(mining)
data = HttpService:JSONDecode(response, Enum.HttpContentType.ApplicationJson)
local function tprint(t)
for k,v in pairs(t) do print(k,v) end
end
tprint(data)
local a = table.remove(data, 4)
local b = table.remove(data, 3)
local c = table.remove(data, 2)
local d = table.remove(data, 1)
The solution ended up being so simple, and yet so profound. I can now use this to link crypto, bank accounts, credit cards, and anything else I want directly into Roblox or any other lua based program.
a = (data["result"]["amount"])
Before I dig into the error you're seeing, first some background information.
Lua tables have two methods of indexing values: numerically, and by keys. Often times you will see these two different methods be used to describe the kind of data structure that uses it.
Arrays and lists are tables that use numeric keys to index information.
local arr = {}
arr[1] = "abc"
arr[2] = 123
arr[3] = true
-- print the length of the array
print(#arr) -- 3
-- print the contents of the array
for i, v in ipairs(arr) do
print(i, v)
-- 1 abc
-- 2 123
-- 3 true
end
On the other side of things, dictionaries and hash maps and associative arrays use keys to store information :
local dict = {}
dict["foo"] = "abc"
dict["bar"] = 123
dict["blah"] = true
dict["katz"] = { 1, 2, 3 }
-- print the number of numerical keys in the dictionary
print(#dict) -- 0
-- print the contents of the dictionary
for k, v in pairs(dict) do
print(k, v)
-- foo abc
-- bar 123
-- blah true
-- katz table
end
While lua allows a table to use both of these indexing methods simultaneously, it's important never to mix the two, as behaviors can get real funky when you do. When a table has keys, treat it like a dictionary. When a table has numerical indices, treat it like an array.
When you use HttpService to decode a JSON string into a table, it generates a dictionary that reflects the heirarchical structure of the original data.
The table library, which you call with table.insert() and table.remove() expects that the table you're working with is an array.
When your data is arranged like this :
local data = {}
data["Success"] = true
data["StatusCode"] = 200
data["StatusMessage"] = "Success"
data["Headers"] = {} -- a dictionary of headers
data["Body"] = {
result = {
amount = 1,
depositAddress = "blah",
},
} -- after HttpService:JSONDecode() is called...
And you tell it to remove a numbered index with table.remove(data, 4), it won't work because there's no data stored at index number 4. data is a dictionary, not an array.
Often, it is annoying to try to print out the contents of a table with multiple layers of data, especially JSON tables, as the pairs function will only index one level at a time. Thankfully, Roblox's print function and Output widget are smart enough to do this for you. You can simply print(data) and it will show you the full table in the output and allow you to inspect each level.
Then once you know how your data is structured you can step through it value by value.
local amount = data["Body"]["result"]["amount"]
-- or
local amount = data.Body.result.amount
I was wondering if it would make sense to pass a meta table by reference vs declaring it in-line in setmetatable() when you want to use the same meta table for multiple tables.
My goal is to save memory, but only if it really makes a significant difference.
What I'm talking about is this:
-- Passing the meta table by reference:
JSON1 = {
metaTable = {
__index = function (t, k)
-- ...
end;
__call = function()
-- ...
end
};
parse = function(filePath)
local fakeParsedJson = {}
setmetatable(fakeParsedJson, JSON1.metaTable) -- Right here
return fakeParsedJson(filePath)
end;
}
VS
-- Passing the table in-line:
JSON2 = {
parse = function(filePath)
local fakeParsedJson = {}
setmetatable(fakeParsedJson, { -- Right here:
__index = function (t, k)
-- ...
end;
__call = function()
-- ...
end
})
return fakeParsedJson(filePath)
end;
}
I tried to find out if there is a significant difference in memory usage, but the only way I could find was to compare the gcinfo:
local start1 = gcinfo()
local example2_1 = JSON2.parse('example2_1.json')
local example2_2 = JSON2.parse('example2_2.json')
local example2_3 = JSON2.parse('example2_3.json')
local example2_4 = JSON2.parse('example2_4.json')
local example2_5 = JSON2.parse('example2_5.json')
print(gcinfo()-start1) -- Prints 1
local start2 = gcinfo()
local example1_1 = JSON1.parse('example1_1.json')
local example1_2 = JSON1.parse('example1_2.json')
local example1_3 = JSON1.parse('example1_3.json')
local example1_4 = JSON1.parse('example1_4.json')
local example1_5 = JSON1.parse('example1_5.json')
print(gcinfo()-start2) -- Prints 1
Here's my fiddle: https://repl.it/HfwS/34
It doesn't really look like there is a difference. But I just don't know what is actually happening under the hood.
When you call setmetatable(myTable,myMetaTable), will that write a complete copy of myMetaTable somewhere into myTable or will it just store a simple reference? Because if it would just store a reference, then it would make a lot of sense to have all my tables pointing to the same meta table.
(On x86_64, in Lua 5.3) every (empty) table costs 56 bytes. Every key/value entry in the table costs 32 bytes (but the number of entries is rounded up to the next power of two). (Byte counts may differ for different versions/platforms, but will be roughly the same +/- a power of two or so.)
If you have two entries in the metatable, that's 120 bytes per metatable. (You're also creating closures (function() … end), so it may actually be even more.)
Having the table constructor in argument position for the call to setmetatable means that every time that call is executed, a new independent table is created (+ new closures for the functions, …). (Also read the section on table constructors in the reference manual.) There is no smart compiler / no de-duplication / … In fact, there can't be, because other code could (potentially) modify a metatable, and then there's a clear semantic / observable difference between a single shared metatable and one metatable per thing. If that's not obvious, compare
Foo = { __name = "Foo", dump = print } ; Foo.__index = Foo
function newFoo( ) return setmetatable( { }, Foo ) end
and
function newFoo( )
local mt = { __name = "Foo", dump = print }
mt.__index = mt
return setmetatable( { }, mt )
end
If you say
t = { newFoo( ), newFoo( ), newFoo( ) }
getmetatable( t[1] ).dump = function( self ) print "<Foo>" end
for _, v in ipairs( t ) do v:dump( ) end
the first version will print
<Foo>
<Foo>
<Foo>
while the second one will print (e.g.)
<Foo>
Foo: 0x1267010
Foo: 0x1267120
which is clearly different behavior. So the compiler/… can not de-duplicate identical metatables, because other code (that was not yet seen) might modify one of the metatables, and then the observed behavior would be different.
▶ This means that if you create multiple (meta)tables, they must be kept somewhere. Storing several tables necessarily uses more memory than storing a single one, so having a table constructor in argument position for the call to setmetatable will use more memory than creating a table first and then passing a reference to it in the call.
That said, worrying about memory use should not be your primary concern. The semantics / "meaning" / observable behavior of your code are more important.
If you modify the metatable, should the behavior of all "objects" / values change? Or do you want to determine object types by metatable identity (getmetatable( x ) == Foo)? Then you must use a shared metatable (or an equivalent construction).
If you modify the metatable, should the behavior of only a single "object" change? Then you must construct & use separate metatables per "object" / value.
Only if you know that you will never modify the metatable, will not compare metatable references to determine object types, will not …, then these different approaches will show the same externally visible behavior, and only then you are free to choose based on secondary concerns (like memory usage, convenience / code brevity, …).
(In general, needing separately modifiable metatables is very rare, so using shared metatables (create first and pass the reference to setmetatable) is the usual approach – it saves memory and is nicer for debugging.)
Aside: gcinfo is very old and only returns integer approximations to the amount of memory used. Use collectgarbage "count" instead, and then you'll see a difference. (It returns kilobytes used, so multiply by 1024 to get bytes.)
I am trying to build a CNN using Torch 7. I am very new to Lua. I was trying to follow this link. I encountered something called setmetatable in the following code block:
setmetatable(train_set,
{
__index = function(t, i)
return {t.data[i], t.label[i]}
end
});
I understand that the second argument acts as the metatable for the table train_set.
1) Is t the metatable or is t just another name for train_set?
2) Whenever a function is used against __index, does the interpreter assume the first argument (t) to be a table (or metatable, depending on answer to first question)? And is the second argument always the key or index?
3) My understanding is that if I use train_set.data[1], it will invoke the __index. The answer here says that __index is invoked when key does not exist in the table. But is t.data[1] same as train_set.data[1]? If so, how does the interpreter know that?
setmetatable(train_set,
{
__index = function(t, i)
return {t.data[i], t.label[i]}
end
})
Here we have some table named train_set. With this function call we set its metatable to
{
__index = function(t, i)
return {t.data[i], t.label[i]}
end
}
This is an anonymous table. If this is hard to read for you, you could also write:
local my_metatable = {
__index = function(t, i)
return {t.data[i], t.label[i]}
end
}
setmetatable(train_set, my_metatable)
Inside that metatable we implement the metamethod __index. By doing this we tell Lua what to do when someone is indexing a field in train_set that does not exist.
So when we ask Lua to give us the value stored in train_set[4] for example and train_set[4] is nil, Lua will go check if __index is implemented. If so it will call __index(train_set, 4) and give you its return value or otherwise return nil
So the interpreter knows that t.data[1] is the same as train_set.data[1], because he's the one who put train_set into __index.
So when you implement __index it will always be called using the indexed table as first and the index as second argument.
First, two important links:
metatables in Programming in Lua;
metatable events in the Lua wiki.
Now the answers:
t, the first parameter of the __index method in a metatable, refers to the table that has the metatable, here train_set. This parameter allows to reuse the same metatable for several tables.
__index is a special function inside metatables (look at the metatable events), that is called whenever a field in the metatabled table is accessed but missing. For instance, if train_set does not contain the key k and you read train_set.k, there will be a call to __index (train_set, "k") in its metatable.
From what i can infer from the code, the usage pattern in your example looks like local x = train_set [1] that will return a table containing { train_set.data[i], train_set.label[i] }.
This question already has an answer here:
lua: iterate through all pairs in table
(1 answer)
Closed 8 years ago.
I am trying to initialize and print a table. It just isnt working. Any idea what is wrong with this code?
--!/usr/bin/env lua
local retv = {}
retv["test"] = 1000
for k,v in ipairs(retv) do
print (k,v)
end
It prints nothing. I am sure I am missing something very basic but I cant figure this out.
There are two forms of the for-loop in Lua:
The numeric and the generic for-loop.
ipairs(t) is an iterator constructor returning up to three arguments suitable for the generic for, allowing you to iterate over the initial sequence (indices 1,2,3,...) in order.
Possible implementations:
function ipairs(t)
local i = 0
return function()
i = i + 1
if t[i] ~= nil then
return i, t[i]
end
end
end
local function ipairs_helper(t, i)
i = i + 1
if t[i] ~= nil then
return i, t[i]
end
end
function ipairs(t)
return ipairs_helper, t, 0
end
As you can see, that will never return your entry with key "test".
What you want instead, is pairs(t), which is equivalent to next, t.
That will iterate all elements.
You need to use pairs instead of ipairs. pairs iterates over all keys, ipairs only iterates over keys that form a sequence of integers starting from 1 without gaps. (Whether these keys are stored in the array or the hash part of the table is an implementation detail and may change during the lifetime of the table.)
For example, ipairs({'a', 'b', nil, 'c'}) iterates over keys 1 and 2, stopping at (and not including) 3, as that key is missing from the table.
I'm trying to make an __index function in my table which can process ALL of the field it receives.. What I want to do is that if I call the table in the following way
mytable.str1.str2.str3
I should be able to return the table
{"str1", "str2", "str3"}
Note that str1,str2,str3 are undefined, they are just strings. I am not trying to create subtables str1, str2, I just want __index to see everything beyond the first period.
Unfortunately what I have seems that __index only captures str1, and complains that "attempt to index field 'str1' (a nil value)"
Anyone know how this can be done?
I'm not sure why you'd want to do this, but here's how you do it. The comments explain the trick, but basically you need a second metatable to handle the table that's returned from the first call to the __index metamethod.
If this isn't clear, let me know and I can explain in more detail.
-- This metatable appends the new key to itself, then returns itself
stringtablemeta = {}
function stringtablemeta.__index(self, key)
table.insert(self, key)
return self
end
-- In response to the question in the comments:
function stringtablemeta.__tostring(self)
local str = ""
for i, v in ipairs(self) do
if i > 1 then str = str .. "-" end
str = str .. v
end
return str
end
-- This metatable creates a new table, with stringmetatable as its metatable
mytablemeta = {}
function mytablemeta.__index(self, key)
local temp = { key }
setmetatable(temp, stringtablemeta)
return temp
end
-- set mytable to have mymetatable as it's metatable. This makes it so when
-- you index into it, it will call the mytablemeta.__index method.
--
-- That will return a talb with a single string, the key that was passed
-- in. that table will have it's own metatable, the stringmetatable, which
-- will cause it to append keys that are called on it with its own __index
-- metamethod
mytable = {}
setmetatable(mytable, mytablemeta)
test = mytable.str1.str2.str3
for k, v in pairs(test) do
print(k, v)
end
It can't. Not without having a metatable on each of those tables.
mytable is a table. str1 is a different table. So you can do the same thing by doing this:
local temp = mytable.str1
temp.str2.str3
And as far as Lua is concerned, these are equivalent. Therefore, the only way to know what was done at each stage is to give all of them a special metatable. How you concatenate the different values into a table is something you'll have to investigate on your own.
As Nicol said, you cannot do that directly in Lua. However, by returning specially crafted tables, you can achieve a similar result to what you want. Take a look at AutomagicTables at the Lua-users Wiki for inspiration.