How can I loop through nested tables in Lua when the nested tables are mixed in with other data types? - lua

I'm trying loop though a very large table in Lua that consists of mixed data types many nested tables. I want to print the entire data table to the console, but I'm having trouble with nested loops. When I do a nested loop to print the next level deep Key Value pairs I get this error bad argument #1 to 'pairs' (table expected, got number) because not all values are tables.
I've tried adding a if type(value) == table then before the nested loop but it never triggers, because type(value) returns userdata whether they are ints, strings or tables.
EDIT: I was wrong, only tables are returned as type userdata
My table looks something like this but hundreds of pairs and can be several nested tables. I have a great built in method printall() with the tool I'm using for this but it only works on the first nested table. I don't have any control over what this table looks like, I'm just playing with a game's data, any help is appreciated.
myTable = {
key1 = { value1 = "string" },
key2 = int,
key3 = { -- printall() will print all these two as key value pairs
subKey1 = int,
subKey2 = int
},
key4 = {
innerKey1 = { -- printall() returns something like : innerKey1 = <int32_t[]: 0x13e9dcb98>
nestedValue1 = "string",
nestedValue2 = "string"
},
innerKey2 = { -- printall() returns something like : innerKey2 = <vector<int32_t>[41]: 0x13e9dcbc8>
nestedValue3 = int,
nestedValue4 = int
}
},
keyN = "string"
}
My loop
for key, value in pairs(myTable) do
print(key)
printall(value)
for k,v in pairs(value) do
print(k)
printall(v)
end
end
print("====")
end
ANSWER : Here is my final version of the function that fixed this, it's slightly modified from the answer Nifim gave to catch edge cases that were breaking it.
function printFullObjectTree(t, tabs)
local nesting = ""
for i = 0, tabs, 1 do
nesting = nesting .. "\t"
end
for k, v in pairs(t) do
if type(v) == "userdata" then -- all tables in this object are the type `userdata`
print(nesting .. k .. " = {")
printFullObjectTree(v, tabs + 1)
print(nesting .. "}")
elseif v == nil then
print(nesting .. k .. " = nil")
elseif type(v) == "boolean" then
print(nesting .. k .. " = " .. string.format("%s", v))
else
print(nesting .. k .. " = " .. v)
end
end
end

type(value) returns a string representing the type of value
More information on that Here:
lua-users.org/wiki/TypeIntrospection
Additionally your example table has int as some of the values for some keys, as this would be nil those keys are essentially not part of the table for my below example i will change each instance of int to a number value.
It would also make sense to recurse if you hit a table rather than making a unknown number of nested loops.
here is an example of working printAll
myTable = {
key1 = { value1 = "string" },
key2 = 2,
key3 = { -- printall() will print all these two as key value pairs
subKey1 = 1,
subKey2 = 2
},
key4 = {
innerKey1 = { -- printall() returns something like : innerKey1 = <int32_t[]: 0x13e9dcb98>
nestedValue1 = "string",
nestedValue2 = "string"
},
innerKey2 = { -- printall() returns something like : innerKey2 = <vector<int32_t>[41]: 0x13e9dcbc8>
nestedValue3 = 3,
nestedValue4 = 4
}
},
keyN = "string"
}
function printAll(t, tabs)
local nesting = ""
for i = 0, tabs, 1 do
nesting = nesting .. "\t"
end
for k, v in pairs(t) do
if type(v) == "table" then
print(nesting .. k .. " = {")
printAll(v, tabs + 1)
print(nesting .. "}")
else
print(nesting .. k .. " = " .. v)
end
end
end
print("myTable = {")
printAll(myTable, 0)
print("}")

Related

Deserialization of simple Lua table stored as string

I'm transfering a lua table literal in a string from a web application in to PICO-8 that I'm trying to deserialize back in to a lua table in PICO-8.
The string is in the form '{"top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}'
To try and keep things simple I'm only going to include lowercase characters in the strings and only strings are allowed in the nested tables.
I feel like I've got a grasp on parsing the characters, but I don't know how to keep track of where I am in the recreated data, both the depth of the structure and the index.
How is this usually done?
The catch is that as PICO-8 lua doesn't contain load or loadstring the parsing has to be done manually. The following code is using table.insert and string.sub instead of the PICO-8 equivalents because I'm using a lua REPL to help prototype this code.
Here is what I have so far with print statements what I think I need to do where.
Any help would be greatly appreciated.
test_obj = {"top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}
data_string = '{"top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}'
data = nil
string = ''
level = 0
while #data_string > 0 do
local d=string.sub(data_string,1,1)
if stringChar(d) then
string = string..d
end
if comma(d) then
print(string)
table.insert(data, string)
string = ''
end
if openBracket(d) then
if data == nil then
data = {}
print('new table')
else
print('insert table')
end
level = level + 1
print('on level', level)
end
if closeBracket(d) then
print('end of table')
level = level - 1
print('up to level', level)
end
data_string=string.sub(data_string,2)
end
Use Lua patterns to avoid parsing each character
local function read_exp_list(s)
local exps, res = {}, {}
local function save(v)
exps[#exps + 1] = v
return ('\0'):rep(#exps)
end
s = s:gsub('%b{}', function(s) return save{read_exp_list(s:sub(2, -2))} end) -- arrays
s = s:gsub('"(.-)"', save) -- strings
s = s:gsub('%-?%d+', function(s) return save(tonumber(s)) end) -- integer numbers
for k in s:gmatch'%z+' do
res[#res + 1] = exps[#k]
end
return (table.unpack or unpack)(res)
end
local data_string = '{-42, "top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}'
local obj = read_exp_list(data_string)
-- obj == {-42, "top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}
Strings must be enclosed in " and must not contain characters {}\ inside. String may be empty.
Numbers must be integer in decimal notation with optional minus.
Arrays must contain only strings, numbers and subarrays. Array may be empty.
UPDATE:
The following code only uses functions string.sub, table.insert, tonumber, type
local function is_digit(c)
return c >= '0' and c <= '9'
end
local function read_from_string(input)
if type(input) == 'string' then
local data = input
local pos = 0
function input(undo)
if undo then
pos = pos - 1
else
pos = pos + 1
return string.sub(data, pos, pos)
end
end
end
local c
repeat
c = input()
until c ~= ' ' and c ~= ','
if c == '"' then
local s = ''
repeat
c = input()
if c == '"' then
return s
end
s = s..c
until c == ''
elseif c == '-' or is_digit(c) then
local s = c
repeat
c = input()
local d = is_digit(c)
if d then
s = s..c
end
until not d
input(true)
return tonumber(s)
elseif c == '{' then
local arr = {}
local elem
repeat
elem = read_from_string(input)
table.insert(arr, elem)
until not elem
return arr
end
end
local data_string = '{-42, "top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}'
local obj = read_from_string(data_string)
-- obj == {-42, "top", {"one", {"one a", "one b"}}, {"two", {"two a", "two b"}}}
Strings must be enclosed in " and must not contain character \ inside. String may be empty.
Numbers must be integer in decimal notation with optional minus.
Arrays must contain only strings, numbers and subarrays. Array may be empty.

Lua 4 "n" property of tables

In Lua 4, many tables have an "n" property which tracks the number of items inside the table.
Do all tables have this property?
Can it be overridden?
I ask, because I'm trying to develop a routine that prints all of a table's elements recursively in valid Lua syntax, and want to know if it's safe to filter all "n" items out of the result?
Thanks.
[edit]
Here's the script:
-- ThoughtDump v1.4.0
-- Updated: 2017/07/25
-- *****************
-- Created by Thought (http://hw2.tproc.org)
-- Updated by Mikali
-- DESCRIPTION
-- ***********
-- Parses the globals table and __TDPrints its contents to "HW2.log".
-- Can also be used to parse (i.e., pretty-print) generic tables in some cases.
-- Note: functions & variables must actually be declared in order to be parsed.
-- Otherwise, they are ignored.
-- Note: if parsing a table other than the globals table, the __TDPrinted table
-- values may be in a different order than was originally written. Values with
-- numerical indices are moved to the "top" of the table, followed by values
-- with string indices, followed by tables. Functions appear in different
-- locations, depending on whether they are indexed using a number or a string.
-- Note: despite the fact that nil values cannot be stored in tables, they are
-- still handled.
-- Note: even though functions may be referenced within tables, a function will
-- only be parsed correctly if it is indexed using a string that is the same as
-- the name of the function.
__TDOutputString = ""
function __TDParse(name, value, level, verbose, numbers, collapse)
if ((name == "__TDParse") or (name == "__TDSortHash") or (name == "__TDPrint") or (name == "__TDPrintGlobals()") or (name == "__TDOutputString")) then
return
end
local Element = nil
local ValType = type(value)
local NamType = type(name)
local PreLevel = ""
if (collapse == 0) then
for i = 1, level do
PreLevel = PreLevel .. "\t"
end
end
local ComLevel = ""
if (level ~= 0) then
ComLevel = ","
end
if ((ValType == "function") or (ValType == "userdata")) then
if (NamType == "string") then
Element = PreLevel .. name .. " = " .. name .. ComLevel
elseif (numbers == 1) then
Element = PreLevel .. "[" .. name .. "] = " .. name .. ComLevel
else
Element = PreLevel .. name .. ComLevel
end
elseif (ValType == "string") then
if (NamType == "string") then
Element = PreLevel .. name .. " = \"" .. value .. "\"" .. ComLevel
elseif (numbers == 1) then
Element = PreLevel .. "[" .. name .. "] = \"" .. value .. "\"" .. ComLevel
else
Element = PreLevel .. "\"" .. value .. "\"" .. ComLevel
end
elseif (ValType == "number") then
if (NamType == "string") then
Element = PreLevel .. name .. " = " .. value .. ComLevel
elseif (numbers == 1) then
Element = PreLevel .. "[" .. name .. "] = " .. value .. ComLevel
else
Element = PreLevel .. value .. ComLevel
end
elseif (ValType == "table") then
if (NamType == "string") then
Element = PreLevel .. name .. " ="
elseif (numbers == 1) then
Element = PreLevel .. "[" .. name .. "] ="
else
Element = ""
end
elseif (ValType == "nil") then
if (NamType == "string") then
Element = PreLevel .. name .. " = nil" .. ComLevel
elseif (numbers == 1) then
Element = PreLevel .. "[" .. name .. "] = nil" .. ComLevel
else
Element = PreLevel .. "nil" .. ComLevel
end
else
Element = PreLevel .. "-- unknown object type " .. ValType .. " for object " .. name
end
if (verbose == 1) then
Element = Element .. " -- " .. ValType .. ", tag: " .. tag(value)
end
if (((ValType == "table") and (NamType == "number") and (numbers == 0)) or (collapse == 1)) then
__TDPrint(Element, 0)
else
__TDPrint(Element, 1)
end
if (ValType == "table") then
__TDPrint(PreLevel .. "{", collapse == 0)
__TDSortHash(__TDParse, value, level + 1, verbose, numbers, collapse)
__TDPrint(PreLevel .. "}" .. ComLevel, 1)
end
end
function __TDSortHash(func, tabl, level, verbose, numbers, collapse)
local typesarray = {}
local typescount = {}
local keycount = 1
local keyarray = {}
for i, iCount in tabl do
local thistype = type(iCount)
if not (typesarray[thistype]) then
typescount[thistype] = 0
typesarray[thistype] = {}
end
typescount[thistype] = typescount[thistype] + 1
typesarray[thistype][typescount[thistype]] = i
end
sort(typesarray)
for i, iCount in typesarray do
sort(iCount)
for j, jCount in iCount do
keyarray[keycount] = tostring(jCount)
keycount = keycount + 1
end
end
for i, iCount in keyarray do
local tempcount = tonumber(iCount)
if (tempcount) then
iCount = tempcount
end
func(iCount, tabl[iCount], level, verbose, numbers, collapse)
end
end
function __TDPrint(instring, newline)
__TDOutputString = __TDOutputString .. instring
if (newline == 1) then
__TDOutputString = __TDOutputString .. "\n"
end
end
function __TDPrintGlobals()
__TDOutputString = ""
__TDPrint("globals =", 1)
__TDPrint("{", 1)
__TDSortHash(__TDParse, globals(), 1, 0, 0, 0)
__TDPrint("}\n", 1)
local WriteFile = "$test_globals_write.lua"
writeto(WriteFile)
write(__TDOutputString)
writeto()
end
__TDPrintGlobals()
In Lua 4.x in tables, n is just an element of the table like any other elements a table could contain, but it's not part of the table mechanism itself.
So, it can be overwritten or removed.
Some functions use it, like tinsert() ,and other table functions:
local tbl = { n=0 }
tinsert(tbl, 123)
print(tbl.n) --> 1
This is very useful as the getn() function gives only the highest number index of the table. But if there is only named elements in the table or a mix or number indexes and named indexes, then getn() doesn't reflect the real number of elements in the table. If elements are always inserted (or removed) using table functions like tinsert() then n is the accurate number of elements in the table.
Lua 4.x --> Lua 5.x equivalent:
getn(tbl) #tbl
tinsert(tbl,e) table.insert(tbl,e) or tbl:insert(e)
Of course you can still add elements in a table using the simple table access. But as n can be very useful, try to keep it updated as well.
tbl["Bla"] = 234
tbl.Bli = 345
tbl.n = tbl.n + 2
If n doesn't exist in a table but is needed by the code somewhere, it can be added using the for loop:
local tbl = {1,2,3,4,5,6}; tbl.a=11; tbl.b=22; tbl.c=33
local n = 0
for ie, e in tbl do
n = n + 1
end
tbl.n = n
or the foreach loop:
local tbl = {1,2,3,4,5,6}; tbl.a=11; tbl.b=22; tbl.c=33
tbl.n = 0
foreach(tbl, function() %tbl.n = %tbl.n + 1 end )
Note 1: The initialization of tbl.n to 0 will give the number of elements in the table, including n. Here the result of tbl.n is 10.
As eventually we don't want n to be counted as a real element of the table (which is is) but only counting other elements, we should init n to -1 then.
Note 2: The Lua 4.x upvalue operator % is used here because the tbl variable is not reachable in the function (not in the scope) for the foreach loop. It can be reached using %tbl. However, a upvalue is always read only, so the tbl variable can't be changed. The following will generate an error in the function:
%tbl = { } -- change the reference to another table
%tbl = 135 -- change the ref to the table for a number (or a string, ...)
As tbl variable contains in fact a reference to a table, the referenced table can be modifiable and therefore the element n can be changed without problem (as well as other elements of the table).
%tbl.n = %tbl.n + 1 -- increment the element n of the referenced table
Note 3: A global variable tbl could have been used, but it's good practice to always use local variable. Access to local variables is also faster than global.
Not all tables have this property.
It can be overwritten.
Why not traverse the table using a for loop? Or if possible, use Lua 5.3 ;)
In Lua this was called table for loop, in modern Lua it's called generic for loop.
The table for statement traverses all pairs (index,value) of a given
table. It has the following syntax:
stat ::= for name `,' name in exp1 do block end
A for statement like
for index, value in exp do block end
is equivalent to the code:
do
local _t = exp
local index, value = next(t, nil)
while index do
block
index, value = next(t, index)
end
end
Note the following:
_t is an invisible variable. The name is here for explanatory purposes only.
The behavior is undefined if you assign to index inside the block.
The behavior is undefined if you change the table _t during the traversal.
The variables index and value are local to the statement; you cannot use their values after the for ends.
You can use break to exit a for. If you need the value of index or value, assign them to other variables before breaking.
The order that table elements are traversed is undefined, even for numerical indices. If you want to traverse indices in numerical order,
use a numerical for.
Refer to the Lua manual 4.4.4
https://www.lua.org/manual/4.0/manual.html#4.4

Lua: Table inside Object

I'm using OO abstraction in LUA and I'm having trouble with tables inside objects.
I'm using this approach to implement OO support: Object Orientation Tutorial
If i define a table inside the Class like this:
fields = {}
It will be shared by all instances of that Class. Something like an static attribute in Java. However, I'd like to get a normal class attribute instead.
That sounds odd to me, since it works correctly for non-table items like this:
attribute = 0
That is not going to be shared between all instances.
Here is a complete example:
local function C1Constructor( self )
print( "C1: Constructor... " )
self.fields = {}
end
local function C1SetName( self, name )
self.name = name
end
local function C1Add( self, name, value )
print( "C1: (" .. self.name .. ") :: Add: " .. name .. " = " .. value )
self.fields[ name ] = value
end
local function C1Show( self )
print( "C1: (" .. self.name .. ") :: Show:" )
for name, value in pairs( self.fields ) do
print(" " .. name .. " = " .. value )
end
end
C1 = {
name = "",
constructor = C1Constructor,
setName = C1SetName,
add = C1Add,
show = C1Show,
fields = {},
}
function C1.new( o )
o = o or { }
setmetatable( o, { __index = C1 } )
o:constructor()
return o;
end
c1a = C1.new()
c1b = C1.new()
c1a:setName( "Obj A" )
c1b:setName( "Obj B" )
c1a:show()
c1b:show()
c1a:add( "k1", "v1" )
c1b:add( "k2", "v2" )
c1a:show()
c1b:show()
I know how to fix it. But I don't know what is happening behind the scenes:
local function C1Constructor( self )
print( "C1: Constructor... " )
self.fields = {}
end
What am I missing here?
Thanks very much,
Tables are referenced values. When you store a table in a variable you are storing a reference to the table and not a primitive value (like for a string or a number). As such if you create one table and share it among instances you only have one table. When you create a table per-instance (e.g. in your constructor) then you have one table per-instance and not a shared table.

LPeg Increment for Each Match

I'm making a serialization library for Lua, and I'm using LPeg to parse the string. I've got K/V pairs working (with the key explicitly named), but now I'm going to add auto-indexing.
It'll work like so:
#"value"
#"value2"
Will evaluate to
{
[1] = "value"
[2] = "value2"
}
I've already got the value matching working (strings, tables, numbers, and Booleans all work perfectly), so I don't need help with that; what I'm looking for is the indexing. For each match of #[value pattern], it should capture the number of #[value pattern]'s found - in other words, I can match a sequence of values ("#"value1" #"value2") but I don't know how to assign them indexes according to the number of matches. If that's not clear enough, just comment and I'll attempt to explain it better.
Here's something of what my current pattern looks like (using compressed notation):
local process = {} -- Process a captured value
process.number = tonumber
process.string = function(s) return s:sub(2, -2) end -- Strip of opening and closing tags
process.boolean = function(s) if s == "true" then return true else return false end
number = [decimal number, scientific notation] / process.number
string = [double or single quoted string, supports escaped quotation characters] / process.string
boolean = P("true") + "false" / process.boolean
table = [balanced brackets] / [parse the table]
type = number + string + boolean + table
at_notation = (P("#") * whitespace * type) / [creates a table that includes the key and value]
As you can see in the last line of code, I've got a function that does this:
k,v matched in the pattern
-- turns into --
{k, v}
-- which is then added into an "entry table" (I loop through it and add it into the return table)
Based on what you've described so far, you should be able to accomplish this using a simple capture and table capture.
Here's a simplified example I knocked up to illustrate:
lpeg = require 'lpeg'
l = lpeg.locale(lpeg)
whitesp = l.space ^ 0
bool_val = (l.P "true" + "false") / function (s) return s == "true" end
num_val = l.digit ^ 1 / tonumber
string_val = '"' * l.C(l.alnum ^ 1) * '"'
val = bool_val + num_val + string_val
at_notation = l.Ct( (l.P "#" * whitesp * val * whitesp) ^ 0 )
local testdata = [[
#"value1"
#42
# "value2"
#true
]]
local res = l.match(at_notation, testdata)
The match returns a table containing the contents:
{
[1] = "value1",
[2] = 42,
[3] = "value2",
[4] = true
}

how to access data in such table?

I'm writing a program with lua. I have data that organized in the following way:
t= {
i1 = {
p1 = { value = "i1p1" },
p2 = { value = "i1p2" },
-- etc
pm = { value = "i1pm" }
},
i2 = {
p1 = { value = "i2p1" },
p2 = { value = "i2p2" },
-- etc
pm = { value = "i2pm" }
},
im = {
p1 = { value = "imp1" },
p2 = { value = "imp2" },
-- etc
pm = { value = "impm" }
}
} --(inner tables)
In another way each group of data is indexed by two variables i&p,I am sure that the data is kept correctly but I want a way to print the data from their tables because I won't know the values of i and p to iterate over them or even the numbers n & m any body know how to do this with lua?
If you know the depth of your nested (inner) tables, you can iterate explicitly to that depth:
for k1,v1 in pairs(t) do
for k2,v2 in pairs(v1) do
for k3, v3 in pairs(v2) do
print(k3, ":", v3)
end
end
end
Alternatively, you can recursively iterate into your nested structure:
function print_tbl(tbl)
if type(tbl) == "table" then
for _,v in pairs(tbl) do
print_tbl(v)
end
else
print(tbl)
end
end
print_tbl(t)
This is just an example. If your tables contain functions, contains userdata, or your nesting has cycles, you'll need a different approach. Take a look at table serialization on the Lua user wiki. Serialization requires sensible handling of tables with nesting, functions, cycles, etc. You may be able to use lessons learned on your data.

Resources