Finding duplicates in a multi-dimensional table - lua

Question

A slightly altered version of the below permitted me to filter unique field values out of a multi-dimensional table (dictionary style).
[ url ] http://rosettacode.org/wiki/Remove_duplicate_elements#Lua
items = {1,2,3,4,1,2,3,4,"bird","cat","dog","dog","bird"}`
flags = {}
io.write('Unique items are:')
for i=1,#items do
if not flags[items[i]] then
io.write(' ' .. items[i])
flags[items[i]] = true
end
end
io.write('\n')`
What I'm lost at is what 'if not ... then ... end' part actually does. To me this is sillyspeak but hey, it works ;-) Now i want to know what happens under the hood.
I hope multi-dimensional does not offend anyone, I'm referring to a table consisting of multiple row containing multiple key-value pairs on each row.
Here's the code i'm using, no brilliant adaptation but good enough to filter unique values on a fieldname
for i=1,#table,1 do
if not table2[table[i].fieldname] then
table2[table[i].fieldname] = true
end
end
for k,v in pairs(table2) do
print(k)
end

Answer 1

function findDuplicates(t)
seen = {} --keep record of elements we've seen
duplicated = {} --keep a record of duplicated elements
for i = 1, #t do
element = t[i]
if seen[element] then --check if we've seen the element before
duplicated[element] = true --if we have then it must be a duplicate! add to a table to keep track of this
else
seen[element] = true -- set the element to seen
end
end
return duplicated
end
The logic behind the if seen[element] then, is that we check if we've already seen the element before in the table. As if they key doesn't exist nill will be returned which is evaluated which is false (this is not the same as boolean false, there are two types of false in lua!).
You can use this function like so:
t = {'a','b','a','c','c','c','d'}
for key,_ in pairs(findDuplicates(t)) do
print(key)
end
However that function won't work with multidimensional tables, this one will however:
function findDuplicates(t)
seen = {} --keep record of elements we've seen
duplicated = {} --keep a record of duplicated elements
local function traverse(subt)
for i=1, #subt do
element = subt[i]
if type(element) == 'table' then
traverse(element)
else
if seen[element] then
duplicated[element] = true
else
seen[element] = true
end
end
end
end
traverse(t)
return duplicated
end
Example useage:
t = {'a',{'b','a'},'c',{'c',{'c'}},'d'}
for k,_ in pairs(findDuplicates(t)) do
print(k)
end
Outputs
a
c
t = {a='a',b='b',c='c',d='c',e='a',f='d'}
function findDuplicates(t)
seen = {}
duplicated = {}
for key,val in pairs(t) do
if seen[val] then
duplicated[val] = true
else
seen[val] = true
end
end
return duplicated
end
This works the same way as before but checks if the same value is associated with a different key and if so, makes note of that value as being duplicated.

Answer 2

Eventually this is the code which worked for me. I've been asked to post it as a separate answer so here goes.
for i=1,#table1,1 do
if not table2[table1[i].fieldname] then
table2[table1[i].fieldname] = true
end
end
for k,v in pairs(table2) do
print(k)
end