I need clarification on Metatable.__index - lua

Question

I asked earlier why my methods for a metatable weren't being located by Lua, and was told that by setting __index to my metatable, that it would resolve the issue, so I assumed that a method when called was searching by index in the metatable, but I've ran into an issue now that I need to use indexing brackets [ and ] on my metatable, so __indexis assigned to return an index from a table inside of it, how do I resolve the functionality needs of both using methods, and use of indexing brackets
I wrote a minimal example indicating the problem:
TestMetatable = {DataTable = {}}
TestMetatable.__index = TestMetatable
function TestMetatable.new()
local Tmp = {}
setmetatable(Tmp,TestMetatable)
Tmp.DataTable = {1}
return Tmp
end
function TestMetatable:TestMethod()
print("Ran Successfully")
end
function TestMetatable.__index(self,index)
return self.DataTable[index]
end
local Test = TestMetatable.new()
-- both functionalities are needed
print(Test[1])
Test:TestMethod()

Answer 1

You need to understand the difference between __index and __newindex, and their relationship with the current contents of the main table.
__newindex is only called/accessed when all the following are true:
When you are setting a value into the main table, via tbl[index] = expr (or equivalent syntax, like tbl.name = expr).
When the key you are trying to set into the main table does not already exist in the main table.
The second one trips people up often. And that's your problem here, because __index is only accessed when:
When the key being read from the main table does not already exist in the main table.
So if you want to filter every read from and write to a table, then that table must always be empty. Therefore, those reads and writes need to go into some other table you create for each new object. So your new function needs to create two tables: one that remains empty and one that has all the data in it.
Honestly, I wish Lua had a way to create just an empty piece of userdata that you could bind a user-defined metatable to, just to avoid these issues.

Answer 2

the way I resolved this problem, according to Nicol Bolas's solution, if it might give clarity to anyone else's confusion :-)
TestMetatable = {DataTable = {}, FunctionTable = {}}
function TestMetatable.new()
local Tmp = {}
setmetatable(Tmp,TestMetatable)
Tmp.DataTable = {1}
Tmp.FunctionTable = TestMetatable
return Tmp
end
function TestMetatable:TestMethod()
print("Ran Successfully")
end
function TestMetatable.__index(self,index)
if type(index) == "string" then
return self.FunctionTable[index]
else
return self.DataTable[index]
end
end
local Test = TestMetatable.new()
-- both functionalities are needed
print(Test[1])
Test:TestMethod()