Lua metatable Objects cannot be purge from memory? - lua

I'm using a proprietary platform that reported memory usage in realtime on screen. I decided to use a Class.lua I found on http://lua-users.org/wiki/SimpleLuaClasses
However, I noticed memory issues when purging object created by this using a simple Account class. Specifically, I would start with say 146k of memory used, create 1000 objects of a class that just holds an integer instance variable and store each object into a table.
The memory used is now 300k
I would then exit, iterating through the table and setting each element in the table to nil. But would never get back the 146k, usually after this I am left using 210k or something similar. If I run the load sequence again during the same session, it does not exceed 300k so it is not a memory leak.
I have tried creating 1000 integers in a table and setting these to nil, which does give me back 146k.
In addition I've tried a simpler class file (Account2.lua) that doesn't rely on a class.lua. This still incurs memory fragmentation but not as much as the one that uses Class.lua
Can anybody explain what is going on here? How can I purge these objects and get back the memory?
here is the code
--------Class.lua------
-- class.lua
-- Compatible with Lua 5.1 (not 5.0).
--http://lua-users.org/wiki/SimpleLuaClasses
function class(base,ctor)
local c = {} -- a new class instance
if not ctor and type(base) == 'function' then
ctor = base
base = nil
elseif type(base) == 'table' then
-- our new class is a shallow copy of the base class!
for i,v in pairs(base) do
c[i] = v
end
c._base = base
end
-- the class will be the metatable for all its objects,
-- and they will look up their methods in it.
c.__index = c
-- expose a ctor which can be called by ()
local mt = {}
mt.__call = function(class_tbl,...)
local obj = {}
setmetatable(obj,c)
if ctor then
ctor(obj,...)
else
-- make sure that any stuff from the base class is initialized!
if base and base.init then
base.init(obj,...)
end
end
return obj
end
c.init = ctor
c.instanceOf = function(self,klass)
local m = getmetatable(self)
while m do
if m == klass then return true end
m = m._base
end
return false
end
setmetatable(c,mt)
return c
end
--------Account.lua------
--Import Class template
require 'class'
local classname = "Account"
--Declare class Constructor
Account = class(function(acc,balance)
--Instance variables declared here.
if(balance ~= nil)then
acc.balance = balance
else
--default value
acc.balance = 2097
end
acc.classname = classname
end)
--------Account2.lua------
local account2 = {}
account2.classname = "unnamed"
account2.balance = 2097
-----------Constructor 1
do
local metatable = {
__index = account2;
}
function Account2()
return setmetatable({}, metatable);
end
end
--------Main.lua------
require 'Account'
require 'Account2'
MAX_OBJ = 5000;
test_value = 1000;
Obj_Table = {};
MODE_ACC0 = 0 --integers
MODE_ACC1 = 1 --Account
MODE_ACC2 = 2 --Account2
TEST_MODE = MODE_ACC0;
Lua_mem = 0;
function Load()
for i=1, MAX_OBJ do
if(TEST_MODE == MODE_ACC0 )then
table.insert(Obj_Table, test_value);
elseif(TEST_MODE == MODE_ACC1 )then
table.insert(Obj_Table, Account(test_value)); --Account.lua
elseif(TEST_MODE == MODE_ACC2 )then
table.insert(Obj_Table, Account2()); --Account2.lua
Obj_Table[i].balance = test_value;
end
end
end
function Purge()
--metatable purge
if(TEST_MODE ~= MODE_ACC0)then
--purge stage 0:
print("set each elements metatable to nil")
for i=1, MAX_OBJ do
setmetatable(Obj_Table[i], nil);
end
end
--purge stage 1:
print("set table element to nil")
for i=1, MAX_OBJ do
Obj_Table[i] = nil;
end
--purge stage 2:
print("start table.remove...");
for i=1, MAX_OBJ do
table.remove(Obj_Table, i);
end
print("...end table.remove");
--purge stage 3:
print("create new object_table {}");
Obj_Table= {};
--purge stage 4:
print("collectgarbage('collect')");
collectgarbage('collect');
end
--Loop callback, called every tick
function OnUpdate()
Lua_mem = collectgarbage('count');
collectgarbage('collect');
end
--Loop rendering callback
function OnRender()
DrawText(Lua_mem );
end
-------------------
--NOTE:
--code starts in idle awaiting input from user
--On first input, runs Load(), on exit runs Purge()
--Where DrawText() draws the string parameter passed, to screen.
--Update
I've updated the code with suggestions from comments below, and will post my findings later today.
--Update 2
Well I've tried the above code, and it does seem collectgarbage("count") reports lua giving me back all the memory in all three scenarios.
Here are my results for collectgarbage('count')
ACC0
- On start: 25.567K used
- On Load: 89.334K used
- On Purge: 25.567K used
ACC1
- On start: 25.567K used
- On Load: 440.567k used
- On Purge: 25.567K used
ACC2
- On start: 25.327K used
- On Load: 245.34K used
- On Purge: 25.327K used

You need to force the garbage collector to reclaim the memory (see collectgarbage("collect")).

Main.lua:
--purge stage 0:
print("set each elements metatable to nil")
for i=1, MAX_OBJ do
setmetatable(Obj_Table[i], nil);
end
According to the documentation of Lua you can not reset a tables metatable by trying to set it to nil. It just equals the function getmetatable(). So the objects keep being linked.

Most dynamic languages don't really release memory back to system, but keep it ready for future allocations instead. Try to create some objects after Lua reported memory decrease - Lua should use this freed memory it kept to yourself and platform memory consumption won't increase.

Related

LUA: Looking for a specific table by its variable

I'm currently starting work on a text adventure game in Lua--no addons, just pure Lua for my first project. In essence, here is my problem; I'm trying to find out how I can do a "reverse lookup" of a table using one of its variables. Here's an example of what I've tried to do:
print("What are you trying to take?")
bag = {}
gold = {name="Gold",ap=3}
x = io.read("*l")
if x == "Gold" then
table.insert(bag,gold)
print("You took the " .. gold.name .. ".")
end
Obviously, writing a line like this with every single object in the game would be very... exhausting--especially since I think I'll be able to use this solution for not just taking items but movement from room to room using a reverse lookup with each room's (x,y) coordinates. Anyone have any ideas on how to make a more flexible system that can find a table by the player typing in one of its variables? Thanks in advance!
-blockchainporter
This doesn't directly answer your question as you asked it, but I think it would serve the purpose of what you are trying to do. I create a table called 'loot' which can hold many objects, and the player can place any of these in their 'bag' by typing the name.
bag = {}
loot = {
{name="Gold", qty=3},
{name="Axe", qty=1},
}
print("What are you trying to take?")
x = io.read("*l")
i = 1
while loot[i] do
if (x == loot[i].name) then
table.insert(bag, table.remove(loot,i))
else
i = i + 1
end
end
For bonus points, you could check 'bag' to see if the player has some of that item already and then just update the quantity...
while loot[i] do
if (x == loot[i].name) then
j, found = 1, nil
while bag[j] do
if (x == bag[j].name) then
found = true
bag[j].qty = bag[j].qty + loot[i].qty
table.remove(loot,i)
end
j = j + 1
end
if (not found) then
table.insert(bag, table.remove(loot,i))
end
else
i = i + 1
end
end
Again, this isn't a 'reverse lookup' solution like you asked for... but I think it is closer to what you are trying to do by letting a user choose to loot something.
My disclaimer is that I don't use IO functions in my own lua usage, so I have to assume that your x = io.read("*l") is correct.
PS. If you only ever want objects to have a name and qty, and never any other properties (like condition, enchantment, or whatever) then you could also simplify my solution by using key/val pairs:
bag = {}
loot = { ["Gold"] = 3, ["Axe"] = 1 }
print("What are you trying to take?")
x = io.read("*l")
for name, qty in pairs(loot) do
if x == name then
bag.name = (bag.name or 0) + qty
loot.name = nil
end
end
I have a few notes to start before I specifically address your question. (I just want to do this before I forget, so please bear with me!)
I recommend printing to the terminal using stderr instead of stdout--the Lua function print uses the latter. When I am writing a Lua script, I often create a C-style function called eprintf to print formatted output to stderr. I implement it like this:
local function eprintf(fmt, ...)
io.stderr:write(string.format(fmt, ...))
return
end
Just be aware that, unlike print, this function does not automatically append a newline character to the output string; to do so, remember to put \n at the end of your fmt string.
Next, it may be useful to define a helper function that calls io.read("*l") to get an entire line of input. In writing some example code to help answer your question, I called my function getline--like the C++ function that has similar behavior--and defined it like this:
local function getline()
local read = tostring(io.read("*l"))
return read
end
If I correctly understand what it is you are trying to do, the player will have an inventory--which you have called bag--and he can put items into it by entering item names into stdin. So, for instance, if the player found a treasure chest with gold, a sword, and a potion in it and he wanted to take the gold, he would type Gold into stdin and it would be placed in his inventory.
Based on what you have so far, it looks like you are using Lua tables to create these items: each table has a name index and another called ap; and, if a player's text input matches an item's name, the player picks that up item.
I would recommend creating an Item class, which you could abstract nicely by placing it in its own script and then loading it as needed with require. This is a very basic Item class module I wrote:
----------------
-- Item class --
----------------
local Item = {__name = "Item"}
Item.__metatable = "metatable"
Item.__index = Item
-- __newindex metamethod.
function Item.__newindex(self, k, v)
local err = string.format(
"type `Item` does not have member `%s`",
tostring(k)
)
return error(err, 2)
end
-- Item constructor
function Item.new(name_in, ap_in)
assert((name_in ~= nil) and (ap_in ~= nil))
local self = {
name = name_in,
ap = ap_in
}
return setmetatable(self, Item)
end
return Item
From there, I wrote a main driver to encapsulate some of the behavior you described in your question. (Yes, I know my Lua code looks more like C.)
#!/usr/bin/lua
-------------
-- Modules --
-------------
local Item = assert(require("Item"))
local function eprintf(fmt, ...)
io.stderr:write(string.format(fmt, ...))
return
end
local function printf(fmt, ...)
io.stdout:write(string.format(fmt, ...))
return
end
local function getline()
local read = tostring(io.read("*l"))
return read
end
local function main(argc, argv)
local gold = Item.new("Gold", 3)
printf("gold.name = %s\ngold.ap = %i\n", gold.name, gold.ap)
return 0
end
main(#arg, arg)
Now, as for the reverse search which you described, at this point all you should have to do is check the user's input against an Item's name. Here it is in the main function:
local function main(argc, argv)
local gold = Item.new("Gold", 3)
local bag = {}
eprintf("What are you trying to take? ")
local input = getline()
if (input == gold.name) then
table.insert(bag, gold)
eprintf("You took the %s.\n", gold.name)
else
eprintf("Unrecognized item `%s`.\n", input)
end
return 0
end
I hope this helps!

OOP Help - How do I get this code block to recognize one of its arguments?

I'm learning Lua and how to implement OOP. Trying out a test example of an object seems to return one of the argument of the object as 'null' despite being assigned one.
function Character(Name, Level, Class) --Constructor
return {GetName = T.GetName, GetLevel = T.GetLevel, GetClass = T.GetClass}
end
end
-- Snippets
Player = Character("Bob", 1, "Novice")
When I try printing Player.GetName() it returns null instead of Bob. Where have I gone wrong?
Here is the full code.
OOP in Lua takes a bit more than what you have done there, you will want to make use of metatables and upvalues.
-- How you could define your character structure.
local Character = {}
function Character.GetName(self)
return self.name
end
function Character.new(Name, Level, Class)
local _meta = {}
local _private = {}
_private.name = Name
_private.level = Level
_private.class = Class
_meta.__index = function(t, k) -- This allows access to _private
return rawget(_private, k) or rawget(Character, k)
end
_meta.__newindex = function(t, k, v) -- This prevents the value from being shaded
if rawget(_private, k) or rawget(Character, k) then
error("this field is protected")
else
rawset(t, k, v)
end
end
return setmetatable({}, _meta) --return an empty table with our meta methods implemented
end
This creates a local table _private when you create a new instance of a Character. That local table is an upvalue to the _meta.__index and it cannot be accessed outside the scope of the Character.new function. _private can be accessed when __index is called because it is an upvalue.
-- How to use the character structure
player = Character.new("Bob", 10, "Novice")
npc = Character.new("Alice", 11, "Novice")
print(player:GetName())
I use player:GetName(), but in all honesty you can just do player.name as well.
Resources for more on this topic:
http://tutorialspoint.com/lua/lua_metatables.htm
http://lua-users.org/wiki/ObjectOrientationTutorial

Trying to implement object-oriented programming in Lua, but it's not quite working

Ok, so I'm trying to follow the instructions found here: https://www.lua.org/pil/16.1.html to do something resembling OO programming in Lua (and the LOVE game framework), but it's not working. Here's the core of my code. I have a generic Object class:
local Object = {}
function Object:load(arg)
end
function Object:update(dt)
end
function Object:draw()
end
function Object:new(arg)
o = {}
setmetatable(o, self)
self.__index = self
o:load(arg)
return o
end
return Object
and a Ship class that inherits from it:
Object = require('objects.object')
local Ship = Object:new()
Ship.sprite = love.graphics.newImage('assets/sprites/ship.png')
Ship.sprite:setFilter('nearest', 'nearest', 0)
Ship.px = Ship.sprite:getWidth()/2
Ship.py = Ship.sprite:getHeight()/2
function Ship:load(arg)
self.x = arg.x or 0
self.y = arg.y or 0
self.sx = arg.sx or arg.s or 1
self.sy = arg.sy or arg.s or 1
self.rot = arg.rot or 0
self.tint = arg.tint or {255, 255, 255}
end
function Ship:draw()
love.graphics.setColor(self.tint)
love.graphics.draw(self.sprite, self.x, self.y, self.rot,
self.sx, self.sy, self.px, self.py)
love.graphics.setColor({255, 255, 255})
end
return Ship
Now the problem is, I create two of these Ships as members of another object with this code:
self.ship1 = Ship:new({x=50, y=self.y1, s=2, tint={0, 0.5, 1}})
self.ship2 = Ship:new({x=750, y=self.y2, s=-2, tint={1, 0.5, 0}})
But when I draw them, I only see one - the second. As it turns out, it's like the code above doesn't assign the new instances of Ship to ship1 and ship2, but directly to self, for reasons that I can't understand. Did I do something wrong or is this a weird bug of the interpreter?
It is not bug of the interpreter, it is designed behavior of the language. o={} creates global variable, which is not expected by the programmer here. "Why it is so?" is a frequent question to the language creator. There are many efforts to take control over that behavior simpler .
o = {} without local creates global variable global variable is accessible and shared between all the functions in the program, unless you use fancy environment scoping techniques. Using global variable inside a function opens up doors for various side effects, you should be careful with side effects.
I've removed some of the syntactic sugar and added code above can be equivalently written as follows:
Object.new = function(table_before_colon,arg)
highlander = {} --global variable, there can be only one
setmetatable(highlander,table_before_colon);
table_before_colon.__index = table_before_colon;
highlander:load(arg) -- table_before_colon.__index.load(highlander,arg)
return highlander
end
local Ship = Object:new()
--global highlander == Ship
--Ship.new points to Object.new
function Ship:load(arg)--equivalent to: Ship.load=function(self,arg)
--code that sets fields of the `self` object and is called from within new
self.x = arg.x or 0 -- equivalently highlander.x=arg.x or 0
end
Now, the presence of the global variable would not matter, if nothing happened to it in the period from the start of the new till the new returns. But, apparently, your other code is similar to this:
local OtherObject = Object:new()
--otherObject == highlander
OtherObject.load = function(new_other_obj,arg)
--highlander == new_other_obj
new_other_obj.ship1 = Ship:new({x=50, y=self.y1, s=2, tint={0, 0.5, 1}})
--highlander == new_other_obj.ship1
new_other_obj.ship2 = Ship:new({x=750, y=self.y2, s=-2, tint={1, 0.5, 0}})
--highlander == new_other_obj.ship2
end
So, OtherObject.load calls other functions, and those functions also access and modify the same global variable.
local some_object = OtherObject:new()
returns the global variable as it is at the end of the call, which is last set to the ship2 inside the call to Ship:new inside the call to OtherObject.load inside call to OtherObject:new.
Solved! Apparently, what was needed was this little snippet:
function Object:new(arg)
local o = {}
setmetatable(o, self)
self.__index = self
o:load(arg)
return o
end
Making o local in all my new methods solved everything. I don't know how it worked exactly but I assume having it in the global variable space broke something when the meta tables are set.

Re-initialize table without losing references

I'd like to re-initialize a table without losing references to it.
What I want to achieve is defining tables in files, and when a file is changed (with a text editor) the file is reloaded, changing the table. Of course this doesn't change the table but creates a new instance, old references will still point to the old table.
Any suggestions?
EDIT: I want to elaborate on what I want to achieve. An example with game characters and weapons. I want to modify the weapons.lua and so affect the characters.
-- weapons.lua
sword = { damage = 3 }
-- characters.lua
character = { weapon = sword }
Adding a level of indirection (putting "sword" inside "weapons") like suggested by JWT doesn't help, unless I split character into { weaponTable = weapons, weaponKey = "sword" } but I don't see this as an option.
Anchor everything that needs to survive in the global environment. Nesting is fine, and this doesn't have to be your primary reference. (You can still local things, but make sure to initialize those local variables from the global environment and update the global if you change the local.)
To initialize the global values, say
foo = foo or value -- if foo is always true-ish
bar = (bar == nil) and value or bar -- if bar may be `false`
To initialize or update tables, you can
foo = foo or { }
foo.bar = foo.bar or 23
foo.baz = foo.baz or 42
-- and so on...
but that's kinda icky, so maybe say
function reinit( new, old ) -- (re)initialize one level, prefer old
if old == nil then return new end
if type( old ) ~= "table" then return old end
for k, v in pairs( new ) do
if old[k] == nil then old[k] = v end
end
return old
end
function reset( new, old ) -- (re)initialize one level, prefer new
if old == nil then return new end
if type( old ) ~= "table" then return new end
for k, v in pairs( new ) do old[k] = v end
return old
end
and then just
foo = reinit( { bar = 23, baz = 42 }, foo ) -- only setting if not defined
-- or
foo = reset( { bar = 23, baz = 42 }, foo ) -- always overriding given fields
or maybe make it even more fancy and say
function traverse( t, path )
local here, last, lastk, created = t
-- follow path of several keys starting from t, creating tables as needed
for k in path:gmatch "[^.]+" do
k = tonumber( k ) or k -- convert all-number keys to integer (for arrays)
local next = here[k]
if not next then
next, created = { }, true
here[k] = next
else
created = false
end
lastk, last, here = k, here, next
end
return here, last, lastk, created
end
function repopulate( path, value, update )
update = update or reinit -- pass 'reset' as 'update' for the other behavior
-- or something entirely different if that's what you need
local here, last, lastk, created = traverse( _G, path )
if type( value ) == "table" then
update( value, here )
else
if created then last[lastk] = nil end -- created one level too much
update( { [lastk] = value }, last )
end
end
and then (with arbitrary nesting)
-- No need to create 'state' first if it doesn't exist yet!
-- (If it exists, it will get updated, otherwise it's created)
repopulate( "state.player.weapon", { kind = "sword", damage = 11 } )
-- Do keep in mind that generally update order is relevant -- you may want to
-- give a newly created player a default inventory, but you may not want to
-- "refill" the player's inventory on every reload. So generally `repopulate`
-- with the parent and all child nodes for from-scratch creation, then
-- `repopulate` the child nodes that need to be checked & potentially updated
-- as well.
-- (So here you'd probably repopulate `state.player` first and then
-- `state.player.weapon` or other fields only if they should be updated anyway.)
-- e.g.:
repopulate( "state.player", {
x = 0, y = 0, hp = 100, mp = 100, level = 0, -- and so on
weapon = { kind = "sword", damage = 11 }, -- etc. etc.
} )
-- on reload always force a sword-kind weapon, leave rest (damage, ...) as-is
repopulate( "state.player.weapon", { kind = "sword" }, reset )
-- or alternatively: only if player has no weapon, give them a sword
repopulate( "state.player.weapon", { kind = "sword", damage = 3 } )
And you can go further, add metamethods to hide some of that shuffling, define different update policies, ... – you've seen some of the possibilities, now go and build your own version that fits your style and your code.
(While you're free to use the above code in any way, please note that it was written ad-hoc in the browser. I did some testing, fixed some glitches, and it seems to work now, but don't be surprised if there's still one or two bugs hiding in there. So play with this, change it, break it (and see how/why it breaks), adapt and extend it, ... – but unless you completely understand what it does and can fix any bugs, I strongly suggest you write your own version, or just stick to the basics. You probably don't need everything that this does, and you're likely to need other things that this doesn't do. As this is a central part of the reloading/live-coding infrastructure and everything has to be adapted to be reload-compatible, any mismatch between your tooling and what you actually need will result in a lot of pain everywhere in your code. So if you need something like this, put in a day or two to make it work the way you need it to, or you will regret it.)
(Free bonus warning: If you do OOP, you'll probably have to store and retrieve your classes instead of creating them every time, otherwise old objects from previous iterations will miss code updates and still run their old methods. I've forgotten about that more than just a couple of times and wasted several hours pondering "why isn't it fixed now?!?" after repeatedly re-loading code... So remember to anchor your metatables, anchor your classes!)
You could nest the tables in another table.
Before:
local a = { 1, 2, 3 }
local b = { 7, 8, 9 }
print(a[2] + b[2]) -- #=> 10
After:
local lookup = {
a = { 1, 2, 3 },
b = { 7, 8, 9 }
}
print(lookup.a[2] + lookup.b[2]) -- #=> 10
Then you can fully replace (or just update) a table in the lookup table and any dependent statements will use that updated value:
lookup.a = { 100, 50, 0 }
print(lookup.a[2] + lookup.b[2]) -- #=> 58
I don't know if it's exactly what you needed (As an ID is necessary) but I hope it will fit your needs.
meta = {
tables = {},
__call = function(arg, t)
for k, v in pairs(t) do
arg[k] = v
end
end,
__bnot = function(arg)
return arg.__key
end,
__newindex = function(arg, key, val)
meta.tables[arg.__key][key] = val
end,
__index = function(arg, key)
return meta.tables[arg.__key][key]
end
}
function RefTable(arg)
local newtable = {}
if arg ~= nil then
newtable.__key = arg
setmetatable(newtable, meta)
if meta.tables[arg] == nil then
meta.tables[arg] = {}
end
else
error("RefTable can't have nil key")
end
return newtable
end
-- Using the RefTable
sword = RefTable("Sword")
sword({damage = 3})
sword.cooldown = 10
character = {sword = sword}
print("base", sword.damage, sword.cooldown)
print("embed", character.sword.damage, character.sword.cooldown)
sword = RefTable("Sword")
sword({damage = 8, cooldown = 50})
print("embed2", character.sword.damage, character.sword.cooldown)
print(sword.__key, sword.cooldown)
ref = RefTable("Sword")
ref.cooldown = 1000
print(sword.cooldown)

Can I extract stack values from a parent function in Lua?

My game engine pushes a value on to the lua stack as a parameter to a function and then invokes it using lua_pcall. The lua code will run and call additional lua functions. Eventually this lua code will invoke a C function. Is it possible for this function to retrieve the value that was originally pushed on to the stack?
Its like this:
<engine function A>
pushes parameter value X on to stack for lua
<lua func>
<lua func>
<lua func>
<engine function B>
can I extract the values X that was pushed by function A here?
Yes, with a combination of getinfo, getlocal and getupvalue you can get all that information (you can even change those values using set* functions).
Here is a fragment from MobDebug that returns stack information along with a table of locals and upvalues at each level. The variables at each level will be indexed in the same order they appear in the code (starting from parameters). For each get* function you can use their C equivalents (lua_getinfo, lua_getlocal, and lua_getupvalue), but the logic should be exactly the same.
local function stack(start)
local function vars(f)
local func = debug.getinfo(f, "f").func
local i = 1
local locals = {}
while true do
local name, value = debug.getlocal(f, i)
if not name then break end
if string.sub(name, 1, 1) ~= '(' then locals[name] = {value, tostring(value)} end
i = i + 1
end
i = 1
local ups = {}
while func and true do -- check for func as it may be nil for tail calls
local name, value = debug.getupvalue(func, i)
if not name then break end
ups[name] = {value, tostring(value)}
i = i + 1
end
return locals, ups
end
local stack = {}
for i = (start or 0), 100 do
local source = debug.getinfo(i, "Snl")
if not source then break end
table.insert(stack, {
{source.name, source.source, source.linedefined,
source.currentline, source.what, source.namewhat, source.short_src},
vars(i+1)})
if source.what == 'main' then break end
end
return stack
end

Resources