In the Lua manual we read:
A reference is a unique integer key.
As long as you do not manually add
integer keys into table t, luaL_ref
ensures the uniqueness of the key it
returns. You can retrieve an object
referred by reference r by calling
lua_rawgeti(L, t, r). Function
luaL_unref frees a reference and its
associated object.
Suppose I create a reference to an object, push it onto the API stack, save it under a global variable, and then call luaL_unref.... does it get freed despite being pointed to in Lua?
Example code:
lua_newtable( L );
int index = luaL_ref( L, LUA_REGISTRYINDEX );
lua_rawgeti( L, LUA_REGISTRYINDEX, index );
lua_setglobal( L, "test" );
luaL_unref( L, LUA_REGISTRYINDEX, index );
lua_getglobal( L, "test" ); // ...?
It would be.
Lua Registry is merely a table. No magic here.
So, your code is roughly equivalent to following (with exception of how lua_ref works with indices):
local t = { }
local index = #_R + 1 -- Assume that fictional _R is registry
_R[index] = t
_G["test"] = t -- Non-fictional _G is a global environment
_R[index] = nil
Also, note that your example does not make much sense. (I assume that it is oversimplified.) You don't need to put table into a registry before saving it as a global variable.
For your "unreferenced" table to be destroyed, you need GC to kick in. It cannot kick in between lua_newtable and lua_setglobal, if you call them one after another without returning control to Lua inbetween. Until you return control to Lua, your table is "referenced" in the Lua stack.
No, it is not explicitly freed.
I think there may be some confusion between the question and the previous answer.
The luaL_unref function merely unreferences the object, it does not actually perform a free operation. So if a variable still references the object it remains alive and is not freed.
The problem is with the wording of the reference. The associated object is "freed from the registry" but it is not "freed from the memory system".
Related
In Dart, looking at the code below, does it 'pass by reference' for list and 'pass by value' for integers? If that's the case, what type of data will be passed by reference/value? If that isn't the case, what's the issue that causes such output?
void main() {
var foo = ['a','b'];
var bar = foo;
bar.add('c');
print(aoo); // [a, b, c]
print(bar); // [a, b, c]
var a = 3;
int b = a;
b += 2;
print(a); // 3
print(b); // 5
}
The question your asking can be answered by looking at the difference between a value and a reference type.
Dart like almost every other programming langue makes a distinction between the two. The reason for this is that you divide memory into the so called stack and the heap. The stack is fast but very limited so it cannot hold that much data. (By the way, if you have too much data stored in the stack you will get a Stack Overflow exception which is where the name of this site comes from ;) ). The heap on the other hand is slower but can hold nearly infinite data.
This is why you have value and reference types. The value types are all your primitive data types (in Dart all the data type that are written small like int, bool, double and so on). Their values are small enough to be stored directly in the stack. On the other hand you have all the other data types that may potentially be much bigger so they cannot be stored in the stack. This is why all the other so called reference types are basically stored in the heap and only an address or a reference is stored in the stack.
So when you are setting the reference type bar to foo you're essentially just copying the storage address from bar to foo. Therefore if you change the data stored under that reference it seems like your changing both values because both have the same reference. In contrast when you say b = a your not transferring the reference but the actual value instead so it is not effected if you make any changes to the original value.
I really hope I could help answering your question :)
In Dart, all type are reference types. All parameters are passed by value. The "value" of a reference type is its reference. (That's why it's possible to have two variables containing the "same object" - there is only one object, but both variables contain references to that object). You never ever make a copy of an object just by passing the reference around.
Dart does not have "pass by reference" where you pass a variable as an argument (so the called function can change the value bound to the variable, like C#'s ref parameters).
Dart does not have primitive types, at all. However (big caveat), numbers are always (pretending to be) canonicalized, so there is only ever one 1 object in the program. You can't create a different 1 object. In a way it acts similarly to other languages' primitive types, but it isn't one. You can use int as a type argument to List<int>, unlike in Java where you need to do List<Integer>, you can ask about the identity of an int like identical(1, 2), and you can call methods on integers like 1.hashCode.
If you want to clone or copy a list
var foo = ['a', 'b'];
var bar = [...foo];
bar.add('c');
print(bar); // [a, b, c]
print(foo); // [a, b]
var bar_two = []; //or init an empty list
bar_two.addAll([...bar]);
print(bar_two); // [a, b, c]
Reference link
Clone a List, Map or Set in Dart
I was wondering if it would make sense to pass a meta table by reference vs declaring it in-line in setmetatable() when you want to use the same meta table for multiple tables.
My goal is to save memory, but only if it really makes a significant difference.
What I'm talking about is this:
-- Passing the meta table by reference:
JSON1 = {
metaTable = {
__index = function (t, k)
-- ...
end;
__call = function()
-- ...
end
};
parse = function(filePath)
local fakeParsedJson = {}
setmetatable(fakeParsedJson, JSON1.metaTable) -- Right here
return fakeParsedJson(filePath)
end;
}
VS
-- Passing the table in-line:
JSON2 = {
parse = function(filePath)
local fakeParsedJson = {}
setmetatable(fakeParsedJson, { -- Right here:
__index = function (t, k)
-- ...
end;
__call = function()
-- ...
end
})
return fakeParsedJson(filePath)
end;
}
I tried to find out if there is a significant difference in memory usage, but the only way I could find was to compare the gcinfo:
local start1 = gcinfo()
local example2_1 = JSON2.parse('example2_1.json')
local example2_2 = JSON2.parse('example2_2.json')
local example2_3 = JSON2.parse('example2_3.json')
local example2_4 = JSON2.parse('example2_4.json')
local example2_5 = JSON2.parse('example2_5.json')
print(gcinfo()-start1) -- Prints 1
local start2 = gcinfo()
local example1_1 = JSON1.parse('example1_1.json')
local example1_2 = JSON1.parse('example1_2.json')
local example1_3 = JSON1.parse('example1_3.json')
local example1_4 = JSON1.parse('example1_4.json')
local example1_5 = JSON1.parse('example1_5.json')
print(gcinfo()-start2) -- Prints 1
Here's my fiddle: https://repl.it/HfwS/34
It doesn't really look like there is a difference. But I just don't know what is actually happening under the hood.
When you call setmetatable(myTable,myMetaTable), will that write a complete copy of myMetaTable somewhere into myTable or will it just store a simple reference? Because if it would just store a reference, then it would make a lot of sense to have all my tables pointing to the same meta table.
(On x86_64, in Lua 5.3) every (empty) table costs 56 bytes. Every key/value entry in the table costs 32 bytes (but the number of entries is rounded up to the next power of two). (Byte counts may differ for different versions/platforms, but will be roughly the same +/- a power of two or so.)
If you have two entries in the metatable, that's 120 bytes per metatable. (You're also creating closures (function() … end), so it may actually be even more.)
Having the table constructor in argument position for the call to setmetatable means that every time that call is executed, a new independent table is created (+ new closures for the functions, …). (Also read the section on table constructors in the reference manual.) There is no smart compiler / no de-duplication / … In fact, there can't be, because other code could (potentially) modify a metatable, and then there's a clear semantic / observable difference between a single shared metatable and one metatable per thing. If that's not obvious, compare
Foo = { __name = "Foo", dump = print } ; Foo.__index = Foo
function newFoo( ) return setmetatable( { }, Foo ) end
and
function newFoo( )
local mt = { __name = "Foo", dump = print }
mt.__index = mt
return setmetatable( { }, mt )
end
If you say
t = { newFoo( ), newFoo( ), newFoo( ) }
getmetatable( t[1] ).dump = function( self ) print "<Foo>" end
for _, v in ipairs( t ) do v:dump( ) end
the first version will print
<Foo>
<Foo>
<Foo>
while the second one will print (e.g.)
<Foo>
Foo: 0x1267010
Foo: 0x1267120
which is clearly different behavior. So the compiler/… can not de-duplicate identical metatables, because other code (that was not yet seen) might modify one of the metatables, and then the observed behavior would be different.
▶ This means that if you create multiple (meta)tables, they must be kept somewhere. Storing several tables necessarily uses more memory than storing a single one, so having a table constructor in argument position for the call to setmetatable will use more memory than creating a table first and then passing a reference to it in the call.
That said, worrying about memory use should not be your primary concern. The semantics / "meaning" / observable behavior of your code are more important.
If you modify the metatable, should the behavior of all "objects" / values change? Or do you want to determine object types by metatable identity (getmetatable( x ) == Foo)? Then you must use a shared metatable (or an equivalent construction).
If you modify the metatable, should the behavior of only a single "object" change? Then you must construct & use separate metatables per "object" / value.
Only if you know that you will never modify the metatable, will not compare metatable references to determine object types, will not …, then these different approaches will show the same externally visible behavior, and only then you are free to choose based on secondary concerns (like memory usage, convenience / code brevity, …).
(In general, needing separately modifiable metatables is very rare, so using shared metatables (create first and pass the reference to setmetatable) is the usual approach – it saves memory and is nicer for debugging.)
Aside: gcinfo is very old and only returns integer approximations to the amount of memory used. Use collectgarbage "count" instead, and then you'll see a difference. (It returns kilobytes used, so multiply by 1024 to get bytes.)
I try to make efficiently a copy of a lua table. I have written the following function copyTable() that works well (see below). But I imagined I could have something more efficient using the "passing by value" mechanism of the functions. I made a few tests to explore this mechanism :
function nop(x)
return x
end
function noop(x)
x={}
return x
end
function nooop(x)
x[#x+1]=4
return x
end
function copyTable(datatable)
local tblRes={}
if type(datatable)=="table" then
for k,v in pairs(datatable) do tblRes[k]=copyTable(v) end
else
tblRes=datatable
end
return tblRes
end
tab={1,2,3}
print(tab) -->table: 0x1d387e0 tab={1,2,3}
print(nop(tab)) -->table: 0x1d387e0 tab={1,2,3}
print(noop(tab)) -->table: 0x1e76f90 tab={1,2,3}
print(nooop(tab)) -->table: 0x1d387e0 tab={1,2,3,4}
print(tab) -->table: 0x1d387e0 tab={1,2,3,4}
print(copyTable(tab)) -->table: 0x1d388d0
We can see that the reference to the table is transferred unchanged through the functions (when I just read it or add things) except within noop() where I try a radical modification of the existing.
I read Bas Bossink and the answer made by Michael Anderson in this Q/A. Regarding the passing or tables as arguments, they emphasized the difference between "arguments passed by ref" and "arguments passed by values and tables are references" with examples where this difference appears.
But what does that mean precisely ? Do we have a copy of the reference, but what difference does that make with a passing through ref since the data pointed and therefore manipulated is still the same, not copied ? Is the mechanism in noop() specific when we try to affect nil to the table, specific to avoid the deletion of the table or in which cases does it trigger (we can see with nooop() that it is not always the case when the table is modified) ?
My question : how the mechanism of passing tables really works ? Is there a way to make a more efficient way to copy the data of a table without the burden of my copyTable ?
The rules of argument passing in Lua is similarly to C: everything is pass by value, but tables and userdata are passed around as pointers. Passing a copy of a reference does not appear so different in usage, but it is completely different than passing by reference.
For example, you brought this part up specifically.
function noop(x)
x={}
return x
end
print(noop(tab)) -->table: 0x1e76f90 tab={1, 2, 3}
You are assigning the value for the new table[1] into variable x (x now holds a new pointer value). You didn't mutate the original table, the tab variable still holds the pointer value to the original table. When you return from noop you are passing back the value of the new table, which is empty. Variables hold values, and a pointer is a value, not a reference.
Edit:
Missed your other question. No, if you want to deep-copy a table, a function similar to what you wrote is the only way. Deep copies are very slow when tables get large. To avoid performance issues, you might use a mechanism like "rewind tables", which keep track of changes made to them so they can be undone at later points in time (very useful in recursive with backtrack contexts). Or if you just need to keep users from screwing with table internals, write a "freezable" trait.
[1] Imagine the {} syntax is a function that constructs a new table and returns a pointer to the new table.
If you are sure that those 3 assumptions (A) are valid for "tab" (the table being copied):
There are no table keys
t1 = {}
tab = {}
tab[t1] = value
There are no repeated table values
t1 = {}
tab = {}
tab.a = t1
tab.b = t1
-- or
-- tab.a.b...x = t1
There are no recursive tables:
tab = {}
tab.a = tab
-- or
-- tab.a.b...x = tab
Then the code you provided is the smallest and almost as efficient as possible.
If A1 doesn't hold (i.e. you have table keys), then you must change your code to:
function copyTable(datatable)
local tblRes={}
if type(datatable)=="table" then
for k,v in pairs(datatable) do
tblRes[copyTable(k)] = copyTable(v)
end
else
tblRes=datatable
end
return tblRes
end
If A2 doesn't hold (i.e. you have repeated table values), then you could change your code to:
function copyTable(datatable, cache)
cache = cache or {}
local tblRes={}
if type(datatable)=="table" then
if cache[datatable] then return cache[datatable]
for k,v in pairs(datatable) do
tblRes[copyTable(k, cache)] = copyTable(v, cache)
end
cache[datatable] = tblRes
else
tblRes=datatable
end
return tblRes
end
This approach only pays off, though, if you have lots of repeated large tables. So, it is a matter of evaluating which version is faster for your actual production scenario.
If A3 doesn't hold (i.e. you have recursive tables), then your code (and both adjustments above) will enter an infinite recursive loop and eventually throw a stack overflow.
The simplest way to handle that is keeping a backtrack and throwing an error if table recursion happens:
function copyTable(datatable, cache, parents)
cache = cache or {}
parents = parents or {}
local tblRes={}
if type(datatable)=="table" then
if cache[datatable] then return cache[datatable]
assert(not parents[datatable])
parents[datatable] = true
for k,v in pairs(datatable) do
tblRes[copyTable(k, cache, parents)]
= copyTable(v, cache, parents)
end
parents[datatable] = false
cache[datatable] = tblRes
else
tblRes=datatable
end
return tblRes
end
My solution for a deepcopy function which handles recursive tables, preserving the original structure may be found here: https://gist.github.com/cpeosphoros/0aa286c6b39c1e452d9aa15d7537ac95
I am trying to "Skip" a variable, by either never declaring it or just having it garbage collected immediately, but I don't know if it's possible.
Example:
function TestFunc()
return 1, 2
end
function SecondFunction()
local nodeclare, var = TestFunc()
end
Basically what I wanted was for "nodeclare" to not even exist. So if I did print(nodeclare, var) it would do nil, 2.
The same thing would be if I was doing a pairs loop and I didn't need to use the keyvalue.
Is there some special thing I can put as the variable name for this to happen? If say I was doing a pairs loop over 100 values, would that even have a signifigant impact?
First of all, variables are not garbage collected, objects are. In this case, there's nothing to garbage collect.
However, let's say that TestFunc was creating objects (say, tables):
function TestFunc()
return {1}, {2}
end
function SecondFunction()
local nodeclare, var = TestFunc()
end
Now nodeclare is referencing a table returned by TestFunc. That's an object, allocated on the heap, that we don't want hanging around forever.
That object will eventually be collected if there is nothing left referring to it. In your case, as soon as SecondFunction returns, the local nodeclare goes out of scope and goes away. As long as there's nothing else referencing that table, the table will be collected (during next collection cycle).
You can avoid declaring nodeclare entirely by skipping the first return value of TestFunc like this:
local var = select(2, TestFunc())
However, when you're talking about a temporary local variable, as in your example, you normally just create the temporary variable then ignore it. This avoids the overhead of the call to select. Sometimes you use a variable name that indicates it's trash:
local _, var = TestFunc()
If say I was doing a pairs loop over 100 values, would that even have a signifigant impact?
None whatsoever. You're just continually overwriting the value of a local variable.
What impact do you mean exactly? Memory? Performance?
According to the Programming in Lua book, you can sort of skip the second return value, but not ignore the first and use the second:
x,y = foo2() -- x='a', y='b'
x = foo2() -- x='a', 'b' is discarded
x,y,z = 10,foo2() -- x=10, y='a', z='b'
Here's my code, I confuse the local variable 'count' in the return function(c1,c2) with memory strack and where does they store in?
function make_counter()
local count = 0
return function()
count = count + 1
return count
end
end
c1 = make_counter()
c2 = make_counter()
print(c1())--print->1
print(c1())--print->2
print(c1())--print->3
print(c2())--print->1
print(c2())--print->2
in the return function(c1,c2) with memory strack and where does they store in?
It's stored in the closure!
c1 is not a closure, it is the function returned by make_counter(). The closure is not explicitly declared anywhere. It is the combination of the function returned by make_counter() and the "free variables" of that function. See closures # Wikipedia, specifically the implementation:
Closures are typically implemented with a special data structure that contains a pointer to the function code, plus a representation of the function's lexical environment (e.g., the set of available variables and their values) at the time when the closure was created.
I'm not quite sure what you're asking exactly, but I'll try to explain how closures work.
When you do this in Lua:
function() <some Lua code> end
You are creating a value. Values are things like the number 1, the string "string", and so forth.
Values are immutable. For example, the number 1 is always the number 1. It can never be the number two. You can add 1 to 2, but that will give you a new number 3. The same goes for strings. The string "string" is a string and will always be that particular string. You can use Lua functions to take away all 'g' characters in the string, but this will create a new string "strin".
Functions are values, just like the number 1 and the string "string". Values can be stored in variables. You can store the number 1 in multiple variables. You can store the string "string" in multiple variables. And the same goes for all other kinds of values, including functions.
Functions are values, and therefore they are immutable. However, functions can contain values; these values are not immutable. It's much like tables.
The {} syntax creates a Lua table, which is a value. This table is different from every other table, even other empty tables. However, you can put different stuff in tables. This doesn't change the unique value of the table, but it does change what is stored within that table. Each time you execute {}, you get a new, unique table. So if you have the following function:
function CreateTable()
return {}
end
The following will be true:
tableA = CreateTable()
tableB = CreateTable()
if(tableA == tableB) then
print("You will never see this")
else
print("Always printed")
end
Even though both tableA and tableB are empty tables (contain the same thing), they are different tables. They may contain the same stuff, but they are different values.
The same goes for functions. Functions in Lua are often called "closures", particularly if the function has contents. Functions are given contents based on how they use variables. If a function references a local variable that is in scope at the location where that function is created (remember: the syntax function() end creates a function every time you call it), then the function will contain a reference to that local variable.
But local variables go out of scope, while the value of the function may live on (in your case, you return it). Therefore, the function's object, the closure, must contain a reference to that local variable that will cause it to continue existing until the closure itself is discarded.
Where do the values get stored? It doesn't matter; only the closure can access them (though there is a way through the C Lua API, or through the Lua Debug API). So unlike tables, where you can get at anything you want, closures can truly hide data.
Lua Closures can also be used to implement prototype-based classes and objects. Closure classes and objects behave slightly differently than normal Lua classes and their method of invocation is somewhat different:
-- closure class definition
StarShip = {}
function StarShip.new(x,y,z)
self = {}
local dx, dy, dz
local curx, cury, curz
local engine_warpnew
cur_x = x; cur_y = y; cur_z = z
function setDest(x,y,z)
dx = x; dy=y; dz=z;
end
function setSpeed(warp)
engine_warpnew = warp
end
function self.warp(x,y,z,speed)
print("warping to ",x,y,x," at warp ",speed)
setDest(x,y,z)
setSpeed(speed)
end
function self.currlocation()
return {x=cur_x, y=cur_y, z=cur_z}
end
return self
end
enterprise = StarShip.new(1,3,9)
enterprise.warp(0,0,0,10)
loc = enterprise.currlocation()
print(loc.x, loc.y, loc.z)
Produces the following output:
warping to 0 0 0 at warp 10
1 3 9
Here we define a prototype object "StarShip" as an empty table.
Then we create a constructor for the StarShip in the "new" method. The first thing it does is create a closure table called self that contains the object's methods. All methods in the closure (those defined as 'function self.') are "closed" or defined for all values accessible by the constructor. This is why it's called a closure. When the constructor is done it returns the closure object "return self".
A lot more information on closure-based objects is available here:
http://lua-users.org/wiki/ObjectOrientationClosureApproach