So I am exposing complicated data objects that are both created entirely within a C++ environment but more recently I have made it so Lua can also create an instances of these objects. I use the UserData to track where the allocation happened so when __gc is called it knows what it is supposed to do.
Is Lua smart enough to know a reference to a user data it created is stored in a user data block that it did not create?
Code Sample
function AddNewSubjObject(containerId, subObjectType)
local containerObject = meta_ContainerObject:getContainer(containerId)
local newSubObject = meta_SubObject:CreateNew(subObjectType)
containerObject.AddChild(newSubObject)
return containerObject;
end
We get a local variable to the userdata object that can contain other objects. We then create a new object that its only tracking is in Lua. Finally we call a method that will store the sub object in the container object.
When this function returns, to me it looks like it has removed all references to the newSubObject so I fear it will delete the memory allocated for it even though it is still being referenced..
Will this either leak memory by not cleaning up the objects properly or will it corrupt memory by deleting it when it is still be potentially used? Basically I am wondering if this is a situation that has to be considered and handled when exposing userdata via Lua.
The Lua garbage collector does not look inside userdata.
Lua provides weak tables to manage these kinds of references. See PiL Chapter 17. You can use a table with weak keys of subObjects and values of the container of the subObject. When the container is collected so will the subObjects.
Related
I would like one instance of a model in memory to serve as a template for creating other objects for performance reasons, so that duplicates look like the original object but otherwise share no common components with the object they are initialized from, as if they were loaded with Model.find(template_object.id). I've tried some of the available solutions but none seems to do what I need: .dup and .deep_dup will create a new object with nil id and .clone will make some of the fields common to both the initializer and the initialized.
Currently my API is giving out the original objects that I keep as class variables, but I discovered that it leads to obscure memory leaks when the code using the objects manipulates their associations - these are kept in memory indefinitely. I hope that by giving out copies the associations of the template objects will stay untouched and the leak will be gone.
This sounds like the use case for defining a class and just initializing instances. You can customize whatever properties you want shared in the MyClass#new method. Without knowing more about your needs I will add that if you must store a template in memory you could store it as a class variable perhaps MyClass##template but would need to hear more to opine further. 😄
What I found when browsing rails source is the .instantiate method:
MyModel.instantiate(#my_other_instance.attributes_before_type_cast.deep_dup)
I would like to get the lua state from inside lua so I can pass it too an external program that cannot be hooked up using ffi or dll. I just need a pointer to it and the ability to share it(shared memory across program boundaries.
That or I can create a lua state in my program and then pass that so I would simply need to set the lua state to it inside lua(and it would have to work with shared memory).
I've thought about sharing data using json but ideally I would like to directly access objects.
Lua is pretty good about avoiding heap allocation and global pointers to allocated memory. lua_newstate takes an allocator function as a parameter. The provided function will be used to allocate/deallocate all memory associated with the lua_State object. Including the pointer returned by lua_newstate.
So hypothetically, you could provide an allocator function that allocates/deallocates interprocess shared memory. And then, you can just pass the lua_State to some other process and access it.
First, you clearly cannot do this "from inside lua"; that kind of low-level thing just ain't happening. You cannot access the lua_State object from within Lua. You must be in control of the lua_State creation process for that to be a possibility. So we're talking about C (equivalent) code here, not in-Lua code.
Now, you can expose a C function to Lua which returns a light userdata that just so happens to be the exact lua_State* in question. But Lua can't really do much with light userdata other than pass it to other C function APIs.
Second, while the Lua system provides a guarantee that it will only allocate memory through the allocator, the system does not provide a guarantee that what you're trying to do will work. It is entirely possible that the Lua implementation does use process global memory, so long as it does it in such a way that different threads can access that global memory without breaking threading guarantees.
Obviously, you can inspect the Lua implementation to see if it does anything of the kind. But my point is that the guarantees are that each independent lua_State will be thread-isolated from each other and that each lua_State will only allocate memory through the given allocator. There is no guarantee that Lua's implementation doesn't have some global storage that it uses for some purpose.
So simply sharing the memory allocated by the Lua state may not be enough.
Also, even if this works, the two processes cannot access the same lua_State object at the same time, just like two threads in the same process cannot access the lua_State at the same time.
The lua state is not designed to leave the program / thread it is executing in.
Doing a query on a running lua_state could result in a crash, because it is only notionally consistent when a lua call returns, or a C api function is called. During execution, some un-locked modifications could cause uninitialized memory access, or ininite loops due to lists being inconsistent.
I'm somewhat new to Delphi, and this question is just me being curious. (I also just tried using it by accident only to discover I'm not supposed to.)
If you look at the documentation for TObject.InitInstance it tells you not to use it unless you're overriding NewInstance. The method is also public. Why not make it protected if the user is never supposed to call it?
Since I was around when this whole Delphi thing got started back around mid-1992, there are likely several answers to this question. If you look at the original declaration for TObject in Delphi 1, there weren't any protected/private members on TObject. That was because very early on in the development of Delphi and in concert with the introduction of exceptions to the language, exceptions were allocated from a different heap than other objects. This was the genesis of the NewInstance/InitInstance/CleanupInstance/FreeInstance functions. Overriding these functions on your class types you can literally control where an object is allocated.
In recent years I've used this functionality to create a cache of object instances that are literally "recycled". By intercepting NewInstance and FreeInstance, I created a system where instances are not returned to the heap upon de-allocation, rather they are placed on a lock-free/low-lock linked list. This makes allocating/freeing instances of a particular type much faster and eliminates a lot of excursions into the memory manager.
By having InitInstance public (the opposite of which is CleanupInstance), this would allow those methods to be called from other utility functions. In the above case I mentioned, InitInstance could be called on an existing block of memory without having to be called only from NewInstance. Suppose NewInstance calls a general purpose function that manages the aforementioned cache. The "scope" of the class instance is lost so the only way to call InitInstance is of it were public.
One of these days, we'll likely ship the code that does what I described above... for now it's part of an internal "research" project.
Oh, as an aside and also a bit of a history lesson... Prior to the Delphi 1 release, the design of how Exception instances were allocated/freed was returned to using the same heap as all the other objects. Because of an overall collective misstep it was assumed that we needed to allocate all Exception object instances to "protect" the Out of memory case. We reasoned that if we try and raise an exception because the memory manager was "out of memory", how in the blazes would we allocate the exception instance!? We already know there is no memory at that point! So we decided that a separate heap was necessary for all exceptions... until either Chuck Jazdzewski or Anders Heijlsberg (I forget exactly which one), figured out a simple, rather clever solution... Just pre-allocate the out of memory exception on startup! We still needed to control whether or not the exception should ever actually be freed (Exception instances are automatically freed once handled), so the whole NewInstance/FreeInstance mechanism remained.
Well never say never. In the VCL too much stuff is private and not virtual as it is, so I kinda like the fact that this stuff is public.
It isn't really necessary for normal use, but in specific cases, you might use it to allocate objects in bulk. NewInstance reserves a bit of memory for the object and then calls InitInstance to initialize it. You could write a piece of code that allocates memory for a great number of objects in one go, and then calls InitInstance for different parts of that large block to initialize different blocks in it. Such an implementation could be the base for a flyweight pattern implementation.
Normally you wouln't need such a thing at all, but it's nice that you can if you really want/need to.
How it works?
The fun thing is: a constructor in Delphi is just some method. The Create method itself doesn't do anything special. If you look at it, it is just a method as any other. It's even empty in TObject!
You can even call it on an instance (call MyObject.Create instead of TMyObject.Create), and it won't return a new object at all. The key is in the constructor keyword. That tells the compiler, that before executing the TAnyClass.Create method, it should also construct an actual object instance.
That construction means basically calling NewInstance. NewInstance allocates a piece of memory for the data of the object. After that, it calls InitInstance to do some special initialization of that memory, starting with clearing it (filling with zeroes).
Allocating memory is a relatively expensive task. A memory manager (compiled into your application) needs to find a free piece of memory and assign it to your object. If it doesn't have enough memory available, it needs to make a request to Windows to give it some more. If you have thousands or even millions of objects to create, then this can be inefficient.
In those rare cases, you could decide to allocate the memory for all those objects in one go. In that case you won't call the constructor at all, because you don't want to call NewInstance (because it would allocate extra memory). Instead, you can call InitInstance yourself to initialize pieces of your big chunk of memory.
Anyway, this is just a hypotheses of the reason. Maybe there isn't a reason at all. I've seen so many irrationally applied visibility levels in the VCL. Maybe they just didn't think about it at all. ;)
It gives developers a way to create object not using NewInstance (memory from stack/memory pool)
I'm currently working on a MOAI project using Lua. I"m trying to set up some stress tests for some of the game objects, and in turn tracking when the Lua objects I have are created and destroyed during a game session. I can easily track when an "class" object/table has been initialized by incrementing the count in the constructor or initializer. However, because Lua does not have destructors, I'm not sure how I can track when an object has been removed from memory.
Would appreciate any help or suggestions on this matter. Thanks!
To be notified when a Lua object (I assume full userdata or table) is gone, you set a _gc metamethod for it.
Perhaps weak tables is your answer, with nesting. Here's a snippet:
objectArray={}
function newObj(...)
--your OOP code here
--obj is the new table you made
objectArray[#objectArray+1]=setmetatable({obj},{__mode='v'})
end
Now, in a function/block that runs every frame:
for i=1,#objectArray do --no pairs for efficiency, being run every frame this matters
local stillThere=#objectArray[i]
stillThere=stillThere==1
if not stillThere then deconstruct() end
end
Unfortunately, you can't get the table back. I'm not sure if there's an easy solution to this, because __index halts GC.
When making Lua bindings for C++ classes, should I return tables or userdata objects?
Does anyone know any of the pros and cons for each method?
I recommend returning userdata. Regardless of approach, there has to be somewhere to put the pointer
to the C++ data, or the actual C++ data itself, and there's nowhere
safe to do this with a table.
Returning tables would make sense in some situations, because they can
be 'annotated' in Lua with extra attributes without one's having to do
anything extra to support this. Unfortunately the C++ object pointer
has to go somewhere, and there's nowhere sensible for it to go other
than an actual entry in the table itself.
This is not a very safe place for it to go. It can be found by Lua
code, and removed or replaced. This could be by accident, or on
purpose, it doesn't really matter.
My preference therefore is to return userdata objects. They can be
made to work like tables if one really must insist upon that, but they
also have a "secret" area (the actual userdata itself) where the C++
object pointer can be stored, safe from overwriting by Lua code.
(Userdata objects also have an "environment" pointer, which is another
place to store object-specific data. As with the userdata payload
itself, this value is inaccessible to Lua code and can't be damaged
that way.)