When making Lua bindings for C++ classes, should I return tables or userdata objects?
Does anyone know any of the pros and cons for each method?
I recommend returning userdata. Regardless of approach, there has to be somewhere to put the pointer
to the C++ data, or the actual C++ data itself, and there's nowhere
safe to do this with a table.
Returning tables would make sense in some situations, because they can
be 'annotated' in Lua with extra attributes without one's having to do
anything extra to support this. Unfortunately the C++ object pointer
has to go somewhere, and there's nowhere sensible for it to go other
than an actual entry in the table itself.
This is not a very safe place for it to go. It can be found by Lua
code, and removed or replaced. This could be by accident, or on
purpose, it doesn't really matter.
My preference therefore is to return userdata objects. They can be
made to work like tables if one really must insist upon that, but they
also have a "secret" area (the actual userdata itself) where the C++
object pointer can be stored, safe from overwriting by Lua code.
(Userdata objects also have an "environment" pointer, which is another
place to store object-specific data. As with the userdata payload
itself, this value is inaccessible to Lua code and can't be damaged
that way.)
Related
Status: Sort of solved. Switching Lua.Ref (close equivalent to LuaD LuaObject) to struct as suggested in answer has solved most issues related to freeing references, and I changed back to similar mechanism LuaD uses. More about this in the end.
In one of my project, I am working with Lua interface. I have mainly borrowed the ideas from LuaD. The mechanism in LuaD uses lua_ref & lua_unref to be able to move lua table/function references in D space, but this causes heavy problems because the calls to destructors and their order is not guaranteed. LuaD usually segfaults at least at the program exit.
Because it seems that LuaD is not maintained anymore, I decided to write my own interface for my purposes. My Lua interface class is here: https://github.com/mkoskim/games/blob/master/engine/util/lua.d
Usage examples can be found here:
https://github.com/mkoskim/games/blob/master/demo/luasketch/luademo.d
And in case you need, the Lua script used by the example is here:
https://github.com/mkoskim/games/blob/master/demo/luasketch/data/test.lua
The interface works like this:
Lua.opIndex pushes global table and index key to stack, and return Top object. For example, lua["math"] pushes _G and "math" to stack.
Further accesses go through Top object. Top.opIndex goes deeper in the table hierarchy. Other methods (call, get, set) are "final" methods, which perform an operation with the table and key at the top of the stack, and clean the stack afterwards.
Close everything works fine, except this mechanism has nasty quirk/bug that I have no idea how to solve it. If you don't call any of those "final" methods, Top will leave table and key to the stack:
lua["math"]["abs"].call(-1); // Works. Final method (call) called.
lua["math"]["abs"]; // table ref & key left to stack :(
What I know for sure, is that playing with Top() destructor does not work, as it is not called immediately when object is not referenced anymore.
NOTE: If there is some sort of operator to be called when object is accessed as rvalue, I could replace call(), set() and get() methods with operator overloads.
Questions:
Is there any way to prevent users to write such expressions (getting Top object without calling any of "final" methods)? I really don't want users to write e.g. luafunc = lua["math"]["abs"] and then later try to call it, because it won't work at all. Not without starting to play with lua_ref & lua_unref and start fighting with same issues that LuaD has.
Is there any kind of opAccess operator overloading, that is, overloading what happens when object is used as rvalue? That is, expression "a = b" -> "a.opAssign(b.opAccess)"? opCast does not work, it is called only with explicit casts.
Any other suggestions? I internally feel that I am looking solution from wrong direction. I feel that the problem reside in the realm of metaprogramming: I am trying to "scope" things at expression level, which I feel is not that suitable for classes and objects.
So far, I have tried to preserve the LuaD look'n'feel at interface user's side, but I think that if I could change the interface to something like following, I could get it working:
lua.call(["math", "abs"], 1); // call lua.math.abs(2)
lua.get(["table", "x", "y", "z"], 2); // lua table.x.y.z = 2
...
Syntactically that would ensure that reference to lua object fetched by indexing is finally used for something in the expression, and the stack would be cleaned.
UPDATE: Like said, changing Lua.Ref to struct solved problems related to dereferencing, and I am again using reference mechanism similar to LuaD. I personally feel that this mechanism suits the LuaD-style syntax I am using, too, and it can be quite a challenge to make the syntax working correctly with other mechanisms. I am still open to hear if someone has ideas to make it work.
The system I sketched to replace references (to tackle the problem with objects holding references living longer than lua sandbox) would probably need different kind of interface, something similar I sketched above.
You also have an issue when people do
auto math_abs = lua["math"]["abs"];
math_abs.call(1);
math_abs.call(3);
This will double pop.
Make Top a struct that holds the stack index of what they are referencing. That way you can use its known scoping and destruction behavior to your advantage. Make sure you handle this(this) correctly as well.
Only pop in the destructor when the value is the actual top value. You can use a bitset in LuaInterface to track which stack positions are in use and put the values in it using lua_replace if you are worried about excessive stack use.
I can obtain a userdata from inside my C++ code. I want to know if I can cast it to something so I can dereference its fields and invoke its methods without going through Lua.
In other words:
Once I obtain an userdata and put it in the Lua stack, how can I get it out and use it as an object of a certain class? I know all fields and methods that I'm interested in (their Lua names), but not necessarily all fields and method the userdata underlying implementation provides.
Is there a way to somehow create an interface that would know the offsets of each member?
I believe the question is, how do I know the offset of each member in the Lua view of the userdata, so I can create a struct to cast the void * returned by lua_touserdata to?
Userdata is just a binary blob. Array of bytes. How those bytes will be interpreted - depends entirely on interface provided with that userdata, be it metatable with methods, or some other agreements.
If that userdata created by your code, then you either know what type of native object is there (knowing that some func generate or accept only specific type), or you can place type identifier as first bytes of userdata blob, so you can check that id and switch/cast userdata pointer to a specific class.
If that userdata created by someone's code - you can only rely on methods provided within its metatable (you can access it from native code), or just treat is as array of bytes, without possibly knowing actual type of data stored there. Or do some hackish guesses, making it working in some cases, and crashing badly if your guess wasn't correct.
Is there a way to return a struct from an imported function in MQL4, without having to pass it as a parameter and making a memcpy?
Be cautious with any kind of DLL-interfacing, MQL4 Documentation states:
Passing ParametersAll parameters of simple types are passed by values unless it is explicitly indicated that they are passed by reference. When a string is passed, the address of the buffer of the copied string is passed; if a string is passed by reference, the address of the buffer of this string without copying it is passed to the function imported from DLL.Structures that contain dynamic arrays[], strings, classes, other complex structures, as well as static or dynamic arrays[] of the enumerated objects, can't be passed as a parameter to an imported function.When passing an array to DLL, the address of the beginning of the data buffer is always passed (irrespective of the AS_SERIES flag). A function inside a DLL knows nothing about the AS_SERIES flag, the passed array is a static array of an undefined length; an additional parameter should be used for specifying the array size.
More glitches apply... Then how to make it work?
Maybe a straight, heterogeneous multi-party distributed processing, which communicates rather results than function calls, independent of all nightmares of maintaining just DLL-imported functions API changes, is a way safer way to go. Using this approach for the last few years and since than have no problems with New-MQL4.56789 string-s that seized to remain string-s and silently started to become struct-s etc.
Worth to know about.
Anyway, welcome and enjoy the Wild Worlds of MQL4 -- may enjoy to click and read other posts on issues in MQL4/DLL integration and/or signalling/messaging in MQL4 domains. Feel free to ask more
So I am exposing complicated data objects that are both created entirely within a C++ environment but more recently I have made it so Lua can also create an instances of these objects. I use the UserData to track where the allocation happened so when __gc is called it knows what it is supposed to do.
Is Lua smart enough to know a reference to a user data it created is stored in a user data block that it did not create?
Code Sample
function AddNewSubjObject(containerId, subObjectType)
local containerObject = meta_ContainerObject:getContainer(containerId)
local newSubObject = meta_SubObject:CreateNew(subObjectType)
containerObject.AddChild(newSubObject)
return containerObject;
end
We get a local variable to the userdata object that can contain other objects. We then create a new object that its only tracking is in Lua. Finally we call a method that will store the sub object in the container object.
When this function returns, to me it looks like it has removed all references to the newSubObject so I fear it will delete the memory allocated for it even though it is still being referenced..
Will this either leak memory by not cleaning up the objects properly or will it corrupt memory by deleting it when it is still be potentially used? Basically I am wondering if this is a situation that has to be considered and handled when exposing userdata via Lua.
The Lua garbage collector does not look inside userdata.
Lua provides weak tables to manage these kinds of references. See PiL Chapter 17. You can use a table with weak keys of subObjects and values of the container of the subObject. When the container is collected so will the subObjects.
I'm not sure if this is the right place to ask this, since it's not really a technical question but more a question of style and coding practices...
I've always ben a fan of using "const" to define variables that will not be changing throughout their lifetime, most especially when they are parameters to functions/methods. This probably stems from my history with C++, where objects could be passed by reference rather than by pointer, but you wanted to ensure that the original value wasn't accidentally altered, either by you or by someone else on your team who was working on the same code snippet.
When looking through the headers for both Objective-C in general and Cocos2d specifically, I've noticed that there is a noticeable lack of use of this item. Now, I'm not against developing code as quickly as possible, and leaving off constraints such as these leave the developer the option to modify values as their code develops and evolves, but there are some instances where I believe that this laxity does not belong.
For example, in Cocos2D/UIKit, the "UIFont fontWithName" method takes "(NSString *)" as the parameter for the font name: does this method really need to reserve the right to alter the original string that was passed in? I personally like to define constant strings as "const" items, and I don't like the necessity of casting these as non-"const" when calling these methods.
Enough proselytizing: My question - Is the direction now moving towards less well-defined interfaces and more towards "lazy references" (which I do not consider to be a derogative term)?
Thanks in advance for any feedback....
Const wouldn't mean anything for Objective C class pointers, because it would have to be overloaded in a very confusing way for Objective C types. This is because there's no way to mark a method as const, as there is in C++, so the compiler could never enforce it.
That said, at my old company, we did declare global string constants using something like:
NSString* const kMyCoolString = #"Hello, world!";
The point being that it at least couldn't be reassigned to something else.
The closest analog in Objective C/Cocoa/Foundation are mutable/immutable versions of data structures, which doesn't really help your case.