After I've created a TRecordSymbol, how do I add a constructor and methods to it? I've tried using TMethodSymbol with little success, as I can't find a way to define the execution of the method.
Method symbols (like all TFuncSymbol) defer the execution to an interface (defined through the Executable property) for "normal" execution (ie. with a stack frame and parameters evaluated and pushed on the stack). "Magic" function symbols on the other hand require a dedicated expression class, and they take over the whole function call (so less overhead, but you've got to guard yourself against everything).
For samples, you can look at what the dwsMathComplex & 3d units do, they introduce records with custom methods.
Related
Status: Sort of solved. Switching Lua.Ref (close equivalent to LuaD LuaObject) to struct as suggested in answer has solved most issues related to freeing references, and I changed back to similar mechanism LuaD uses. More about this in the end.
In one of my project, I am working with Lua interface. I have mainly borrowed the ideas from LuaD. The mechanism in LuaD uses lua_ref & lua_unref to be able to move lua table/function references in D space, but this causes heavy problems because the calls to destructors and their order is not guaranteed. LuaD usually segfaults at least at the program exit.
Because it seems that LuaD is not maintained anymore, I decided to write my own interface for my purposes. My Lua interface class is here: https://github.com/mkoskim/games/blob/master/engine/util/lua.d
Usage examples can be found here:
https://github.com/mkoskim/games/blob/master/demo/luasketch/luademo.d
And in case you need, the Lua script used by the example is here:
https://github.com/mkoskim/games/blob/master/demo/luasketch/data/test.lua
The interface works like this:
Lua.opIndex pushes global table and index key to stack, and return Top object. For example, lua["math"] pushes _G and "math" to stack.
Further accesses go through Top object. Top.opIndex goes deeper in the table hierarchy. Other methods (call, get, set) are "final" methods, which perform an operation with the table and key at the top of the stack, and clean the stack afterwards.
Close everything works fine, except this mechanism has nasty quirk/bug that I have no idea how to solve it. If you don't call any of those "final" methods, Top will leave table and key to the stack:
lua["math"]["abs"].call(-1); // Works. Final method (call) called.
lua["math"]["abs"]; // table ref & key left to stack :(
What I know for sure, is that playing with Top() destructor does not work, as it is not called immediately when object is not referenced anymore.
NOTE: If there is some sort of operator to be called when object is accessed as rvalue, I could replace call(), set() and get() methods with operator overloads.
Questions:
Is there any way to prevent users to write such expressions (getting Top object without calling any of "final" methods)? I really don't want users to write e.g. luafunc = lua["math"]["abs"] and then later try to call it, because it won't work at all. Not without starting to play with lua_ref & lua_unref and start fighting with same issues that LuaD has.
Is there any kind of opAccess operator overloading, that is, overloading what happens when object is used as rvalue? That is, expression "a = b" -> "a.opAssign(b.opAccess)"? opCast does not work, it is called only with explicit casts.
Any other suggestions? I internally feel that I am looking solution from wrong direction. I feel that the problem reside in the realm of metaprogramming: I am trying to "scope" things at expression level, which I feel is not that suitable for classes and objects.
So far, I have tried to preserve the LuaD look'n'feel at interface user's side, but I think that if I could change the interface to something like following, I could get it working:
lua.call(["math", "abs"], 1); // call lua.math.abs(2)
lua.get(["table", "x", "y", "z"], 2); // lua table.x.y.z = 2
...
Syntactically that would ensure that reference to lua object fetched by indexing is finally used for something in the expression, and the stack would be cleaned.
UPDATE: Like said, changing Lua.Ref to struct solved problems related to dereferencing, and I am again using reference mechanism similar to LuaD. I personally feel that this mechanism suits the LuaD-style syntax I am using, too, and it can be quite a challenge to make the syntax working correctly with other mechanisms. I am still open to hear if someone has ideas to make it work.
The system I sketched to replace references (to tackle the problem with objects holding references living longer than lua sandbox) would probably need different kind of interface, something similar I sketched above.
You also have an issue when people do
auto math_abs = lua["math"]["abs"];
math_abs.call(1);
math_abs.call(3);
This will double pop.
Make Top a struct that holds the stack index of what they are referencing. That way you can use its known scoping and destruction behavior to your advantage. Make sure you handle this(this) correctly as well.
Only pop in the destructor when the value is the actual top value. You can use a bitset in LuaInterface to track which stack positions are in use and put the values in it using lua_replace if you are worried about excessive stack use.
The question came up while I was trying to write a lookup table and during debugging I had the impression, that the array might not be initialized at compile or load time, but rather in a lazy way... Unfortunately I could not find the answer in the MSDN chapter about arrays.
Lets look at some sample code first. The code is in a compiled application, not a script, btw.
module Foo =
let bar = [| for i in 0..9 -> yield (i*i) |]
When does Foo.bar get initialized?
At compile time?
At load time?
Lazy, when first accessed?
Never? (Stays an IEnumerator based sequence despite its array type?)
On a side note, my sequence expression is more complicated than the example above and also uses other functions within scope.
Are there cases which are handled in different ways, such as trivial vs complex eypression or long vs short array etc?
Foo.bar gets initialized at load time - or, when your application starts.
There are slight differences between how this is done for libraries and for applications. For applications, the compiler inserts appropriate initialization into the Main method. For libraries, the initialization check is inserted into static constructors of (I believe) all types, so when you access any type from a library, the initialization is done (this may be sometime after Main, but still before you run any code from the library).
It does not really depend on what the code is - if it is a value, it will be initialized. There are some values like Lazy<T> or IEnumerable<T> that do not immediately fully evaluate, but the value is initialized nevertheless.
Is it possible to dynamically modify symbol table at runtime in C (in elf format on Linux)?
My eventual goal is the following:
Inside certain function say foo, I want to override malloc function to my custom handler my_malloc. But outside foo, any malloc should still call to malloc as in glibc.
Note: this is different from LD_PRELOAD which would override malloc during the entire program execution.
Is it possible to dynamically modify symbol table at runtime in C (in elf format on Linux)?
In theory this is possible, but in practice it's too hard to do.
Inside certain function say foo, I want to override malloc function to my custom handler my_malloc. But outside foo, any malloc should still call to malloc as in glibc.
Modifying symbol table (even if it were possible) would not get you to your desired goal.
All calls from anywhere inside your ELF binary (let's assume foo is in the main executable), resolve to the same PLT import slot malloc#plt. That slot is resolved to glibc malloc on the first call (from anywhere in your program, assuming you are not using LD_BIND_NOW=1 or similar). After that slot has been resolved, any further modification to the symbol table will have no effect.
You didn't say how much control over foo you have.
If you can recompile it, the problem becomes trivial:
#define malloc my_malloc
int foo() {
// same code as before
}
#undef malloc
If you are handed a precompiled foo.o, you are linking it with my_malloc.o, and you want to redirect all calls from inside foo.o from malloc to my_malloc, that's actually quite simple to do at the object level (i.e. before final link).
All you have to do is go through foo.o relocation records, and change the ones that say "put address of external malloc here" to "put address of external my_malloc here".
If foo.o contains additional functions besides foo, it's quite simple to limit the relocation rewrite to just the relocations inside foo.
Is it possible to dynamically modify symbol table at runtime in C (in elf format on Linux)?
Yes, it is not easy, but the functionality can be packaged into a library, so at the end of the day, it can be made practical.
Typemock Isolator++
(https://www.typemock.com/isolatorpp-product-page/isolate-pp/)
This is free-to-use, but closed source solution. The usage example from documentation should be instructive
TEST_F(IsolatorPPTests, IsExpired_YearIs2018_ReturnTrue) {
Product product;
// Prepare a future time construct
tm* fakeTime = new tm();
fakeTime->tm_year = 2018;
// Fake the localtime method
FAKE_GLOBAL(localtime);
// Replace the returned value when the method is called
// with the fake value.
WHEN_CALLED(localtime(_)).Return(fakeTime);
ASSERT_TRUE(product.IsExpired());
}
Other libraries of this kind
Mimick, from Q: Function mocking in C?
cpp-stub, from Q: Creating stub functionality in C++
Elfspy, for C++, but sometimes it's OK to test C code from C++ unittests, from Q: C++ mock framework capable of mocking non-virtual methods and C functions
HippoMocks, from Q: Mocking C functions in MSVC (Visual Studio)
the subprojects in https://github.com/coolxv/cpp-stub/tree/master/other
... there is still more, feel free to append ...
Alternate approaches
ld's --wrap option and linker scripts, https://gitlab.com/hedayat/powerfake
various approaches described in answers for Q: Advice on Mocking System Calls
and in answers to Q: How to mock library calls?
Rewrite code to make it testable
This is easier in other languages than C, but still doable even in C. Structure code into small functions without side-effects that can be unit-tested without resorting to trickery, and so on. I like this blog Modularity. Details. Pick One. about the tradeoffs this brings. Personally, I guturally dislike the "sea of small functions and tons of dependency injection" style of code, but I realize that that's the easiest to work with, all things considered.
Excursion to other languages
What you are asking for is trivial to do in Python, with the unittest.mock.patch, or possibly by just assigning the mock into the original function directly, and undoing that at the end of the test.
In Java, there is Mockito/PowerMock, which can be used to replace static methods for the duration of a test. Static methods in Java approximately correspond to regular functions in C.
In Go, there is Mockey, which works similarly to what needs to be done in C. It has similar limitations in that inlining can break this kind of runtime mocking. I am not sure if in C you can hit the issue that very short methods are unmockable because there is not enough space to inject the redirection code; I think more likely not, if all calls go through the Procedure Linkage Table.
Destructors in Delphi are usually named "Destroy", however as far as i understand you can also
name destructors differently
have multiple destructors
Is there any reason why this was implemented this way? What are the possible use cases for differently named / multiple destructors?
In theory you can manually call different destructors to free different external resources, like breaking ref-counting loops, deleting or just closing file, etc.
Also, since the Object Pascal language does not have those magical new/delete operations, there just should be some identifier to call for disposing of the object.
I'd prefer to look at that in retrospect.
"Turbo Pascal with Objects" style objects have both - you call a "magical" Dispose procedure but explicitly specify a destructor to call, since language itself did not knew what to choose. Similarly "magic" procedure New had to be supplied with a manually selected constructor.
http://www.freepascal.org/docs-html/rtl/system/dispose.html
http://putka.acm.si/langref/turboPascal/0547.html
http://www.freepascal.org/docs-html/rtl/system/new.html
http://putka.acm.si/langref/turboPascal/04A4.html
This however violates DRY principle: compiler knows that we are calling d-tor or c-tor, but yet we have to additionally call those "New" and "Dispose" functions. In theory that probably provided to decouple memory allocation and information feeding and combine them anyway we'd like. But i don't think this feature was actually used anything wide.
Interesting that the same design is used in Apple Objective C. You 1st allocate memory for the object and after that you call a constructor for that new instance: http://en.wikipedia.org/wiki/Objective-C#Instantiation
When that model was streamlined for Delphi few decisions was made to make things more simplified (and unified). Memory [de]allocation strategy was shifted to the class level, rather than call-site. That made the redundancy of both calling "New" and named constructor very contrast. One had to be dropped.
C++/C#/Java chosen to retain a special language-level keywords for it, using overloaded functions to provide different c-tors. Perhaps that corresponds to USA style of computer languages.
However Pascal at its core has two ideas: verbosity and small vocabulary. Arguably they can be tracked in other European-school languages like Scala. If possible, the keywords should be removed from language itself and moved to external modules - libraries that you can add or remove from project. And overloaded functions were introduced much later to the language and early preference was to surely have two differently named (self-documenting) function names.
This both ideas probably caused Delphi to remove "magic" procedures and to deduce object creation/destruction at the call-site just by used function names. If you call MyVar.Destroy then compiler looks at the declaration of .Destroy and knows we are deleting the object. Similarly it knows TMyType.CreateXXX(YYY,ZZZ) is an object instanbtiation due to the way CreateXXX was declared.
To make c-tor and d-tor no-named like in C++, Delphi would have to introduce two more keywords to the language level, like those C++ new and delete. And there seems to be no clear advantage in that. At least personally i better like Delphi way.
PS. I had to add there one assumption: we are talking about real C++ and Delphi languages as they were around 1995. They only featured manual memory control for heap-allocated objects, no garbage collection and no automatic ref-counting. You could not trigger object destruction by assigning variable with nil/NULL pointer.
I'd like to get a firmer grasp of how frequently the runtime in any language that requires one is being called. In this case, I'm specifically interested in knowing:
Of all the function calls getting executed on an OS X or iOS system in any given second (approximations are of course necessary) how many of those are Objective-C runtime functions (i.e. functions that are defined by the runtime)?
Of course it depends on your application, but in general the answer is "a whole lot". Like, a whole freaking lot.
If you really want to see numbers, I'd recommend using dtrace to log all runtime functions as they're called. This blog entry talks about how to do such a thing.
A lot. Here are just a few examples.
Every time you send a message, the actual message sending is done by a runtime function (this is in fact the most called runtime function in pretty much any objective C program).
NSObject class and protocol are not part of the standard library but part of the runtime, therefore any method that ends up executing to the default NSObject implementation is in fact executing runtime code.
Every time you execute a default property accessor (either read or write), that's part of the runtime.
If you use ARC, every time you access a weak reference (either for reading or writing it) that's a runtime function.
Objc runtime includes the C runtime, so anything that involves a C runtime function (for example passing a large structure by value or returning it) is in fact calling into the runtime.
and more.