We have Lua integrated into a project but we've found an odd test case that crashes consistently on ARM:
data = {"A","B","C","D","E","F","G","H","I","J"};
function OnTick(_object)
local params = {};
return 1;
end
Here is the basics of how the function is being called from C++:
lua_getglobal(Lua, function_name->c_str()); // Push function name that we want to call onto the stack
if (lua_isnil(Lua, -1))
{
// Error
lua_pop(Lua, 1);
return -1;
}
lua_pushlightuserdata(Lua, (void*)object); // Push the reference object onto the stack
if (lua_pcall(Lua, 1, 1, 0) != 0)
{
// Error
lua_pop(Lua, 1);
return -1;
}
lua_pop(Lua, 1);
return 1;
OnTick crashes after being called around 5 times.
Lua appears to be crashing when the garbage collector tries to clean up. Anyone else come across something like this and solved it?
Resolved this issue, the client code was corrupting the Lua state
Related
I am an intermediately skilled Linux/Unix user trying to compile software for an iPad on a (jailbroken) iPad.
Many builds (for example, make and tex-live) fail with some Operation not permitted error. This will either look like Can't exec "blah": Operation not permitted or execvp: blah: Operation not permitted where blah is aclocal, a configure script, libtool, or just about anything. Curiously, finding the offending line in a Makefile or configure script and prefixing it with sudo -u mobile -E will solve the error for that line, only for it to reappear for on a later line or in another file. Since I am running the build scripts as mobile, I do not understand how this could possibly fix the issue, yet it does. I have confirmed that making these changes does actually allow for the script to work successfully up to that point. Running the build script with sudo or sudo -u mobile -E and/or running the entire build as root does not solve the issue; with either, I still must edit build scripts to add sudo’s.
I would like to know why this is happening, and if possible how I could address the issue without editing build scripts. Any information about these types of errors would be interesting to me even if they do not solve my problem. I am aware that the permissions/security/entitlements system is unusual on iOS and would like to learn more about how it works.
I am using an iPad Pro 4 on jailbroken iOS 13.5 with the build tools from sbingner’s and MCApollo’s repos (repo.bingner.com and mcapollo.github.io/Public). In particular, I am using a build of LLVM 5 (manually installed from sbingner’s old debs), Clang 10, Darwin CC tools 927 and GNU Make 4.2.1. I have set CC, CXX, CFLAGS, etc. to point to clang-10 and my iOS 13.5 SDK with -isysroot and have confirmed that these settings are working. I would like to replace these with updated versions, but I cannot yet build these tools for myself due to this issue and a few others. I do have access to a Mac for cross-compilation if necessary, but I would rather use only my iPad because I like the challenge.
I can attach any logs necessary or provide more information if that would be useful; I do not know enough about this issue to know what information is useful. Thanks in advance for helping me!
For anyone who ends up needing to address this issue on a jailbreak that does not have a fix for this issue, I have written (pasted below) a userland hook based on the posix_spawn implementation from the source of Apple’s xnu kernel.
Compile it with Theos, and inject it into all processes spawned by your shell by setting environment variable DYLD_INSERT_LIBRARIES to the path of the resulting dylib. Note: some tweak injectors (namely libhooker, see here) reset DYLD_INSERT_LIBRARIES, so if you notice this behavior, be sure to inject only your library.
Because the implementation of the exec syscalls in iOS call out to posix_spawn, this hook fixes all of the exec-related issue’s I’ve run into so far.
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <spawn.h>
// Copied from bsd/kern/kern_exec.c
#define IS_WHITESPACE(ch) ((ch == ' ') || (ch == '\t'))
#define IS_EOL(ch) ((ch == '#') || (ch == '\n'))
// Copied from bsd/sys/imgact.h
#define IMG_SHSIZE 512
// Here, we provide an alternate implementation of posix_spawn which correctly handles #!.
// This is based on the implementation of posix_spawn in bsd/kern/kern_exec.c from Apple's xnu source.
// Thus, I am fairly confident that this posix_spawn has correct behavior relative to macOS.
%hookf(int, posix_spawn, pid_t *pid, const char *orig_path, const posix_spawn_file_actions_t *file_actions, const posix_spawnattr_t *attrp, char *const orig_argv[], char *const envp[]) {
// Call orig before checking for anything.
// This mirrors the standard implementation of posix_spawn because it first checks if we are spawning a binary.
int err = %orig;
// %orig returns EPERM when spawning a script.
// Thus, if err is anything other than EPERM, we can just return like normal.
if (err != EPERM)
return err;
// At this point, we do not need to check for exec permissions or anything like that.
// because posix_spawn would have returned that error instead of EPERM.
// Now we open the file for reading so that we can check if it's a script.
// If it turns out not to be a script, the EPERM must be from something else
// so we just return err.
FILE *file = fopen(orig_path, "r");
if (file == NULL) {
return err;
}
if (fseek(file, 0, SEEK_SET)) {
return err;
}
// In exec_activate_image, the data buffer is filled with the first PAGE_SIZE bytes of the file.
// However, in exec_shell_imgact, only the first IMG_SHSIZE bytes are used.
// Thus, we read IMG_SHSIZE bytes out of our file.
// The buffer is filled with newlines so that if the file is not IMG_SHSIZE bytes,
// the logic reads an IS_EOL.
char vdata[IMG_SHSIZE] = {'\n'};
if (fread(vdata, 1, IMG_SHSIZE, file) < 2) { // If we couldn't read at least two bytes, it's not a script.
fclose(file);
return err;
}
// Now that we've filled the buffer, we don't need the file anymore.
fclose(file);
// Now we follow exec_shell_imgact.
// The point of this is to confirm we have a script
// and extract the usable part of the interpreter+arg string.
// Where they return -1, we don't have a shell script, so we return err.
// Where they return an error, we return that same error.
// We don't bother doing any SUID stuff because SUID scripts should be disabled anyway.
char *ihp;
char *line_startp, *line_endp;
// Make sure we have a shell script.
if (vdata[0] != '#' || vdata[1] != '!') {
return err;
}
// Try to find the first non-whitespace character
for (ihp = &vdata[2]; ihp < &vdata[IMG_SHSIZE]; ihp++) {
if (IS_EOL(*ihp)) {
// Did not find interpreter, "#!\n"
return ENOEXEC;
} else if (IS_WHITESPACE(*ihp)) {
// Whitespace, like "#! /bin/sh\n", keep going.
} else {
// Found start of interpreter
break;
}
}
if (ihp == &vdata[IMG_SHSIZE]) {
// All whitespace, like "#! "
return ENOEXEC;
}
line_startp = ihp;
// Try to find the end of the interpreter+args string
for (; ihp < &vdata[IMG_SHSIZE]; ihp++) {
if (IS_EOL(*ihp)) {
// Got it
break;
} else {
// Still part of interpreter or args
}
}
if (ihp == &vdata[IMG_SHSIZE]) {
// A long line, like "#! blah blah blah" without end
return ENOEXEC;
}
// Backtrack until we find the last non-whitespace
while (IS_EOL(*ihp) || IS_WHITESPACE(*ihp)) {
ihp--;
}
// The character after the last non-whitespace is our logical end of line
line_endp = ihp + 1;
/*
* Now we have pointers to the usable part of:
*
* "#! /usr/bin/int first second third \n"
* ^ line_startp ^ line_endp
*/
// Now, exec_shell_imgact copies the interpreter into another buffer and then null-terminates it.
// Then, it copies the entire interpreter+args into another buffer and null-terminates it for later processing into argv.
// This processing is done in exec_extract_strings, which goes through and null-terminates each argument.
// We will just do this all at once since that's much easier.
// Keep track of how many arguments we have.
int i_argc = 0;
ihp = line_startp;
while (true) {
// ihp is on the start of an argument.
i_argc++;
// Scan to the end of the argument.
for (; ihp < line_endp; ihp++) {
if (IS_WHITESPACE(*ihp)) {
// Found the end of the argument
break;
} else {
// Keep going
}
}
// Null terminate the argument
*ihp = '\0';
// Scan to the beginning of the next argument.
for (; ihp < line_endp; ihp++) {
if (!IS_WHITESPACE(*ihp)) {
// Found the next argument
break;
} else {
// Keep going
}
}
if (ihp == line_endp) {
// We've reached the end of the arg string
break;
}
// If we are here, ihp is the start of an argument.
}
// Now line_startp is a bunch of null-terminated arguments possibly padded by whitespace.
// i_argc is now the count of the interpreter arguments.
// Our new argv should look like i_argv[0], i_argv[1], i_argv[2], ..., orig_path, orig_argv[1], orig_argv[2], ..., NULL
// where i_argv is the arguments to be extracted from line_startp;
// To allocate our new argv, we need to know orig_argc.
int orig_argc = 0;
while (orig_argv[orig_argc] != NULL) {
orig_argc++;
}
// We need space for i_argc + 1 + (orig_argc - 1) + 1 char*'s
char *argv[i_argc + orig_argc + 1];
// Copy i_argv into argv
int i = 0;
ihp = line_startp;
for (; i < i_argc; i++) {
// ihp is on the start of an argument
argv[i] = ihp;
// Scan to the next null-terminator
for (; ihp < line_endp; ihp++) {
if (*ihp == '\0') {
// Found it
break;
} else {
// Keep going
}
}
// Go to the next character
ihp++;
// Then scan to the next argument.
// There must be another argument because we already counted i_argc.
for (; ihp < line_endp; ihp++) {
if (!IS_WHITESPACE(*ihp)) {
// Found it
break;
} else {
// Keep going
}
}
// ihp is on the start of an argument.
}
// Then, copy orig_path into into argv.
// We need to make a copy of orig_path to avoid issues with const.
char orig_path_copy[strlen(orig_path)+1];
strcpy(orig_path_copy, orig_path);
argv[i] = orig_path_copy;
i++;
// Now, copy orig_argv[1...] into argv.
for (int j = 1; j < orig_argc; i++, j++) {
argv[i] = orig_argv[j];
}
// Finally, add the null.
argv[i] = NULL;
// Now, our argv is setup correctly.
// Now, we can call out to posix_spawn again.
// The interpeter is in argv[0], so we use that for the path.
return %orig(pid, argv[0], file_actions, attrp, argv, envp);
}
I am trying to evaluate Saxon-C 1.2.1 HE on CentOS8 and installation seems to have gone ok. Trying out the samples by cd samples/cppTests && build64-linux.sh though leads to a myriad of compilation errors to the tune of the following:
../../Saxon.C.API/SaxonProcessor.h:599:32: error: division ‘sizeof (JNINativeMethod*) / sizeof (JNINativeMethod)’ does not compute the number of array elements [-Werror=sizeof-pointer-div]
gMethods, sizeof(gMethods) / sizeof(gMethods[0]));
Before I summarily and trustfully switched off -Werror=sizeof-pointer-div i checked the source code and what's going on there do seem dubious.
bool registerCPPFunction(char * libName, JNINativeMethod * gMethods=NULL){
if(libName != NULL) {
setConfigurationProperty("extc", libName);
}
if(gMethods == NULL && nativeMethodVect.size()==0) {
return false;
} else {
if(gMethods == NULL) {
//copy vector to gMethods
gMethods = new JNINativeMethod[nativeMethodVect.size()];
}
return registerNativeMethods(sxn_environ->env, "com/saxonica/functions/>
gMethods, sizeof(gMethods) / sizeof(gMethods[0]));
}
return false;
}
more specifically sizeof(gMethods) / sizeof(gMethods[0]) would not seem to calculate anything useful by any margin. The intention was probably rather to output some code that would arrive at the same value as nativeMethodVect.size() but seeing this project's source for the very first time i might be mistaking and the division is in fact intentional ?
I am inclined to guess the intention was in fact closer to b than to a in the following example:
#include <cstdio>
struct test
{
int x, y, z;
};
int main()
{
test *a = new test[32], b[32];
printf("%d %d\n", sizeof(a)/sizeof(a[0]), sizeof(b)/sizeof(b[0]));
return 0;
}
which output 0 32 which is expected as the sizeof(a) gives the size of a pointer not the size of an array's memory region.
That bit of code is to support the feature of user defined extension functions in XSLT stylesheets and XQuery queries. If a user is not using these features then they don't need that bit of code. In fact User defined extension functions is only available in Saxon-PE/C and Saxon-EE/C so it should not be in the Saxon-HE/C code base. I have created the following bug issue to investigate the error above and to https://saxonica.plan.io/issues/4477
I would think the workaround would be to either remove the code in question if the extension function feature is not used or remove the compile flag -Werror=sizeof-pointer-div.
The intent was code is as follows:
jobject JNICALL cppNativeCall(jstring funcName, jobjectArray arguments, jobjectArray argTypes){
//native call code here
}
JNINativeMethod cppMethods[] =
{
{
fname,
funcParameters,
(void *)&cppNativeCall
}
};
bool nativeFound = processor->registerNativeMethods(env, "NativeCall",
cppMethods, sizeof(cppMethods) / sizeof(cppMethods[0]));
I want to get current function call stack efficiently, Is there a api or sth to do this?
I tried a method like this, but its too slow.
local function GetStackDepth()
local depth = 0
while true do
if not debug.getinfo(3 + depth) then
break
end
depth = depth + 1
end
return depth
end
edit:
The really problem is Im writing a profiler tool, and using debug.sethook do sth in call and return event. but on lua5.1 or lua-jit situation, when a tail return happend i got two call event and just one return event like this:
call ------ the 1st call event
call
return
so my solution to this problem is get the call stack depth of the current event, on a return event when the depth is less than the 1st call event's depth, i know its a tail return, then i can handle it properly.
But I found the GetStackDepth() its self cost a lot time (too slow), which affect my profiler result.
i can't change the lua version.
For tail calls, you get a different event (or get istailcall in the activation record). I have to admit I am using the C API for the profiling code, but it should not be too tricky to translate that into native lua; although if you have performance issues, note that Roberto Ierusalimschy writes in Programming in Lua, 3rd ed, chapter 24 “The Debug Library”:
For performance reasons, the official interface to these primitives is through the C API.
I have a main C-style debug hook that forwards calls to event handlers:
void debugHook(lua_State * const ls, lua_Debug * const ar)
{
ASSERT(ls != nullptr);
ASSERT(ar != nullptr);
CallGraphTracker & dbg = getCallGraphTracker();
switch(ar->event)
{
case LUA_HOOKCALL:
dbg.handleEventCall(ls, ar);
break;
case LUA_HOOKTAILCALL:
dbg.handleEventTailCall(ls, ar);
break;
case LUA_HOOKRET:
dbg.handleEventReturn(ls, ar);
break;
default:
SHOULD_NEVER_BE_REACHED();
}
}
The event handlers, in my case, build a call graph based on the lua function pointers; the CallGraphTracker has Nodes (one root node and the further ones being added as subNodes) and keeps track of the curNode. (Actually, I also build function infos, but I have removed that code, to keep this simple. Obviously, the Nodes could store all kinds of additional info, too.)
The function Node::addCall just increases a number if a subnode for the same function pointer (luaPtr), line (ar->currentline), and “tailcall-ness” (true or false) already exists, otherwise a new Node is created. In both cases, the relevant subnode is returned and used as the new currentNode.
void CallGraphTracker::handleEventCall(lua_State * ls, lua_Debug * ar)
{
// Get function info and transform into simple function pointer.
lua_getinfo(ls, "f", ar);
ASSERT(lua_isfunction(ls, -1));
void const * luaPtr = lua_topointer(ls, -1);
ASSERT(luaPtr != nullptr);
// Add call.
lua_getstack(ls, 1, ar);
lua_getinfo(ls, "l", ar);
curNode = curNode->addCall(luaPtr, ar->currentline, false);
}
void CallGraphTracker::handleEventTailCall(lua_State * ls, lua_Debug * ar)
{
// Get function info and transform into simple function pointer.
lua_getinfo(ls, "nSf", ar);
ASSERT(lua_isfunction(ls, -1));
void const * luaPtr = lua_topointer(ls, -1);
ASSERT(luaPtr != nullptr);
// Add tail call.
lua_getstack(ls, 1, ar);
lua_getinfo(ls, "l", ar);
curNode = curNode->addCall(luaPtr, ar->currentline, true);
}
void CallGraphTracker::handleEventReturn(lua_State *, lua_Debug *)
{
while(curNode->isTailCall())
{
ASSERT(curNode->getCallerNode() != nullptr);
curNode = curNode->getCallerNode();
}
ASSERT(curNode->getCallerNode() != nullptr);
curNode = curNode->getCallerNode();
}
You'll notice that in handleEventReturn I traverse back the tailcalls first and then do the “proper” return. You can't assume a fixed number of tailcalls (strictly speaking) in case of tailcall-recursion of the form
function f(num)
if num == 0 then
return
end
-- Do something
return f(num-1)
end
Unfortunately, while this handles the tailcalls correctly, my current problem is that under error handling there can still be a mismatch between calls and returns, so I was hoping to be able to ask lua directly for the depth of the call stack, rather than having to do the slow “trial and error”.
My current plan (I have not yet implemented it) is to keep track of my expected call depth, then get information on that level. If that is what I expected, great, no further tests nececssary, moderate cost involved, otherwise I walk up (or is that down?) the stack until I get to a point where my stored call graph information fits the info returned from lua for that stack depth.
I working with Keil, MDK-ARM Pro 4.71 for a Cortex-M3 target(STM32F107).
I have compiled the Lua interpreter and a Lua "timer" module that interfaces the chip's timers. I'd like to call a lua function when the timer is elapsed.
Here is a sample use :
t = timer.open()
t.event = function() print("Bing !") end
t:start()
Until here, everything works fine :-) ! I see the "Bing !" message being printed each time the timer elapses.
Now if I use a closure :
t = timer.open()
i = 0
t.event = function() i = i + 1; print(i); end
t:start()
I'm having a bad memory access in the GC after some amount of timer's updates. Since it's an embedded context with very few memory, I may be running out of memory quite fast if there is a leak.
Here is the "t.event" setter (ELIB_TIMER is a C structure representing my timer) :
static int ElibTimerSetEvent(lua_State* L)
{
ELIB_TIMER* pTimer_X = ElibCheckTimer(L, 1, TRUE);
if (pTimer_X->LuaFuncKey_i != LUA_REFNIL)
{
luaL_unref(L, LUA_REGISTRYINDEX, pTimer_X->LuaFuncKey_i);
pTimer_X->LuaFuncKey_i = LUA_REFNIL;
}
if (!lua_isnil(L, 2))
{
pTimer_X->LuaFuncKey_i = luaL_ref(L, LUA_REGISTRYINDEX);
}
return 0;
}
And here is the native callback implementation :
static void ElibTimerEventHandler(SYSEVT_HANDLE Event_H)
{
ELIB_TIMER* pTimer_X = (ELIB_TIMER*)SWLIB_SYSEVT_GetSideData(Event_H);
lua_State* L = pTimer_X->L;
int i = lua_gettop(L);
if (pTimer_X->LuaFuncKey_i != LUA_REFNIL)
{
lua_rawgeti(L, LUA_REGISTRYINDEX, pTimer_X->LuaFuncKey_i);
lua_call(L, 0, 0);
lua_settop(L, i);
}
}
This is synchronized externally, so this isn't a synchronization issue.
Am I doing something wrong ?
EDIT
Here is the callstack (with a lua_pcall instead of lua_call, but it is the same). The first line is my hard fault handler.
I have found the problem ! I ran out of stack (native stack, not Lua) space :p.
I guess this specific script was causing a particularly long call stack. After increasing the allocated memory for my native stack, the problem is gone. On the opposite, if I reduce it, I can't even initialize the interpreter.
Many thanks to those who tried to help here.
Found a bug in your C code. You broke the lua stack in static int ElibTimerSetEvent(lua_State* L)
luaL_ref will pop the value on the top of Lua stack: http://www.lua.org/manual/5.2/manual.html#luaL_ref
So you need to copy the value to be refed, before the call to luaL_ref:
lua_pushvalue(L, 2); // push the callback to stack top, and then it will be consumed by luaL_ref()
Please fix this and try again.
My pthread_detach calls fail with a "Bad file descriptor" error. The calls are in the destructor for my class and look like this -
if(pthread_detach(get_sensors) != 0)
printf("\ndetach on get_sensors failed with error %m", errno);
if(pthread_detach(get_real_velocity) != 0)
printf("\ndetach on get_real_velocity failed with error %m", errno);
I have only ever dealt with this error when using sockets. What could be causing this to happen in a pthread_detach call that I should look for? Or is it likely something in the thread callback that could be causing it? Just in case, the callbacks look like this -
void* Robot::get_real_velocity_thread(void* threadid) {
Robot* r = (Robot*)threadid;
r->get_real_velocity_thread_i();
}
inline void Robot::get_real_velocity_thread_i() {
while(1) {
usleep(14500);
sensor_packet temp = get_sensor_value(REQUESTED_VELOCITY);
real_velocity = temp.values[0];
if(temp.values[1] != -1)
real_velocity += temp.values[1];
} //end while
}
/*Callback for get sensors thread*/
void* Robot::get_sensors_thread(void* threadid) {
Robot* r = (Robot*)threadid;
r->get_sensors_thread_i();
} //END GETSENSORS_THREAD
inline void Robot::get_sensors_thread_i() {
while(1) {
usleep(14500);
if(sensorsstreaming) {
unsigned char receive;
int read = 0;
read = connection.PollComport(port, &receive, sizeof(unsigned char));
if((int)receive == 19) {
read = connection.PollComport(port, &receive, sizeof(unsigned char));
unsigned char rest[54];
read = connection.PollComport(port, rest, 54);
/* ***SET SENSOR VALUES*** */
//bump + wheel drop
sensor_values[0] = (int)rest[1];
sensor_values[1] = -1;
//wall
sensor_values[2] = (int)rest[2];
sensor_values[3] = -1;
...
...
lots more setting just like the two above
} //end if header == 19
} //end if sensors streaming
} //end while
} //END GET_SENSORS_THREAD_I
Thank you for any help.
The pthread_* functions return an error code; they do not set errno. (Well, they may of course, but not in any way that is documented.)
Your code should print the value returned by pthread_detach and print that.
Single Unix Spec documents two return values for this function: ESRCH (no thread by that ID was found) and EINVAL (the thread is not joinable).
Detaching threads in the destructor of an object seems silly. Firstly, if they are going to be detached eventually, why not just create them that way?
If there is any risk that the threads can use the object that is being destroyed, they need to be stopped, not detached. I.e. you somehow indicate to the threads that they should shut down, and then wait for them to reach some safe place after which they will not touch the object any more. pthread_join is useful for this.
Also, it is a little late to be doing that from the destructor. A destructor should only be run when the thread executing it is the only thread with a reference to that object. If threads are still using the object, then you're destroying it from under them.