Today I have read an article which is about golang goroutine. It says that if there are too many recursive call, the goroutine space will be extended. In my mind, while program running, each function call will create a new stack, system only push a pointer-like object(some mechine code) which point to the created stack to caller function stack top. When cpu load this object, it will save current context and jump to created stack. After called function having returned, cpu will write the returned value which is in register back to the object. If my understanding is right, there is only a little space cast for recursive function call at calling stack. As reading that article, I doubt my understanding is wrong. May each function call make the whole called function code been pushed into calling function stack but not a pointer-like object. If this is true, a function call will cast much space at calling stack. I searched this question by google, but no result. Is there anyone can help me?
Update: I found the answer https://www.geeksforgeeks.org/memory-layout-of-c-program/.
Related
I define a static global array of struct which has some elements defined as pointers. This array will be filled and managed by a static library I use in the project. I find that the array is correctly filled by the library. After a couple of seconds, some values in this array is overwritten somewhere else. After debugging, I found that the Stack Pointer (SP) points to the same memory location where some members of the array are saved and therefore will overwrite the value when pushing to the stack when for example a function is called and its local values are saved in the stack.
My question is: what can cause the stack pointer to point to a memory location that is occupied by another variable? How can I debug this problem further?
I know that my question is general but unfortunately I can't provide a minimal code that reproduces the problem as it is a big project and I don't know exactly which part of the code can cause such a problem.
This is almost always caused by stack overflow and by a bad RAM memory layout causing the stack to overflow into the .data segment. Also obviously avoid bad practices such as intentional/accidental use of recursion.
Troubleshooting:
Soon as the SP is initialized by the CRT/start-up code, then have your debugger fill the whole stack area with some known magic number like 0xAA.
Set a write breakpoint at the very bottom of the stack (at the first 0xAA, assuming down-counting stack).
If you just halt the program at a random place, you'll be able to see how many of the 0xAA your program has munched up. If all of them are gone, your stack is toast.
If you reach the write breakpoint at the bottom of the stack, you'll be able to tell which code was responsible for the most recent write.
A modern MCU and a half-decent debugger with trace may reveal more detailed info if you view the backtrace.
Other less likely but also possible causes:
Array out-of-bounds or pointer corruption bugs overwriting things like return addresses on the stack or writing at random locations.
"Runaway code" caused by something like the above, the program counter executing at arbitrary memory locations.
Beginner mistakes such as returning pointers to local variables from a function or otherwise using pointers to local data which has gone out of scope.
Messing up manual push/pops or function calling convention in inline assembly.
This answer is for the beginners or others who may face the same problem to clarify the problem and to explain how did I figured it out.
A stack is a section in the memory where local variables are stored. Local variables are the non-static variables defined inside a function. Another usage of the stack is to store the address of the latest executed command before jumping to another function or interrupt service routine (ISR) so that CPU knows from where should it continue when it finishes executing the ISR or the other function. When a local variable is defined, its value is saved in the stack. As soon as the function finishes executing, this local variable doesn't exist anymore and its address in the stack is free to be used.
The problem as #the_busybee pointed out was because of defining a local array and assigning its address (which is in the stack) to a globally defined pointer. Then the global pointer itself is saved in .bss memory section while the address it is pointing at is in the stack.
Now how did I figured out
After posting the question I was trying to edit it to make it more clear. This helps you clarifying your thoughts (like the rubber duck debugging method. Many times I found the answer to the question just by trying to formulate it in a clear way.
Then I was thinking that why does the stack pointer point to the address of a globally defined variable.
To make sure that this assumption is correct I checked the address where the global variable is defined. It was 0x20003F4. Then I checked the address saved in the globally defined pointer. It was 0x20006FD0.
I thought then, what if this pointed-at address is a part of the stack area? I checked the .map file and found that .stack starts at address 0x20006c00 and has the size of 0x400. This means that 0x20006FD0 is indeed part of the stack.
Then I remembered that I defined the array pointed at by the global pointer locally in a function.
At this point I saw the comment of the_busybee and cancelled editing the question and start writing this answer :)
I hope this answer helps
Status: Sort of solved. Switching Lua.Ref (close equivalent to LuaD LuaObject) to struct as suggested in answer has solved most issues related to freeing references, and I changed back to similar mechanism LuaD uses. More about this in the end.
In one of my project, I am working with Lua interface. I have mainly borrowed the ideas from LuaD. The mechanism in LuaD uses lua_ref & lua_unref to be able to move lua table/function references in D space, but this causes heavy problems because the calls to destructors and their order is not guaranteed. LuaD usually segfaults at least at the program exit.
Because it seems that LuaD is not maintained anymore, I decided to write my own interface for my purposes. My Lua interface class is here: https://github.com/mkoskim/games/blob/master/engine/util/lua.d
Usage examples can be found here:
https://github.com/mkoskim/games/blob/master/demo/luasketch/luademo.d
And in case you need, the Lua script used by the example is here:
https://github.com/mkoskim/games/blob/master/demo/luasketch/data/test.lua
The interface works like this:
Lua.opIndex pushes global table and index key to stack, and return Top object. For example, lua["math"] pushes _G and "math" to stack.
Further accesses go through Top object. Top.opIndex goes deeper in the table hierarchy. Other methods (call, get, set) are "final" methods, which perform an operation with the table and key at the top of the stack, and clean the stack afterwards.
Close everything works fine, except this mechanism has nasty quirk/bug that I have no idea how to solve it. If you don't call any of those "final" methods, Top will leave table and key to the stack:
lua["math"]["abs"].call(-1); // Works. Final method (call) called.
lua["math"]["abs"]; // table ref & key left to stack :(
What I know for sure, is that playing with Top() destructor does not work, as it is not called immediately when object is not referenced anymore.
NOTE: If there is some sort of operator to be called when object is accessed as rvalue, I could replace call(), set() and get() methods with operator overloads.
Questions:
Is there any way to prevent users to write such expressions (getting Top object without calling any of "final" methods)? I really don't want users to write e.g. luafunc = lua["math"]["abs"] and then later try to call it, because it won't work at all. Not without starting to play with lua_ref & lua_unref and start fighting with same issues that LuaD has.
Is there any kind of opAccess operator overloading, that is, overloading what happens when object is used as rvalue? That is, expression "a = b" -> "a.opAssign(b.opAccess)"? opCast does not work, it is called only with explicit casts.
Any other suggestions? I internally feel that I am looking solution from wrong direction. I feel that the problem reside in the realm of metaprogramming: I am trying to "scope" things at expression level, which I feel is not that suitable for classes and objects.
So far, I have tried to preserve the LuaD look'n'feel at interface user's side, but I think that if I could change the interface to something like following, I could get it working:
lua.call(["math", "abs"], 1); // call lua.math.abs(2)
lua.get(["table", "x", "y", "z"], 2); // lua table.x.y.z = 2
...
Syntactically that would ensure that reference to lua object fetched by indexing is finally used for something in the expression, and the stack would be cleaned.
UPDATE: Like said, changing Lua.Ref to struct solved problems related to dereferencing, and I am again using reference mechanism similar to LuaD. I personally feel that this mechanism suits the LuaD-style syntax I am using, too, and it can be quite a challenge to make the syntax working correctly with other mechanisms. I am still open to hear if someone has ideas to make it work.
The system I sketched to replace references (to tackle the problem with objects holding references living longer than lua sandbox) would probably need different kind of interface, something similar I sketched above.
You also have an issue when people do
auto math_abs = lua["math"]["abs"];
math_abs.call(1);
math_abs.call(3);
This will double pop.
Make Top a struct that holds the stack index of what they are referencing. That way you can use its known scoping and destruction behavior to your advantage. Make sure you handle this(this) correctly as well.
Only pop in the destructor when the value is the actual top value. You can use a bitset in LuaInterface to track which stack positions are in use and put the values in it using lua_replace if you are worried about excessive stack use.
Note: There are some questions below that illustrate my thinking, but the only answer I'm looking for is the answer to the actual question, in the title. Not asking for "a book" here, or itemized responses to all of those.
I'm trying to start a coroutine from the C API, let it yield, and continue it later (possibly after executing resumes from other coroutines). This is a fairly simple use case, but the documentation for lua_resume() is extremely confusing:
int lua_resume (lua_State *L, lua_State *from, int nargs);
Starts and resumes a coroutine in the given thread L.
To start a coroutine, you push onto the thread stack the main function plus
any arguments; then you call lua_resume, with nargs being the number of
arguments. This call returns when the coroutine suspends or finishes its
execution. When it returns, the stack contains all values passed to
lua_yield, or all values returned by the body function. lua_resume returns
LUA_YIELD if the coroutine yields, LUA_OK if the coroutine finishes its
execution without errors, or an error code in case of errors (see
lua_pcall).
In case of errors, the stack is not unwound, so you can use the debug API
over it. The error message is on the top of the stack.
To resume a coroutine, you remove any results from the last lua_yield, put
on its stack only the values to be passed as results from yield, and then
call lua_resume.
The parameter from represents the coroutine that is resuming L. If there is
no such coroutine, this parameter can be NULL.
The meaning of "represents the coroutine that is resuming L" is extremely unclear here. When exactly is "from" nil, which of L and from are the "thread stack", and what are the exact requirements for resuming a coroutine? Can the state L be modified between the initial lua_resume() and the second one which is actually resuming? If so, how does the state know where/what function to resume? If not (ie. one thread per lua_State) what is the proper way to create a new thread, so that it shares an execution context (globals, environment, etc) with the parent thread, and what is the proper way to call lua_resume() and unwind the call for each start/resume in this case? Could the 'from' argument do / be related to any of these things?
And finally, in either case - how can I get a full stack trace for debugging in an error handler (for example invoked on error from lua_pcall) which respects / is aware of calls across resumes? Does lua_getinfo() report correctly through resumes? Is that what the 'from' argument is for?
I'd really like a complete, working example of a coroutine start/resume from the C API that illustrates the use of that second argument. There's this example on gist, but it's years old and the API has changed since then - lua_resume() takes 3 arguments now, not 2...
when i connect a signal to a callback function the callback functions gets passed parameters. Is the reference counter increased before the objects get passed to my callback function or do i have to increase it by myself.
I guess there must be some sort of convention for that because nothing like that is mentioned in the documentation of gtk or libgobject.
Generally, you do not assume a reference on an object when it is passed to your callback. You only assume a reference when the object is the return value of a method which is annotated with "transfer full". You can see these annotations in the documentation.
(I say "generally" because there may always be badly constructed libraries whose API violates these guidelines. You can't do a whole lot about that, though.)
The Question
Excerpt from the documentation:
Decreases the reference count on a GMainLoop object by one.
If the result is zero, free the loop and free all associated memory.
I could not find information regarding this reference counter. What is it initially set to and how is it used?
Details
In particular, I'm confused about this piece of example code (in the main method) (note that set_cancel is a static method:
void (*old_sigint_handler)(int);
old_sigint_handler = signal (SIGINT, set_cancel);
/* Create a new glib main loop */
data.main_loop = g_main_loop_new (NULL, FALSE);
old_sigint_handler = signal (SIGINT, set_cancel);
/* Run the main loop */
g_main_loop_run (data.main_loop);
signal (SIGINT, old_sigint_handler);
g_main_loop_unref (data.main_loop);
If g_main_loop is blocking, how it ever going to stop? I could not find information on this signal method either. But that might be native to the library (although I do not know).
Note: I reduced the code above to what I thought was the essential part. It is from a camera interface library called aravis under docs/reference/aravis/html/ArvCamera.html
I could not find information regarding this reference counter. What is it initially set to and how is it used?
It is initially set to 1. Whenever you store a reference to the object you increment the reference counter, and whenever you remove a reference you decrement the reference counter. It's a form of manual garbage collection. Just google "reference counting" and you'll get lots of information.
If g_main_loop is blocking, how it ever going to stop?
Somewhere someone will call g_main_loop_quit. Judging by the question I'm guessing you're not very familiar with the concept of an event loop—GLib's manual isn't a very gentle introduction to the basic concept, you may want to try the Wikipedia article or just search for "event loop".
I could not find information on this signal method either. But that might be native to the library (although I do not know).
The signal is a standard function (both C and POSIX). Again, there is lots of information out there, including good old man pages (man 2 signal).