Lua 'require' but files are only in memory - lua

Setting: I'm using Lua from a C/C++ environment.
I have several lua files on disk. Those are read into memory and some more memory-only lua files become available during runtime. Think e.g. of an editor, with additional unsaved lua files.
So, I have a list<identifier, lua_file_content> in memory. Some of these files have require statements in them. When I try to load all these files to a lua instance (currently via lua_dostring) I get attempt to call global require (a nil value).
Is there a possibility to provide a require function, which replaces the old one and just uses the provided in memory files (those files are on the C side)?
Is there another way of allowing require in these files without having the required files on disk?
An example would be to load the lua stdlib from memory only without altering it. (This is actually my test case.)

Instead of replacing require, why not add a function to package.loaders? The code is nearly the same.
int my_loader(lua_State* state) {
// get the module name
const char* name = lua_tostring(state);
// find if you have such module loaded
if (mymodules.find(name) != mymodules.end())
{
luaL_loadbuffer(state, buffer, size, name);
// the chunk is now at the top of the stack
return 1;
}
// didn't find anything
return 0;
}
// When you load the lua state, insert this into package.loaders
http://www.lua.org/manual/5.1/manual.html#pdf-package.loaders

A pretty straightforward C++ function that would mimic require could be: (pseudocode)
int my_require(lua_State* state) {
// get the module name
const char* name = lua_tostring(state);
// find if you have such module loaded
if (mymodules.find(name) != mymodules.end())
luaL_loadbuffer(state, buffer, size, name);
// the chunk is now at the top of the stack
lua_call(state)
return 1;
}
Expose this function to Lua as require and you're good to go.
I'd also like to add that to completely mimic require's behaviour, you'd probably need to take care of package.loaded, to avoid the code to be loaded twice.

There is no package.loaders in lua 5.2
It called package.searchers now.
#include <stdio.h>
#include <string>
#include <lua.hpp>
std::string module_script;
int MyLoader(lua_State *L)
{
const char *name = luaL_checkstring(L, 1); // Module name
// std::string result = SearchScript(name); // Search your database.
std::string result = module_script; // Just for demo.
if( luaL_loadbuffer(L, result.c_str(), result.size(), name) )
{
printf("%s", lua_tostring(L, -1));
lua_pop(L, 1);
}
return 1;
}
void SetLoader(lua_State* L)
{
lua_register(L, "my_loader", MyLoader);
std::string str;
// str += "table.insert(package.loaders, 2, my_loader) \n"; // Older than lua v5.2
str += "table.insert(package.searchers, 2, my_loader) \n";
luaL_dostring(L, str.c_str());
}
void SetModule()
{
std::string str;
str += "print([[It is add.lua]]) \n";
str += "return { func = function() print([[message from add.lua]]) end } \n";
module_script=str;
}
void LoadMainScript(lua_State* L)
{
std::string str;
str += "dev = require [[add]] \n";
str += "print([[It is main.lua]]) \n";
str += "dev.func() \n";
if ( luaL_loadbuffer(L, str.c_str(), str.size(), "main") )
{
printf("%s", lua_tostring(L, -1));
lua_pop(L, 1);
return;
}
}
int main()
{
lua_State* L = luaL_newstate();
luaL_openlibs(L);
SetModule(L); // Write down module in memory. Lua not load it yet.
SetLoader(L);
LoadMainScript(L);
lua_pcall(L,0,0,0);
lua_close(L);
return 0;
}

Related

lua, call script from C with limited modules, error on string.len

I am testing about calling LUA scripts from C. LUA 5.3.5.
To tighten the libraries used, in C I am using
L = luaL_newstate();
luaopen_base(L);
luaopen_string(L);
luaL_dofile(L, szFilename);
int tipo = lua_getglobal(L, "check");
int error = lua_pcall(L, 0, 0, 0);
(code simplified and without error handling)
This is the sample LUA script:
function check()
local buffer = "hello"
buffer = buffer .. " world!"
print(buffer)
print(string.len(buffer))
end
Why does string.len return error?
attempt to index a nil value (global 'string')
I thought luaopen_string loads the string library.
luaopen_string does not define a global variable string: it just leaves the table on the stack. So do
luaopen_string(L);
lua_setglobal(L,"string");
Alternatively, use luaL_requiref instead of calling luaopen_string directly:
luaL_requiref(L,"string", luaopen_string,1);
See the last paragraph of section 6 in the manual;
To have access to these libraries, the C host program should call the luaL_openlibs function, which opens all standard libraries. Alternatively, the host program can open them individually by using luaL_requiref to call ... luaopen_string (for the string library) ... These functions are declared in lualib.h.
Finally, if you want to customize the set of standard libraries that your program uses, edit linit.c and add it to your project. Change the list in loadedlibs. Then call luaL_openlibs.
After talking on the LUA mailing list, the best hint I got is to look at the source code.
luaL_openlibs is defined in linit.c
LUALIB_API void luaL_openlibs (lua_State *L) {
const luaL_Reg *lib;
/* "require" functions from 'loadedlibs' and set results to global table */
for (lib = loadedlibs; lib->func; lib++) {
luaL_requiref(L, lib->name, lib->func, 1);
lua_pop(L, 1); /* remove lib */
}
}
So to load only the libraries base and string
the solution for LUA 5.3 is
luaL_requiref(L, "_G", luaopen_base, 1);
lua_pop(L, 1);
luaL_requiref(L, LUA_STRLIBNAME, luaopen_string, 1);
lua_pop(L, 1);
And for LUA 5.4
luaL_requiref(L, LUA_GNAME, luaopen_base, 1);
lua_pop(L, 1);
luaL_requiref(L, LUA_STRLIBNAME, luaopen_string, 1);
lua_pop(L, 1);

is it safe to have two lua thread run parallel on the same lua state without concurrent execution?

we are developing game server using lua.
the server is single threaded, we'll call lua from c++.
every c++ service will create a lua thread from a global lua state which is shared by all service.
the lua script executed by lua thread will call a c api which will make a rpc call to remote server.
then the lua thread is suspened, because it's c function never return.
when the rpc response get back, we'll continue the c code ,which will return to the lua script.
so, we will have multiple lua thread execute parallel on a same global lua state, but they will never run concurrently. and the suspend is not caused but lua yield function, but from the c side.
is it safe to do something like this?
#include <stdio.h>
#include <stdlib.h>
#include <ucontext.h>
#include "lua/lua.hpp"
static ucontext_t uctx_main, uctx_func1, uctx_func2;
lua_State* gLvm;
int gCallCnt = 0;
static int proc(lua_State *L) {
int iID = atoi(lua_tostring(L, -1));
printf("begin proc, %s\n", lua_tostring(L, -1));
if(iID == 1)
{
swapcontext(&uctx_func1, &uctx_main);
}
else
{
swapcontext(&uctx_func2, &uctx_main);
}
printf("end proc, %s\n", lua_tostring(L, -1));
return 0;
}
static void func1(void)
{
gCallCnt++;
printf("hello, func1\n");
lua_State*thread = lua_newthread (gLvm);
lua_getglobal(thread, "proc");
char szTmp[20];
sprintf(szTmp, "%d", gCallCnt);
lua_pushstring(thread, szTmp);
int iRet = lua_resume(thread, gLvm, 1);
printf("lua_resume return:%d\n", iRet);
}
static void func2(void)
{
gCallCnt++;
printf("hello, func2\n");
lua_State*thread = lua_newthread (gLvm);
lua_getglobal(thread, "proc");
char szTmp[20];
sprintf(szTmp, "%d", gCallCnt);
lua_pushstring(thread, szTmp);
int iRet = lua_resume(thread, gLvm, 1);
printf("lua_resume return:%d\n", iRet);
}
int main(int argc, char *argv[]){
int iRet = 0;
gLvm = luaL_newstate();
luaL_openlibs(gLvm);
lua_pushcfunction(gLvm, proc);
lua_setglobal(gLvm, "proc");
char func1_stack[16384];
char func2_stack[16384];
getcontext(&uctx_func1);
uctx_func1.uc_stack.ss_sp = func1_stack;
uctx_func1.uc_stack.ss_size = sizeof(func1_stack);
uctx_func1.uc_link = &uctx_main;
makecontext(&uctx_func1, func1, 0);
getcontext(&uctx_func2);
uctx_func2.uc_stack.ss_sp = func2_stack;
uctx_func2.uc_stack.ss_size = sizeof(func2_stack);
uctx_func2.uc_link = &uctx_main;
makecontext(&uctx_func2, func2, 0);
swapcontext(&uctx_main, &uctx_func1);
swapcontext(&uctx_main, &uctx_func2);
swapcontext(&uctx_main, &uctx_func1);
swapcontext(&uctx_main, &uctx_func2);
printf("hello, main\n");
return 0;
}

Interfacing readline into Rust

I try to do this tutorial in rust, so far I have a lot of problems interfacing the C library into Rust.
C equivalent code:
#include <stdio.h>
#include <stdlib.h>
#include <editline/readline.h>
#include <editline/history.h>
int main(int argc, char** argv) {
/* Print Version and Exit Information */
puts("Lispy Version 0.0.0.0.1");
puts("Press Ctrl+c to Exit\n");
/* In a never ending loop */
while (1) {
/* Output our prompt and get input */
char* input = readline("lispy> ");
/* Add input to history */
add_history(input);
/* Echo input back to user */
printf("No you're a %s\n", input);
/* Free retrived input */
free(input);
}
return 0;
}
So far i got this:
extern crate libc;
use std::c_str;
#[link(name = "readline")]
extern {
fn readline (p: *const libc::c_char) -> *const libc::c_char;
}
fn rust_readline (prompt: &str) -> Option<Box<str>> {
let cprmt = prompt.to_c_str();
cprmt.with_ref(|c_buf| {
unsafe {
let ret = c_str::CString::new (readline (c_buf), true);
ret.as_str().map(|ret| ret.to_owned())
}
})
}
fn main() {
println!("Lispy Version 0.0.1");
println!("Press Ctrl+c to Exit.\n");
// I want to have "history" in Linux of this:
//
// loop {
// print!("lispy> ");
// let input = io::stdin().read_line().unwrap();
// print!("No you're a {}", input);
// }
loop {
let val = rust_readline ("lispy> ");
match val {
None => { break }
_ => {
let input = val.unwrap();
println!("No you're a {}", input);
}
}
}
}
Especifically, I'm having issues with rust_readline function, i don't understand very well what's doing inside.
Edit Cargo.toml, put this:
[dependencies.readline]
git = "https://github.com/shaleh/rust-readline"
Code corrected:
extern crate readline;
fn main() {
println!("Lispy Version 0.0.1");
println!("Press Ctrl+c to Exit.\n");
loop {
let input = readline::readline("lispy> ").unwrap();
readline::add_history(input.as_str());
println!("No you're a {}", input);
}
}
Happy Lisping :-) .
Funny thing is, that I found this question reading the very same book, but not including any book specific information in my search query.
There is now a crate for this. Facon's solution worked for me, but the prompt string always printed as garbage using that library, so I looked for another crate and found one working nicely. Here is an updated example:
cargo.toml:
[dependencies]
# https://crates.io/crates/rustyline
rustyline = "3.0.0"
main.rs:
extern crate rustyline;
use rustyline::error::ReadlineError;
use rustyline::Editor;
const HISTORY_FILENAME: &str = "history.txt";
fn main() {
println!("Lispy Version 0.0.1");
println!("Press Ctrl+c to Exit.\n");
// We create an editor for the readline history.
let mut readline_editor = Editor::<()>::new();
// And then load the history, if it exists.
if readline_editor.load_history(HISTORY_FILENAME).is_err() {
println!("No previous history.");
}
loop {
// We read some input from CLI.
let readline = readline_editor.readline("LISPY>> ");
// The reading of the input could fail, if a user uses special
// key combinations. So we match against the readline Result
// type. Result can either be some `Ok` or an some `Err`.
match readline {
Ok(line) => {
readline_editor.add_history_entry(line.as_ref());
println!("No, you are {}", line);
},
Err(ReadlineError::Interrupted) => {
println!("CTRL-C");
break
},
Err(ReadlineError::Eof) => {
println!("CTRL-D");
break
},
Err(err) => {
println!("Error: {:?}", err);
break
}
}
readline_editor.save_history(HISTORY_FILENAME).expect("Could not save to readline history.");
}
}

ObjectiveC Lua: Error in C function call from Lua

I am using two libraries for objectiveC-lua bridge.
One is Lua library (with C API) provided by Lua official web page. Since that library
does not provide API to pass objectiveC object to lua, I opt for another library
for this purpose, and Wax seems to be the only option for now.
This program starts when runLua() is called, where it will invoke test.lua
iOS implementation
-(void) runLua{
[self initLua];
[self invokeLua:#"require test"];
}
-(void)initLua{
// initialize Lua and our load our lua file
L = luaL_newstate(); // create a new state structure for the interpreter
luaL_openlibs(L); // load all the standard libraries into the interpreter
lua_settop(L, 0);
initDone = TRUE;
}
-(void) invokeLua:(NSString*)luaSrcStr{
//NSLog(#"%#",luaSrcStr);
if(!initDone){
NSLog(#"inside invokeLua: not yet init");
[self initLua];
}
lua_settop(L, 0);
int ok = luaL_loadstring(L, [luaSrcStr UTF8String]);
if(ok == 0){
lua_getglobal(L,"debug");
lua_getfield(L,-1, "traceback");
lua_remove(L,-2);
lua_insert(L,-2);
ok = lua_pcall(L, 0, 0, -2);
if (ok != 0){
luaL_error(L, "cannot run lua file: %s",
lua_tostring(L, -1));
return;
}
}
test.lua
local testObject = myLib.newiOSObjWithName("TestObject")
testObject:setObjectLength(3.6)
myLib.passTestObjectToC(testObject)
There is no problem in executing 1st and 2nd line, only 3rd line, error occurs.
Error
PANIC: unprotected error in call to Lua API (cannot run lua file: attempt to index a nil
value stack traceback:
[C]: in function 'passTestObjectToC'
...-451C-9F1D-9CE481B4F9A0/test.app/test.lua:6: in main chunk
[C]: in function 'require'
[string "..."]:3: in main chunk)
The error seems to indicate that the function passTestObjectToC is not found in lua, which in this case is not true as the function is registered to lua when luaL_openlibs(L) is called.
My guess, the problem comes where objectiveC object is passed to lua using wax_instance_create(luaState *L, ...). After passing the object, Wax probably has changed the luaState *L that it doesnt remember that passTestObjectToC is already registered.
The C implementation as below.
C implementation
static int newiOSObjWithName(lua_State *L){
NSString *className = [NSString stringWithCString:lua_tostring(L, -1)
encoding:NSUTF8StringEncoding];
className = [className stringByReplacingCharactersInRange:NSMakeRange(0,1) withString:
[[className substringToIndex:1] uppercaseString]];
id classId = [[NSClassFromString(className) alloc] init];
luaopen_wax_instance(L);
wax_instance_create(L, classId, YES);
return 1;
}
static int passTestObjectToC(lua_State *L){
// something to be done here
return 0;
}
static const struct luaL_Reg myLib [] = {
{"newiOSObjWithName", newiOSObjWithName},
{"passTestObjectToC", passTestObjectToC},
{NULL, NULL}
}
int luaopen_mylib (lua_State *L){
luaL_register(L, "myLib", myLib);
return 1;
}
linit.c of Lua Library
static const luaL_Reg lualibs[] = {
{"", luaopen_base},
{LUA_LOADLIBNAME, luaopen_package},
{LUA_TABLIBNAME, luaopen_table},
{LUA_IOLIBNAME, luaopen_io},
{LUA_OSLIBNAME, luaopen_os},
{LUA_STRLIBNAME, luaopen_string},
{LUA_MATHLIBNAME, luaopen_math},
{LUA_DBLIBNAME, luaopen_debug},
{"myLib", luaopen_mylib},
{NULL, NULL}
};
LUALIB_API void luaL_openlibs (lua_State *L) {
const luaL_Reg *lib = lualibs;
for (; lib->func; lib++) {
lua_pushcfunction(L, lib->func);
lua_pushstring(L, lib->name);
lua_call(L, 1, 0);
}
}
Anyone can tell me why there is an error for passTestObjectToC call? What is the reason?
There is no example of Wax implementation for passing iOSObject available. I hv not found it.
So I am not sure if this is the right way to do it. Most examples use Wax to implement iOS classes/functions directly in Lua script instead.
My mistake, there is no problem in the codes above. Function passTestObjectToC is successfully called, the error occurred because of the implementation inside passTestObjectToC and not during the calling of passTestObjectToC.
iOS object is successfully passed from iOS to Lua using Wax as above.

Forcing a Lua script to exit

How do you end a long running Lua script?
I have two threads, one runs the main program and the other controls a user supplied Lua script. I need to kill the thread that's running Lua, but first I need the script to exit.
Is there a way to force a script to exit?
I have read that the suggested approach is to return a Lua exception. However, it's not garanteed that the user's script will ever call an api function ( it could be in a tight busy loop). Further, the user could prevent errors from causing his script to exit by using a pcall.
You could use setjmp and longjump, just like the Lua library does internally. That will get you out of pcalls and stuff just fine without need to continuously error, preventing the script from attempting to handle your bogus errors and still getting you out of execution. (I have no idea how well this plays with threads though.)
#include <stdio.h>
#include <setjmp.h>
#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"
jmp_buf place;
void hook(lua_State* L, lua_Debug *ar)
{
static int countdown = 10;
if (countdown > 0)
{
--countdown;
printf("countdown: %d!\n", countdown);
}
else
{
longjmp(place, 1);
}
}
int main(int argc, const char *argv[])
{
lua_State* L = luaL_newstate();
luaL_openlibs(L);
lua_sethook(L, hook, LUA_MASKCOUNT, 100);
if (setjmp(place) == 0)
luaL_dostring(L, "function test() pcall(test) print 'recursing' end pcall(test)");
lua_close(L);
printf("Done!");
return 0;
}
You could set a variable somewhere in your program and call it something like forceQuitLuaScript. Then, you use a hook, described here to run every n instructions. After n instructions, it'll run your hook which just checks if forceQuitLuaScript is set, and if it is do any clean up you need to do and kill the thread.
Edit: Here's a cheap example of how it could work, only this is single threaded. This is just to illustrate how you might handle pcall and such:
#include <stdlib.h>
#include "lauxlib.h"
void hook(lua_State* L, lua_Debug *ar)
{
static int countdown = 10;
if (countdown > 0)
{
--countdown;
printf("countdown: %d!\n", countdown);
}
else
{
// From now on, as soon as a line is executed, error
// keep erroring until you're script reaches the top
lua_sethook(L, hook, LUA_MASKLINE, 0);
luaL_error(L, "");
}
}
int main(int argc, const char *argv[])
{
lua_State* L = luaL_newstate();
luaL_openlibs(L);
lua_sethook(L, hook, LUA_MASKCOUNT, 100);
// Infinitely recurse into pcalls
luaL_dostring(L, "function test() pcall(test) print 'recursing' end pcall(test)");
lua_close(L);
printf("Done!");
return 0;
}
The way to end a script is to raise an error by calling error. However, if the user has called the script via pcall then this error will be caught.
It seems like you could terminate the thread externally (from your main thread) since the lua script is user supplied and you can't signal it to exit.
If that isn't an option, you could try the debug API. You could use lua_sethook to enable you to regain control assuming you have a way to gracefully terminate your thread in the hook.
I haven't found a way to cleanly kill a thread that is executing a long running lua script without relying on some intervention from the script itself. Here are some approaches I have taken in the past:
If the script is long running it is most likely in some loop. The script can check the value of some global variable on each iteration. By setting this variable from outside of the script you can then terminate the thread.
You can start the thread by using lua_resume. The script can then exit by using yield().
You could provide your own implementation of pcall that checks for a specific type of error. The script could then call error() with a custom error type that your version of pcall could watch for:
function()
local there_is_an_error = do_something()
if (there_is_an_error) then
error({code = 900, msg = "Custom error"})
end
end
possibly useless, but in the lua I use (luaplayer or PGELua), I exit with
os.exit()
or
pge.exit()
If you're using coroutines to start the threads, you could maybe use coroutine.yield() to stop it.
You might wanna take look at
https://github.com/amilamad/preemptive-task-scheduler-for-lua
project. its preemptive scheduler for lua.
It uses a lua_yeild function inside the hook. So you can suspend your lua thread. It also uses longjmp inside but its is much safer.
session:destroy();
Use this single line code on that where you are want to destroy lua script.
lua_KFunction cont(lua_State* L);
int my_yield_with_res(lua_State* L, int res) {
cout << " my_yield_with_res \n" << endl;
return lua_yieldk(L, 0, lua_yield(L, res), cont(L));/* int lua_yieldk(lua_State * L, int res, lua_KContext ctx, lua_KFunction k);
Приостанавливает выполнение сопрограммы(поток). Когда функция C вызывает lua_yieldk, работающая
сопрограмма приостанавливает свое выполнение и вызывает lua_resume, которая начинает возврат данной сопрограммы.
Параметр res - это число значений из стека, которые будут переданы в качестве результатов в lua_resume.
Когда сопрограмма снова возобновит выполнение, Lua вызовет заданную функцию продолжения k для продолжения выполнения
приостановленной C функции(смотрите §4.7). */
};
int hookFunc(lua_State* L, lua_Debug* ar) {
cout << " hookFunc \n" << endl;
return my_yield_with_res(L, 0);// хук./
};
lua_KFunction cont(lua_State* L) {// функция продолжения.
cout << " hooh off \n" << endl;
lua_sethook(L, (lua_Hook)hookFunc, LUA_MASKCOUNT, 0);// отключить хук foo.
return 0;
};
struct Func_resume {
Func_resume(lua_State* L, const char* funcrun, unsigned int Args) : m_L(L), m_funcrun(funcrun), m_Args(Args) {}
//имена функций, кол-во агрументов.
private:
void func_block(lua_State* L, const char* functionName, unsigned int Count, unsigned int m_Args) {
lua_sethook(m_L, (lua_Hook)hookFunc, LUA_MASKCOUNT, Count); //вызов функции с заданной паузой.
if (m_Args == 0) {
lua_getglobal(L, functionName);// получить имя функции.
lua_resume(L, L, m_Args);
}
if (m_Args != 0) {
int size = m_Args + 1;
lua_getglobal(L, functionName);
for (int i = 1; i < size; i++) {
lua_pushvalue(L, i);
}
lua_resume(L, L, m_Args);
}
};
public:
void Update(float dt) {
unsigned int Count = dt * 100.0;// Время работы потока.
func_block(m_L, m_funcrun, Count, m_Args);
};
~Func_resume() {}
private:
lua_State* m_L;
const char* m_funcrun; // имя функции.
unsigned int m_Count;// число итерации.
unsigned int m_Args;
};
const char* LUA = R"(
function main(y)
--print(" func main arg, a = ".. a.." y = ".. y)
for i = 1, y do
print(" func main count = ".. i)
end
end
)";
int main(int argc, char* argv[]) {
lua_State* L = luaL_newstate();/*Функция создает новое Lua состояние. */
luaL_openlibs(L);
luaL_dostring(L, LUA);
//..pushlua(L, 12);
pushlua(L, 32);
//do {
Func_resume func_resume(L, "main", 2);
func_resume.Update(1.7);
lua_close(L);
// } while (LUA_OK != lua_status(L)); // Пока поток не завершен.
return 0;
};

Resources