In my application, I have all the Lua libraries exposed from the C backend. Now, I have a need to load a Lua module. The method for this seems to be :
lua_getglobal(L, "require");
lua_pushstring(L, libname);
lua_pcall(L, 1, 0, 0);
which will search the package.path to find <libname>.lua and load it.
Is it possible to build-in the Lua module into the C application (so that the module becomes part of the C application) ? so that I don't have to separately package the Lua module. Somehow I am not able to find any references or examples of this! :(
p.s. I am using LuaJIT-2.0.2, and the library in question is SciLua/Time (uses ffi)
Yes.
luajit -b Module.lua Module_bc.c
will compile a module to bytecode and output a C array initializer containing that bytecode.
If you build with shared libraries enabled and export this array from the main executable, require will find it (and will not need to look for Module.lua.)
To test that it is working, set package.path = "" before requireing the module. If it still works, you know the preload is working and it is not just using the Module.lua file from the current directory.
http://luajit.org/running.html
Other things to keep in mind:
If the module depends on an external file (using io.open), that file still needs to be present. For example some ffi modules try to open a C header file, to pass to ffi.cdef
You need to keep Module_bc.c in sync with Module.lua, e.g. with a Makefile recipe, or you will see some confusing bugs!
Related
I'm trying to repackage into a Docker container some Lua library that is made of the main module and some helper modules. The helper modules are kept inside a subfolder of the library so that imports from the main file are done as
require 'helpers/SomeHelper'
The problem is: because of the way I want the Docker container to work, it would be extremely helpful if I can invoke this library from a different working folder. That is, my call to the main program would be something like
th /app/main.lua
regardless of the actual working directory I'm standing. Unfortunately, relative imports seem to fail when the working directory is different from the directory where the main file is located.
Is there any way I can configure LUA_PATH or any other mechanism to make these imports work correctly? Note that changing the code of the library itself would be a poor solution, as it wasn't developed by me and I would like to be able to update it to newer versions easily.
If you don't care about the working directory, you can just load lfs / LuaFileSytem and use lfs.chdir( src_dir ) to change to the source directory (potentially saving the current working directory with lfs.currentdir( ) first.)
You can also extend the search path of Lua so that it will search those extra directories. The search is driven by package.searchpath. To add a directory /foo/bar/ to the search in a way that supports all normally supported library layouts, add
/foo/bar/?.lua;/foo/bar/?/init.lua to package.path
/foo/bar/?.so (or .dylib or .dll on other OSen) to package.cpath
You can use several ways to extend the path.
One option that works well is to set the LUA_PATH / LUA_CPATH environment variables. (A ;; sequence in one of them will expand to the full default path.) This can be done from .profile or other setup scripts via an earlier export LUA_PATH="..." or (if started from a wrapper script) inline by setting variables just for that call LUA_PATH="..." lua /foo/bar.lua. (Note that if you export this variable in too broad a scope, other Lua scripts will also get their path extended and may find potentially incompatible Lua libraries.)
(You can also manually modify package.(c)path from LUA_INIT. That way, you won't be able to independently disable LUA_INIT or LUA_PATH, but you can use all of Lua to generate the path dynamically.)
A third option (this may be best in your specific case) is to put the extension of package.path at the top of your main script, as in
do
local dir = (arg[0]:match "^(.*)/$")
if dir then -- else cwd is . which works by default
package.path = dir.."/?.lua;"..dir.."/?/init.lua;"..package.path
package.cpath = dir.."/?.so;"..package.cpath
end
end
-- rest of your program goes here
When running a script with the Lua interpreter, arg[0] is the script. So this extends the path to include the program's directory no matter where it is located, and it will only affect the search path of this particular script / program.
You should not forget about that not all modules are loaded from FS directly.
E.g. to improve perfomance it is possible read/compile file to memory and then
use preload table to provide way to load module from memory.
Basic example
--- preload code. It can be done by host application.
local FooUtils = function()
return {
print = function(...)
print("foo", ...)
end
}
end
local Foo = function()
local Utils = require "foo.utils"
return {
foo = function()
Utils.print"hello"
end
}
end
package.preload['foo.utils'] = FooUtils
package.preload['foo'] = Foo
--- main application
require "foo".foo()
In this example assume that FooUtils and Foo just example of compiled modules.
E.g. it can be like FooUtils = loadstring('path/to/utils.lua) and it can be done
even in separate Lua state and then used in any other.
It is important to remember that Foo module have no idea about how host application does lookup of foo.utils.
So there no standart way to provide original file path or relevant paths.
So if you write some module wich relays on relative paths then this module
may be not working in some environments.
So I just suggest use full namespace like require 'foo.utils' instead of require 'utils'
I'm trying to embed Lua 5.2 into my C program.
I want to let the Lua script to be able to require and load another script. How to do it?
Suppose the Lua part has such file structure:
lua_script
- main.lua
+ utils
- custom_loader.lua
+ globals
- scene_globals.lua
- scene_levels.lua
The main.lua will try to require and import functions from custom_loader.lua, and so on...
Is it possible to do this without writing a kind-of wrapper in C?
Can the Lua script automagically just load everything it needs?
(Ps. I don't really need sandboxing for now, so it's okay for the script to do what it want.)
As decribed by the Lua require man page, it searches for the file in a path.
This path can be defined in C.
Have a look to this post : "Setting the global LUA_PATH variable from C++/C"
The require function is very pratical to load modules and libraries defined in .lua files.
This is a problem that plagued me for quite a while.
After I wrote and compiled my custom functions into a shared library, require('mylib') would only work when the shared library was directly in the same directory as the script I called it from.
Any efforts to try require('/path/to/mylib') or similar absolute paths failed. Furthermore, "backtracking" through a relative path (i.e. using .. failed as well.
So how can one specify the bin directory, or wherever the shared library is output to?
Well according to the Lua documentation on the require call (using Lua 5.2 here), there are a few places the loader looks for these loadable modules.
It seems that require() uses what are called "searchers" (docs linked to in above) to determine where to find these modules. There are four searchers in total. From the docs:
The first searcher simply looks for a loader in the package.preload
table.
The second searcher looks for a loader as a Lua library, using the
path stored at package.path. The search is done as described in
function package.searchpath.
The third searcher looks for a loader as a C library, using the path
given by the variable package.cpath. Again, the search is done as
described in function package.searchpath. For instance, if the C path
is the string "./?.so;./?.dll;/usr/local/?/init.so" the searcher for module foo will try to open the files ./foo.so, ./foo.dll, and
/usr/local/foo/init.so, in that order. Once it finds a C library, this
searcher first uses a dynamic link facility to link the application
with the library. Then it tries to find a C function inside the
library to be used as the loader. The name of this C function is the
string "luaopen_" concatenated with a copy of the module name where
each dot is replaced by an underscore. Moreover, if the module name
has a hyphen, its prefix up to (and including) the first hyphen is
removed. For instance, if the module name is a.v1-b.c, the function
name will be luaopen_b_c.
The fourth searcher tries an all-in-one loader. It searches the C path
for a library for the root name of the given module. For instance,
when requiring a.b.c, it will search for a C library for a. If found,
it looks into it for an open function for the submodule; in our
example, that would be luaopen_a_b_c. With this facility, a package
can pack several C submodules into one single library, with each
submodule keeping its original open function.
The searcher of use to us is the third one: it is used for any shared libraries (.dll or .so) which is generally how our custom C modules are built.
Using the template string (the one with the question marks), the searcher will look in each of the specified paths, substituting the argument of require() in place of the question mark. In order to specify the path for this third searcher, one must set (or append to) package.cpath and then call require().
So perhaps you have a directory structure as
- ROOT
|-lua
|-bin
where lua contains script.lua and bin contains mylib.so
To load mylib.so, you just need these two lines of code in script.lua:
package.cpath = '/ROOT/bin/?.so;' .. package.cpath
libfuncs = require('mylib')
NOTE: Notice the semicolon. If you append (as opposed to the prepending above), make sure to lead with the semicolon on your added path. It is not there buy default. Otherwise your new path will be merged to the current default cpath, which is just ./?.so.
If you whant just configure your system to use some path to store
libraries there then best way just use LUA_PATH and LUA_CPATH env variables.
e.g.LUA_CPATH=/path/to/?.so. Or for specific Lua version like LUA_CPATH_5_3=/path/to/?.so.
But if you have full path to some library and whant just load this particular library from this directory you can use package.loadlib function.
local function loadlib(path, name)
local sep = string.sub(package.config, 1, 1)
local file = path .. sep .. name .. ((sep == '/') and '.so' or '.dll')
local func = 'luaopen_' .. name
local loader, msg = package.loadlib(file, func)
assert(loader, msg)
return assert(loader())
end
local mylib = loadlib([[/path/to]], 'mylib')
I'm trying to use the sockets module in a script and I keep encountering a issue where the script is unable to find socket.core. Is there anyway for me to point to exactly where the core.dll is? I've tried using cpath and I can never seem to get it to work. I just want to be able to say "C:/folder/folder/folder/core.dll"
package.cpath = 'F:/Folder/Foldertwo/Game/agame/Beta/Scripts/libs/socket/?.dll;' .. package.cpath
#EgorSkriptunoff is correct in his comment: socket.lua (which is a lua module) loads socket.core (which is a dynamic library), so you won't be able to load it from folder/core.dll as the default searcher will be looking for socket/core.dll.
If you really want to load it from folder/core.dll, you may try to load it yourself and assign the returned value to package.preload['socket.core']. This way when socket.lua loads the module, it will get the value to return from package.preload key without loading the module.
I want to let perl use the DBI module in my own path(suppose, /home/users/zdd/perl5/lib/DBI), but the sysem also has a DBI module which is /usr/lib/perl5/lib/DBI.
when I write the following code in my script, perl use the system path be default, how to force it use the one under my path?
use lib './perl5/lib/DBI';
use DBI;
sub test {
....
}
/usr/lib/perl5/lib/DBI was added to the PATH environment variable in my bash profile, it was used by many scripts, so I can't disable it.
The file for the main DBI module is in ./perl5/lib. So your path is not pointing to it.
The DBI folder contains sub-modules of DBI, e.g. DBI::Foo (the :: in module names is a representation of your module directory structure).
Try using ./perl5/lib as your library instead.
Also, using a relative path will fail if the current directory is not what you think it is. If you are in doubt, have your script call cwd to see what the current directory is.
For debugging purposes, it may be helpful to use:
no lib '[main Perl module library path here]';
That way you can be sure you are only using your custom module path. Any failure to find a module will cause an error, rather than silently using the system version.
Update: For more information, see Perldoc on use lib. Perl will use the library that you have specified first. If it does not, that indicates it is not actually finding the module in the location you have given.
In addition to what dan1111 suggested, I would also recommend you print out #INC (just before your use DBI statement) and dump %INC (just after your use DBI statement) to see what your script is doing. That may help you debug the issue.