Is it possible to check if a lua script contains errors without executing it? I have fallowing code:
if(luaL_loadbuffer(L, data, size, name))
{
fprintf (stderr, "%s", lua_tostring (L, -1));
lua_pop (L, 1);
}
if(lua_pcall(L, 0, 0, 0))
{
fprintf (stderr, "%s", lua_tostring (L, -1));
lua_pop (L, 1);
}
But if the script contains errors it passes first if and it is executed. I want to know if it contains errors when I load it, not when I execute it. Is this possible?
You can use the LUA Compiler. It will only compile your file to bytecode without executing it.
Your program will also have the advantage the run faster if it is compiled.
You can even use the -p option to only perform a syntax checking, according to the linked man page :
-p load files but do not generate any output file. Used mainly for syntax checking or testing precompiled chunks: corrupted files will probably generate errors when loaded. For a thourough integrity test, use -t.
(This was originally meant as a reply to the first comment to Krtek's question, but I ran out of space there and to be honest it works as an answer just fine.)
Functions are essentially values, and thus a named function is actually a variable of that name. Variables, by their very definition, can change as a script is executed. Hell, someone might accidentally redefine one of those functions. Is that bad? To sum my thoughts up: depending on the script, parameters passed and/or actual implementations of those pre-defined functions you speak of (one might unset itself or others, for example), it is not possible to guarantee things work unless you are willing to narrow down some of your demands. Lua is too dynamic for what you are looking for. :)
If you want a flawless test: create a dummy environment with all bells and whistles in place, and see if it crashes anywhere along the way (loading, executing, etc). This is basically a sort of unit test, and as such would be pretty heavy.
If you want a basic check to see if a script has a valid syntax: Krtek gave an answer for that already. I am quite sure (but not 100%) that the lua equivalent is to loadfile or loadstring, and the respective C equivalent is to try and lua_load() the code, each of which convert readable script to bytecode which you would already need to do before you could actually execute the code in your normal all-is-well usecase. (And if that contained function definitions, those would need to be executed later on for the code inside those to execute.)
However, these are the extent of your options with regards to pre-empting errors before they actually happen. Lua is a very dynamic language, and what is a great strength also makes for a weakness when you want to prove correctness. There are simply too many variables involved for a perfect solution.
In general it is not possible, as Lua is a dynamic language, and most of errors happen in runtime.
If you want to check for syntax errors, use luac -p option. I use it as a part of my pre-commit hook, for example.
Other common errors are triggering by misusing the global variables. You may analyze output of luac -l to catch these cases. See here: http://lua-users.org/wiki/DetectingUndefinedVariables.
If you want something more advanced, there are several more-or-less functional static analysis tools for Lua code. Start with LuaInspect.
In any case, you are advised to write unit tests instead of just relying on static code checks. Less pain, more gain.
Related
In python, there is a common construction of the form if __name__ == "__main__": to detect whether the file was imported or executed directly. Usually, the only action taken in this conditional is to execute some "sensible, top level" function. This allows the same file to be used as a basic script and as a library module (and also as something an interactive user can import and use).
I was wondering if there is a clean and reliable way to do this in lua. I thought I could use the _REQUIREDNAME global variable, but it turns out that this was changed in Lua 5.1. Currently, the lua require passes arguments (in the variadic ...), so in principle, these can be examined. However, this is either not reliable, not clean, or probably both, because obviously when a script is executed arguments can be passed. So to do this safely, you would have to examine the arguments.
FWIW, require passes the module name as argument 1 (the string you called require on), and the path to the file it eventually found as argument 2. So there is a obviously some examination that can be done to try to detect this, which if not nearly as nice as if __name__ == "__main__": and can always be bypassed by a user by passing two suitably constructed arguments to the script. Not exactly a security threat, but I would hope there is a better solution.
I also experimented with another method, which I found very ugly but promising. This was to use debug.traceack(). If the script is executed directly, the traceback is very predictable, in fact, it only has 3 lines. I thought this might be it, although, like I said, an ugly hack for sure.
Do any more frequent lua users have advice? In effect, if I am writing module X, I want to either return X.main_func() in script mode or return X in import mode.
EDIT:
I took out an item which was actually incorrect (and makes my traceback solution workable). Additionally, the link provided in the comment by Egor Skriptunoff did provide another trick from the debug library which is even cleaner than using the traceback. Other than that, it seems that everyone ran into the same issues as me and the lua team has been disinterested in providing an official means to support this.
Based on the links provided by Egor, the current cleanest and safest way to do this seems to be as outlined here:
How to determine whether my code is running in a lua module?
which I repeat for ease of reference:
if pcall(debug.getlocal, 4, 1) then
print("in package")
else
print("in main script")
end
There is a whole thread about it here:
http://lua.2524044.n2.nabble.com/Modules-with-standalone-main-program-td7681497.html
Like I said, it seems this is a popular feature which is going to remain unsupported for the time being, but the debug.getlocal method seems to be what common lua developers have settled on for now.
A require() returning in package.loaded.
So simply check package.loaded for what you want.
Also as a good start for more experience in that methodic i suggest to write package.preload functions for require() stuff without manipulating the path.
A simple example for showing this...
# /usr/local/bin/lua -i
Lua 5.4.3 Copyright (C) 1994-2021 Lua.org, PUC-Rio
> package.preload.mymod=function() return {dump=function(...)
>> local args={...}
>> local test,dump=pcall(assert,args[1])
>> if test then
>> for key,value in pairs(dump) do
>> io.write(string.format("%s=%s\n",key,value)):flush()
>> end
>> return true
>> else
>> return test,dump
>> end
>> end} end
> mymod=require('mymod')
> mymod.dump(package.loaded)
string=table: 0x56693590
mymod=table: 0x566aa890
utf8=table: 0x56694d80
package=table: 0x56691ed0
math=table: 0x56693cb0
table=table: 0x566920d0
_G=table: 0x56690730
debug=table: 0x566950e0
coroutine=table: 0x56692310
os=table: 0x56692e60
io=table: 0x566921b0
true
I'm reading some source codes of a project, which is a combination of c++ and lua, they are interwined through luabind.
There is a la.lua file, in which there is a function exec(arg). The lua file also uses functions/variables from other lua file, so it has statements as below in the beginning
module(..., package.seeall);
print("Loading "..debug.getinfo(1).source.."...")
require "client_config"
now I want to run la.exec() from interactive terminal(on linux), but I get errors like
attempt to index global 'lg' (a nil value)
if I want to import la.lua, I get
require "la"
Loading #./la.lua...
./la.lua:68: attempt to index global 'ld' (a nil value)
stack traceback:
./lg.lua:68: in main chunk
[C]: in function 'require'
stdin:1: in main chunk
[C]: ?
what can I do?
Well, what could be going wrong?
(Really general guesswork following, there's not much information in what you provided…)
One option is that you're missing dependencies because the files don't properly require all the things they depend on. (If A depends on & requires B and then C, and C depends on B but doesn't require it because it's implicitly loaded by A, directly loading C will fail.) So if you throw some hours at tracking down & fixing dependencies, things might suddenly work.
(However, depending on how the modules are written this may be impossible without a lot of restructuring. As an example, unless you set package.loaded["foo"] to foo's module table in foo before loading submdules, those submodules cannot require"foo". (Luckily, module does that, in newer code without module that's often forgotten – and then you'll get an endless loop (until the stack overflows) of foo loading other modules which load foo which loads other modules which …) Further, while "fixing" things so they load in the interpreter you might accidentally break the load order used by the program/library under normal operation which you won't notice until you try to run that one normally again. So it may simply cost too much time to fix dependencies. You might still be able to track down enough to construct a long lua -lfoo-lbar… one-off dependency list which might get things to run, but don't depend on it.)
Another option is that there are missing parts provided by C(++) modules. If these are written in the style of a Lua library (i.e. they have luaopen_FOO), they might load in the interpreter. (IIRC that's unlikely for C++ because it expects the main program to be C++-aware but lua is (usually? always?) plain C.) It's also possible that these modules don't work that way and need to be loaded in some other way. Yet another possibility might be that the main program pre-defines things in the Lua state(s) that it creates, which means that there is no module that you could load to get those things.
While there are some more variations on the above, these should be all of the general categories. If you suspect that your problem is the first one (merely missing dependency information), maybe throw some more time at this as you have a pretty good chance of getting it to work. If you suspect it's one of the latter two, there's a very high chance that you won't get it to work (at least not directly).
You might be able to side-step that problem by patching the program to open up a REPL and then do whatever it is you want to do from there. (The simplest way to do that is to call debug.debug(). It's really limited (no multiline, no implicit return, crappy error information), but if you need/want something better, something that behaves very much like the normal Lua REPL can be written in ~30 lines or so of Lua.)
I'm writing tests with EUnit and some of the Units Under Test need to read a data file via file:consult/1. My tests make assumptions about the data that will be available in /priv, but the data will be different in production. What's the best way to accomplish this?
I'm a total newbie in Erlang and I have thought of a few solutions that feel a bit ugly to me. For example,
Put both files in /priv and use a macro (e.g., "-ifdef(EUNIT)") to determine which one to pass to file:consult/1. This seems too fragile/error-prone to me.
Get Rebar to copy the right file to /priv.
Also please feel free to point out if I'm trying to do something that is fundamentally wrong. That may well be the case.
Is there a better way to do this?
I think both of your solutions would work. It is rather question of maintaining such tests, and both of those rely on some external setup (file existing, and having wright data).
For me easiest way to keep contents of such file local to given test is mocking, and making file:consult/1 return value you want.
7> meck:new(file, [unstick, passthrough]).
ok
8> meck:expect(file, consult, fun( _File ) -> {some, data} end).
ok
9> file:consult(any_file_at_all).
{some,data}
It will work, but there are two more things you could do.
First of all, you should not be testing file:consult/1 at all. It is part of standard library, and can assume it works all wright. Rather than doing that you should test functions that use data you read from this file; and of course pass to them some "in-test-created" data. It will give you some nice separation between data source, and parsing (acting on) it. And later it might be simpler to replace file:consult with call to external service, or something like that.
Other thing is that problem with testing something should be sign of bad smell for you. You might think a little about redesigning your system. I'm not saying that you have to, but such problems are good indicator to justify on . If you testing some functionality x, and you would like it to behave one way in production and other in tests (read one file or another), than maybe this behaviour should be injected to it. Or in other words, maybe file your function should read should be a parameter in this function. If you would like to still have some "default file to read" functionality in your production code, you could use something like this
function_with_file_consult(A, B, C) ->
function_with_file_consult(A, B, C, "default_file.dat").
function_with_file_consult(A, B, C, File) ->
[ ... actual function logic ... ]
It will allow you to use shorter version in production, and longer just for your tests.
I've read on the Lua wiki / here / etc. on how to sandbox lua code generally. But I haven't been able to find something that disallows function creation. For example, the example here provides a sample code as:
assert(run [[function f(x) return x^2 end; t={2}; t[1]=f(t[1])]])
And that's with an empty environment. But I want to eliminate the ability to create a function (the 1st part of the code) - e.g., just allow expressions. Any idea on how to do that? Does it have to be in C somehow? Thanks in advance!
If you want to evaluate expressions only, you could try this:
function run(s) return loadstring("return "..s)() end
(error handling omitted)
This simple solution will prevent most `attacks', but not eliminate them because one can say
(function () f=function(x) print"hello" end end)()
which defines a new function named f.
Your best bet is to use a sandbox and not worry about what the user does to the environment, because it'll not be your environment.
You can try detecting the creation of functions by looking for the string "function" before allowing the execution of the lua script. For example from your C/C++ backend.
If "function" appears throw a "you are not allowed to create functions" error and don't execute the code.
A couple notes:
You might want to try to customize the detection a bit more - only throw errors if you detect function followed by blanks and an opening parenthesis, for example. I'm leaving that as an exercise.
You should be aware that there are some standard lua functions that kindof expect the users to be able to create functions - for example, the string table has several of those. Without creating functions, it'll be very difficult for your users to work with strings (it is already difficult enough with functions...)
I'm using a closed-source application that loads Lua scripts and allows some customization through modifying these scripts. Unfortunately that application is not very good at generating useful log output (all I get is 'script failed') if something goes wrong in one of the Lua scripts.
I realize that dynamic languages are pretty much resistant to static code analysis in the way C++ code can be analyzed for example.
I was hoping though, there would be a tool that runs through a Lua script and e.g. warns about variables that have not been defined in the context of a particular script.
Essentially what I'm looking for is a tool that for a script:
local a
print b
would output:
warning: script.lua(1): local 'a' is not used'
warning: script.lua(2): 'b' may not be defined'
It can only really be warnings for most things but that would still be useful! Does such a tool exist? Or maybe a Lua IDE with a feature like that build in?
Thanks, Chris
Automated static code analysis for Lua is not an easy task in general. However, for a limited set of practical problems it is quite doable.
Quick googling for "lua lint" yields these two tools: lua-checker and Lua lint.
You may want to roll your own tool for your specific needs however.
Metalua is one of the most powerful tools for static Lua code analysis. For example, please see metalint, the tool for global variable usage analysis.
Please do not hesitate to post your question on Metalua mailing list. People there are usually very helpful.
There is also lua-inspect, which is based on metalua that was already mentioned. I've integrated it into ZeroBrane Studio IDE, which generates an output very similar to what you'd expect. See this SO answer for details: https://stackoverflow.com/a/11789348/1442917.
For checking globals, see this lua-l posting. Checking locals is harder.
You need to find a parser for lua (should be available as open source) and use it to parse the script into a proper AST tree. Use that tree and a simple variable visibility tracker to find out when a variable is or isn't defined.
Usually the scoping rules are simple:
start with the top AST node and an empty scope
item look at the child statements for that node. Every variable declaration should be added in the current scope.
if a new scope is starting (for example via a { operator) create a new variable scope inheriting the variables in the current scope).
when a scope is ending (for example via } ) remove the current child variable scope and return to the parent.
Iterate carefully.
This will provide you with what variables are visible where inside the AST. You can use this information and if you also inspect the expressions AST nodes (read/write of variables) you can find out your information.
I just started using luacheck and it is excellent!
The first release was from 2015.