I want to have two lexers in one project, and I don't want to run into problems with having multiple yylex functions in the build. Can I make lex output with a different prefix?
You can use the -Pprefix parameter for flex in your makefile. Using flex -Pfoo you would effectively prefix all yy generated functions. Have a look at the manual page for further details.
flex lets you do that. Just define the YY_DECL macro. Dunno about actual Unix(tm) lex(1) though.
You could build a C++ lexer. This means all the state information is held in an object.
Then it is just a matter of using the correct object!
Related
I am trying to learn a bit about Vala and wanted to create a Calculator to test how Gtk worked. The problem is that I coded everything around the supposition that there would be a way to parse a string that contained the required operations. Something like this:
string operation = "5+2/3*4"
I have done this with Python and it is as simple as using the compilers parser. I understand Python is math oriented, but I thought that perhaps there would be Vala library waiting for me as an answer... I haven't found it if it does exist, but as I was looking at the string documentation, I noticed this part:
/* Strings prefixed with '#' are string templates. They can evaluate
* embedded variables and expressions prefixed with '$'.
* Since Vala 0.7.8.
*/
string name = "Dave";
println (#"Good morning, $name!");
println (#"4 + 3 = $(4 + 3)");
So... I thought that maybe there was a way to make it work that way, maybe something like this:
stdout.printf(#"$(operation)")
I understand that this is not an accurate supposition as it will just substitute the variable and require a further step to actually evaluate it.
Right now the two main doubts I am having are: a) Is there a library function capable of doing this? and b) Is it possible to work out a solution using string templates?
Here's something I found that would do the work. I used the C++ libmatheval library, for this I first required a vapi file to bind it to Vala. Which I found here. There are a lot of available vapi files under the project named vala-extra-apis, and they are recognized in GNOME's Vala List of Bindings although they are not included at install.
You could parse the expression using libvala (which is part of the compiler).
The compiler creates a CodeContext and runs the Vala parser over a (or several) .vala file(s).
You could then create your own CodeVisitor decendant class that visits the necessary nodes of the parse tree and evaluates expressions.
As far as I can see there is no expression evaluator that does this, yet. That is because normally vala code is translated to C code and the C compiler then does compile time expression evaluation or the finished executable does the run time evaluation.
Python is different, because it is primarily a scripting language and has evaluation build directly into the runtime / interpreter.
My delphi application runs scripts using JvInterpreter (from the Jedi project).
A feature I use is runtime evaluation of expressions.
Script Example:
[...]
ShowMessage(X_SomeName);
[...]
JvInterpreter doesn't know X_SomeName.
When X_SomeName's value is required the scripter calls its OnGetValue-callback.
This points to a function I handle. There I lookup X_SomeName's value and return it.
Then JvInterpreter calls ShowMessage with the value I provided.
Now I consider switching to DelphiWebScript since it has a proper debug-interface and should also be faster than JvInterpreter.
Problem: I didn't find any obvious way to implement what JvInterpreter does with its OnGetValue/OnSetValue functions, though.
X_SomeName should be considered (and actually is, most of the time) a variable which is handled by the host application.
Any Ideas?
Thanks!
You can do that through the language extension mechanism, which has a FindUnknownName method that allows to register symbols on the spot.
It is used in the asm lib module demo, and you can also check the new "AutoExternalValues" test case in ULanguageExtensionTests, which should be closer to what you're after.
I've read on the Lua wiki / here / etc. on how to sandbox lua code generally. But I haven't been able to find something that disallows function creation. For example, the example here provides a sample code as:
assert(run [[function f(x) return x^2 end; t={2}; t[1]=f(t[1])]])
And that's with an empty environment. But I want to eliminate the ability to create a function (the 1st part of the code) - e.g., just allow expressions. Any idea on how to do that? Does it have to be in C somehow? Thanks in advance!
If you want to evaluate expressions only, you could try this:
function run(s) return loadstring("return "..s)() end
(error handling omitted)
This simple solution will prevent most `attacks', but not eliminate them because one can say
(function () f=function(x) print"hello" end end)()
which defines a new function named f.
Your best bet is to use a sandbox and not worry about what the user does to the environment, because it'll not be your environment.
You can try detecting the creation of functions by looking for the string "function" before allowing the execution of the lua script. For example from your C/C++ backend.
If "function" appears throw a "you are not allowed to create functions" error and don't execute the code.
A couple notes:
You might want to try to customize the detection a bit more - only throw errors if you detect function followed by blanks and an opening parenthesis, for example. I'm leaving that as an exercise.
You should be aware that there are some standard lua functions that kindof expect the users to be able to create functions - for example, the string table has several of those. Without creating functions, it'll be very difficult for your users to work with strings (it is already difficult enough with functions...)
Is it possible to check if a lua script contains errors without executing it? I have fallowing code:
if(luaL_loadbuffer(L, data, size, name))
{
fprintf (stderr, "%s", lua_tostring (L, -1));
lua_pop (L, 1);
}
if(lua_pcall(L, 0, 0, 0))
{
fprintf (stderr, "%s", lua_tostring (L, -1));
lua_pop (L, 1);
}
But if the script contains errors it passes first if and it is executed. I want to know if it contains errors when I load it, not when I execute it. Is this possible?
You can use the LUA Compiler. It will only compile your file to bytecode without executing it.
Your program will also have the advantage the run faster if it is compiled.
You can even use the -p option to only perform a syntax checking, according to the linked man page :
-p load files but do not generate any output file. Used mainly for syntax checking or testing precompiled chunks: corrupted files will probably generate errors when loaded. For a thourough integrity test, use -t.
(This was originally meant as a reply to the first comment to Krtek's question, but I ran out of space there and to be honest it works as an answer just fine.)
Functions are essentially values, and thus a named function is actually a variable of that name. Variables, by their very definition, can change as a script is executed. Hell, someone might accidentally redefine one of those functions. Is that bad? To sum my thoughts up: depending on the script, parameters passed and/or actual implementations of those pre-defined functions you speak of (one might unset itself or others, for example), it is not possible to guarantee things work unless you are willing to narrow down some of your demands. Lua is too dynamic for what you are looking for. :)
If you want a flawless test: create a dummy environment with all bells and whistles in place, and see if it crashes anywhere along the way (loading, executing, etc). This is basically a sort of unit test, and as such would be pretty heavy.
If you want a basic check to see if a script has a valid syntax: Krtek gave an answer for that already. I am quite sure (but not 100%) that the lua equivalent is to loadfile or loadstring, and the respective C equivalent is to try and lua_load() the code, each of which convert readable script to bytecode which you would already need to do before you could actually execute the code in your normal all-is-well usecase. (And if that contained function definitions, those would need to be executed later on for the code inside those to execute.)
However, these are the extent of your options with regards to pre-empting errors before they actually happen. Lua is a very dynamic language, and what is a great strength also makes for a weakness when you want to prove correctness. There are simply too many variables involved for a perfect solution.
In general it is not possible, as Lua is a dynamic language, and most of errors happen in runtime.
If you want to check for syntax errors, use luac -p option. I use it as a part of my pre-commit hook, for example.
Other common errors are triggering by misusing the global variables. You may analyze output of luac -l to catch these cases. See here: http://lua-users.org/wiki/DetectingUndefinedVariables.
If you want something more advanced, there are several more-or-less functional static analysis tools for Lua code. Start with LuaInspect.
In any case, you are advised to write unit tests instead of just relying on static code checks. Less pain, more gain.
I'm using a closed-source application that loads Lua scripts and allows some customization through modifying these scripts. Unfortunately that application is not very good at generating useful log output (all I get is 'script failed') if something goes wrong in one of the Lua scripts.
I realize that dynamic languages are pretty much resistant to static code analysis in the way C++ code can be analyzed for example.
I was hoping though, there would be a tool that runs through a Lua script and e.g. warns about variables that have not been defined in the context of a particular script.
Essentially what I'm looking for is a tool that for a script:
local a
print b
would output:
warning: script.lua(1): local 'a' is not used'
warning: script.lua(2): 'b' may not be defined'
It can only really be warnings for most things but that would still be useful! Does such a tool exist? Or maybe a Lua IDE with a feature like that build in?
Thanks, Chris
Automated static code analysis for Lua is not an easy task in general. However, for a limited set of practical problems it is quite doable.
Quick googling for "lua lint" yields these two tools: lua-checker and Lua lint.
You may want to roll your own tool for your specific needs however.
Metalua is one of the most powerful tools for static Lua code analysis. For example, please see metalint, the tool for global variable usage analysis.
Please do not hesitate to post your question on Metalua mailing list. People there are usually very helpful.
There is also lua-inspect, which is based on metalua that was already mentioned. I've integrated it into ZeroBrane Studio IDE, which generates an output very similar to what you'd expect. See this SO answer for details: https://stackoverflow.com/a/11789348/1442917.
For checking globals, see this lua-l posting. Checking locals is harder.
You need to find a parser for lua (should be available as open source) and use it to parse the script into a proper AST tree. Use that tree and a simple variable visibility tracker to find out when a variable is or isn't defined.
Usually the scoping rules are simple:
start with the top AST node and an empty scope
item look at the child statements for that node. Every variable declaration should be added in the current scope.
if a new scope is starting (for example via a { operator) create a new variable scope inheriting the variables in the current scope).
when a scope is ending (for example via } ) remove the current child variable scope and return to the parent.
Iterate carefully.
This will provide you with what variables are visible where inside the AST. You can use this information and if you also inspect the expressions AST nodes (read/write of variables) you can find out your information.
I just started using luacheck and it is excellent!
The first release was from 2015.