I have an application that uses lua files for some of its more obscure configuration options. As such it mostly contains calls into the app to create things and alter properties; most C functions don't have a return value but some do.
I now have a need to read these same configuration files into a different application, and perform significantly different things when the functions are called (so I can't use common code). In addition, I'm only interested in a subset of the possible functions, and I think I can get away with by default ignoring (and/or returning nil) any other function call.
So I'm wondering what the best approach is here. How (from a C++ app), can I load and execute a lua script such that expressions etc are evaluated as normal but I can intercept and process certain app-defined C functions while simply ignoring (returning nil if required) calls to any other C functions?
(Note: I do have access to the vocabulary of the original app, which mostly uses luabind; I could just use the same definitions and change the implementation, but that's too fragile since the original app can have more functions added to it later. I would like something more generic.)
The goal is to get a bit of C code which I can use as a generic placeholder; the end result being "anything that's defined (standard library routines, functions defined in Lua, and C functions explicitly registered), call it as normal; for anything else, call one specific routine that simply does nothing, instead of raising an error". And preferably something compatible with luabind.
The whole process is initiated by a bit of C code that sets up the Lua environment, loads a set of files, calls one function, and then destroys the environment. There won't be anything ongoing.
Set a __call metamethod for nil:
debug.setmetatable(nil, { __call=function () end })
The _index metamethod for _G or other tables does not work because the name is resolved before the call.
(You don't need to use the debug API or library if you're setting this from C.)
How about using setfenv and a metatable? You can replace global environment table of certain function with an empty table. And then, set the placeholder function to ignore C-defined function.
local env = {} -- empty environment
local metatbl = {}
function metatbl.__index (tbl, key) -- provides placeholder function
return function()
print(key .. " called")
return(nil)
end
end
setmetatable(env, metatbl)
function samplefunc() -- your Lua code goes here
globalfunction "xyz" -- calls placeholder function
end
setfenv(samplefunc, env)
samplefunc()
If you want to use build-in function such as print, you can push it into env talbe like:
local env = {print = print}
If you have some way of differentiating the C functions from the ordinary Lua functions in your script, you can iterate over all functions defined in your Lua system, and for each C function that isn't on the list of functions you care about, replace that function's implementation with a simple nil-returning one. This is really easy if all the c-binding functions are defined tidily together in a table; if they're in _G you have a bit of a job ahead of you. The functions you do care about get bound via luabind as normal.
Related
So I have a Wow addon which is many, many thousands of lines long. Sometimes, Blizzard removes Lua functions from the game and I'm not always sure whether the functions I've called in the addon are valid anymore.
Is there a way that I can scan an entire Lua file for functions that no longer exist?
I know that I can do something like this:
if not RemovedFunction then print("Function does not exist") end
But that requires me to check every function name one at a time so that's not realistic (there are hundreds).
I would like to be able to scan the entire Lua file and alert me if any function names that I've called are no longer present in World of Warcraft API.
The solution can be written in any language (does not have to be Lua script although that would be preferable).
You can find a changelog here
https://wow.gamepedia.com/Patch_9.0.1/API_changes
https://github.com/Ketho/BlizzardInterfaceResources/compare/8.3.7...9.0.1#diff-ca64e26bbd77a0dd7b6d3699c75cbf60dd7ac03e379b8085dd7efcca4a52510e
The GitHub project shows how those functions are scanned, it's essentially just scanning _G[] and Blizzard's FrameXML code
You can also upload a zip to Globe which will tell you about any removed API, e.g.
https://www.townlong-yak.com/globe/#h:2a8385bff94e98a6bad8f5f09b45a148-reads
A good API would list removed function names in their changelog. Just search your script for those names.
You can use a linter like luacheck to find undefined stuff in your script. This would require you to maintain a list of existing function names.
This also works for other globals.
Or you search your script for function calls and search that name in a list of existing function names. Or you check if this name is nil in your environment. A function call is any name that is followed by optional whitespace and either of (, " or {.
For functions this is trivial, for other variables it becomes a bit more complicated and you'll end up writing your own linter so you could as well just use an existing one.
Scan if the function name is nil
function isFunctionExist(functionName) --Function name in string
local func = load("return "..functionName")
if func == nil then
error("Invalid function name!") --There an invalid letter in your function name that cause load unable to load
end
return type(func()) == "function"
end
It also work on any lua interpreter not only in World of Warcraft addon
Lua will cause attempt to call nil value or similiar
So before run it we check whether is it nil or a function
load("return "..functionName) will return the functionName variable content even it nil then we check it with type if it an existent function it return "function"
type(func()) == "function" this line do the checking
Everybody knows that variables in Lua, if not explicitly defined as "local", will be global. This will sometimes cause problems, like overriding library functions, or unexpectedly providing a value for another global variable with the same name. So it should be very helpful if there's a way to find all global variables that is defined in a single Lua code file.
However, I failed to find any clue on this seemingly quite-popular problem. The best answer I can get online is using _G to print all global variables in the environment, which isn't of much help. I'm currently coding Lua in Intellij Idea with Emmylua, a powerful tool that can show global variables in a special style, and it can easily trace a global variable to its definition; but when the code becomes quite long, this will not help much either.
So basically, I just want to get a list of global variables defined in a given Lua code file. Either with a tool or with a wonderful function. If it can make things easier, we may presume the code file is a module. If it can further print the definition locations for these global variables, that's even better. Can somebody help me?
Lua doesn't have a way to tell when or where a global was introduced.
In the special that the value is a function, debug.getinfo may be able to help by telling you where the function is defined (which is often but not always the same place where the function is made global).
You can capture the needed information at the time the global is introduced. This can be done by setting a metatable with a __newindex method on the global table. This method will be called when a new global is introduced (but not when an existing global is overridden). In this method, you can figure out where the caller came from with debug.getinfo. Also beware, if any of your other code is trying to use a metatable on the global environment, you must play nicely with it. (It can only have one metatable.)
You can also avoid using the global table. One in-between way of doing this is to override the environment. In Lua 5.2 and Lua 5.3, this is done by declaring a local table called _ENV -- all accesses to the global table will instead access this table. (Actually, global accesses always use _ENV and _ENV is _G by default.) You can make this mostly invisible by giving this _ENV a metatable that forwards accesses to _G (or whatever other environment). The difference here is that __newindex will still be called even if a binding exists in _G, so this method can detect overrides.
Using _ENV, though is inherently local to a scope (e.g. each file needs to override it). Such a hook could be installed globally as well though. If you load your modules manually with the load function (unlikely), you can just supply a custom _ENV as an argument. If you use require, it is possible to get a hold of the loaded file before it is executed by overriding (or monkey patching) the Lua searcher in package.searchers[2]. This is the built-in function that require calls to find the file in your filesystem and then load it. The return value is the loaded function which require then runs. So, after it is loaded but before it is returned back to require, you could use debug.setupvalue to override the default _ENV value (if any).
Example code (only lightly tested):
local global_info = {}
local default_searcher2 = package.searchers[2]
package.searchers[2] = function(...)
local result = default_searcher2(...)
local parent_environment = _G
local my_env = setmetatable({}, {
__index = parent_environment,
__newindex = function(self, k, v)
local new_info = debug.getinfo(2)
-- keeping rich data like this could be a memory leak
-- if some globals are assigned repeatedly, but that
-- may still be okay in a debugging scenario
local history = global_info[k]
if history == nil then
history = {}
global_info[k] = history
end
table.insert(history, {info = new_info, value = v})
parent_environment[k] = v
end,
})
if type(result) == "function" then
debug.setupvalue(result, 1, my_env)
end
return result
end
function gethistory(name)
local history = global_info[name]
if history == nil then
print('"' .. name .. '" has never been defined...')
else
print('History for "' .. name .. '":')
for _, record in ipairs(history) do
print(record.info.short_src .. ": " .. record.info.currentline)
end
end
end
Note that the hook here will only apply to files required after this code has been run, and basically only applies to Lua files (not C libs) that get included via the built-in require. It doesn't set a metatable on the global environment, so not conflict there, but it could be circumvented if files access _G directly (or e.g. setup access to _G instead of _ENV in their own _ENV tables). Such things can also be accounted for, but it can be a rabbit hole depending on how "invisible" you need this patch to be.
In Lua 5.1, instead of _ENV, you have setfenv which I believe can be used to similar effect.
Also note that all the methods I'm outlining can only detect global accesses that actually get executed at runtime.
Yes. Local versus global is a binding issue, which is primarily established at compile-time. Setting a variable is, of course, well-determined at compile-time.
Lua provides the luac compiler, which takes the argument -l for list.
In Lua 5.1, there is the opcode SETGLOBAL. A column indicates the line number of the statement and the comment indicates the name of the global.
In 5.2 and later, there is the opcode SETTABUP. A column indicates the line number of the statement and the comment indicates the name of the table and key. "Globals" are in the table referenced by the _ENV upvalue.
So, you can easily find the line number of any statement that sets a global variable with the tools Lua provides.
BTW—under many module systems, a module script would not set any global variables.
Take a look at this code:
local urgent = {}
local capi =
{
client = client,
}
local client
do
client = setmetatable({}, {
__index = function(_, k)
client = require("awful.client")
return client[k]
end,
__newindex = error -- Just to be sure in case anything ever does this
})
end
I'm having trouble understanding what it does. It's from the awesome-wm project. These are the things I'm having trouble understanding:
client = client in the declaration of capi
setmetatable stuff inside do-end
client = client in the declaration of capi
This is defining what portion of the capi is available in this file's scope, If you look at the client.lua file you will see that the capi defined in it has client, mouse, screen, and awesome.
For each item defined in the capi table there is a corresponding .c file. These files define objects such as client. urgent.lua has visibility of that object, likely it is a global variable, that is how we can set client = client the second client refers to the global variable.
Here is an example of 2 files:
main.lua
bar = "Hello World!"
local foo = require('foo')
print(foo.bar)
foo.lua
local foo = {
bar = bar
}
return foo
The print function in main.lua will result in Hello World!
setmetatable stuff inside do-end
Here by warping the setmetatable in a do-end block the code is executing in a restricted scope. This is normally done to contain the block's local variables so that they do not persist after the code's execution.
That said that is not the purpose of this block as the block has no local variables. As I see it, the blocking is simply to show that the object being modified is the local variable of client and not the global variable of client.
Additionally the metatable here is used to prevent circular dependency loops, this is mentioned comments in some of the places where similar code appears in the project, such as client.lua where local screen is defined.
#Nifim answer is excellent. I just want to add more context on why this code exist in its proper historical context. Before Lua 5.2, the module system was different. There was a magic module() function defined in the core Lua library. When you made a module, you had to first make local version of all global variables before calling module() because otherwise it would run in its own global environment. "capi" stands for "Core API" or "C (language) API" depending on the weather. If Awesome was written today with all the knowledge we now have, there would not be a public "C language" API and they would always be hidden in the private section to increase flexibility. Right now setting "c.my_own_property" do a couple round trips between capi.client and awful.client just to accommodate all the legacy constraints.
Now, the metatable magic is a Lua pattern called meta-lazy-loading. Because the urgent is a submodule of awful.client, it cannot directly import awful.client without causing a circular dependency. Over time, as Awesome APIs became better defined, more and more refactoring were made and they often introduced weird dependencies to maintain some degree of backward compatibility. In the best universe, we would have disregarded all users config and just re-engineered the whole code to avoid these circular dependencies. However every time we do that all users of the said APIs wake up one morning and they cannot login into their computer anymore. So this kind of workaround exist to prevent such events in return for some weird code and maintenance burden.
I've heard mumblings around the internets about being able to hot-swap code in Lua similar to how it's done in Java, Erlang, Lisp, etc. However, 30 minutes of googling for it has turned up nothing. Has anyone read anything substantial about this? Anyone have any experience doing it? Does it work in LuaJIT or only in the reference VM?
I'm more interested in the technique as a shortcut in development/debugging than an upgrade path in a live environment.
Lua, and most scripting languages for that matter, do not support the most generalized form of "hot swapping" as you define it. That is, you cannot guaranteeably change a file on disk and have any changes in it propagate itself into an executing program.
However, Lua, and most scripting languages for that matter, are perfectly capable of controlled forms of hot swapping. Global functions are global functions. Modules simply load global functions (if you use them that way). So if a module loads global functions, you can reload the module again if it is changed, and those global function references will change to the newly loaded functions.
However, Lua, and most scripting languages for that matter, makes no guarantees about this. All that's happening is the changing of global state data. If someone copied an old function into a local variable, they can still access it. If your module uses local state data, the new version of the module cannot access the old module's state. If a module creates some kind of object that has member functions, unless those members are fetched from globals, these objects will always refer to the old functions, not the new ones. And so forth.
Also, Lua is not thread safe; you can't just interrupt a lua_State at some point and try to load a module again. So you would have to set up some specific point in time for it to check stuff out and reload changed files.
So you can do it, but it isn't "supported" in the sense that it can just happen. You have to work for it, and you have to be careful about how you write things and what you put in local vs. global functions.
As Nicol said, the language itself doesn't do it for you.
If you want to implement something like this yourself though, it's not that hard, the only thing "preventing" you is any "leftover" references (which will still point to the old code) and the fact require caches its return value in package.loaded.
The way I'd do it is by dividing your code into 3 modules:
the reloading logic at entry point (main.lua)
any data you want to preserve across reloads (data.lua)
the actual code to reload (payload.lua), making sure you don't keep any references to that (which is sometimes not possible when you e.g. have to give callbacks to some library; see below).
-- main.lua:
local PL = require("payload")
local D = require("data")
function reload(module)
package.loaded[module]=nil -- this makes `require` forget about its cache
return require(module)
end
PL.setX(5)
PL.setY(10)
PL.printX()
PL.printY()
-- .... somehow detect you want to reload:
print "reloading"
PL = reload("payload") -- make sure you don't keep references to PL elsewhere, e.g. as a function upvalue!
PL.printX()
PL.printY()
-- data.lua:
return {} -- this is a pretty dumb module, it's literally just a table stored in `package.loaded.data` to make sure everyone gets the same instance when requiring it.
-- payload.lua:
local D = require("data")
local y = 0
return {
setX = function(nx) D.x = nx end, -- using the data module is preserved
setY = function(ny) y = ny end, -- using a local is reset upon reload
printX = function() print("x:",D.x) end,
printY = function() print("y:", y) end
}
output:
x: 5
y: 10
reloading
x: 5
y: 0
you could flesh out that logic a bit better by having a "registry module" that keeps track of all the requiring/reloading for you and abstracts away any access into modules (thus allowing you to replace the references), and, using the __index metatable on that registry you could make it pretty much transparent without having to call ugly getters all over the place. this also means you can supply "one liner" callbacks that then actually just tail-call through the registry, if any 3rd party library needs that.
I am using the C Fuzzy API and I want to load function module contained in a file lets say mycalculator.lua. This seem to run fine however when I later try to run another file A.lua that requires 'mycalculator' it does not work unless the mycalculator.lua file is available on the file system to reload. I am trying to just load it into the system and then have it available without having the mycalculator.lua in the file system. It there any way to have lua system keep the definition without loading it again? Basically I convert the mycalculator.lua into a string and then run it. I don't want to put mycalculator.lua file into the file system, I just want to hand it over as a string and then be able to require it in the next string I pass to the stack Thanks
There is a difference between simply executing a Lua script and loading a Lua module. If you wish to load a Lua module, then you must actually load a Lua module exactly as a script would: by calling require.
Since you appear to be new to Lua, I should probably explain this. You've probably seen code like this in Lua scripts:
require 'mycalculator'
That is not some special statement to Lua. That is a function call. It is just some syntactic sugar for:
require('mycalculator')
Functions in Lua can be called with NAME VALUE syntax instead of NAME(...) syntax, but it only allows you to send one parameter. And the parameter must be a literal (or table constructor).
In order to call the Lua require function from C, you must use the Lua stack. You must fetch the function from the global table by using lua_getfield(L, LUA_GLOBALSINDEX, "require"); Then, you push a string onto the stack containing the name of the module to load. Then, you use lua_pcall or whatever Lua function calling function to call it.