So I was reading Programing in Lua 2nd Ed and I came across this paragraph here:
It is good programming style to use local variables whenever
possible. Local variables help you avoid cluttering the global
environment with unnecessary names. Moreover, the access to local
variables is faster than to global ones.
Could anyone explain why this is? And is this "feature" only in Lua, or is it in other languages aswell? (e.g. C, C++, Java)
The difference in running time is due to the difference between hash table lookup and array lookup. An interpreter might be able to place a local variable in a CPU register, but even without such cleverness local variables are faster to access.
Global variables in Lua are stored in tables. Generally, anyone can modify these tables, and therefore the interpreter has to lookup a value anew every time it is being accessed. Local variables on the other hand disappear only when they go out of scope. Therefore they can have fixed locations in an array.
The benchmark program below calls a dummy function in a loop. The benchmark shows how the running time goes up the more tables the program has to jump through.
Other dynamic languages should be expected to have similar characteristics; see for example the Python benchmark at the very end.
Some relevant links:
Optimising Using Local Variables (Lua)
Local Variables (Python performance tips)
Optimizing Global Variable/Attribute Access. A (withdrawn) Python proposal on the lookup of global vs. local objects.
File demo.lua:
local M = {}
_G.demo = M
function M.op(x) return x end
return M
File main.lua:
local M = require "demo"
op = demo.op
local outer_op = demo.op
function iter_op(n)
local inner_op = demo.op
for i = 1, n do
-- Example running times for n = 100,000,000 (Lua 5.2.0):
-- Lookup a table (demo or _G), then lookup 'op'
-- within that table:
--
-- demo.op(i) --> 0:40
-- _G.op(i) --> 0:39
-- Lookup 'op' within a known local table (M or the table of
-- globals):
--
-- M.op(i) --> 0:30
-- op(i) --> 0:30
-- Dereference a local variable declared inside or outside
-- of this iter_op() function:
--
-- inner_op(i) --> 0:23
-- outer_op(i) --> 0:22
end
end
iter_op(100000000)
File main.py:
import demo # Contains 'def op(x): return x'.
global_op = demo.op
def iter_op(n):
local_op = demo.op
for i in xrange(n):
# Example running times for n = 50,000,000 (Python 2.6.5):
# demo.op(i) # 0:50
# global_op(i) # 0:41
local_op(i) # 0:36
iter_op(50000000)
In any language local variables will be faster. You will need to understand what a register is and what the thread stack is to understand my explanation. Most local variables are implemented as a register variable or pushed near the top of the local stack, so they are generally accessed much more quickly. Global variables are stored further up the stack (if they are not on the heap) so computing their address to access them is slower.
I'm making some assumptions here about the inner workings of Lua, but this makes sense from a computer architecture standpoint.
Related
I have actually been wondering about this question for a long time, and I can recall reading something somewhere on the internet about loading all local variables into a table, but I could never find it again.
What I am wondering is if there is basically any way to dynamically create a local variable, similar to how you can dynamically create a global variable.
In Lua you can create dynamic global variables simply by saying:
local my_env = getfenv(1);
for i = 1, 10 do
my_env["OMG_DYNAMIC_GLOBAL_VARIABLE_"..i] = i * i;
end
But I have always wondered if there is a way to essentially do the same or a similar thing for local variables. Something like
for i = 1, 10 do
local["OMG_DYNAMIC_LOCAL_VARIABLE_"..i] = i * i;
end
would kind of be pseudo code for what I am trying to do.
As a second part to my question, I am wondering if there is any way to possibly iterate through every local variable in the current scope? I guess pseudo code for what I am trying to do would look something like:
local a = 123;
local b = 321;
LocalVars = {(local)}; --the table with the local variables in it.
for i,v in pairs(LocalVars) do print(i, v); end
I would appreciate any help regarding either part of my question.
Thanks
the table with the local variables in it
There is no table with local variables. Local variables do not exist in compiled Lua. There are simply locations on the Lua stack and/or upvalues. Even the names of local variables are lost during the compilation process.
Local variables are always static constructs of the Lua source code. They never exist within a table. If you need a non-static construct, use a table. That's what tables are for. That's (part of) why Lua globals are in a table.
I don't see the advantage of what you proposed over a regular local table except for syntactic sugar.
local locals = {}
locals.a = 123
locals.b = 321
for i,v in pairs(locals) do
print(i, v)
end
The syntax you proposed does not make the program clearer or easier to read in any way. In fact it even creates new problems. Does the following print c as well or does it not because it is in a nested local scope?
for i,v in pairs(LocalVars) do
local c = 1729
print(i, v)
end
Suppose I have the following two Lua files:
In a.lua:
local x = 5
f = dofile'b.lua'
f()
In b.lua:
local fun = function()
print(x)
end
return fun
Then if I run luajit a.lua in shell it prints nil since x cannot be seen in the function defined in b.lua. The expected printing should be 5. However if I put everything in a single file then it's exactly what I want:
In aa.lua:
local x = 5
local f = function()
print(x)
end
f()
Run luajit aa.lua it prints 5.
So why x cannot be seen in the first case?
As their name suggests, local variables are local to the chunk.
dofile() loads the chunk from another file. Since it's another chunk, it makes sense that the local variable x in the first chunk isn't seen by it.
I agree that it is somewhat unintuitive that this doesn't work.
You'd like to say, at any point in the code there is a clear set of variables that are 'visible' -- some may be local, some may be global, but there is some map that the interpreter can use to resolve names of either kind.
When you load a chunk using dofile, then it can see whatever global variables currently exist, but apparently it can't see any local variables. We know that 'dofile' is not like C/C++ inclusion macros, which would give exactly the behavior you describe for local variables, but still you might reasonably expect that this part of it would work the same.
Ultimately there's no answer but "that's just not how they specified the language". The only satisfying answer is probably along the lines 'because otherwise it would cause non-obvious problem X' or 'because then use-case Y would go slower'.
I think the best answer is that, if all names were dynamically rebound according to the scope in which they are loaded when you use loadfile / dofile, that would inhibit a lot of optimization and such when compiling chunks into bytecode. In the lua system, name resolution works like 'either it is local in this scope, and then it binds to that (known) object, or, it is a lookup in the (unique) global table.' This system is pretty simple, there are only a few options and not a lot of room for complexity.
I don't think that running byte code even keeps track of the names of local variables, it discards them after the chunk is compiled. They would have to undo that optimization if they wanted to allow dynamic name resolution at chunk loading time like you suggest.
If your question is not really why but how can I make it work anyways, then one way you can do it is, in the host script, put any local variables that you want to be visible in the environment of the script that is called. When you do this you need to split dofile into a few calls. It's slightly different in lua 5.1 vs lua 5.2.
In lua 5.1:
In a.lua:
local shared = { x = 5 }
temp = loadfile('b.lua')
setfenv(temp, shared)
f = temp()
f()
In lua 5.2:
In a.lua:
local shared = { x = 5 }
temp = loadfile('b.lua', 't', shared)
f = temp()
f()
The x variable defined in module a.lua cannot be seen from b.lua because it was declared as local. The scope of a local variable is its own module.
If you want x to be visible from b.lua, just need to declare it global. A variable is either local or global. To declare a variable as global, just simply do not declare it as local.
a.lua
x = 5
f = dofile'b.lua'
f()
b.lua
local fun = function()
print(x)
end
return fun
This will work.
Global variables live within the global namespace, which can be accessed at any given time via the _G table. When Lua cannot solve a variable, because it's not defined inside the module where is being used, Lua searches that variable in the global namespace. In conclusion, it's also possible to write b.lua as:
local fun = function()
print(_G["x"])
end
return fun
At the begining of some lua package files, sometimes there will be the line local base = _G or local base = ....
What's the benefits for doing this?
What's the differences between these two lines?
For the first question, you can refer: Why make global Lua functions local?
For your second one,
What's the differences between these two lines?
When you do local base = _G, you are assigning base to be a synonym for the global environment table. On the other hand, in the statement local base = ...; the ... refer to vararg feature of lua.
It can be shown in better detail with the following example:
local a = {...}
for k, v in pairs(a) do print(k, v) end
and then, executing it as follows:
─$ lua temp.lua some thing is passed "here within quotes"
1 some
2 thing
3 is
4 passed
5 here within quotes
As you see, ... is just a list of arguments passed to the program. Now, when you have
local base = ...
lua assigns the first argument to the variable base. All other parameters will be ignored in the above statement.
In Programming in Lua (3rd Ed.) by Roberto Ierusalimschy it is stated that
A common idiom in Lua is
local foo = foo
This code creates a local
variable, foo, and initializes it with the value of the global
variable foo. (The local foo becomes visible only after its
declaration.) This idiom is useful when the chunk needs to preserve
the original value of foo even if later some other function changes
the value of the global foo; it also speeds up the access to foo.
Could someone explain this more in detail and provide a simple example?
At the moment, the only use I can think of for this idiom is to manage local variables (in a given block) that have the same names as global variables, so that the global variable is left unchanged after the block.
An example:
foo = 10
do
local foo = foo
foo = math.log10(foo)
print(foo)
end
print(foo)
this gives:
1
10
But the same could be accomplished without using the idiom at all:
bar = 10
do
local bar = math.log10(bar)
print(bar)
end
print(bar)
that gives the same result. So my explanation doesn't seem to hold.
I've seen this used more often as a optimization technique than as a way to preserve original values. With the standard Lua interpreter, every global variable access and module access requires a table lookup. Local variables, on the other hand, have statically-known locations at bytecode-compile time and can be placed in VM registers.
In more depth: Why are local variables accessed faster than global variables in lua?
The explanation is correct; I'm not sure why you are not satisfied with your example. To give you a real example:
local setfenv = setfenv
if not setfenv then -- Lua 5.2+
setfenv = function() ..... end
end
Another reason is to preserve the value as it is at this moment, so that other functions that use that value (in a file or a module) would have the same expectations about that value.
Wrapping a global:
do
local setmetatable = setmetatable
function _ENV.setmetatable(...)
-- Do your thing
return setmetatable(...)
end
end
Reducing overhead by using a local instead of doing a lookup in the globals-table (which is a local btw.):
local type = type
for k, v in next, bigtable do
if type(v) == "string" then
-- Do one thing
else
-- Do other thing
end
end
I think you're splitting hairs, unintentionally.
local bar = math.log10(bar)
is essentially the same as local bar = bar in spirit, but we it would be less useful to claim that the idiom is local bar = a(bar), because we may want to handle the local in some way other than passing it to a function first -- e.g. appending it to something.
This point is that we want to refer to the local bar, just as you say, not exactly how the conversion from global to local is done.
I just started with Lua as part of a school assignment. I'd like to know if there's an easy way to implement profiling for Lua? I need something that displays allocated memory, variables in use regardless of their type, etc.
I've been finding C++ solutions which I have been able to compile successfully but I don't know how to import them to the Lua environment.
I also found Shinny but I couldn't find any documentation about how to make it work.
There are several profilers available that you can check, but most of them target execution time (and are based on debug hook).
To track variables, you will need to use debug.getlocal and debug.getupvalue (from your code or from debug hook).
To track memory usage you can use collectgarbage(count) (probably after collectgarbage(collect)), but this only tells you total memory in use. To track individual data structures, you may need traverse global and local variables and calculate the amount of space they take. You can check this discussion for some pointers and implementation details.
Something like this would be the simplest profiler that tracks function calls/returns (note that you should not trust absolute numbers it generates, only relative ones):
local calls, total, this = {}, {}, {}
debug.sethook(function(event)
local i = debug.getinfo(2, "Sln")
if i.what ~= 'Lua' then return end
local func = i.name or (i.source..':'..i.linedefined)
if event == 'call' then
this[func] = os.clock()
else
local time = os.clock() - this[func]
total[func] = (total[func] or 0) + time
calls[func] = (calls[func] or 0) + 1
end
end, "cr")
-- the code to debug starts here
local function DoSomethingMore(x)
x = x / 2
end
local function DoSomething(x)
x = x + 1
if x % 2 then DoSomethingMore(x) end
end
for outer=1,100 do
for inner=1,1000 do
DoSomething(inner)
end
end
-- the code to debug ends here; reset the hook
debug.sethook()
-- print the results
for f,time in pairs(total) do
print(("Function %s took %.3f seconds after %d calls"):format(f, time, calls[f]))
end