I would like to load a module into a modulefile (to resolve dependencies).
MyModule:
#%Module########################################
##
## Modulefile
#
proc ModulesHelp { } {
puts stderr "Env for MyProg"
}
proc addPath {var val} {
prepend-path $var $val
}
module load MyOtherModule
addPath PATH /opt/MyModule/bin
MyOtherModule:
#%Module########################################
##
## Modulefile
#
proc ModulesHelp { } {
puts stderr "Env for MyOtherProg"
}
proc addPath {var val} {
prepend-path $var $val
}
addPath PATH /opt/MyOtherModule/bin
When I run module load MyModule, both modules seem to be loaded but environment is not right :
$module list
Currently Loaded Modulefiles:
1) MyModule 2) MyOtherModule
$echo $PATH
/opt/MyModule/bin:/usr/bin:/bin
If I add the line foreach p [array names env] { set tmp $env($p) } or at least set tmp $env(PATH) in the MyModule after the module load MyOtherModule line, the environment is correctly modified. It also work fine if I don't use my function addPath but I use the prepend-path command directly, which is a bit annoying because I would like to do more things in the addPath function of course.
Anyone as an idea on what is going on and what I am missing ?
The prepend-path is probably doing some “clever” stuff to manage a variable; what exactly it is is something I don't know and don't need to know, because we can solve it all using generic Tcl. To make your wrapping of it work, use uplevel to evaluate the code in the proper scope, though you need to consider whether to use the global scope (name #0) or the caller's scope (1, which is the default); they're the same when your procedure addPath is called from the global level, but otherwise can be quite different, and I don't know what other oddness is going on with the modules system processing.
To demonstrate, try this addPath:
proc addPath {var val} {
puts stderr "BEFORE..."
uplevel 1 [list prepend-path $var $val]
puts stderr "AFTER..."
}
We use list to construct the thing to evaluate in the caller's scope, as it is guaranteed to generate substitution-free single-command scripts. (And valid lists too.) This is the whole secret to doing code generation in Tcl: keep it simple, use list to do any quoting required, call a helper procedure (with suitable arguments) when things get complicated, and use uplevel to control evaluation scope.
(NB: upvar can also be useful — it binds local variables to variables in another scope — but isn't what you're recommended to use here. I mention it because it's likely to be useful if you do anything more complex…)
Related
I have a Rake task in my Rails project which executes openssl through a system call.
The code looks like this:
system('bash', '-c', 'openssl cms -verify...')
I need to run the command in bash rather than dash (which is default on Ubuntu) to use process substitution in the command.
I need to create a test with rspec which checks that, in this case, the argument verify was passed as expected.
I have tried the following:
expect(Kernel).to receive(:system) do |args|
expect(args[2]).to match(/verify/)
end
However, this only gives me the third letter in the first string sent to system - i.e. the letter s from bash - rather than the third argument sent in the system call.
What am I doing wrong here? Any suggestions would be much appreciated.
Args are being passed to the block as sequential arguments, so if you want to treat them as an array, you need a splat operator in do |*args|:
expect(Kernel).to receive(:system) do |*args|
expect(args[2]).to match(/verify/)
end
Just to take a step back, it's important to understand how block arguments work, since they are different from methods. For example:
def my_fn(*args)
yield(*args)
end
my_fn(1,2,3) { |args| print args }
# => 1
my_fn(1,2,3) { |a, b, c| print [a,b,c] }
# => [1,2,3]
my_fn(1,2,3) { |*args| print args }
# => [1,2,3]
So if you did do |args| (without the splat), you are assigning the args variable to the first argument passed to the block ("bash") and ignoring the other arguments.
When require is called in testt.lua which is one of two files the return is movee and movee.lua.
movee are for the most part a class to be required, but should be able to accept to be called direct with parameter.
movee.lua
local lib = {} --this is class array
function lib.moveAround( ... )
for i,direction in ipairs(arg) do
print(direction)
end
end
function lib.hello()
print("Hello water jump")
end
lib.moveAround(...)
return lib
testt.la
local move = require("movee")
Expected result is not to call lib.moveAround or print of file name when require is called.
Your expectations are incorrect. Lua, and most scripting languages for that matter, does not recognize much of a distinction between including a module and executing the Lua file which provides that module. Every function statement is a statement whose execution creates a function object. Until those statements are executed, those functions don't exist. Same goes for your local lib = {}. And so on.
Now, if you want to make a distinction between when a user tries to require your script as a module and when a user tries to execute your script on the command line (or via just loadfile or similar), then I would suggest doing the following.
Check the number of arguments the script was given. If no arguments were given, then your script was probably required, so don't do the stuff you don't want to do when the user requires your script:
local nargs = select("#", ...)
if(nargs > 0) then
lib.moveAround(...)
end
Solved by replacing
lib.moveAround(...)
with
local argument = {...}
if argument[1] ~= "movee" and argument[2] ~= "movee" then
lib.moveAround(...)
end
require("movee")
will execute the code within movee.lua
lib.moveAround(...)
is part of that code. Hence if you require "movee" you call lib.moveAround
If the expected result is not to call it, remove that line from your code or don't require that file.
I would like to create a procedure like this simple example:
proc name {args} {
foreach val $args {
puts $val
}
}
But I would like the procedure to handle variables that don't exist, something like the code shown below:
proc name {args} {
foreach val $args {
if [info exists $val] {
puts $val
}
}
}
The problem is that the code is not executed because as soon as I call the procedure with an unexisting variable it immediately stalls, prior to go into the code, saying that there is a variable that doesn't exist. Is it probable because the procedure checks argument existance before entering the body?.
I can make it work by changing args by several optional variables with predefined values, but that limits the procedure and makes it look bad.
Can I make a proc able to handle unexisting variables?
You can't pass a variable as an argument: arguments have to be values. You can pass a variable name as an argument and use that as a reference to the variable inside the procedure. Example:
proc name args {
foreach varname $args {
upvar 1 $varname var
if {[info exists var]} {
puts $var
}
}
}
(The call to upvar creates a link between the variable whose name is the value of the variable varname outside the procedure and the variable called var inside the procedure. This is one way to "pass a variable to a procedure".)
Then you can do this:
% set foo 1 ; set baz 3
% name foo bar baz
1
3
Note that if you try to invoke the procedure as
% name $bar
where bar is undefined, the interpreter tries (and fails) to evaluate it before calling the procedure. That might be what you are seeing.
Documentation:
upvar
If we look at the point where you are calling the command (procedures are commands; they're a subclass really) you'll see something like this in your code:
name $a $b $c
That's fine if all those variables exist, but if one doesn't, it will blow up even before name is called. In Tcl, $a means exactly “read the variable a and use its contents here”, unlike in some other languages where $ means “look out language, here comes a variable name!”
Because of this, we need to change the calling convention to be one that works with this:
name a b c
That's going to require the use of upvar. Like this:
proc name {args} {
foreach varName $args {
# Map the caller's named variable to the local name “v”
upvar 1 $varName v
# Now we can work with v in a simple way
if {[info exists v]} {
puts $v
}
}
}
You made a mistake here
if [info exists $val]
When info exists is used it should be checked against variable name, not the variable value.
Lets come to your actual question.
You can pass the arguments to the procedure as a key-value pair, then it is pretty simple.
proc user_info {args} {
#Converting the arguments into array
if {[catch {array set aArgs $args}]} {
puts "Please pass the arguments as key-value pair"
return 1
}
#Assume, we need to ensure these 3 arguments passed for sure.
set mandatoryArgs "-name -age -country"
foreach mArg $mandatoryArgs {
if {![info exists aArgs($mArg)]} {
puts "Missing mandatory argument '$mArg'"
return 1
}
}
}
user_info -name Dinesh
tl;dr: What design pattern allows you to split Lua code over multiple files that need to share some information without affecting the global table?
Background
It is considered bad form to create a library in Lua where requiring the library affects the global namespace:
--> somelib.lua <--
SomeLib = { ... }
--> usercode.lua <--
require 'somelib'
print(SomeLib) -- global key created == bad
Instead, it is considered a best practice to create a library that uses local variables and then returns them for the user to assign as they see fit:
--> somelib.lua <--
local SomeLib = { ... }
return SomeLib
--> usercode.lua <--
local theLib = require 'somelib' -- consumers name lib as they wish == good
The above pattern works fine when using a single file. However, this becomes considerably harder when you have multiple files that reference each other.
Concrete Example
How can you rewrite the following suite of files so that the assertions all pass? Ideally the rewrites will leave the same files on disk and responsibilities for each file. (Rewriting by merging all code into a single file is effective, but not helpful ;)
--> test_usage.lua <--
require 'master'
assert(MASTER.Simple)
assert(MASTER.simple)
assert(MASTER.Shared)
assert(MASTER.Shared.go1)
assert(MASTER.Shared.go2)
assert(MASTER.Simple.ref1()==MASTER.Multi1)
assert(pcall(MASTER.Simple.ref2))
assert(_G.MASTER == nil) -- Does not currently pass
--> master.lua <--
MASTER = {}
require 'simple'
require 'multi'
require 'shared1'
require 'shared2'
require 'shared3'
require 'reference'
--> simple.lua <--
MASTER.Simple = {}
function MASTER:simple() end
--> multi.lua <--
MASTER.Multi1 = {}
MASTER.Multi2 = {}
--> shared1.lua <--
MASTER.Shared = {}
--> shared2.lua <--
function MASTER.Shared:go1() end
--> shared3.lua <--
function MASTER.Shared:go2() end
--> reference.lua <--
function MASTER.Simple:ref1() return MASTER.Multi1 end
function MASTER.Simple:ref2() MASTER:simple() end
Failure: Setting the Environment
I thought to solve the problem by setting the environment to my master table with a self-reference. This does not work when calling functions like require however, as they change the environment back:
--> master.lua <--
foo = "original"
local MASTER = setmetatable({foo="captured"},{__index=_G})
MASTER.MASTER = MASTER
setfenv(1,MASTER)
require 'simple'
--> simple.lua <--
print(foo) --> "original"
MASTER.Simple = {} --> attempt to index global 'MASTER' (a nil value)
You are giving master.lua two responsibilities:
It defines the common module table
It imports all of the submodules
Instead you should create a separate module for (1) and import it in all of the submodules:
--> common.lua <--
return {}
--> master.lua <--
require 'simple'
require 'multi'
require 'shared1'
require 'shared2'
require 'shared3'
require 'reference'
return require'common' -- return the common table
--> simple.lua <--
local MASTER = require'common' -- import the common table
MASTER.Simple = {}
function MASTER:simple() end
etc.
Finally, change the first line of test_usage.lua to use a local variable:
--> test_usage.lua <--
local MASTER = require'master'
...
The tests should now pass.
I have a systematic way to solve that problem. I have refactored your module in a Git repository to show you how it works: https://github.com/catwell/dont-touch-global-namespace/commit/34b390fa34931464c1dc6f32a26dc4b27d5ebd69
The idea is that you should have the sub-parts return a function that takes the main module as an argument.
If you cheat by opening the source files in master.lua, append a header and a footer and use loadstring, you can even use them unmodified (only master.lua has to be modified, but it is more complex). Personally, I prefer to keep it explicit, which is what I have done here. I don't like magic :)
EDIT: it is very close to Andrew Stark's first solution, except I patch the MASTER table directly in the sub-modules. The advantage is that you can define several things at once, like in your simple.lua, multi.lua and reference.lua files.
We can solve the problem by changing the master file to modify the environment in which all required code is run:
--> master.lua <--
local m = {} -- The actual master table
local env = getfenv(0) -- The current environment
local sandbox = { MASTER=m } -- Environment for all requires
setmetatable(sandbox,{__index=env}) -- ...also exposes read access to real env
setfenv(0,sandbox) -- Use the sandbox as the environment
-- require all files as before
setfenv(0,env) -- Restore the original environment
return m
The sandbox is an empty table that inherits values from _G but that also has a reference to the MASTER table, simulating a global from the perspective of later code. Using this sandbox as the environment causes all later requires to evaluate their "global" code in this context.
We save the real environment for later restoration, so that we don't mess with any later code that might want to actually set a global variable.
The question concerns:
Not polluting the global space when making modules.
Making modules in such a way that they might be split into multiple files, for maintenance reasons, among others.
My solution to the above problem lies in tweaking the "return as table" idiom in Lua such that instead of returning a table, you return a function that returns a table, when state needs to be passed between sub-modules.
This works well for sub-modules that are entirely dependent upon some root-module. If they are loaded independently, then they require the user to know that they need to call the module before they can use it. This is unlike every other module that has a collection of methods, ready to go from local a = require('a').
At any rate, this works like so:
--callbacks.lua a -- sub-module
return function(self)
local callbacks = {}
callbacks.StartElement = function(parser, elementName, attributes)
local res = {}
local stack = self.stack
---awesome stuff for about 150 lines...
return callbacks
end
To use it, you can...
local make_callbacks = require'callbacks'
self.callbacks = make_callbacks(self)
Or, better yet, simply call the return value of require when assigning the callback table to the parent module, like so:
self.callbacks = require'trms.xml.callbacks'(self)
Most often, I try not to do this. If I'm passing state or self between submodules, I find that I'm often doing it wrong. My internal policy is that if I'm doing something that is highly-related to another file, I might be okay. More likely, I'm putting something in the wrong spot and there is a way to do it without passing anything between modules.
The reason that I don't like this is that which I pass by table has methods and properties unseen in the file that I am working within. I'm not free to refactor the internal implementation of one of my files, without horking the others. So, I humbly suggest that this idiom is a yellow flag, but probably not a red one. :)
While this solves the problem of state-sharing without globals, it doesn't really protect the user from the accidental omission of local. If I may speak to that implied question...
The first thing that I do is remove access to the global environment from my module. Remembering that it's only available as long as I don't
reset _ENV, reseting it is the first thing that I do. This is done by packing only what is needed into a new _ENV table.
_ENV = {print = print,
pairs = pairs, --etc
}
However, constantly re-typing all of the things that I need from lua into each file is a giant, error-prone pain. To avoid this, I make one file in my module's base directory and use it as the home for all of my modules' and sub-modules' common environments. I call it _ENV.lua.
Note: I cannot use "init.lua" or any other root-module for this purpose, because I need to be able to load it from the sub-modules, which are being loaded by
the root-module, which loads the sub-modules, which are...
My abbreviated _ENV.lua file looks something like the following:
--_ENV.lua
_ENV = {
type = type, pairs = pairs, ipairs = ipairs, next = next, print =
print, require = require, io = io, table = table, string = string,
lxp = require"lxp", lfs = require"lfs",
socket = require("socket"), lpeg = require'lpeg', --etc..
}
return _ENV
With this file, I now have a common base from which to work.
All of my other modules load this first, using the following command:
_ENV = require'root_mod._ENV' --where root_mod is the base of my module.
This facility was critical for me, for two reasons. First, it keeps me
out of global space. If I see that I am missing something from the global environment _G (took me a surprisingly long time before I saw that I didn't have
tostring!), I can go back into my _ENV.lua file and add it. As
a required file, this only gets loaded one time, so having it applied
to all of my submodules is 0 calories.
Second, I find that it gives me everything that I really needed for using
the "return module as table" protocol, with only a few exceptions where "return a function that returns a table" is needed.
TL;DR: Don't return the module, set package.loaded[...] = your_module as early as possible (can still be empty), then just require the module in submodules and it will be properly shared.
The clean way to do this is to explicitly register the module and not rely on require to implicitly register it at the end. The documentation says:
require (modname)
Loads the given module. The function starts by looking into the
package.loaded table to determine whether modname is already loaded.
If it is, then require returns the value stored at
package.loaded[modname]. [This gets you the caching behavior that
every file is run only once.] Otherwise, it tries to find a loader for
the module. [And one of the searchers is looking for Lua files to run,
which gets you the usual file loading behavior.]
[…]
Once a loader is found, require calls the loader with two arguments:
modname and an extra value dependent on how it got the loader. (If the
loader came from a file, this extra value is the file name.) If the loader
returns any non-nil value [e.g. your file returns the module table],
require assigns the returned value to package.loaded[modname]. If the
loader does not return a non-nil value and has not assigned any value to
package.loaded[modname], then require assigns true to this entry.
In any case, require returns the final value of
package.loaded[modname].
(emphasis, [comments] added by me.)
With the return mymodule idiom, the caching behavior fails if you have a loop in your dependencies – the cache is updated too late. (As a result, files may be loaded several times (you may even get endless loops!) and sharing will fail.) But explicitly saying
local _M = { } -- your module, however you define / name it
package.loaded[...] = _M -- recall: require calls loader( modname, something )
-- so `...` is `modname, something` which is shortened
-- to just `modname` because only one value is used
immediately updates the cache, so that other modules can already require your module before its main chunk returned. (Of course, at that time they can only actually use what's already been defined. But that's not usually a problem.)
The package.loaded[...] = mymodule approach works in 5.1–5.3 (incl. LuaJIT).
For your example, you would adjust the start of master.lua to
1c1,2
< MASTER = {}
---
> local MASTER = {}
> package.loaded[...] = MASTER
and for all other files
0a1
> local MASTER = require "master"
and you're done.
In django, we can do this:
views.py :
def A(request):
context = {test : 'test'}
return render_to_response('index.html', context , context_instance = RequestContext(request))
def B(request):
context = {}
return render_to_response('index.html', context , context_instance = RequestContext(request))
index.html:
{% if test %}
{{ test }}
{% endif %}
And have our template render without error, even if i use method B, where variable 'test' does not exist, but I still can put it in the template.
I want to do the same with pylons + mako, in controller :
foo.py
def A(self):
c.test = 'test'
return render('index.html')
def B(self):
return render('index.html')
index.html :
% if c.test:
${'c.test'}
% endif
In Django, I can do that, but in Pylons, I get an error, is there anyway to check wheter 'c.test' exists or not?
the error : AttributeError: 'ContextObj' object has no attribute 'test'
I had a similar issue where I had multiple views using the same template and needed to test if a variable was set. I looked at the docs chris referenced and found another way to solve this problem regardless of how mako.strict_undefined is set. Essentially you call the get() method on the context object. In your example you could do the following:
% if context.get('test', UNDEFINED) is not UNDEFINED:
${test}
% endif
or
${context.get('test', '')}
That will print the same as ${test} if it exists, and print an empty string if it doesn't.
Unfortunately you can't seem to use an in operator on context which would be the most intuitive.
From the mako Docs on Context Variables:
% if someval is UNDEFINED:
someval is: no value
% else:
someval is: ${someval}
% endif
The docs describe this as referencing variable names not in the current context. Mako will set these variables to the value UNDEFINED.
I check for variables like so:
% if not someval is UNDEFINED:
(safe to use someval)
However, if pylons/pyramid has strict_undefined=True setting, attempts to use the undefined variable results in a NameError being raised. They give a brief philisophical justification for doing it this way, instead of simply replacing un-set variables with empty strings which seems consistent with Python philosophy. Took me a while to find this, but reading that entire section on the Mako Runtime will clear up how Mako recieves, sets, and uses context variables.
Edit:
For completions sake, the documents explain the strict_undefined setting here. You can change this variable by setting it in one of your .ini files:
[app:main]
...
mako.strict_undefined = false
a bit late, so whenever you use a variable on your template that doesn't exist on your controller, pylons will raise an error, to disable the error, just put this in your environment.py :
config['pylons.strict_tmpl_context'] = False