This question is related to but distinct from Exporting non-S3-methods with dots in the name using roxygen2 v4. From that post I learned that one needs to use #export function.name in order for the NAMESPACE to be written correctly by roxygen. I have done that and the NAMESPACE is correctly written.
My problem comes when I do R CMD Check. I have legacy code that has a function tail.g(). R CMD Check throws a NOTE noting that an apparent S3 method has been exported but not registered.
A reproducible example is below. Observe that xxxx.g doesn't have the NOTE, which leads me to believe that because tail is a generic in the utils package I need some special work-around. I'd prefer not to rename tail.g to tail_g because this is legacy code. I wish to eliminate all notes for a successful CRAN submission.
library(roxygen2)
package.skeleton("test")
writeLines(
"#' Check an argument
#'
#' Checks an argument.
#' #param ... Some arguments.
#' #return A value.
#' #export tail.g
tail.g <- function(...) 0",
"test/R/tail.g.R"
)
writeLines(
"#' Check an argument
#'
#' Checks an argument.
#' #param ... Some arguments.
#' #return A value.
#' #export xxxx.g
xxxx.g <- function(...) 0",
"test/R/xxxx.g.R"
)
roxygenise("test")
setwd("./test")
devtools::check(document=FALSE)
Gives the NOTE:
checking S3 generic/method consistency ... NOTE
Found the following apparent S3 methods exported but not registered:
tail.g
How do I eliminate the NOTE for tail.g() without renaming?
This is a dirty hack, but it works: just register the functions as methods, in addition to the export.
So your NAMESPACE could have two lines like this:
export(tail.g)
S3method(tail, g)
This appears to be enough to eliminate the warning and get a clean CRAN submission. The only negative side effect I can think of for regular users is a few bogus entries in the output of things like methods(class="g") or methods("tail"). Most regular users shouldn't even notice.
With roxygen2, you can use the #rawNamespace tag to include the required S3method directive in your NAMESPACE file:
writeLines(
"#' Check an argument
#'
#' Checks an argument.
#' #param ... Some arguments.
#' #return A value.
#' #export tail.g
#' #rawNamespace S3method(tail, g)
tail.g <- function(...) 0",
"test/R/tail.g.R"
)
Related
kedro recommends storing parameters in conf/base/parameters.yml. Let's assume it looks like this:
step_size: 1
model_params:
learning_rate: 0.01
test_data_ratio: 0.2
num_train_steps: 10000
And now imagine I have some data_engineering pipeline whose nodes.py has a function that looks something like this:
def some_pipeline_step(num_train_steps):
"""
Takes the parameter `num_train_steps` as argument.
"""
pass
How would I go about and pass that nested parameters straight to this function in data_engineering/pipeline.py? I unsuccessfully tried:
from kedro.pipeline import Pipeline, node
from .nodes import split_data
def create_pipeline(**kwargs):
return Pipeline(
[
node(
some_pipeline_step,
["params:model_params.num_train_steps"],
dict(
train_x="train_x",
train_y="train_y",
),
)
]
)
I know that I could just pass all parameters into the function by using ['parameters'] or just pass all model_params parameters with ['params:model_params'] but it seems unelegant and I feel like there must be a way. Would appreciate any input!
(Disclaimer: I'm part of the Kedro team)
Thank you for your question. Current version of Kedro, unfortunately, does not support nested parameters. The interim solution would be to use top-level keys inside the node (as you already pointed out) or decorate your node function with some sort of a parameter filter, which is not elegant either.
Probably the most viable solution would be to customise your ProjectContext (in src/<package_name>/run.py) class by overwriting _get_feed_dict method as follows:
class ProjectContext(KedroContext):
# ...
def _get_feed_dict(self) -> Dict[str, Any]:
"""Get parameters and return the feed dictionary."""
params = self.params
feed_dict = {"parameters": params}
def _add_param_to_feed_dict(param_name, param_value):
"""This recursively adds parameter paths to the `feed_dict`,
whenever `param_value` is a dictionary itself, so that users can
specify specific nested parameters in their node inputs.
Example:
>>> param_name = "a"
>>> param_value = {"b": 1}
>>> _add_param_to_feed_dict(param_name, param_value)
>>> assert feed_dict["params:a"] == {"b": 1}
>>> assert feed_dict["params:a.b"] == 1
"""
key = "params:{}".format(param_name)
feed_dict[key] = param_value
if isinstance(param_value, dict):
for key, val in param_value.items():
_add_param_to_feed_dict("{}.{}".format(param_name, key), val)
for param_name, param_value in params.items():
_add_param_to_feed_dict(param_name, param_value)
return feed_dict
Please also note that this issue has already been addressed on develop and will become available in the next release. The fix uses the approach from the snippet above.
As mentioned by Dmitry, kedro 0.16.0 introduced nested parameter values inside the node inputs which can be accessed via . operator:
node(func, "params:a.b", None)
whereas kedro 0.17.6 enabled overriding nested parameters with params in CLI, e.g.
kedro run --params="model.model_tuning.booster:gbtree"
I would like to create a procedure like this simple example:
proc name {args} {
foreach val $args {
puts $val
}
}
But I would like the procedure to handle variables that don't exist, something like the code shown below:
proc name {args} {
foreach val $args {
if [info exists $val] {
puts $val
}
}
}
The problem is that the code is not executed because as soon as I call the procedure with an unexisting variable it immediately stalls, prior to go into the code, saying that there is a variable that doesn't exist. Is it probable because the procedure checks argument existance before entering the body?.
I can make it work by changing args by several optional variables with predefined values, but that limits the procedure and makes it look bad.
Can I make a proc able to handle unexisting variables?
You can't pass a variable as an argument: arguments have to be values. You can pass a variable name as an argument and use that as a reference to the variable inside the procedure. Example:
proc name args {
foreach varname $args {
upvar 1 $varname var
if {[info exists var]} {
puts $var
}
}
}
(The call to upvar creates a link between the variable whose name is the value of the variable varname outside the procedure and the variable called var inside the procedure. This is one way to "pass a variable to a procedure".)
Then you can do this:
% set foo 1 ; set baz 3
% name foo bar baz
1
3
Note that if you try to invoke the procedure as
% name $bar
where bar is undefined, the interpreter tries (and fails) to evaluate it before calling the procedure. That might be what you are seeing.
Documentation:
upvar
If we look at the point where you are calling the command (procedures are commands; they're a subclass really) you'll see something like this in your code:
name $a $b $c
That's fine if all those variables exist, but if one doesn't, it will blow up even before name is called. In Tcl, $a means exactly “read the variable a and use its contents here”, unlike in some other languages where $ means “look out language, here comes a variable name!”
Because of this, we need to change the calling convention to be one that works with this:
name a b c
That's going to require the use of upvar. Like this:
proc name {args} {
foreach varName $args {
# Map the caller's named variable to the local name “v”
upvar 1 $varName v
# Now we can work with v in a simple way
if {[info exists v]} {
puts $v
}
}
}
You made a mistake here
if [info exists $val]
When info exists is used it should be checked against variable name, not the variable value.
Lets come to your actual question.
You can pass the arguments to the procedure as a key-value pair, then it is pretty simple.
proc user_info {args} {
#Converting the arguments into array
if {[catch {array set aArgs $args}]} {
puts "Please pass the arguments as key-value pair"
return 1
}
#Assume, we need to ensure these 3 arguments passed for sure.
set mandatoryArgs "-name -age -country"
foreach mArg $mandatoryArgs {
if {![info exists aArgs($mArg)]} {
puts "Missing mandatory argument '$mArg'"
return 1
}
}
}
user_info -name Dinesh
I would like to load a module into a modulefile (to resolve dependencies).
MyModule:
#%Module########################################
##
## Modulefile
#
proc ModulesHelp { } {
puts stderr "Env for MyProg"
}
proc addPath {var val} {
prepend-path $var $val
}
module load MyOtherModule
addPath PATH /opt/MyModule/bin
MyOtherModule:
#%Module########################################
##
## Modulefile
#
proc ModulesHelp { } {
puts stderr "Env for MyOtherProg"
}
proc addPath {var val} {
prepend-path $var $val
}
addPath PATH /opt/MyOtherModule/bin
When I run module load MyModule, both modules seem to be loaded but environment is not right :
$module list
Currently Loaded Modulefiles:
1) MyModule 2) MyOtherModule
$echo $PATH
/opt/MyModule/bin:/usr/bin:/bin
If I add the line foreach p [array names env] { set tmp $env($p) } or at least set tmp $env(PATH) in the MyModule after the module load MyOtherModule line, the environment is correctly modified. It also work fine if I don't use my function addPath but I use the prepend-path command directly, which is a bit annoying because I would like to do more things in the addPath function of course.
Anyone as an idea on what is going on and what I am missing ?
The prepend-path is probably doing some “clever” stuff to manage a variable; what exactly it is is something I don't know and don't need to know, because we can solve it all using generic Tcl. To make your wrapping of it work, use uplevel to evaluate the code in the proper scope, though you need to consider whether to use the global scope (name #0) or the caller's scope (1, which is the default); they're the same when your procedure addPath is called from the global level, but otherwise can be quite different, and I don't know what other oddness is going on with the modules system processing.
To demonstrate, try this addPath:
proc addPath {var val} {
puts stderr "BEFORE..."
uplevel 1 [list prepend-path $var $val]
puts stderr "AFTER..."
}
We use list to construct the thing to evaluate in the caller's scope, as it is guaranteed to generate substitution-free single-command scripts. (And valid lists too.) This is the whole secret to doing code generation in Tcl: keep it simple, use list to do any quoting required, call a helper procedure (with suitable arguments) when things get complicated, and use uplevel to control evaluation scope.
(NB: upvar can also be useful — it binds local variables to variables in another scope — but isn't what you're recommended to use here. I mention it because it's likely to be useful if you do anything more complex…)
I have a script like this:
param(
[Alias('a')]
[string]$aval,
[Alias('b')]
[switch]$bval,
[Alias('c')]
[string]$cval
)
if($aval.length -gt 1)
{
Do-Something
}
elseif($bval)
{
Do-Something-Else
}
elseif($cval.length -gt 1)
{
Do-Another-Thing
}
else
{
Do-This
}
If someone calls my script like so, an ugly error is displayed saying it is missing an argument for parameter 'aval/bval/cval':
PS C:\> .\MyScript.ps1 -a
C:\MyScript.ps1 : Missing an argument for parameter 'aval'. Specify a
parameter of type 'System.String' and try again.
At line:1 char:18
+ .\MyScript.ps1 -n <<<<
+ CategoryInfo : InvalidArgument: (:) [MyScript.ps1], ParameterBindingException
+ FullyQualifiedErrorId : MissingArgument,MyScript.ps1
Is there any way to make a cleaner, possibly one line, error appear instead? Also, is there a better way to handle parameters then a list of elseif statements (my actual script has ~10 parameters)?
The script sometimes passes an argument with a parameter as well:
EX:
PS C:\> .\MyScript.ps1 -b ServerName
Thanks for any help!
There are a few things that you can look at here. First, if the parameter will never have an associated value and you just want to know if the script was called with the parameter or not, then use a [switch] parameter instead of a string.
Here is a very simple example of using a switch parameter:
param(
[switch]$a
)
if($a){
'Switch was present'
}else{
'No switch present'
}
Save that as a script and run it with and without the -a parameter.
If you will sometimes have the parameter present with some value being passed in but other times without the value, then give the parameter a default value when you define it:
[Alias('a')]
[string]$aval = '',
Then in your logic if something was passed in, the length of the string will be gt 1.
As for the if-then structure that you have, there are a plethora of options for handling this sort of logic. with the little bit of information that you have shared, I suspect that using switch structure will be the best plan:
Get-Help about_Switch
In django, we can do this:
views.py :
def A(request):
context = {test : 'test'}
return render_to_response('index.html', context , context_instance = RequestContext(request))
def B(request):
context = {}
return render_to_response('index.html', context , context_instance = RequestContext(request))
index.html:
{% if test %}
{{ test }}
{% endif %}
And have our template render without error, even if i use method B, where variable 'test' does not exist, but I still can put it in the template.
I want to do the same with pylons + mako, in controller :
foo.py
def A(self):
c.test = 'test'
return render('index.html')
def B(self):
return render('index.html')
index.html :
% if c.test:
${'c.test'}
% endif
In Django, I can do that, but in Pylons, I get an error, is there anyway to check wheter 'c.test' exists or not?
the error : AttributeError: 'ContextObj' object has no attribute 'test'
I had a similar issue where I had multiple views using the same template and needed to test if a variable was set. I looked at the docs chris referenced and found another way to solve this problem regardless of how mako.strict_undefined is set. Essentially you call the get() method on the context object. In your example you could do the following:
% if context.get('test', UNDEFINED) is not UNDEFINED:
${test}
% endif
or
${context.get('test', '')}
That will print the same as ${test} if it exists, and print an empty string if it doesn't.
Unfortunately you can't seem to use an in operator on context which would be the most intuitive.
From the mako Docs on Context Variables:
% if someval is UNDEFINED:
someval is: no value
% else:
someval is: ${someval}
% endif
The docs describe this as referencing variable names not in the current context. Mako will set these variables to the value UNDEFINED.
I check for variables like so:
% if not someval is UNDEFINED:
(safe to use someval)
However, if pylons/pyramid has strict_undefined=True setting, attempts to use the undefined variable results in a NameError being raised. They give a brief philisophical justification for doing it this way, instead of simply replacing un-set variables with empty strings which seems consistent with Python philosophy. Took me a while to find this, but reading that entire section on the Mako Runtime will clear up how Mako recieves, sets, and uses context variables.
Edit:
For completions sake, the documents explain the strict_undefined setting here. You can change this variable by setting it in one of your .ini files:
[app:main]
...
mako.strict_undefined = false
a bit late, so whenever you use a variable on your template that doesn't exist on your controller, pylons will raise an error, to disable the error, just put this in your environment.py :
config['pylons.strict_tmpl_context'] = False