I have a YAML file with this structure
key1: value 1
key2:
- value 21
- value 22
key3:
- key31: value 311
key32: value 321
key33: value 331
And I need to access value 331, so I have this code:
#Grab('org.yaml:snakeyaml:1.26')
import org.yaml.snakeyaml.Yaml
// parse the yaml
def yaml = new Yaml().load(new FileReader("file.yaml"))
def var1 = "key3"
def var2 = "key33"
println yaml.key3.key33 // this is getting value 331
println yaml.$var1.$var2 // this is getting null
Is there any other way to get this value with given variables?
Note this is part of a Jenkins pipeline.
Thank you so much.
This should work for you. You can simply use readYaml step to read the YAML file within the Jenkins Pipeline
def yaml = readYaml file: 'file.yaml'
def var1 = "key3"
def var2 = "key33"
println yaml.key3.key33
println yaml."$var1"."$var2"
As you can see above I have surrounded your dynamic variables with double quotes, which will tell Jenkins to interpolate them.
Related
I am completely new to Groovy and Jenkins. I have some pre defined variable in Groovy Script (of Jenkins pipeline) and need to pick any one variable from them dynamically based on job/user input.
The example context of requirement is as provided below.
Here variable env is my input and based on that I should get correct userid.
env = "dev" //Input
def stg_userid = "abc"
def dev_userid = "xyz"
uid_var_name = "${env}_userid"
print "${uid_var_name}" // It is giving "dev_userid"
print 'abc' if we give 'stg' for env ;
print 'xyz' if we give 'dev' for env
Tried searching online for dynamic variable name use case in Groovy , but didn't got anything useful.
usually it's question of complex variable (Map) that holds parameters for all possible environments
and you could get section of this configuration by environment name
env = "dev"
def config = [
dev: [
user: 'aaa',
url: 'aaa-url'
],
stg: [
user: 'zzz',
url: 'zzz-url'
]
]
def uid_var_name = config[env].user // returns "aaa"
I have a properties file where certain string shoudl be replaced with Jenkins parameter. I ahve tried using the variable directly in the Properties file which did not work.
properties file
DOCKER_TAG_SUFFIX=-REPLACE_RELEASE_VERSION
PROPERTY_FILE_PATH=someproperty
Jenkinsfile snippet
def jboss_parameters = readProperties file: jboss_propfile
jboss_parameters .replaceAll("RELEASE_VERSION",params.RELEASE_VERSION) # try1
jboss_parameters = readFile(jboss_propfile).replaceAll("REPLACE_RELEASE_VERSION",params.RELEASE_VERSION) # try2
# try 3
jboss_parameters.each{k,v ->
if (v == "REPLACE_RELEASE_VERSION" )
jboss_parameters.setProperty($k,params.RELEASE_VERSION)
}
# try 4
def jboss_source_file = new File(jboss_propfile)
def jboss_parameters = jboss_source_file.text.replace("REPLACE_RELEASE_VERSION",params.RELEASE_VERSION)
I am not able to find another way that works for me.
println jboss_parameters output
{DOCKER_TAG_SUFFIX=-REPLACE_RELEASE_VERSION, PROPERTY_FILE_PATH=someproperty}
The readProperties step returns a dictionary (map), not a string, that is crated from the properties file.
Your first attempt (# try1) fails because maps in groovy do not have a replaceAll function like strings have and therefore you will get an error.
Your third attempt (# try3) is failing because you are comparing the map values to REPLACE_RELEASE_VERSION without the - character and therefore the comparison always fails and no values are changed.
I tested the second attempt (# try 2) and it seems to be working, so i am not sure what is your issue, but it is easier to handle properties as a map instead of a string that is retuned from the readFile method.
So if you have only specific properties that need to be updated you can update them directly:
def jboss_parameters = readProperties file: jboss_propfile
jboss_parameters.DOCKER_TAG_SUFFIX = params.RELEASE_VERSION // update relevant property
Or if you have multiple properties that should be modified you can iterate and update each value using the collectEntries method. Something like:
def jboss_parameters = readProperties file: jboss_propfile
updated_parameters = jboss_parameters.collectEntries { key, value ->
[key, value.replaceAll("REPLACE_RELEASE_VERSION",params.RELEASE_VERSION)]
}
I have a build pipeline. I am still learning groovy. I am doing something simple like this
stage('test'){
def temp = 3 \\ reading this value from other env variable
while(temp != 1) {
temp=temp-1
echo temp
}
}
It always echoes 3 and while is never-ending.
When you read the value from the environment variable, you're getting it as a String
You need to convert it to an Integer
def temp = env.SOMETHING.toInteger()
Or, define it as an integer instead of using def:
int temp = env.SOMETHING
I have a set of child JenkinsFiles which get called from a main JenkinsFile. The main JenkinsFile defines a few variables, which I wan't to avoid defining in the child JenkinsFiles.
Is that possible?
Example of main JenkinsFile:
#!groovy
String var1 = "something"
String var2 = "something"
load 'JenkinsFile-child1'
load 'JenkinsFile-child2'
JenkinsFile-child1:
#!groovy
echo var1
echo var2
pipeline script:
node{
load 'main.groovy'
}
main.groovy
var1 = "world"
load 'child.groovy'
child.groovy
echo "hello $var1"
will print hello world
the key that in script you can declare variable (field) for current scope (for the whole script class)
and you do if with assignment to undeclared variable in script.
normally the following annotation also should work, but unfortunately not:
import groovy.transform.Field;
#Field String var1 = "world";
I use the following code to temporarily modify environment variables.
#contextmanager
def _setenv(**mapping):
"""``with`` context to temporarily modify the environment variables"""
backup_values = {}
backup_remove = set()
for key, value in mapping.items():
if key in os.environ:
backup_values[key] = os.environ[key]
else:
backup_remove.add(key)
os.environ[key] = value
try:
yield
finally:
# restore old environment
for k, v in backup_values.items():
os.environ[k] = v
for k in backup_remove:
del os.environ[k]
This with context is mainly used in test cases. For example,
def test_myapp_respects_this_envvar():
with _setenv(MYAPP_PLUGINS_DIR='testsandbox/plugins'):
myapp.plugins.register()
[...]
My question: is there a simple/elegant way to write _setenv? I thought about actually doing backup = os.environ.copy() and then os.environ = backup .. but I am not sure if that would affect the program behavior (eg: if os.environ is referenced elsewhere in the Python interpreter).
I suggest you the following implementation:
import contextlib
import os
#contextlib.contextmanager
def set_env(**environ):
"""
Temporarily set the process environment variables.
>>> with set_env(PLUGINS_DIR=u'test/plugins'):
... "PLUGINS_DIR" in os.environ
True
>>> "PLUGINS_DIR" in os.environ
False
:type environ: dict[str, unicode]
:param environ: Environment variables to set
"""
old_environ = dict(os.environ)
os.environ.update(environ)
try:
yield
finally:
os.environ.clear()
os.environ.update(old_environ)
EDIT: more advanced implementation
The context manager below can be used to add/remove/update your environment variables:
import contextlib
import os
#contextlib.contextmanager
def modified_environ(*remove, **update):
"""
Temporarily updates the ``os.environ`` dictionary in-place.
The ``os.environ`` dictionary is updated in-place so that the modification
is sure to work in all situations.
:param remove: Environment variables to remove.
:param update: Dictionary of environment variables and values to add/update.
"""
env = os.environ
update = update or {}
remove = remove or []
# List of environment variables being updated or removed.
stomped = (set(update.keys()) | set(remove)) & set(env.keys())
# Environment variables and values to restore on exit.
update_after = {k: env[k] for k in stomped}
# Environment variables and values to remove on exit.
remove_after = frozenset(k for k in update if k not in env)
try:
env.update(update)
[env.pop(k, None) for k in remove]
yield
finally:
env.update(update_after)
[env.pop(k) for k in remove_after]
Usage examples:
>>> with modified_environ('HOME', LD_LIBRARY_PATH='/my/path/to/lib'):
... home = os.environ.get('HOME')
... path = os.environ.get("LD_LIBRARY_PATH")
>>> home is None
True
>>> path
'/my/path/to/lib'
>>> home = os.environ.get('HOME')
>>> path = os.environ.get("LD_LIBRARY_PATH")
>>> home is None
False
>>> path is None
True
EDIT2
A demonstration of this context manager is available on GitHub.
_environ = dict(os.environ) # or os.environ.copy()
try:
...
finally:
os.environ.clear()
os.environ.update(_environ)
I was looking to do the same thing but for unit testing, here is how I have done it using the unittest.mock.patch function:
def test_function_with_different_env_variable():
with mock.patch.dict('os.environ', {'hello': 'world'}, clear=True):
self.assertEqual(os.environ.get('hello'), 'world')
self.assertEqual(len(os.environ), 1)
Basically using unittest.mock.patch.dict with clear=True, we are making os.environ as a dictionary containing solely {'hello': 'world'}.
Removing the clear=True will let the original os.environ and add/replace the specified key/value pair inside {'hello': 'world'}.
Removing {'hello': 'world'} will just create an empty dictionary, os.envrion will thus be empty within the with.
In pytest you can temporarily set an environment variable using the monkeypatch fixture. See the docs for details. I've copied a snippet here for your convenience.
import os
import pytest
from typing import Any, NewType
# Alias for the ``type`` of monkeypatch fixture.
MonkeyPatchFixture = NewType("MonkeyPatchFixture", Any)
# This is the function we will test below to demonstrate the ``monkeypatch`` fixture.
def get_lowercase_env_var(env_var_name: str) -> str:
"""
Return the value of an environment variable. Variable value is made all lowercase.
:param env_var_name:
The name of the environment variable to return.
:return:
The value of the environment variable, with all letters in lowercase.
"""
env_variable_value = os.environ[env_var_name]
lowercase_env_variable = env_variable_value.lower()
return lowercase_env_variable
def test_get_lowercase_env_var(monkeypatch: MonkeyPatchFixture) -> None:
"""
Test that the function under test indeed returns the lowercase-ified
form of ENV_VAR_UNDER_TEST.
"""
name_of_env_var_under_test = "ENV_VAR_UNDER_TEST"
env_var_value_under_test = "EnvVarValue"
expected_result = "envvarvalue"
# KeyError because``ENV_VAR_UNDER_TEST`` was looked up in the os.environ dictionary before its value was set by ``monkeypatch``.
with pytest.raises(KeyError):
assert get_lowercase_env_var(name_of_env_var_under_test) == expected_result
# Temporarily set the environment variable's value.
monkeypatch.setenv(name_of_env_var_under_test, env_var_value_under_test)
assert get_lowercase_env_var(name_of_env_var_under_test) == expected_result
def test_get_lowercase_env_var_fails(monkeypatch: MonkeyPatchFixture) -> None:
"""
This demonstrates that ENV_VAR_UNDER_TEST is reset in every test function.
"""
env_var_name_under_test = "ENV_VAR_UNDER_TEST"
expected_result = "envvarvalue"
with pytest.raises(KeyError):
assert get_lowercase_env_var(env_var_name_under_test) == expected_result
For unit testing I prefer using a decorator function with optional parameters. This way I can use the modified environment values for a whole test function. The decorator below also restores the original environment values in case the function raises an Exception:
import os
def patch_environ(new_environ=None, clear_orig=False):
if not new_environ:
new_environ = dict()
def actual_decorator(func):
from functools import wraps
#wraps(func)
def wrapper(*args, **kwargs):
original_env = dict(os.environ)
if clear_orig:
os.environ.clear()
os.environ.update(new_environ)
try:
result = func(*args, **kwargs)
except:
raise
finally: # restore even if Exception was raised
os.environ = original_env
return result
return wrapper
return actual_decorator
Usage in unit tests:
class Something:
#staticmethod
def print_home():
home = os.environ.get('HOME', 'unknown')
print("HOME = {0}".format(home))
class SomethingTest(unittest.TestCase):
#patch_environ({'HOME': '/tmp/test'})
def test_environ_based_something(self):
Something.print_home() # prints: HOME = /tmp/test
unittest.main()