I am trying to replace the the DB cred based on the env name in Jenkins, but I am unable to achieve the same.
I have a JSON Config Files like this named 'JsonConfig'
{
"production": {
"DB_USERNAME": "userABC"
},
"development": {
"DB_USERNAME": "userXYZ"
}
}
and this what I have in Jenkinsfile
def getEnvName() {
if ("master".equals(env.BRANCH_NAME)) {
return "production";
}
return env.BRANCH_NAME;
}
def config;
node(){
configFileProvider([configFile(fileId: 'secret-credentials', targetLocation: 'JsonConfig')]) {
config = readJSON file: 'JsonConfig'
}
}
pipeline {
agent any
stages {
stage("Setup") {
when {
beforeAgent true
anyOf { branch 'master', branch 'development' }
}
steps {
sh """
sed -i 's#__DB_USERNAME__#config.${getEnvName()}.DB_USERNAME#' ./secret-data.yml
cat ./secret-data.yml
"""
//Alternative
sh "sed -i 's#__DB_USERNAME__#${config.getEnvName().DB_USERNAME}#' ./secret-data.yml"
}
}
}
}
If I statically pass the var name like this, then it is working fine.
sh "sed -i 's#__DB_USERNAME__#${config.production.DB_USERNAME}#' ./k8s/secret-data.yml"
I want to make "production" dynamic so that it reads the value which is returned from getEnvName() method.
The problematic line is
sh """
sed -i 's#__DB_USERNAME__#config.${getEnvName()}.DB_USERNAME#' ./secret-data.yml
"""
This will evaluate as the shell command
sed -i 's#__DB_USERNAME__#config.production.DB_USERNAME#' ./secret-data.yml
But you want to be evaluated to
sed -i 's#__DB_USERNAME__#userABC#' ./secret-data.yml
Since the config is a Groovy object representing the parsed JSON file we can access its properties dynamically using the subscript operator ([]):
sh """
sed -i 's#__DB_USERNAME__#${config[getEnvName()].DB_USERNAME}#' ./secret-data.yml
"""
Related
I am looking to use a database username/password in my config.ini file. I have the following withCredentials line in my Jenkinsfile:
withCredentials([usernamePassword(credentialsId: 'database', usernameVariable: 'DATABASE_USER', passwordVariable: 'DATABASE_PASSWORD')])
I don't explicitly call this config.ini file in my Jenkinsfile, however I do use a bash script to:
export CONFIG_FILE='config.ini'
Is there any way to set these accordingly in my config.ini:
DB_USERNAME = {DATABASE_USER}
DB_PASSWORD = {DATABASE_PASSWORD}
Bash can do this for you. You have two options:
Use envsubst. You'll need to install it on all of your nodes (it's usually part of the gettext package).
Use evil eval
Full example:
pipeline {
agent {
label 'linux' // make sure we're running on Linux
}
environment {
USER = 'theuser'
PASSWORD = 'thepassword'
}
stages {
stage('Write Config') {
steps {
sh 'echo -n "user=$USER\npassword=$PASSWORD" > config.ini'
}
}
stage('Envsubst') {
steps {
sh 'cat config.ini | envsubst > config_envsubst.ini'
sh 'cat config_envsubst.ini'
}
}
stage('Eval') {
steps {
sh 'eval "echo \"$(cat config.ini)\"" > config_eval.ini'
sh 'cat config_eval.ini'
}
}
}
}
This this Stackexchange question for more options.
I have a shared library that accept parameters i setup to compress files into a tar. The jenkinspipline looks like this.
stage("Package"){
steps{
compress_files("arg1", "arg2")
}
}
The shared library compress_file looks like this
#!/usr/bin/env groovy
// Process any number of arguments.
def call(String... args) {
sh label: 'Create Directory to store tar files.', returnStdout: true,
script: """ mkdir -p "$WORKSPACE/${env.PROJECT_NAME}" """
args.each {
sh label: 'Creating project directory.', returnStdout: true,
script: """ mkdir -p "$WORKSPACE/${env.PROJECT_NAME}" """
sh label: 'Coping contents to project directory.', returnStdout: true,
script: """ cp -rv ${it} "$WORKSPACE/${env.PROJECT_NAME}/." """
}
sh label: 'Compressing project directory to a tar file.', returnStdout: true,
script: """ tar -czf "${env.PROJECT_NAME}.tar.gz" "${env.PROJECT_NAME}" """
sh label: 'Remove the Project directory..', returnStdout: true,
script: """ rm -rf "$WORKSPACE/${env.PROJECT_NAME}" """
}
New requirement is to use an array instead of updating the argument values. How or can we pass an arrayname in the jenkinsfile stage
Yes it’s possible, from Jenkinsfile you can define the array inside stage() or outside stage() and make that use of, like
In declarative pipeline :
def files = ["arg1", "arg2"] as String[]
pipeline {
agent any
stages {
stage("Package") {
steps {
// script is optional
script {
// you can manipulate the variable value of files here
}
compress_files(files)
}
}
}
}
In scripted pipeline:
node() {
//You can define the value here as well
// def files = ["arg1", "arg2"] as String[]
stage("Package"){
def files = ["arg1", "arg2"] as String[]
compress_files(files)
}
}
And in the shared library, the method will be like
// var/compress_files.groovy
def call(String[] args) {
args.each {
// retrive the value from ${it} and proceed with your logic
}
}
or
def call(String... args) {
args.each {
// retrive the value from ${it} and proceed with your logic
}
}
I have a fairly simple script, but the Jenkinsfile never substitutes the variable (since) and I am not sure why.
I have tried the $since and ${since} syntax and each time the substitution is empty. The parameters work just fine.
since = ''
pipeline {
agent any
options {
buildDiscarder(logRotator(numToKeepStr: '5', artifactNumToKeepStr: '5'))
}
parameters {
string(defaultValue: '', description: 'The service name you wish to display status', name: 'serviceName')
choice(choices: ['swarm-group-test', 'swarm-group-prod'], description: 'Swarm environment to remove from', name: 'swarmEnv')
string(defaultValue: '5', description: 'Number of minutes to look back in logs', name: 'numMinutes')
}
tools {
nodejs "node-js-11.12.0"
}
stages {
stage('Invoke Playbook') {
steps {
script {
sh (script: "node -p -e \"var a = new Date(); a.setMinutes(a.getMinutes() - ${numMinutes}); a.toISOString();\"", returnStdout: true).trim()
}
echo "since: ${since}"
ansiColor('xterm') {
sh '''
export ANSIBLE_FORCE_COLOR=true
export ANSIBLE_STDOUT_CALLBACK=debug
ansible-playbook -b -v -u jenkins playbooks/display-service-status.yml -k --extra-vars="serviceName=$serviceName swarmEnv=$swarmEnv since=$since" -i playbooks/swarm-hosts
'''
}
}
}
}
}
Output:
since: 2019-09-30T12:43:12.134Z
ansible-playbook -b -v -u jenkins playbooks/display-service-status.yml -k '--extra-vars=serviceName=TEST_openam swarmEnv=swarm-group-test since=' -i playbooks/swarm-hosts
I believe that you cannot substitute Jenkins variables in single quote strings.
Here's a helpful document: https://gist.github.com/Faheetah/e11bd0315c34ed32e681616e41279ef4
Relevant question. A few things to remember:
Environment variables should be declared within an environment (reference). They can have global or stage scope.
Variables can also be declared with the def keyword in some specific locations/scopes.
We can reference variables using groovy-like interpolation, that is, $ and curly braces inside double quote strings (reference) .
Example:
environment {
since = 'hello'
varThatUsesAnotherVar = "${since} world"
}
stages {
stage('Invoke Playbook') {
steps {
echo "since: ${since}"
echo "varThatUsesAnotherVar : ${varThatUsesAnotherVar}"
}
}
}
I have defined global variable in Jenkins pipeline
def BUILDNRO = '0'
pipeline { ...
Then i manipulate variable with shell script to enable running builds parallel by using job build number as identifier so we don't mix different docker swarms.
stage('Handle BUILD_NUMBER') {
steps {
script {
BUILDNRO = sh( script: '''#!/bin/bash
Build=`echo ${BUILD_NUMBER} | grep -o '..$'`
# Check if BUILD first character is 0
if [[ $Build:0:1 == "0" ]]; then
# replace BUILD first character from 0 to 5
Build=`echo $Build | sed s/./5/1`
fi
echo $Build
''',returnStdout: true).trim()
}
}
}
i get value out from previos stage and trying to get global variable on next stage
stage('DOCKER: Init docker swarm') {
steps {
echo "BUILDNRO is: ${BUILDNRO}" --> Value is here.
sh '''#!/bin/bash
echo Buildnro is: ${BUILDNRO} --> This is empty.
...
}
}
This will out give global variable empty. why? in previous stage there was value in it.
EDIT 1.
Modified code blocks to reflect current status.
I managed to figure it out. Here is solution how i managed to did it.
BUILDNRO is groovy variable and if wanting to used in bash variable it have to pass using withEnv. BUILD_NUMBER in first stage is bash variable hence it can be used directly script in first stage.
def BUILDNRO = '0'
pipeline {
....
stages {
stage('Handle BUILD_NUMBER') {
steps {
script {
BUILDNRO = sh( script: '''#!/bin/bash
Build=`echo ${BUILD_NUMBER} | grep -o '..$'`
''',returnStdout: true).trim()
}
}
}
stage('DOCKER: Init docker swarm') {
steps {
dir("prose_env/prose_api_dev_env") {
withEnv(["MYNRO=${BUILDNRO}"]) {
sh(returnStdout: false, script: '''#!/bin/bash
echo Buildnro is: ${MYNRO}`
'''.stripIndent())
}
}
}
}
}
}
If you are using single quotes(```) in the shell module, Jenkins treats every variable as a bash variable. The solution is using double quotes(""") but then if you made bash variable you have to escape it. Below an example with working your use case and escaped bash variable
pipeline {
agent any
stages {
stage('Handle BUILD_NUMBER') {
steps {
script {
BUILDNRO = sh(script: 'pwd', returnStdout: true).trim()
echo "BUILDNRO is: ${BUILDNRO}"
}
}
}
stage('DOCKER: Init docker swarm') {
steps {
sh """#!/bin/bash
echo Buildnro is: ${BUILDNRO}
variable=world
echo "hello \${variable}"
sh """
}
}
}
}
output of the second stage:
Buildnro is: /var/lib/jenkins/workspace/stack1
hello world
I am trying to run the JenkinsFile below which contains two sed commands. But I faced different issues with string interpolation when I cat the file.
Do you know how I can run it inside the JenkinsFile?
Thanks in advance.
pipeline {
agent any
tools {nodejs "NodeJS 6.7.0"}
stages {
stage('checking out gitlab branch master') {
steps {
checkout([$class: 'GitSCM', branches: [[name: '*/development']]])
}
}
stage('executing release process') {
environment {
ARTIFACTORY_APIKEY = credentials('sandbox-gms-password')
}
steps {
sh 'cp bowerrc.template .bowerrc'
sh 'sed -i -e "s/username/zest-jenkins/g" .bowerrc'
sh 'sed -i -e "s/password/${ARTIFACTORY_APIKEY}/g" .bowerrc'
sh 'cat .bowerrc'
}
}
}
}
Put the commands in single "sh" block, please take the reference from the below:-
pipeline {
agent any
tools {nodejs "NodeJS 6.7.0"}
stages {
stage('checking out gitlab branch master') {
steps {
checkout([$class: 'GitSCM', branches: [[name: '*/development']]])
}
}
stage('executing release process') {
environment {
ARTIFACTORY_APIKEY = credentials('sandbox-gms-password')
}
steps {
sh '''
cp bowerrc.template .bowerrc
sed -i -e "s/username/zest-jenkins/g" .bowerrc
sed -i -e "s/password/${ARTIFACTORY_APIKEY}/g" .bowerrc
cat .bowerrc
'''
}
}
}
}