I have a fairly simple script, but the Jenkinsfile never substitutes the variable (since) and I am not sure why.
I have tried the $since and ${since} syntax and each time the substitution is empty. The parameters work just fine.
since = ''
pipeline {
agent any
options {
buildDiscarder(logRotator(numToKeepStr: '5', artifactNumToKeepStr: '5'))
}
parameters {
string(defaultValue: '', description: 'The service name you wish to display status', name: 'serviceName')
choice(choices: ['swarm-group-test', 'swarm-group-prod'], description: 'Swarm environment to remove from', name: 'swarmEnv')
string(defaultValue: '5', description: 'Number of minutes to look back in logs', name: 'numMinutes')
}
tools {
nodejs "node-js-11.12.0"
}
stages {
stage('Invoke Playbook') {
steps {
script {
sh (script: "node -p -e \"var a = new Date(); a.setMinutes(a.getMinutes() - ${numMinutes}); a.toISOString();\"", returnStdout: true).trim()
}
echo "since: ${since}"
ansiColor('xterm') {
sh '''
export ANSIBLE_FORCE_COLOR=true
export ANSIBLE_STDOUT_CALLBACK=debug
ansible-playbook -b -v -u jenkins playbooks/display-service-status.yml -k --extra-vars="serviceName=$serviceName swarmEnv=$swarmEnv since=$since" -i playbooks/swarm-hosts
'''
}
}
}
}
}
Output:
since: 2019-09-30T12:43:12.134Z
ansible-playbook -b -v -u jenkins playbooks/display-service-status.yml -k '--extra-vars=serviceName=TEST_openam swarmEnv=swarm-group-test since=' -i playbooks/swarm-hosts
I believe that you cannot substitute Jenkins variables in single quote strings.
Here's a helpful document: https://gist.github.com/Faheetah/e11bd0315c34ed32e681616e41279ef4
Relevant question. A few things to remember:
Environment variables should be declared within an environment (reference). They can have global or stage scope.
Variables can also be declared with the def keyword in some specific locations/scopes.
We can reference variables using groovy-like interpolation, that is, $ and curly braces inside double quote strings (reference) .
Example:
environment {
since = 'hello'
varThatUsesAnotherVar = "${since} world"
}
stages {
stage('Invoke Playbook') {
steps {
echo "since: ${since}"
echo "varThatUsesAnotherVar : ${varThatUsesAnotherVar}"
}
}
}
Related
I am trying to replace the the DB cred based on the env name in Jenkins, but I am unable to achieve the same.
I have a JSON Config Files like this named 'JsonConfig'
{
"production": {
"DB_USERNAME": "userABC"
},
"development": {
"DB_USERNAME": "userXYZ"
}
}
and this what I have in Jenkinsfile
def getEnvName() {
if ("master".equals(env.BRANCH_NAME)) {
return "production";
}
return env.BRANCH_NAME;
}
def config;
node(){
configFileProvider([configFile(fileId: 'secret-credentials', targetLocation: 'JsonConfig')]) {
config = readJSON file: 'JsonConfig'
}
}
pipeline {
agent any
stages {
stage("Setup") {
when {
beforeAgent true
anyOf { branch 'master', branch 'development' }
}
steps {
sh """
sed -i 's#__DB_USERNAME__#config.${getEnvName()}.DB_USERNAME#' ./secret-data.yml
cat ./secret-data.yml
"""
//Alternative
sh "sed -i 's#__DB_USERNAME__#${config.getEnvName().DB_USERNAME}#' ./secret-data.yml"
}
}
}
}
If I statically pass the var name like this, then it is working fine.
sh "sed -i 's#__DB_USERNAME__#${config.production.DB_USERNAME}#' ./k8s/secret-data.yml"
I want to make "production" dynamic so that it reads the value which is returned from getEnvName() method.
The problematic line is
sh """
sed -i 's#__DB_USERNAME__#config.${getEnvName()}.DB_USERNAME#' ./secret-data.yml
"""
This will evaluate as the shell command
sed -i 's#__DB_USERNAME__#config.production.DB_USERNAME#' ./secret-data.yml
But you want to be evaluated to
sed -i 's#__DB_USERNAME__#userABC#' ./secret-data.yml
Since the config is a Groovy object representing the parsed JSON file we can access its properties dynamically using the subscript operator ([]):
sh """
sed -i 's#__DB_USERNAME__#${config[getEnvName()].DB_USERNAME}#' ./secret-data.yml
"""
I'm in the process of passing a from the Jenkins Global Variable Reference variable called JOB_BASE_NAME to the groovy script. I'm using extendedChoice parameter with Groovy script and it is responsible for listing container images from the ECR on a specific repository. In my case Jenkins job names and ECR repository names are equivalent.
Ex:
Jenkins Job Name = http://jenkins.localhost/job/application-abc
ECR Repo name = abc/application-abc
I tried several things but all time I ended up with an empty response to the container images listing part.
Please help me to figure out is it outofthebox or how can i implement this thing
Thanks
Here is my Code
pipeline {
agent {
label 'centos7-slave'
}
stages {
stage('Re Tag RELEASE TAG AS UAT') {
environment {
BRANCH = "${params.GITHUB_BRANCH_TAG}"
}
input {
message 'Select tag'
ok 'Release!'
parameters {
extendedChoice(
bindings: '',
groovyClasspath: '',
multiSelectDelimiter: ',',
name: 'DOCKER_RELEASE_TAG',
quoteValue: false,
saveJSONParameterToFile: false,
type: 'PT_SINGLE_SELECT',
visibleItemCount: 5,
groovyScript: '''
import groovy.json.JsonSlurper
def AWS_ECR = ("/usr/local/bin/aws ecr list-images --repository-name abc/${JOB_BASE_NAME} --filter tagStatus=TAGGED --region ap-southeast-1").execute()
def DATA = new JsonSlurper().parseText(AWS_ECR.text)
def ECR_IMAGES = []
DATA.imageIds.each {
if(("$it.imageTag".length()>3))
{
ECR_IMAGES.push("$it.imageTag")
}
}
return ECR_IMAGES.grep( ~/.*beta.*/ ).sort().reverse()
'''
)
}
}
steps {
script {
def DOCKER_TAG = sh(returnStdout: true, script:"""
#!/bin/bashF
set -e
set -x
DOCKER_TAG_NUM=`echo $DOCKER_RELEASE_TAG | cut -d "-" -f1`
echo \$DOCKER_TAG_NUM
""")
DOCKER_TAG = DOCKER_TAG.trim()
DOCKER_TAG_NUM = DOCKER_TAG
}
sh "echo ${AWS_ECR} | docker login --username AWS --password-stdin ${ECR}"
sh "docker pull ${ECR}/${REPOSITORY}:${DOCKER_RELEASE_TAG}"
sh " docker tag ${ECR}/${REPOSITORY}:${DOCKER_RELEASE_TAG} ${ECR}/${REPOSITORY}:${DOCKER_TAG_NUM}-rc"
sh "docker push ${ECR}/${REPOSITORY}:${DOCKER_TAG_NUM}-rc"
}
}
}
}
You can leverage Groovy String Interpolation to replace the job base name in the script for the parameter, but the script can't access any variable out of the scope of the script.
You can try as following:
Use a function to compose the Groovy script for parameter
The function accept the JOB_BASE_NAME value
Use Groovy string interpolation to replace to real value.
pipeline {
agent {
label 'centos7-slave'
}
stages {
stage('Re Tag RELEASE TAG AS UAT') {
environment {
BRANCH = "${params.GITHUB_BRANCH_TAG}"
}
input {
message 'Select tag'
ok 'Release!'
parameters {
extendedChoice(
bindings: '',
groovyClasspath: '',
multiSelectDelimiter: ',',
name: 'DOCKER_RELEASE_TAG',
quoteValue: false,
saveJSONParameterToFile: false,
type: 'PT_SINGLE_SELECT',
visibleItemCount: 5,
groovyScript: list_ecr_images("${env.JOB_BASE_NAME}")
)
}
}
steps {
script {
def DOCKER_TAG = sh(returnStdout: true, script:"""
#!/bin/bashF
set -e
set -x
DOCKER_TAG_NUM=`echo $DOCKER_RELEASE_TAG | cut -d "-" -f1`
echo \$DOCKER_TAG_NUM
""")
DOCKER_TAG = DOCKER_TAG.trim()
DOCKER_TAG_NUM = DOCKER_TAG
}
sh "echo ${AWS_ECR} | docker login --username AWS --password-stdin ${ECR}"
sh "docker pull ${ECR}/${REPOSITORY}:${DOCKER_RELEASE_TAG}"
sh " docker tag ${ECR}/${REPOSITORY}:${DOCKER_RELEASE_TAG} ${ECR}/${REPOSITORY}:${DOCKER_TAG_NUM}-rc"
sh "docker push ${ECR}/${REPOSITORY}:${DOCKER_TAG_NUM}-rc"
}
}
}
}
def list_ecr_images(jobBaseName) {
def _script = """
import groovy.json.JsonSlurper
def AWS_ECR = [
'/usr/local/bin/aws',
'ecr list-images',
"--repository-name abc/${jobBaseName}",
'--filter tagStatus=TAGGED',
'--region ap-southeast-1'
].execute().text
def DATA = new JsonSlurper().parseText(AWS_ECR)
def ECR_IMAGES = []
DATA.imageIds.each {
if((it.imageTag.length()>3))
{
ECR_IMAGES.push(it.imageTag)
}
}
return ECR_IMAGES.grep( ~/.*beta.*/ ).sort().reverse()
"""
return _script.stripIndent()
}
I have defined global variable in Jenkins pipeline
def BUILDNRO = '0'
pipeline { ...
Then i manipulate variable with shell script to enable running builds parallel by using job build number as identifier so we don't mix different docker swarms.
stage('Handle BUILD_NUMBER') {
steps {
script {
BUILDNRO = sh( script: '''#!/bin/bash
Build=`echo ${BUILD_NUMBER} | grep -o '..$'`
# Check if BUILD first character is 0
if [[ $Build:0:1 == "0" ]]; then
# replace BUILD first character from 0 to 5
Build=`echo $Build | sed s/./5/1`
fi
echo $Build
''',returnStdout: true).trim()
}
}
}
i get value out from previos stage and trying to get global variable on next stage
stage('DOCKER: Init docker swarm') {
steps {
echo "BUILDNRO is: ${BUILDNRO}" --> Value is here.
sh '''#!/bin/bash
echo Buildnro is: ${BUILDNRO} --> This is empty.
...
}
}
This will out give global variable empty. why? in previous stage there was value in it.
EDIT 1.
Modified code blocks to reflect current status.
I managed to figure it out. Here is solution how i managed to did it.
BUILDNRO is groovy variable and if wanting to used in bash variable it have to pass using withEnv. BUILD_NUMBER in first stage is bash variable hence it can be used directly script in first stage.
def BUILDNRO = '0'
pipeline {
....
stages {
stage('Handle BUILD_NUMBER') {
steps {
script {
BUILDNRO = sh( script: '''#!/bin/bash
Build=`echo ${BUILD_NUMBER} | grep -o '..$'`
''',returnStdout: true).trim()
}
}
}
stage('DOCKER: Init docker swarm') {
steps {
dir("prose_env/prose_api_dev_env") {
withEnv(["MYNRO=${BUILDNRO}"]) {
sh(returnStdout: false, script: '''#!/bin/bash
echo Buildnro is: ${MYNRO}`
'''.stripIndent())
}
}
}
}
}
}
If you are using single quotes(```) in the shell module, Jenkins treats every variable as a bash variable. The solution is using double quotes(""") but then if you made bash variable you have to escape it. Below an example with working your use case and escaped bash variable
pipeline {
agent any
stages {
stage('Handle BUILD_NUMBER') {
steps {
script {
BUILDNRO = sh(script: 'pwd', returnStdout: true).trim()
echo "BUILDNRO is: ${BUILDNRO}"
}
}
}
stage('DOCKER: Init docker swarm') {
steps {
sh """#!/bin/bash
echo Buildnro is: ${BUILDNRO}
variable=world
echo "hello \${variable}"
sh """
}
}
}
}
output of the second stage:
Buildnro is: /var/lib/jenkins/workspace/stack1
hello world
I am using the extendedChoice plugin for my Jenkins pipeline. It fetches the s3 objects from the bucket and provides the list of values using a short Groovy script. The issue is that I need to parametrize the s3 bucket by using the corresponding variable defined within the pipeline's environment section. How can I do this?
So I tried a lot of different snippets to get the env vars though with no result.
import jenkins.model.*
// This will print out the requested var from the global Jenkins config.
def envVars = Jenkins.instance.getGlobalNodeProperties()[0].getEnvVars()
return envVars['S3_BUCKET']
// This will print out values from the env vars of the node itself where the Jenkins is running.
def env = System.getenv('S3_BUCKET')
return env
// This is what I have now
def domainsList = "aws s3api list-objects-v2 --bucket someRandomBucket --output text --delimiter /".execute() | 'cut -d / -f 1'.execute() | 'sed 1d'.execute()
domainsList.waitFor()
def output = domainsList.in.text
return output.split('COMMONPREFIXES')
// This is the Jenkinsfile
pipeline {
agent any
environment {
DOMAIN_NAME = "${params.DOMAIN_NAME}"
MODEL_VERSION = "${params.MODEL_VERSION}"
S3_BUCKET = "someRandomBucket"
}
parameters {
extendedChoice(
bindings: '',
defaultValue: '',
description: '',
descriptionPropertyValue: '',
groovyClasspath: '',
groovyScript: '''
def domainsList = "aws s3api list-objects-v2 --bucket someRandomBucket --output text --delimiter /".execute() | 'cut -d / -f 1'.execute() | 'sed 1d'.execute()
domainsList.waitFor()
def output = domainsList.in.text
return output.split('COMMONPREFIXES')
''',
multiSelectDelimiter: ',',
name: 'DOMAIN_NAME',
quoteValue: false,
saveJSONParameterToFile: false,
type: 'PT_SINGLE_SELECT',
visibleItemCount: 10)
choice(
choices: ['a', 'b'],
description: 'Select a model version for processing',
name: 'MODEL_VERSION')
}
stages {
stage('Clean workdir') {
steps {
cleanWs()
}
}
stage('build') {
steps {
sh "echo $S3_BUCKET"
sh "echo $DOMAIN_NAME"
sh "echo $MODEL_VERSION"
}
}
}
}
As I mentioned above I need to substitue the someRandomBucket hardcode with the S3_BUCKET env var value in the groovy script within the extendedChoice parameter
RESOLVED - Environment variables can be injected particarly for the parameter via the Jenkins job UI
I'm trying to configure a parameter in Jenkins pipeline and then execute it within bat command:
pipeline {
agent {
label 'master'
}
parameters {
string (
defaultValue: '"someExe.exe"',
description: '',
name : 'varExe'
)
}
stages {
stage("hi") {
steps {
script {
bat '${params.varExe}'
}
}
}
}
}
Unfortunately, i'm getting this error:
'${varExe}'is not recognized as an internal or external command
For some reason, Jenkins doesn't use varExe value.
I've also tried bat '${varExe}' but still no luck.
Any ideas ?
You need to use a double quote here to replace the variable.
bat "${params.varExe}"
You have to be careful with single and double quotes. For the following example, the first one would echo someExe.exe, while the second one would throw a Bad substitution error.
pipeline {
agent any
parameters {
string (
defaultValue: '"someExe.exe"',
description: '',
name : 'varExe')
}
stages {
stage ('Test') {
steps {
script {
sh "echo '${params.varExe}'"
sh 'echo "${params.varExe}"'
}
}
}
}
}
I think for bat command should be like below
bat ''' echo %varExe% '''
reference : pass parameter from jenkins parameterized build to windows batch command