Bad substitution when passing parameter to shell script in Jenkinsfile - jenkins

In a Jenkinsfile I'm attempting to set an environment variable by setting the stdOut of a shell script. The script contains an AWS command that returns an InstanceID:
stage('Set InstanceID') {
steps {
script {
env.IID = sh (script: 'scripts/get-node-id.sh "${params.ENVIRONMENT}" "${params.NODE}"', returnStdout: true).trim()
}
}
}
No matter what I do or how many backslashes I use to escape the quotes, nothing works. I get a bad substitution error. I've also tried without double quotes.
If I hardcode in the shell script arguments, it runs fine.
How do I get this working if I want to use the parameter values here?

Groovy (the language of the Jenkinsfile) and Bash share the same substitution syntax. As you're using single quotes in your example code, the Groovy substitution does not work (see https://groovy-lang.org/syntax.html#_single_quoted_string). So Bash will try to do the substitution, but does not know these variables as they are Jenkins parameter values.
So solve this you need to use double quotes for your script and escape the double quotes in it (or use singe quotes):
stage('Set InstanceID') {
steps {
script {
env.IID = sh (script: "scripts/get-node-id.sh \"${params.ENVIRONMENT}\" \"${params.NODE}\"", returnStdout: true).trim()
}
}
}

I managed to resolve this, the comment from #yong above was almost what I needed. I needed three lots of double quotes and to unquote the variables:
stage('Set InstanceID') {
steps {
script {
env.IID = sh (script: """scripts/get-node-id.sh ${params.ENVIRONMENT} ${params.NODE}""", returnStdout: true).trim()
}
}
}

Related

Jenkins pipeline returns "Bad Subtitution" for shell command

I'm attempting to run the following command in a shell block in my Jenkins pipeline:
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:${ACCOUNT_NUMBER}:task-definition/${TASK_NAME}:${NEW_REVISION}"'"'
This command works perfectly fine when I run it directly on the Jenkins node in shell.
When I insert it into the Pipeline like this:
stage('process json') {
steps {
dir('mydir') {
sh """
NEW_REVISION=\$(cat revision.txt)
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:\${env.AWS_ACCOUNT_NUMBER}:task-definition/\${env.TASK_NAME}:\${NEW_REVISION}"'"'
"""
}
}
}
I get a Bad substitution error without any more information. As far as I know, I'm escaping variables and quotation correctly. I can bypass the error if I remove the double quotes like this:
jq '.Resources[].TargetService.Properties.TaskDefinition = "arn:aws:ecs:us-east-1:${ACCOUNT_NUMBER}:task-definition/${TASK_NAME}:${NEW_REVISION}"'
But that ends up processing the variables literally.
Notes: I'm aware of the security issue by not passing jq --arg and prepared to modify my command after I can get the simpler format working. revision.txt contains a numeric value. The env.* variables are declared earlier as part of the pipeline environment.
env is a Jenkins Object and you seem to be escaping env.* variables as well. If you have already exported these variables as Environment variables they should be available to you in the shell environment. So simply drop the env part from the variables or remove the escape characters from such variables and let Jenkins interpolate them.
stage('process json') {
steps {
dir('mydir') {
sh """
NEW_REVISION=\$(cat revision.txt)
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:\${AWS_ACCOUNT_NUMBER}:task-definition/\${TASK_NAME}:\${NEW_REVISION}"'"'
"""
}
}
}

In a Jenkins script, can I bind credentials inside a conditional expression?

I am trying to run a script to dictate whether a Jenkins step runs. This script needs credentials, which we access by calling withCredentials.
Here's a slightly reduced example:
stage('Sample test stage') {
when {
expression {
withCredentials([usernamePassword(credentialsId: 'MY_CREDENTIALS_ID', usernameVariable: 'MY_USERNAME', passwordVariable: 'MY_TOKEN')]) {
return sh([returnStdout: true, script: 'node bin/my-script ${MY_TOKEN}']).trim().startsWith('success')
}
}
}
steps {
echo 'I should only run when the script passes'
}
}
However, this doesn't work. Jenkins returns the error: "syntax error: bad substitution."
I'm assuming it's down to the syntax used when trying to pass in the credentials. I've tried a couple of variations, such as returning withCredentials and wrapping the whole expression in withCredentials, but both of them had fairly dramatic errors.
I cannot find any documentation on this, which makes me wonder if it's illegal
I feel quite silly. There's a difference between " and ' in Groovy. Within single quotes, only Bash substitutions will work. Within double quotes, both Groovy and Bash substitutions will work.

Accessing parameters in Jenkins

How do I access this variable in a shell script.
I have tried
echo params.$STATES;
echo $STATES;
Output for the first one is params. . Output for the second one is an empty string. The output I am expecting is the string I am passing when I build that job with parameters.
If you are using the 'Execute Shell' block in a Freestyle Project, you will need a $ (dollar sign) before the variable. Since you already tried this, there could be an issue not wrapping variable within {}
try echo ${STATES}
More info on curly braces around variables,
See: codeforester answer on usage of curly braces around shell variables
If you are using a Jenkinsfile without a shell block (valid in Scripted and Declarative)
Use dollar sign with double quotes
node(){
echo "$STATES"
}
pipeline {
agent any
stages {
stage('Hello') {
steps {
echo "$STATES"
}
}
}
}
or without Double quotes and dollar sign
node(){
echo STATES
}
pipeline {
agent any
stages {
stage('Hello') {
steps {
echo STATES
}
}
}
}
If you are using a Shell block in the Jenkinsfile pipeline, use
sh "echo $STATES" or sh "echo ${STATES}"
(as in the Freestyle Execute Shell block, Double quotes are needed for interpolation)
See
String interpolation in Jenkinsfiles
Handling parameters in Jenkinsfiles

How to access groovy variable from pipeline into shell script?

I've a global variable in pipeline say BACKUP_DIR_NAME and in shell script which is inside pipeline, I want to build path using it hence have following code -
BACKUP_DIR_NAME="10-04-2020"
pipeline {
agent any
stages {
stage('First') {
steps {
script {
sh '''
BACKUP_DIR_PATH="/home/oracle/SeleniumFramework/SeleniumResultsBackup/"$BACKUP_DIR_NAME"/"
echo "Directory path is "$BACKUP_DIR_PATH
'''
}
}
}
}
}
When executed this, I can see value of BACKUP_DIR_NAME is evaluated as empty. Could you please help me to correct above code?
You mix two types of variables in your sh step. In the first line, you are trying to access the Groovy variable and interpolate its value to construct shell variable. In the second line, you expect to access this shell variable.
To satisfy the first part, you need to use double quotes to construct a Groovy string that supports variables interpolation. To satisfy the second part, you need to escape \$ to prevent $BACKUP_DIR_PATH from being interpolated.
BACKUP_DIR_NAME="10-04-2020"
pipeline {
agent any
stages {
stage('First') {
steps {
script {
sh """
BACKUP_DIR_PATH="/home/oracle/SeleniumFramework/SeleniumResultsBackup/"$BACKUP_DIR_NAME"/"
echo "Directory path is "\$BACKUP_DIR_PATH
"""
}
}
}
}
}

How to use environment variable inside Jenkinsfile

I am having similar issue as mentioned here
I am trying to deploy an application via Jenkinsfile. For which I have to run this command on the deploy stage in Jenkins (if I hardcode the value then it works fine):
xldDeploy serverCredentials: 'usernam', environmentId: 'Environments/SysTest1/SysTest1_1', packageId: 'Applications/Testapp/testapp_1.0.4.5.Build39_TAG-test'
"testapp_1.0.4.5.Build39_TAG-test" is getting generated at running time. Which can be created by concating "${TagVersion}.Build${env.BUILD_NUMBER}_${ComponentTagName}"
I tried below code in my Jenkins pipeline:
stage('Deploy') {
node('noibuild01') {
if ("${env.Build_WildflyCPECommon}" == 'true') {
echo "${TagVersion}"
echo "${ComponentTagName}"
echo "${env.BUILD_NUMBER}"
script {
env.buildNumber = "${TagVersion}.Build${env.BUILD_NUMBER}_${ComponentTagName}"
env.packageid = "'Applications/Testapp/${env.buildNumber}'"
}
echo "${env.buildNumber}"
echo "${env.packageid}"
xldDeploy serverCredentials: 'nex8voo', environmentId: 'Environments/SysTest1/SysTest1_1', packageId: "${env.packageid}"
}
}
}
I checked the output, it is showing correctly:
echo "${env.buildNumber}" giving
testapp_1.0.4.5.Build39_TAG-test
echo "${env.packageid}" giving
'Applications/Testapp/testapp_1.0.4.5.Build39_TAG-test'
But xldDeploy serverCredentials: 'username', environmentId: 'Environments/SysTest1/SysTest1_1', packageId: "${env.packageid}"
is taking as:
[/repository/ci/'Applications/Testapp/testapp_1.0.4.5.Build39_TAG-test']
Repository entity: ['Applications/Testapp/testapp_1.0.4.5.Build39_TAG-test'] not found
I think I can't use packageId: "${env.packageid}".
Is there anything I could try? Maybe Groovy or Python code?
Your packageid environment variable is not being assigned a concatenated string correctly. You have literal quotes inside the string interpolation quotes. You should change it to:
env.packageid = "Applications/Testapp/${env.buildNumber}"
to only interpolate the string, which is the functionality you want here.
Additionally, you do not need to interpolate the environment variable inside an empty string for your method parameter, so your method invocation can be cleaned up as:
xldDeploy serverCredentials: 'nex8voo', environmentId: 'Environments/SysTest1/SysTest1_1', packageId: env.packageid

Resources