I have a Jenkinsfile which uses three tick marks surrounding a command to execute as in:
sh ''' command '''
We have no idea why three tick marks are required or what role they perform.
This syntax is seen in the Jenkinsfile doc set.
This has nothing at all to do with bash (in which triple-quotes have no special meaning at all), and everything to do with Groovy (the separate, non-bash interpreter that parses Jenkinsfiles).
In Groovy, but not in bash, strings must use triple-quotes to span multiple lines.
In the context of a sh directive in a Jenkinsfile, the content of your triple-quoted string is passed to a shell as a script to execute; however, the syntax is parsed by Groovy, so it's only Groovy that cares about the quotes themselves (as opposed to the quoted content).
Can you give more idea about what kind of command is it, is it a unix command or some script ?
The single quote and its variation like '''(3 ticks) as mentioned in question skip the variable expansion, and it could used to show what is being executed.
echo '''Updating JAVA_HOME variable :
export $JAVA_HOME="$NEW_JAVA_HOME" '''
However in your question, a command (some string) is enclosed between 3 ticks marks and sh tries to execute this command or script. One such example below
$ echo "echo hello" > /tmp/tesh.sh
$ sh '''/tmp/test.sh'''
hello
Related
I'm attempting to run the following command in a shell block in my Jenkins pipeline:
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:${ACCOUNT_NUMBER}:task-definition/${TASK_NAME}:${NEW_REVISION}"'"'
This command works perfectly fine when I run it directly on the Jenkins node in shell.
When I insert it into the Pipeline like this:
stage('process json') {
steps {
dir('mydir') {
sh """
NEW_REVISION=\$(cat revision.txt)
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:\${env.AWS_ACCOUNT_NUMBER}:task-definition/\${env.TASK_NAME}:\${NEW_REVISION}"'"'
"""
}
}
}
I get a Bad substitution error without any more information. As far as I know, I'm escaping variables and quotation correctly. I can bypass the error if I remove the double quotes like this:
jq '.Resources[].TargetService.Properties.TaskDefinition = "arn:aws:ecs:us-east-1:${ACCOUNT_NUMBER}:task-definition/${TASK_NAME}:${NEW_REVISION}"'
But that ends up processing the variables literally.
Notes: I'm aware of the security issue by not passing jq --arg and prepared to modify my command after I can get the simpler format working. revision.txt contains a numeric value. The env.* variables are declared earlier as part of the pipeline environment.
env is a Jenkins Object and you seem to be escaping env.* variables as well. If you have already exported these variables as Environment variables they should be available to you in the shell environment. So simply drop the env part from the variables or remove the escape characters from such variables and let Jenkins interpolate them.
stage('process json') {
steps {
dir('mydir') {
sh """
NEW_REVISION=\$(cat revision.txt)
jq '.Resources[].TargetService.Properties.TaskDefinition = "'"arn:aws:ecs:us-east-1:\${AWS_ACCOUNT_NUMBER}:task-definition/\${TASK_NAME}:\${NEW_REVISION}"'"'
"""
}
}
}
I have a legacy project in Jenkins that hast to be pipelined (for
later parallelization), hence moving from simple tcsh script to
pipeline
running the script as
#!/bin/tcsh
source ./mysetting.sh
update
works but the same pipeline step fails due to missing alias expansion
stage ('update') {
steps {
//should be working but alias expansion fails
sh 'tcsh -c "source ./mysettings.sh; alias; update"'
//manually expanding the alias works fine
sh 'tcsh -c "source ./mysettings.sh; alias; python update.py;"'
}
}
calling alias in the steps properly lists all the set aliases, so I
can see them, but not use them.
I know in bash alias expansion has to be set
#enable shell option for alias_expansion
shopt -s expand_aliases
but in csh/tcsh that should be taken care of by source.
what am I missing?
found the solution:
sh '#!/bin/tcsh \n' +
'source ./mysettings.sh \n' +
'echo "Calling my alias" \n' +
'my_alias \n'
every line starting with sh launches a new shell, so it has to be in one line including line breaks.
further adding to the confusing was that documentation of jenkins says that it starts "a bash" but it launched /bin/sh which in my case pointed to something else
I am unable to properly pass a variable with spaces in Jenkinsfile to shell command
I have tried using quotes (double and single) backspaces and various other combination. Jenkins will treat the string like as 2 item and single quoted them.
The variable in question in the Jenkins file.
MYTIME = 2022-01-02 03:04:05
In Jenkinsfile
stage('list') {
steps{
sh script:'ansible-playbook -i ./hosts.ini ./ping_playbook.yml -e time=\\"$TIME\\" --limit ${ENV}', label: "ping test"
}
}
Jenkins will run it as
ansible-playbook -i ./lzhxjp-test-update/lzhxjp/hosts.ini .ping_playbook.yml -e 'time="2022-01-02' '11:22:33"' --limit sdktest
How do I put it so that Jenkins interpret it as -e time="2022-01-02 11:22:33"
Inside single-quoted strings no variables are expanded by Jenkins (Groovy to be exact).
Single-quoted strings are plain java.lang.String and don’t support interpolation.
https://groovy-lang.org/syntax.html#all-strings
Thus it is passed directly into the shell and therefore already subject to shell quoting, you can remove the \\ before the " and then the shell the will expand "$TIME" correctly.
I am using Choice param in my jenkins file to select environment as follows:
pipeline {
agent any
parameters {
choice(
name: 'ENVIRONMENT_URL',
choices: "https://beta1.xyz.com\nhttps://beta2.xyz.com\nhttps://beta3.xyz.com",
description: 'interesting stuff' )
}
in the Stage section, i have the following piece
stage('execute tests') {
steps {
script {
sh """URL=${ENVIRONMENT_URL} npm run e2e:tests"""
sh 'echo -e "\nTest Run Completed.\n"'
}
}
}
However, when i run the pipeline job by selecting the choice parameter i added, the following gets executed (the inserted choice param creates a line break):
+ URL=https://beta1.xyz.com
+ npm run e2e:tests
Using the variable is causing a line break and that's what is causing issue.
I have tried different ways to avoid line break. Tried using a variable but that didn't help. tried with different quotations, that didn't either.
What can i do to avoid line break ?
You can use the trim method on a String class type to remove trailing whitespace and newline:
sh "URL=${params.ENVIRONMENT_URL.trim()} npm run e2e:tests"
Note I also specified your parameter is in the params map and removed the triple quotes as those are for multiline string formatting.
Alternatively, you can specify the choices as an array instead of as a multiline string. The choices argument would then appear like:
choice(
name: 'ENVIRONMENT_URL',
choices: ['https://beta1.xyz.com', 'https://beta2.xyz.com', 'https://beta3.xyz.com'],
description: 'interesting stuff'
)
Either solution would fix your issue.
I am having these lines in my Jenkinsfile:
parameters {
string(name: 'DATABASE', defaultValue: 'jenkinsdatabase',
description: 'The name of the database')
}
(...)
Now I want to use the value of ${params.DATABASE} in a step of a stage e.g.
sh 'mysql --user ${USER} -p${PASSWORD} --host ${HOST} -e "DROP DATABASE IF EXISTS ${params.DATABASE};CREATE DATABASE ${params.DATABASE} DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; commit;";export GRADLE_OPTS="-Xms1536m -Xmx1536m"'
But this ends with a exeception: Bad substitution
Can anybody help me?
Your sh script body uses single quotes around it.
sh '...'
Since these are single quotes, string interpolation will not occur. This means that all of your $ strings will use the shell variable substitution instead of the Groovy substitution. You are getting a Bad substitution in the shell because the shell won't be able to substitute ${params.DATABASE} because that is an invalid shell substitution.
params is a Groovy variable, so switching your single quotes to double quotes will perform string interpolation of the variables.
sh "..."