I start building a Jenkins job with a Jenkinsfile. In one of my steps, I create a variable. This variable echo's as expected, now I try to insert this variable into a new function and need to escape, everything looked fine until I see the last double quote on a new line and not on the end.
The line of code that create my variable is:
gitTag = sh(returnStdout: true, script: "git describe --tags--always")
The line code that use this variable:
sh "echo WEBSERVICE_VERSION=\"$gitTag\" > ${WORKSPACE}/webservice/src/webservice_version.py"
expected output WEBSERVICE_VERSION="$gitTag", but the output I see is WEBSERVICE_VERSION=$gitTag.
Did I make any mistake in escaping?
This should work for you
node('master'){
def gitTag="mytag"
sh "touch webservice_version.py"
sh "echo \\\"${gitTag}\\\" > webservice_version.py"
sh "cat webservice_version.py"
}
Related
In a Jenkinsfile I'm attempting to set an environment variable by setting the stdOut of a shell script. The script contains an AWS command that returns an InstanceID:
stage('Set InstanceID') {
steps {
script {
env.IID = sh (script: 'scripts/get-node-id.sh "${params.ENVIRONMENT}" "${params.NODE}"', returnStdout: true).trim()
}
}
}
No matter what I do or how many backslashes I use to escape the quotes, nothing works. I get a bad substitution error. I've also tried without double quotes.
If I hardcode in the shell script arguments, it runs fine.
How do I get this working if I want to use the parameter values here?
Groovy (the language of the Jenkinsfile) and Bash share the same substitution syntax. As you're using single quotes in your example code, the Groovy substitution does not work (see https://groovy-lang.org/syntax.html#_single_quoted_string). So Bash will try to do the substitution, but does not know these variables as they are Jenkins parameter values.
So solve this you need to use double quotes for your script and escape the double quotes in it (or use singe quotes):
stage('Set InstanceID') {
steps {
script {
env.IID = sh (script: "scripts/get-node-id.sh \"${params.ENVIRONMENT}\" \"${params.NODE}\"", returnStdout: true).trim()
}
}
}
I managed to resolve this, the comment from #yong above was almost what I needed. I needed three lots of double quotes and to unquote the variables:
stage('Set InstanceID') {
steps {
script {
env.IID = sh (script: """scripts/get-node-id.sh ${params.ENVIRONMENT} ${params.NODE}""", returnStdout: true).trim()
}
}
}
I am stuck in trying to get a Jenkinsfile to work. It keeps failing on sh step and gives the following error
process apparently never started in /home/jenkins/workspace
...
(running Jenkins temporarily with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true might make the problem clearer)
I have tried adding
withEnv(['PATH+EXTRA=/usr/sbin:/usr/bin:/sbin:/bin'])
before sh step in groovy file
also tried to add
/bin/sh
in Manage Jenkins -> Configure System in the shell section
I have also tried replacing the sh line in Jenkinsfile with the following:
sh "docker ps;"
sh "echo 'hello';"
sh ./build.sh;"
sh ```
#!/bin/sh
echo hello
```
This is the part of Jenkinsfile which i am stuck on
node {
stage('Build') {
echo 'this works'
sh 'echo "this does not work"'
}
}
expected output is "this does not work" but it just hangs and returns the error above.
what am I missing?
It turns out that the default workingDir value for default jnlp k8s slave nodes is now set to /home/jenkins/agent and I was using the old value /home/jenkins
here is the config that worked for me
containerTemplate(name: 'jnlp', image: 'lachlanevenson/jnlp-slave:3.10-1-alpine', args: '${computer.jnlpmac} ${computer.name}', workingDir: '/home/jenkins/agent')
It is possible to get the same trouble with the malformed PATH environment variable. This prevents the sh() method of the Pipeline plugin to call the shell executable. You can reproduce it on a simple pipeline like this:
node('myNode') {
stage('Test') {
withEnv(['PATH=/something_invalid']) {
/* it hangs and fails later with "process apparently never started" */
sh('echo Hello!')
}
}
}
There is variety of ways to mangle PATH. For example you use withEnv(getEnv()) { sh(...) } where getEnv() is your own method which evaluates the list of environment variables depending on the OS and other conditions. If you make a mistake in the getEnv() method and PATH gets overwritten you get it reproduced.
I have a Jenkinsfile which uses three tick marks surrounding a command to execute as in:
sh ''' command '''
We have no idea why three tick marks are required or what role they perform.
This syntax is seen in the Jenkinsfile doc set.
This has nothing at all to do with bash (in which triple-quotes have no special meaning at all), and everything to do with Groovy (the separate, non-bash interpreter that parses Jenkinsfiles).
In Groovy, but not in bash, strings must use triple-quotes to span multiple lines.
In the context of a sh directive in a Jenkinsfile, the content of your triple-quoted string is passed to a shell as a script to execute; however, the syntax is parsed by Groovy, so it's only Groovy that cares about the quotes themselves (as opposed to the quoted content).
Can you give more idea about what kind of command is it, is it a unix command or some script ?
The single quote and its variation like '''(3 ticks) as mentioned in question skip the variable expansion, and it could used to show what is being executed.
echo '''Updating JAVA_HOME variable :
export $JAVA_HOME="$NEW_JAVA_HOME" '''
However in your question, a command (some string) is enclosed between 3 ticks marks and sh tries to execute this command or script. One such example below
$ echo "echo hello" > /tmp/tesh.sh
$ sh '''/tmp/test.sh'''
hello
I am trying to replace the '/' from Git branch name with '_' in my jenkinsfile so that I can tag my docker image with the branch name. In bash the below command works fine
echo "${git_branch_name//\//_}"
But when use the above command in jenkinsfile as below it throws an error.
#!/usr/bin/env groovy
def commit_id
def imagetag
def branch_name
def git_branch_name
node('Nodename') {
stage('checkout') {
checkout (scm).$Branch_Param
sh "git rev-parse --short HEAD > .git/commit-id"
commit_id = readFile('.git/commit-id').trim()
sh "git rev-parse --abbrev-ref HEAD > .git/branch-name"
git_branch_name = readFile('.git/branch-name').trim()
branch_name= sh "echo ${git_branch_name//\//_}"
sh "echo ${commit_id}"
sh "echo ${branch_name}"
sh "echo Current branch is ${branch_name}"
}
}
WorkflowScript: 15: end of line reached within a simple string 'x' or "x" or /x/;
solution: for multi-line literals, use triple quotes '''x''' or """x""" or /x/ or $/x/$ # line 15, column 28.
sh "branch_name = echo ${git_branch_name//\//_}"
What am I doing wrong here? Should I use Groovy regular expression instead of shell? why is shell not being interpreted correctly?
Thank you
The issue is that you're asking Groovy itself to interpret the expression ${git_branch_name//\//_}, not the shell. Using double-quotes around the string you pass to the sh step is what causes that. So if you instead write the following, this first error will go away:
sh 'echo ${git_branch_name//\\//_}' // <- Note the single-quotes
Basically, always use single-quotes unless you specifically need to use groovy's string interpolation (see the very last echo at the bottom of this answer).
Interestingly, it seems when I tested I didn't need the shebang (#!/bin/bash) to specify bash as some comments suggest; this ${variable//x/y} replace syntax worked in an sh step as-is. I guess the shell spawned was bash. I don't know if that's always the case, or if our Jenkins box has been specifically setup that way.
Also note you need to escape the escape sequence ('\\/') because what you're passing to the sh step is a string literal in groovy code. If you don't add that extra backslash, the line passed to the shell to be interpreted by it will be echo ${git_branch_name////_}, which it won't understand.
But there are other issues as well. First, assigning the output of the sh step to branch_name as you do means branch_name will always equal null. To get the stdout from a line of shell code you need to pass the extra parameter returnStdout: true to sh:
branch_name = sh (
script: 'echo ${git_branch_name//\\//_}',
returnStdout: true
).trim () // You basically always need to use trim, because the
// stdout will have a newline at the end
For bonus points, we could wrap that sh call in a closure. I find myself using it often enough to make this a good idea.
// Get it? `sh` out, "shout!"
def shout = { cmd -> sh (script: cmd, returnStdout: true).trim () }
//...
branch_name = shout 'echo ${git_branch_name//\\//_}'
But finally, the major problem is that bash (or whatever shell is actually spawned) doesn't have access to groovy variables. As far as it knows, echo ${git_branch_name} outputs an empty string, and therefore so does echo ${git_branch_name//\//_}.
You have a couple of choices. You could skip the creation of .git/branch-name and just immediately output the string-replaced result of git rev-parse:
branch_name = shout 'name=$(git rev-parse --abbrev-ref HEAD) && echo ${name//\\//_}'
Or to simplify that further you could use groovy's string replace function rather than the bash syntax:
branch_name = shout ('git rev-parse --abbrev-ref HEAD').replace ('/', '_')
Personally, I find the latter quite a bit more readable. YMMV. So bringing it all together at last:
#!groovy
def shout = { cmd -> sh (script: cmd, returnStdout: true).trim () }
// Note that I'm not declaring any variables up here. They're not needed.
// But you can if you want, just to clearly declare the environment for
// future maintainers.
node ('Nodename') {
stage ('checkout') {
checkout (scm).$Branch_Param
commit_id = shout 'git rev-parse --short HEAD'
branch_name = shout ('git rev-parse --abbrev-ref HEAD').replace ('/', '_')
echo commit_id
echo branch_name
echo "The most recent commit on branch ${branch_name} was ${commit_id}"
}
}
Given the following pipeline:
stages {
stage ("Checkout SCM") {
steps {
checkout scm
sh "echo ${CHANGE_AUTHOR_EMAIL}"
sh "echo ${CHANGE_ID}"
}
}
}
Why do these variables fail to resolve and provide a value?
Eventually I want to use these environment variables to send an email and merge a pull request:
post {
failure {
emailext (
attachLog: true,
subject: '[Jenkins] $PROJECT_NAME :: Build #$BUILD_NUMBER :: build failure',
to: '$CHANGE_AUTHOR_EMAIL',
replyTo: 'iadar#...',
body: '''<p>You are receiving this email because your pull request was involved in a failed build. Check the attached log file, or the console output at: $BUILD_URL to view the build results.</p>'''
)
}
}
and
sh "curl -X PUT -d '{\'commit_title\': \'Merge pull request\'}' <git url>/pulls/${CHANGE_ID}/merge?access_token=<token>"
Oddly enough, $PROJECT_NAME, $BUILD_NUMBER, $BUILD_URL do work...
Update: this may be an open bug... https://issues.jenkins-ci.org/browse/JENKINS-40486 :-(
Is there any workaround to get these values?
You need to be careful about how you refer to environment variables depending on whether it is shell or Groovy code, and how you are quoting.
When you do sh "echo ${CHANGE_ID}", what actually happens is that Groovy will interpolate the string first, by replacing ${CHANGE_ID} with the Groovy property CHANGE_ID, and that's where your error message is from. In Groovy, the environment variables are wrapped in env.
If you want to refer to the environment variables directly from your shell script, you either have to interpolate with env, use single quotes, or escape the dollar sign. All of the following should work:
sh 'echo $CHANGE_ID'
sh "echo \$CHANGE_ID"
sh "echo ${env.CHANGE_ID}"
For anyone who may come across this, these variables are available only if the checkbox for Build origin PRs (merged with base branch) was checked (this is in a multi-branch job).
See more in this other Jenkins issue: https://issues.jenkins-ci.org/browse/JENKINS-39838