Why do I get the following error:
++ cat package.json
++ jq .version
+ TAG = '"0.0.5"' /****/script.sh:
line 12: TAG: command not found
When running the below in a Jenkinsfile:
sh '''
...
...
TAG = \$(cat package.json | jq '.version')
git tag -a v${TAG} -m "New release: ${TAG}"
'''
As was mentioned in the comments, the problem was with the spaces before and after =. Additionally, I added escaping in the second line so it'll display as v0.0.6 and not v"0.0.6".
sh '''
...
...
TAG=\$(jq -r '.version' package.json)
git tag -a "v${TAG}" -m "New release: ${TAG}"
'''
Related
I'm writing a jenkins pipeline jenkinsfile and within the script clause I have to ssh to a box and run some commands. I think the problem has to do with the env vars that I'm using within the quotes. I'm getting a ENDSSH command not found error and I'm at a loss. Any help would be much appreciated.
stage("Checkout my-git-repo"){
steps {
script {
sh """
ssh -o StrictHostKeyChecking=accept-new -o LogLevel=ERROR -o UserKnownHostsFile=/dev/null -i ${JENKINS_KEY} ${JENKINS_KEY_USR}#${env.hostname} << ENDSSH
echo 'Removing current /opt/my-git-repo directory'
sudo rm -rf /opt/my-git-repo
echo 'Cloning new my-git-repo repo into /opt'
git clone ssh://${JENKINS_USR}#git.gitbox.com:30303/my-git-repo
sudo mv /home/jenkins/my-git-repo /opt
ENDSSH
"""
}
}
}
-bash: line 6: ENDSSH: command not found
I'm personally not familiar with jenkins, but I'd guess the issue is the whitespace before ENDSSH
White space in front of the delimiter is not allowed.
(https://linuxize.com/post/bash-heredoc/)
Try either removing the indentation:
stage("Checkout my-git-repo"){
steps {
script {
sh """
ssh -o StrictHostKeyChecking=accept-new -o LogLevel=ERROR -o UserKnownHostsFile=/dev/null -i ${JENKINS_KEY} ${JENKINS_KEY_USR}#${env.hostname} << ENDSSH
echo 'Removing current /opt/my-git-repo directory'
sudo rm -rf /opt/my-git-repo
echo 'Cloning new my-git-repo repo into /opt'
git clone ssh://${JENKINS_USR}#git.gitbox.com:30303/my-git-repo
sudo mv /home/jenkins/my-git-repo /opt
ENDSSH
"""
}
}
}
OR ensure that the whitespace is only tabs and replace << with <<-:
Appending a minus sign to the redirection operator <<-, will cause all
leading tab characters to be ignored. This allows you to use
indentation when writing here-documents in shell scripts. Leading
whitespace characters are not allowed, only tab.
I have a file 'README.txt' which contains the line -
"version": "1.0.0-alpha-test.7"
Using a Jenkins Pipeline, I want to replace this line with
"version": "1.0.0-alpha-test.{BUILD_NUMBER}"
The following sed command works when I try it on a linux cluster
sed -i -E "s#(\"version\"[ ]*:[ ]*\".+alpha-test\.)[0-9]+\"#\1${BUILD_NUMBER}#g" README.txt
The same command does not work using a Jenkins Pipeline.
Tried with the following query but it doesn't work -
sh """
sed -i -E "s|([\"]version[\"][ ]*:[ ]*[\"].+alpha-test\\.)[0-9]+\"|\1${BUILD_NUMBER}|g" README.txt
cat README.txt
"""
/home/jenkins/workspace/test/test-pipeline#tmp/durable-eb774fcf/script.sh:
3:
/home/jenkins/workspace/test/test-pipeline#tmp/durable-eb774fcf/script.sh:
Syntax error: ")" unexpected
Best to use perl command instead...
script{
old_version = (sh (returnStdout: true, script:'''old_version=`cat version.cfg |grep VERSION=|cut -d "=" -f2`
echo $old_version''')).toString().trim()
sh """
if [ "$old_version" != $new_version ]; then
perl -pi -e "s,$old_version,$new_version,g" version.cfg
##-- git push operation --##
fi
"""
}
I am having problems figuring out how to pass some variables into the parallel runs in the Jenkins groovy script below:
#!/usr/bin/env groovy
def call(version, project) {
sh '''#!/bin/bash
[[ ! -e ${WORKSPACE}/target/rpm/${project}/RPMS/ ]] && mkdir -p ${WORKSPACE}/target/rpm/${project}/RPMS/
(( $(ls ${WORKSPACE}/target/rpm/${project}/RPMS/*.rpm | wc -l) != 0 )) && rm ${WORKSPACE}/target/rpm/${project}/RPMS/*.rpm
cd ${WORKSPACE}/scripts/fpm_requirements && bundle install && bundle show fpm
'''
parallel (
"package foo": {
sh '''#!/bin/bash
export PATH=$PATH:~/bin:~/.gem/ruby/gems
cd ${WORKSPACE}/scripts/fpm_requirements
echo Project is ${project}
echo Version is ${version}
echo Iteration is $(echo ${version} | cut -d . -f 3)
'''
},
"package bar": {
sh '''#!/bin/bash
export PATH=$PATH:~/bin:~/.gem/ruby/gems
cd ${WORKSPACE}/scripts/fpm_requirements
echo Project is ${project}
echo Version is ${version}
echo Iteration is $(echo ${version} | cut -d . -f 3)
'''
}
)
}
So the version and project variables are populated in the first shell that is called but when they hit the two parallel runs they are not being pulled in.
I have tried a few different options to pass them in but none have worked.
Does anyone have any relevant ideas that might help?
You should change the ''' to """. In Groovy, string inside single/triple quote won't trigger string interpolation, but string inside single/triple double quote will do that.
So the ${version} and ${project} in your Shell script will be treated as variable from Shell context, but actually they are exist in Groovy context.
More about Groovy String at here, Below option 2 more suitable for your issue.
Option 1) using "" or """
"package foo": {
sh """#!/bin/bash
export PATH=\$PATH:~/bin:~/.gem/ruby/gems
cd \${WORKSPACE}/scripts/fpm_requirements
echo Project is ${project}
echo Version is ${version}
echo Iteration is \$(echo ${version} | cut -d . -f 3)
"""
},
"package bar": {
sh """#!/bin/bash
export PATH=\$PATH:~/bin:~/.gem/ruby/gems
cd \${WORKSPACE}/scripts/fpm_requirements
echo Project is ${project}
echo Version is ${version}
echo Iteration is \$(echo ${version} | cut -d . -f 3)
"""
}
Attention: need to escape the $ ahead of ${WORKSPACE} and $(echo ..), because we hope $ be kept after interpolation.
Option 2) using ' or ''' and inject version and project into Environment Variables of Shell context.
def call(version, project) {
env.version=version
env.project=project
// Groovy env api used to inject groovy value into environment variable
// so that you can refer groovy value later in shell script
// still use ''' in following code, no need to change
...
Script in package.json:
"scripts": {
"version": "echo $npm_package_version"
},
One of the stage in Jenkins pipeline:
stage('Build'){
sh 'npm install'
def packageVersion = sh 'npm run version'
echo $packageVersion
sh 'VERSION=${packageVersion} npm run build'
}
I got version from npm script output, but following line
echo $packageVersion
return null
Is packageVersion value not correctly assigned?
Edited:
when using Pipeline Utility Steps with following code
stage('Build'){
def packageJSON = readJSON file: 'package.json'
def packageJSONVersion = packageJSON.version
echo packageJSONVersion
sh 'VERSION=${packageJSONVersion}_${BUILD_NUMBER}_${BRANCH_NAME} npm run build'
}
I get
[Pipeline] echo
1.1.0
[Pipeline] sh
[...] Running shell script + VERSION=_16_SOME_BRANCH npm run build
So I am able to extract version, but still cannot pass it when running script
After your edit using readJSON, you now get string interpolation wrong. Variables within single quotes are not replaced in Groovy, only within double quotes.
sh 'VERSION=${packageJSONVersion}_${BUILD_NUMBER}_${BRANCH_NAME} npm run build'
must be
sh "VERSION=${packageJSONVersion}_${BUILD_NUMBER}_${BRANCH_NAME} npm run build"
The sh step by default returns nothing, so packageVersion should be null.
To return the output of the executed command, use it like this:
sh(script: 'npm run version', returnStdout: true)
This variant of sh returns the output instead of printing it.
Actually, I am wondering, why echo $packageVersion doesn't fail with an error, as this variable is not defined, but should be echo packageVersion.
For my it this:
sh(script: "grep \"version\" package.json | cut -d '\"' -f4 | tr -d '[[:space:]]'", returnStdout: true)
This worked for me:
Full version number:
PACKAGE_VERSION = sh returnStdout: true, script: '''grep 'version' package.json | cut -d '"' -f4 | tr '\n' '\0''''
echo "Current package version: $PACKAGE_VERSION"
$ > 1.2.3
Major version only:
PACKAGE_VERSION = sh returnStdout: true, script: '''grep 'version' package.json | cut -d '"' -f4 | cut -d '.' -f1 | tr '\n' '\0''''
echo "Current package Major version: $PACKAGE_VERSION"
$ > 1
stage('Read JSON') {
steps {
script {
def packageJson = readJSON file: 'package.json'
def packageVersion = packageJSON.version
echo "${packageJSONVersion}"
}
}
}
Below snippet worked for me: Credits to #Ferenc Takacs
version = sh(returnStdout: true, script: "grep 'version' package.json | cut -d '"' -f4 | tr '\n' '\0'")
This command will take exact property version: ... in package.json and can work on both Mac and Linux. The other solutions using grep will not give you correct answer in case of you have > 1 version keyword in your package.json (it'll return all of them instead of just the one you want)
awk -F'"' '/"version": ".+"/{ print $4; exit; }' package.json
To access an npm environment variable outside the scope of a run-script, parse the variable with bash:
$ npm run env | grep npm_package_version | cut -d '=' -f 2
Author: https://remarkablemark.org/blog/2018/08/14/package-json-version/
I am trying to create a Jenkins pipeline where I need to execute multiple shell commands and use the result of one command in the next command or so. I found that wrapping the commands in a pair of three single quotes ''' can accomplish the same. However, I am facing issues while using pipe to feed output of one command to another command. For example
stage('Test') {
sh '''
echo "Executing Tests"
URL=`curl -s "http://localhost:4040/api/tunnels/command_line" | jq -r '.public_url'`
echo $URL
RESULT=`curl -sPOST "https://api.ghostinspector.com/v1/suites/[redacted]/execute/?apiKey=[redacted]&startUrl=$URL" | jq -r '.code'`
echo $RESULT
'''
}
Commands with pipe are not working properly. Here is the jenkins console output:
+ echo Executing Tests
Executing Tests
+ curl -s http://localhost:4040/api/tunnels/command_line
+ jq -r .public_url
+ URL=null
+ echo null
null
+ curl -sPOST https://api.ghostinspector.com/v1/suites/[redacted]/execute/?apiKey=[redacted]&startUrl=null
I tried entering all these commands in the jenkins snippet generator for pipeline and it gave the following output:
sh ''' echo "Executing Tests"
URL=`curl -s "http://localhost:4040/api/tunnels/command_line" | jq -r \'.public_url\'`
echo $URL
RESULT=`curl -sPOST "https://api.ghostinspector.com/v1/suites/[redacted]/execute/?apiKey=[redacted]&startUrl=$URL" | jq -r \'.code\'`
echo $RESULT
'''
Notice the escaped single quotes in the commands jq -r \'.public_url\' and jq -r \'.code\'. Using the code this way solved the problem
UPDATE: : After a while even that started to give problems. There were certain commands executing prior to these commands. One of them was grunt serve and the other was ./ngrok http 9000. I added some delay after each of these commands and it solved the problem for now.
The following scenario shows a real example that may need to use multiline shell commands. Which is, say you are using a plugin like Publish Over SSH and you need to execute a set of commands in the destination host in a single SSH session:
stage ('Prepare destination host') {
sh '''
ssh -t -t user#host 'bash -s << 'ENDSSH'
if [[ -d "/path/to/some/directory/" ]];
then
rm -f /path/to/some/directory/*.jar
else
sudo mkdir -p /path/to/some/directory/
sudo chmod -R 755 /path/to/some/directory/
sudo chown -R user:user /path/to/some/directory/
fi
ENDSSH'
'''
}
Special Notes:
The last ENDSSH' should not have any characters before it. So it
should be at the starting position of a new line.
use ssh -t -t if you have sudo within the remote shell command
I split the commands with &&
node {
FOO = world
stage('Preparation') { // for display purposes
sh "ls -a && pwd && echo ${FOO}"
}
}
The example outputs:
- ls -a (the files in your workspace
- pwd (location workspace)
- echo world