Here is my groovy file
timestamps{
node('cf_slave'){
checkout scm
stage('Read the file') {
def PWD = pwd()
withEnv(["prj_option=${params.project}"]) {
def response =sh(returnStdout: true, script: 'sh \'jenkins/security/get_values.sh\'')
}
}
This is my get_values.sh file
echo "The project option is:" $prj_option
prj_name=$(echo "$prj_option" | tr '[:upper:]' '[:lower:]')
file_name="va_input_file_$prj_name.txt"
echo "The project option is:" $file_name
ls -la
chmod 775 jenkins/security/$file_name
ls -la
get_input_values() {
file=$1
IFS=''
while read line
do
if [ `echo ${line} | grep -E -c -w "NAME_SPACE" ` -gt 0 ]; then
NAME_SPACE=$(echo " ${line}" | cut -d'=' -f2)
echo "The name space value is $NAME_SPACE"
elif [ `echo ${line} | grep -E -c -w "IMAGE_NAMES" ` -gt 0 ]; then
values=$(echo " ${line}" | cut -d'=' -f2)
echo "THE DOCKERIMAGES are $DOCKER_IMAGES_NAMES"
else
echo "Please provide input for namespace and docker images to be scanned by VA_TOOl"
fi
done < ${file}
}
images=$(get_input_values ${file_name})
so here my text file is under jenkins/security folder of gitrepo but unfortunately I am getting this error:
16:05:28 + sh jenkins/security/get_values.sh
16:05:28 jenkins/security/get_values.sh: 16: jenkins/security/get_values.sh: cannot open va_input_file_icp.txt: No such file```
Unfortunately, there is a ticket for this in Jenkins (https://issues.jenkins-ci.org/browse/JENKINS-51245) which was closed as a duplicate of this ticket: (https://issues.jenkins-ci.org/browse/JENKINS-27413)
JENKINS-27413 was raised in 2015, and is still open. The File Parameter appears not to work in Jenkins Pipeline. It does however work when used in a Freestyle project. While not ideal, I would recommend changing your job to be a Freestyle project if that's feasible.
I have a file 'README.txt' which contains the line -
"version": "1.0.0-alpha-test.7"
Using a Jenkins Pipeline, I want to replace this line with
"version": "1.0.0-alpha-test.{BUILD_NUMBER}"
The following sed command works when I try it on a linux cluster
sed -i -E "s#(\"version\"[ ]*:[ ]*\".+alpha-test\.)[0-9]+\"#\1${BUILD_NUMBER}#g" README.txt
The same command does not work using a Jenkins Pipeline.
Tried with the following query but it doesn't work -
sh """
sed -i -E "s|([\"]version[\"][ ]*:[ ]*[\"].+alpha-test\\.)[0-9]+\"|\1${BUILD_NUMBER}|g" README.txt
cat README.txt
"""
/home/jenkins/workspace/test/test-pipeline#tmp/durable-eb774fcf/script.sh:
3:
/home/jenkins/workspace/test/test-pipeline#tmp/durable-eb774fcf/script.sh:
Syntax error: ")" unexpected
Best to use perl command instead...
script{
old_version = (sh (returnStdout: true, script:'''old_version=`cat version.cfg |grep VERSION=|cut -d "=" -f2`
echo $old_version''')).toString().trim()
sh """
if [ "$old_version" != $new_version ]; then
perl -pi -e "s,$old_version,$new_version,g" version.cfg
##-- git push operation --##
fi
"""
}
I am having problems figuring out how to pass some variables into the parallel runs in the Jenkins groovy script below:
#!/usr/bin/env groovy
def call(version, project) {
sh '''#!/bin/bash
[[ ! -e ${WORKSPACE}/target/rpm/${project}/RPMS/ ]] && mkdir -p ${WORKSPACE}/target/rpm/${project}/RPMS/
(( $(ls ${WORKSPACE}/target/rpm/${project}/RPMS/*.rpm | wc -l) != 0 )) && rm ${WORKSPACE}/target/rpm/${project}/RPMS/*.rpm
cd ${WORKSPACE}/scripts/fpm_requirements && bundle install && bundle show fpm
'''
parallel (
"package foo": {
sh '''#!/bin/bash
export PATH=$PATH:~/bin:~/.gem/ruby/gems
cd ${WORKSPACE}/scripts/fpm_requirements
echo Project is ${project}
echo Version is ${version}
echo Iteration is $(echo ${version} | cut -d . -f 3)
'''
},
"package bar": {
sh '''#!/bin/bash
export PATH=$PATH:~/bin:~/.gem/ruby/gems
cd ${WORKSPACE}/scripts/fpm_requirements
echo Project is ${project}
echo Version is ${version}
echo Iteration is $(echo ${version} | cut -d . -f 3)
'''
}
)
}
So the version and project variables are populated in the first shell that is called but when they hit the two parallel runs they are not being pulled in.
I have tried a few different options to pass them in but none have worked.
Does anyone have any relevant ideas that might help?
You should change the ''' to """. In Groovy, string inside single/triple quote won't trigger string interpolation, but string inside single/triple double quote will do that.
So the ${version} and ${project} in your Shell script will be treated as variable from Shell context, but actually they are exist in Groovy context.
More about Groovy String at here, Below option 2 more suitable for your issue.
Option 1) using "" or """
"package foo": {
sh """#!/bin/bash
export PATH=\$PATH:~/bin:~/.gem/ruby/gems
cd \${WORKSPACE}/scripts/fpm_requirements
echo Project is ${project}
echo Version is ${version}
echo Iteration is \$(echo ${version} | cut -d . -f 3)
"""
},
"package bar": {
sh """#!/bin/bash
export PATH=\$PATH:~/bin:~/.gem/ruby/gems
cd \${WORKSPACE}/scripts/fpm_requirements
echo Project is ${project}
echo Version is ${version}
echo Iteration is \$(echo ${version} | cut -d . -f 3)
"""
}
Attention: need to escape the $ ahead of ${WORKSPACE} and $(echo ..), because we hope $ be kept after interpolation.
Option 2) using ' or ''' and inject version and project into Environment Variables of Shell context.
def call(version, project) {
env.version=version
env.project=project
// Groovy env api used to inject groovy value into environment variable
// so that you can refer groovy value later in shell script
// still use ''' in following code, no need to change
...
Why do I get the following error:
++ cat package.json
++ jq .version
+ TAG = '"0.0.5"' /****/script.sh:
line 12: TAG: command not found
When running the below in a Jenkinsfile:
sh '''
...
...
TAG = \$(cat package.json | jq '.version')
git tag -a v${TAG} -m "New release: ${TAG}"
'''
As was mentioned in the comments, the problem was with the spaces before and after =. Additionally, I added escaping in the second line so it'll display as v0.0.6 and not v"0.0.6".
sh '''
...
...
TAG=\$(jq -r '.version' package.json)
git tag -a "v${TAG}" -m "New release: ${TAG}"
'''
I am using a Jenkinsfile in a pipeline on version 2.32.2.
For various reasons I want to extract the version string from the pom. I was hoping I wouldn't have to add the maven help plugin and use evaluate.
I quickly came up with a little sed expression to get it out of the pom which uses pipes and works on the commandline in the jenkins workspace on the executor.
$ sed -n '/<version>/,/<version/p' pom.xml | head -1 | sed 's/[[:blank:]]*<\/*version>//g'
1.0.0-SNAPSHOT
It could probably be optimized, but I want to understand why the pipeline seems to be failing on piped sh commands. I've played with various string formats and am currently using a dollar slashy string.
The pipeline step looks like the following to allow for easy output of the command string:
script {
def ver_script = $/sed -n '/<version>/,/<version/p' pom.xml | head -1 | sed 's/[[:blank:]]*<\/*version>//g'/$
echo "${ver_script}"
POM_VERSION = sh(script: "${ver_script}", returnStdout: true)
echo "${POM_VERSION}"
}
When run in the jenkins pipeline I get the following console output where it seems to be separating the piped commands into separate commands:
[Pipeline] script
[Pipeline] {
[Pipeline] echo
sed -n '/<version>/,/<version/p' pom.xml | head -1 | sed 's/[[:blank:]]*<\/*version>//g'
[Pipeline] sh
[FRA-198-versioned-artifacts-44SD6DBQOGOI54UEF7NYE4ECARE7RMF7VQYXDPBVFOHS5CMSTFLA] Running shell script
+ sed -n /<version>/,/<version/p pom.xml
+ head -1
+ sed s/[[:blank:]]*<\/*version>//g
sed: couldn't write 89 items to stdout: Broken pipe
[Pipeline] }
[Pipeline] // script
Any guidance out there on how to properly use piped commands in a jenkinsfile ?
I finally put some thought into it and realized that pipe subshells are probably causing the issue. I know some of the evils of eval but I ended up wrappping this in an eval:
script {
def ver_script = $/eval "sed -n '/<version>/,/<version/p' pom.xml | head -1 | sed 's/[[:blank:]]*<\/*version>//g'"/$
echo "${ver_script}"
POM_VERSION = sh(script: "${ver_script}", returnStdout: true)
echo "${POM_VERSION}"
}
I know this kind of late answer, but whoever you who needs the solution without eval you can use /bin/bash -c "script" to make pipe works
script {
POM_VERSION = sh(script: "/bin/bash -c 'sed -n \'/<version>/,/<version/p\' pom.xml | head -1 | sed \'s/[[:blank:]]*<\/*version>//g\'\''", returnStdout: true)
echo "${POM_VERSION}"
}
The only problem with this method is hellish escape yet this way the subshell of pipe will be handled by our boy /bin/bash -c
If your environment allows it, I've found a simple solution to this problem to be to place your script containing pipes into a file, and then run that with sh, like so:
script.sh:
#!/bin/sh
kubectl exec --container bla -i $(kubectl get pods | awk '/foo-/{ print $1 }') -- php /code/dostuff
Jenkinsfile:
stage('Run script with pipes') {
steps {
sh "./script.sh"
}
}
The pipeline-utility-steps plugin nowadays includes a readMavenPom step, which allows to access the version as follows:
version = readMavenPom.getVersion()
So nothing detailed above worked for me using the scripted Jenkinsfile syntax with Groovy. I was able to get it working, however. The type of quotations you use are important. In the example below, I am trying to fetch the latest git tag from GitHub.
...
stage("Get latest git tag") {
if (env.CHANGE_BRANCH == 'master') {
sh 'git fetch --tags'
TAGGED_COMMIT = sh(script: 'git rev-list --branches=master --tags --max-count=1', returnStdout: true).trim()
LATEST_TAG = sh(script: 'git describe --abbrev=0 --tags ${TAGGED_COMMIT}', returnStdout: true).trim()
VERSION_NUMBER = sh(script: "echo ${LATEST_TAG} | cut -d 'v' -f 2", returnStdout: true).trim()
echo "VERSION_NUMBER: ${VERSION_NUMBER}"
sh 'echo "VERSION_NUMBER: ${VERSION_NUMBER}"'
}
}
...
Notice how the shell execution to assign LATEST_TAG works as expected (assigning the variable to v2.1.0). If we were to try the same thing (with single quotes) to assign VERSION_NUMBER, it would NOT work - the pipe messes everything up. Instead, we wrap the script in double quotes.
The first echo prints VERSION_NUMBER: 2.1.0 but the second prints VERSION_NUMBER:. If you want VERSION_NUMBER to be available in the shell commands, you have to assign the output of the shell command to env.VERSION_NUMBER as shown below:
...
stage("Get latest git tag") {
if (env.CHANGE_BRANCH == 'master') {
sh 'git fetch --tags'
TAGGED_COMMIT = sh(script: 'git rev-list --branches=master --tags --max-count=1', returnStdout: true).trim()
LATEST_TAG = sh(script: 'git describe --abbrev=0 --tags ${TAGGED_COMMIT}', returnStdout: true).trim()
env.VERSION_NUMBER = sh(script: "echo ${LATEST_TAG} | cut -d 'v' -f 2", returnStdout: true).trim()
echo "VERSION_NUMBER: ${VERSION_NUMBER}"
sh 'echo "VERSION_NUMBER: ${VERSION_NUMBER}"'
}
}
...
The first echo prints VERSION_NUMBER: 2.1.0 and the second prints VERSION_NUMBER: 2.1.0.
I am also struggling with the usage of pipe inside my jenkins pipeline but as a side note, if you want a simple way to extract the version of a maven pom, here's a very clean one I found in another post and that I'm using :
stage('Preparation') {
version = getVersion()
print "version : " + version
}
def getVersion() {
def matcher = readFile('pom.xml') =~ '<version>(.+)</version>'
matcher ? matcher[0][1] : null
}
gives you :
[Pipeline] echo
releaseVersion : 0.1.24
[Pipeline] sh