How to make file expressions work with Jenkinsfile?
I want to run simple command rm -rv !(_deps) to remove all populated files except _deps directory. What I've tried so far:
sh '''rm -rv !(_deps)''' which caused script.sh: line 1: syntax error near unexpected token `('
sh 'rm -rv !\\(_deps\\)' which caused rm: cannot remove '!(_deps)': No such file or directory (but it DOES exist)
sh 'rm -rv !\(_deps\)' which caused unexpected char: '\'
sh 'rm -rv !(_deps)' which caused syntax error near unexpected token `('
In Bash to use pattern matching you have to enable extglob:
So the following will work. Make sure your shell executor is set to bash.
sh """
shopt -s extglob
rm -rv !(_deps)
"""
If you don't want to enable extglob you can use a different command to get the directories deleted. Following is one option.
ls | grep -v _deps | xargs rm -rv
In Jenkins
sh """
ls | grep -v _deps | xargs rm -rv
"""
Related
I'm writing a jenkins pipeline jenkinsfile and within the script clause I have to ssh to a box and run some commands. I think the problem has to do with the env vars that I'm using within the quotes. I'm getting a ENDSSH command not found error and I'm at a loss. Any help would be much appreciated.
stage("Checkout my-git-repo"){
steps {
script {
sh """
ssh -o StrictHostKeyChecking=accept-new -o LogLevel=ERROR -o UserKnownHostsFile=/dev/null -i ${JENKINS_KEY} ${JENKINS_KEY_USR}#${env.hostname} << ENDSSH
echo 'Removing current /opt/my-git-repo directory'
sudo rm -rf /opt/my-git-repo
echo 'Cloning new my-git-repo repo into /opt'
git clone ssh://${JENKINS_USR}#git.gitbox.com:30303/my-git-repo
sudo mv /home/jenkins/my-git-repo /opt
ENDSSH
"""
}
}
}
-bash: line 6: ENDSSH: command not found
I'm personally not familiar with jenkins, but I'd guess the issue is the whitespace before ENDSSH
White space in front of the delimiter is not allowed.
(https://linuxize.com/post/bash-heredoc/)
Try either removing the indentation:
stage("Checkout my-git-repo"){
steps {
script {
sh """
ssh -o StrictHostKeyChecking=accept-new -o LogLevel=ERROR -o UserKnownHostsFile=/dev/null -i ${JENKINS_KEY} ${JENKINS_KEY_USR}#${env.hostname} << ENDSSH
echo 'Removing current /opt/my-git-repo directory'
sudo rm -rf /opt/my-git-repo
echo 'Cloning new my-git-repo repo into /opt'
git clone ssh://${JENKINS_USR}#git.gitbox.com:30303/my-git-repo
sudo mv /home/jenkins/my-git-repo /opt
ENDSSH
"""
}
}
}
OR ensure that the whitespace is only tabs and replace << with <<-:
Appending a minus sign to the redirection operator <<-, will cause all
leading tab characters to be ignored. This allows you to use
indentation when writing here-documents in shell scripts. Leading
whitespace characters are not allowed, only tab.
I have a file 'README.txt' which contains the line -
"version": "1.0.0-alpha-test.7"
Using a Jenkins Pipeline, I want to replace this line with
"version": "1.0.0-alpha-test.{BUILD_NUMBER}"
The following sed command works when I try it on a linux cluster
sed -i -E "s#(\"version\"[ ]*:[ ]*\".+alpha-test\.)[0-9]+\"#\1${BUILD_NUMBER}#g" README.txt
The same command does not work using a Jenkins Pipeline.
Tried with the following query but it doesn't work -
sh """
sed -i -E "s|([\"]version[\"][ ]*:[ ]*[\"].+alpha-test\\.)[0-9]+\"|\1${BUILD_NUMBER}|g" README.txt
cat README.txt
"""
/home/jenkins/workspace/test/test-pipeline#tmp/durable-eb774fcf/script.sh:
3:
/home/jenkins/workspace/test/test-pipeline#tmp/durable-eb774fcf/script.sh:
Syntax error: ")" unexpected
Best to use perl command instead...
script{
old_version = (sh (returnStdout: true, script:'''old_version=`cat version.cfg |grep VERSION=|cut -d "=" -f2`
echo $old_version''')).toString().trim()
sh """
if [ "$old_version" != $new_version ]; then
perl -pi -e "s,$old_version,$new_version,g" version.cfg
##-- git push operation --##
fi
"""
}
I am trying to create a Jenkins pipeline where I need to execute multiple shell commands and use the result of one command in the next command or so. I found that wrapping the commands in a pair of three single quotes ''' can accomplish the same. However, I am facing issues while using pipe to feed output of one command to another command. For example
stage('Test') {
sh '''
echo "Executing Tests"
URL=`curl -s "http://localhost:4040/api/tunnels/command_line" | jq -r '.public_url'`
echo $URL
RESULT=`curl -sPOST "https://api.ghostinspector.com/v1/suites/[redacted]/execute/?apiKey=[redacted]&startUrl=$URL" | jq -r '.code'`
echo $RESULT
'''
}
Commands with pipe are not working properly. Here is the jenkins console output:
+ echo Executing Tests
Executing Tests
+ curl -s http://localhost:4040/api/tunnels/command_line
+ jq -r .public_url
+ URL=null
+ echo null
null
+ curl -sPOST https://api.ghostinspector.com/v1/suites/[redacted]/execute/?apiKey=[redacted]&startUrl=null
I tried entering all these commands in the jenkins snippet generator for pipeline and it gave the following output:
sh ''' echo "Executing Tests"
URL=`curl -s "http://localhost:4040/api/tunnels/command_line" | jq -r \'.public_url\'`
echo $URL
RESULT=`curl -sPOST "https://api.ghostinspector.com/v1/suites/[redacted]/execute/?apiKey=[redacted]&startUrl=$URL" | jq -r \'.code\'`
echo $RESULT
'''
Notice the escaped single quotes in the commands jq -r \'.public_url\' and jq -r \'.code\'. Using the code this way solved the problem
UPDATE: : After a while even that started to give problems. There were certain commands executing prior to these commands. One of them was grunt serve and the other was ./ngrok http 9000. I added some delay after each of these commands and it solved the problem for now.
The following scenario shows a real example that may need to use multiline shell commands. Which is, say you are using a plugin like Publish Over SSH and you need to execute a set of commands in the destination host in a single SSH session:
stage ('Prepare destination host') {
sh '''
ssh -t -t user#host 'bash -s << 'ENDSSH'
if [[ -d "/path/to/some/directory/" ]];
then
rm -f /path/to/some/directory/*.jar
else
sudo mkdir -p /path/to/some/directory/
sudo chmod -R 755 /path/to/some/directory/
sudo chown -R user:user /path/to/some/directory/
fi
ENDSSH'
'''
}
Special Notes:
The last ENDSSH' should not have any characters before it. So it
should be at the starting position of a new line.
use ssh -t -t if you have sudo within the remote shell command
I split the commands with &&
node {
FOO = world
stage('Preparation') { // for display purposes
sh "ls -a && pwd && echo ${FOO}"
}
}
The example outputs:
- ls -a (the files in your workspace
- pwd (location workspace)
- echo world
When not using --nonall, the --workdir ... option ("create working dirs under ~/.parallel/tmp/ on the remote computers") works as expected:
$ parallel -S target-server --workdir ... pwd ::: ""
/home/myuser/.parallel/tmp/my-machine-7285-1
However, when I add the --nonall option, it no longer has any effect:
$ parallel --nonall -S target-server --workdir ... pwd ::: ""
/home/myuser
Even when specifying an explicit working directory such as /home, this works:
$ parallel -S target-server --workdir /home pwd ::: ""
/home
...but this doesn't:
$ parallel --nonall -S target-server --workdir /home pwd ::: ""
/home/myuser
Any ideas why parallel is ignoring --workdir when using --nonall?
It is clearly a bug. Now registered in the bug list: https://savannah.gnu.org/bugs/index.php?46819
Feel free to add yourself as Cc if you want to be kept informed.
Of course some shell commands are restricted from use on cloudbees. Is there any workaround for Facebook-iOS-SDK installation via cocoapods?
I just run
pod install
and in the installation of facebook sdk there's a script that is blocked by the system:
Installing Facebook-iOS-SDK (3.16.2)
[!] /bin/bash
set -e
find src -name \*.png | grep -v # | grep -v -- - | sed -e 's|\(.*\)/\([a-zA-Z0-9]*\).png|scripts/image_to_code.py -i \1/\2.png -c \2 -o src|' | sh && find src -name \*.wav | grep -v # | grep -v -- - | sed -e 's|\(.*\)/\([a-zA-Z0-9]*\).wav|scripts/audio_to_code.py -i \1/\2.wav -c \2 -o src|' | sh
sh: line 12: src/FacebookSDKApplicationTests/ReferenceImages/FBLikeControlTests/testStyleStandard_1_123_2_2.png: Permission denied
sh: line 13: src/FacebookSDKApplicationTests/ReferenceImages/FBLikeControlTests/testStyleStandard_0_123_1_0.png: Permission denied
and so on..
This is just copying of resources so I wonder how can I manage to install it?
The key error message is below, i believe you have same in bottom of error output.
sh: scripts/image_to_code.py: /usr/bin/python^M: bad interpreter: No such file or directory
The problem was caused by file formatter.
To quick solve this problem is enter this command :
sudo ln -s /usr/bin/python /usr/bin/python^M
refer:
https://github.com/CocoaPods/CocoaPods/issues/2418