I am using Jenkins to take a number of parameters, generate an ansible-playbook command and run it. My Jenkins server is also my Ansible server.
My shell says ::
echo $ESXi_IP
echo $VM_NAME
echo $NIC1_MAC
echo $NIC2_MAC
echo $NIC3_MAC
echo $NIC4_MAC
echo $ESXi_HOSTNAME
echo $PLAYBOOK
ansible-playbook $PLAYBOOK --extra-vars "esxi_ip=$ESXi_IP vm_name=$VM_NAME nic1_mac=$NIC1_MAC nic2_mac=$NIC2_MAC nic3_mac=$NIC3_MAC nic4_mac=$NIC4_MAC esxi_hostname=$ESXi_HOSTNAME"
When I run the Job, the output is ::
+ ansible-playbook /root/ansible/sc-ece.yaml --extra-vars 'esxi_ip=5.232.66.49 vm_name=JenkinsTest nic1_mac=00:50:C0:A8:01:02 nic2_mac=00:50:0A:A9:37:A5 nic3_mac=00:50:0A:FF:FE:4C nic4_mac=00:50:AC:10:01:65 esxi_hostname=tmolab13-14iamesxi4'
ERROR! the playbook: /root/ansible/sc-ece.yaml could not be found
The playbook path is correct. there is no issue in it at all.
What seems to be missing here ?
You are correct Matt & Dave. Permissions to the folder was an issue. Thanks !
Related
I have a legacy project in Jenkins that hast to be pipelined (for
later parallelization), hence moving from simple tcsh script to
pipeline
running the script as
#!/bin/tcsh
source ./mysetting.sh
update
works but the same pipeline step fails due to missing alias expansion
stage ('update') {
steps {
//should be working but alias expansion fails
sh 'tcsh -c "source ./mysettings.sh; alias; update"'
//manually expanding the alias works fine
sh 'tcsh -c "source ./mysettings.sh; alias; python update.py;"'
}
}
calling alias in the steps properly lists all the set aliases, so I
can see them, but not use them.
I know in bash alias expansion has to be set
#enable shell option for alias_expansion
shopt -s expand_aliases
but in csh/tcsh that should be taken care of by source.
what am I missing?
found the solution:
sh '#!/bin/tcsh \n' +
'source ./mysettings.sh \n' +
'echo "Calling my alias" \n' +
'my_alias \n'
every line starting with sh launches a new shell, so it has to be in one line including line breaks.
further adding to the confusing was that documentation of jenkins says that it starts "a bash" but it launched /bin/sh which in my case pointed to something else
I try to integrate ansible aws_ec2 dynamic inventory into a Jenkins, but I have this kind of problem:
ansible-playbook -i aws_ec2.yaml ping-pb.yaml --private-key **** -e ansible_ssh_user=ec2-user
usage: ansible-playbook [-h] [--version] [-v] [-k]
[--private-key PRIVATE_KEY_FILE] [-u REMOTE_USER]
---
[--list-tags] [--step] [--start-at-task START_AT_TASK]
playbook [playbook ...]
ansible-playbook: error: unrecognized arguments: provisioning configuration#tmp/secretFiles/bbe6eeff-027e-480b-ae88/keyforjenk010422.pem
I added my .pem file into a "secret file" storage of Jenkins credentials.
This is how my pipeline looks like:
steps {
withCredentials([file(credentialsId: 'rmp-pem', variable: 'PRIVATE')]) {
dir('Ansible') {
sh "ansible-playbook -i aws_ec2.yaml ping-pb.yaml --private-key $PRIVATE -e 'ansible_ssh_user=ec2-user'" } } } }
When I don't use the credentials but store the file in my Github repo it works fine, so the problem is.. in Jenkins?
I also tried to store my private key as sshUserPrivateKey in Jenkins, but the result is the same.
It's because Jenkins seems to have generated a filename that contain spaces, so you'll need to quote that $PRIVATE shell variable to present it as one argument to --private-key
Due to the existing " string literal used by Groovy, you will need to escape the quote characters:
sh "ansible-playbook -i aws_ec2.yaml ping-pb.yaml --private-key \"$PRIVATE\" -e 'ansible_ssh_user=ec2-user'"
or, you can use Groovy's alternative syntax
sh """ansible-playbook -i aws_ec2.yaml ping-pb.yaml --private-key "$PRIVATE" -e 'ansible_ssh_user=ec2-user'"""
You need to use " specifically because '$PRIVATE' would not expand the shell variable
I'm trying to set up a Jenkins pipeline which will run a liquibase update whenever something is pushed to the master branch. The liquibase runner plugin for Jenkins has a security risk and therefore, I can't install it and run liquibase updates from that.
My liquibase* file (the bash script) is in my repository at the following path
/repo/liquibase/liquibase/liquibase*
I've set up the pipeline to run the following shell script. NOTE: I have the command set to liquibase --help for test purposes, but normally I'd want to run an update command.
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase/liquibase
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase/liquibase/liquibase
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase/liquibase/jre/bin
cd liquibase
ls -ltr
chmod 755 liquibase/liquibase
chmod 755 liquibase/jre/bin/java.exe
liquibase --help
The liquibase --help command runs fine from the directory path /repo/liquibase in git bash. However, when I run it from Jenkins, I get the following error.
/var/lib/jenkins/workspace/Database_and_Repos/liquibase/liquibase/liquibase/jre/bin/java: No such file or directory
Build step 'Execute shell' marked build as failure
My liquibase file looks like this and it is the last line in the file that is causing the error.
#!/usr/bin/env bash
if [ ! -n "${LIQUIBASE_HOME+x}" ]; then
# echo "LIQUIBASE_HOME is not set."
## resolve links - $0 may be a symlink
PRG="$0"
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`"/$link"
fi
done
LIQUIBASE_HOME=`dirname "$PRG"`
# make it fully qualified
LIQUIBASE_HOME=`cd "$LIQUIBASE_HOME" && pwd`
# echo "Liquibase Home: $LIQUIBASE_HOME"
fi
# build classpath from all jars in lib
if [ -f /usr/bin/cygpath ]; then
CP=.
for i in "$LIQUIBASE_HOME"/liquibase*.jar; do
i=`cygpath --windows "$i"`
CP="$CP;$i"
done
for i in "$LIQUIBASE_HOME"/lib/*.jar; do
i=`cygpath --windows "$i"`
CP="$CP;$i"
done
else
if [[ $(uname) = MINGW* ]]; then
CP_SEPARATOR=";"
else
CP_SEPARATOR=":"
fi
CP=.
for i in "$LIQUIBASE_HOME"/liquibase*.jar; do
CP="$CP""$CP_SEPARATOR""$i"
done
CP="$CP""$CP_SEPARATOR""$LIQUIBASE_HOME/lib/"
for i in "$LIQUIBASE_HOME"/lib/*.jar; do
CP="$CP""$CP_SEPARATOR""$i"
done
fi
if [ -z "${JAVA_HOME}" ]; then
#JAVA_HOME not set, try to find a bundled version
if [ -d "${LIQUIBASE_HOME}/jre" ]; then
JAVA_HOME="$LIQUIBASE_HOME/jre"
elif [ -d "${LIQUIBASE_HOME}/.install4j/jre.bundle/Contents/Home" ]; then
JAVA_HOME="${LIQUIBASE_HOME}/.install4j/jre.bundle/Contents/Home"
fi
fi
if [ -z "${JAVA_HOME}" ]; then
JAVA_PATH="$(which java)"
if [ -z "${JAVA_PATH}" ]; then
echo "Cannot find java in your path. Install java or use the JAVA_HOME environment variable"
fi
else
#Use path in JAVA_HOME
JAVA_PATH="${JAVA_HOME}/bin/java"
fi
# add any JVM options here
JAVA_OPTS="${JAVA_OPTS-}"
"${JAVA_PATH}" -cp "$CP" $JAVA_OPTS liquibase.integration.commandline.Main ${1+"$#"}
Has anyone run into this problem with liquibase commands in Jenkins? I've been googling all day, but haven't found much similar to this exact issue. Any help in the right direction would be great.
(We're updating the Liquibase Runner plugin. We have a release that is being reviewed for the security issues by the Jenkins team now.)
The error message seems to say that your "Execute shell" command in your changelog is not working correctly. Maybe the command is not installed, maybe it's calling a script that is not on your build machine.
One way to explore this to add an "echo" of the "Execute shell" command prior. Also, I'd pass --logLeve=DEBUG to Liquibase to get a better idea on the command it's trying to run.
Thanks for using Liquibase and Jenkins! I'll be talking about it here next month: https://www.cloudbees.com/devops-world/.
You could use the liquibase-maven-plugin and just call the maven phase in the pipeline:
sh mvn resources:resources liquibase:update
As for me, it is the best decision. Follow the official documentation
My scenario is, I have parameterized build and inside the build section, I have executed shell where I define a variable and then echo to print it. But it doesn't print anything in the console output.
I hope I have made myself clear. Could anyone please answer my question?
current_folder=`date +%Y%m%d-%H%M%S`
echo $current_folder
enter image description here
I'm using Jenkins ver. 2.32.3 and a simple freestyle job, running on mac OS, using an execute shell build step of:
current_folder=`date +%Y%m%d-%H%M%S`
echo $current_folder
Gives output of:
$ /bin/sh -xe /var/folders/kh/4fl0eeldofefmmsfd/T/hudson89388543547899686.sh
++ date +%Y%m%d-%H%M%S
+ current_folder=20180613-081712
+ echo 20180613-081712
20180613-081712
Finished: SUCCESS
In a similar fashion, setting the shell:
#!/bin/bash
current_folder=`date +%Y%m%d-%H%M%S`
echo $current_folder
Gives:
$ /bin/bash /var/folders/kh/by0kd93dfew5fgjhy000h6/T/hudson62702345565786787.sh
20180613-081655
Finished: SUCCESS
The same applies to a parameter that is defined as part of the Jenkins job, underneath the
This project is parameterized checkbox once set. For example, if you have a string parameter called userName with a default value of User1, then you can print it's value in an Execute Shell build step using:
echo $userName
echo ${userName}
echo "In a string ${userName}"
Giving:
User1
User1
In a string User1
TL;DR I want to use the sh step even though Jenkins is running on Windows. I do not want to use the bat step, unless you can show me how to easily reproduce what I need done using bat
I've been converting some old Jenkins jobs over to 2.x Pipeline script. One of my jobs uses the "Publish over SSH plugin" to:
Send artifacts to a remote server
Exec a set of commands on the remote server
For instance:
I wanted to replicate this in Pipeline Script so I've done the following:
stage('Deploy') {
withCredentials([[$class: 'FileBinding', credentialsId: 'bitbucket-key-file', variable: 'SSHKEY']]) {
sh '''
scp -i "$SSHKEY" dsub.tar.gz tprmbbuild#192.168.220.57:dsubdeploy
scp -i "$SSHKEY" deployDsubUi.sh tprmbbuild#192.168.220.57:dsubdeploy
ssh -i "$SSHKEY" -o StrictHostKeyChecking=no 192.168.220.57 <<- EOF
DEPLOY_DIR=/home/tprmbbuild/dsubdeploy
echo '*** dos2unix using sed'
sed -e 's/\r$//' $DEPLOY_DIR/deployDsubUi.sh > $DEPLOY_DIR/deployDsubUi-new.sh
mv $DEPLOY_DIR/deployDsubUi-new.sh $DEPLOY_DIR/deployDsubUi.sh
chmod 755 $DEPLOY_DIR/deployDsubUi.sh
echo '*** Deploying Dsub UI'
$DEPLOY_DIR/deployDsubUi.sh $DEPLOY_DIR/dsub.tar.gz
EOF'''
}
}
Problem is, I get this stack trace when executing my build:
[Pipeline] sh
[E:\Jenkins\jenkins_home\workspace\tpr-ereg-ui-deploy#2] Running shell script
1 [main] sh 3588 E:\Jenkins\tools\Git_2.10.1\usr\bin\sh.exe: *** fatal error - add_item ("\??\E:\Jenkins\tools\Git_2.10.1", "/", ...) failed, errno 1
Stack trace:
Frame Function Args
000FFFF9BB0 0018005C24E (0018023F612, 0018021CC39, 000FFFF9BB0, 000FFFF8B30)
000FFFF9BB0 001800464B9 (000FFFFABEE, 000FFFF9BB0, 1D2345683BEC046, 00000000000)
000FFFF9BB0 001800464F2 (000FFFF9BB0, 00000000001, 000FFFF9BB0, 4A5C3A455C3F3F5C)
000FFFF9BB0 001800CAA8B (00000000000, 000FFFFCE00, 001800BA558, 1D234568CAFA549)
000FFFFCC00 00180118745 (00000000000, 00000000000, 001800B2C5E, 00000000000)
000FFFFCCC0 00180046AE5 (00000000000, 00000000000, 00000000000, 00000000000)
00000000000 00180045753 (00000000000, 00000000000, 00000000000, 00000000000)
000FFFFFFF0 00180045804 (00000000000, 00000000000, 00000000000, 00000000000)
End of stack trace
Agreed with "it is my belief it is failing to spawn the shell". It is trying to run "E:\Jenkins\tools\Git_2.10.1\usr\bin\sh.exe" (using Windows backslash syntax). Unless we have a shell executable configured (sh.exe) in the mentioned directory, it will fail.
Powershell (or Cmd Shell):
If you are oepn to use batch files, you would have to install/configure 3 binaries (ssh, scp, ssh). Everything else falls in place (I see that you are channeling commands to a remote machine using ssh. I assume that the remote server is linux/unix based).
Alternatives:
You can use cygwin or run linux on virtualbox (or any software that emulates linux on windows). But, running just 3 commands may not be worth the trouble (It will definitely be fruitful if you have plans to convert/write more shell scripts in future).
You can use "bat" instead of "sh" in windows.
Also use 2 backslashes to escape the path string correctly. See example below
node {
currentBuild.result = "SUCCESS"
try {
stage('Checkout'){
checkout scm
}
stage('Convert to Binary RPD'){
bat "D:\\oracle\\Middleware\\user_projects\\domains\\bi\\bitools\\bin\\biserverxmlexec -D .\\RPD -P Gl081Reporting -O .\\GLOBI.rpd"
}
stage('Notify'){
echo 'sending email'
// send to email
emailext (
subject: "SUCCESS: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'",
body: """$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS:
Check console output at $BUILD_URL to view the results.""",
to:"girish.lakshmanan#abc.co.uk girish.la#gmail.com"
)
}
}
catch (err) {
currentBuild.result = "FAILURE"
throw err
}
}
You should be able to run the scp.exe from the git installation directly within a batch script.
There is no "here" document for batch as far as I know, but you could just put the script to be run on the server into a separate file.
(Untested)
stage('Deploy') {
withCredentials([[$class: 'FileBinding', credentialsId: 'bitbucket-key-file', variable: 'SSHKEY']]) {
bat '''
E:\\Jenkins\\tools\\Git_2.10.1\\usr\\bin\\scp.exe -i "${SSHKEY}" dsub.tar.gz tprmbbuild#192.168.220.57:dsubdeploy
E:\\Jenkins\\tools\\Git_2.10.1\\usr\\bin\\scp.exe -i "${SSHKEY}" deployDsubUi.sh tprmbbuild#192.168.220.57:dsubdeploy
E:\\Jenkins\\tools\\Git_2.10.1\\usr\\bin\\scp.exe -i "${SSHKEY}" -o StrictHostKeyChecking=no 192.168.220.57 < server_script.sh
}
}