I'm trying to set up a Jenkins pipeline which will run a liquibase update whenever something is pushed to the master branch. The liquibase runner plugin for Jenkins has a security risk and therefore, I can't install it and run liquibase updates from that.
My liquibase* file (the bash script) is in my repository at the following path
/repo/liquibase/liquibase/liquibase*
I've set up the pipeline to run the following shell script. NOTE: I have the command set to liquibase --help for test purposes, but normally I'd want to run an update command.
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase/liquibase
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase/liquibase/liquibase
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase/liquibase/jre/bin
cd liquibase
ls -ltr
chmod 755 liquibase/liquibase
chmod 755 liquibase/jre/bin/java.exe
liquibase --help
The liquibase --help command runs fine from the directory path /repo/liquibase in git bash. However, when I run it from Jenkins, I get the following error.
/var/lib/jenkins/workspace/Database_and_Repos/liquibase/liquibase/liquibase/jre/bin/java: No such file or directory
Build step 'Execute shell' marked build as failure
My liquibase file looks like this and it is the last line in the file that is causing the error.
#!/usr/bin/env bash
if [ ! -n "${LIQUIBASE_HOME+x}" ]; then
# echo "LIQUIBASE_HOME is not set."
## resolve links - $0 may be a symlink
PRG="$0"
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`"/$link"
fi
done
LIQUIBASE_HOME=`dirname "$PRG"`
# make it fully qualified
LIQUIBASE_HOME=`cd "$LIQUIBASE_HOME" && pwd`
# echo "Liquibase Home: $LIQUIBASE_HOME"
fi
# build classpath from all jars in lib
if [ -f /usr/bin/cygpath ]; then
CP=.
for i in "$LIQUIBASE_HOME"/liquibase*.jar; do
i=`cygpath --windows "$i"`
CP="$CP;$i"
done
for i in "$LIQUIBASE_HOME"/lib/*.jar; do
i=`cygpath --windows "$i"`
CP="$CP;$i"
done
else
if [[ $(uname) = MINGW* ]]; then
CP_SEPARATOR=";"
else
CP_SEPARATOR=":"
fi
CP=.
for i in "$LIQUIBASE_HOME"/liquibase*.jar; do
CP="$CP""$CP_SEPARATOR""$i"
done
CP="$CP""$CP_SEPARATOR""$LIQUIBASE_HOME/lib/"
for i in "$LIQUIBASE_HOME"/lib/*.jar; do
CP="$CP""$CP_SEPARATOR""$i"
done
fi
if [ -z "${JAVA_HOME}" ]; then
#JAVA_HOME not set, try to find a bundled version
if [ -d "${LIQUIBASE_HOME}/jre" ]; then
JAVA_HOME="$LIQUIBASE_HOME/jre"
elif [ -d "${LIQUIBASE_HOME}/.install4j/jre.bundle/Contents/Home" ]; then
JAVA_HOME="${LIQUIBASE_HOME}/.install4j/jre.bundle/Contents/Home"
fi
fi
if [ -z "${JAVA_HOME}" ]; then
JAVA_PATH="$(which java)"
if [ -z "${JAVA_PATH}" ]; then
echo "Cannot find java in your path. Install java or use the JAVA_HOME environment variable"
fi
else
#Use path in JAVA_HOME
JAVA_PATH="${JAVA_HOME}/bin/java"
fi
# add any JVM options here
JAVA_OPTS="${JAVA_OPTS-}"
"${JAVA_PATH}" -cp "$CP" $JAVA_OPTS liquibase.integration.commandline.Main ${1+"$#"}
Has anyone run into this problem with liquibase commands in Jenkins? I've been googling all day, but haven't found much similar to this exact issue. Any help in the right direction would be great.
(We're updating the Liquibase Runner plugin. We have a release that is being reviewed for the security issues by the Jenkins team now.)
The error message seems to say that your "Execute shell" command in your changelog is not working correctly. Maybe the command is not installed, maybe it's calling a script that is not on your build machine.
One way to explore this to add an "echo" of the "Execute shell" command prior. Also, I'd pass --logLeve=DEBUG to Liquibase to get a better idea on the command it's trying to run.
Thanks for using Liquibase and Jenkins! I'll be talking about it here next month: https://www.cloudbees.com/devops-world/.
You could use the liquibase-maven-plugin and just call the maven phase in the pipeline:
sh mvn resources:resources liquibase:update
As for me, it is the best decision. Follow the official documentation
Related
I have a legacy project in Jenkins that hast to be pipelined (for
later parallelization), hence moving from simple tcsh script to
pipeline
running the script as
#!/bin/tcsh
source ./mysetting.sh
update
works but the same pipeline step fails due to missing alias expansion
stage ('update') {
steps {
//should be working but alias expansion fails
sh 'tcsh -c "source ./mysettings.sh; alias; update"'
//manually expanding the alias works fine
sh 'tcsh -c "source ./mysettings.sh; alias; python update.py;"'
}
}
calling alias in the steps properly lists all the set aliases, so I
can see them, but not use them.
I know in bash alias expansion has to be set
#enable shell option for alias_expansion
shopt -s expand_aliases
but in csh/tcsh that should be taken care of by source.
what am I missing?
found the solution:
sh '#!/bin/tcsh \n' +
'source ./mysettings.sh \n' +
'echo "Calling my alias" \n' +
'my_alias \n'
every line starting with sh launches a new shell, so it has to be in one line including line breaks.
further adding to the confusing was that documentation of jenkins says that it starts "a bash" but it launched /bin/sh which in my case pointed to something else
We have this workspace and subdirectory:
$WORKSPACE/Refrens data/11. metadata/file.xml
in Jenkins section Execute Shell we have this code:
DEPLOY_FILE=$WORKSPACE/Refrens data/11. metadata/file.xml
if [[ -s $DEPLOY_FILE ]];
then
echo "$WORKSPACE/$DEPLOY_FILE exits in Bitbucket"
echo ""
else
echo "$WORKSPACE/$DEPLOY_FILE does not exits in Bitbucket"
exit 1;
fi
When building it says that this file does not exist and the else part is executed
$WORKSPACE/Referens data/11. metadata/file.xml does not exist in Bitbucket
But our file exist!
I think Jenkins does not understand space in the directory names Reference data and 11. metadata how do I fix it?
hope one of you can help - I'm running a script in jenkins pipeline, so that I can upload source maps to rollbar, so I need to loop through the minified js files - I'm trying to do this with the FIND command but it keeps giving me an error: find parameter format not correct - script below:
stages {
stage('Set correct environment link') {
steps {
script {
buildTarget = params.targetEnv
if(params.targetEnv.equals("prj1")){
linkTarget = "http://xxx.xxx.xx/xxx/"
} else if(params.targetEnv.equals(.......)..etc.
stage('Uploading source maps to Rollbar') {
steps {
sh ''' #!/bin/bash
# Save a short git hash, must be run from a git
# repository (or a child directory)
version=$(git rev-parse --short HEAD)
# Use the post_server_time access token, you can
# find one in your project access token settings
post_server_item=e1d04646bf614e039d0af3dec3fa03a7
echo "Uploading source maps for version $version!"
# We upload a source map for each resulting JavaScript
# file; the path depends on your build config
for path in $(find dist/renew -name "*.js"); do
# URL of the JavaScript file on the web server
url=${linkTarget}/$path
# a path to a corresponding source map file
source_map="#$path.map"
echo "Uploading source map for $url"
curl --silent --show-error https://api.rollbar.com/api/1/sourcemap \
-F access_token=$post_server_item \
-F version=$version \
-F minified_url=$url \
-F source_map=$source_map \
> /dev/null
done
'''
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
From Bash CLI
With the help of #jeb I managed to resolve this FIND issue on jenkins by placing the absolute path for (find) in the shell script - initially it was using the windows FIND cmd, so i needed to point to cygwin find cmd
before: for path in $(find dist/renew -name "*.js"); do
and after: for path in $(/usr/bin/find dist/renew -name "*.js"); do
Thaks to all that commented
I try to execute bower commands in a sh script that is run in the after-success phase o a travis build. I installed bower in the install phase:
install:
- npm install -g bower
[...]
after_success:
- if [ ${TRAVIS_PULL_REQUEST} = "false" ] && [ "$TRAVIS_BRANCH" = "master" ]; then
./my-script.sh;
fi
Unfortunately, if I call bower in the sh script it produces the following output:
./my-script.sh: line 30: ./node_modules/.bin/bower: No such file or directory
I do not know how to proceed to fix the error. Any help would be greatly appreciated, thank you already!
I had to call the script using
bash my-script.sh;
instead of
./my-script.sh;
Now everything is working fine.
I am using Jenkins to take a number of parameters, generate an ansible-playbook command and run it. My Jenkins server is also my Ansible server.
My shell says ::
echo $ESXi_IP
echo $VM_NAME
echo $NIC1_MAC
echo $NIC2_MAC
echo $NIC3_MAC
echo $NIC4_MAC
echo $ESXi_HOSTNAME
echo $PLAYBOOK
ansible-playbook $PLAYBOOK --extra-vars "esxi_ip=$ESXi_IP vm_name=$VM_NAME nic1_mac=$NIC1_MAC nic2_mac=$NIC2_MAC nic3_mac=$NIC3_MAC nic4_mac=$NIC4_MAC esxi_hostname=$ESXi_HOSTNAME"
When I run the Job, the output is ::
+ ansible-playbook /root/ansible/sc-ece.yaml --extra-vars 'esxi_ip=5.232.66.49 vm_name=JenkinsTest nic1_mac=00:50:C0:A8:01:02 nic2_mac=00:50:0A:A9:37:A5 nic3_mac=00:50:0A:FF:FE:4C nic4_mac=00:50:AC:10:01:65 esxi_hostname=tmolab13-14iamesxi4'
ERROR! the playbook: /root/ansible/sc-ece.yaml could not be found
The playbook path is correct. there is no issue in it at all.
What seems to be missing here ?
You are correct Matt & Dave. Permissions to the folder was an issue. Thanks !