Travis CI inline Bash Script - travis-ci

I have the following Travis CI yml file snippet:
script:
- if [ $TRAVIS_BRANCH == "master" ]; then
sbt clean coverage test coverageReport docker:publishLocal;
fi
- if [ $TRAVIS_BRANCH == "/^develop-.*$/" ]; then
sbt clean coverage test coverageReport;
fi
It sort of works, but I could not understand the logs that it produces. It is sort of misleading. Here is the log from the build server:
[success] Total time: 22 s, completed Aug 8, 2017 5:29:28 PM
The command "if [ $TRAVIS_BRANCH == "master" ]; then sbt clean coverage test coverageReport docker:publishLocal; fi" exited with 0.
0.00s
$ if [ $TRAVIS_BRANCH == "/^develop-.*$/" ]; then sbt clean coverage test coverageReport; fi
The command "if [ $TRAVIS_BRANCH == "/^develop-.*$/" ]; then sbt clean coverage test coverageReport; fi" exited with 0.
How could anything that did not run exit with a code 0? I would have rather expected it to just say, it skipped that script. What do you guys think?

"if [ $TRAVIS_BRANCH == "master" ]; then sbt clean coverage test coverageReport docker:publishLocal; fi"
in its entirety is what travis uses as the command for the step. Even if the 'true' branch of the if statement is not executed, travis still executed a command successfully, hence the exit 0. If you want message for skipped configurations, you would have to add them yourself
- if [ $TRAVIS_BRANCH == "master" ]; then
sbt clean coverage test coverageReport docker:publishLocal;
else
echo "Skipped"
fi
Additionally, your second check is is maybe not doing what you want. Using the =~ operator enables Regex matching, == may only use glob pattern-matching, AFAIK.

Related

Jenkins run a shell command when build is failed as a post build action to run failed TestNG plan

I want to rerun, below shell script in the same project, once my build is i completed with errors, so I can rerun my failed test cases in testng-failed.xml.
FILE=./target/surefire-reports/testng-failed.xml
if test -f "$FILE"; then
echo "$FILE exists."
mvn clean test -Dhttp.proxyHost=proxy-dev.aws.skynet.com -Dhttp.proxyPort=8080 -Dproxy-dev.aws.skynet.com -Dhttps.proxyPort=8080 -Dbrowser="${Browser}" -Dbrowser_version="${browser_version}" -Dos_version="10" -Dos="Windows" -Dlocalrun="false" -Dtestxml=FILE
else
echo "$FILE is not there"
fi
But in Jenkins, post build section, I don't see an option to add Execute shell. Please help.
I installed post build plugin , but it's only show for maven projects and not for my free style project.
In free style project, No post steps option.
I managed a walkaround to do this without any plugins and still run in the Jenkins free style project. Add two execute shells and in first shell enter the below shell commands.
export M2_HOME=/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/Maven3
export PATH=$PATH:$M2_HOME/bin
mvn --version
cd CommonPagesStorage
mvn clean install -DskipTests
cd ..
cd MultiWebdriverDemo
mvn clean test -Dmaven.test.failure.ignore=true -Dsurefire.suiteXmlFiles=TestSuites/${TestPlanName}
echo "${TestPlanName}is ran"
#Dmaven.test.failure.ignore=true added this so even the script failed it doesn't mark as fail and mark it as unstable and continue.
In the second shell command ,conditionally checks the testng-failed.xml is exists if so run it to run the failed test cases.
cd MultiWebdriverDemo
export M2_HOME=/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/Maven3
export PATH=$PATH:$M2_HOME/bin
mvn --version
#this to print current directory.
echo "the second Script executed from: ${PWD}"
if [ -e ./target/surefire-reports/testng-failed.xml ]
then
echo "ok file found pass"
mvn test -Dmaven.test.failure.ignore=true -Dsurefire.suiteXmlFiles=target/surefire-reports/testng-failed.xml
else
echo "file not found passed in first attempt ..."
exit 0
#exit 0 means forcefully mark it as passed
fi
if [ -e ./target/surefire-reports/testng-failed.xml ]
then
echo "Rerun also failed exiting with buil state as failure "
exit 1
#exit 1 means forcefully mark it as failed
else
echo "file not found rerun is passed marking the build as passed ..."
exit 0
fi
S0 even in the second time, my webdriver test case is failed, it's failed build mark as a failure.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 16.199 s
[INFO] Finished at: 2021-08-12T14:05:36Z
[INFO] ------------------------------------------------------------------------
+ [ -e ./target/surefire-reports/testng-failed.xml ]
+ echo Rerun also failed exiting with buil state as failure
Rerun also failed exiting with buil state as failure
+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
TestNG Reports Processing: START
Looking for TestNG results report in workspace using pattern: **/testng-results.xml
Saving reports...
Processing '/var/jenkins_home/jobs/August-FreeStyle/builds/37/testng/testng-results.xml'
100.000000% of tests failed, which exceeded threshold of 0%. Marking build as UNSTABLE
TestNG Reports Processing: FINISH
Finished: FAILURE

How to define multiple project keys for quality gate in jenkins pipeline

I have a jenkins pipeline in which one stage is sonarqube analysis.I have a project and multiple subprojects in sonarqube.for ex projectkey=xyz.com subprojectkey=abc.xyz.com. My requirement is if quality gates didn't pass my build should fails . How i can define multiple project keys. I wrote a shell script to do the task but in that also I'm able to define only one project key.
PROJECTKEY="%teamcity.project.id%"
QGSTATUS=`curl -s -u SONAR_TOKEN: http://SONAR_URL:9000/api/qualitygates/project_status?projectKey=$PROJECTKEY | jq '.projectStatus.status' | tr -d '"'`
if [ "$QGSTATUS" = "OK" ]
then
exit 0
elif [ "$QGSTATUS" = "ERROR" ]
then
exit 1
fi

Running liquibase update from script in jenkins pipeline

I'm trying to set up a Jenkins pipeline which will run a liquibase update whenever something is pushed to the master branch. The liquibase runner plugin for Jenkins has a security risk and therefore, I can't install it and run liquibase updates from that.
My liquibase* file (the bash script) is in my repository at the following path
/repo/liquibase/liquibase/liquibase*
I've set up the pipeline to run the following shell script. NOTE: I have the command set to liquibase --help for test purposes, but normally I'd want to run an update command.
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase/liquibase
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase/liquibase/liquibase
export PATH=$PATH:/var/lib/jenkins/workspace/repo/liquibase/liquibase/jre/bin
cd liquibase
ls -ltr
chmod 755 liquibase/liquibase
chmod 755 liquibase/jre/bin/java.exe
liquibase --help
The liquibase --help command runs fine from the directory path /repo/liquibase in git bash. However, when I run it from Jenkins, I get the following error.
/var/lib/jenkins/workspace/Database_and_Repos/liquibase/liquibase/liquibase/jre/bin/java: No such file or directory
Build step 'Execute shell' marked build as failure
My liquibase file looks like this and it is the last line in the file that is causing the error.
#!/usr/bin/env bash
if [ ! -n "${LIQUIBASE_HOME+x}" ]; then
# echo "LIQUIBASE_HOME is not set."
## resolve links - $0 may be a symlink
PRG="$0"
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`"/$link"
fi
done
LIQUIBASE_HOME=`dirname "$PRG"`
# make it fully qualified
LIQUIBASE_HOME=`cd "$LIQUIBASE_HOME" && pwd`
# echo "Liquibase Home: $LIQUIBASE_HOME"
fi
# build classpath from all jars in lib
if [ -f /usr/bin/cygpath ]; then
CP=.
for i in "$LIQUIBASE_HOME"/liquibase*.jar; do
i=`cygpath --windows "$i"`
CP="$CP;$i"
done
for i in "$LIQUIBASE_HOME"/lib/*.jar; do
i=`cygpath --windows "$i"`
CP="$CP;$i"
done
else
if [[ $(uname) = MINGW* ]]; then
CP_SEPARATOR=";"
else
CP_SEPARATOR=":"
fi
CP=.
for i in "$LIQUIBASE_HOME"/liquibase*.jar; do
CP="$CP""$CP_SEPARATOR""$i"
done
CP="$CP""$CP_SEPARATOR""$LIQUIBASE_HOME/lib/"
for i in "$LIQUIBASE_HOME"/lib/*.jar; do
CP="$CP""$CP_SEPARATOR""$i"
done
fi
if [ -z "${JAVA_HOME}" ]; then
#JAVA_HOME not set, try to find a bundled version
if [ -d "${LIQUIBASE_HOME}/jre" ]; then
JAVA_HOME="$LIQUIBASE_HOME/jre"
elif [ -d "${LIQUIBASE_HOME}/.install4j/jre.bundle/Contents/Home" ]; then
JAVA_HOME="${LIQUIBASE_HOME}/.install4j/jre.bundle/Contents/Home"
fi
fi
if [ -z "${JAVA_HOME}" ]; then
JAVA_PATH="$(which java)"
if [ -z "${JAVA_PATH}" ]; then
echo "Cannot find java in your path. Install java or use the JAVA_HOME environment variable"
fi
else
#Use path in JAVA_HOME
JAVA_PATH="${JAVA_HOME}/bin/java"
fi
# add any JVM options here
JAVA_OPTS="${JAVA_OPTS-}"
"${JAVA_PATH}" -cp "$CP" $JAVA_OPTS liquibase.integration.commandline.Main ${1+"$#"}
Has anyone run into this problem with liquibase commands in Jenkins? I've been googling all day, but haven't found much similar to this exact issue. Any help in the right direction would be great.
(We're updating the Liquibase Runner plugin. We have a release that is being reviewed for the security issues by the Jenkins team now.)
The error message seems to say that your "Execute shell" command in your changelog is not working correctly. Maybe the command is not installed, maybe it's calling a script that is not on your build machine.
One way to explore this to add an "echo" of the "Execute shell" command prior. Also, I'd pass --logLeve=DEBUG to Liquibase to get a better idea on the command it's trying to run.
Thanks for using Liquibase and Jenkins! I'll be talking about it here next month: https://www.cloudbees.com/devops-world/.
You could use the liquibase-maven-plugin and just call the maven phase in the pipeline:
sh mvn resources:resources liquibase:update
As for me, it is the best decision. Follow the official documentation

How to not mark Jenkins job as FAILURE when pytest tests fail

I have a Jenkins setup with a pipeline that uses pytest to run some test suites. Sometimes a test fails and sometimes the test environment crashes (random HTTP timeout, external lib error, etc.). The job parses the XML test result but the build is marked as FAILURE as long as pytest returns non-zero.
I want Jenkins to get exit code zero from pytest even if there are failed tests but I also want other errors to be marked as failures. Are there any pytest options that can fix this? I found pytest-custom_exit_code but it can only suppress the empty test suite error. Maybe some Jenkins option or bash snippet?
A simplified version of my groovy pipeline:
pipeline {
stages {
stage ('Building application') {
steps {
sh "./build.sh"
}
}
stage ('Testing application') {
steps {
print('Running pytest')
sh "cd tests && python -m pytest"
}
post {
always {
archiveArtifacts artifacts: 'tests/output/'
junit 'tests/output/report.xml'
}
}
}
}
}
I have tried to catch exit code 1 (meaning some tests failed) but Jenkins still received exit code 1 and marks the build as FAILURE:
sh "cd tests && (python -m pytest; rc=\$?; if [ \$rc -eq 1 ]; then exit 0; else exit \$rc; fi)"
Your attempt does not work because Jenkins runs the shell with the errexit (-e) option enabled and that causes the shell to exit right after the pytest command before it reaches your if statement. There is a one-liner however that will work because it is executed as one statement: https://stackoverflow.com/a/31114992/1070890
So your build step would look like this:
sh 'cd tests && python -m pytest || [[ $? -eq 1 ]]'
My solution was to implement the support myself in pytest-custom-exit-code and create a pull request.
From version 0.3.0 of the plugin I can use pytest --suppress-tests-failed-exit-code to get the desired behavior.

Jenkins build runs script that exit 1 but it doesn't mark the build as a failure

In Jenkins configuration, I have the following code:
sh example.sh test
example.sh has the following code:
aws s3 cp ./Bucket $S3BUCKET --recursive --content-type "video/mp4" --cache-control "max-age=$MAXAGE" --exclude "*" --include "*.mp4" --profile $PROFILE
if [ "$?" -ne 0 ]; then
exit 1
fi
When I look at the console output, there is an error running the command above, but it doesn't exit 1 and mark the build as a failure. I see the error text "Could not deploy files to S3 bucket" but the build isn't marked as a failure. So exit 1 doesn't run.
Why is that happenning?
Use Log Parser Plugin to check if the Could not deploy files to S3 bucket text is in the output and fail the build if so

Resources