Sonar can't read coverage xml report for python application - jenkins

I'm trying to analyse a python application in which I'm using pipenv to manage dependencies. The Jenkinsfile is as the following
stage('Install dependencies') {
steps {
echo('....Install dependencies & Create VirtualEnvironment ..')
sh 'pip3 install --user pipenv'
sh 'pipenv --rm || exit 0'
sh 'pipenv install --pre --dev'
}
}
stage('Unit & Integration tests') {
steps {
echo('.... Unit & Integration tests')
sh "pipenv run coverage run --source=application/src/jobs -m pytest -v --junit-xml=reports/report.xml ."
sh "pipenv run coverage report"
stash includes: '*', name: 'workspace'
stash includes: 'reports/*' , name: 'reports'
}
}
stage('Build & SonarQube analyses') {
agent { label 'node14' }
steps {
echo('Build & SonarQube analyses')
unstash 'reports'
sh '/opt/sonar-scanner/bin/sonar-scanner \
-Dsonar.projectKey=$sonarProjectKey \
-Dsonar.sources=./application/src/ \
-Dsonar.host.url=$sonarHost \
-Dsonar.login=$sonarApiKey \
-Dsonar.python.coverage.reportPaths=./reports/report.xml \
-X '
}
}
The coverage report is generated successfully and I can visualize it by running coverage report But Sonar isn't able to update the coverage
Any suggestion please

The problem was in the xml report format, I used converage xml -i to generate the report and it works
stage('Unit & Integration tests') {
steps {
echo('.... Unit & Integration tests')
sh "pipenv run coverage run --source=application/src/jobs -m pytest -v --junit-xml=reports/report.xml ."
sh "pipenv run coverage report"
sh "pipenv run coverage xml -i" // generate the report that will be parsed by Sonar
stash includes: 'coverage.xml' , name: 'coverage'
}
}
stage('Build & SonarQube analyses') {
agent { label 'node14' }
steps {
echo('Build & SonarQube analyses')
unstash 'coverage'
sh '/opt/sonar-scanner/bin/sonar-scanner \
-Dsonar.projectKey=$sonarProjectKey \
-Dsonar.sources=./application/src/ \
-Dsonar.host.url=$sonarHost \
-Dsonar.login=$sonarApiKey \
-Dsonar.python.coverage.reportPaths=coverage.xml \
-X '
}
}

Related

Executing gcloud command in Jenkins pipeline

I'm trying to run the gcloud command in a Jenkins declarative pipeline just like in the following example:
pipeline {
agent any
stages {
stage('Run gcloud version') {
steps {
sh 'gcloud --version'
}
}
}
}
I downloaded the "GCloud SDK Plugin" and configured it like this (in "Global Tool Configuration" for Jenkins):
but when I try to build the pipeline using the above Jenkinsfile, I'm getting a 'gcloud: not found' error in the pipeline.
I was able to run the command using the following Jenkinsfile:
pipeline {
agent any
stages {
stage('Run gcloud') {
steps {
withEnv(['GCLOUD_PATH=/var/jenkins_home/google-cloud-sdk/bin']) {
sh '$GCLOUD_PATH/gcloud --version'
}
}
}
}
}
Note: I'm running Jenkins in kubernetes, so first I had to install the gcloud sdk in the Jenkins pod
I am running Jenkins 2.176.2 in containers and the GCloud plugin was not able to install the SDK in the slave (agents) containers.
I used the docker file to install it when deploying the agents:
RUN echo "deb http://packages.cloud.google.com/apt cloud-sdk-stretch main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list \
&& curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - \
&& apt-get update -y && apt-get install google-cloud-sdk -y \
&& PATH=$PATH:/root/google-cloud-sdk/bin

How to fix script.sh: line 1: Builing...: not found problem?

I have used both jenkins/jenkins:latest and jenkinsci/blueocean:latest docker images with pipeline script from SCM settings.
General setting "GitHub project" was enabled with https://github.com/alamsarker/test
Now When I build. its shows the following error:
+ Builing...
/var/jenkins_home/workspace/pipeline-test#tmp/durable-2aac8cac/script.sh: line 1: Builing...: not found
Can you please to fix the issue?
I run docker by:
docker run \
-u root \
--rm \
-d \
-p 8080:8080 \
-p 50000:50000 \
-v jenkins-data:/var/jenkins_home \
-v /var/run/docker.sock:/var/run/docker.sock \
jenkinsci/blueocean
My Jenkinsfile is simple as follows:
pipeline {
agent any
stages {
stage('build') {
steps {
sh 'Builing...'
}
}
stage('Test') {
steps {
sh 'Testing...'
}
}
stage('Deploy') {
steps {
sh 'Deploying...'
}
}
}
}
the pipeline step sh is used to execute linux cmd. Building is not a valid linux cmd, that's why you get the error.
If you want to print out some word you can use step echo which is cross-platform or execute the linux cmd: echo via step sh, like sh 'echo Building...' which only work on linux-like agent.
pipeline {
agent any
stages {
stage('build') {
steps {
echo 'Builing...'
}
}
stage('Test') {
steps {
sh 'echo Testing...'
}
}
stage('Deploy') {
steps {
echo 'Deploying...'
}
}
}
}

Jenkins stash not stashing files

I need to stash 2 ear files to be unstashed in the next stage.
This is my code to stash the files.
steps {
sh 'chmod +x gradlew'
echo "Building tms-load and tms-loadRemote ear files"
script {
sh "./gradlew -PjdkHome=${env.JAVA_HOME} -PweblogicHome=${WEBLOGIC_PATH} -Penv=at2 buildAll"
}
stash includes: "./build/staging/deploy/tms/AT2/*", name: "tmsLoadEars"
}
There are 2 ear files in this directory.
The gradle successfully builds the files.
The ant echo shows this:
[ant:echo] Deploying to ear area
'build/staging/deploy/tms/AT2/tms-load'.
The file name is 'tms-load.ear'
When I try to unstash and deploy, I get the error that there were no files included in the stash.
This is my unstash code:
script {
println JAVA_BIN_PATH;
dir('.') {
unstash "tmsLoadEars"
}
sh '''
. ~/.bash_profile
pghAdminConsole="<server url set here>"
wlLevel="L0"
"${JAVA_BIN_PATH}"/java -Xms512M -Xmx512M -cp "${WEBLOGIC_PATH}"/server/lib/weblogic.jar weblogic.Deployer \
-debug -stage -remote -verbose -upload \
-source ./build/staging/deploy/tms/AT2/tms-load.ear \
-targets $cluster -adminurl t3://$pghAdminConsole \
-username <username here> -password <password here> -deploy
'''
}
What am I doing wrong?
Added stash to the gradle command:
steps {
sh 'chmod +x gradlew'
echo "Building tms-load and tms-loadRemote ear files"
script
{
sh "./gradlew -PjdkHome=${env.JAVA_HOME} -PweblogicHome=/opt/weblogic/wl12.1.3.0/wlserver -Penv=at2 buildAll"
}
stash name: 'loadEarL0', includes: '**/tms-load.ear'
stash name: 'loadRemoteEarL0', includes: '**/tms-loadRemote.ear'
}
Then unstashed in the next step:
unstash "loadEarL0"
Defined the source switch in the Deploy program like this:
-source ./build/staging/deploy/tms/AT2/app/tms-load.ear \

Cannot conditionally remove a directory inside the workspace using basic Jenkins pipeline steps

I am trying to remove the directory junit located in the workspace of my Jenkins job using scripted Pipeline which looks somewhat like this:
node {
stage('Build') {
checkout scm
app = docker.build("...")
}
stage('Test') {
app.withRun("--name = ${CONTAINER_ID} ...") {
// sh "mkdir -p junit"
// sh "rm -rf junit/"
dir "junit" {
deleteDir
}
sh "docker exec ${CONTAINER_ID} /bin/bash -c 'source venv/bin/activate && python run.py test -x junit'"
sh "docker cp ${CONTAINER_ID}:/home/foo/junit junit"
}
}
junit 'junit/*.xml'
}
However I am getting the following (red haring?) error, e.g.
java.lang.ClassCastException:
hudson.tasks.junit.pipeline.JUnitResultsStep.testResults expects class
java.lang.String but received class
org.jenkinsci.plugins.workflow.cps.CpsClosure2
However when I am using the shell steps:
sh "mkdir -p junit"
sh "rm -rf junit/"
It works as expected. What am I doing wrong?
Try to use parentheses:
dir ("junit") {
deleteDir()
}

Building Go app with "vendor" directory on Jenkins with Docker

I'm trying to set up a Jenkins Pipeline to build and deploy my first Go project using a Jenkinsfile and docker.image().inside . I can't figure out how to get go to pick up the dependencies in the vendor/ directory.
When I run the build, I get a bunch of errors:
+ goapp test ./...
src/dao/demo_dao.go:8:2: cannot find package "github.com/dgrijalva/jwt-go" in any of:
/usr/lib/go_appengine/goroot/src/github.com/dgrijalva/jwt-go (from $GOROOT)
/usr/lib/go_appengine/gopath/src/github.com/dgrijalva/jwt-go (from $GOPATH)
/workspace/src/github.com/dgrijalva/jwt-go
...why isn't it picking up the Vendor directory?
When I throw in some logging, it seems that after running sh "cd /workspace/src/bitbucket.org/nalbion/go-demo" the next sh command is still in the original ${WORKSPACE} directory. I really like the idea of the Jenkins file, but I can't find any decent documentation for it.
(Edit - there is decent documentation here but dir("/workspace/src/bitbucket.org/nalbion/go-demo") {} doesn't seem to work within docker.image().inside)
My Docker file resembles:
FROM golang:1.6.2
# Google's App Engine Go SDK
RUN wget https://storage.googleapis.com/appengine-sdks/featured/go_appengine_sdk_linux_amd64-1.9.40.zip -q -O go_appengine_sdk.zip && \
unzip -q go_appengine_sdk.zip -d /usr/lib/ && \
rm go_appengine_sdk.zip
ENV PATH /usr/lib/go_appengine:/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ENV GOPATH /usr/lib/go_appengine/gopath
# Add Jenkins user
RUN groupadd -g 132 jenkins && useradd -d "/var/jenkins_home" -u 122 -g 132 -m -s /bin/bash jenkins
And my Jenkinsfile:
node('docker') {
currentBuild.result = "SUCCESS"
try {
stage 'Checkout'
checkout scm
stage 'Build and Test'
env.WORKSPACE = pwd()
docker.image('nalbion/go-web-build:latest').inside(
"-v ${env.WORKSPACE}:/workspace/src/bitbucket.org/nalbion/go-demo " +
"-e GOPATH=/usr/lib/go_appengine/gopath:/workspace") {
// Debugging
sh 'echo GOPATH: $GOPATH'
sh "ls -al /workspace/src/bitbucket.org/nalbion/go-demo"
sh "cd /workspace/src/bitbucket.org/nalbion/go-demo"
sh "pwd"
sh "go vet ./src/..."
sh "goapp test ./..."
}
stage 'Deploy to DEV'
docker.image('nalbion/go-web-build').inside {
sh "goapp deploy --application go-demo --version v${v} app.yaml"
}
timeout(time:5, unit:'DAYS') {
input message:'Approve deployment?', submitter: 'qa'
}
stage 'Deploy to PROD'
docker.image('nalbion/go-web-build').inside {
sh "goapp deploy --application go-demo --version v${v} app.yaml"
}
} catch (err) {
currentBuild.result = "FAILURE"
// send notifications
throw err
}
}
I managed to get it working by including the cd in the same sh statement:
docker.image('nalbion/go-web-build:latest')
.inside("-v ${env.WORKSPACE}:/workspace/src/bitbucket.org/nalbion/go-demo " +
"-e GOPATH=/usr/lib/go_appengine/gopath:/workspace") {
sh """
cd /workspace/src/bitbucket.org/nalbion/go-demo
go vet ./src/...
goapp test ./...
"""
}

Resources