I configure Jenkins to work with sonarqube scanner. The scan are working fine. The jenkins pipeline is working and I don't have any isssue in the jenkins log.
SonarQube Scanner 3.0.3.778
Jenkins: 2.70
SonarQube Scanner for Jenkins plugin: 2.6.1
I use this code:
stage('SonarQube analysis') {
sh 'sed -ie "s|_PROJECT_|${PROJECT_CODE}|g" $WORKSPACE/_pipeline/sonar-project.properties'
// requires SonarQube Scanner 3.0+
def scannerHome = '/opt/sonar/bin/sonar-scanner';
withSonarQubeEnv('mscodeanalysis') {
sh "${scannerHome}/bin/sonar-scanner -Dproject.settings=$WORKSPACE/_pipeline/sonar-project.properties"
}
}
}
}
}
}
// No need to occupy a node
stage("Quality Gate"){
timeout(time: 15, unit: 'MINUTES') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
My problem come from Quality Gate. It never POST the json payload to jenkins. I don't see a json entry inside jenkins log. But I know the connection between jenkins and sonarqube server is working because I was able to send a POST using curl from the sonarqube VM.
Here the jenkins job output:
Timeout set to expire in 15 min
[Pipeline] {
[Pipeline] waitForQualityGate
Checking status of SonarQube task 'AV3irVJXpvBxXXNJYZkd' on server 'mscodeanalysis'
SonarQube task 'AV3irVJXpvBxXXNJYZkd' status is 'PENDING'
Cancelling nested steps due to timeout
Here is my payload that never reach jenkins pipeline:
url: http://sonar-server:9000/api/ce/task?id=AV3irVJXpvBxXXNJYZkd
{"task":{"organization":"default-organization","id":"AV3irVJXpvBxXXNJYZkd","type":"REPORT","componentId":"AV3hrJeCfL_nrF2072FH","componentKey":"POOL-003","componentName":"POOL-003","componentQualifier":"TRK","analysisId":"AV3irVkZszLEB6PsCK9X","status":"SUCCESS","submittedAt":"2017-08-14T21:36:35+0000","submitterLogin":"jenkins","startedAt":"2017-08-14T21:36:37+0000","executedAt":"2017-08-14T21:36:38+0000","executionTimeMs":650,"logs":false,"hasScannerContext":true}}
I can't insert image but the Quality gate is Pass and the analysis task is success.
Let me know if I need to include more information.
Thank you
The issue could be that Jenkins is using https with self-signed certificate. Then solution is:
Generate truststore for SonarQube:
keytool -import -trustcacerts -alias jenkins-host-name -file cert.crt -keystore sonarqube.jks
keystore passw: password
Where cert.crt - is certificate used for ssl for jenkins, jenkins-host-name - is a hostname of jenkins in the docker network (which is used in webhook)
Add truststore to SonarQube Dockerfile:
FROM sonarqube
COPY sonarqube.jks /var/sonar_cert/
COPY sonar.properties /opt/sonarqube/conf/sonar.properties
Update sonar.properties
sonar.ce.javaAdditionalOpts=-Djavax.net.ssl.trustStore=/var/sonar_cert/sonarqube.jks -Djavax.net.ssl.trustStorePassword=password
Then if you have a correct user and password for Jenkins provided in webhook URL everything should work.
Tried: Jenkins 2.107.2, SonarQube 7.1
Here is a quick example of what we did to resolve this issue:
SonarQube randomly hangs at "pending" state. Telling it to retry refreshes it. We set it to 10 seconds in this example
maxRetry = 200
forloop (i=0; i<maxRetry; i++){
try {
timeout(time: 10, unit: 'SECONDS') {
waitForQualityGate()
}
} catch(Exception e) {
if (i == maxRetry-1) {
throw e
}
}
}
Was surprised to find that #Katone Vi's answer worked so well. Based on their answer we added a quick exit on success and used the DSL for the original request:
stage('SonarQube') {
steps {
withSonarQubeEnv('SonarQube') {
sh """
${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=XXX_${env.STAGE}_lambda
"""
}
script {
Integer waitSeconds = 10
Integer timeOutMinutes = 10
Integer maxRetry = (timeOutMinutes * 60) / waitSeconds as Integer
for (Integer i = 0; i < maxRetry; i++) {
try {
timeout(time: waitSeconds, unit: 'SECONDS') {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Sonar quality gate status: ${qg.status}"
} else {
i = maxRetry
}
}
} catch (Throwable e) {
if (i == maxRetry - 1) {
throw e
}
}
}
}
}
}
If you have configured SonarQube to use a HTTP(S) proxy, make sure that your jenkins is either reachable through the proxy or is configured as a "non-proxy host". This can be done with the http.nonProxyHosts property or HTTP_NONPROXYHOSTS environment variable. See also the documentation for further information and syntax.
If you are using Jenkinsfile, this is workaround:
define creadentials:
environment {
CRED = credentials('jenkins_user_pass')
}
then use:
stage("Quality Gate") {
steps {
script {
while(true){
sh "sleep 2"
def url="http://jenkinsURL/job/${env.JOB_NAME.replaceAll('/','/job/')}/lastBuild/consoleText";
def sonarId = sh script: "wget -qO- --content-on-error --no-proxy --auth-no-challenge --http-user=${CRED_USR} --http-password=${CRED_PSW} '${url}' | grep 'More about the report processing' | head -n1 ",returnStdout:true
sonarId = sonarId.substring(sonarId.indexOf("=")+1)
echo "sonarId ${sonarId}"
def sonarUrl = "http://jenkinsURL/sonar/api/ce/task?id=${sonarId}"
def sonarStatus = sh script: "wget -qO- '${sonarUrl}' --no-proxy --content-on-error | jq -r '.task' | jq -r '.status' ",returnStdout:true
echo "Sonar status ... ${sonarStatus}"
if(sonarStatus.trim() == "SUCCESS"){
echo "BREAK";
break;
}
if(sonarStatus.trim() == "FAILED "){
echo "FAILED"
currentBuild.result = 'FAILED'
break;
}
}
}
}
}
I have faced similar issue, while the quality Gate back-end activities in Sonar server takes less than 20 sec to complete its analysis.But quality gate fail/success response from sonar-webhook in jenkins job take lot of time and stuck.
stage('Sonar:QG') {
steps {
**sleep(10) /* Added 10 sec sleep that was suggested in few places*/**
script{
timeout(time: 10, unit: 'MINUTES') {
def qg = waitForQualityGate abortPipeline: true
if (qg.status != 'OK') {
echo "Status: ${qg.status}"
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
}
}
Essentially check below thing:-
Webhook is configured in sonar or not :- SonarQube -> Administration -> Webhooks
http://:/sonarqube-webhook/
or Use localhost in place of IP in http://locahlhost:port/sonarqube-webhook/ solves issue in my case.
I'm made more simple decision but it work a same
stage("Quality gate") {
steps {
retry(3){
waitForQualityGate abortPipeline: true
}
}
}
Adding a sh 'sleep 10' between stage('SonarQube analysis') AND stage("Quality Gate") fix the issue. Now the jenkins job receive
Checking status of SonarQube task 'AV3rHxhp3io6giaQF_OA' on server 'sonarserver'
SonarQube task 'AV3rHxhp3io6giaQF_OA' status is 'SUCCESS'
SonarQube task 'AV3rHxhp3io6giaQF_OA' completed. Quality gate is 'OK'
Related
I am using latest Jenkins in my Linux Box. I am trying to create a pipline with script block like below;
pipeline {
agent any
stages {
stage('TestStage') {
steps {
script {
sh "testfile.sh"
}
}
}
}
}
testfile.sh will return a json text like below;
{
"Worker": [
{
"Status": "running"
}
]
}
The Status can be either running or success or failure. If it is running, the code has to call testfile.sh again and check the status. If its success, pipeline has to continue to next step and if it is failure, pipeline has to terminate. Is it possible to achieve this?
Thanks.
You can achieve by creating a while loop that executes your script, reads the output and checks its value until a given timeout, when the status is no longer running you can check the result and fail the build according to the final status.
Something like:
pipeline {
agent any
stages {
stage('TestStage') {
steps {
script {
timeout(time: 1, unit: 'HOURS') { // timeout for the 'running' period
def status = sh script: 'testfile.sh', returnStdout: true
while (status == 'running') {
sleep time: 10, unit: 'SECONDS' // sleep between iterations
def output = sh script: 'testfile.sh', returnStdout: true
def dict = readJSON text: output
status = dict.Worker.Status
}
}
if(status == 'failure'){
error "Operation has ended with status 'failure'"
}
}
}
}
}
}
You can find more info on the relevant steps: sleep, timeout, error and readJSON.
I am running a SonarQube - Jenkins integration.
I need to achieve the following:
Link each pipeline build with that respective SonarQube analysis.
i.e
Build A - Passes Quality Gate
Build B - Fails Quality Gate
If I click on the SonarQube link associated with build A - It directs to the SonarQube dashboard showing a failure.
If I click on the SonarQube link associated with build B - It directs to the SonarQube dashboard showing a success.
I have tried the following:
sonar.projectVersion = ${env.BUILD_NUMBER}
This just tells with which analysis version to compare the latest analysis.
How can I achieve a direct link with the SonarQube dashboard and that particular build number in a Jenkins script pipeline?
I understand you want read Sonar Quality Gate in your Jenkins job ?
Sonar Quality gate will generate a "report-task.txt" file once analysis completed.
Depending on your project size, it might take a while.
Then, using Sonar rest api, you can get json status and use it.
Here is a copy/paste sample I'm using :
myEcho('DEBUG', "I want to wait for SONAR completion")
if (fileExists('target/sonar/report-task.txt')) {
//echo sh(returnStdout: true, script: 'cat target/sonar/report-task.txt')
def reportTak = readProperties(file: "target/sonar/report-task.txt")
def countLoop = 0
def countMax = 10
def sonarGateIsDone = false
// wait at least 1 second before asking for Sonar feedback
sleep(time: 3000, unit: 'MILLISECONDS')
while (!sonarGateIsDone && (countLoop <= countMax)) {
countLoop++
// loop while status is over OR timeout...
echo sh(returnStdout: true, script: "curl ${SONAR_URL}api/ce/task?id=${reportTak.ceTaskId} -o target/sonar/output.json")
if (fileExists('target/sonar/output.json')) {
def outputJson = readJSON(file: 'target/sonar/output.json')
if ("${outputJson.task.status}" == 'SUCCESS' || "${outputJson.task.status}" == 'CANCELED' || "${outputJson.task.status}" == 'FAILED') {
myEcho('INFO', "Sonar Gate internal analysis finished [${outputJson.task.status}]")
if ("${outputJson.task.status}" == 'SUCCESS') {
// internal process done, lets check Gate status
echo sh(returnStdout: true, script: "curl ${SONAR_URL}api/qualitygates/project_status?analysisId=${outputJson.task.analysisId} -o target/sonar/sonarGate.json")
if (fileExists('target/sonar/sonarGate.json')) {
def sonarGateJson = readJSON(file: 'target/sonar/sonarGate.json')
if ("${sonarGateJson.projectStatus.status}" == 'OK') {
// Gate is OK
myEcho('INFO', "Sonar gate is OK : status=[${sonarGateJson.projectStatus.status}]")
sonarGateIsDone = true
} else {
// Gate is NOK
myEcho('WARN', "Sonar gate is NOK : status=[${sonarGateJson.projectStatus.status}]")
sonarGateIsDone = true
}
} else {
// cannot find SonarGate.json ?!
myEcho('FAIL', 'Sonar gate check failed : cannot find [target/sonar/sonarGate.json]')
}
} else {
myEcho('WARN', "Sonar gate check is [${outputJson.task.status}]...")
}
} else {
// Sonar internal analysis isnt over, keep on going
myEcho('INFO', "Sonar Gate internal analysis still ongoing, wait a little... [${outputJson.task.status}]")
}
} else {
myEcho('FAIL', 'Sonar gate check failed : cannot find [target/sonar/output.json]')
}
// reaching here might probably be for Sonar to get time to make it...
if (!sonarGateIsDone) {
sleep(time: 1000, unit: 'MILLISECONDS')
}
} // while loop
if (!sonarGateIsDone) {
myEcho('WARN', "Waiting too long for completion... gave up !")
}
} else {
myEcho('INFO', "target/sonar/report-task.txt DOES NOT EXISTS")
}
Hope this helps.
Regards
I am executing in a pipeline a command to pass the sound of a project, what I need is that, just like in a normal job the sonar link remains once the job is executed, the same happens in the pipeline, now that when I run it in the pipeline, the SonarQube link is not saved, so I have the steps in groovy:
stage ('QA'){
steps {
echo 'executing sonar'
bat 'mvn sonar:sonar -Dsonar.host.url='+env.SONAR_URL+' -
Dsonar.projectName=QA:%JOB_BASE_NAME% -
Dsonar.projectKey=QA:%JOB_BASE_NAME%'
}
}
stage("Quality Gate"){
timeout(time: 1, unit: 'HOURS') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
Try replacing with:
stage ('QA'){
steps {
echo 'Running SonarQube..'
withSonarQubeEnv('XXXXXXXX') {
bat 'mvn sonar:sonar -Dsonar.host.url='+env.SONAR_URL+' -
Dsonar.projectName=QA:%JOB_BASE_NAME% -
Dsonar.projectKey=QA:%JOB_BASE_NAME%'
timeout(time: 1, unit: 'HOURS') {
script {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Pipeline aborted due to a quality gate failure: ${qg.status}"
}
}
}
}
}
}
Note that the XXXXXXXX should be replaced with the name of a Sonar config you've entered in Jenkins under Manage Jenkins>Configure System>SonarQube servers
i've installed and configure a SonanarQube Qualuity Gates plugin and successfully configured it in a non-pipline job. But when u try to configure it in pipline i face an different errors, when i try to make pipline like this:
stage ("SonarQube analysis") {
dir('source') {
def scannerHome = tool 'sonscanner';
withSonarQubeEnv('sonar'){
timeout(time: 1, unit: 'HOURS');
withSonarQubeEnv('sonar'){
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
}
}
i see this:
java.lang.IllegalStateException: There is no body to invoke
at org.jenkinsci.plugins.workflow.cps.CpsStepContext.newBodyInvoker(CpsStepContext.java:283)
at org.jenkinsci.plugins.workflow.cps.CpsStepContext.newBodyInvoker(CpsStepContext.java:95)
at org.jenkinsci.plugins.workflow.steps.TimeoutStepExecution.start(TimeoutStepExecution.java:46)
When as separate step after SonarQube analytics :
stage("Quality Gate"){
timeout(time: 1, unit: 'HOURS') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {error "Pipeline aborted due to quality gate failure: ${qg.status}"
} else {
echo 'Quality Gate PASSED'
}
}
}
Than
java.lang.IllegalStateException: Unable to get SonarQube task id and/or server name. Please use the 'withSonarQubeEnv' wrapper to run your analysis.
at org.sonarsource.scanner.jenkins.pipeline.WaitForQualityGateStep$Execution.processStepParameters(WaitForQualityGateStep.java:125)
at org.sonarsource.scanner.jenkins.pipeline.WaitForQualityGateStep$Execution.start(WaitForQualityGateStep.java:107)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:224)
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:150)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108)
And when pipline :
stage ("SonarQube analysis") {
dir('source') {
def scannerHome = tool 'sonscanner';
withSonarQubeEnv('sonar'){
sh "${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=android -Dsonar.sources='${WORKSPACE}'/source -Dsonar.language=java -Dsonar.projectBaseDir='${WORKSPACE}'/source -Dsonar.java.binaries='${WORKSPACE}'/source/app/build/intermediates/classes/debug -Dsonar.junit.reportsPath='${WORKSPACE}'/sourceapp/build/test-results/testDebugUnitTest"
}
}
}
stage("Quality Gate"){
timeout(time: 1, unit: 'HOURS') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {error "Pipeline aborted due to quality gate failure: ${qg.status}"
} else {
echo 'Quality Gate PASSED'
}
}
}
This:
Caused: java.lang.IllegalStateException: Unable to parse response from https://sonar//api/ce/task?id=AWCYPE3v1VuUrqmUzLyR:
Unable to parse response from https://sonar//api/ce/task?id=AWCYPE3v1VuUrqmUzLyR:
Note the double // in the url. You can fix this easily, by removing the trailing slash in the configuration of the plugin on Jenkins.
In Jenkins:
Manage Jenkins
Configure System
Scrol to the section called SonarQube servers
Remove the trailing \ from the Server URL ;)
It should look something like this:
https://sonarcloud.io
NOT
https://sonarcloud.io/
Within my Jenkins Pipeline I need to react on the SonarQube Quality Gate.
Is there an easier way to achieve this but looking in the Sonar-Scanner log for the result page (e.g. https://mysonarserver/sonar/api/ce/task?id=xxxx) and parse the JSON Result from there?
I use Jenkins 2.30 and SonarQube 5.3
Thanks in advance
Based on Vincent's answer, and using Pipeline utility steps, here's my updated version that worked for me (using sonarscanner report file) :
withSonarQubeEnv('SONAR 6.4') {
sh "${scannerHome}/bin/sonar-scanner"
sh "cat .scannerwork/report-task.txt"
def props = readProperties file: '.scannerwork/report-task.txt'
echo "properties=${props}"
def sonarServerUrl=props['serverUrl']
def ceTaskUrl= props['ceTaskUrl']
def ceTask
timeout(time: 1, unit: 'MINUTES') {
waitUntil {
def response = httpRequest ceTaskUrl
ceTask = readJSON text: response.content
echo ceTask.toString()
return "SUCCESS".equals(ceTask["task"]["status"])
}
}
def response2 = httpRequest url : sonarServerUrl + "/api/qualitygates/project_status?analysisId=" + ceTask["task"]["analysisId"], authentication: 'jenkins_scanner'
def qualitygate = readJSON text: response2.content
echo qualitygate.toString()
if ("ERROR".equals(qualitygate["projectStatus"]["status"])) {
error "Quality Gate failure"
}
}
Please note the use of a Jenkins Credentials (authentication: 'jenkins_scanner') to retrieve the quality gate in Sonar being auhtenticated.
Using SonarQube Scanner for Jenkins 2.8.1 the solution is available out of the Box:
stage('SonarQube analysis') {
withSonarQubeEnv('My SonarQube Server') {
sh 'mvn clean package sonar:sonar'
} // SonarQube taskId is automatically attached to the pipeline context
}
}
stage("Quality Gate"){
timeout(time: 1, unit: 'HOURS') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
Scan first:
node("sonar") {
deleteDir()
unstash 'sources'
def scannerHome = tool 'sonar-scanner';
withSonarQubeEnv('sonarqube-rec') {
withEnv(["JAVA_HOME=${ tool 'JDK_8.0' }", "PATH+MAVEN=${tool 'M325'}/bin:${env.JAVA_HOME}/bin"]) {
// requires SonarQube Scanner for Maven 3.2+
sh '''
mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar
echo "SONAR_AUTH_TOKEN=$SONAR_AUTH_TOKEN" >> target/sonar/report-task.txt
'''
stash includes: "target/sonar/report-task.txt", name: 'sonar-report-task'
}
}
}
then check the quality gate:
stage("Quality Gate"){
node("sonar") {
deleteDir()
unstash 'sonar-report-task'
def props = utils.getProperties("target/sonar/report-task.txt")
echo "properties=${props}"
def sonarServerUrl=props.getProperty('serverUrl')
def ceTaskUrl= props.getProperty('ceTaskUrl')
def ceTask
def URL url = new URL(ceTaskUrl)
timeout(time: 1, unit: 'MINUTES') {
waitUntil {
ceTask = utils.jsonParse(url)
echo ceTask.toString()
return "SUCCESS".equals(ceTask["task"]["status"])
}
}
url = new URL(sonarServerUrl + "/api/qualitygates/project_status?analysisId=" + ceTask["task"]["analysisId"] )
def qualitygate = utils.jsonParse(url)
echo qualitygate.toString()
if ("ERROR".equals(qualitygate["projectStatus"]["status"])) {
error "Quality Gate failure"
}
}
}
I used ".sonar/report-task.txt" to retrieve the ceTaskUrl - Then I used Pipeline Shared Libraries and wrote my own Pipeline Function to retrieve the quality gate.
http://mySonarQube.com:9001/api/ce/task?id="ceTaskUrl"
Parse "task.analysisId"
Parse quality-gates from http://mySonarQube.com:9001/api/qualitygates/project_status?analysisId="task.analysisId"