Within my Jenkins Pipeline I need to react on the SonarQube Quality Gate.
Is there an easier way to achieve this but looking in the Sonar-Scanner log for the result page (e.g. https://mysonarserver/sonar/api/ce/task?id=xxxx) and parse the JSON Result from there?
I use Jenkins 2.30 and SonarQube 5.3
Thanks in advance
Based on Vincent's answer, and using Pipeline utility steps, here's my updated version that worked for me (using sonarscanner report file) :
withSonarQubeEnv('SONAR 6.4') {
sh "${scannerHome}/bin/sonar-scanner"
sh "cat .scannerwork/report-task.txt"
def props = readProperties file: '.scannerwork/report-task.txt'
echo "properties=${props}"
def sonarServerUrl=props['serverUrl']
def ceTaskUrl= props['ceTaskUrl']
def ceTask
timeout(time: 1, unit: 'MINUTES') {
waitUntil {
def response = httpRequest ceTaskUrl
ceTask = readJSON text: response.content
echo ceTask.toString()
return "SUCCESS".equals(ceTask["task"]["status"])
}
}
def response2 = httpRequest url : sonarServerUrl + "/api/qualitygates/project_status?analysisId=" + ceTask["task"]["analysisId"], authentication: 'jenkins_scanner'
def qualitygate = readJSON text: response2.content
echo qualitygate.toString()
if ("ERROR".equals(qualitygate["projectStatus"]["status"])) {
error "Quality Gate failure"
}
}
Please note the use of a Jenkins Credentials (authentication: 'jenkins_scanner') to retrieve the quality gate in Sonar being auhtenticated.
Using SonarQube Scanner for Jenkins 2.8.1 the solution is available out of the Box:
stage('SonarQube analysis') {
withSonarQubeEnv('My SonarQube Server') {
sh 'mvn clean package sonar:sonar'
} // SonarQube taskId is automatically attached to the pipeline context
}
}
stage("Quality Gate"){
timeout(time: 1, unit: 'HOURS') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
Scan first:
node("sonar") {
deleteDir()
unstash 'sources'
def scannerHome = tool 'sonar-scanner';
withSonarQubeEnv('sonarqube-rec') {
withEnv(["JAVA_HOME=${ tool 'JDK_8.0' }", "PATH+MAVEN=${tool 'M325'}/bin:${env.JAVA_HOME}/bin"]) {
// requires SonarQube Scanner for Maven 3.2+
sh '''
mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar
echo "SONAR_AUTH_TOKEN=$SONAR_AUTH_TOKEN" >> target/sonar/report-task.txt
'''
stash includes: "target/sonar/report-task.txt", name: 'sonar-report-task'
}
}
}
then check the quality gate:
stage("Quality Gate"){
node("sonar") {
deleteDir()
unstash 'sonar-report-task'
def props = utils.getProperties("target/sonar/report-task.txt")
echo "properties=${props}"
def sonarServerUrl=props.getProperty('serverUrl')
def ceTaskUrl= props.getProperty('ceTaskUrl')
def ceTask
def URL url = new URL(ceTaskUrl)
timeout(time: 1, unit: 'MINUTES') {
waitUntil {
ceTask = utils.jsonParse(url)
echo ceTask.toString()
return "SUCCESS".equals(ceTask["task"]["status"])
}
}
url = new URL(sonarServerUrl + "/api/qualitygates/project_status?analysisId=" + ceTask["task"]["analysisId"] )
def qualitygate = utils.jsonParse(url)
echo qualitygate.toString()
if ("ERROR".equals(qualitygate["projectStatus"]["status"])) {
error "Quality Gate failure"
}
}
}
I used ".sonar/report-task.txt" to retrieve the ceTaskUrl - Then I used Pipeline Shared Libraries and wrote my own Pipeline Function to retrieve the quality gate.
http://mySonarQube.com:9001/api/ce/task?id="ceTaskUrl"
Parse "task.analysisId"
Parse quality-gates from http://mySonarQube.com:9001/api/qualitygates/project_status?analysisId="task.analysisId"
Related
I have intergrated Sonarqube in Jenkins Pipeline, it working as below define:
stage('static check') {
steps {
echo "starting codeAnalyze with SonarQube......"
//sonar:sonar.QualityGate should pass
withSonarQubeEnv('sonar') {
sh "mvn -f pom.xml clean compile sonar:sonar -U"
}
script {
timeout(10) {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
echo "failure: ${qg.status}"
# Plan to send notify to Slack, But can NOT to obtain the Sonar report URL or TaskId
}
}
}
}
}
Plan to send corresponding notify to Slack, But can NOT to obtain the Sonar report URL or TaskId
And found some guys suggest that use BUILD_LOG_REGEX: SonarQube result URL: ${BUILD_LOG_REGEX, regex=".*ANALYSIS SUCCESSFUL, you can browse (.*)", showTruncatedLines=false, substText="$1"}, but this only working in Email Extension Plugin.
Another suggest is compose the report URL by this: ProjectURL = host + "/dashboard?id=" + ProjectID, and we not easy to get the ProjectID, since the projectID is compose from Pom.xml as: groupId:artifactId.
Maybe there is some easy way to get the Sonar TaskId or Report URL, Especially whether there is the waitForQualityGate() or other method will return the Sonar TaskId or Report URL ?
You can see the report by reading the file ./target/sonar/report-task.txt
Using def getURL = readProperties file: './target/sonar/report-task.txt'
and extract by calling something like this ${getURL['dashboardUrl']} which is will give url of report. also that above file have taskId.
so it should like this
def qg = waitForQualityGate()
if (qg.status != 'OK') {
echo "failure: ${qg.status}"
def getURL = readProperties file: './target/sonar/report-task.txt
Add your slackmsg here use in that ${getURL['dashboardUrl']}
}
For windows, use
./.scannerwork/report-task.txt
Eg
waitForQualityGate abortPipeline: true
def getURL = readProperties file: './.scannerwork/report-task.txt'
sonarqubeURL = "${getURL['dashboardUrl']}"
echo "${sonarqubeURL }"
I am executing in a pipeline a command to pass the sound of a project, what I need is that, just like in a normal job the sonar link remains once the job is executed, the same happens in the pipeline, now that when I run it in the pipeline, the SonarQube link is not saved, so I have the steps in groovy:
stage ('QA'){
steps {
echo 'executing sonar'
bat 'mvn sonar:sonar -Dsonar.host.url='+env.SONAR_URL+' -
Dsonar.projectName=QA:%JOB_BASE_NAME% -
Dsonar.projectKey=QA:%JOB_BASE_NAME%'
}
}
stage("Quality Gate"){
timeout(time: 1, unit: 'HOURS') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
Try replacing with:
stage ('QA'){
steps {
echo 'Running SonarQube..'
withSonarQubeEnv('XXXXXXXX') {
bat 'mvn sonar:sonar -Dsonar.host.url='+env.SONAR_URL+' -
Dsonar.projectName=QA:%JOB_BASE_NAME% -
Dsonar.projectKey=QA:%JOB_BASE_NAME%'
timeout(time: 1, unit: 'HOURS') {
script {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Pipeline aborted due to a quality gate failure: ${qg.status}"
}
}
}
}
}
}
Note that the XXXXXXXX should be replaced with the name of a Sonar config you've entered in Jenkins under Manage Jenkins>Configure System>SonarQube servers
I have problem with method waitForQualityGate(). I got an error "No such DSL method 'waitForQualityGate' found among steps". Another strange thing is that I must use parameter -DX for sonarscanner. I don't know what is wrong. Thanks for help.
pipeline {
agent { label 'builders' }
tools {
maven 'maven 3.3.9'
}
stages {
stage ('Checkout'){
steps {
git branch: 'develop', credentialsId: 'credential', url: 'ssh://repository'
}
}
stage ('Build'){
steps {
withMaven (
maven: 'maven 3.3.9',
mavenSettingsConfig: 'cc86690e-095d-4714-92b2-b61861241c7a'){
sh 'mvn -U clean package -DskipTests'
}
}
}
stage ('SonarQube Scan'){
steps {
withSonarQubeEnv('SonarQube') {
withMaven (
maven: 'maven 3.3.9',
mavenSettingsConfig: 'cc86690e-095d-4714-92b2-b61861241c7a'){
sh 'mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.3.0.603:sonar ' +
'-DX' +
'-Dsonar.login=login' +
'-Dsonar.password=password' +
'-Dsonar.issuesReport.json.enable=true' +
'-Dsonar.report.export.path=sonar-report.json'
}
} // SonarQube taskId is automatically attached to the pipeline context
}
}
stage ('Quality Gate') {
steps {
timeout(time: 1, unit: 'HOURS') { // Just in case something goes wrong, pipeline will be killed after a timeout
script {
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
}
}
}
}
This will not be an immediate answer, but the reason for the error is, you are calling a custom method without even loading it initially. Load the groovy file that have the method, which jenkins is complaining that dsl doesn't exist... only when you load the groovy file/the class, then you can instantiate it.
Cannot do this, without loading it ... def qg = waitForQualityGate()
If its a method, u have to call it and it should return something...
I configure Jenkins to work with sonarqube scanner. The scan are working fine. The jenkins pipeline is working and I don't have any isssue in the jenkins log.
SonarQube Scanner 3.0.3.778
Jenkins: 2.70
SonarQube Scanner for Jenkins plugin: 2.6.1
I use this code:
stage('SonarQube analysis') {
sh 'sed -ie "s|_PROJECT_|${PROJECT_CODE}|g" $WORKSPACE/_pipeline/sonar-project.properties'
// requires SonarQube Scanner 3.0+
def scannerHome = '/opt/sonar/bin/sonar-scanner';
withSonarQubeEnv('mscodeanalysis') {
sh "${scannerHome}/bin/sonar-scanner -Dproject.settings=$WORKSPACE/_pipeline/sonar-project.properties"
}
}
}
}
}
}
// No need to occupy a node
stage("Quality Gate"){
timeout(time: 15, unit: 'MINUTES') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
My problem come from Quality Gate. It never POST the json payload to jenkins. I don't see a json entry inside jenkins log. But I know the connection between jenkins and sonarqube server is working because I was able to send a POST using curl from the sonarqube VM.
Here the jenkins job output:
Timeout set to expire in 15 min
[Pipeline] {
[Pipeline] waitForQualityGate
Checking status of SonarQube task 'AV3irVJXpvBxXXNJYZkd' on server 'mscodeanalysis'
SonarQube task 'AV3irVJXpvBxXXNJYZkd' status is 'PENDING'
Cancelling nested steps due to timeout
Here is my payload that never reach jenkins pipeline:
url: http://sonar-server:9000/api/ce/task?id=AV3irVJXpvBxXXNJYZkd
{"task":{"organization":"default-organization","id":"AV3irVJXpvBxXXNJYZkd","type":"REPORT","componentId":"AV3hrJeCfL_nrF2072FH","componentKey":"POOL-003","componentName":"POOL-003","componentQualifier":"TRK","analysisId":"AV3irVkZszLEB6PsCK9X","status":"SUCCESS","submittedAt":"2017-08-14T21:36:35+0000","submitterLogin":"jenkins","startedAt":"2017-08-14T21:36:37+0000","executedAt":"2017-08-14T21:36:38+0000","executionTimeMs":650,"logs":false,"hasScannerContext":true}}
I can't insert image but the Quality gate is Pass and the analysis task is success.
Let me know if I need to include more information.
Thank you
The issue could be that Jenkins is using https with self-signed certificate. Then solution is:
Generate truststore for SonarQube:
keytool -import -trustcacerts -alias jenkins-host-name -file cert.crt -keystore sonarqube.jks
keystore passw: password
Where cert.crt - is certificate used for ssl for jenkins, jenkins-host-name - is a hostname of jenkins in the docker network (which is used in webhook)
Add truststore to SonarQube Dockerfile:
FROM sonarqube
COPY sonarqube.jks /var/sonar_cert/
COPY sonar.properties /opt/sonarqube/conf/sonar.properties
Update sonar.properties
sonar.ce.javaAdditionalOpts=-Djavax.net.ssl.trustStore=/var/sonar_cert/sonarqube.jks -Djavax.net.ssl.trustStorePassword=password
Then if you have a correct user and password for Jenkins provided in webhook URL everything should work.
Tried: Jenkins 2.107.2, SonarQube 7.1
Here is a quick example of what we did to resolve this issue:
SonarQube randomly hangs at "pending" state. Telling it to retry refreshes it. We set it to 10 seconds in this example
maxRetry = 200
forloop (i=0; i<maxRetry; i++){
try {
timeout(time: 10, unit: 'SECONDS') {
waitForQualityGate()
}
} catch(Exception e) {
if (i == maxRetry-1) {
throw e
}
}
}
Was surprised to find that #Katone Vi's answer worked so well. Based on their answer we added a quick exit on success and used the DSL for the original request:
stage('SonarQube') {
steps {
withSonarQubeEnv('SonarQube') {
sh """
${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=XXX_${env.STAGE}_lambda
"""
}
script {
Integer waitSeconds = 10
Integer timeOutMinutes = 10
Integer maxRetry = (timeOutMinutes * 60) / waitSeconds as Integer
for (Integer i = 0; i < maxRetry; i++) {
try {
timeout(time: waitSeconds, unit: 'SECONDS') {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Sonar quality gate status: ${qg.status}"
} else {
i = maxRetry
}
}
} catch (Throwable e) {
if (i == maxRetry - 1) {
throw e
}
}
}
}
}
}
If you have configured SonarQube to use a HTTP(S) proxy, make sure that your jenkins is either reachable through the proxy or is configured as a "non-proxy host". This can be done with the http.nonProxyHosts property or HTTP_NONPROXYHOSTS environment variable. See also the documentation for further information and syntax.
If you are using Jenkinsfile, this is workaround:
define creadentials:
environment {
CRED = credentials('jenkins_user_pass')
}
then use:
stage("Quality Gate") {
steps {
script {
while(true){
sh "sleep 2"
def url="http://jenkinsURL/job/${env.JOB_NAME.replaceAll('/','/job/')}/lastBuild/consoleText";
def sonarId = sh script: "wget -qO- --content-on-error --no-proxy --auth-no-challenge --http-user=${CRED_USR} --http-password=${CRED_PSW} '${url}' | grep 'More about the report processing' | head -n1 ",returnStdout:true
sonarId = sonarId.substring(sonarId.indexOf("=")+1)
echo "sonarId ${sonarId}"
def sonarUrl = "http://jenkinsURL/sonar/api/ce/task?id=${sonarId}"
def sonarStatus = sh script: "wget -qO- '${sonarUrl}' --no-proxy --content-on-error | jq -r '.task' | jq -r '.status' ",returnStdout:true
echo "Sonar status ... ${sonarStatus}"
if(sonarStatus.trim() == "SUCCESS"){
echo "BREAK";
break;
}
if(sonarStatus.trim() == "FAILED "){
echo "FAILED"
currentBuild.result = 'FAILED'
break;
}
}
}
}
}
I have faced similar issue, while the quality Gate back-end activities in Sonar server takes less than 20 sec to complete its analysis.But quality gate fail/success response from sonar-webhook in jenkins job take lot of time and stuck.
stage('Sonar:QG') {
steps {
**sleep(10) /* Added 10 sec sleep that was suggested in few places*/**
script{
timeout(time: 10, unit: 'MINUTES') {
def qg = waitForQualityGate abortPipeline: true
if (qg.status != 'OK') {
echo "Status: ${qg.status}"
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
}
}
Essentially check below thing:-
Webhook is configured in sonar or not :- SonarQube -> Administration -> Webhooks
http://:/sonarqube-webhook/
or Use localhost in place of IP in http://locahlhost:port/sonarqube-webhook/ solves issue in my case.
I'm made more simple decision but it work a same
stage("Quality gate") {
steps {
retry(3){
waitForQualityGate abortPipeline: true
}
}
}
Adding a sh 'sleep 10' between stage('SonarQube analysis') AND stage("Quality Gate") fix the issue. Now the jenkins job receive
Checking status of SonarQube task 'AV3rHxhp3io6giaQF_OA' on server 'sonarserver'
SonarQube task 'AV3rHxhp3io6giaQF_OA' status is 'SUCCESS'
SonarQube task 'AV3rHxhp3io6giaQF_OA' completed. Quality gate is 'OK'
Is there any way to break Jenkins build when Sonar quality gate fails with waitForQualityGate() method along with Sonar Scanner for MSbuild? I could not find any documentation for the same. All I could find is the usage of waitForQualityGate() along with Sonar scanner, but the general sonar scanner is not recommended for MSbuild projects.
The below mentioned link does not talk about usage waitForQualityGate with MSBuild.
https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+Jenkins#AnalyzingwithSonarQubeScannerforJenkins-AnalyzinginaJenkinspipeline
That documentation talks about Sonar Scanner, but I am referring to Sonar scanner for MSbuild which is a different scanner altogether. The way I use this scanner is as shown below.
void beginSonarMSBuild(String VERSION){
stage('Begin Sonar Analysis') {
def MSBuildScannerHome = tool 'sonar-scanner-msbuild-3.0.0.629';
withSonarQubeEnv('civil sonar') {
bat "${MSBuildScannerHome}\\SonarQube.Scanner.MSBuild.exe begin /k:mcdc
/n:mc-design-converter /v:${VERSION}.$BUILD_NUMBER /d:sonar.sourceEncoding=UTF-8
}
}
}
void build(){
stage ('Build'){
bat "Nuget restore SOMEHTING.sln"
bat "MSBuild.exe SOMETHING.csproj "
}
}
void endSonarMSBuild(){
stage ('Complete Sonar Analysis'){
def MSBuildScannerHome = tool 'sonar-scanner-msbuild-3.0.0.629';
bat "${MSBuildScannerHome}\\SonarQube.Scanner.MSBuild.exe end"
}
}
Now when I use waitforqualitygate() with beginSonarMSBuild(String VERSION)as shown below:
void beginSonarMSBuild(String VERSION){
stage('Begin Sonar Analysis') {
def MSBuildScannerHome = tool 'sonar-scanner-msbuild-3.0.0.629';
withSonarQubeEnv('civil sonar') {
bat "${MSBuildScannerHome}\\SonarQube.Scanner.MSBuild.exe begin /k:mcdc
/n:mc-design-converter /v:${VERSION}.$BUILD_NUMBER /d:sonar.sourceEncoding=UTF-8
}
}
stage("Quality Gate"){
timeout(time: 1, unit: 'MINUTES') {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
void build(){
scripts here...
}
void endSonarMSBuild(){
scripts here...
}
I get this error msg java.lang.IllegalStateException: Unable to get SonarQube task id and/or server name. Please use the 'withSonarQubeEnv' wrapper to run your analysis.
Also I get the same error when I use waitForQualityGate() with endSonarMSBuild() step as shown below.
void beginSonarMSBuild(String VERSION){
stage('Begin Sonar Analysis') {
scripts here...
}
void build(){
scripts here...
}
void endSonarMSBuild(){
stage ('Complete Sonar Analysis'){
def MSBuildScannerHome = tool 'sonar-scanner-msbuild-3.0.0.629';
bat "${MSBuildScannerHome}\\SonarQube.Scanner.MSBuild.exe end"
}
stage("Quality Gate"){
timeout(time: 1, unit: 'MINUTES') {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
}
So the question I have is, does Sonar scanner for MSBuild even support waitForQualityGate(), if yes, then how to use the same?
On the documentation, the example is made with the scanner for Maven but it should work fine with any scanner as long as you wrap it in a withSonarQubeEnv step.
For the scanner for MSBuild, it is important to wrap the end step (but wrapping the begin step is also a good idea to automatically pass credentials.
void beginSonarMSBuild(String VERSION) {
stage('Begin SonarQube Analysis') {
def MSBuildScannerHome = tool 'sonar-scanner-msbuild-3.0.0.629';
withSonarQubeEnv('civil sonar') {
bat "${MSBuildScannerHome}\\SonarQube.Scanner.MSBuild.exe begin /k:mcdc
/n:mc-design-converter /v:${VERSION}.$BUILD_NUMBER /d:sonar.sourceEncoding=UTF-8
}
}
}
void build() {
stage ('Build') {
bat "Nuget restore SOMEHTING.sln"
bat "MSBuild.exe SOMETHING.csproj"
}
}
void endSonarMSBuild() {
stage ('Complete SonarQube Analysis') {
withSonarQubeEnv('civil sonar') {
def MSBuildScannerHome = tool 'sonar-scanner-msbuild-3.0.0.629';
bat "${MSBuildScannerHome}\\SonarQube.Scanner.MSBuild.exe end"
} // Will collect task id
}
stage("Quality Gate"){
timeout(time: 1, unit: 'MINUTES') {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
}