I am trying to run sonar-scanner and access quality gate results and am kind of stuck after trying various options suggested on forums. This is my first time trying to post, so please let me know if I am missing any details. I do see the json payload in sonarqube server webhooks console but it is in failed status (red cross-mark). Ours is a shared CBJ and SonarQube server with limited access for me on both of those. Any help/guidance is really appreciated. Thank you so much.
======================================
SonarQube Configuration
Project_Name > Administration > Webhooks
Name: Webhook_Name
URL: https://CloudBeesJenkins_Server_FQDN/dev-master/sonarqube-webhook/
Secret: 'webhook_secret_text'
======================================
CBJ Configuration
CredentialsID: 'SonarQubeToken': Value: Scope: Global credentials (unrestricted)
======================================
Jenkins Job - Pipeline Script
/* this stage succeeds */
stage('SonarQube Analysis') {
def scannerHome = tool 'Sonar-Prod';
withSonarQubeEnv('Sonar-Prod') {
sh """${scannerHome}/bin/sonar-scanner -X \
-Dsonar.projectKey=ProjKey \
-Dsonar.sources=src \
-Dsonar.host.url=https://sonarqube_server_fqdn \
-Dsonar.login=sonar_project_secret_text"""
}
}
/* fails at waitForQualityGate */
stage("Quality Gate Status Check") {
timeout(time: 1, unit: 'HOURS')// Just in case something goes wrong, pipeline will be killed after a timeout
// had previously tried using waitForQualityGate() and waitForQualityGate(webhookSecretId: 'webhook_secret_text' with same result
def qg = waitForQualityGate(webhookSecretId: 'webhook_secret_text', credentialsId: 'sonar_project_secret_text') // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
=====================================
Logs from Jenkins Server - Job Running Sonarscanner and qualitygate
SonarQube Scanner 4.2.0.1873
Java 1.8.0_242 Oracle Corporation (64-bit)
Linux 2.6.32-754.27.1.el6.x86_64 amd64
SonarQube server 7.9.1 - Community 7.9.1.27448
[CloudBees Jenkins Enterprise 2.204.3.7-rolling]
09:40:13.671 DEBUG: Upload report
09:40:13.931 DEBUG: POST 200 https://sonarqube_server_fqdn/api/ce/submit?projectKey=ProjKey | time=256ms
09:40:13.935 INFO: Analysis report uploaded in 264ms
09:40:13.938 INFO: ANALYSIS SUCCESSFUL, you can browse https://sonarqube_server_fqdn/dashboard?id=ProjKey
09:40:13.938 INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
09:40:13.938 INFO: More about the report processing at https://sonarqube_server_fqdn/api/ce/task?id=AXDt34Wae-uSoUyAgrS-
[Pipeline] waitForQualityGate
Checking status of SonarQube task 'AXDt34Wae-uSoUyAgrS-' on server 'Sonar-Prod'
org.sonarqube.ws.client.HttpException: Error 401 on https://sonarqube_server_fqdn/api/ce/task?id=AXDt34Wae-uSoUyAgrS-
It was a firewall issue. Communication from Jenkins to SonarQube server was opened but not the other way round. This issue can be closed.
Related
We are using "fortify on-demand (FOD)" platform to scan our source code to find out any security vulnerabilities are present. We integrated the FOD with jenkins to automate the process of uploading and scanning. And we opted the pipeline script method for integration. All the process up to uploading and scanning is running fine and we are capturing policy scan status (passed or failed) also, but the pipeline script of fodPollResults is failing to fail the build when the FOD policy scan is failed. irrespective of the result of policy scan the build is getting success.
jenkins pipeline script
stage('FOD POLL') {
steps {
fodPollResults bsiToken: '', personalAccessToken: 'fortify_personal_access_token', policyFailureBuildResultPreference: 2, pollingInterval: 3, releaseId: '******', tenantId: '', username: ''
}
}
Fortify on Demand Poll Results
the source code of this plugin is located here:
https://github.com/jenkinsci/fortify-on-demand-uploader-plugin/blob/master/src/main/java/org/jenkinsci/plugins/fodupload/steps/FortifyPollResults.java
and there is a bug ticket about this problem here:
https://github.com/jenkinsci/fortify-on-demand-uploader-plugin/issues/118
Following workaround seems to work:
steps {
fodPollResults ...
script {
if (manager.logContains('.*Scan failed established policy check.*')) {
error("Build failed because of negative fortify policy check.")
}
}
}
I try to invoke the below salt script using Jenkins:
create_script:
file.managed:
- name: /tmp/broc/import_props.sh
- source: salt://projects/broc/jboss/files/import.sh.jinja
- template: jinja
Import_properties:
cmd.script:
- name: /tmp/broc/import.sh
- cwd: /tmp/broc`
The Jenkins console output is:
`ID: create_script
Function: file.managed
Name: /tmp/broc/import.sh
Result: True
Comment: File /tmp/broc/import.sh updated
Started: 11:31:13.736928
Duration: 166.319 ms
Changes:
----------
diff:
New file
mode:
0644
ID: Import_properties
Function: cmd.script
Name: /tmp/broc/import.sh
Result: False
Comment: Command '/tmp/broc/import.sh' run
Started: 11:31:13.903378
Duration: 399.825 ms
Changes:
----------
pid:
8292
retcode:
1`
And Jenkins build finished success:
`Succeeded: 21 (changed=22)
Failed: 1
Total states run: 22
Total run time: 30.338 s"}}]
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS`
My question is a saltstack job ID Import_properties Result is False. So a Jenkins build should also finish as FAILURE. In the above case the saltstack result is ignored and build finished SUCCESS. Is there a way the jenkins build be made FAILURE based on saltstack Result?
I see the below Jenkins pipeline:
`try{
saltCmd = "\"salt -E \"($target)\" \ state.apply projects.alip.process-server \
pillar=\'{\"region\": \"${Region}\",\"siteid\":\"${SiteID}\",\"dbuser\":\"${DBUSER}\",\"dbpass\":\"${DBPASS}\"}\' \""
result = salt authtype: 'pam',
clientInterface: local(
arguments: saltCmd,
blockbuild: true,
function: 'cmd.run',
target: "$my_salttarget",
saveFile: true,
targettype: 'glob'),
credentialsId: "$my_saltcred",
servername: "$my_saltserver"
}
}catch(e){
result = e.toString()
currentBuild.result = 'FAILURE'
}finally{
echo result.replace("\\n",'\n')
}
}`
I am new to Jenkins pipeline script, can you help suggest inputs for adding a post build steps under finally to parse the Jenkins console output, identify a string and if it matches mark the build failure. This is similar to a text finder plugin except that we write a pipeline script.
Even if your state failed, Jenkins believe the salt command it evoked still ran successfully.
Jenkins is not able to detect the errors inside salt itself.
It can only tell if salt runs successfully or not.
This is the same as when you ran salt command line. Even if the state fail, the shell command salt still returns 0.
Your issue here is (as said by others) due to jenkins using the return code if the salt command itself and NOT the return code of the action taken by the state you applied.
My 2 cents here is the
--retcode-passthrough
option you can pass to your salt command.
This option allows the salt command return code to match the action taken by the state.
Simply put, if anything fails in a state then the salt command will return a failure return code.
Official doc here
I'm using Jenkins 2.89, SonarQube Scanner for Jenkins 2.6.1 and SonarQube 6.7 configured with a webhook to Jenkins.
I'm triggering Sonar analysis with:
stage("SonarQube Analysis") {
script {
workspace = resolveWorkspacePath()
withEnv(["JAVA_HOME=${ tool 'java-8'}","PATH+MAVEN=${tool 'Maven 3.2.2'}/bin:${env.JAVA_HOME}/bin"]) {
withSonarQubeEnv('Sonar Solem') {
sh "mvn -f ${workspace}/pom.xml org.sonarsource.scanner.maven:sonar-maven-plugin:3.3.0.603:sonar -Dsonar.host.url=http://sonar.mycompany.cl"
}
}
}
and then gathering status at the next stage with:
stage("SonarQube Quality Gate") {
steps {
script {
timeout(time: 1, unit: 'HOURS') {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
echo "Status: ${qg.status}"
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
}
}
The log in console shows:
[Pipeline] script
[Pipeline] {
[Pipeline] timeout
Timeout set to expire in 1 hr 0 min
[Pipeline] {
[Pipeline] waitForQualityGate
Checking status of SonarQube task 'AV-nIGNjEMS3I3uac4Dq' on server 'Sonar MyCompany'
SonarQube task 'AV-nIGNjEMS3I3uac4Dq' status is 'IN_PROGRESS'
[Pipeline] echo
Status: NONE
[Pipeline] error
[Pipeline] }
Looking at logging level, I see the SonarQube webhook POST with a correct payload:
Received POST from 10.0.0.236
Nov 10, 2017 3:27:06 PM FINE org.sonarsource.scanner.jenkins.pipeline.SonarQubeWebHook
Full details of the POST was {"serverUrl":"http://sonar.mycompany.cl","taskId":"AV-nLx-zEMS3I3uac4Ds","status":"SUCCESS","analysedAt":"2017-11-10T15:25:50-0300","changedAt":"2017-11-10T15:25:50-0300","project":{"key":"com.mycompany:mycomponent","name":"My Company Component","url":"http://sonar.mycompany.cl/dashboard?id=com.mycompany%3Amycomponent"},"branch":{"name":"master","type":"LONG","isMain":true,"url":"http://sonar.mycompany.cl/dashboard?id=com.mycompany%3Amycomponent"},"properties":{}}
I didn't find a working solution to this, so I suppose it works correctly for the most of people. I'm using the latest version for every component, maybe a regression?
Regards
I came across a similar situation where waitForQualityGate() fails with Status NONE.
Looking at the console output
[Pipeline] waitForQualityGate
Checking status of SonarQube task 'AWWpiDY2hX3zDQY-CMoe' on server 'Sonar1'
SonarQube task 'AWWpiDY2hX3zDQY-CMoe' status is 'SUCCESS'
SonarQube task 'AWWpiDY2hX3zDQY-CMoe' completed. Quality gate is 'NONE'
Quality gate is 'NONE' ??? I wasnt sure what was causing this as previous test runs returned Quality gate is 'OK'.
Having a look on SonarQube server I notice that the default Quality Gate 'SonarQube Way' was NOT set as default. In my situation I was only working with this one Quality Gate. Resetting this to default resolved my issue.
[Pipeline] waitForQualityGate
Checking status of SonarQube task 'AWWpnnRThX3zDQY-CMpM' on server 'Sonar1'
SonarQube task 'AWWpnnRThX3zDQY-CMpM' status is 'PENDING'
SonarQube task 'AWWpnnRThX3zDQY-CMpM' status is 'SUCCESS'
SonarQube task 'AWWpnnRThX3zDQY-CMpM' completed. Quality gate is 'OK'
The following SonarQube (6.3) analysis stage in a declarative pipeline in Jenkins 2.50 is failing with this error in the console log: http://pastebin.com/t2ja23vC. More specifically:
SonarQube installation defined in this job (SonarGate) does not match any configured installation. Number of installations that can be configured: 1.
Update: after changing "SonarQube" to "SonarGate" in the Jenkins settings (under SonarQube servers, so it'll match the Jenkinsfile), I get a different error: http://pastebin.com/HZZ6fY6V
java.lang.IllegalStateException: Unable to get SonarQube task id and/or server name. Please use the 'withSonarQubeEnv' wrapper to run your analysis.
The stage is a modification of the example from the SonarQube docs: https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+Jenkins#AnalyzingwithSonarQubeScannerforJenkins-AnalyzinginaJenkinspipeline
stage ("SonarQube analysis") {
steps {
script {
STAGE_NAME = "SonarQube analysis"
if (BRANCH_NAME == "develop") {
echo "In 'develop' branch, don't analyze."
}
else { // this is a PR build, run sonar analysis
withSonarQubeEnv("SonarGate") {
sh "../../../sonar-scanner-2.9.0.670/bin/sonar-scanner"
}
}
}
}
}
stage ("SonarQube Gatekeeper") {
steps {
script {
STAGE_NAME = "SonarQube Gatekeeper"
if (BRANCH_NAME == "develop") {
echo "In 'develop' branch, skip."
}
else { // this is a PR build, fail on threshold spill
def qualitygate = waitForQualityGate()
if (qualitygate.status != "OK") {
error "Pipeline aborted due to quality gate coverage failure: ${qualitygate.status}"
}
}
}
}
}
I also created a webhook, sonarqube-webhook, with the URL http://****/sonarqube-webhook/. Should it be like that, or http://****/sonarqube/sonarqube-webhook? To access the server dashboard I use http://****/sonarqube.
In SonarQube's Quality Gates section I created a new quality gate:
I am not sure if the setting in SonarGate is correct. I do use jenkins-mocha to generate an lcov.info file that is used in Sonar to generate the coverage data.
Perhaps the quality gate setting is the wrong setting to do? The end result is to fail the job in Jenkins if coverage % is not met.
Finally, I am not sure if the following configurations in the Jenkins system configuration are at all required:
And
(It's 9000 not 900... cut text in the screen shot)
The SonarQube Jenkins plugin scans the build output for two specific lines, which it uses to get the SonarQube report task properties and project URL. If your invocation of sonar-scanner does not output these lines, the waitForQualityGate() call won't have the task ID to look them up. So you will have to figure out the correct settings to make it more verbose.
See the extractSonarProjectURLFromLogs and extractReportTask methods in the SonarUtils class of the plugin to understand how they work:
ANALYSIS SUCCESSFUL, you can browse <project URL> is used to add a link to the badge (in the build history)
Working dir: <dir with report-task.txt> is used to pass the task ID to the waitForQualityGate step
This was discovered to be a bug in the SonarQube scanner for Jenkins, when using a Jenkins slave for jobs (if the job is run on the master, it'd work). You can read more here: https://jira.sonarsource.com/browse/SONARJNKNS-282
I have tested this using a test build of v2.61 of the scanner plug-in and found it working.
The solution is to upgrade to v2.61 when released.
This stage will then work:
stage ("SonarQube analysis") {
steps {
withSonarQubeEnv('SonarQube') {
sh "../../../sonar-scanner-2.9.0.670/bin/sonar-scanner"
}
def qualitygate = waitForQualityGate()
if (qualitygate.status != "OK") {
error "Pipeline aborted due to quality gate coverage failure: ${qualitygate.status}"
}
}
}
If you're running SonarCube in a docker container check that the memory isn't exhausted. We were maxing out. Which seemed to be the issue.
I try to run sonar tests with maven in my Jenkins pipeline project. The documentations says if the sonar is configured globally and you use the withSonarQube step the environment variables with the globally configured sonar properites are injected. So far so good.
http://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+Jenkins#AnalyzingwithSonarQubeScannerforJenkins-AnalyzingwithSonarQubeScannerforMaven
My pipeline config looks like:
def stash = '********'
def branch = 'dev'
stage('git') {
node {
git branch: branch, credentialsId: 'Buildserver-Private.key', url: stash
}
}
stage('build') {
node {
//....
}
}
stage('sonar') {
node {
withSonarQubeEnv('Sonar') {
sh 'mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar'
}
}
}
The build fails because the sonar plugin trys to connect to the default h2 database instead of the configured one. If i check the log, there are no sonar properties passed to maven.
Injecting SonarQube environment variables using the configuration: Sonar
[Pipeline] {
[Pipeline] tool
[Pipeline] sh
[***********] Running shell script
+ cd .
+ /var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3_3_9/bin/mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar
[INFO] Scanning for projects...
[...]
[INFO] --- sonar-maven-plugin:3.2:sonar (default-cli) # *******.project.build ---
[INFO] User cache: /var/lib/jenkins/.sonar/cache
[INFO] SonarQube version: 4.5.6
[INFO] Default locale: "en_US", source code encoding: "UTF-8" (analysis is platform dependent)
12:23:17.971 INFO - Load global referentials...
12:23:18.071 INFO - Load global referentials done: 102 ms
12:23:18.102 INFO - User cache: /var/lib/jenkins/.sonar/cache
12:23:18.109 INFO - Install plugins
12:23:18.176 INFO - Install JDBC driver
12:23:18.183 INFO - Create JDBC datasource for jdbc:h2:tcp://localhost/sonar
Why is my config ignored? What does the documentation mean if it says?
Since version 2.5 of the SonarQube Scanner for Jenkins, there is an
official support of Jenkins pipeline. We provide a 'withSonarQubeEnv'
block that allow to select the SonarQube server you want to interact
with. Connection details you have configured in Jenkins global
configuration will be automatically passed to the scanner.
It seems they are not ...
Has anybody an idea what am I missing?
You are using an old version of SonarQube (4.5.6, the previous LTS) that requires to pass DB connection parameters (URL, login, password) to the scanners - which is a security issue. withSonarQubeEnv does not propagate those settings in order to fix this flaw.
Since SonarQube 5.2, these parameters are no longer required. So you have to use a version that is more recent. I suggest you to upgrade to the latest LTS version of SonarQube (5.6).