I have intergrated Sonarqube in Jenkins Pipeline, it working as below define:
stage('static check') {
steps {
echo "starting codeAnalyze with SonarQube......"
//sonar:sonar.QualityGate should pass
withSonarQubeEnv('sonar') {
sh "mvn -f pom.xml clean compile sonar:sonar -U"
}
script {
timeout(10) {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
echo "failure: ${qg.status}"
# Plan to send notify to Slack, But can NOT to obtain the Sonar report URL or TaskId
}
}
}
}
}
Plan to send corresponding notify to Slack, But can NOT to obtain the Sonar report URL or TaskId
And found some guys suggest that use BUILD_LOG_REGEX: SonarQube result URL: ${BUILD_LOG_REGEX, regex=".*ANALYSIS SUCCESSFUL, you can browse (.*)", showTruncatedLines=false, substText="$1"}, but this only working in Email Extension Plugin.
Another suggest is compose the report URL by this: ProjectURL = host + "/dashboard?id=" + ProjectID, and we not easy to get the ProjectID, since the projectID is compose from Pom.xml as: groupId:artifactId.
Maybe there is some easy way to get the Sonar TaskId or Report URL, Especially whether there is the waitForQualityGate() or other method will return the Sonar TaskId or Report URL ?
You can see the report by reading the file ./target/sonar/report-task.txt
Using def getURL = readProperties file: './target/sonar/report-task.txt'
and extract by calling something like this ${getURL['dashboardUrl']} which is will give url of report. also that above file have taskId.
so it should like this
def qg = waitForQualityGate()
if (qg.status != 'OK') {
echo "failure: ${qg.status}"
def getURL = readProperties file: './target/sonar/report-task.txt
Add your slackmsg here use in that ${getURL['dashboardUrl']}
}
For windows, use
./.scannerwork/report-task.txt
Eg
waitForQualityGate abortPipeline: true
def getURL = readProperties file: './.scannerwork/report-task.txt'
sonarqubeURL = "${getURL['dashboardUrl']}"
echo "${sonarqubeURL }"
Related
I'm attempting to validate that all Jenkins pipelines, at least in a single group/organization, have published their junit tests. Is there a way to programmatically do this? Also, would it be relegated to Jenkinsfiles or work on all pipelines? Thanks!
I could manually check this via looking for the "Test Results" on the page that I have included the image for below. This indicates that the job has published Test Results to the JUnit plugin.
If I were to write a Jenkinsfile, it might look something like this. But it is possible to attach these to the JUnit pipeline via manual methods as well:
pipeline {
agent any
stages {
stage('Compile') {
steps {
// Login to Repository
configFileProvider([configFile(fileId: 'nexus_maven_configuration', variable: 'MAVEN_SETTINGS')]) {
sh 'mvn -s $MAVEN_SETTINGS compile'
}
}
}
stage('Test') {
steps {
configFileProvider([configFile(fileId: 'nexus_maven_configuration', variable: 'MAVEN_SETTINGS')]) {
sh 'mvn -s $MAVEN_SETTINGS test'
}
}
}
}
post {
always {
junit '**/target/surefire-reports/*.xml'
archive 'target/*.jar'
}
}
}
Here is a script you can use to check whether you have tests attached for Jobs in a specific Subdirectory. You can either run this through a Pipeline or using the Jenkins Script Console.
def subFolderToCheck = "folder1" // We will only check Jobs in a specific sub directory
Jenkins.instance.getAllItems(Job.class).each { jobitem ->
def jobName = jobitem.getFullName()
def jobInfo = Jenkins.instance.getItemByFullName(jobName)
// We will check if the last successfull build has any tests attached.
if(jobName.contains(subFolderToCheck) && jobInfo.getLastSuccessfulBuild() != null) {
def results = jobInfo.getLastSuccessfulBuild().getActions(hudson.tasks.junit.TestResultAction.class).result
println("Job : " + jobName + " Tests " + results.size())
if(results == null || results.size() <= 0) {
print("Job " + jobName + " Does not have any tests!!!!!")
}
}
}
I am trying to create New file in Jenkins Pipeline , by getting error.
error:
java.io.FileNotFoundException: /var/lib/jenkins/workspace/Pipeline-Groovy/test.txt (No such file or directory)
But when i am executing below commands without pipeline , It's created new file
def newFile = new File("/var/lib/jenkins/workspace/test/test.txt")
newFile.append("hello\n")
println newFile.text
If i use same code in Pipeline getting above error
pipeline {
agent any
options {
buildDiscarder(logRotator(numToKeepStr: '5'))
timestamps()
}
stages {
stage('Demo1-stage') {
steps {
deleteDir()
script {
def Jobname = "${JOB_NAME}"
echo Jobname
}
}
}
stage('Demo-2stage') {
steps {
script {
def workspace = "${WORKSPACE}"
echo workspace
def newFile = new File("/var/lib/jenkins/workspace/Pipeline-Groovy/test.txt")
newFile.createNewFile()
sh 'ls -lrt'
}
}
}
}
}
It looks like your folder is not present. Do not give absolute path while creating the file unless it is a requirement. I see that in your case, you need a file in workspace. Always use the ${WORKSPACE} to get the current work directory.
def newFile = new File("${WORKSPACE}/test.txt")
newFile.createNewFile()
I configure Jenkins to work with sonarqube scanner. The scan are working fine. The jenkins pipeline is working and I don't have any isssue in the jenkins log.
SonarQube Scanner 3.0.3.778
Jenkins: 2.70
SonarQube Scanner for Jenkins plugin: 2.6.1
I use this code:
stage('SonarQube analysis') {
sh 'sed -ie "s|_PROJECT_|${PROJECT_CODE}|g" $WORKSPACE/_pipeline/sonar-project.properties'
// requires SonarQube Scanner 3.0+
def scannerHome = '/opt/sonar/bin/sonar-scanner';
withSonarQubeEnv('mscodeanalysis') {
sh "${scannerHome}/bin/sonar-scanner -Dproject.settings=$WORKSPACE/_pipeline/sonar-project.properties"
}
}
}
}
}
}
// No need to occupy a node
stage("Quality Gate"){
timeout(time: 15, unit: 'MINUTES') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
My problem come from Quality Gate. It never POST the json payload to jenkins. I don't see a json entry inside jenkins log. But I know the connection between jenkins and sonarqube server is working because I was able to send a POST using curl from the sonarqube VM.
Here the jenkins job output:
Timeout set to expire in 15 min
[Pipeline] {
[Pipeline] waitForQualityGate
Checking status of SonarQube task 'AV3irVJXpvBxXXNJYZkd' on server 'mscodeanalysis'
SonarQube task 'AV3irVJXpvBxXXNJYZkd' status is 'PENDING'
Cancelling nested steps due to timeout
Here is my payload that never reach jenkins pipeline:
url: http://sonar-server:9000/api/ce/task?id=AV3irVJXpvBxXXNJYZkd
{"task":{"organization":"default-organization","id":"AV3irVJXpvBxXXNJYZkd","type":"REPORT","componentId":"AV3hrJeCfL_nrF2072FH","componentKey":"POOL-003","componentName":"POOL-003","componentQualifier":"TRK","analysisId":"AV3irVkZszLEB6PsCK9X","status":"SUCCESS","submittedAt":"2017-08-14T21:36:35+0000","submitterLogin":"jenkins","startedAt":"2017-08-14T21:36:37+0000","executedAt":"2017-08-14T21:36:38+0000","executionTimeMs":650,"logs":false,"hasScannerContext":true}}
I can't insert image but the Quality gate is Pass and the analysis task is success.
Let me know if I need to include more information.
Thank you
The issue could be that Jenkins is using https with self-signed certificate. Then solution is:
Generate truststore for SonarQube:
keytool -import -trustcacerts -alias jenkins-host-name -file cert.crt -keystore sonarqube.jks
keystore passw: password
Where cert.crt - is certificate used for ssl for jenkins, jenkins-host-name - is a hostname of jenkins in the docker network (which is used in webhook)
Add truststore to SonarQube Dockerfile:
FROM sonarqube
COPY sonarqube.jks /var/sonar_cert/
COPY sonar.properties /opt/sonarqube/conf/sonar.properties
Update sonar.properties
sonar.ce.javaAdditionalOpts=-Djavax.net.ssl.trustStore=/var/sonar_cert/sonarqube.jks -Djavax.net.ssl.trustStorePassword=password
Then if you have a correct user and password for Jenkins provided in webhook URL everything should work.
Tried: Jenkins 2.107.2, SonarQube 7.1
Here is a quick example of what we did to resolve this issue:
SonarQube randomly hangs at "pending" state. Telling it to retry refreshes it. We set it to 10 seconds in this example
maxRetry = 200
forloop (i=0; i<maxRetry; i++){
try {
timeout(time: 10, unit: 'SECONDS') {
waitForQualityGate()
}
} catch(Exception e) {
if (i == maxRetry-1) {
throw e
}
}
}
Was surprised to find that #Katone Vi's answer worked so well. Based on their answer we added a quick exit on success and used the DSL for the original request:
stage('SonarQube') {
steps {
withSonarQubeEnv('SonarQube') {
sh """
${scannerHome}/bin/sonar-scanner -Dsonar.projectKey=XXX_${env.STAGE}_lambda
"""
}
script {
Integer waitSeconds = 10
Integer timeOutMinutes = 10
Integer maxRetry = (timeOutMinutes * 60) / waitSeconds as Integer
for (Integer i = 0; i < maxRetry; i++) {
try {
timeout(time: waitSeconds, unit: 'SECONDS') {
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Sonar quality gate status: ${qg.status}"
} else {
i = maxRetry
}
}
} catch (Throwable e) {
if (i == maxRetry - 1) {
throw e
}
}
}
}
}
}
If you have configured SonarQube to use a HTTP(S) proxy, make sure that your jenkins is either reachable through the proxy or is configured as a "non-proxy host". This can be done with the http.nonProxyHosts property or HTTP_NONPROXYHOSTS environment variable. See also the documentation for further information and syntax.
If you are using Jenkinsfile, this is workaround:
define creadentials:
environment {
CRED = credentials('jenkins_user_pass')
}
then use:
stage("Quality Gate") {
steps {
script {
while(true){
sh "sleep 2"
def url="http://jenkinsURL/job/${env.JOB_NAME.replaceAll('/','/job/')}/lastBuild/consoleText";
def sonarId = sh script: "wget -qO- --content-on-error --no-proxy --auth-no-challenge --http-user=${CRED_USR} --http-password=${CRED_PSW} '${url}' | grep 'More about the report processing' | head -n1 ",returnStdout:true
sonarId = sonarId.substring(sonarId.indexOf("=")+1)
echo "sonarId ${sonarId}"
def sonarUrl = "http://jenkinsURL/sonar/api/ce/task?id=${sonarId}"
def sonarStatus = sh script: "wget -qO- '${sonarUrl}' --no-proxy --content-on-error | jq -r '.task' | jq -r '.status' ",returnStdout:true
echo "Sonar status ... ${sonarStatus}"
if(sonarStatus.trim() == "SUCCESS"){
echo "BREAK";
break;
}
if(sonarStatus.trim() == "FAILED "){
echo "FAILED"
currentBuild.result = 'FAILED'
break;
}
}
}
}
}
I have faced similar issue, while the quality Gate back-end activities in Sonar server takes less than 20 sec to complete its analysis.But quality gate fail/success response from sonar-webhook in jenkins job take lot of time and stuck.
stage('Sonar:QG') {
steps {
**sleep(10) /* Added 10 sec sleep that was suggested in few places*/**
script{
timeout(time: 10, unit: 'MINUTES') {
def qg = waitForQualityGate abortPipeline: true
if (qg.status != 'OK') {
echo "Status: ${qg.status}"
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
}
}
Essentially check below thing:-
Webhook is configured in sonar or not :- SonarQube -> Administration -> Webhooks
http://:/sonarqube-webhook/
or Use localhost in place of IP in http://locahlhost:port/sonarqube-webhook/ solves issue in my case.
I'm made more simple decision but it work a same
stage("Quality gate") {
steps {
retry(3){
waitForQualityGate abortPipeline: true
}
}
}
Adding a sh 'sleep 10' between stage('SonarQube analysis') AND stage("Quality Gate") fix the issue. Now the jenkins job receive
Checking status of SonarQube task 'AV3rHxhp3io6giaQF_OA' on server 'sonarserver'
SonarQube task 'AV3rHxhp3io6giaQF_OA' status is 'SUCCESS'
SonarQube task 'AV3rHxhp3io6giaQF_OA' completed. Quality gate is 'OK'
Within my Jenkins Pipeline I need to react on the SonarQube Quality Gate.
Is there an easier way to achieve this but looking in the Sonar-Scanner log for the result page (e.g. https://mysonarserver/sonar/api/ce/task?id=xxxx) and parse the JSON Result from there?
I use Jenkins 2.30 and SonarQube 5.3
Thanks in advance
Based on Vincent's answer, and using Pipeline utility steps, here's my updated version that worked for me (using sonarscanner report file) :
withSonarQubeEnv('SONAR 6.4') {
sh "${scannerHome}/bin/sonar-scanner"
sh "cat .scannerwork/report-task.txt"
def props = readProperties file: '.scannerwork/report-task.txt'
echo "properties=${props}"
def sonarServerUrl=props['serverUrl']
def ceTaskUrl= props['ceTaskUrl']
def ceTask
timeout(time: 1, unit: 'MINUTES') {
waitUntil {
def response = httpRequest ceTaskUrl
ceTask = readJSON text: response.content
echo ceTask.toString()
return "SUCCESS".equals(ceTask["task"]["status"])
}
}
def response2 = httpRequest url : sonarServerUrl + "/api/qualitygates/project_status?analysisId=" + ceTask["task"]["analysisId"], authentication: 'jenkins_scanner'
def qualitygate = readJSON text: response2.content
echo qualitygate.toString()
if ("ERROR".equals(qualitygate["projectStatus"]["status"])) {
error "Quality Gate failure"
}
}
Please note the use of a Jenkins Credentials (authentication: 'jenkins_scanner') to retrieve the quality gate in Sonar being auhtenticated.
Using SonarQube Scanner for Jenkins 2.8.1 the solution is available out of the Box:
stage('SonarQube analysis') {
withSonarQubeEnv('My SonarQube Server') {
sh 'mvn clean package sonar:sonar'
} // SonarQube taskId is automatically attached to the pipeline context
}
}
stage("Quality Gate"){
timeout(time: 1, unit: 'HOURS') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
Scan first:
node("sonar") {
deleteDir()
unstash 'sources'
def scannerHome = tool 'sonar-scanner';
withSonarQubeEnv('sonarqube-rec') {
withEnv(["JAVA_HOME=${ tool 'JDK_8.0' }", "PATH+MAVEN=${tool 'M325'}/bin:${env.JAVA_HOME}/bin"]) {
// requires SonarQube Scanner for Maven 3.2+
sh '''
mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar
echo "SONAR_AUTH_TOKEN=$SONAR_AUTH_TOKEN" >> target/sonar/report-task.txt
'''
stash includes: "target/sonar/report-task.txt", name: 'sonar-report-task'
}
}
}
then check the quality gate:
stage("Quality Gate"){
node("sonar") {
deleteDir()
unstash 'sonar-report-task'
def props = utils.getProperties("target/sonar/report-task.txt")
echo "properties=${props}"
def sonarServerUrl=props.getProperty('serverUrl')
def ceTaskUrl= props.getProperty('ceTaskUrl')
def ceTask
def URL url = new URL(ceTaskUrl)
timeout(time: 1, unit: 'MINUTES') {
waitUntil {
ceTask = utils.jsonParse(url)
echo ceTask.toString()
return "SUCCESS".equals(ceTask["task"]["status"])
}
}
url = new URL(sonarServerUrl + "/api/qualitygates/project_status?analysisId=" + ceTask["task"]["analysisId"] )
def qualitygate = utils.jsonParse(url)
echo qualitygate.toString()
if ("ERROR".equals(qualitygate["projectStatus"]["status"])) {
error "Quality Gate failure"
}
}
}
I used ".sonar/report-task.txt" to retrieve the ceTaskUrl - Then I used Pipeline Shared Libraries and wrote my own Pipeline Function to retrieve the quality gate.
http://mySonarQube.com:9001/api/ce/task?id="ceTaskUrl"
Parse "task.analysisId"
Parse quality-gates from http://mySonarQube.com:9001/api/qualitygates/project_status?analysisId="task.analysisId"
I am using the Pipeline plugin in Jenkins by Clouldbees (the name was Workflow plugin before), I am trying to get the user name in the Groovy script but I am not able to achieve it.
stage 'checkout svn'
node('master') {
// Get the user name logged in Jenkins
}
Did you try installing the Build User Vars plugin? If so, you should be able to run
node {
wrap([$class: 'BuildUser']) {
def user = env.BUILD_USER_ID
}
}
or similar.
To make it work with Jenkins Pipeline:
Install user build vars plugin
Then run the following:
pipeline {
agent any
stages {
stage('build user') {
steps {
wrap([$class: 'BuildUser']) {
sh 'echo "${BUILD_USER}"'
}
}
}
}
}
Here's a slightly shorter version that doesn't require the use of environment variables:
#NonCPS
def getBuildUser() {
return currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserId()
}
The use of rawBuild requires that it be in a #NonCPS block.
It is possible to do this without a plugin (assuming JOB_BASE_NAME and BUILD_ID are in the environment):
def job = Jenkins.getInstance().getItemByFullName(env.JOB_BASE_NAME, Job.class)
def build = job.getBuildByNumber(env.BUILD_ID as int)
def userId = build.getCause(Cause.UserIdCause).getUserId()
There is also a getUserName, which returns the full name of the user.
This works for me without the Build User plugin:
// get first entry of JSONArray
def buildCause = currentBuild.getBuildCauses()[0]
def buildPrincipal = [type:"unknown", name:""]
if (buildCause._class ==~ /.+BranchEventCause/) {
def branchCause = currentBuild.getRawBuild().getCause(jenkins.branch.BranchEventCause)
buildPrincipal = [type:"branch",name:buildCause.shortDescription]
} else
if (buildCause._class ==~ /.+TimerTriggerCause/) {
def timerCause = currentBuild.getRawBuild().getCause(hudson.triggers.TimerTrigger.TimerTriggerCause)
buildPrincipal = [type:"timer", name:"Timer event"]
} else
if (buildCause._class ==~ /.+UserIdCause/) {
def buildUserCause = currentBuild.getRawBuild().getCause(hudson.model.Cause.UserIdCause)
buildPrincipal = [type:"user", name:buildCause.userId]
} else
// ... other causes
def jobUserId, jobUserName
//then somewhere
wrap([$class: 'BuildUser']) {
jobUserId = "${BUILD_USER_ID}"
jobUserName = "${BUILD_USER}"
}
//then
println("Started By: ${jobUserName}")
We were using this plugin : Build User Vars Plugin. More variables are available.
//Below is a generic groovy function to get the XML metadata for a Jenkins build.
//curl the env.BUILD_URL/api/xml parse it with grep and return the string
//I did an or true on curl, but possibly there is a better way
//echo -e "some_string \c" will always return some_string without \n char
//use the readFile() and return the string
def GetUserId(){
sh """
/usr/bin/curl -k -s -u \
\$USERNAME:\$PASSWORD -o \
/tmp/api.xml \
\$BUILD_URL/api/xml || true
THE_USERID=`cat /tmp/api.xml | grep -oP '(?<=<userId>).*?(?=</userId>)'`
echo -e "\$THE_USERID \\c" > /tmp/user_id.txt
"""
def some_userid = readFile("/tmp/user_id.txt")
some_userid
}
I modified #shawn derik response to get it to work in my pipeline:
stage("preserve build user") {
wrap([$class: 'BuildUser']) {
GET_BUILD_USER = sh ( script: 'echo "${BUILD_USER}"', returnStdout: true).trim()
}
}
Then I can reference that variable later on by passing it or in the same scope as ${GET_BUILD_USER} . I installed the same plugin referenced.
Edit: I re-read the question - the below only gets you the user running the build (which technically is often more interesting), not the one triggering the build in the frontend (be it REST-API or WebUI).
If you have Jenkins impersonation enabled, then I believe the result should be equivalent, otherwise this will only get you the user who owns the jenkins agent on the build machine.
Original answer:
Another way would be to
sh 'export jenkins_user=$(whoami)'
Downside: Linux-dependent, difficult to port across multiple agents in a single build (but then, the auth context may be different on each slave)
Upside: No need to install plugins (which on shared/large Jenkins instances can be tricky)
The Build User Vars Plugin is useful when you are executing the stage on an agent.
The alternative is to use the current build clause (see https://code-maven.com/jenkins-get-current-user), which also works when your stage is set with agent none.
The following code is inspired by Juergen's solution but I added more possible trigger reason and display them in a formatted manner:
String getTriggerReason() {
def buildCause = currentBuild.getBuildCauses()[0]
if (buildCause._class ==~ /.+(BranchEventCause|BranchIndexingCause)/) {
if (env.JOB_BASE_NAME == 'master') {
return 'Triggered by master commit'
} else {
return "Triggered by ${buildCause.shortDescription}"
}
}
if (buildCause._class ==~ /.+TimerTriggerCause/) {
return 'Triggered by timer'
}
if (buildCause._class ==~ /.+BuildUpstreamCause/) {
return "Triggered by build #${buildCause.upstreamBuild}"
}
if (buildCause._class ==~ /.+UserIdCause/) {
def userName = buildCause.userName.replaceFirst(/\s?\(.*/, '')
return "Triggered by user ${userName}"
}
return 'Unknown trigger'
}