I want to perform SonarQube analysis of a git repository code and I want to use SonarScanner from Docker Container, not from within Jenkins Configuration.
I've tried to create this pipeline:
pipeline {
agent { docker { image 'emeraldsquad/sonar-scanner:latest' } }
stages {
stage('build && SonarQube analysis') {
steps {
withSonarQubeEnv('sonar.tools.devops.****') {
sh 'sonar-scanner \\ -Dsonar.projectKey=myProject \\ -Dsonar.sources=./src \\'
}
}
}
stage("Quality Gate") {
steps {
timeout(time: 1, unit: 'HOURS') {
// Parameter indicates whether to set pipeline to UNSTABLE if Quality Gate fails
// true = set pipeline to UNSTABLE, false = don't
// Requires SonarScanner for Jenkins 2.7+
waitForQualityGate abortPipeline: true
}
}
}
}
}
The build fails on stage build && SonarQube analysis, with build output:
Injecting SonarQube environment variables using the configuration: sonar.tools.devops.*****
[Pipeline] {
[Pipeline] sh
+ sonar-scanner ' -Dsonar.projectKey=myProject' ' -Dsonar.sources=./src' '\'
ERROR: Unrecognized option: -Dsonar.sources=./src
INFO:
INFO: usage: sonar-scanner [options]
INFO:
INFO: Options:
INFO: -D,--define <arg> Define property
INFO: -h,--help Display help information
INFO: -v,--version Display version information
INFO: -X,--debug Produce execution debug output
I would try to remove double backslashes between the arguments:
sh 'sonar-scanner -Dsonar.projectKey=myProject -Dsonar.sources=./src'
The backslashes escape spaces which won't be trucated by the shell and are added to the argument's name.
Related
I would like to install maven and npm via docker agent using Jenkins declarative pipeline. But When I would like to use below script Jenkins throws an error as below. It might be using agent none but how can I use node with docker agent via declarative pipeline jenkins.
ERROR: Attempted to execute a step that requires a node context while
‘agent none’ was specified. Be sure to specify your own ‘node { ... }’
blocks when using ‘agent none’.
I try to set agent any but this time I received an error "Still waiting to schedule task
Waiting for next available executor"
pipeline {
agent none
// environment{
proxy = https://
// stable_revision = sh(script: 'curl -H "Authorization: Basic $base64encoded"
// }
stages {
stage('Build') {
agent {
docker { image 'maven:3-alpine'}
}
steps {
sh 'mvn --version'
echo "$apigeeUsername"
echo "Stable Revision: ${env.stable_revision}"
}
}
stage('Test') {
agent { docker { image 'maven:3-alpine' image 'node:8.12.0' } }
environment {
HOME = '.'
}
steps {
script{
try{
sh 'npm install'
sh 'node --version'
//sh 'npm test/unit/*.js'
}catch(e){
throw e
}
}
}
}
// stage('Policy-Code Analysis') {
// steps{
// sh "npm install -g apigeelint"
// sh "apigelint -s wiservice_api_v1/apiproxy/ -f codeframe.js"
// }
// }
stage('Promotion'){
steps{
timeout(time: 2, unit: 'DAYS') {
input 'Do you want to Approve?'
}
}
}
stage('Deployment'){
steps{
sh "mvn -f wiservice_api_v1/pom.xml install -Ptest -Dusername=${apigeeUsername} -Dpassword=${apigeePassword} -Dapigee.config.options=update"
//sh "mvn apigee-enterprise:install -Ptest -Dusername=${apigeeUsername} -Dpassword=${apigeePassword} "
}
}
}
}
Basically your error message tells you everything you need to know:
ERROR: Attempted to execute a step that requires a node context while
‘agent none’ was specified. Be sure to specify your own ‘node { ... }’
blocks when using ‘agent none’.
so what is the issue here? You use agent none for your pipeline which means you do not specify a specific agent for all stages. An agent executes a specific stage. If a stage has no agent it can't be executed and this is your issue here.
The following 2 stage have no agent which means no docker-container / server or whatever where it can be executed.
stage('Promotion'){
steps{
timeout(time: 2, unit: 'DAYS') {
input 'Do you want to Approve?'
}
}
}
stage('Deployment'){
steps{
sh "mvn -f wiservice_api_v1/pom.xml install -Ptest -Dusername=${apigeeUsername} -Dpassword=${apigeePassword} -Dapigee.config.options=update"
//sh "mvn apigee-enterprise:install -Ptest -Dusername=${apigeeUsername} -Dpassword=${apigeePassword} "
}
}
so you have to add agent { ... } to both stage seperately or use a global agent like following and remove the agent from your stages:
pipeline {
agent {
docker { image 'maven:3-alpine'}
} ...
For further information see guide to set up master and agent machines or distributed jenkins builds or the official documentation.
I think you meant to add agent any instead of agent none, because each stage requires at least one agent (either declared at the top for the pipeline or per stage).
Also, I see some more issues.
Your Test stage specifies two images for the same stage.
agent { docker { image 'maven:3-alpine' image 'node:8.12.0' } } although, your stage is executing only npm commands. I believe only one of the image will be downloaded.
To clarify bit more on mkemmerz answer, your Promotion stage is designed correctly. If you plan to have an input step in the pipeline, do not add an agent for the pipeline because input steps block the executor context. See this link https://jenkins.io/blog/2018/04/09/whats-in-declarative/
I have a Jenkins pipeline that periodically pull from gitlab and build different repos, build a multi-component platform, run and test it. Now I installed a sonarqube server on the same machine (Ubuntu 18.04) and I want to connect my Jenkins to sonarqube.
In Jenkins:
I set up the sonarqube scanner at Global Tool Configuration as below:
I generated a token in sonarqube and in Jenkins at configuration I set up the server as below BUT I couldn't find any place to insert the token (and I think this is the problem):
In the jenkins pipeline this is how I added a stage for sonarqube:
stage('SonarQube analysis') {
steps{
script {
scannerHome = tool 'SonarQube';
}
withSonarQubeEnv('SonarQube') {
sh "${scannerHome}/bin/sonar-scanner"
}
}
}
But this fails with below logs and ERROR: script returned exit code 127:
[Pipeline] { (SonarQube analysis)
[Pipeline] script
[Pipeline] {
[Pipeline] tool
Invalid tool ID
[Pipeline] }
[Pipeline] // script
[Pipeline] withSonarQubeEnv
Injecting SonarQube environment variables using the configuration: SonarQube
[Pipeline] {
[Pipeline] sh
+ /var/lib/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarQube/bin/sonar-scanner
/var/lib/jenkins/workspace/wws-full-test#tmp/durable-2c68acd1/script.sh: 1: /var/lib/jenkins/workspace/wws-full-test#tmp/durable-2c68acd1/script.sh: /var/lib/jenkins/tools/hudson.plugins.sonar.SonarRunnerInstallation/SonarQube/bin/sonar-scanner: not found
[Pipeline] }
WARN: Unable to locate 'report-task.txt' in the workspace. Did the SonarScanner succeeded?
[Pipeline] // withSonarQubeEnv
[Pipeline] }
[Pipeline] // stage
And when I check my jenkinstools on the disk sonnar plugin is not there:
$ ls /var/lib/jenkins/tools/
jenkins.plugins.nodejs.tools.NodeJSInstallation
Can someone please let me know how I can connect Jenkins to sonarqube?
Create and add token to be able to connect to SonarQube.
You have create project in SonarQube and use it as a parameter:
sh """
${scannerHome}/bin/sonar-scanner \
-Dsonar.projectKey=your_project_key_created_in_sonarqube_as_project \
-Dsonar.sources=. \
"""
I'm facing a problem in implementing SonarQube scanner for my repository in Jenkinsfile. I don't know where should I add the properties of SonarQube scanner in the Jenkinsfile.
I've set Jenkins locally on my windows system. The projects are purely based on Python, Ruby & React.
agent {label 'master'}
triggers {
GenricTrigger ([
genricVariables: [
key: 'pr_from_branch', value: '$.pullrequest.source.branch.name'],
[
expressionType: 'JsonPath',
regexpFilter: '',
defaultValue: ''],
token: 'test'])
}
options {
buildDiscarder (
logRotator(numToKeepStr:'5'))
}
stages {
stage ('Initialize & SonarQube Scan') {
steps {
def scannerHome = tool 'sonarScanner';
withSonarQubeEnv('My SonarQube Server') {
bat """
${scannerHome}/bin/sonar-runner.bat
pip install -r requirements.txt
"""
}
}
}
stage('Quality Gate') {
sleep time: 3000, unit: 'MILLISECONDS'
timeout(time: 1, unit: 'MINUTES') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
stage ('Smoke Test') {
steps {
bat """
pytest -s -v tests/home/login_test.py
currentBuild.result = 'SUCCESS'
"""
}
}
}
}
The properties include:
-----------------Sonarqube configuration........................
sonar.projectKey=<*****>
sonar.projectName=<project name>
sonar.projectVersion=1.0
sonar.login=<sonar-login-token>
sonar.sources=src
sonar.exclusions=**/*.doc,**/*.docx,**/*.ipch,/node_modules/,
sonar.host.url=http://<url>/
-----------------Sonar for bitbucket plugin configuration...................
sonar.bitbucket.repoSlug=<project name>
sonar.bitbucket.accountName=<name>
sonar.bitbucket.oauthClientKey=<OAuth_Key>
sonar.bitbucket.oauthClientSecret=<OAuth_secret>
sonar.analysis.mode=issues
I can manually add these properties in sonar-project.properties file and set this file in my project root directly but it will be running locally not on the server. So to avoid that I want to add these properties to Jenkinsfile
We run Sonar scanner as a Docker container but it should give you a fair idea of how to use your properties for the same in Jenkinsfile.
stage("Sonar Analysis"){
sh "docker pull docker.artifactory.company.com/util-sonar-runner:latest"
withSonarQubeEnv('sonarqube'){
sh "docker run --rm -v ${workspace}:/opt/spring-service -w /opt/spring-service -e SONAR_HOST_URL=${SONAR_HOST_URL} -e SONAR_AUTH_TOKEN=${SONAR_AUTH_TOKEN} docker.artifactory.company.com/util-sonar-runner:latest /opt/sonar-scanner/bin/sonar-scanner -Dsonar.host.url=${SONAR_HOST_URL} -Dsonar.login=${SONAR_AUTH_TOKEN} -Dsonar.projectKey=spring-service -Dsonar.projectName=spring-service -Dsonar.projectBaseDir=. -Dsonar.sources=./src -Dsonar.java.binaries=./build/classes -Dsonar.junit.reportPaths=./build/test-results/test -Dsonar.jacoco.reportPaths=./build/jacoco/test.exec -Dsonar.exclusions=src/test/java/**/* -Dsonar.fortify.reportPath=fortifyResults-${IMAGE_NAME}.fpr -Dsonar.password="
}
}
You run the pipeline step like this. The sonar server properties can be defined under the profile of the pom.xml file.
steps {
withSonarQubeEnv('SonarQube') {
sh 'mvn -Psonar -Dsonar.sourceEncoding=UTF-8 org.sonarsource.scanner.maven:sonar-maven-plugin:3.0.2:sonar'
}
}
The SonarQube scanner needs to be defined on Jenkins Global tool Configuration section.
I have a Docker container running Jenkins (2.150.1) and another Docker container running SonarQube (7.4). Jenkins is using the SonarQube Scanner for Jenkins plugin and the scanning is done on the Jenkins container. The Jenkinsfile for the project looks like this:
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'building...'
}
}
stage('Test') {
steps {
echo 'testing...'
withSonarQubeEnv ('SonarQube') {
sh '/var/jenkins_home/sonar-scanner/sonar-scanner-3.2.0.1227-linux/bin/sonar-scanner'
}
echo 'really finished testing2'
}
}
stage("Quality Gate") {
steps {
timeout(time: 1, unit: 'MINUTES') {
waitForQualityGate abortPipeline: true
}
}
}
stage('Deployment') {
steps {
echo 'deploying...'
}
}
}
}
To get the scanning to work as part of a Jenkins pipeline job, I manually installed sonar-scanner on the Jenkins container by downloading the zip file and unzipping it to: /var/jenkins_home/sonar-scanner/sonar-scanner-3.2.0.1227-linux
This is working well, but I want to improve it by:
taking out the harcoded path to sonar-scanner from my Jenkinsfile
specify a non local location of sonar-scanner because I now need to run the scan on another VM/container instead of on the Jenkins container
I tried using Manage Jenkins > Global Tool Configuration > SonarQube Scanner and updated my Jenkinsfile to use SONAR_RUNNER_HOME instead of the hard coded path, but that didn't work and I got an error that sonar-scanner can't be found.
In Manage Jenkins > Global Tool Configuration > SonarQube Scanner check install automatically.
Then go to Manage Jenkins > Configure System and add the following
The Name should be the same as the parameter in the line in your Jenkinsfile: withSonarQubeEnv('SonarQube')
I am trying to set an environment variable with name "TEST_CONFIG_ROOT" in my Jenkins pipeline, I am referring to examples here:
https://jenkins.io/doc/book/pipeline/jenkinsfile/#working-with-the-environment
But, When I am executing my test, it appears that env variable is not set, because my test still complains that it did not get the value of the variable "TEST_CONFIG_ROOT" which it was supposed to get from env.
Please see my jenkinsFile below:
node('node1'){
def buildInput;
echo 'Deploying my build'
if(!params.buildName) {
buildInput = input(
id: 'userInput', message: 'What is the build name?', parameters: [
[$class: 'StringParameterDefinition', defaultValue: 'abcd-1', description: 'Environment', name: 'buildName']
])
}
buildToUse = params.buildName ? params.buildName : buildInput;
echo ("Env: "+buildToUse);
if ( "${params.buildParam}" == 'prequal' || !params.buildParam ){
stage('Prequal') {
}
}
node('nodename'){
if ( "${params.buildParam}" == 'test' || !params.buildParam ){
withMaven(
maven: 'M2Slave',
mavenSettingsConfig: 'MavenSettingsXML',
mavenLocalRepo: '${HOME}/.m2/repository') {
stage('Test') {
echo 'Testing my build'
echo " my work space is ${env.WORKSPACE}"
checkout scm
environment {
TEST_CONFIG_ROOT = '${env.WORKSPACE}/testsE2e/src/main/resources'
}
dir ( 'testsE2e'){
sh 'mvn clean verify surefire-report:report surefire-report:failsafe-report-only -Dtestngxml=testng.xml -Dhttp.proxyHost=proxy02.com -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy02.com -Dhttps.proxyPort=8080 -Djavax.xml.accessExternalSchema=all'
}
}
}
}
}
}
I also tried executing the export command using the shell script as below, but this is also not helping.
echo " my work space is ${env.WORKSPACE}"
sh 'export TEST_CONFIG_ROOT="${WORKSPACE}/testsE2e/src/main/resources"'
Find below snippet of log when the pipeline job is executed:
[Pipeline] echo
my work space is /usr/home/nodename/Jenkins/workspace/workspace/someName-RCZ4A4MA3GRP4PSVYT4HTVVIKU4J7TW2667CKTZU22CHR2CBEM5Q
[Pipeline] dir
Running in /usr/home/nodename/Jenkins/workspace/workspace/someName-RCZ4A4MA3GRP4PSVYT4HTVVIKU4J7TW2667CKTZU22CHR2CBEM5Q/testsE2e
[Pipeline] {
[Pipeline] sh
[testsE2e] Running shell script
+ mvn clean verify surefire-report:report surefire-report:failsafe-report-only -Dtestngxml=testng.xml -Dhttp.proxyHost=proxy02.com -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy02.com -Dhttps.proxyPort=8080 -Djavax.xml.accessExternalSchema=all
----- withMaven Wrapper script -----
Picked up JAVA_TOOL_OPTIONS: -Dmaven.ext.class.path="/usr/home/nodename/Jenkins/workspace/workspace/someName-RCZ4A4MA3GRP4PSVYT4HTVVIKU4J7TW2667CKTZU22CHR2CBEM5Q#tmp/withMaven00e87287/pipeline-maven-spy.jar" -Dorg.jenkinsci.plugins.pipeline.maven.reportsFolder="/usr/home/nodename/Jenkins/workspace/workspace/someName-RCZ4A4MA3GRP4PSVYT4HTVVIKU4J7TW2667CKTZU22CHR2CBEM5Q#tmp/withMaven00e87287"
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T08:41:47-08:00)
Maven home: /opt/maven/apache-maven-3.3.9
Java version: 1.8.0_111, vendor: Oracle Corporation
Java home: /opt/oracle/jdk1.8.0_111/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-573.7.1.el6.x86_64", arch: "amd64", family: "unix"
[jenkins-maven-event-spy] INFO generate /usr/home/nodename/Jenkins/workspace/workspace/someName-RCZ4A4MA3GRP4PSVYT4HTVVIKU4J7TW2667CKTZU22CHR2CBEM5Q#tmp/withMaven00e87287/maven-spy-20170924-225639-49.log ...
[INFO] Scanning for projects...
I would say you're mixing up declarative pipelines with scripted ones (see Pipeline Syntax in the docs).
The following code snippet belongs to declarative ones, but you have a scripted one:
environment {
TEST_CONFIG_ROOT = "${env.WORKSPACE}/testsE2e/src/main/resources"
}
With scripted pipelines, it's actually a bit easier:
env.TEST_CONFIG_ROOT = "${env.WORKSPACE}/testsE2e/src/main/resources"
So, this is what worked in my case, just mentioning for others reference too.
withEnv(["TEST_CONFIG_ROOT=${env.WORKSPACE}/testsE2e/src/main/resources"]) {
dir ( 'testsE2e'){
sh 'mvn clean verify surefire-report:report surefire-report:failsafe-report-only -Dtestngxml=testng.xml -Dhttp.proxyHost=proxy02.com -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy02.com -Dhttps.proxyPort=8080 -Djavax.xml.accessExternalSchema=all'
}
}