I am trying to set an environment variable with name "TEST_CONFIG_ROOT" in my Jenkins pipeline, I am referring to examples here:
https://jenkins.io/doc/book/pipeline/jenkinsfile/#working-with-the-environment
But, When I am executing my test, it appears that env variable is not set, because my test still complains that it did not get the value of the variable "TEST_CONFIG_ROOT" which it was supposed to get from env.
Please see my jenkinsFile below:
node('node1'){
def buildInput;
echo 'Deploying my build'
if(!params.buildName) {
buildInput = input(
id: 'userInput', message: 'What is the build name?', parameters: [
[$class: 'StringParameterDefinition', defaultValue: 'abcd-1', description: 'Environment', name: 'buildName']
])
}
buildToUse = params.buildName ? params.buildName : buildInput;
echo ("Env: "+buildToUse);
if ( "${params.buildParam}" == 'prequal' || !params.buildParam ){
stage('Prequal') {
}
}
node('nodename'){
if ( "${params.buildParam}" == 'test' || !params.buildParam ){
withMaven(
maven: 'M2Slave',
mavenSettingsConfig: 'MavenSettingsXML',
mavenLocalRepo: '${HOME}/.m2/repository') {
stage('Test') {
echo 'Testing my build'
echo " my work space is ${env.WORKSPACE}"
checkout scm
environment {
TEST_CONFIG_ROOT = '${env.WORKSPACE}/testsE2e/src/main/resources'
}
dir ( 'testsE2e'){
sh 'mvn clean verify surefire-report:report surefire-report:failsafe-report-only -Dtestngxml=testng.xml -Dhttp.proxyHost=proxy02.com -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy02.com -Dhttps.proxyPort=8080 -Djavax.xml.accessExternalSchema=all'
}
}
}
}
}
}
I also tried executing the export command using the shell script as below, but this is also not helping.
echo " my work space is ${env.WORKSPACE}"
sh 'export TEST_CONFIG_ROOT="${WORKSPACE}/testsE2e/src/main/resources"'
Find below snippet of log when the pipeline job is executed:
[Pipeline] echo
my work space is /usr/home/nodename/Jenkins/workspace/workspace/someName-RCZ4A4MA3GRP4PSVYT4HTVVIKU4J7TW2667CKTZU22CHR2CBEM5Q
[Pipeline] dir
Running in /usr/home/nodename/Jenkins/workspace/workspace/someName-RCZ4A4MA3GRP4PSVYT4HTVVIKU4J7TW2667CKTZU22CHR2CBEM5Q/testsE2e
[Pipeline] {
[Pipeline] sh
[testsE2e] Running shell script
+ mvn clean verify surefire-report:report surefire-report:failsafe-report-only -Dtestngxml=testng.xml -Dhttp.proxyHost=proxy02.com -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy02.com -Dhttps.proxyPort=8080 -Djavax.xml.accessExternalSchema=all
----- withMaven Wrapper script -----
Picked up JAVA_TOOL_OPTIONS: -Dmaven.ext.class.path="/usr/home/nodename/Jenkins/workspace/workspace/someName-RCZ4A4MA3GRP4PSVYT4HTVVIKU4J7TW2667CKTZU22CHR2CBEM5Q#tmp/withMaven00e87287/pipeline-maven-spy.jar" -Dorg.jenkinsci.plugins.pipeline.maven.reportsFolder="/usr/home/nodename/Jenkins/workspace/workspace/someName-RCZ4A4MA3GRP4PSVYT4HTVVIKU4J7TW2667CKTZU22CHR2CBEM5Q#tmp/withMaven00e87287"
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T08:41:47-08:00)
Maven home: /opt/maven/apache-maven-3.3.9
Java version: 1.8.0_111, vendor: Oracle Corporation
Java home: /opt/oracle/jdk1.8.0_111/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-573.7.1.el6.x86_64", arch: "amd64", family: "unix"
[jenkins-maven-event-spy] INFO generate /usr/home/nodename/Jenkins/workspace/workspace/someName-RCZ4A4MA3GRP4PSVYT4HTVVIKU4J7TW2667CKTZU22CHR2CBEM5Q#tmp/withMaven00e87287/maven-spy-20170924-225639-49.log ...
[INFO] Scanning for projects...
I would say you're mixing up declarative pipelines with scripted ones (see Pipeline Syntax in the docs).
The following code snippet belongs to declarative ones, but you have a scripted one:
environment {
TEST_CONFIG_ROOT = "${env.WORKSPACE}/testsE2e/src/main/resources"
}
With scripted pipelines, it's actually a bit easier:
env.TEST_CONFIG_ROOT = "${env.WORKSPACE}/testsE2e/src/main/resources"
So, this is what worked in my case, just mentioning for others reference too.
withEnv(["TEST_CONFIG_ROOT=${env.WORKSPACE}/testsE2e/src/main/resources"]) {
dir ( 'testsE2e'){
sh 'mvn clean verify surefire-report:report surefire-report:failsafe-report-only -Dtestngxml=testng.xml -Dhttp.proxyHost=proxy02.com -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy02.com -Dhttps.proxyPort=8080 -Djavax.xml.accessExternalSchema=all'
}
}
Related
In Jenkins Server, there are two global environment variables defined. It's in Manage Jenkins -> Configure System -> Global Properties -> Environment variables
Name: MAVEN_HOME
Value: /var/home/tools/hudson.tasks.Maven_MavenInstallation/maven3.5.2
Name: PATH+EXTRA
$PATH:/usr/local/bin:$MAVEN_HOME/bin
I see that PATH+EXTRA will add the MAVEN PATH to the PATH environment variable. This is how my existing Server set up is. Now I need to update Jenkins with Maven 3.8.2, so I downloaded Maven 3.8.2 in the server using Manage Jenkins -> Global Tool Configuration -> Maven Installations. Now I am trying to override the global MAVEN_HOME and PATH to point to MAVEN_3.8.2 path.
In the Jenkins pipeline script
def maven_version = 'maven_3.8.2'
pipeline {
agent any
stages {
stage ('build') {
steps {
withEnv(["PATH+MAVEN=${tool maven_version}/bin"]) {
echo "PATH is: $PATH"
echo env.PATH
echo env.MAVEN_HOME
sh 'env'
sh 'mvn --version'
}
}
}
}
}
Results:
echo "PATH is: $PATH" =>
/var/home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.8.2/bin:/opt/java/jdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/var/home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.5.2/bin:/opt/java/jdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
echo env.PATH => /var/home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.8.2/bin:/opt/java/jdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/var/home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.5.2/bin:/opt/java/jdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
echo env.MAVEN_HOME =>
/var/home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.5.2
sh 'env' => prints all the environment variables. Noticed following:
MAVEN_HOME=/var/home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.5.2
PATH=$PATH:/usr/local/bin:/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.5.2/bin:/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.8.2/bin:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.5.2/bin:/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Why is the PATH is being appending with Maven 3.5.2 in the front of the path. How can I let PATH point to Maven 3.8.2?
sh 'mvn --version' => Apache Maven 3.5.2
Maven home: /var/home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.5.2
How do I get the mvn --version result with maven3.8.2?
Note: I also tried with free style project, and used following commands to override the values but the mvn --version is always printing 3.5.2. Any idea if it is a bug with Jenkins unable to override the path or is there any way to do it?
export MAVEN_HOME=/var/home/tools/hudson.tasks.Maven_MavenInstallation/maven_3.8.2
export PATH=$PATH:$MAVEN_HOME/bin
The format you used to modify the PATH variable uses concatenation that prepends the new value to the existing one. It means that
PATH+MAVEN=${tool maven_version}/bin
is an equivalent of:
PATH=${tool maven_version}/bin:$PATH
You can solve this issue by overriding the PATH variable explicitly and putting the new path at the end of the variable. Try to test the pipeline like this one:
def maven_version = 'maven_3.8.2'
pipeline {
agent any
stages {
stage ('build') {
steps {
withEnv(["PATH=${tool maven_version}/bin:$PATH"]) {
echo "PATH is: $PATH"
echo env.PATH
echo env.MAVEN_HOME
sh 'env'
sh 'mvn --version'
}
}
}
}
}
I am Unable to add the above circled functionality in attached image as Declarative Pipeline Syntax.
PS I am new to this, i Searched for this on others answers but no one matches my requirements.
For example if there is a Parameter in jenkins named VERSION, maven command should become
clean deploy -B -s pathtosettings.xml -DVERSION=valueinparameter
Below is my current code
NOte : I WANT ALL THE PARAMETERS AUTOMATICALLY -DVERSION=${params.VERSION} doesnt help me
pipeline {
agent any
stages {
stage('Checkout Scm') {
steps {
git 'ssh://git#XXXXXXXXXXXXXXXXXXXXXXXXX.git'
}
}
stage('Maven Build 0') {
steps {
configFileProvider([configFile(fileId:'0c0631a5-6510-4b4a-833d-4b80fa67d5f3', targetLocation: 'settings.xml', variable: 'SETTINGS_XML')]) {
withMaven{
sh "mvn clean deploy -B -s ${SETTINGS_XML}
}
}
}
}
tools {
jdk 'JDK_1.8'
}
parameters {
string(name: 'VERSION', defaultValue: '3_12_0', description: 'version to be in maven')
}
}
First, I think you doesn't need targetLocation to perform this.
To access to your parameter value, you need to use params prefix.
This is how I'm using the configFileProvider to make it work :
configFileProvider([configFile(fileId: 'configFileId', variable: 'SETTINGS_XML')]) {
sh "mvn clean deploy -s \$SETTINGS_XML -B -DVERSION=$params.VERSION"
}
With this, the variable which link the settings file is not replaced and it's correctly used in my pipeline and the version is replaced in the command. Don't forget to use a
'Maven settings.xml' type of file in the configFileProvider.
steps {
script{
foo= " "
params.each {param ->
foo = "${foo} -D${param.key}=${param.value} "
}
}
configFileProvider([configFile(fileId:'XXXX', targetLocation: 'settings.xml', variable: 'SETTINGS_XML')]) {
withMaven{
sh "mvn clean deploy -B -s ${SETTINGS_XML} - ${foo}"
}
}
This is the Only Approach found
I'm facing a problem in implementing SonarQube scanner for my repository in Jenkinsfile. I don't know where should I add the properties of SonarQube scanner in the Jenkinsfile.
I've set Jenkins locally on my windows system. The projects are purely based on Python, Ruby & React.
agent {label 'master'}
triggers {
GenricTrigger ([
genricVariables: [
key: 'pr_from_branch', value: '$.pullrequest.source.branch.name'],
[
expressionType: 'JsonPath',
regexpFilter: '',
defaultValue: ''],
token: 'test'])
}
options {
buildDiscarder (
logRotator(numToKeepStr:'5'))
}
stages {
stage ('Initialize & SonarQube Scan') {
steps {
def scannerHome = tool 'sonarScanner';
withSonarQubeEnv('My SonarQube Server') {
bat """
${scannerHome}/bin/sonar-runner.bat
pip install -r requirements.txt
"""
}
}
}
stage('Quality Gate') {
sleep time: 3000, unit: 'MILLISECONDS'
timeout(time: 1, unit: 'MINUTES') { // Just in case something goes wrong, pipeline will be killed after a timeout
def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
stage ('Smoke Test') {
steps {
bat """
pytest -s -v tests/home/login_test.py
currentBuild.result = 'SUCCESS'
"""
}
}
}
}
The properties include:
-----------------Sonarqube configuration........................
sonar.projectKey=<*****>
sonar.projectName=<project name>
sonar.projectVersion=1.0
sonar.login=<sonar-login-token>
sonar.sources=src
sonar.exclusions=**/*.doc,**/*.docx,**/*.ipch,/node_modules/,
sonar.host.url=http://<url>/
-----------------Sonar for bitbucket plugin configuration...................
sonar.bitbucket.repoSlug=<project name>
sonar.bitbucket.accountName=<name>
sonar.bitbucket.oauthClientKey=<OAuth_Key>
sonar.bitbucket.oauthClientSecret=<OAuth_secret>
sonar.analysis.mode=issues
I can manually add these properties in sonar-project.properties file and set this file in my project root directly but it will be running locally not on the server. So to avoid that I want to add these properties to Jenkinsfile
We run Sonar scanner as a Docker container but it should give you a fair idea of how to use your properties for the same in Jenkinsfile.
stage("Sonar Analysis"){
sh "docker pull docker.artifactory.company.com/util-sonar-runner:latest"
withSonarQubeEnv('sonarqube'){
sh "docker run --rm -v ${workspace}:/opt/spring-service -w /opt/spring-service -e SONAR_HOST_URL=${SONAR_HOST_URL} -e SONAR_AUTH_TOKEN=${SONAR_AUTH_TOKEN} docker.artifactory.company.com/util-sonar-runner:latest /opt/sonar-scanner/bin/sonar-scanner -Dsonar.host.url=${SONAR_HOST_URL} -Dsonar.login=${SONAR_AUTH_TOKEN} -Dsonar.projectKey=spring-service -Dsonar.projectName=spring-service -Dsonar.projectBaseDir=. -Dsonar.sources=./src -Dsonar.java.binaries=./build/classes -Dsonar.junit.reportPaths=./build/test-results/test -Dsonar.jacoco.reportPaths=./build/jacoco/test.exec -Dsonar.exclusions=src/test/java/**/* -Dsonar.fortify.reportPath=fortifyResults-${IMAGE_NAME}.fpr -Dsonar.password="
}
}
You run the pipeline step like this. The sonar server properties can be defined under the profile of the pom.xml file.
steps {
withSonarQubeEnv('SonarQube') {
sh 'mvn -Psonar -Dsonar.sourceEncoding=UTF-8 org.sonarsource.scanner.maven:sonar-maven-plugin:3.0.2:sonar'
}
}
The SonarQube scanner needs to be defined on Jenkins Global tool Configuration section.
I want to create a Jenkins (v2.126) Declarative syntax pipeline, which has stages with when() clauses checking the value of an environment variable. Specifically I want to set a Jenkins job parameter (so 'build with parameters', not pipeline parameters) and have this determine if a stage is executed.
I have stage code like this:
stage('plan') {
when {
environment name: ExecuteAction, value: 'plan'
}
steps {
sh 'cd $dir && $tf plan'
}
}
The parameter name is ExecuteAction. However, when ExecuteAction is set via a Job "Choice" parameter to: plan, this stage does not run. I can see the appropriate value is coming in via environment variable by adding this debug stage:
stage('debug') {
steps {
sh 'echo "ExecuteAction = $ExecuteAction"'
sh 'env'
}
}
And I get Console output like this:
[Pipeline] stage
[Pipeline] { (debug)
[Pipeline] sh
[workspace] Running shell script
+ echo 'ExecuteAction = plan'
ExecuteAction = plan
[Pipeline] sh
[workspace] Running shell script
+ env
...
ExecuteAction=plan
...
I am using the when declarative syntax from Jenkins book pipeline syntax, at about mid-page, under the when section, built-in conditions.
Jenkins is running on Gnu/Linux.
Any ideas what I might be doing wrong?
Duh! You need to quote the environment variable's name in the when clause.
stage('plan') {
when {
environment name: 'ExecuteAction', value: 'plan'
}
steps {
sh 'cd $dir && $tf plan'
}
}
I believe you need to use params instead of environment. Try the following:
when {
expression { params.ExecuteAction == 'plan' }
}
I want to pass a variable which I read in stage A towards stage B somehow. I see in some examples that people write it to a file, but I guess that is not really a nice solution. I tried writing it to an environment variable, but I'm not really successful on that. How can I set it up properly?
To get it working I tried a lot of things and read that I should use the """ instead of ''' to start a shell and escape those variables to \${foo} for example.
Below is what I have as a pipeline:
#!/usr/bin/env groovy
pipeline {
agent { node { label 'php71' } }
environment {
packageName='my-package'
packageVersion=''
groupId='vznl'
nexus_endpoint='http://nexus.devtools.io'
nexus_username='jenkins'
nexus_password='J3nkins'
}
stages{
// Package dependencies
stage('Install dependencies') {
steps {
sh '''
echo Skip composer installation
#composer install --prefer-dist --optimize-autoloader --no-interaction
'''
}
}
// Unit tests
stage('Unit Tests') {
steps {
sh '''
echo Running PHP code coverage tests...
#composer test
'''
}
}
// Create artifact
stage('Package') {
steps {
echo 'Create package refs'
sh """
mkdir -p ./build/zpk
VERSIONTAG=\$(grep 'version' composer.json)
REGEX='"version": "([0-9]+.[0-9]+.[0-9]+)"'
if [[ \${VERSIONTAG} =~ \${REGEX} ]]
then
env.packageVersion=\${BASH_REMATCH[1]}
/usr/bin/zs-client packZpk --folder=. --destination=./build/zpk --name=${env.packageName}-${env.packageVersion}.zpk --version=${env.packageVersion}
else
echo "No version found!"
exit 1
fi
"""
}
}
// Publish ZPK package to Nexus
stage('Publish packages') {
steps {
echo "Publish ZPK Package"
sh "curl -u ${env.nexus_username}:${env.nexus_password} --upload-file ./build/zpk/${env.packageName}-${env.packageVersion}.zpk ${env.nexus_endpoint}/repository/zpk-packages/${groupId}/${env.packageName}-${env.packageVersion}.zpk"
archive includes: './build/**/*.{zpk,rpm,deb}'
}
}
}
}
As you can see the packageVersion which I read from stage Package needs to be used in stage Publish as well.
Overall tips against the pipeline are of course always welcome as well.
A problem in your code is that you are assigning version of environment variable within the sh step. This step will execute in its own isolated process, inheriting parent process environment variables.
However, the only way of passing data back to the parent is through STDOUT/STDERR or exit code. As you want a string value, it is best to echo version from the sh step and assign it to a variable within the script context.
If you reuse the node, the script context will persist, and variables will be available in the subsequent stage. A working example is below. Note that any try to put this within a parallel block can be of failure, as the version information variable can be written to by multiple processes.
#!/usr/bin/env groovy
pipeline {
environment {
AGENT_INFO = ''
}
agent {
docker {
image 'alpine'
reuseNode true
}
}
stages {
stage('Collect agent info'){
steps {
echo "Current agent info: ${env.AGENT_INFO}"
script {
def agentInfo = sh script:'uname -a', returnStdout: true
println "Agent info within script: ${agentInfo}"
AGENT_INFO = agentInfo.replace("/n", "")
env.AGENT_INFO = AGENT_INFO
}
}
}
stage("Print agent info"){
steps {
script {
echo "Collected agent info: ${AGENT_INFO}"
echo "Environment agent info: ${env.AGENT_INFO}"
}
}
}
}
}
Another option which doesn't involve using script, but is just declarative, is to stash things in a little temporary environment file.
You can then use this stash (like a temporary cache that only lives for the run) if the workload is sprayed out across parallel or distributed nodes as needed.
Something like:
pipeline {
agent any
stages {
stage('first stage') {
steps {
// Write out any environment variables you like to a temporary file
sh 'echo export FOO=baz > myenv'
// Stash away for later use
stash 'myenv'
}
}
stage ("later stage") {
steps {
// Unstash the temporary file and apply it
unstash 'myenv'
// use the unstashed vars
sh 'source myenv && echo $FOO'
}
}
}
}