I have asked this question on the Jenkins mailing list.
I have an upstream component, libraryA, I build, archive, and deploy via a Maven job, jobA. This works great. I have a downstream Maven job, jobB, that has a dependency on libraryA. This also works great, except…
I have a completely separate pipeline job, pipelineA specified by Jenkinsfile. Within that Jenkinsfile, I build a specific branch of libraryA I don’t want archived or deployed. In my Jenkinsfile I have “withMaven(mavenLocalRepo: ‘libraryA/.repository’, publisherStrategy: ‘EXPLICIT’)”, and inside that, “sh “””[…]mvn clean package sonar:sonar[…]””” (Any typos here are probably the fault of my typing here as I did not copy-paste. There are no errors from Jenkins when executing these steps.) I have also tried “options: [artifactsPublisher(disabled: true)]” in place of “publisherStrategy: ‘EXPLICIT’” and had the same results. I have verified when pipelineA builds libraryA, it does NOT get deployed to my remote Maven repository, and I expect it not to get deployed there. Good.
So, what happens?
Well, if I build pipelineA followed by jobB, jobB gets its copy of libraryA from pipelineA, causing the build to fail. If I then run jobA, jobB succeeds as expected.
I could change the version of libraryA in the branch pipelineA builds, but I’d rather not do that as it’s not correct for my particular use case. What else could I do? What did I miss? (I do not admin this Jenkins instance, so my access is limited in that respect.)
For clarity here is an approximation of the pipeline's Jenkinsfile:
#!groovy
pipeline {
environment {
GIT_CREDENTIAL_ID = 'git_credential_name'
}
agent any
tools {
maven 'internal_maven'
jdk 'openjdk'
git 'internal_git'
}
stages {
stage('Build Parent') {
steps {
checkoutRepos([[checkoutDir: 'parent', branch: 'gold', url: 'ssh://git#fake-git-url/parent.git']], env.GIT_CREDENTIAL_ID)
withMaven(publisherStrategy: 'EXPLICIT') {
sh """
cd parent
mvn -U clean verify
"""
}
stash name: 'parent', includes: 'pom.xml'
}
}
stage('Build libraryParent') {
steps {
unstash 'parent'
checkoutRepos([[checkoutDir: 'libraryParent', branch: 'gold', url: 'ssh://git#fake-git-url/libraryParent.git']], env.GIT_CREDENTIAL_ID)
withMaven(mavenLocalRepo: 'libraryParent/.repository', publisherStrategy: 'EXPLICIT') {
sh """
mvn install:install-file -Dpackaging=pom -Dfile=pom.xml -DpomFile=pom.xml
cd libraryParent
mvn clean verify -P jacoco sonar:sonar -U -Dsonar.host.url=https://fake-sonar-url -Dsonar.scm.provider=git
"""
}
dir('libraryParent/libraryA/target/') {
stash name: 'libraryA', includes: 'libraryA-1000-SNAPSHOT.jar'
}
}
}
stage('Build appA') {
steps {
sonarAnApp 'appA'
}
}
stage('Build appB') {
steps {
sonarAnApp 'appB'
}
}
stage('Build appC') {
steps {
sonarAnApp 'appC'
}
}
}
}
def sonarAnApp(final String appName) {
unstash 'parent'
unstash 'libraryA'
checkoutRepos([[checkoutDir: appName, branch: 'gold', url: "ssh://git#fake-git-url/${appName}.git"]], env.GIT_CREDENTIAL_ID)
withMaven(mavenLocalRepo: "$appName/.repository", publisherStrategy: 'EXPLICIT') {
sh """
mvn install:install-file -Dpackaging=pom -Dfile=pom.xml -DpomFile=pom.xml
mvn install:install-file -Dfile=libraryA-1000-SNAPSHOT.jar
cd $appName
mvn clean verify -P jacoco sonar:sonar -U -Dsonar.host.url=https://fake-sonar-url -Dsonar.scm.provider=git
"""
}
}
Related
I have created a multibranch pipeline project and thus created jenkinsfile and put that in dev branch.
In one of the stage, I have to run mvn sonar:sonar -Dsonar.scm.branch=branch-${env.BRANCH_NAME} but it's giving error as bad substitution branch-${env.BRANCH_NAME}.
I need branch-${env.BRANCH_NAME} as a branch-name so that at sonar i can see branch-dev at branches section in sonar dashboard.
if i use mvn sonar:sonar -Dsonar.scm.branch=env.BRANCH_NAME then it provides output but it will act as short-lived branch in sonar. but at sonar we want branch as long-lived branch.
!/usr/bin/env groovy
pipeline {
agent { label 'ol73_slave-jdk8u192-git' }
options {
timestamps()
timeout(time: 2, unit: 'HOURS')
buildDiscarder(logRotator(numToKeepStr: '10'))
disableConcurrentBuilds()
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Unit Test and Code Scan') {
steps {
echo "*****JUnit Tests, JaCoCo Code Coverage, & SonarQube Code Qualiy Scan*****"
withMaven(jdk: 'jdk8_u192', maven: 'maven-3.3.9', mavenSettingsConfig: '79ecf9bd-8cbc-4d5e-b7d1-200241e16b52') {
sh '''
cd DARC
mvn clean package sonar:sonar -Dsonar.host.url=***** -Dsonar.login=******* -Dsonar.exclusions=file:**/src/test/** -B -Pcoverage -Dsonar.branch.name=branch-${env.BRANCH_NAME}
'''
}
}
}
}
}
In groovy Strings are encapsulated in single quotes '' and GStrings in double quotes ""
In order to do interpolation you need to be using GStrings. In your example, this should simply be
sh """
cd DARC
mvn clean package sonar:sonar -Dsonar.host.url=***** -Dsonar.login=******* -Dsonar.exclusions=file:**/src/test/** -B -Pcoverage -Dsonar.branch.name=branch-${env.BRANCH_NAME}
"""
I want to create a Jenkins job which does following:
Git>Mvn build> copy jar to some location of server.
So this can be done using a single job or 2 jobs?
Or which is preferred way of doing this , is pipeline preferred over creating a maven job?
I have created this pipeline script, but this does not copy the current build jar to the server location, it copies the previous build artifact jar.
node {
def mvnHome
stage('Preparation') { // for display purposes
// Get some code from a GitHub repository
git 'git#github.pie.ABC.com:abcdef/BoltRepo.git'
mvnHome = tool 'M2'
}
stage('Build') {
// Run the maven build
if (isUnix()) {
sh "'${mvnHome}/bin/mvn' -Dmaven.test.failure.ignore clean package"
} else {
bat(/"${mvnHome}binmvn" -Dmaven.test.failure.ignore clean package/)
}
}
stage('Results') {
archiveArtifacts 'target/*/BoltRepo*.jar'
}
stage('Deploy Artifact') {
copyArtifacts(
projectName: currentBuild.projectName,
filter: 'target/*/BoltRepo*.jar',
fingerprintArtifacts: true,
target: '/ngs/app/boltd/bolt/bolt_components/bolt_provision/test',
flatten: true )
}
}
What is the best way of achieving this.
I haven't used the pipeline before, but I have done what you want using "ArtifactDeployer" from the "Post-build Actions" in the job's configurations
Note: you will need to install "Artifact Deployer Plug-in"
I have created very basic Multibranch Pipeline on my local Jenkins via BlueOcean UI. From default config I removed almost all behaviors except one for discovering branches. The config looks line follows:
Within Jenkinsfile I'm trying to setup following scenario:
Checkout branch
(optionally) Merge it to master branch
Build Back-end
Build Front-end
Snippet from my Jenkinsfile:
pipeline {
agent none
stages {
stage('Setup') {
agent {
label "master"
}
steps {
sh "git checkout -f ${env.BRANCH_NAME}"
}
}
stage('Merge with master') {
when {
not {
branch 'master'
}
}
agent {
label "master"
}
steps {
sh 'git checkout -f origin/master'
sh "git merge --ff-only ${env.BRANCH_NAME}"
}
}
stage('Build Back-end') {
agent {
docker {
image 'openjdk:8'
}
}
steps {
sh './gradlew build'
}
}
stage ('Build Front-end') {
agent {
docker {
image 'saddeveloper/node-chromium'
}
}
steps {
dir ('./front-end') {
sh 'npm install'
sh 'npm run buildProd'
sh 'npm run testHeadless'
}
}
}
}
}
Pipeline itself and building steps works fine, but the problem is that Jenkins adds "Check out from version control" step before each stage. The step looks for new branches, fetches refs, but also checks out current branch. Here is relevant output from full build log:
// stage Setup
> git checkout -f f067047bbdd3a5d5f9d1f2efae274bc175829595
sh git checkout -f my-branch
// stage Merge with master
> git checkout -f f067047bbdd3a5d5f9d1f2efae274bc175829595
sh git checkout -f origin/master
sh git merge --ff-only my-branch
// stage Build Back-end
> git checkout -f f067047bbdd3a5d5f9d1f2efae274bc175829595
sh ./gradlew build
// stage Build Front-end
> git checkout -f f067047bbdd3a5d5f9d1f2efae274bc175829595
sh npm install
sh npm run buildProd
sh npm run testHeadless
So as you see it effectively resets working directory to particular commit before every stage git checkout -f f067...595.
Is there any way to disable this default checkout behavior?
Or any viable option how to implement such optional merging to master branch?
Thanks!
By default, git scm will be executed in a Jenkins pipeline. You can disable it by doing:
pipeline {
agent none
options {
skipDefaultCheckout true
}
...
Also, I'd recommend take a look to other useful pipeline options https://jenkins.io/doc/book/pipeline/syntax/#options
I'm trying to create some Docker images. For that I want to use the version number specified in the Maven pom.xml file as tag. I am however rather new to the declarative Jenkins pipelines and I can't figure out how to change my environment variable so that VERSION contains the right version for all stages.
This is my code
#!groovy
pipeline {
tools {
maven 'maven 3.3.9'
jdk 'Java 1.8'
}
environment {
VERSION = '0.0.0'
}
agent any
stages {
stage('Checkout') {
steps {
git branch: 'master', credentialsId: '290dd8ee-2381-4c5b-8d33-5631d03ee7be', url: 'git#gitlab.crosslang.local:company/SOME-API.git'
sh "git clean -f && git reset --hard origin/master"
}
}
stage('Build and Test Java code') {
steps {
script {
def pom = readMavenPom file: 'pom.xml'
VERSION = pom.version
}
echo "${VERSION}"
sh "mvn clean install -DskipTests"
}
}
stage('Build Docker images') {
steps {
dir('whales-microservice/src/main/docker'){
sh 'cp ../../../target/whales-microservice-${VERSION}.jar whales-microservice.jar'
script {
docker.build "company/whales-microservice:${VERSION}"
}
}
}
}
}
}
The problem is the single quote of the statement
sh 'cp ../../../target/whales-microservice-${VERSION}.jar whales-microservice.jar'
single quotes don't expand variables in groovy: http://docs.groovy-lang.org/latest/html/documentation/#_string_interpolation
so you have to double quote your shell statement:
sh "cp ../../../target/whales-microservice-${VERSION}.jar whales-microservice.jar"
I just wanted to mention that if you have pipeline-utility-steps plugin installed you can use readMavenPom() in the environment part, too. It looks like this:
environment {
VERSION = readMavenPom().getVersion()
}
When our browser based tests fail, we take a screenshot of the browser window to better illustrate the problem. However, I don't understand how to archive them in my pipeline, because the pipeline stops after the failure. Same for the junit.xml, I'd also like to use it in error cases.
I've checked, the screenshots are generated and stored correctly.
My definition looks like this (irrelevant things mostly trimmed):
node {
stage('Build docker container') {
checkout([$class: 'GitSCM', ...])
sh "docker build -t webapp ."
}
stage('test build') {
sh "mkdir -p rspec screenshots"
sh "docker run -v /var/jenkins_home/workspace/webapp/rspec/junit.xml:/myapp/junit.xml -v /var/jenkins_home/workspace/webapp/screenshots:/myapp/tmp/capybara -v webapp bundle exec rspec"
}
stage('Results') {
junit 'rspec/junit*.xml'
archive 'screenshots/*'
}
}
You can use simple Java try/catch to avoid pipeline failure on test failure, or Jenkins catchError like this :
node {
catchError {
// Tests that might fail...
}
// Archive your tests artifacts
}
From here, you can use the post section in your pipeline:
pipeline {
agent any
stages {
stage('Build') {
...
}
stage('Test') {
...
}
}
post {
always {
archive 'build/libs/**/*.jar'
}
}
}