When a new merge is made the previous job keeps building keeping the new one in queue.
What I Want: When a new merge is made, I want jenkin to focus on that job stopping the previous build using powershell or cmd.
Is that possible.
Screenshot of a problem
Thank You in advance.
You can use the sample to abort the previous job:
pipeline {
agent any
stages {
stage('Abort previous running builds') {
steps {
abortPreviousRunningBuilds()
}
}
stage('Build') {
steps {
sleep(5)
}
}
stage('Test') {
steps {
sleep(5)
}
}
}
}
def abortPreviousRunningBuilds() {
def previousBuild = currentBuild.getRawBuild().getPreviousBuildInProgress()
while (previousBuild != null) {
if (previousBuild.isInProgress()) {
def executor = previousBuild.getExecutor()
if (executor != null) {
echo ">> Aborting older build #${previousBuild.number}"
executor.interrupt(Result.ABORTED, new CauseOfInterruption.UserInterruption("Aborted by newer build #${currentBuild.number}"))
}
}
previousBuild = previousBuild.getPreviousBuildInProgress()
}
}
If you get some RejectedAccessException, you need to allow in-process script approval by this way.
Update
Or you can use the solution: install Jenkins plugin - Pipeline: Milestone Step and write like this:
pipeline {
agent any
stages {
stage('Abort previous running builds') {
steps {
abortPreviousRunningBuilds()
}
}
stage('Build') {
steps {
sleep(5)
}
}
stage('Test') {
steps {
sleep(5)
}
}
}
}
def abortPreviousRunningBuilds() {
def buildNumber = env.BUILD_NUMBER as int
if (buildNumber > 1) milestone(buildNumber - 1)
milestone(buildNumber)
}
Related
I was assigned to create CI/CD workflow for our project combining Openshift + Jenkins pipeline. I can build by using following:
stage('build') {
steps {
script {
openshift.withCluster() {
openshift.withProject() {
def buildSelector = openshift.selector("bc", "test")
buildSelector.startBuild()
buildSelector.logs('-f')
}
}
} // script
} // steps
} // stage
But I want to also use dotnet restore and other commands but pipeline returns:
/var/lib/jenkins/jobs/test-namespace/jobs/test-pipeline/workspace#tmp/durable-4265d26a/script.sh: line 1: dotnet: command not found
also when trying to use
agent {
docker { image 'mcr.microsoft.com/dotnet/aspnet:3.1' }
}
/var/lib/jenkins/jobs/test-namespace/jobs/test-pipeline/workspace#tmp/durable-4265d26a/script.sh: line 1: docker: command not found
This is my current script
pipeline {
agent {
node {
label ''
}
}
options {
// set a timeout of 20 minutes for this pipeline
timeout(time: 20, unit: 'MINUTES')
}
stage('build') {
steps {
script {
openshift.withCluster() {
openshift.withProject() {
def buildSelector = openshift.selector("bc", "test")
buildSelector.startBuild()
buildSelector.logs('-f')
}
}
} // script
} // steps
} // stage
stage('clean'){
steps {
script{
echo "${workspace} I want to call dotnet clean"
}
} // steps
} // stage
stage('restore'){
steps {
script{
echo "${workspace} I want to call dotnet restore"
}
} // steps
} // stage
stage('tests'){
steps {
script{
echo "${workspace} I want to call dotnet test *.sln"
}
} // steps
} // stage
stage('deploy') {
steps {
script {
openshift.withCluster() {
openshift.withProject() {
def deploySelector = openshift.selector("dc", "test")
deploySelector.startDeploy()
deploySelector.logs('-f')
}
}
} // script
} // steps
} // stage
}
}
How can I install those binaries to Jenkins? I am using minishift on MacOS.
I'm creating a declarative Jenkins pipeline, that looks like this:
pipeline {
agent {
label 'mylabel'
}
stages {
stage('Install dependencies') {
milestone()
steps {
sh "yarn install"
}
}
stage('Lint files') {
steps {
sh "eslint src"
}
}
stage('Create bundle') {
steps {
sh "yarn run build:server"
sh "yarn run build:client"
}
}
stage('Publish') {
steps {
timeout(time: 15, unit: 'SECONDS') {
input(message: 'Deploy this build to QA?')
}
// deployment steps
}
}
}
}
It works great, however, if the timeout step fails (because we don't want to deploy this build, or nobody is there to push the button, and so on), the build is marked with status "aborted". Unfortunately this means that for example Github marks our pull requests as "checks failing".
Is there a way to declare the build with the status that it had before the timeout() step? Eg. if the build was a success up until the timeout step, it should be marked as success, even if the timeout happens.
Since all you want is to let the build abort without marking it as failed, you can just add a simple try/catch to your code.
stage('Publish') {
steps {
script {
def proceed = true
try {
timeout(time: 15, unit: 'SECONDS') {
input(message: 'Deploy this build to QA?')
}
} catch (err) {
proceed = false
}
if(proceed) {
// deployment steps
}
}
}
}
If a user aborts the build or it times out, the error is suppressed, the build is still a success, and the deployment steps won't execute.
We have a situation where we don't want to start a build if there are no user commits. Because of a bug in the scm trigger prevention on message or user.
What we then do is fail the build with a NOT_BUILT result.
Maybe this will also work for your situation
Try the following in your script
try {
stage ('wait') {
timeout(time: 15, unit: 'SECONDS') {
input(message: 'Deploy this build to QA?')
}
}
} catch (err) {
def user = err.getCauses()[0].getUser()
if('SYSTEM' == user.toString()) { //timeout
currentBuild.result = "SUCCESS"
}
}
If you don't want the job to be marked as "aborted" or "failed":
For a Declarative pipeline (not a scripted) you would do something like this
stage('Foo') {
steps {
script {
env.PROCEED_TO_DEPLOY = 1
try {
timeout(time: 2, unit: 'MINUTES') {
// ...
}
} catch (err) {
env.PROCEED_TO_DEPLOY = 0
}
}
}
}
stage('Bar') {
when {
expression {
env.PROCEED_TO_DEPLOY == 1
}
}
//rest of your pipeline
stage "Bar" would be skipped but as for the rest of the stages the job will be marked as passed (assuming nothing wrong happened before)
You could probably adjust currentBuild.result in post section of declarative pipeline.
E.g.
pipeline {
stages {
...
}
post {
aborted {
script {
currentBuild.result = 'SUCCESS'
}
}
}
}
I have a multibranch pipeline with a Jenkinsfile in my repo and I am able to have my CI workflow (build & unit tests -> deploy-dev -> approval -> deploy-QA -> approval -> deploy-prod) on every commit.
What I would like to do is add SonarQube Analysis on nightly builds in the first phase build & unit tests.
Since my build is triggerd by Gitlab I have defined my pipeline triggers as follow :
pipeline {
...
triggers {
gitlab(triggerOnPush: true, triggerOnMergeRequest: true, branchFilterType: 'All')
}
...
}
To setup my nightly build I have added
triggers {
...
cron('H H * * *')
}
But now, how to execute analysis step if we are only building the job triggered by the cron expression at night ?
My simplified build stage looks as follow :
stage('Build & Tests & Analysis') {
// HERE THE BEGIN SONAR ANALYSIS (to be executed on nightly builds)
bat 'msbuild.exe ...'
bat 'mstest.exe ...'
// HERE THE END SONAR ANALYSIS (to be executed on nightly builds)
}
There is the way how to get build trigger information. It is described here:
https://jenkins.io/doc/pipeline/examples/#get-build-cause
It is good for you to check this as well:
how to get $CAUSE in workflow
Very good reference for your case is https://hopstorawpointers.blogspot.com/2016/10/performing-nightly-build-steps-with.html. Here is the function from that source that exactly matches your need:
// check if the job was started by a timer
#NonCPS
def isJobStartedByTimer() {
def startedByTimer = false
try {
def buildCauses = currentBuild.rawBuild.getCauses()
for ( buildCause in buildCauses ) {
if (buildCause != null) {
def causeDescription = buildCause.getShortDescription()
echo "shortDescription: ${causeDescription}"
if (causeDescription.contains("Started by timer")) {
startedByTimer = true
}
}
}
} catch(theError) {
echo "Error getting build cause"
}
return startedByTimer
}
This works in declarative pipeline
when {
triggeredBy 'TimerTrigger'
}
For me the easiest way is to define a cron in build trigger and verify the hour on the nightly stage using a when expression:
pipeline {
agent any
triggers {
pollSCM('* * * * *') //runs this pipeline on every commit
cron('30 23 * * *') //run at 23:30:00
}
stages {
stage('nightly') {
when {//runs only when the expression evaluates to true
expression {//will return true when the build runs via cron trigger (also when there is a commit at night between 23:00 and 23:59)
return Calendar.instance.get(Calendar.HOUR_OF_DAY) in 23
}
}
steps {
echo "Running the nightly stage only at night..."
}
}
}
}
You could check the build cause like so:
stage('Build & Tests & Analysis') {
when {
expression {
for (Object currentBuildCause : script.currentBuild.rawBuild.getCauses()) {
return currentBuildCause.class.getName().contains('TimerTriggerCause')
}
}
steps {
bat 'msbuild.exe ...'
bat 'mstest.exe ...'
}
}
}
However, this requires the following entries in script-approval.xml:
<approvedSignatures>
<string>method hudson.model.Run getCauses</string>
<string>method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild</string>
</approvedSignatures>
This can also be approved via https://YOURJENKINS/scriptApproval/.
Hopefully, this won't be necessary after JENKINS-41272 is fixed.
Until then, a workaround could be to check the hour of day in the when expression (keep in mind that these times refer to to the timezone of Jenkins)
when { expression { return Calendar.instance.get(Calendar.HOUR_OF_DAY) in 0..3 } }
I've found a way, which does not use "currentBuild.rawBuild" which is restricted. Begin your pipeline with:
startedByTimer = false
def buildCauses = "${currentBuild.buildCauses}"
if (buildCauses != null) {
if (buildCauses.contains("Started by timer")) {
startedByTimer = true
}
}
Test the boolean where you need it, for example:
stage('Clean') {
when {
anyOf {
environment name: 'clean_build', value: 'Yes'
expression { (startedByTimer == true) }
}
}
steps {
echo "Cleaning..."
...
Thanks to this you can now do this without needing to the the use the non-whitelisted currentBuild.getRawBuild().getCauses() function which can give you Scripts not permitted to use method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild depending on your setup:
#NonCPS
def isJobStartedByTimer() {
def startedByTimer = false
try {
def buildCauses = currentBuild.getBuildCauses()
for ( buildCause in buildCauses ) {
if (buildCause != null) {
def causeDescription = buildCause.shortDescription
echo "shortDescription: ${causeDescription}"
if (causeDescription.contains("Started by timer")) {
startedByTimer = true
}
}
}
} catch(theError) {
echo "Error getting build cause"
}
return startedByTimer
}
I'm trying to create a declarative pipeline which does a number (configurable via parameter) jobs in parallel, but I'm having trouble with the parallel part.
Basically, for some reason the below pipeline generates the error
Nothing to execute within stage "Testing" # line .., column ..
and I cannot figure out why, or how to solve it.
import groovy.transform.Field
#Field def mayFinish = false
def getJob() {
return {
lock("finiteResource") {
waitUntil {
script {
mayFinish
}
}
}
}
}
def getFinalJob() {
return {
waitUntil {
script {
try {
echo "Start Job"
sleep 3 // Replace with something that might fail.
echo "Finished running"
mayFinish = true
true
} catch (Exception e) {
echo e.toString()
echo "Failed :("
}
}
}
}
}
def getJobs(def NUM_JOBS) {
def jobs = [:]
for (int i = 0; i < (NUM_JOBS as Integer); i++) {
jobs["job{i}"] = getJob()
}
jobs["finalJob"] = getFinalJob()
return jobs
}
pipeline {
agent any
options {
buildDiscarder(logRotator(numToKeepStr:'5'))
}
parameters {
string(
name: "NUM_JOBS",
description: "Set how many jobs to run in parallel"
)
}
stages {
stage('Setup') {
steps {
echo "Setting it up..."
}
}
stage('Testing') {
steps {
parallel getJobs(params.NUM_JOBS)
}
}
}
}
I've seen plenty of examples doing this in the old pipeline, but not declarative.
Anyone know what I'm doing wrong?
At the moment, it doesn't seem possible to dynamically provide the parallel branches when using a Declarative Pipeline.
Even if you have a stage prior where, in a script block, you call getJobs() and add it to the binding, the same error message is thrown.
In this case you'd have to fall back to using a Scripted Pipeline.
I need to launch a dynamic set of tests in a declarative pipeline.
For better visualization purposes, I'd like to create a stage for each test.
Is there a way to do so?
The only way to create a stage I know is:
stage('foo') {
...
}
I've seen this example, but I it does not use declarative syntax.
Use the scripted syntax that allows more flexibility than the declarative syntax, even though the declarative is more documented and recommended.
For example stages can be created in a loop:
def tests = params.Tests.split(',')
for (int i = 0; i < tests.length; i++) {
stage("Test ${tests[i]}") {
sh '....'
}
}
As JamesD suggested, you may create stages dynamically (but they will be sequential) like that:
def list
pipeline {
agent none
options {buildDiscarder(logRotator(daysToKeepStr: '7', numToKeepStr: '1'))}
stages {
stage('Create List') {
agent {node 'nodename'}
steps {
script {
// you may create your list here, lets say reading from a file after checkout
list = ["Test-1", "Test-2", "Test-3", "Test-4", "Test-5"]
}
}
post {
cleanup {
cleanWs()
}
}
}
stage('Dynamic Stages') {
agent {node 'nodename'}
steps {
script {
for(int i=0; i < list.size(); i++) {
stage(list[i]){
echo "Element: $i"
}
}
}
}
post {
cleanup {
cleanWs()
}
}
}
}
}
That will result in:
dynamic-sequential-stages
If you don't want to use for loop, and generated pipeline to be executed in parallel then, here is an answer.
def jobs = ["JobA", "JobB", "JobC"]
def parallelStagesMap = jobs.collectEntries {
["${it}" : generateStage(it)]
}
def generateStage(job) {
return {
stage("stage: ${job}") {
echo "This is ${job}."
}
}
}
pipeline {
agent none
stages {
stage('non-parallel stage') {
steps {
echo 'This stage will be executed first.'
}
}
stage('parallel stage') {
steps {
script {
parallel parallelStagesMap
}
}
}
}
}
Note that all generated stages will be executed into 1 node.
If you are willing to executed the generated stages to be executed into different nodes.
def agents = ['master', 'agent1', 'agent2']
// enter valid agent name in array.
def generateStage(nodeLabel) {
return {
stage("Runs on ${nodeLabel}") {
node(nodeLabel) {
echo "Running on ${nodeLabel}"
}
}
}
}
def parallelStagesMap = agents.collectEntries {
["${it}" : generateStage(it)]
}
pipeline {
agent none
stages {
stage('non-parallel stage') {
steps {
echo 'This stage will be executed first.'
}
}
stage('parallel stage') {
steps {
script {
parallel parallelStagesMap
}
}
}
}
}
You can of course add more than 1 parameters and can use collectEntries for 2 parameters.
Please remember return in function generateStage is must.
#Jorge Machado: Because I cannot comment I had to post it as an answer. I've solved it recently. I hope it'll help you.
Declarative pipeline:
A simple static example:
stage('Dynamic') {
steps {
script {
stage('NewOne') {
echo('new one echo')
}
}
}
}
Dynamic real-life example:
// in a declarative pipeline
stage('Trigger Building') {
when {
environment(name: 'DO_BUILD_PACKAGES', value: 'true')
}
steps {
executeModuleScripts('build') // local method, see at the end of this script
}
}
// at the end of the file or in a shared library
void executeModuleScripts(String operation) {
def allModules = ['module1', 'module2', 'module3', 'module4', 'module11']
allModules.each { module ->
String action = "${operation}:${module}"
echo("---- ${action.toUpperCase()} ----")
String command = "npm run ${action} -ddd"
// here is the trick
script {
stage(module) {
bat(command)
}
}
}
}
You might want to take a look at this example - you can have a function return a closure which should be able to have a stage in it.
This code shows the concept, but doesn't have a stage in it.
def transformDeployBuildStep(OS) {
return {
node ('master') {
wrap([$class: 'TimestamperBuildWrapper']) {
...
} } // ts / node
} // closure
} // transformDeployBuildStep
stage("Yum Deploy") {
stepsForParallel = [:]
for (int i = 0; i < TargetOSs.size(); i++) {
def s = TargetOSs.get(i)
def stepName = "CentOS ${s} Deployment"
stepsForParallel[stepName] = transformDeployBuildStep(s)
}
stepsForParallel['failFast'] = false
parallel stepsForParallel
} // stage
Just an addition to what #np2807 and #Anton Yurchenko have already presented: you can create stages dynamically and run the in parallel by simply delaying list of stages creation (but keeping its declaration), e.g. like that:
def parallelStagesMap
def generateStage(job) {
return {
stage("stage: ${job}") {
echo "This is ${job}."
}
}
}
pipeline {
agent { label 'master' }
stages {
stage('Create List of Stages to run in Parallel') {
steps {
script {
def list = ["Test-1", "Test-2", "Test-3", "Test-4", "Test-5"]
// you may create your list here, lets say reading from a file after checkout
// personally, I like to use scriptler scripts and load the as simple as:
// list = load '/var/lib/jenkins/scriptler/scripts/load-list-script.groovy'
parallelStagesMap = list.collectEntries {
["${it}" : generateStage(it)]
}
}
}
}
stage('Run Stages in Parallel') {
steps {
script {
parallel parallelStagesMap
}
}
}
}
}
That will result in Dynamic Parallel Stages:
I use this to generate my stages which contain a Jenkins job in them.
build_list is a list of Jenkins jobs that i want to trigger from my main Jenkins job, but have a stage for each job that is trigger.
build_list = ['job1', 'job2', 'job3']
for(int i=0; i < build_list.size(); i++) {
stage(build_list[i]){
build job: build_list[i], propagate: false
}
}
if you are using Jenkinsfile then, I achieved it via dynamically creating the stages, running them in parallel and also getting Jenkinsfile UI to show separate columns. This assumes parallel steps are independent of each other (otherwise don't use parallel) and you can nest them as deep as you want (depending upon the # of for loops you'll nest for creating stages).
Jenkinsfile Pipeline DSL: How to Show Multi-Columns in Jobs dashboard GUI - For all Dynamically created stages - When within PIPELINE section see here for more.