I have a scripted pipeline and I'd like to execute different operations:
every day: run tests
every weekend: run very long static analysis task
I know I can define multiple triggers with
properties(
pipelineTriggers([cron("0 12 * * *"), cron("* * * * 6")])
)
But I don't know how I can then define the job later
if (???) {
sh "run complex task"
} else if (???) {
sh "run tests"
}
How can I find out which of the cron rules triggered my task?
I believe you can't get to the cron information during the build. TimerTriggerCause contains only information that the build was triggered by timer.
node {
properties([
pipelineTriggers([cron("* * * * *")])
])
def timeTriggerCause = currentBuild.rawBuild.getCause(hudson.triggers.TimerTrigger.TimerTriggerCause)
println timeTriggerCause?.getShortDescription()
}
Couple of solutions:
Check date during the build
Use multiple pipelines. You can separate all the logic into one pipeline with boolean parameter (i.e. RunComplexTask). Other pipelines (triggered by timer) would call this pipeline and pass proper value for boolean parameter.
EDIT: I've added example of multiple pipeline setup
PIPELINE_RUN_COMPLEX_TASK:
node {
properties([pipelineTriggers([cron('* * * * 6')])])
build job: 'PIPELINE_MAIN', parameters: [booleanParam(name: 'RunComplexTask', value: true)]
}
PIPELINE_RUN_TESTS:
node {
properties([pipelineTriggers([cron('0 12 * * *')])])
build job: 'PIPELINE_MAIN', parameters: [booleanParam(name: 'RunComplexTask', value: false)]
}
PIPELINE_MAIN:
if(RunComplexTask.toBoolean())
{
echo "Running complex task"
}
else
{
echo "Running tests"
}
Pipeline main has this boolean parameter I've mentioned.
Related
I have 3 downstream build jobs which are triggered when 1 upstream job 'Project U' has been built successfully. Example:
triggers {
pollSCM('H/5 * * * *')
upstream(upstreamProjects: 'Project U', threshold: hudson.model.Result.SUCCESS)
}
This works as expected, however, if code changes are committed to all parts at the same time, the upstream and downstream builds start building simultaneously.
I want to avoid this, because the downstream builds will run twice, and the first run is quite useless as the upstream commit has not been built yet. So I would like to configure the downstream jobs to block their build while the upstream job is building.
I know how to do this in Jenkins Freestyle job in the user interface (also see this answer):
But I cannot find how to do this in a Jenkins declarative pipeline?
This approach works:
waitUntil {
def job = Jenkins.instance.getItemByFullName("Project U")
!job.isBuilding() && !job.isInQueue()
}
When this downstream is started, it will check if the upstream job is either active or queued, and if so, will wait until its build has finished.
I haven't been able to find out how to programmatically access the current job's upstream job(s), so the job names need to be copied. (There is a method getBuildingUpstream(), which would be much more convenient, but I haven't found a way to obtain the current Project object from the Job instance.)
I finally ended up creating this function in the Jenkins shared library:
/*
vars/waitForJobs.groovy
Wait until none of the upstream jobs is building or being queued any more
Parameters:
upstreamProjects String with a comma separated list of Jenkins jobs to check
(use same format as in upstream trigger)
*/
def call( Map params) {
projects = params['upstreamProjects'].split(', *')
echo 'Checking the following upstream projects:'
if ( projects.size() == 0) {
echo 'none'
} else {
projects.each {project ->
echo "${project}"
}
}
waitUntil {
def running = false
projects.each {project ->
def job = Jenkins.instance.getItemByFullName(project)
if (job == null) {
error "Project '${project} not found"
}
if (job.isBuilding()) {
echo "Waiting for ${project} (executing)"
running = true
}
if (job.isInQueue()) {
echo "Waiting for ${project} (queued for execution)"
running = true
}
}
return !running
}
}
The nice thing is that I can just copy the parameter from the upstream trigger, because it uses the exact same format. Here's an example of how it looks like:
pipeline {
[...]
triggers {
pollSCM('H/5 * * * *')
upstream(upstreamProjects: 'Project U1, Project U2, Project U3', threshold: hudson.model.Result.SUCCESS)
}
[...]
stages {
stage('Wait') {
steps {
script{
// Wait for upstream jobs to finish
waitForJobs(upstreamProjects: 'Project U1, Project U2, Project U3')
}
}
[...]
}
}
}
I would like to run my Jenkins multibranch pipeline regulary during a week. Therefore I tried to set the Cron property for my pipeline directly in the Jenkinsfile as follows:
#Library('pipelines#master) _
properties([pipelineTriggers([cron('*/5 * * * *')])])
runPipeline()
In can see in the output of the build, that the properties step has been executed, but the pipeline does not start automatically.
Did I configure the Cron trigger correctly? How can I check if the trigger has been configured?
You can set the cron trigger in declarative way like below.
pipeline {
agent any
triggers{
cron('*/5 * * * *')
}
stages {
stage ("Test Stage1"){
steps {
script {
echo "Hello Test Stage1"
}
}
}
stage("Test Stage2"){
steps{
echo "Hello Test Stage2"
}
}
}
}
i have here one pipeline that running test against app with specific environment variable and triggers at specific time.
what i want to do is to use the same pipeline to be executed periodically at different times but each build is with different environment variable.
here is sample of my jenkinsfile the run against only set of environment available at certain one time
pipeline {
environment {
mvnHome = tool name: 'myMvn', type: 'maven'
mvnCMD = "${mvnHome}/bin/mvn"
APP_NAME = 'test'
APP_PACKAGE = 'test1'
APP_ACTIVITY = 'test2'
}
agent {
node {
label 'master'
}
}
triggers {
cron('15 20 * * *')
}
stages {
stage('SCM Checkout') {
steps {
git(branch: 'APP', url: 'https://gitlab.test.ba/amrka/framework.git', poll: true, credentialsId: 'GitlabCred')
}
}
stage('Testing') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh(label: 'Test Process', script: "${mvnCMD} test")
}
}
}
}
}
You can use parameterized-scheduler plugin. In Jenkins dashboard, goto
Manage Jenkins -> Manage Plugins -> Parameterized Scheduler.
To schedule a job at different times that needs to use different environments, you have to use the parameterized-scheduler plugin.
Parameter1
H/20 * * * * %Parameter1 where Parameter1 is your environment variable for a specific time you want.
Parameter2
H/30 * * * * %Parameter2 where Parameter2 is your a different environment variable for a specific time you want.
The parameters should be setup already for the parameterized-scheduler plugin to work. It can be used only for jobs with parameters.
Remember you have to have your parameters already setup because the plugin is visible only for jobs with parameters. See more documentation including usage examples in the README.
We would like to run the Jenkins pipeline:
nightly, but only if there was any commit in the chosen branches (dev or master)
and after every commit
With the caveat that nightly build should also tigger additional step (long-running integration tests).
Is it possible? How can we configure this in Jenkinsfile?
So, your question boils down to several:
How can I run a nightly build?
How can I stop a build if no commits were made in the last 24 hours?
How can I run a "long-running" stage, but not always?
Let's address these one by one.
You can have a nightly build by using cron or parameterizedCron, if you install the ParameterizedCron plugin. As you seem to want to run your job in all the branches, this may look like e.g.
pipeline {
agent any
triggers {
// schedule a nightly build at random time after 0am on dev branch, 1am on master
cron(env.BRANCH_NAME == 'dev' ? '0 * * * *' : '')
cron(env.BRANCH_NAME == 'master' ? '1 * * * *' : '')
To address the other points, you need to know why your build is running. It may be triggered by a person, or cron, or a commit. Once you make sure it's cron that triggered the build, you may want to explore git log for the time of the latest commit.
def currentBuildReason() {
def timerCause = currentBuild.rawBuild.getCause(hudson.triggers.TimerTrigger.TimerTriggerCause)
if (timerCause) { echo "Build reason: Build was started by timer"; return "TIMER" }
timerCause = currentBuild.rawBuild.getCause(org.jenkinsci.plugins.parameterizedscheduler.ParameterizedTimerTriggerCause)
if (timerCause) { echo "Build reason: Build was started by parameterized timer"; return "TIMER" }
def userCause = currentBuild.rawBuild.getCause(hudson.model.Cause$UserIdCause)
if (userCause) { echo "Build reason: Build was started by user"; return "USER" }
println "here are the causes"
echo "${currentBuild.buildCauses}"
return "UNKNOWN"
}
and later:
if ( currentBuildReason() == "TIMER" and env.BRANCH == "master" ) {
def gitLog = sh script: "git log", returnStdout: true
// look for the date, and stop the build
Finally, you can run steps on condition. One way to do it is to define a parameter for your job, indicating whether the integration tests should run. This will allow a person to run them even if not nightly, when they select the box. E.g.
pipeline {
parameters {
booleanParam(name: 'RUN_LONG_TEST', defaultValue: false,
description: "Should long-running tests execute?")
}
and later
stage('Integration tests') {
agent { node { label "integration_test_slave" }}
when {
beforeAgent true
expression {params.RUN_LONG_TEST}
}
steps {
// run integration test
}
}
To schedule the nightly with the param selected, you need the parameterizedCron plugin:
pipeline {
agent any
triggers {
// schedule a nightly build at random time after 0am on dev branch, 1am on master
parameterizedCron(env.BRANCH_NAME == 'dev' ? '0 * * * * % RUN_LONG_TEST=true' : '')
parameterizedCron(env.BRANCH_NAME == 'master' ? '1 * * * * % RUN_LONG_TEST=true'' : '')
Alternatively, you may disregard the parameter if only the nightlies should run it, and analyze the value of currentBuildReason(). When it equals to TIMER, these tests should run.
I have three pipeline projects, project-a, project-b and project-c. project-c takes a parameter. On successful completion of either project-a or project-b I want to trigger a build of project-c with a parameter.
I can do this in project-a and project-b with this code in the pipeline:
stage('trigger-project-c') {
def job = build job: 'project-c', parameters: [[$class: 'StringParameterValue', name: 'MY_PARAM', value: 'somevalue']]
}
But this requires two executors. I want project-a or project-b to completely finish before running project-c with parameters.
Your pipeline most likely looks like this:
node {
stage('build') {
// sh "make"
}
// ...
stage('trigger-project-c') {
def job = build job: 'project-c', parameters: [[$class: 'StringParameterValue', name: 'MY_PARAM', value: 'somevalue']]
}
}
By wrapping everything inside the node closure, the downstream job project-c is triggered inline, without the upstream job being paused / releasing an executor.
Therefore, things that do essentially nothing for a long time should not be wrapped within a node step, in order to not block an executor. A very similar case is when using the input step to wait for user feedback.
Instead, your pipeline should look e.g. as follows, which is - so to say - the best practice (as you don't block your executor):
stage('build') {
node {
// sh "make"
}
}
// or
node {
stage('build') {
// sh "make"
}
stage('unit') {
// sh "make"
}
} // node
// note: the following code is _not_ wrapped inside a `node` step
stage('trigger-project-c') {
def job = build job: 'project-c', parameters: [[$class: 'StringParameterValue', name: 'MY_PARAM', value: 'somevalue']]
}
There is no need to wrap the build step within a node, i.e., block an executor for it. For other steps (like sh), pipeline execution would trigger an error and remind you that it cannot be run outside of a node allocation.
Add the parameter wait: false to the build step to continue pipeline execution without waiting for the downstream job.
EDIT: this would help, if you don't care for the success of the other downstream job for this pipeline. If you need to wait until it is finished and then continue the own (upstream job's) pipeline, then see my other answer.