how to schedule parameterized pipelines to run just once with one parameter and the rest with another one? - jenkins

I have a pipeline which I just added 2 parameters to build release or debug (parameters are called release or debug). The pipeline uses cron syntax to check for changes in the SCM every 10 mins, the pipeline checks for every commit and then build release (C++ program) but I would like to build debug once a day, let's say everyday every coomit pushed from 12 to 13 will be build in debug. All of this without me having to run the pipeline and changing the parameter manually (it is set to release by default). Is there any way to do this? This is a very short version of what the pipeline looks like:
pipeline {
stages {
stage('Setup parameters') {
steps {
script {
properties([
parameters([
choice(
defaultValue: 'RELEASE',
choices: ['RELEASE', 'DEBUG'],
name: 'BUILD_CONFIG'
),
])
])
}
}
}
stage('Build release'){
when {
expression {
return params.BUILD_CONFIG == 'RELEASE'
}
}
steps{
script {
def msbuild = tool name: 'MSBuild', type: 'hudson.plugins.msbuild.MsBuildInstallation'
bat "\"${msbuild}\" /Source/project-GRDK.sln /t:Rebuild /p:configuration=\"Release Steam D3D11\""
}
}
}
stage('Build debug'){
when {
expression {
return params.BUILD_CONFIG == 'DEBUG'
}
}
steps{
script {
def msbuild = tool name: 'MSBuild', type: 'hudson.plugins.msbuild.MsBuildInstallation'
bat "\"${msbuild}\" /Source/project-GRDK.sln /t:Rebuild /p:configuration=\"Debug Steam D3D11\""
}
}
}
}
}

parameterizedCron plugin does what you need:
pipeline {
agent any
parameters {
choice(name: 'BUILD_CONFIG', choices: ['RELEASE', 'DEBUG'], defaultValue: 'RELEASE')
}
triggers {
parameterizedCron('''
1,2,3,4,5,6,7,8,9,10 * * * * % BUILD_CONFIG=RELEASE
12 * * * * % BUILD_CONFIG=DEBUG
''')
}

Another option would be to create a second job, which triggers the build job with the right parameters:
pipeline {
agent any
triggers {
cron('H 12 * * *')
}
stages {
stage('Build xxx debug') {
steps {
build job: "your-job-name-here", parameters: [
choice(name: 'BUILD_CONFIG', value: 'DEBUG')
]
}
}
}
}

It is possible to determine the cause of the build with currentBuild.rawBuild.getCause(Class<T> type). The type you are looking for is UserIdCause. Following would build a stage in case the job was not triggered by an user (manually). In this stage steps are from Build debug stage.
stage('Build debug if time triggered') {
when {
expression {
return currentBuild.rawBuild.getCause(hudson.model.Cause$UserIdCause) == null
}
}
steps {
script {
def msbuild = tool name: 'MSBuild', type: 'hudson.plugins.msbuild.MsBuildInstallation'
bat "\"${msbuild}\" /Source/project-GRDK.sln /t:Rebuild /p:configuration=\"Debug Steam D3D11\""
}
}
You will also need to add an expression to Build release and Build debug stages, in order to prevent building if the job is not triggered by an user (manually).
stage('Build release'){
when {
allOf {
expression {
return params.BUILD_CONFIG == 'RELEASE'
}
expression {
return currentBuild.rawBuild.getCause(hudson.model.Cause$UserIdCause) != null
}
}
}
...
Docu:
https://javadoc.jenkins-ci.org/hudson/model/Cause.html
https://javadoc.jenkins-ci.org/hudson/model/Run.html
How to differentiate build triggers in Jenkins Pipeline
EDIT
If you want to keep everything in one pipeline, then you need to create two new variables. Following code creates Calendar object for 12 am today and converts it to milliseconds.
Calendar date = new GregorianCalendar()
date.set(Calendar.HOUR_OF_DAY, 12);
date.set(Calendar.MINUTE, 0);
date.set(Calendar.SECOND, 0);
date.set(Calendar.MILLISECOND, 0);
def start = date.getTime().getTime()
In same way you could create a Calendar object for 1 pm today (e.g. end). With currentBuild.rawBuild.getTimestamp() you get Calendar object, when the build was scheduled. If the scheduled time is between start and end set for example a boolean variable and check it in the pipeline when block.
def buildDebug = false
def scheduled = currentBuild.rawBuild.getTimestamp().getTime().getTime()
if(scheduled > start && scheduled < end)
buildDebug = true
...
stage('Build release'){
when {
allOf {
expression {
return params.BUILD_CONFIG == 'RELEASE'
}
expression {
return currentBuild.rawBuild.getCause(hudson.model.Cause$UserIdCause) == null
}
expression {
return buildDebug == false
}
}
}
...
stage('Build debug'){
when {
allOf {
expression {
return params.BUILD_CONFIG == 'RELEASE'
}
expression {
return currentBuild.rawBuild.getCause(hudson.model.Cause$UserIdCause) == null
}
expression {
return buildDebug == true
}
}
}
How to create a Java Date object of midnight today and midnight tomorrow?

Related

How to configure Jenkinsfile to only run a specific stage when running a daily cron job?

I've set a cron job to run every night, however I only want it to run stage B within the Jenkinsfile not all of them.
pipeline {
agent any
triggers {
cron('#midnight')
}
}
stages {
stage('A') {
...
}
stage('B'){
when {
allOf {
expression { env.CHANGE_TARGET == "master" }
branch "PR-*"
}
}
steps {
sh """
echo 'running script'
make run-script
"""
}
}
stage('C') {
...
}
Without removing the conditionals in Stage B, I can't seem to figure out how to specify the cron to explicitly only run Stage B of the Jenkinsfile - I need to run that makefile script only when those conditionals are met OR during the daily midnight cron job
You can achieve what you want with the Parameterized Scheduler Plugin which enables you to define cron triggers that trigger the job with a specific environment variable, you can then use this variable as a condition to determine which step to execute.
in your case you can use that environment variable in the when directive of each stage to determine if it should run or not according to the variable.
Something like:
pipeline {
agent any
parameters {
booleanParam(name: 'MIDNIGHT_BUILD', defaultValue: 'true', description: 'Midnight build')
}
triggers {
parameterizedCron('''
0 * * * * %MIDNIGHT_BUILD=true
''')
}
stages {
stage('A') {
when {
expression { !env.MIDNIGHT_BUILD }
}
steps {
...
}
}
stage('B') {
when {
expression { env.MIDNIGHT_BUILD || env.CHANGE_TARGET == "master" }
}
steps {
sh """
echo 'running script'
make run-script
"""
}
}
stage('C') {
when {
expression { !env.MIDNIGHT_BUILD }
}
steps {
...
}
}
}
}

Jenkinsfile - How to make a cron trigger kick off only a specifc stage?

I have a Jenkinsfile with the following triggers:
triggers {
cron('0 * * * 1-5')
}
So it will trigger at the top of the hour, every hour, Monday through Friday.
In the Jenkinsfile I have a number of stages like:
stage('CI Build and push snapshot') {
when {
anyOf { branch 'PR-*';branch 'develop' }
}
.
.
.
stage('Build Release') {
when {
branch 'master'
}
.
.
.
stage('Integration Tests') {
when {
? // not sure what goes here
}
What I want to do is, when that trigger is kicked off, I only want the Integration Tests stage to run. How do I achieve this? I think with what I have now every stage is going to be run.
Thanks!
I was able to get it working using something like:
stage('CI Build and push snapshot') {
when {
anyOf { branch 'PR-*';branch 'develop' }
not {
expression { return currentBuild.rawBuild.getCause(hudson.triggers.TimerTrigger$TimerTriggerCause) }
}
}
stage('Integration Tests') {
when {
branch 'develop'
expression { return currentBuild.rawBuild.getCause(hudson.triggers.TimerTrigger$TimerTriggerCause) }
}
Note this is using shared library functions and scripted syntax (not declarative), you will need to use script {} blocks in order to implement.
For organisation purposes, I put this into its own function in a shared library file called jobCauses.groovy under /vars, you can keep it in-line if you like, or put it at the bottom of the Jenkinsfile etc.
/**
* Checks if job cause is Cron
*
* #return boolean
*/
boolean hasTriggeredCause() {
List jobCauses = currentBuild.rawBuild.getCauses().collect { it.getClass().getCanonicalName().tokenize('.').last() }
return jobCauses.contains('TimerTriggerCause')
}
Then in your pipeline:
stage('Integration Tests') {
script {
if ( jobCauses.hasTriggeredCause() ) {
//do the thing
}
}
}

Access which stage failed in previous Jenkins build

I have written a Jenkinsfile script which gets whether documents are updated or code is updated in the current Github commit and starts all the stages accordingly. If only documents are updated I don't run the code testing stage again.
So now if the previous build failed and now in the current Git commit only documents are updated then it will not run the code testing stage. So I want a method/way to know which stage failed during the last Jenkins build and if needed run the current Jenkins build.
For example if the code testing stage failed in the previous build, I'll need to run the code testing stage for this build, otherwise I can just run the documents zipping stage.
As a workaround to get failed stages from Jenkins build such function can be used. I could not find a simpler way to do it. But this code requires to run without Groovy sandbox or you need to whitelist a lot of Jenkins method signatures (which is not recommeded). Also blueocean plugin has to be installed.
import io.jenkins.blueocean.rest.impl.pipeline.PipelineNodeGraphVisitor
import io.jenkins.blueocean.rest.impl.pipeline.FlowNodeWrapper
import org.jenkinsci.plugins.workflow.flow.FlowExecution
import org.jenkinsci.plugins.workflow.graph.FlowNode
import org.jenkinsci.plugins.workflow.job.WorkflowRun
#NonCPS
List getFailedStages(WorkflowRun run) {
List failedStages = []
FlowExecution exec = run.getExecution()
PipelineNodeGraphVisitor visitor = new PipelineNodeGraphVisitor(run)
def flowNodes = visitor.getPipelineNodes()
for (node in flowNodes) {
if (node.getType() != FlowNodeWrapper.NodeType.STAGE ) { continue; }
String nodeName = node.getDisplayName()
def nodeResult = node.getStatus().getResult()
println String.format('{"displayName": "%s", "result": "%s"}',
nodeName, nodeResult)
def resultSuccess = io.jenkins.blueocean.rest.model.BlueRun$BlueRunResult.SUCCESS
if (nodeResult != resultSuccess) {
failedStages.add(nodeName)
}
}
return failedStages
}
// Ex. Get last build of "test_job"
WorkflowRun run = Jenkins.instance.getItemByFullName("test_job")._getRuns()[0]
failedStages = getFailedStages(run)
I thing it could fit. Use buildVariables from previous build, timeout \ input in case You need to change something, try \ catch for setup stages status. Code example:
// yourJob
// with try/catch block
def stageOneStatus;
def stageTwoStatus;
def stageThreeStatus;
pipeline {
agent any
stages {
stage("STAGE 1") {
// For initial run every stage
when { expression { params.stageOne == "FAILURE" } }
steps {
script {
try {
// make thing
} catch (Exception e) {
stageOneStatus = "FAILURE";
}
}
}
}
stage("STAGE 2") {
when { expression { params.stageTwo == "FAILURE" } }
steps {
script {
try {
// make thing
} catch (Exception e) {
stageTwoStatus = "FAILURE";
}
}
}
}
stage("STAGE 3") {
when { expression { params.stageThree == "FAILURE" } }
steps {
script {
try {
// make thing
} catch (Exception e) {
stageThreeStatus = "FAILURE";
}
}
}
}
}
}
// Checking JOB
def pJob;
pipeline {
agent any
stages {
// Run job with inheriting variable from build
stage("Inheriting job") {
steps {
script {
pJob = build(job: "yourJob", parameters: [
[$class: 'StringParameterValue', name: 'stageOne', value: 'FAILURE'],
[$class: 'StringParameterValue', name: 'stageTwo', value: 'FAILURE'],
[$class: 'StringParameterValue', name: 'stageThree', value: 'FAILURE']
], propagate: false)
if (pJob.result == 'FAILURE') {
error("${pJob.projectName} FAILED")
}
}
}
}
// Wait for fix, and re run job
stage ('Wait for fix') {
timeout(time: 24, unit: 'HOURS') {
input "Ready to rerun?"
}
}
// Re run job after changes in code
stage("Re-run Job") {
steps {
script {
build(
job: "yourJob",
parameters: [
[$class: 'StringParameterValue',name: 'stageOne',value: pJob.buildVariables.stageOneStatus ],
[$class: 'StringParameterValue',name: 'stageTwo',value: pJob.buildVariables.stageTwoStatus ],
[$class: 'StringParameterValue',name: 'stageThree',value: pJob.buildVariables.stageThreeStatus ]
]
)
}
}
}
}
}

Declarative Pipeline When Conditions

I'm in need of help with the when condition on Jenkins. I want to take the words FULL_BUILD from the merge request title and then execute different stages based on whether it is a FULL_BUILD or someone just submitting a merge request to master that doesn't need to go through Veracode, SonarQube, etc etc (these stages are not pasted in as they are just repeats of the when conditions below). However, I have to repeat this crazy when condition on EVERY stage, as well as sometimes creating a special stage that only executes on FULL_BUILD or "regular" builds.
Has anyone created a #NonCPS script to set a true/false variable? Or is there a crafty way to execute the script on startup to set a reusable variable?
I want to have the users execute everything from their Gitlab MR and not have to go to Jenkins to hit a button (hence I do not use a parameter of boolean).
pipeline {
agent {
node {
label 'master'
}
}
parameters{
//I am trying to pull the information for a full build from the Merge Request
}
environment {
//Assume random variables all work fine
}
options {
skipDefaultCheckout()
gitLabConnection('GitLab_Generic')
timeout(time: 60, unit: 'MINUTES')
}
triggers {
gitlab(triggerOnPush: true, triggerOnMergeRequest: true, branchFilterType: 'All')
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Build') {
steps {
dir("${env.WORKSPACE}\\${params.PROJECT_NAME}") {
bat 'nuget.exe restore %PROJECT_NAME%.sln'
}
dir("${env.WORKSPACE}\\${params.PROJECT_NAME}\\build") {
bat "\"${tool 'msbuild'}\" %COMPONENT%.XML /p:Configuration=Debug "
}
dir("${env.WORKSPACE}") {
echo "Creating a Build status file"
writeFile file: "output/MR_Title.txt", text: "BUILD STATUS:"
}
}
}
stage('Check MR FULL_BUILD' ){
when {
branch 'master'
}
steps{
dir("${env.WORKSPACE}") {
//writeFile file: "MR_Title.txt", text: "BUILD STATUS:"
powershell '& "./build/scripts/MergeRequestAPI.ps1" -GIT_CREDENTIALS $env:GIT_API_TOKEN -PROJECT_ID $env:GIT_PROJECT_ID | Out-File output/MR_Title.txt -Encoding utf8"'
}
}
}
stage('Package Snapshot') {
when {
allOf {
branch 'master'
not {
expression {
return readFile('output/MR_Title.txt').contains("FULL BUILD")
}
}
}
}
steps {
dir("${env.WORKSPACE}\\${params.PROJECT_NAME}\\build") {
bat "\"${tool 'msbuild'}\" %COMPONENT%.XML /t:Publish /p:version=${env.SnapshotComponentVersion} "
}
}
}
stage('Package Full Build') {
when {
allOf {
branch 'master'
expression {
return readFile('output/MR_Title.txt').contains("FULL BUILD")
}
}
}
steps {
dir("${env.WORKSPACE}\\${params.PROJECT_NAME}\\build") {
bat "\"${tool 'msbuild'}\" %COMPONENT%.XML /t:Publish /p:version=${env.ComponentVersion} "
}
}
}
}
}

How to handle nightly build in Jenkins declarative pipeline

I have a multibranch pipeline with a Jenkinsfile in my repo and I am able to have my CI workflow (build & unit tests -> deploy-dev -> approval -> deploy-QA -> approval -> deploy-prod) on every commit.
What I would like to do is add SonarQube Analysis on nightly builds in the first phase build & unit tests.
Since my build is triggerd by Gitlab I have defined my pipeline triggers as follow :
pipeline {
...
triggers {
gitlab(triggerOnPush: true, triggerOnMergeRequest: true, branchFilterType: 'All')
}
...
}
To setup my nightly build I have added
triggers {
...
cron('H H * * *')
}
But now, how to execute analysis step if we are only building the job triggered by the cron expression at night ?
My simplified build stage looks as follow :
stage('Build & Tests & Analysis') {
// HERE THE BEGIN SONAR ANALYSIS (to be executed on nightly builds)
bat 'msbuild.exe ...'
bat 'mstest.exe ...'
// HERE THE END SONAR ANALYSIS (to be executed on nightly builds)
}
There is the way how to get build trigger information. It is described here:
https://jenkins.io/doc/pipeline/examples/#get-build-cause
It is good for you to check this as well:
how to get $CAUSE in workflow
Very good reference for your case is https://hopstorawpointers.blogspot.com/2016/10/performing-nightly-build-steps-with.html. Here is the function from that source that exactly matches your need:
// check if the job was started by a timer
#NonCPS
def isJobStartedByTimer() {
def startedByTimer = false
try {
def buildCauses = currentBuild.rawBuild.getCauses()
for ( buildCause in buildCauses ) {
if (buildCause != null) {
def causeDescription = buildCause.getShortDescription()
echo "shortDescription: ${causeDescription}"
if (causeDescription.contains("Started by timer")) {
startedByTimer = true
}
}
}
} catch(theError) {
echo "Error getting build cause"
}
return startedByTimer
}
This works in declarative pipeline
when {
triggeredBy 'TimerTrigger'
}
For me the easiest way is to define a cron in build trigger and verify the hour on the nightly stage using a when expression:
pipeline {
agent any
triggers {
pollSCM('* * * * *') //runs this pipeline on every commit
cron('30 23 * * *') //run at 23:30:00
}
stages {
stage('nightly') {
when {//runs only when the expression evaluates to true
expression {//will return true when the build runs via cron trigger (also when there is a commit at night between 23:00 and 23:59)
return Calendar.instance.get(Calendar.HOUR_OF_DAY) in 23
}
}
steps {
echo "Running the nightly stage only at night..."
}
}
}
}
You could check the build cause like so:
stage('Build & Tests & Analysis') {
when {
expression {
for (Object currentBuildCause : script.currentBuild.rawBuild.getCauses()) {
return currentBuildCause.class.getName().contains('TimerTriggerCause')
}
}
steps {
bat 'msbuild.exe ...'
bat 'mstest.exe ...'
}
}
}
However, this requires the following entries in script-approval.xml:
<approvedSignatures>
<string>method hudson.model.Run getCauses</string>
<string>method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild</string>
</approvedSignatures>
This can also be approved via https://YOURJENKINS/scriptApproval/.
Hopefully, this won't be necessary after JENKINS-41272 is fixed.
Until then, a workaround could be to check the hour of day in the when expression (keep in mind that these times refer to to the timezone of Jenkins)
when { expression { return Calendar.instance.get(Calendar.HOUR_OF_DAY) in 0..3 } }
I've found a way, which does not use "currentBuild.rawBuild" which is restricted. Begin your pipeline with:
startedByTimer = false
def buildCauses = "${currentBuild.buildCauses}"
if (buildCauses != null) {
if (buildCauses.contains("Started by timer")) {
startedByTimer = true
}
}
Test the boolean where you need it, for example:
stage('Clean') {
when {
anyOf {
environment name: 'clean_build', value: 'Yes'
expression { (startedByTimer == true) }
}
}
steps {
echo "Cleaning..."
...
Thanks to this you can now do this without needing to the the use the non-whitelisted currentBuild.getRawBuild().getCauses() function which can give you Scripts not permitted to use method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild depending on your setup:
#NonCPS
def isJobStartedByTimer() {
def startedByTimer = false
try {
def buildCauses = currentBuild.getBuildCauses()
for ( buildCause in buildCauses ) {
if (buildCause != null) {
def causeDescription = buildCause.shortDescription
echo "shortDescription: ${causeDescription}"
if (causeDescription.contains("Started by timer")) {
startedByTimer = true
}
}
}
} catch(theError) {
echo "Error getting build cause"
}
return startedByTimer
}

Resources