I want my pipeline to continuously build and run unit test during the day and automatically deploy the latest build at midnight. Also need to be able to manually force deploy if needed. My problem is that the first build of the day is not cancelled, so the pipeline will deploy twice.
Maybe if I create a separate job that locks the deploy resource just before midnight and releases it some time after just to make sure that all builds are in queue for lock so that 'inversePrecedence' prunes all older builds. Is there a simpler solution?
stage('build')
node('myNode') {
sh 'build.ksh'
sh 'runtest.ksh'
stash includes: 'Builds/', name: 'Builds'
}
// Wait for midnight to deploy
milestone 1
stage('wait for midnight')
if (!isMidnight()) {
waitForMidnightOrManualTrigger()
}
milestone 2
stage('deploy')
lock(resource: 'deploy-server', inversePrecedence: true) {
node('myNode') {
unstash 'Builds'
sh "deploy.ksh"
}
}
milestone 3
def isMidnight() {
int hour = Calendar.getInstance().get(Calendar.HOUR_OF_DAY);
return hour < 1
}
def waitForMidnightOrManualTrigger() {
echo 'Wait until midnight. Or Force deploy. Or abort.'
try {
waitTime = getMillisToMidnight()
timeout(time: waitTime, unit: 'MILLISECONDS') {
input 'Click Proceed to force deploy now. Or wait till midnight for automatic deploy.'
}
} catch (org.jenkinsci.plugins.workflow.steps.FlowInterruptedException e){
if (isAborted(e)) {
echo "The job was aborted by user"
throw e
} else {
echo "Timeout reached, proceed!"
}
}
}
def getMillisToMidnight() {
Calendar c = Calendar.getInstance();
c.add(Calendar.DAY_OF_MONTH, 1);
c.set(Calendar.HOUR_OF_DAY, 0);
c.set(Calendar.MINUTE, 0);
c.set(Calendar.SECOND, 0);
c.set(Calendar.MILLISECOND, 0);
return c.getTimeInMillis() - System.currentTimeMillis();
}
// There might be an easier way to differ between if the timeout throws the
// exception or if it is the user that has aborted the build...
def isAborted(e) {
def stackTrace = e.getStackTrace()
for (int i=0; i<stackTrace.size(); ++i) {
if (stackTrace[i].toString().contains("InputStepExecution.stop")) {
return false
}
}
return true
}
Related
I'd like to start a pipeline job manually. This job should then run daily and after seven days stop automatically. Is there any way to do this?
AFAIK There is no OOB solution for this. But you can implement something with Groovy to achieve what you need. For example, check the following pipeline. In the below Pipeline, I'm adding a Cron expression to run every day if manually triggered and then removing the corn expression after a predefined number of runs elapse. You should be able to fine-tune the below and achieve what you need.
def expression = getCron()
pipeline {
agent any
triggers{ cron(expression) }
stages {
stage('Example') {
steps {
script {
echo "Build"
}
}
}
}
}
def getCron() {
def runEveryDayCron = "0 9 * * *" //Runs everyday at 9
def numberOfRunsToCheck = 7 // Will run 7 times
def currentBuildNumber = currentBuild.getNumber()
def job = Jenkins.getInstance().getItemByFullName(env.JOB_NAME)
for(int i=currentBuildNumber; i > currentBuildNumber - numberOfRunsToCheck; i--) {
def build = job.getBuildByNumber(i)
if(build.getCause(hudson.model.Cause$UserIdCause) != null) { //This is a manually triggered Build
return runEveryDayCron
}
}
return ""
}
I maintain a jenkins pipeline which has the requirement to finish before 10am in the morning.
Is there a way to detect that the job finished after 10am and print a warning for example?
Here is a sample Pipeline to achieve your requirement. Here I have used a post-build step to check the time. You can decide what you want to do after checking the time. Eg: Setting the build to unstable, sending a mail etc.
pipeline {
agent any
stages{
stage('Build') {
steps{
echo "RUNNING THE BUILD!!!!"
}
}
}
post {
always {
script{
echo 'Checking the time'
def timeToCheckBefore = [hourOfDay: 10, minute: 0, second: 0] // 10PM will be 23, 0, 0
def now = new Date()
def check = now.clone()
check.set(timeToCheckBefore)
if(now.after(check)) {
echo "Time is Passed: Current Time : $now, Should finish before: $check"
} else {
echo "Job finished timely: Current Time : $now, Should finish before: $check"
}
}
}
}
}
I have a stage which runs one shell script at remote node. If the script takes very long time to execute, the stage should wait for sometime and should move to next stage without aborting the subsequent stages.
Could you please let me know the required syntax to achieve this.
in a declarative pipeline, add this stage:
stage ("do-and-skip-for-timeout") {
steps {
script {
try {
timeout (time: 10, unit: 'MINUTES') {
echo "do something here, that can take some time" // replace this line with your code
}
}
catch (error) {
def user = error.getCauses()[0].getUser()
if('SYSTEM' == user.toString()) { // SYSTEM means timeout.
echo "Timeout reached, continue to next stage"
}
else {
throw new Exception("[ERROR] stage failed!")
}
}
}
}
I have a multibranch pipeline with a Jenkinsfile in my repo and I am able to have my CI workflow (build & unit tests -> deploy-dev -> approval -> deploy-QA -> approval -> deploy-prod) on every commit.
What I would like to do is add SonarQube Analysis on nightly builds in the first phase build & unit tests.
Since my build is triggerd by Gitlab I have defined my pipeline triggers as follow :
pipeline {
...
triggers {
gitlab(triggerOnPush: true, triggerOnMergeRequest: true, branchFilterType: 'All')
}
...
}
To setup my nightly build I have added
triggers {
...
cron('H H * * *')
}
But now, how to execute analysis step if we are only building the job triggered by the cron expression at night ?
My simplified build stage looks as follow :
stage('Build & Tests & Analysis') {
// HERE THE BEGIN SONAR ANALYSIS (to be executed on nightly builds)
bat 'msbuild.exe ...'
bat 'mstest.exe ...'
// HERE THE END SONAR ANALYSIS (to be executed on nightly builds)
}
There is the way how to get build trigger information. It is described here:
https://jenkins.io/doc/pipeline/examples/#get-build-cause
It is good for you to check this as well:
how to get $CAUSE in workflow
Very good reference for your case is https://hopstorawpointers.blogspot.com/2016/10/performing-nightly-build-steps-with.html. Here is the function from that source that exactly matches your need:
// check if the job was started by a timer
#NonCPS
def isJobStartedByTimer() {
def startedByTimer = false
try {
def buildCauses = currentBuild.rawBuild.getCauses()
for ( buildCause in buildCauses ) {
if (buildCause != null) {
def causeDescription = buildCause.getShortDescription()
echo "shortDescription: ${causeDescription}"
if (causeDescription.contains("Started by timer")) {
startedByTimer = true
}
}
}
} catch(theError) {
echo "Error getting build cause"
}
return startedByTimer
}
This works in declarative pipeline
when {
triggeredBy 'TimerTrigger'
}
For me the easiest way is to define a cron in build trigger and verify the hour on the nightly stage using a when expression:
pipeline {
agent any
triggers {
pollSCM('* * * * *') //runs this pipeline on every commit
cron('30 23 * * *') //run at 23:30:00
}
stages {
stage('nightly') {
when {//runs only when the expression evaluates to true
expression {//will return true when the build runs via cron trigger (also when there is a commit at night between 23:00 and 23:59)
return Calendar.instance.get(Calendar.HOUR_OF_DAY) in 23
}
}
steps {
echo "Running the nightly stage only at night..."
}
}
}
}
You could check the build cause like so:
stage('Build & Tests & Analysis') {
when {
expression {
for (Object currentBuildCause : script.currentBuild.rawBuild.getCauses()) {
return currentBuildCause.class.getName().contains('TimerTriggerCause')
}
}
steps {
bat 'msbuild.exe ...'
bat 'mstest.exe ...'
}
}
}
However, this requires the following entries in script-approval.xml:
<approvedSignatures>
<string>method hudson.model.Run getCauses</string>
<string>method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild</string>
</approvedSignatures>
This can also be approved via https://YOURJENKINS/scriptApproval/.
Hopefully, this won't be necessary after JENKINS-41272 is fixed.
Until then, a workaround could be to check the hour of day in the when expression (keep in mind that these times refer to to the timezone of Jenkins)
when { expression { return Calendar.instance.get(Calendar.HOUR_OF_DAY) in 0..3 } }
I've found a way, which does not use "currentBuild.rawBuild" which is restricted. Begin your pipeline with:
startedByTimer = false
def buildCauses = "${currentBuild.buildCauses}"
if (buildCauses != null) {
if (buildCauses.contains("Started by timer")) {
startedByTimer = true
}
}
Test the boolean where you need it, for example:
stage('Clean') {
when {
anyOf {
environment name: 'clean_build', value: 'Yes'
expression { (startedByTimer == true) }
}
}
steps {
echo "Cleaning..."
...
Thanks to this you can now do this without needing to the the use the non-whitelisted currentBuild.getRawBuild().getCauses() function which can give you Scripts not permitted to use method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild depending on your setup:
#NonCPS
def isJobStartedByTimer() {
def startedByTimer = false
try {
def buildCauses = currentBuild.getBuildCauses()
for ( buildCause in buildCauses ) {
if (buildCause != null) {
def causeDescription = buildCause.shortDescription
echo "shortDescription: ${causeDescription}"
if (causeDescription.contains("Started by timer")) {
startedByTimer = true
}
}
}
} catch(theError) {
echo "Error getting build cause"
}
return startedByTimer
}
Within a Jenkinfile pipeline script, how do you query the running job state to tell if it has been aborted?
Normally a FlowInterruptedException or AbortException (if a script was running) will be raised but these can be caught and ignored. Also scripts will not exit immediately if it has multiple statements.
I tried looking at 'currentBuild.Result' but it doesn't seem to be set until the build has complete. Something in 'currentBuild.rawBuild' perhaps?
There is nothing that would automatically set the build status if the exception has been caught. If you want such exceptions to set a build status, but let the script continue, you can write for example
try {
somethingSlow()
} catch (InterruptedException x) {
currentBuild.result = 'ABORTED'
echo 'Ignoring abort attempt at this spot'
}
// proceed
You could implement a watchdog branch in a parallel step. It uses a global to keep track of the watchdog state which could be dangerous, I don't know if accessing globals in 'parallel' is threadsafe. It even works if 'bat' ignores the termination and doesn't raise an exception at all.
Code:
runWithAbortCheck { abortState ->
// run all tests, print which failed
node ('windows') {
for (int i = 0; i < 5; i++) {
try {
bat "ping 127.0.0.1 -n ${10-i}"
} catch (e) {
echo "${i} FAIL"
currentBuild.result = "UNSTABLE"
// continue with remaining tests
}
abortCheck(abortState) // sometimes bat doesn't even raise an exception! so check here
}
}
}
def runWithAbortCheck(closure) {
def abortState = [complete:false, aborted:false]
parallel (
"_watchdog": {
try {
waitUntil { abortState.complete || abortState.aborted }
} catch (e) {
abortState.aborted = true
echo "caught: ${e}"
throw e
} finally {
abortState.complete = true
}
},
"work": {
try {
closure.call(abortState)
}
finally {
abortState.complete = true
}
},
"failFast": true
)
}
def _abortCheckInstant(abortState) {
if (abortState.aborted) {
echo "Job Aborted Detected"
throw new org.jenkinsci.plugins.workflow.steps.FlowInterruptedException(Result.ABORTED)
}
}
def abortCheck(abortState) {
_abortCheckInstant(abortState)
sleep time:500, unit:"MILLISECONDS"
_abortCheckInstant(abortState)
}