I have some Jenkins jobs (say job-A and job-B), but need to prevent any of them running simultaneously if a particular parameter is the same. So if job-A is started with parameter foo=42, and an attempt is made to run job-B with parameter foo=42, that attempt must be prevented. But running job-B with parameter foo=17 (while job-A|foo=42) is running is fine (as is running job-A|foo=17).
I can figure out a way to do this using shell scripts and lock files, but wondering if it is possible within Jenkins itself or using a plugin.
Here is how you can get this done with the Lockable Resource Plugin, which allows you to acquire locks on resources and wait until they are released to perform certain pipeline operations.
First, create a lock called foo42lock, and the following are two sample pipelines. Since you don't want to wait until the lock is released you can set skipIfLocked: true flag when acquiring the lock, so the pipeline will continue if the lock is not acquirable.
JobA
pipeline {
agent any
parameters { string(name: 'foo', defaultValue: '', description: '') }
stages {
stage("StageA") {
steps {
script {
if(foo == "42") {
lock(resource: 'foo42lock', skipIfLocked: true) {
echo "Foo is 42 Lock aquired and Job A Running"
runSteps()
}
} else {
echo "No lock needed"
runSteps()
}
}
}
}
}
}
def runSteps(){
echo "Storing the steps externally to reuse. Running Job A"
sleep 30
}
JobB
pipeline {
agent any
parameters { string(name: 'foo', defaultValue: '', description: '') }
stages {
stage("StageB") {
steps {
script {
if(foo == "42") {
lock(resource: 'foo42lock', skipIfLocked: true) {
echo "Foo is 42 Lock aquired and Job B Running"
runSteps()
}
} else {
echo "No lock needed"
runSteps()
}
}
}
}
}
}
def runSteps(){
echo "Storing the steps externally to reuse. Running Job B steps"
sleep 30
}
Another way to achieve this would be, whenever either JobA or JobB is triggered if foo==42 then JobA can check if there is a build happening for JobB and whether the parameter is set to foo=42 and if a build is happening it can stop the build, and vice versa. Implementing this is also not that complex and can use Groovy scripts to implement the logic.
Related
I have two nodes added in Jenkins,
- master
- node-1
Here is my pipeline script, I would like to lock all the executors on "node-1" before executing anything on "master"
node("master") {
stage("stage-1") {
// here lock node-1
//Execute script
}
}
Is there a way to achieve this? (ie: lock node-1)
My strategy is to mark the node offline as soon as your buyild job catches an executor, and then wait for all the other executors to complete. At this stage, your executor is still active, but the other executors can't get a new build since it's offline, so the node is all for yourself. When you are done, you can markl the node online again.
This requires some serious admin approval for the script.
For instance:
final int sleeptimeSeconds = 10
final String agentname = 'node-1'
echo "Waiting for an executor for ${agentname}..."
node(agentname) {
try {
timeout(time: timeoutminutes, unit: 'MINUTES') {
markAgentOnlineOfOffline(agentname, false)
sleep 5
Computer computer = Jenkins.getInstance().getComputer(agentname)
echo "Waiting for other executors to complete on ${agentname}..."
while (computer.countBusy() > 1) {
sleep sleeptimeSeconds
}
echo "Ready to do work on '${agentname}' in exclusive mode..."
...
}
} catch (e) {
markAgentOnlineOfOffline(agentname, true)
throw e
}
}
def markAgentOnlineOfOffline(String nodeName, boolean online) {
...
}
That last function markAgentOnlineOfOffline can be implemented in a logical (e.g. I use the "offline" label myself, which my jobs reject (li.e. the label requirement includes !offline). But you could use the Jenkins api to mark the node truly offline.
In my pipeline, I have a stage that checks to see if a specific computer (node) is offline. If it is, I want to skip the next stage. However, the next stage is set to use the offline agent, so it doesn't seem to be able to check the When clause.
Here's a simplified version of my pipeline:
pipeline {
agent none
environment {
CONTINUERUN = true
}
stages {
stage('Check Should Run') {
agent any
steps {
script {
CONTINUERUN = false
}
}
}
stage('Skip this stage') {
agent {
label 'offlineAgent'
}
when {
expression {
CONTINUERUN
}
}
steps {
//Do stuff here
}
}
}
}
When I run this, the build just hangs at the 'Skip this stage' stage. I'm assuming, because the agent is offline. How can I skip this stage, when the agent is known to be offline?
In order to evaluate expression before allocating agent, you need to add beforeAgent directive to the when block.
Relevant part of documentation:
Evaluating when before entering agent in a stage
By default, the when condition for a stage will be evaluated after entering the agent for that stage, if one is defined. However, this can be changed by specifying the beforeAgent option within the when block. If beforeAgent is set to true, the when condition will be evaluated first, and the agent will only be entered if the when condition evaluates to true.
I am building pipeline workflow in Jenkins v.2.8. What I would like to achieve is to build one step which would trigger same job multiple times as same time with different parameters.
Example: I have a worfklow called "Master" which has one step, this step is reading my parameter "Number" which is a check box with multiple selection option. So user can trigger workflow and select option for Number like "1, 2, 3". Now what I would like to achieve when this step is executed that it calls my job "Master_Child" and triggers "Master_Child" with 3 different parameters at the same time.
I tried to do it in this way:
stage('MyStep') {
steps {
echo 'Deploying MyStep'
script {
env.NUMBER.split(',').each {
build job: 'Master_Child', parameters: [string(name: 'NUMBER', value: "$it")]
}
}
}
}
But with this it reads first parameter triggers the Mast_Child with parametre 1 and it waits until the jobs is finished, when job is finished then it triggers the same the job with parameter 2.
If I use wait: false on job call, then pipeline workflow just calls this jobs with different parameters but it is not depended if sub job fails.
Any ideas how to implement that ?
Thank you in advance.
I resolved my problem in this way.
stage('MyStage') {
steps {
echo 'Deploying MyStep'
script {
def numbers = [:]
env.NUMBER.split(',').each {
numbers["numbers${it}"] = {
build job: 'Master_Child', parameters: [string(name: 'NUMBER', value: "$it")]
}
}
parallel numbers
}
}
}
Set the wait in the build job syntax to false wait: false
stage('MyStep') {
steps {
echo 'Deploying MyStep'
script {
env.NUMBER.split(',').each {
build job: 'Master_Child', parameters: [string(name: 'NUMBER', value: "$it")], wait: false
}
}
}
}
A Jenkins pipeline project is configured to fetch its Jenkinsfile from a Git repo:
If I change the list of parameters, for example, from:
properties([
parameters([
string(name: 'FOO', description: 'Choose foo')
])
])
to:
properties([
parameters([
string(name: 'FOO', description: 'Choose foo'),
string(name: 'BAR', description: 'Choose bar')
])
])
And run the build, the first run does not show the newly added BAR parameter:
As the updated Jenkins file expects the BAR parameter to be present, this causes the first build after the change to fail as the user is not presented with an input to enter this value.
Is there a way to prevent this? To make sure the Jenkinsfile is up-to-date before showing the parameter entry page?
Short answer: No. It would be nice if there was some facility for parsing and processing the Jenkinsfile separate from the build, but there's not.
Jenkins does not know about the new parameters until it retrieves, parses, and runs the Jenkinsfile, and the only way to do that is to...run a build.
In effect, the build history will always be "one run behind" the Jenkinsfile; when you change something in the Jenkinsfile, the next build will run with the "old" Jenkinsfile, but pick up and process the new Jenkinsfile for the build after that.
The only solution to this problem afaik is to manually add an "skip_run" boolean parameter, than add a when{} clause to every stage of the job.
properties([
parameters([
BooleanParameter(name: 'skip_run', description: 'Skips all stages. Used to update parameters in case of changes.', default: False)
])
])
...
stage('Doing Stuff') {
when {
expression { return params.skip_run ==~ /(?i)(N|NO|F|FALSE|OFF|STOP)/ }
}
steps {
...
}
}
This is, of course, very prone to error.
Alternatively, you could add a single stage as the very beginning of the pipeline and fail the build on purpose.
stage('Update Build Info only') {
when {
expression { return params.skip_run ==~ /(?i)(Y|YES|T|TRUE|ON|RUN)/ }
}
steps {
error("This was done deliberately to update the build info.")
}
}
UPDATE:
Thanks to Abort current build from pipeline in Jenkins, i came up with this solution:
To prevent the build from actually appearing red, you could wrap this with a try - catch and exit the build gracefully.
final updateOnly = 'updateOnly'
try {
stage('Update Build Info only') {
when {
expression { return params.skip_run ==~ /(?i)(Y|YES|T|TRUE|ON|RUN)/ }
}
steps {
error(updateOnly)
}
}
...
//other stages here
...
} catch (e) {
if (e.message == updateOnly) {
currentBuild.result = 'ABORTED'
echo('Skipping the Job to update the build info')
// return here instead of throwing error to keep the build "green"
return
}
// normal error handling
throw e
}
I have a function that skips the build unless the job has all the required parameters, something like:
if (job.hasParameters(['FOO', 'BAR'])) {
// pipeline code
}
An issue was reported a few years ago in Jenkins related with this issue
https://issues.jenkins-ci.org/browse/JENKINS-41929
Still open, so no elegant solution yet.
try..
parameters {
string(name: 'GRADLE_ARGS', defaultValue: '--console=plain', description: 'Gradle arguments')
}
environment{
GRADLE_ARGS = "${params.GRADLE_ARGS}"
}
I'm hoping to add a conditional stage to my Jenkinsfile that runs depending on how the build was triggered. Currently we are set up such that builds are either triggered by:
changes to our git repo that are picked up on branch indexing
a user manually triggering the build using the 'build now' button in the UI.
Is there any way to run different pipeline steps depending on which of these actions triggered the build?
The following code should works to determine if a user has started the pipeline or a timer/other trigger:
def isStartedByUser = currentBuild.rawBuild.getCause(hudson.model.Cause$UserIdCause) != null
In Jenkins Pipeline without currentBuild.rawBuild access the build causes could be retrieved in the following way:
// started by commit
currentBuild.getBuildCauses('jenkins.branch.BranchEventCause')
// started by timer
currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')
// started by user
currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause')
You can get a boolean value with:
isTriggeredByTimer = !currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause').isEmpty()
Or, as getBuildCauses() returns an array, the array's size will work correctly with Groovy truthy semantics:
if (currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')) {
The ability to get causes for a workflow run was released in version 2.22 (2018 Nov 02) to the Pipeline Supporting APIs Plugin. The feature was requested in JENKINS-41272.
A couple methods were added to the currentBuild global variable with that release:
getBuildCauses
Returns a JSON array of build causes for the current build
EXPERIMENTAL - MAY CHANGE getBuildCauses(String causeClass)
Takes a string representing the fully qualified Cause class and returns a JSON array of build causes filtered by that type for the current build, or an empty JSON array if no causes of the specified type apply to the current build
And an example from me submitting:
echo "${currentBuild.buildCauses}" // same as currentBuild.getBuildCauses()
echo "${currentBuild.getBuildCauses('hudson.model.Cause$UserCause')}"
echo "${currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause')}"
And the output:
[Pipeline] echo
[[_class:hudson.model.Cause$UserIdCause, shortDescription:Started by user anonymous, userId:null, userName:anonymous], [_class:org.jenkinsci.plugins.workflow.cps.replay.ReplayCause, shortDescription:Replayed #12]]
[Pipeline] echo
[]
[Pipeline] echo
[]
[Pipeline] End of Pipeline
Finished: SUCCESS
NOTE
There appears to be an issue with the currentBuild.getBuildCauses(type) when the type is a type of Cause contributed by a plugin. For example, currentBuild.getBuildCauses('org.jenkinsci.plugins.workflow.cps.replay.ReplayCause') fails with a java.lang.ClassNotFoundException. This was reported in JENKINS-54673 for the 2.22 version of the Pipeline: Supporting APIs (workflow-support) plugin. It is reportedly fixed in the 2.24 version.
I might be missing something, but you can achieve what you want easily by making use of the when directive:
pipeline {
agent any
stages {
stage('Always') {
steps {
echo "I am always executed"
}
}
stage('ManualTimed') {
steps {
echo "I am only executed when triggered manually or timed"
}
when {
beforeAgent true
anyOf {
triggeredBy 'TimerTrigger'
triggeredBy cause: 'UserIdCause'
}
}
}
stage('GitLabWebHookCause') {
steps {
echo "I am only executed when triggered by SCM push"
}
when {
beforeAgent true
triggeredBy 'GitLabWebHookCause'
}
}
}
}
You will find many similar useful examples for various use cases in the documentation of the when directive.
Edit:
thanks to Jean-Francois Larvoire's answer, I was able to figure out 'my trigger' GitLabWebHookCause I required for my use case.
#vitalii-blagodir:
Your answer works for detecting builds triggered by users and timers, but not by commits.
Instead, I found this to work in my case:
def isTriggeredByIndexing = currentBuild.getBuildCauses('jenkins.branch.BranchIndexingCause').size()
def isTriggeredByCommit = currentBuild.getBuildCauses('com.cloudbees.jenkins.GitHubPushCause').size()
def isTriggeredByUser = currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause').size()
def isTriggeredByTimer = currentBuild.getBuildCauses('hudson.triggers.TimerTrigger$TimerTriggerCause').size()
The .size() suffix returns 0 if the object is missing, or 1 if it's present. This makes the result usable as a boolean.
For finding the object name to use, I found it convenient to display this in the log:
echo "# Build causes"
def buildCauses = currentBuild.buildCauses
def numCause = 0
for (cause in buildCauses) {
echo "${numCause++}: ${cause.shortDescription}" // Display a human-readable index and description
echo "${cause}" // Display the object class name. This allows knowing what names to use in getBuildCauses(name) calls below.
}
Finally, if the goal is to abort a pipeline build in specific cases, then the test must be done before the beginning of the pipeline.
For example, we had a problem with the branch indexing triggering extra useless builds. This was fixed by adding this before the pipeline:
// Avoid useless buils: The branch indexing should only trigger the initial build of a new branch.
def isTriggeredByBranchIndexing = currentBuild.getBuildCauses('jenkins.branch.BranchIndexingCause').size()
if (isTriggeredByBranchIndexing && currentBuild.previousBuild) { // Then it's not the initial build.
echo "# Reindexing a branch already built. It is useless to rebuild it now. Aborting."
currentBuild.result = 'SUCCESS' // Make sure the build is not displayed in red in the Jenkins UI.
return // Abort before the pipeline even starts. (Inside the pipeline, this would only abort one stage.)
}
I think that the answers here are incomplete and do not provide an actual ready to use answer. Here's my code to get it working:
import com.cloudbees.groovy.cps.NonCPS
#NonCPS
def isStartedByTimer() {
def buildCauses = currentBuild.rawBuild.getCauses()
echo buildCauses
boolean isStartedByTimer = false
for (buildCause in buildCauses) {
if ("${buildCause}".contains("hudson.triggers.TimerTrigger\$TimerTriggerCause")) {
isStartedByTimer = true
}
}
echo isStartedByTimer
return isStartedByTimer
}
// [...]
// Other pipeline stuff
script {
isStartedByTimer()
}
When started by user:
00:00:01.353 [hudson.model.Cause$UserIdCause#fa5cb22a]
[Pipeline] echo
00:00:01.358 false
When started by timer:
00:00:01.585 [hudson.triggers.TimerTrigger$TimerTriggerCause#5]
[Pipeline] echo
00:00:01.590 true
Note: the NonCPS decorator is needed because otherwise the next non-script step will throw.
Assuming the two different build causes are "timer" and "push" (to a git repo), you can add the following stage to your Jenkinsfile (in a declarative Jenkins pipeline) to make use of getBuildCauses():
pipeline {
stages {
stage('preparation') {
steps {
script {
// get build cause (time triggered vs. SCM change)
def buildCause = currentBuild.getBuildCauses()[0].shortDescription
echo "Current build was caused by: ${buildCause}\n"
// e.g. "Current build was caused by: Started by GitHub push by mirekphd"
// vs. "Started by timer"
}
}
}
}
}
Then I can decide whether to perform certain stages conditionally (depending on the build cause). For example, pulling a docker base image and inspecting for changes in system libraries (likely security updates) should be done periodically, regardless of whether there was a source code change or not.
We can use "BUILD_CAUSE" variable for getting the information about who initiated the run
for [jenkins-pipeline] you may use
currentBuild.rawBuild.getCauses()
(see github.com/jenkinsci/pipeline-examples/blob/master/… for more details)
There was a similar requirement, where user detail who triggered the build should be there in success / failure notification. The job was already had time based triggered, hence could not use wrap([$class: 'BuildUser']) directly.
I used below step, which print username if the job is triggered manually or timer triggered. So, I used this:
pipeline {
agent any
stages {
stage('Test') {
steps {
script{
env.buildCauses = currentBuild.rawBuild.getCauses()
if (buildCauses.contains("hudson.triggers.TimerTrigger")){
env.builduser = "TimerTrigger"
} else {
wrap([$class: 'BuildUser']) {
env.builduser = "${BUILD_USER}"
}
}
}
echo "Initiated by: ${env.builduser}"
}
}
}
}