How to make jenkins pipeline script programmatically wait until previous/last job execution is currently running or succeeded?
There is option to disable concurrent builds, but that is a really bad option that will stack commits.
Related
We are moving over our build system from Hudson to Jenkins and also to declarative pipelines in SCM. Alas, it looks like there are some hiccups. In Hudson, when a job was scheduled and waiting in the queue, no new jobs were scheduled for that project, which makes all the sense. In Jenkins, however, I observe there are e.g. 5 instances of a job started at the same time, triggered by various upstream or SCM change events. They have all even kind of started, one of them is actually running on the build node and the rest are waiting in "Waiting for next available executor on (build node)". When the build node becomes available, they all dutifully start running in turn and all dutifully run through, most of them without no purpose at all as there are no more changes, and this all takes a huge amount of time.
The declarative pipeline script in SCM starts with the node declaration:
pipeline {
agent {
label 'BuildWin6'
}
...
I guess the actual problem is that Jenkins starts to run these jobs even though the specified build node is busy. Maybe it thinks I might have changed the Jenkinsfile in the SCM and specified another build node to run the thing on? Anyway, how to avoid this? This is probably something obvious as googling does not reveal any similar complaints.
For the record, answering myself. It looks like the best solution is to define another trigger job which is triggered itself by SCM changes. It should do nothing else, only checks out the needed svn repos (with depthOption: 'empty' for space and speed). The job needs to be bound to run on the same agent than the main job.
The main job is triggered only by the first job, not by SCM changes. Now if the main job is building for an hour, and there are 10 svn commits during that time, Jenkins will schedule 10 trigger job builds to run. They are all waiting in the queue as the agent is busy. When the agent becomes available, they all run quickly through and trigger the main job. The main job is triggered only once, for that one must ensure its grace/quiet period is larger than the trigger job run time.
I have one Jenkins job that triggers another job via "Trigger/call builds on other projects."
The triggered downstream job sometimes fails due to environmental reasons. I'd like to be able to restart the triggered job multiple times until it passes.
More specifically, I have a job which does the following:
Triggers a downstream job to configure my test environment. This process is sensitive to environmental issues and may fail. I'd like this to restart multiple times over a period of about an hour or two until it succeeds.
Trigger another job to run tests in the configured environment. This should not restart multiple times because any failure here should be inspected.
I've tried using Naginator for step 1 above (the configuration step). The triggered job does indeed re-run until it passes. Naginator looks so promising, but I'm disappointed to find that when the first execution of the job fails, the upstream job fails immediately despite a subsequent rebuild of the triggered job passing. I need the upstream job to block until the downstream set of jobs passes (or fails to pass) via Naginator.
Can someone help me know what my options are to accomplish this? Can I configure things differently for the upstream job so it relates to the Naginator-managed job better? I'm not wed to Naginator and am open to other plugins or options.
In case its helpful, my organization is currently using Jenkins 1.609.3 which is a few years old. I'd consider upgrading if that leads to a solution.
Trouble aborting pipeline jobs
We recently converted some of our jobs over to pipeline jobs (specifically, multibranch pipeline jobs), and since we did so, stopping builds has become much more problematic.
Back when they were freestyle jobs, the builds would immediately stop when we clicked the red X next to a build.
But when we do that in a pipeline job, often it won't stop. In the console output, we'll get something like this:
Aborted by [USERNAME]
Sending interrupt signal to process
Click here to forcibly terminate running steps
We have to click that link in the third line, and then it often leaves mercurial repositories in a bad state.
Why did stopping builds work fine with freestyle jobs, but not with Pipeline jobs, and is there any way to get them working well with Pipeline jobs?
How do you throttle Jenkins pipeline build so the queue doesn't get filled with multiple pipeline jobs but still allow someone to force start a build if they need it?
Currently, the throttle builds property will prevent the job from starting at all (during the throttling period) through the jenkins web interface. "Execute concurrent builds if necessary" or "Quiet Period" may be a better option but I cannot set that in the Jenkinsfile.
Let's assume that with Jenkins, I have a queue that is piled up with 10 jobs waiting for execution.
One of the executing jobs has failed, and I managed to fix the issue, and push the change.
I know that nobody will be able to push their work, until all job types in build radiator are "green".
Therefore I would like the ability to bypass the jobs in the queue so the failed job will run again.
I saw the following plugin - Priority sorter plugin - but I'm not sure that I can use this for the above scenario :
Let's say I configure all jobs to run with priority 50, and then I re-configure the failed job to run with priority one, will it bypass all the jobs in the queue?
The closest to that functionality would be the Accelerated Build Now Plugin.
[The plugin] allows Jenkins users to launch a project's build right away, even if the queue is long (moving it to the top of the queue) and even if no executor is available (killing and rescheduling builds not launched by "humans")