I'm working on a busy Jenkins server with many big matrix jobs being built all the time. Working with another job can be really annoying if you want instantaneous results... Is there an option to give a job temporary priority for just one build so it skips the queue and gets built on the next available executor?
You can try using Priority Sorter Plugin and add your jobs with higher priority than the rest
Related
I want to run one job multiple times (each build with different parameters) on 2 executors.
I want to execute them based on their build priority value.
Unfortunately Priority Sorter plugin doesn't help in my case, it doesn't sort it correctly - my builds are being executed based on the timestamp they were added to the queue instead of priority.*
I believe this priority mechanism should be implemented before, on a queue level.
How to achieve that?
*-I tested it on the newest Jenkins version and the newest Priority Sorter plugin version
I think what you should try is Accelerated Build Now plugin. This plugin allows the Jenkins users to launch a build right there, even if the queue is long. It prioritises human launched jobs and brings them to the top.
The Priority Sorter I am using (v3.6.0 and Jenkins v.2.73.3) will not see a job unless you have enabled the Execute Concurrent builds if necessary.
So, allow parallel builds for that job and maybe decrease the number of cores to 1. See if that works. If not, you can try Throttle Concurrent Builds Plugin. This allows you to assign as many cores as you want in the specific job.
Here is a patch for Priority Queue plugin. For your case it could be this patch.
I think this is related to the "id" field maybe, previously we weren't setting that and had issues where the multibranch job was reindexed all branches as new. That's fixed now but there is another issue.
Every time the job dsl runs it causes a branch scan job to kick off for all of our multibranchPipelineJobs.
Why does this happen ? Is there a way to prevent this ? For a few jobs it's not a big deal but we have almost 200 multibranchPipelineJobs. So this huge branch scan queue builds up every time the seed job is run. Also, according to Cloudbees there is no way to increase the number of scan jobs Jenkins processes at a time. So it always takes forever to burn down.
This is stupid, am I doing something wrong ? This happens even if there are no changes but frankly I don't think it should happen even if there are. I notice if I modify the config of a Jenkins job and save it, it usually just kicks off a branch scan job so maybe this is Jenkins behavior?
It seems like the ugliest way to handle this, but can you have the job dsl kill scan jobs in the queue for jobs it just configured and not affect other scan jobs that aren't related to the seed job run?
How can I setup Jenkins to automatically cancel/kill low-priority jobs once higher priority jobs are available to run?
Some background -- there's a feature request for this capability:
https://issues.jenkins-ci.org/browse/JENKINS-8405
In the absence of such a feature being implemented, how might I accomplish this? Some ideas I have are trying to read a potentially-existent Jenkins file that contains the list of jobs in the build. And, I could launch the low priority jobs with a wrapper that spawns a separate process that monitors this file and kills the low-priority processes whenever a high-priority job needs to run.
But the above is fairly involved, and so I'd like to avoid doing that. I could use linux "nice", except that the memory requirements are high, so it's really better to kill the processes.
One partial solution is to use the Accelerated Build Now Plugin, which lets you cancel low priority jobs when you click the button. That being said, an automatic version would be better.
https://wiki.jenkins-ci.org/display/JENKINS/Accelerated+Build+Now+Plugin
You could create a supervisor job that runs periodically and checks if high priority jobs are in the queue and aborts low priority jobs if needed.
Naturally you need to make sure that the supervisor job itself never gets stuck in the queue. One way to accomplish this would be to create a dedicated dummy slave for the supervisor job.
I would like to monitor the estimated time of all of my builds to catch the cases where this value is shown as 'N/A'.
In these cases the build gets stuck (probably due to network issues in my environment) and it won't start new builds for that job until killed manually.
What I am missing is how to get that data for each job, either from api or other source.
I would appreciated any suggestion.
Thanks.
For each job, you can click "Trend" on the job run history table, and it will show you the currently executing progress along with a graph of "usual" execution times.
Using the API, you can go to http://jenkins/job/<your_job_name>/<build_number>/api/xml (or /json) and the information is under <duration> and <estimatedDuration> fields.
Finally, there is a Jenkins Timeout Plugin that you can use to automatically take care of "stuck" builds
we are running Jenkins with lots of jobs. At the moment these jobs are kind of grouped by using "master jobs". These do nothing but start all jobs of one group. But, if one of these master jobs runs, it starts around 10 other jobs at one time. Depending on the duration of these jobs and the number of build processores (at the moment 6) Jenkins is blocked for a longer time (up to an hour). The other thing is, that these jobs are not really suitable for such massive parallelization.
To solve this, I'm looking for a way (a plugin), that allows to group some jobs and start them parallel, but limit the build processors used for the jobs of this group to a fixed number (e.g. 2). So it would be possible to run a group of jobs that compile java projects and parallel another group of jobs that installs test databases.
I tried the Build flow plugin, but it's not really the right one: you must separate the jobs manually to the sub-groups that run parallel and if a job in one sub-group failes, the following jobs of this group are not started.
So, maybe someone knows a Jenkins plugin that fits better? Thanks a lot in advance!
Frank
Throttle Concurrent Builds Plugin
Create some category my-group.
Add all the jobs into this group.
Set Maximum Total Concurrent Builds and Maximum Concurrent Builds Per Node.