There some Jenkins jobs that have very high memory requirements. With the aim to avoid memory-related issues I'd like to configure these jobs to block execution of any other jobs from any project on the same node while they are running.
I played with Throttle Concurrent Builds Plugin, but it looks to limit jobs only from the same project. Other plugin - Build Blocker Plugin doesn't block other jobs to be run while the configured job is running.
Are there any plugins that allow to configure "single process" mode for some jobs?
In the "Throttle Concurrent Builds" Plugin there is a "Multi-Project Throttle Categories" option. Make a category "memory hogs" and add all jobs using a lot of memory to that category, than configure limits with "Throttle this project as part of one or more categories"
Related
Anyone has idea how to limit number of concurrent builds on Jenkins for multi-branch pipeline?
I was searching out but almost every approach was about putting properties([disableConcurrentBuilds()]) which doesn't work in my case.
We use resource lock for unit tests, so they don't overlap. In my case no matter how many build executors I have they will just wait on lock to be released so they continue one by one while reserving build slot instead waiting in queue. I found some similar post:
Jenkins limit multibranch
for gitlab pipelines use parameter resource_group for create a resource group that ensures a job is mutually exclusive across different pipelines for the same project:
https://docs.gitlab.com/ee/ci/yaml/#resource_group
for jenkins: plugin and parameter "Block build if certain jobs are running"
https://plugins.jenkins.io/build-blocker-plugin/
I have an instance of Jenkins that uses the mesos plugin. Nearly all of my jobs get triggered via Mesos tasks. I would like to make worker generation a bit more aggressive.
The current issue is that, for the mesos plugin, I have all of the jobs marking the mesos tasks as one-time usage slaves and when a build is in progress on one of these slaves Jenkins forces any queued jobs to wait for a potential executor on these slaves, as opposed to spinning up new instances.
Based on the logs, it also seems like Jenkins has a timer that periodically checks to see if any slaves should be spun up based on the # of queued jobs / excess workload. Is it possible to decrease the polling interval for that process?
From Mesos Jenkins Plugin Readme: over provisioning flags
By default, Jenkins spawns slaves conservatively. Say, if there are 2 builds in queue, it won't spawn 2 executors immediately. It will spawn one executor and wait for sometime for the first executor to be freed before deciding to spawn the second executor. Jenkins makes sure every executor it spawns is utilized to the maximum. If you want to override this behavior and spawn an executor for each build in queue immediately without waiting, you can use these flags during Jenkins startup:
-Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
How do you throttle Jenkins pipeline build so the queue doesn't get filled with multiple pipeline jobs but still allow someone to force start a build if they need it?
Currently, the throttle builds property will prevent the job from starting at all (during the throttling period) through the jenkins web interface. "Execute concurrent builds if necessary" or "Quiet Period" may be a better option but I cannot set that in the Jenkinsfile.
How do i create job categories on Jenkins for use with the throttle concurrent build plugin.
The answer for the question in the link below didnt help.
How to put jobs in a category for the Throttle Concurrent Builds plugin for Jenkins
I cant see any option to configure categories in the global jenkins configuration.
Categories can be configured at 'Manage Jenkins -> Configure System -> Throttle Concurrent Builds'.
I have a Jenkins server with half a dozen builds. Each of these builds is composed of a parent job that triggers anywhere between 4 and 6 smaller jobs that run in parallel. I use the EC2 plugin to keep the number of active slaves in line with the number of queued builds. Or in other words, slaves are coming and going all the time. Each slave has 7 executors (parent job + max(4, 6)).
It is absolutely crucial that all jobs of a build are executed on the same machine. I also cannot allow any jobs from build A to execute on a machine that has jobs from build B running.
What I'm looking for is a way that prevents Jenkins from using any inactive executors of a node as long as any jobs from a previous build are still active on it.
I've spent the day experimenting with a combination of the Throttle Concurrent Builds Plugin and the NodeLabel Parameter Plugin. Unfortunately, there seems to be a bug somewhere that causes throttled builds to not contribute to the Load Statistics of a slave. This in turn means that throttled build will never trigger Jenkins to spin up additional slaves. Obviously this is totally unacceptable.
You can try and use "This build is parameterized"
and pass the $NODE_NAME as a parameter between the builds and then use it at the "Restrict where this project can be run"