What is the impact on the running jobs if I change the number of executors in the production environment?
It depends on hardware used for master/slaves as well as on how much resource-intensive are your jobs.
We set the executors on our master to 0 and to our slaves - one executor per core.
Higher number of executors can result in degraded performance and higher flakiness of the jobs, especially true if they depend/share some data/application (database).
Related
I am new to Jenkins and I have configured master-slave nodes as shown below, but I need help to configure the no of executors in each of the below slave nodes
Currently, I have configured 100 executors in each slave nodes
How many no of executors I can configure in each slave node and what fact(memory, RAM, etc) need to take consider when increasing the no of executors?
Maximum how many no of executors I can configure in each server?
Well it totally depends on your usage. There are multiple factors such as how much cpu and memory is available, how the build are going to execute and what kind of builds, how frequent these build should run etc. But I can clearly say that 100 is too big number. I would suggest go with 20 builds (if builds run frequently and have fair amount of CPU and memory) first and observe if is there any issue with numbers or not then you can increase accordingly.
here is very nice article check this out https://www.avantica.com/blog/jenkins-balance-load-master-slave-setup#:~:text=Jobs%20are%20built%20using%20executors,to%20build%20two%20different%20tasks.
Is Jenkins capable of managing a large amount of jobs? More specifically, how powerful the Jenkins Master server have to be if it will be connected to, for example, 100 agents with each of them having 2 executors?
Although Jenkins Master is not actually running the real jobs but it still has to monitor the status of the agents and display console outputs, but I have no idea how resource-demanding these tasks are that's why I'm looking for some feedback.
Thanks :D
To make a right decision about resources and architecture you can read official documentation:
calculating
hardware recommendations
Or you can read whole article regarding your task
architecting for scale
Short atricle:
The equation for estimating the number of masters and executors needed
when the number of configured jobs is known is as follows:
masters = number of jobs/500
executors = number of jobs * 0.03
The equation for estimating the maximum number of jobs, masters, and
executors needed for an organization based on the number of developers
is as follows:
number of jobs = number of developers * 3.333
number of masters = number of jobs/500
number of executors = number of jobs * 0.03
By default, slave resources such as CPU and RAM are handled in a very basic way in Jenkins: by setting the number of executors, which is the number of allowed concurrent builds on this particular slave.
However, different jobs have different needs in terms of CPUs and RAM, so this model can quickly become sub optimal if you have jobs with different constraints: high RAM use, multi-threading, short and long jobs, etc.
Is there a way, for example, to say that a slave has 16GB of RAM, and then declare RAM consumption for each job?
After digging a bit more, it looks like the heavy job plugin [1] allows crude resource management, which is enough for my needs. I'd still like to know if people have better solutions to propose, though.
[1] https://wiki.jenkins.io/display/JENKINS/Heavy+Job+Plugin
I have a multi-configuration job that uses a large amount of VMs for testing.
The Axis are something like:
30 VM slaves, 5 configurations, 5 different configurations
I would not like to run these sequentially, as the jobs would take forever. However, the default number of simultaneous runs is using up enough resources that I am getting random failures and disconnects.
Is there are way to specify the maximum number of simultaneous tests within this single running job?
I think you have to use a matrix job to trigger the builds of a separate job doing the real build. Then
you can use the Throttle Concurrent Builds Plugin to limit the number of parallel executions of that job you start by the matrix.
For multi project configuration
First you need to create a throttle category. In this case, the name is qa-aut and I limiting the number of execution to 2 for concurrent builds and concurrent builds per node. The node will have 4 executors available.
In your job configuration, make sure you don't run the multi-project sequentially:
Set up throttling builds, selecting "Throttle this project as part of one or more categories", "Multi-Project Throttle Category"(qa-aut) and "Throttle Matrix configuration builds". You can leave in blank the rest of the values
Make sure your node/master has enough executors available. In this case, the master will have available 4 executors
Execute your multi-project job
Instead of using 4 executors (all the availability), you will see it's using only 2 executors (2 threads) as specified in the category.
Is there any difference between I create two slaves, or one slave with two executors on the same Windows server?
Yes, there is a difference: It's about memory consumption and effort of maintenance/administration.
Starting a slave on a system starts a (main) process. This process costs (private) main memory to run and connects to the master.
Each executor is a sub-process of the main process.
It is therefore apparent that running two executors on one slave costs less memory in total compared to running two slaves (with one executor each), as there would be the memory consumption of the main process twice:
2 * Main Processes + 2 * Executors > 1 * Main Process + 2 * Executors
Moreover, administrating a slave is some more effort than just an executor: Whilst an executor has virtually nothing to worry, there are numerous things to configure for a slave. Additionally, the capabilities of the two slaves are anyhow the same (they are running on the same OS as you said), so there is little value-add to also assign it different labels.
In short, if there are no other boundary conditions, which make me do it differently, I always would prefer running two executors on one slave, as this is easier to administrate and some memory is saved.
A slave is a "machine". An executor is an "OS Process" in the slave.
So ideally we always add executors - they do the work and can run in parallel, and the simple theoretic answer to your question is "2 executors on one slave"
In practice we need to add slaves in several use cases:
We need more resources (more cpu, more memory, more "machines")
We need a different setting (Different OSes, Different hardware)
We have global resources that would create a conflict for executors on same machine (shared browser for a UI testing process)
Make the decision based on your use case.
One benefit which immediately comes to my mind for running 1 executor on given node, is to prevent conflicts between processes run at the same time.
On other hand you could prevent job conflicts using existing Jenkins plugins, ie. Heavy Job, Build Blocker.