I am trying to allocate resources in a Jenkins instance. I can create one agent a node and multiple executors under that agent, or I can create multiple agents with 1 executor each for a node. What would be the optimal strategy?
Related
I test in a Jenkins environment two applications, appA and appB: appA needs the resource resourceA, appB needs the resource resourceB. appA and appB can run on same node, they are independent and the required specific resource are independent too.
I have two nodes connected to jenkins: node1 and node2. Each node has resourceA and resourceB, and configured with 1 executor.
Currently my two free-style jobs work fine but the nodes are underemployed regarding available CPU, RAM, network and other resources. And I want to add two other jobs to execute occasionally a test of appA or of appB in this testing environment.
So I am looking for a Jenkins job configuration to define two executors per node and to be able to execute a job for an app on a node where the executor and required resource are available.
I tried to define the node as job parameter (node and label parameter plugin) but this solution is not automatic, on job start it is necessary to specify the node.
I tried the lockable resource plugin but I did not find the way to lock the resource of node1 when the job is started on node1 or the resource of node2 if the job is started on node2
I looked matrix plugin but I think it is not automatic too
Do you know if my idea is feasible? If so, what I need?
Thank you for idea.
Best regards
I have access to an HPC cluster that uses SLURM to schedule jobs. I would like to evaluate having Jenkins submit jobs to the cluster.
As a simple test, it works to add the HPC login node to Jenkins, and configure a Jenkins job that executes srun ... ./myprogram on the login node. The srun command tells SLURM to assign a compute node and execute ./myprogram on it.
This is not an ideal solution. Every Jenkins job has to be written to use srun. Scripts of more than one command are difficult. The compute node is not running the Jenkins agent so features are limited.
Can I have Jenkins instead spawn new nodes on-demand whenever a job is in the queue? It could then execute srun ... jenkins-agent on the login node to spawn a Jenkins agent on some compute node. And then the compute node would connect back to Jenkins and take the enqueued job. I would briefly a node hpc-job-1643 appear and then it would disappear after the job completes.
Yes, you can have Jenkins launch nodes on-demand through Slurm.
Create a new node. Set the remote root directory to something accessible from all nodes of your Slurm cluster.
Set a label that your jobs will use to bring up this node.
Set Usage to "Only build jobs with label expression matching this node".
Select the "Launch agents via SSH" launch method (you need a plugin for this, I think)
Set the host to the hostname of your Slurm control node.
Set up the credentials and host key verification strategy as appropriate.
Click the "Advanced ..." button for the SSH launch method.
Set the Prefix Start Agent Command to srun --pty [options for whatever resources and constraints you need] sh -c '. Note the single quote at the end.
Set the Suffix Start Agent Command to '. That's a single quote.
For Availability, select "Bring this agent online when in demand, and take offline when idle". You'll need to set an in-demand delay (I use 0) and an idle delay.
Currently, we have two machines. One has Jenkins installed and is hosted as master in Jenkins and another one is Slave. Number of executors for both Nodes are set to 1.
I am not exactly sure how Jenkins work behind the scenes but currently when I triggered 2 build jobs simultaneously, it somehow runs only on slave node (and put another build job in queue), if I disconnect the slave and leave only master, then it would run on master(and put another build job in queue).
How to configure Jenkins so that it leverage all my available nodes (master and slave). In other words, I would like to have all available nodes consumes the queue and not just for one of the Nodes.
As I understand, you need to enable Execute concurrent builds if necessary option in your job configuration and then you will be able to run your job simultaneously on all available nodes.
In addition to the above answer. We can also restrict the job to a particular node on which it should run.
For eg
A setup of 3 servers(2 Linux and one windows )
1 Linux server acts as master
1 Linux server acts as node
1 window server as as node
If we have a job that needs to be run on the windows node you can go to the job configuration and restrict the job to run on that node using the node name or label.
Additionally, the no. of executes define the instances of the slave or master node that can be executed parallelly across different jobs.
For running same job you need to check the enable concurrent build option and assign a label having more than 1 nodes in it
Cheers,
Yash
I am curios if it's possible to have one computer/node that would be shared between two jenkins instances?
I know that one can make two same copies of node configurations on Jenkins instances and change workspace, but I am aiming for this scenario.
Jenkins A with Node A
Jenkins B with Node A
Jenkins A starts job on Node A
Jenkins B puts Node A offline
Jenkins B starts job on Node A
Jenkins A puts Node A offline
Node A can only be used once at a time, and there should not be parallel jobs running no matter of different workspaces.
I know that it can be achieved when you have one jenkins instance, but how to do it if you have to use two of them at least?
NOTE: both Jenkins instances are located on different machines.
I believe jenkins mesos plugin is what you needed.
Use following steps to install & configure:
step1
step2
step3
step4
Of course, you had to install a mesos cluster for yourself before do above, see mesos official site. This may cost some time if you are not familar with mesos.
And in step4, you'd better set slave cpus equal with the number of your real slave node.
The principle is:
Mesos first sends resource offer to Jenkins Master A, then jenkins master A will launch tasks on the slave node A, as the resource was used by jenkins master A now. Resource offer will not be sent to Jenkins Master B again, so Jenkins Master B is waiting.
After Jenkins Master A finish task, mesos resource offer for slave node A will be sent to Jenkins Master B, then Jenkins Master B can start task.
The slave can be used by Jenkins Mesos A & Jenkins Mesos B, but never at the same time.
Hope it help you.
This may be a crazy idea but I'm just throwing it.
Is it possible to have one Jenkins master's executors available as slave(s) (executors) from another Jenkins Master?
I.e. Let's say JenkinsMaster1 (has 10 executors). It has bunch of slaves (in various OS with various # of executors per slave) but all of them are used/running something.
There's another JenkinsMaster2 and this instance has the same setup (bunch of slaves with N no. of executors) but this one has some/a lot of free executors (on master or it's slaves).
The question is NOT, why I can't just create a new slave for JenkinsMaster1 if I need a job configured in JenkinsMaster1 instance to run (while every other executor in JenkinsMaster1/its slave are in use) or why not add more/increase executors of JenkinsMaster1 master/slaves BUT how can/is it even possible to use JenkinsMaster2's executors (or it's slaves i.e. owned by JenkinsMaster2) to run a job which is configured on JenkinsMaster1.