I have a very lightweight job that should be executed immediately when it is triggered, rather than waiting an hours for current jobs to finish.
As I understand, a flyweight task is what I want. It will create a ephemeral executor, just for that task.
How can I make a job be run as flyweight?
I have recently had the same problem. My company has a lot of jenkins projects and some have more precedence over others, and we limit the number of executors to only 4.
Therefore, we decided to create some slaves, instead of always building on the master. Create a slave node that only builds your "very lightweight job".
Go to Manage Jenkin -> Manage nodes -> New node -> Dumb slave.
Then configure your slave node to your liking. Now configure the "very lightweight job". Make sure that This build is parameter is checked, then Add parameter -> Node.
Then select the slave node that you just created. There are a lot of configurations, such as which node do you want to default, but I think you can customize that to your liking.
Try out this FlyWeightProject Plugin. It is an extension of the Freestyle type that runs in Flyweight.
AFAIU the issue is that all your executors are occupied when it comes to running this high priority job.
What about:
Establishing another slave node; in a VM, for instance; with an appropriate number of executors, in case there are more of these high priority jobs
Assigning a label like high-priority to this new node
Restricting where this high priority job can be run to this high-priority label
Assigning a label like long-running to all other nodes
Restricting where all other jobs can be run to this long-running label
Another possibility is to:
Configure the new node's Usage: Only build jobs with label restrictions matching this node
Restrict where the high priority job can be run to the new node
This avoids having to create and assign labels to all jobs, as mentioned above, but is less flexible for future adaptions and extensions.
Related
I have a Jenkins master and two agents. However the connectivity to one agent(agentA) is bit shaky and I want to use the other agent(agentB) when the connectivity to the first one is not available.
I am only using the Jenkins web interface and have not used scripts. I am trying to figure out how it can be done using the "Restrict where this project can be run" option in job's configuration. I tried using agentA|| agentB but when agentA is not available it hangs saying "pending - agentA is offline"
Is it possible to have a configuration to achieve what I need?
I can;t leave it blank because I have other agent (agentC, agentD) which do not want this job to run in.
I am not an admin of the Jenkins server, hence adding new plugins is not my preferred option but it can be done.
As noted in Least Load plugin,
By default Jenkins tries to allocate a jobs to the last node is was executed on. This can result in nodes being left idle while other nodes are overloaded.
As you generalized the example, I'm not 100% sure if your situation can be solved by simply better labelling of your nodes or you want to look at least load plugin (it is designed for balancing the load across nodes). Your example appears to show Node names (ie; agentA/agentB). The Queue allocation logic may be "Only A or Only B", then Jenkins sticks to it. Load balancing may not address that as while a Node (a Computer) name is also a label, it may have additional logic tied to it.
If you label the pair of nodes in a pool with a common label, say "CapabilityA", and constrain your jobs to run where "CapabilityA" rather than the node names, you may find jobs float across the pool (to B if A is not available. That's how we have our nodes labelled - by Capability, and we see jobs floating across nodes, but only once the first node is full (4 executors each), so not balanced.
Nodes can have many labels and you can use label conditions to have complex constraints.
The way we're using Jenkins requires us to have two nodes defined for each machine. One Jenkins node runs as a normal user (called Normal), and the other runs as the administrator (called Admin). So they show up as two separate nodes, even though they exist on the same slave machine.
But, we're running into a concurrency problem. Because our job switches between the two nodes, there is a possibility of another job (Job B) being assigned to (for example) the Normal node, while the Admin node is working on its part of (e.g.) Job A.
Is there a way to tell Jenkins that if either the Normal node or the Admin node of a machine is being used, then it should NOT give the other node some other job?
To elaborate on this question--we have a test suite that we currently run serially. All of our Jenkins masters have multiple slaves, so naturally we would like to take advantage of parallelization, so the suite doesn't spent 2 hours using one machine while the other ones sit idle. So it's not really a matter of ensuring only one job runs at once, it really is a matter of telling Jenkins not to use a node when its partner node is busy.
The issue is not related to two nodes on the same machine or one privileged or not; it's a matter of blocking one job from running while the other is still running.
I trust you are using labels to restrict what jobs can run on what nodes
You can use Build Blocker plugin to block the job from running while others are . There are other plugin options which may work for you as well.
You can also use the Paramterized Trigger to in-line the execution of the other job. It can be run as a build step or a post-build step.
You can also restrict the number of executors on a given node via ${JENKINS_URL}/computer/myNode/configure | # of executors, so you don't run multiple jobs one the same node if that's an issue.
Here's the way I solved this problem:
Set the number of executors on each slave node to 1.
Force my job to take an executor for the whole length of the job.
Specifically, in the groovy script that we use for all our jobs, at the very top, after we find which two (admin and normal, running on the same slave) nodes we need, we use the following:
n
node(myNormalNode)
{
//all the rest of the job, including:
node(myAdminNode)
{
//other commands for the admin node
}
//back to commands for the normal node
node('master')
{
//code to run on master
}
//and so forth
}
This makes Jenkins not assign any other jobs to this computer until the first job is done.
Got a pipeline job who can run at 4 different nodes with one label. Previously i got the problem that they randomly tried to run at the same node, so i installed the lockable recources plugin and tried this:
node('TEST') {
try {
notifyBuild('STARTED')
lock(env.NODE_NAME){
This works generally, but it seems to be random wich node from the Label TEST the job chooses. For example the first two job executions can choose the same node and so the 2nd job will have to wait even if there are free nodes available. Is there a way to secure that all nodes are used before jobs have to wait?
Better solution is the https://github.com/jenkinsci/throttle-concurrent-builds-plugin which also works for pipeline jobs. This plugin doesn´t checks if recources are available before it blocks them. Also all recources are used before jobs have to wait.
We use jenkins for automation for our test infrastructure. The requirement is to give users the ability to use a jenkins node for their private test or debug using private jenkins jobs and then put back in the pool of nodes marked with labels; so that other jobs that were marked to run on particular labels can be run without interference.
We can achieve this by letting users alter label, but that didnt workout as users (nearly 50) are making their own label names and it takes time for admin to reassign the nodes (even with process) and precious test time is getting affected.
we are looking for some solution such as ability to provide buttons like take this node offline (cant use this option since jenkins cannot see the node anymore and so users cannot run jenkins job on the node) but may be with the ability to run scripts.
I have done some research on this but have to compromise on some requirements, so i decided to seek help from the community... SUGGESTIONS?
Did you have a look to this question:
How to take Jenkins master node offline using CLI?
In the 1st question, there are some CLI to make a node offline.
Maybe you can create a dedicated job on the master with one parameter (the node name). This job will call the Jenkins CLI to stop your node.
Given several nodes available Jenkins nodes (NodeA, NodeB, ...), is there a way to define a job such that it gets 2 nodes?
For example, Job1 gets assigned to NodeA & NodeD.
To make a job run TWICE - both on node-A AND node-D, use the Matrix-build Plugin (note it is a little tricky to configure and monitor).
To make a job run ONCE - either on node-A OR node-D, give those two nodes the same LABEL, and RESTRICT the job to run on that Label.
Cheers
You'll need to submit the job twice and use the NodeLabel Parameter Plugin to choose which node to execute each job on. To submit both these jobs at the same time, you could try using the Build Flow plugin
Why are you trying to do this? Are you perhaps trying to use slave agents to install software? Perhaps you could consider using an orchestration tool like rundeck (Which also has a Jenkins plugin)