For internal reasons, one of my jobs is able to run concurrently, but new builds abort themselves if another build is already running (disabling concurrency doesn't help, since I don't want new jobs to be scheduled for execution once the current build is done).
However, this behaviour is detrimental to the job status preview (the colored ball next to the job name when inside the job's parent folder). It often shows the status as "aborted", which is undesirable - I want to view the latest running build as the source of the job status.
I tried deleting aborted builds from within their own execution, but that's unfortunately neither trivial nor stable, and thus not suitable for this situation. I could probably get a workaround running that deletes them from a separate job, but that's not ideal either.
Anyway, I'm now hoping that I can just tell Jenkins to ignore "aborted" builds in the calculation of the job preview. Unfortunately, I wasn't able to find a setting or plugin that allows me to do this.
Is this possible at all? Or do I need to find another way?
Would something like this help?
https://plugins.jenkins.io/build-blocker-plugin/
I haven't used it myself but it supports blocking builds completely if a build is already running.
Related
Doesn't anyone know of a Jenkins plugin that ensures that when a job is re-run, it runs on a different node to the previous one? Problem is not all developers have permission to take problematic nodes offline, so they end up not being able to build their branch because Jenkins keeps trying to re-run on the same one.
Assuming all your nodes have same configuration; You can group them under same Label;
To re-run the build you can try -
https://wiki.jenkins.io/display/JENKINS/Naginator+Plugin
You might want to configure node to get offline when idle for certain time.
And aim to re-trigger the build after that time; There is a fair chance the build would run on another Node.
But the question is, what are the chances of build getting successful if nothing has changed. Re-trigger without changes doesn't give successful build unless there is a temporary issue.
I currently have hundreds of builds in the build queue. It will take several days for them all to be built.
I want to bring a specific build to the front of the queue to have it build next.
Currently, the only way I have of doing that is to kill all of the other jobs in the build queue and then re-trigger them.
It seems that surely there must be a better way.
Is there?
Looks like there is no such built-in functionality.
Please, have a look at the jira issue: Ability to move job to top of queue
Although, there is a plugin which seems like a possible solution for you: Accelerated Build Now Plugin
The Jenkins Accelerated Build Now Plugin allows Jenkins users to launch a project's build right away, even if the queue is long (moving it to the top of the queue) and even if no executor is available (killing and rescheduling builds not launched by "humans")
Maybe this plugin will help you : https://wiki.jenkins-ci.org/display/JENKINS/Accelerated+Build+Now+Plugin
https://plugins.jenkins.io/simple-queue/ is a plugin that allows changing a build queue order from UI manually. And it seems to be a viable solution in 2022.
I have two Jenkins jobs, COMPILE and TEST, COMPILE triggers TEST, COMPILE is quick, TEST is slow. COMPILE re-creates data which is used by TEST, so if COMPILE runs while TEST is running, TEST might fail due to the necessary data being temporarily unavailable or incomplete.
Sometimes, COMPILE gets triggered a lot (via CMS, busy development). The standard way would be to synchronize COMPILE and TEST via a lock, so that both never run at the same time but instead wait for the other to finish before starting. This waiting does not really suit me as it delays the COMPILE jobs too much.
An alternative might be to turn TEST to concurrent running, but in my case TEST requires too many resources to be able to run concurrently.
So my approach now is to configure COMPILE so that it first aborts a running TEST job (in case one is running) and then starts its work (eventually triggering TEST again in the end). Several quickly performed COMPILE builds will then each start a TEST build which will all be aborted (gray bubble). Only the last TEST build will be completed and show a decent red or green (or yellow) bubble. But that will be enough in my case (and I accept the drawback that this way I cannot detect exactly which commit broke the build).
Now the question is: How can I make COMPILE abort a TEST build? (Preferably in an elegant way.)
I only found a way to generally abort a job from the outside using Jenkins's REST interface, but that doesn't seem to be very elegant. Is there a more elegant way, maybe using a Jenkins-internal feature I don't know or maybe by using a suitable plugin?
I would suggest consolidating the two jobs into a single job. That would allow for multiple simultaneous executions and will still fail if the compile fails.
You can set the number of Executors for the node to "1" . By this we can make sure only one jobs run at a time in the node , even if it is executed in parallel it will wait for the 1st job to complete and then start the second
General Symptoms
We use Jenkins to build & test on multiple platforms. We use the Configuration Matrix plugin to help with this. Occasionally (increasingly often) Jenkins will mark the Configuration Matrix master job or subjobs as failed when the jobs seem to have succeeded (the console output reports success). We have no idea why this is happening. Any suggestions?
Some clues:
The exit code of the Jenkins job's script is not relevant. We've had test sub-jobs that simply exit 0 and they can still exhibit this bad behavior.
The failures are bunchy. They seem to come in groups.
The failures tend to effect our Windows platforms more heavily but the issue occurs on our Mac nodes as well.
The failures clump on a single node for a time but they are not exclusive to any 1 node.
We've noticed that the failures happen most often with load, particularly failed sub-jobs are started later than their successful sibling sub-jobs (often after other siblings have already completed).
We suspect that the sub-jobs are somehow considered completed by the master before they actually complete. Since they're not done the master sees them as failed. Later the sub-job really does complete (thus the console output says Success). We suspect this because we've added comments to "failed" jobs which look incomplete only to return later and see additional console logs.
It turns out that there is a bug in either the Jenkins "Set Build Name" plugin or the "Configuration Matrix" plugin. When you use them both you're subject to a few bugs. First is the symptoms I described above. Second is that the names set on builds can be wrong (race conditions mean that the wrong name can be applied).
There is a ticket open against Jenkins here. Unfortunately there isn't currently a posted work-around. We may simply stop using one of the plugins.
My fairly-large project uses gated builds, with a heavily-customized process template (XAML). For reasons beyond the scope of this question, our process has a single SharedResourceScope, so multiple builds don't run in parallel (I know you're supposed to do that with an Agent scope, but we switch agents in the middle, so wrapped everything with the SharedResourceScope).
Now, if there are several check-ins in queue, all of them go into "In Progress" state, and all but one wait on the SharedResourceScope. This means that:
People can't know which build is actually running
Even if I set a new queued check-in to be high-priority, it can't overtake all those who are in progress and waiting on SharedResourceScope, so the whole priority setting has little meaning.
I've experimented using DefaultProcessTemplate.XAML, and I see that usually only one build is In Progress (though Occasionally I see 2 builds).
Questions:
When exactly does a build start, and therefore goes into "In progress" mode? What prevents all builds from starting immediately, and blocking on AgentScope / SharedResourceScope?
Is there something I can author in my XAML to prevent all builds to go in progress?
A build starts when the build controller has capacity to start a new build. Since builds can use 0 to n agents, the controller doesn't wait to get an agent before the build starts. The controller determines its capacity based on the "Maximum number of concurrent builds" setting on the controller properties (in code: MaxConcurrentBuilds).
Default setting: "Default to number of agents".
No way in XAML to control this behavior.
Also, there's a bug in TFS2010 regarding this, hotfix: KB2567437
Using TFS Build Extensions, you get a number of activities with which you can administer agent control through the process: QueryBuildAgentStatus, IsBuildRunning, SetBuildAgentStatus to name a few that could be useful to you. They're fairly undocumented as of now, so you'll have to experiment to get them running. But there is a help file included in the package that is useful.
As for the "In Progress" mode, I've noticed that this is the status even if a build is waiting for a free agent; check the log of such a build and you'll see. This is quite confusing, and I hope that MS will add a "Queued (but not started)" mode.