I have a job that kicks off on any commit. It takes 5-10 minutes to run.
But if (say) 4 or 5 git commits come back-to-back I don't want 4 or 5 jobs run - just one job for the last commit. So basically if there is a job of type "X" in the build queue I don't want another job of type "X" in the queue.
That should be the default behavior if you're using the SCM trigger, default job parameters, and don't check the 'Execute concurrent builds if necessary' option.
First job is going to queue and run immediately.
On source change, next job is going to queue and wait until first one is complete.
A third SCM change would detect job already in queue and not do anything.
When first job is done, next one will start - and will use whatever is in the SCM at the moment it starts (not the moment it was scheduled).
That behavior can be changed using parameters, concurrent builds, job throttling, etc. My knowledge there might also be outdated (Jenkins is evolving pretty fast).
On a side note: multiple builds are not necessarily a bad thing - they give you failure locality, which might allow you faster identification of the offending commit. It doesn't matter much for 10 minutes builds, but if your build grows larger than that it can be a problem (with a large team, you can have a LOT of commits in 30 minutes).
Basically you just want to check if there is a new commit every 5 or 10 minutes? You can do that inside the triggering configuration: monitor source control every X minutes (CRON syntax: */15 * * * * for every 15mins)
If you check every 15 minutes if a new commit happened and your jobs only takes 10 minutes to run, there is no chance you would have another execution pending (unless someone ask for a "manual" construction...).
To avoid the latter case, you may consider the Throttle Concurrent Builds plugin
Related
I have a job and it can run concurrently. The trick part here is that this job needs to have an interval to one another, at least for 30 seconds. So when Build1 runs, Build2 should wait for 30 seconds to start.
I already tried using the quiet period but it does not fit my needs (it only works when a job is not triggered by Build Now or Build With Parameters)
Is there a way for me to be able to do this kind of condition?
You can try enumerating other builds of this job and sleeping until all the other builds have been running above 30 seconds. See example code in this answer.
I am building a project in Jenkins and want to launch tests right after it, wait
until the tests are finished and than run another job to analyze the results. The testing system is a close system (I can't modify it) so in order to check if the tests are finish I need to query the system every X seconds. One way to so that is to create a job that will query the system but it will take a slot (I can create 1000 slots but it looks like a hack). is there another way to make the job "sleep" while it is waiting for the next X seconds so it will not take a slot while waiting for another process to finish ?
You can trigger one Jenkins job from another. No need to make jobs sleep or anything complicated like that. Look at upstream and downstream triggers using the parameterized build plugin.
https://plugins.jenkins.io/parameterized-trigger
My use case:
Job A is set to run Monday through Friday at 18:00.
Job B is dependent upon Job A succeeding but should only run Monday through Friday at 06:00. (Monday morning's run would depend upon Friday evening's run). I prefer set times rather than delays between jobs.
On any given morning, if I see that Job A failed (thus Job B never ran), I would like to be able to run (fix) Job A then immediately trigger Job B.
What I have found so far only offers part of this use case. I have tinkered with Pipeline and recently upgraded my Jenkins instance to 2.89.3, so I have access to the most recent features and plugins. Filesystem triggering seems doable.
Any suggestions are appreciated.
You can use the options available in "Build Triggers".
Ex:
Build Trigger
Hope this work for you!
This is a tricky Use Case as generally you want a job to immediately follow on from another one rather than waiting for potentially three days.
Further complicated by wanting it to run straight away when you want it to.
I do not believe there is a "I have finished so kick this job at this time" downstream trigger So for the first part the only things I can think of are:
Job A kicks Job B as soon as it is finished and job B sits there with a time checker in it and starts its task when the time matches.
or Job A artefacts a file with its exit status and job B has a cron trigger for 6am mon-fri and picks up this artefact and then runs or doesn't dependent on the file contents
For the second part you could get the build Cause (see how to get $CAUSE in workflow for pipeline implementation and vote on https://issues.jenkins-ci.org/browse/JENKINS-41272 to get the feature when using sandbox).
And then get your pipeline to behave differently depending on trigger
i.e. if you went for the second option above then In job B you could do if triggered by Cron then read the artefact and do as needed. If triggered by Upstream then just run regardless.
I am dealing with a build pipeline that has some very long wait times in between builds due to outside dependencies. I've found that you can indeed tell a build to sleep before it executes its build steps here. However, I was wondering if there is a limit to how long the sleep can last for. In some cases I'd like builds to wait for 24 hours between builds in the pipeline, inputting 24hrs as 86400 seconds is a little unsettling, but I suppose it's not that unreasonable.
There is no limit implicitly within Jenkins. It will bee limited by your infrastructure reliability and the like.
If you are using jenkins pipeline, ensure that the wait's do not occur whilst consuming an executor (inside a node block).
It may be better, (again if using pipeline) to use a timeout() block rather than a arbitrary sleep, so it resumes as soon as ready.
https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/#code-timeout-code-enforce-time-limit
I have around three Jenkins slave that are configured to run the same job allowing only one concurrent run on each slave. Each of these slave is connected to an embedded hardware that we run the job on. The total duration of the job is around 2 hours. The first 1 hour 50 mins is just taken to compile and configure the slave and the last 10 mins is when the embedded device is used. So basically I was looking for something that I can lock on for the last 10 mins. This would allow us to run multiple concurrent builds on the same slave.
Locks and Latches locks are shared across nodes.
What I am looking for is a node specific lock
If you can separate the problematic section from the compilation process you can just create another job to handle the last 10 minutes and call it using Parameterized Trigger Plugin. This job will run one instance at a time and will act as a native blocker for the run. That way, you can configure concurrent executions and throttling (if needed) on the main job, and create a "gate" to the problematic section.