I have a pod template declared in configure clouds section and I am using jenkins/inbound-agent:4.3-4 image, build agents are coming up fine but they are coming up with just one executor, is there a way I can increase that number?
The reason I would like to increase the number of executors is, I want to create a job which triggers other jobs sequentially and I want to all the downstream projects to run on the same agent as the main job.
I don't see any option in configure cloud section, any heap or clue on workarounds is appreciated.
I met the same issue with the same situation.
I found a post which similiar with this situation,maybe the kubernetes plugin is as same as amazon-ecs plugin,they both hard coded the executors value as 1.
Jenkins inbound-agent container via ecs/fargate plugin - set # executors for node
So, run pipeline steps one by one is the only way I know to avoid this issue.
If you need call other job,set wait:false would be work, like this
build(job: "xxxx", wait: false, parameters: [string(name: "key", value: "value")])
Related
I have multiple jobs set on Jenkins. There are only one executor on masters, there are many other slaves. I want to create a job which would run on a separate job queue or a separate executor concurrently. How can I achieve this in simplest way? Can it be achieved without modifying slaves?
The next thing is how can I achieve running on any new build for this job in parallel. Rest of the jobs set should not be disturbed and interfered.
As I can understand you need to configure a particular job to run on a slave node...
In order to achieve that you can 1st add a node to the Jenkins master. There you have a place to add a label to that slave node. The in the pipeline script of your job you can edit it as follows.
pipeline {
agent {
node {
label '<YOUR SLAVE NODE LABEL>'
}
}
}
In the latter part of the question, you have asked to run parallel jobs. Please refer to the parallel job documentation If you need to add sequential stages refer to the sequential stage documentation If you want to create a dynamic parallel job this link would help you.
Still waiting to schedule task
‘docker’ is offline
JenkinsFile:
pipeline {
agent {
node {
label 'docker'
customWorkspace "workspace/${JOB_NAME}/${BUILD_NUMBER}"
}
}
...
What is the cause of this error, and how can I diagnose it further?
I don't see any related containers running via docker ps.
Relating to the agent named docker. They can be viewed on the "Build executor status" / "Nodes" / https://jenkins-url.local/computer/
If you use node {} with a specific label, and don't have any nodes with that label set-up, the build will be stuck forever. You also need to make sure you have at least 2 executors set-up when using a single node (like 'master'), otherwise pipeline builds will usually be stuck, as they consist of a root build and several sub-builds for the steps.
I am trying to setup various Jenkins pipelines whose last stage is always to run some acceptance tests. To cut a long story short, acceptance tests and test data (much of which is shared) for all products are checked into the same repository which is about 0.5 GB in size. It therefore seemed best to have a separate job for the acceptance tests and trigger it with a "build" step from each pipeline with the appropriate arguments to run the relevant tests. (It is also sometimes useful to rerun these tests without rebuilding the product)
stage('AcceptanceTest') {
steps {
build job: 'run-tests', parameters: ..., wait: true
}
}
So far I have seen that I can either:
trigger the job as normal. But this uses an extra agent/executor,
there doesn't seem to be a way to tell it to reuse the one from the
build (main pipeline). Both pipelines start with "agent { label 'master' }" but that
seems to mean "allocate a new agent on a node matching master".
trigger the job with the "wait: false" argument. This doesn't
block an executor but it does mean I can't report the results of the
tests in the main pipeline. It gives the impression that the test
stage has always succeeded.
Is there a better way?
I seem to have solved this, by adding "agent none" at the top of my main pipeline and moving "agent { label 'master' }" into the build stage. I can then leave my 'AcceptanceTest' stage without an agent and define it in the 'run-tests' job as before. I was under the impression from the docs that if you put agents in stages then all stages needed to have one, but it seems not to be the case. Which is lucky for this usecase...
I don't think that there's another way for declarative pipeline.
On the other hand for scripted pipeline you could execute this outside of node {} and it would just hold onto one executor on master releasing the one on slave.
stage("some") {
build job: 'test'
node {
...
Related question: Jenkis - Trigger another pipeline job in same machine - without creating new "Executor"
I'm trying to test out the parallel functionality for a Jenkins pipeline job, but for some reason the individual build steps of the parallel job never get passed off to an executor and processed. Normal single-threaded pipeline jobs have no issue processing. I tried restarting the Jenkins server in case some resources were locked up, but it did not help.
The full script I'm trying to execute is:
def branches = [:]
branches["setup"] = {node("nsetup") {
echo "hello world"
}}
parallel branches
I have only one node, the master, and it has 5 available executors. It is configured to "use as often as possible". I'm pretty new to Jenkins and setting up a server for the first time, so maybe there's something I missed in the configuration that isn't related to the job.
Does anybody have any suggestions?
And 2 minutes after I post I figure it out! Every time.
Turns out I just didn't have any idea how the "node" command really works. By specifying a parameter in the parentheses, it was preventing it from releasing to an executor. I'm guessing that must tell it to try executing on a certain node matched by label, and I was using it like it was some random logging field. Oops!
I'm new to Jenkins and I'm trying to setup a project which will use few build executors.
The flow shall be as follows:
two build executors with webservice label return their IP addresses and wait for the third build executor to finish its job
third build executor with tester label collects those IP addresses and performs some long running job (e.x. sends HTTP requests to the webservices deployed on those two agents)
How to achieve such behavior in Jenkins?
I've found that when an build executor finishes its job it is immediately released and I don't know how to make it wait for other build executors to finish their jobs.
Edit:
I forgot to mention that I want the build executors with the webservice label to be reserved (not available for other jobs) till the build executor with the tester label will finish its long-running job.
Also all these build executors should be on separate slaves each. That means each slave has only one build executor.
I've finally managed to do this using Pipeline and below script:
node('webservice') {
def firstHostname = getHostname()
node('webservice') {
def secondHostname = getHostname()
node('tester') {
println 'Running tests against ' + firstHostname + ' and ' + secondHostname
// ...
}
}
}
def getHostname() {
sh 'hostname > output'
readFile('output').trim()
}
It acquires two build executors with webservice label. I'm getting their hostnames (I'm using them instead of the IP addresses) and pass them to the build executor with a tester label. Finally the tester runs some long-running tests.
Those two webservice build executors are blocked till the tester finishes its job, and no other project may use them during that time.
As Alex O mentioned, you can configure the master and slave relationship between the projects /executors inside the Jenkins projects /executors. There is option for that, "Build Triggers" -> Build after other projects are built
or use plugin to achieve it
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Trigger+Plugin
or
https://wiki.jenkins-ci.org/display/JENKINS/Join+Plugin
What you actually want is probably that your job uses three slaves at the same time.
Re-thinking the setup in that way, it won't be necessary to consider the collection of IPs and the subsequent usage of the slaves as three different steps that must be aligned in some way.
Unfortunately, Jenkins does not support using multiple slaves for one build out-of-the box, but it will be possible to achieve what you want e.g. using the Multijob plugin and the Join plugin that Aaron mentioned already.
See also this question for information on how to use two slaves at the same time.