Our test suite relies on a number of subsidiary services being present - database, message queue, redis, and so on. I would like to set up a Jenkins build that spins up all the correct services (docker containers, most likely) and then runs the correct tests, followed by some other steps.
Can someone point me to a good example for doing such a thing? I've seen a plug-in for mongo, and some general guides on spinning up agents, but their relationship to what I'm trying to do is unclear.
One possibility is to use the JenkinsCI Kubernetes plugin and jenkinsCI Kubernetes pipeline plugin: they will allow you to
launch docker slaves automatically,
with container group support through podTemplate and containerTemplate.
Currently, I am working in a quality process so as to ensure that the code is acceptable. For that, I'm integrating Jenkins, SonarQube and GitLab, which are running in different servers (actually they are in different docker containers).
The idea is to check with SonarQube everytime the code is pushed against GitLab and block commits, merges, and so on, whether SonarQube has not passed.
I have already integrated Jenkins with SonarQube, but Jenkins checks the code inside his workspace, so imagine a situation where a developer in his laptop needs to push his changes.
My conceptual question is simple: Is it possible to integrate these technologies in order to do this? And, if the question is yes, which steps are necessary?
PD: I don't need to see code, configuration files,and so on. I just need something like:
Configure SonarQube to work with Jenkins
Do an script so as to copy that file in that folder,
...
First, in docker means each tool is in its own container.
They only need to see each other through the network, which is where a Docker Engine in Swarm mode comes in.
Second "configure Jenkins to work with SonarQube"... that is what I have done in my shop, and there isn't much to it.
Once the Jenkins SonarQube plugin is installed, and the address for the SonarQube server entered, you can configure your job and call sonar (for instance with maven: $SONAR_MAVEN_GOAL -Dsonar.host.url=$SONAR_HOST_URL)
The analysis done in the Jenkins workspace will then be published in the SonarQube server.
A swarm server is the more modern version of this 2015 docker-compose.yml file from the marcelbirkner/docker-ci-tool-stack project.
The idea remains the same though: each element is isolated in its own container.
I haven't tried It myself but https://gitlab.talanlabs.com/gabriel-allaigre/sonar-gitlab-plugin could be interesting in your setup.
I've been struggling with setting up the Jenkins Kubernetes Plugin on the Google Container Engine.
I have the plugin installed but I think all my builds are still running on master.
I haven't found any good documentation or guides on configuring this.
UPDATE
I removed the master executor from my Jenkins image. So now my builds aren't running on master but now they have no executor so they don't run at all. Just waits in the queue forever.
You'll need to tell Jenkins how and where to run your builds by adding your Kubernetes cluster as a 'cloud' in the Jenkins configuration. Go to Manage Jenkins -> Configure System -> Cloud -> Add new cloud and select 'Kubernetes'. You'll find the server certificate key, user name and password in your local kubectl configuration (usually in ~/.kube/config). The values for 'Kubernetes URL' and 'Jenkins URL' depend on your cluster setup.
Next, you'll need to configure the docker images that should be used to run your builds by selecting 'Add Docker Template'. Use labels to define which tasks should be run with which image!
Here's a good video tutorial and here you'll find a nice tutorial which explains everything in detail.
The important bit after you've installed the plugin, set up access to your Kubernetes cluster, and set up your first Kubernetes Pod Template with a label like jnlp-slave, is that in your Jenkinsfile you need to begin with something like node('jnlp-slave') {}. Then the pod will be started when you trigger a build.
There's also a helm chart for easy deployment if that helps :)
This example might also help once you've set the plugin up too.
I'm trying not (not hard enough it seems) to get our jenkins server to provision a jenkins-slave using docker.
I have installed the Docker-plugin and configured it according to the description on the page. I have also tested the connectivity and at least this part works.
I have also configured 1 label in the plugin and in my job. I even get a nice page showing me the connected jobs for this slave.
When I then try to start a build nothing really happens. A build is scheduled, but never started - (pending—Waiting for next available executor).
From the message it would seem like jenkins is not able to start the slave via docker....
I'm using docker 1.6.2 and the plugin is 0.10.1.
Any clue to what is going on would be much appreciated!
It seems the problem was that I had added the docker version in the plugin config. That is apparently a no-go according to this post
Jenkins won't execute any jobs. Having viewed this question, I have disabled all slave nodes but a simple job won't even run on the Master node.
What is wrong?
The Jenkins admin console can run, even with the Master node offline. This can happen when Jenkins runs out of disk space.
To confirm, do the following (with thanks to geekride - jenkins-pending-waiting-for-next-available-executor):
go to Jenkins -> Manage Jenkins -> Manage Nodes
examine the "master" node to see if it is offline. It may be reporting that the master node is out of disk space.
go to Jenkins -> Manage Jenkins -> Manage Nodes
examine the "master"
node.(Click on configure icon)
In my case No of executors was set to 0.
Increased it and issue got fixed.
In my case, I had the following set in my JenkinsFile
node('node'){
...
}
There was no node called 'node', only master (the value had been left in there after following some basic tutorials). Changing the value to 'master' got the build working.
In my case it was caused by number of executors (I had 1) and running Jenkins Job (Project) from Pipeline (my pipeline script started other Job in Jenkins). It caused deadlock - my pipeline held executor and was waiting for its job, but the job was waiting for free executor.
The solution may be increasing of # of executors in Jenkins -> Manage Jenkins -> Manage Nodes -> Configure (icon on required node).
I ran into a similar problem because my master was set to "Leave this machine for tied jobs only." So, even though I disabled the Slave, Jenkins kept on bypassing the Master, looking for something else.
Go to Jenkins --> Manage Jenkins --> Manage Nodes, and click on the configure button of your master node (looks like a screwdriver and a wrench). Check the Usage and make sure it's on "Utilize this slave as much as possible."
In my case, I had just installed the "Authorize Project" plugin and incorrectly setup the strategy in "Manage Jenkins -> Configure Global Security -> Access Control for Builds" as "Run as anonymous". So 'anonymous' had no rights to execute the job.
Setting the first strategy as "Run as User who Triggered Build" unlocked the queued jobs.
I'm a little late to the game, but this may help others.
In my case my jenkins master has a shared external resource, which is allocated to jenkins jobs by the external-resource-dispatcher-plugin. Due to bug JENKINS-19439 in the plugin (which is in beta), I found that my resource had been locked by a previous job, but wasn't unlocked when that previous job was cancelled.
To find out if a resource is currently in the locked state, navigate to the affected jenkins node, Jenkins -> Manage Jenkins -> Manage Nodes -> master
You should see the current state of any external resources. If any are unexpectedly locked this may be the reason why jobs are waiting for an executor.
I couldn't find any details of how to manually resolve this issue.
Restarting jenkins didn't resolve the problem.
In the end I went with the brutal approach:
Remove the external resource
(see Jenkins -> Manage Jenkins -> Manage Nodes -> master -> configure)
Restart jenkins
Re-create the external resource
In my case, I noticed this behavior when the box was out of memory (RAM)
I went to Jenkins -> Manage Jenkins -> Manage Nodes and found an out of memory exception.
I just freed up some memory on the machine and the jobs started to go into the executors.
Short answer:
Kill all the jobs which are running on the master.
In my case there were 3 jobs hung on the master for more than 10 days which were unnoticed. We usually do not run any jobs directly on the master, everything is run on slaves. I killed these 3 jobs which were hung, automatically the executors on the slave started picking up jobs.
Point to note that even though we have 8 slaves only 1 slave was in this affected state.
[EDIT] We found the answer to why only one slave was in this affected state.
When a Jenkins slave goes down, all the pending jobs automatically get transferred over to the master. All the 3 hung jobs which I killed were from this slave, so its likely a connection issue between the master and this particular slave.
For me below solution worked.
Jenkins --> Manage Jenkins --> Manage Nodes --> master -> configure -->
Node properties --> Restrict Jobs execution at node - is enabled and given access to specific users. I have given access myself and then job started to run.
If Restrict Jobs execution at node is enabled scheduled tasks cannot run.
In my case it is similar to #Michael Easter: I got a problem in a job due to lack of disk space. I cleared some space, restarted Jenkins but still the problem persisted.
The solution was to go to Jenkins -> Manage Jenkins -> Manage Nodes and just Click on the button to update the status.
In my case I've to set Execute concurrent builds if necessary in job's General settings.
I had just added a stage with a docker agent. Since I only had one node, master, I had to tell the container to reuse the node from the earlier stages:
agent {
docker {
image 'bitnami/mongodb:latest'
reuseNode true
}
}
You can vote to have this be the default behavior for agents and prevent this kind of lock.
In case you have installed the Parameterized Trigger plugin, a job waiting in the build queue might be a known issue JENKINS-47792.
A workaround fix is to downgrade the Parameterized Trigger plugin to version 2.35.1.
Note that you may be required to downgrade dependencies, such as the git plugin, as well.
What worked for me: I finally noticed the Build Executor Status window on the left on the main Jenkins dashboard. I run a dev/test instance on my local system with 2 executors. Both were currently occupied with builds that were not running. Upon cancelling these to jobs, my third (pending) job was able to run.
For me I have to restart the executors manually. Click on "Dead" under "Build Executor Status" and push the restart button.
I ran into a similar problem because my master was set to " # of executor (The maximum number of concurrent builds that Jenkins may perform on this agent).
Go to Jenkins --> Manage Jenkins --> Manage Nodes, and click on the configure button of your master node (increase the number of executor to run mutiple jobs at a time).
You may have milestones set up and your job won't run until the previous job is complete.
In my environment (images jenkins:lts-jdk11, jnlp-agent-maven e jnlp-agent-maven:jdk11), doing a pull for update jenkins image, reports no connectivity problem on agents. But all jobs stayed blocked with message:
pending—Waiting for next available executor on
So, the solution for me was stop and make a pull in both agents image:
docker pull jenkins/jnlp-agent-maven (jdk8)
docker pull jenkins/jnlp-agent-maven:jdk11
when you met this
just restart jenkins mater ,then recovery.