I want to run my Jenkins behind k8s. We can achieve that with any standard helm chart or our own manifest files. In this case, Jenkins (master only) will run inside a container (Pod).
Now I also want to have a pipeline job that uses docker agent as described here
I am getting confused, about
how and where this docker container will be run (on the same node where Jenkins is running? and suppose the node capacity is over then it needs to run docker agent on a different node)
how does Jenkins will authenticate to run containers on k8s nodes?
I saw the Kubernetes plugin/docker plugin. But those plugins create containers beforehand (or at least we need to set up a template, which decides how containers will start, which image will be used and many more) and connects Jenkins with help of JNLP / ssh. I lose the flexibility to have an image as an agent in that case.
going further, I also like to build custom images on the fly with help of Dockerfile shipped along with code. An example is available in the same link.
I believe this documentation is answering all of your questions: https://devopscube.com/jenkins-build-agents-kubernetes/
With this method, you are not losing your flexibility because your Jenkins master going to create a K8s pod on the fly. Yes, additionally you need JNLP authentication but you can think of that as a sidecar container.
About your first question: If you use exactly that way, your Jenkins jobs going to run under Jenkins master with the same Docker that your Jenkins Master is using.
Related
New to Kubernetes, a little complex question needs help.
Background
Using Jenkins in GKE (Google Kubernetes Engine)
Want to use jenkins-docker plugin to provide the specific test environment for each type of tests
Don't want to mixin docker binary in the Jenkins image (because it is large)
Don't want docker-in-docker
More specifically, I don't want the Jenkins Pod be a new Docker Server
What I want
Each test environment can create a new pod in GKE Cluster, rather than creating containers inside the Jenkins Pod
P.S.
I have just read some articles, but half of them are telling about "how to use K8S to scale up the Jenkins (using jenkins-slave + jenkins-kubernates plugin)", another half are telling about how to "use docker plugin in a dockerized jenkins container on a bare metal machine (you can use /var/run/docker.sock to communicate between the host and the docker container)", but I cannot find **how to use docker plugin (to provide a specific environment) in a dockerized jenkins container inside K8S
I have a ec2 instance running as my jenkins master. I would like to run a container in that instance that will be used as another build executor so I can tun a few builds simultaneously.
Im having pronblems connecting these.
In the docker hub jenkins docs it says under the relevant section:
You can run builds on the master out of the box.
But if you want to attach build slave servers through JNLP (Java Web
Start): make sure you map the port: -p 50000:50000 - which will be
used when you connect a slave agent.
If you are only using SSH slaves, then you do NOT need to put that
port mapping.
but when I try to add a node in the jenkins config, it asks remote root directory (probably should be /var/jenkins ? ) and a launch method.
I don't quite understand what I should give it as its launch method to make this work and I don't understand where the port number comes into play.
What you need is Jenkins Docker plugin (link below) and follow the instructions listed here
https://wiki.jenkins.io/display/JENKINS/Docker+Plugin
I followed those instructions and was able to setup dynamic slaves in Jenkins which are dynamically provisioned.
We have a Jenkins setup with the Docker plugin installed and want to run our build jobs in Docker Containers based on private images.
Here is what we have:
Jenkins master runs on "bare metal" VM, no containerization
We have a second VM with a Docker Engine running on it, the Docker engine port from this VM is exposed and accessible from the Jenkins master via TCP
We created several Docker Templates (in the global Jenkins settings) and can use them in our build jobs, as long as we use the "never pull" strategy for the images
The problems occurs when we try to pull an image from our private registry (we use Artifactory for this and it is accessible from the Docker Engine, since we can successfully push images from our Docker VM).
Whenever we start a job in Jenkins that uses such an image which should always be pulled from the private registry, we see All nodes of label ‘OUR_IMAGE_LABEL’ are offline and the job is hanging forever.
The strange thing is that we don't see anything related to such a job in the Jenkins log (/var/log/jenkins/jenkins.log on the Jenkins master) nor do we see anything in the Docker logs (/var/log/messages on the VM with the Docker Engine).
Things work perfectly fine, if we switch back to the "Never pull" strategy and have the image locally available on the Docker engine.
Any ideas
why things are not working
how we could get at least some log messages regarding what Jenkins is doing, when it shows All nodes of label ‘OUR_IMAGE_LABEL’ are offline
I saw couple of tutorials on continuous deployment (on docker.com, on codecentric.de, on devopscube.com).
Overall I saw two approaches:
Set two types of jenkins server (master and slave). Master is in a docker container and slave on the host machine.
Jenkins server in docker container. They set up the link to the host and using that link the jenkins can create or recreate docker images.
In the first approach - I do not understand why they set up additional jenkins server residing inside the docker container. Is not it enough just to have jenkins server on host machine alongside with docker container?
The second approach seems to me a bit insecure because process from container is accessing host OS. Does it have any benefits?
Thanks for any useful info.
I wanted to be able to start multiple linked containers on demand, with a restrict where this build run tag like I do with docker plugin for one single container.
I'm currently running Jenkins inside a docker container and configured a slave cloud using docker plugin to provide a single slave container per job, this provisioning is done on demand by the plugin.
But now I have some new requirements, example:
Starting nodejs application container linked to selenium grid container for protractor e2e testing
Starting a container with a nodejs application linked to a redis server in another container.
Currently, docker plugin does not support linked containers so how should I approach those scenarios?
I know how to start multiple linked containers with docker-compose but there are currently no Jenkins plugins for compose.
I was able to get docker-in-docker working, and thought about having a DIND job with using compose in a pre-setup, but I'm finding this a quite inelegant solution.
Is there a plugin-wise solution?
Docker Slaves Plugin new version's side container feature solves that problem now!