New to Kubernetes, a little complex question needs help.
Background
Using Jenkins in GKE (Google Kubernetes Engine)
Want to use jenkins-docker plugin to provide the specific test environment for each type of tests
Don't want to mixin docker binary in the Jenkins image (because it is large)
Don't want docker-in-docker
More specifically, I don't want the Jenkins Pod be a new Docker Server
What I want
Each test environment can create a new pod in GKE Cluster, rather than creating containers inside the Jenkins Pod
P.S.
I have just read some articles, but half of them are telling about "how to use K8S to scale up the Jenkins (using jenkins-slave + jenkins-kubernates plugin)", another half are telling about how to "use docker plugin in a dockerized jenkins container on a bare metal machine (you can use /var/run/docker.sock to communicate between the host and the docker container)", but I cannot find **how to use docker plugin (to provide a specific environment) in a dockerized jenkins container inside K8S
Related
I want to run my Jenkins behind k8s. We can achieve that with any standard helm chart or our own manifest files. In this case, Jenkins (master only) will run inside a container (Pod).
Now I also want to have a pipeline job that uses docker agent as described here
I am getting confused, about
how and where this docker container will be run (on the same node where Jenkins is running? and suppose the node capacity is over then it needs to run docker agent on a different node)
how does Jenkins will authenticate to run containers on k8s nodes?
I saw the Kubernetes plugin/docker plugin. But those plugins create containers beforehand (or at least we need to set up a template, which decides how containers will start, which image will be used and many more) and connects Jenkins with help of JNLP / ssh. I lose the flexibility to have an image as an agent in that case.
going further, I also like to build custom images on the fly with help of Dockerfile shipped along with code. An example is available in the same link.
I believe this documentation is answering all of your questions: https://devopscube.com/jenkins-build-agents-kubernetes/
With this method, you are not losing your flexibility because your Jenkins master going to create a K8s pod on the fly. Yes, additionally you need JNLP authentication but you can think of that as a sidecar container.
About your first question: If you use exactly that way, your Jenkins jobs going to run under Jenkins master with the same Docker that your Jenkins Master is using.
With the Kubernetes orchestrator now available in the stable version of Docker Desktop for Win/Mac, I've been playing around with running an existing compose stack on Kubernetes locally.
This works fine, e.g., docker stack deploy -c .\docker-compose.yml myapp.
Now I want to go to the next step of running this same application in a production environment using the likes of Amazon EKS or Azure AKS. These services expect proper Kubernetes YAML files.
My question(s) is what's the best way to get these files, or more specifically:
Presumably, docker stack is performing some conversion from Compose YAML to Kubernetes YAML 'under the hood'. Is there documentation/source code links as to what is going on here and can that converted YAML be exported?
Or should I just be using Kompose?
It seems that running the above docker stack deploy command against a remote context (e.g., AKS/EKS) is not possible and that one must do a kubectl deploy. Can anyone confirm?
docker stack deploy with a Compose file to Kube only works on Docker's Kubernetes distributions - Docker Desktop and Docker Enterprise.
With the recent federation announcement you'll be able to manage AKS and EKS with Docker Enterprise, but using them direct means you'll have to use Kubernetes manifest files and kubectl.
According to the DataDog Docker Integration Docs:
There are two ways to run the [DataDog] Agent: directly on each host, or within a docker-dd-agent container. We recommend the latter.
Why is a Docker-based agent installation preferred over just installing the DataDog agent directly as a service on the box that's running the Docker containers?
One of Dockers main features is portability and it makes sense to bind datadog into that environment. That way they are packaged and deployed together and you don't have the overhead of installing datadog manually everywhere you choose to deploy.
What they are also implying is that you should use docker-compose and turn your application / docker container into an multi-container Docker application, running your image(s) alongside the docker agent. Thus you will not need to write/build/run/manage a container via Dockerfile, but rather add the agent image to your docker-compose.yml along with its configuration. Starting your multi-container application will still be easy via:
docker-compose up
Its really convenient and gives you additional features like their autodiscovery service.
I run Rstudio server mostly from an EC2 instance. However, I´d also like to run it from a cluster at work. They tell me that I can setup docker with rstudio and make it run. Now, I´d also like the Rstudios both on EC2 and work to have the same packages and the same versions available. Any idea how I can do this? Can I have both version point to a dropbox folder? In that case, how can I mount a dropbox folder?
You should setup a docker repository on dockerhub or aws ec2 container service (ecs). ECS is a managed service that allows you to easily deploy docker containers onto a cluster of 1 or more ec2 instances that are running the ecs agent (an aws program that helps that cluster work with the ecs). The Dockerfile should install all packages that you need at build time of the image. I suggest referencing the aws ecs documentation, which includes a walkthrough to get you going very quickly (assuming you have an idea of how docker works): https://aws.amazon.com/documentation/ecs/
You should then always run from that docker image, whether you are running on a local or remote machine. One key advantage of docker is that it keeps your application's environments the same (assuming you use the same build of the image) regardless of the host environment.
I am not sure why would not always run on ECS (we have multiple analysts using RStudio, and ECS lets us provision cpu/memory resources to each one, as well as autoscale as needed). You could install docker on EC2 and manage it that way, but probably easier to just install the ecs agent (or use the ecs optimized ec2 ami which has it preinstalled - the docs above walk through configuring it), and use ECS to launch rstudio services.
I wanted to be able to start multiple linked containers on demand, with a restrict where this build run tag like I do with docker plugin for one single container.
I'm currently running Jenkins inside a docker container and configured a slave cloud using docker plugin to provide a single slave container per job, this provisioning is done on demand by the plugin.
But now I have some new requirements, example:
Starting nodejs application container linked to selenium grid container for protractor e2e testing
Starting a container with a nodejs application linked to a redis server in another container.
Currently, docker plugin does not support linked containers so how should I approach those scenarios?
I know how to start multiple linked containers with docker-compose but there are currently no Jenkins plugins for compose.
I was able to get docker-in-docker working, and thought about having a DIND job with using compose in a pre-setup, but I'm finding this a quite inelegant solution.
Is there a plugin-wise solution?
Docker Slaves Plugin new version's side container feature solves that problem now!