I have deployed a pool of self hosted GitHub runners as pods to my kubernetes cluster. Some of our pipelines contain jobs which run container actions. Is it possible to run those jobs in this type of runner?
Docker in Docker is configured in the deployment, and I can build docker images and push them to the container registry.
I note that the GitHub docs state:
If you want to run workflows that use Docker container actions or service containers, you must use a Linux machine and Docker must be installed.
I've struggled to find any definitive answers to this online
Related
I have Azure devops pipeline, building dockerfile on AKS, as AKS is deprecating docker with the latest release, kindly suggest best practice to have a dockerfile build without docker on AKS cluster.
Exploring on Kaniko, buildah to build without docker..
Nothing has changed. You can still use docker build and docker push on your developer or CI system to build and push the Docker image to a repository. The only difference is that using Docker proper as the container backend within your Kubernetes cluster isn't a supported option any more, but this is a low-level administrator-level decision that your application doesn't know or care about.
Unless you were somehow building using the host docker socket within your Kubernetes cluster, this change will not affect you. And if you were mounting the docker socket from the host in a kubernetes cluster, I'd consider that a security concern that you want to fix.
Docker Desktop runs a docker engine as a container on top of containerd, allowing developers to build and run containers in that environment. Similar can be done with DinD build patterns that run the docker engine inside a container, the difference is the underlying container management tooling is containerd instead of a full docker engine, but the containerized docker engine is indifferent to that.
As an alternative to building within the full docker engine, I'd recommend looking at buildkit which is the current default build tool in docker as of 20.10. It uses containerd and they ship a selection of manifests to run builds directly in kubernetes as a standalone builder.
I've just connected my gitlab account to an existing kubernetes cluster. Then I generated a private runner on the cluster. In the pipeline, I wanted to build a docker image, but I always get the error, can't connect to the docker daemon. I realized, probably because privileged is not true, but I can't really find the solution, how can I create the private runner as a privileged one.
edit: I created the runner through the gitlab GUI, https://gitlab.com/ORG/PROJ/-/clusters/CLUSTER_ID?tab=apps and there I clicked the install runner button.
I installed gitlab and docker in Ubuntu. Now I need to install gitlab-runner using docker executor. Is it necessary for the gitlab to be running in docker or is it enough if both runs on the same machine?
GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. So it just needs connectivity to Gilab and its done by registering the runner. Registering runner
Registering a Runner is the process that binds the Runner with a GitLab instance.
If you want to use Docker, GitLab Runner requires a minimum of Docker v1.13.0.
Allows to run:
Multiple jobs concurrently.
Use multiple tokens with multiple server (even per-project).
Limit number of concurrent jobs per-token.
Jobs can be run:
Locally.
Using Docker containers.
Using Docker containers and executing job over SSH.
Using Docker containers with autoscaling on different clouds and virtualization hypervisors.
Connecting to remote SSH server.
The GitLab Runner version should be in sync with the GitLab version, features may be not available or work properly if there’s a version difference.
I have installed CoreOS as my build environment. I installed Jenkins server as a docker container in CoreOS. And I created a free style project on the Jenkins server to build my project. How can I configure the build run on docker containers on the CoreOS?
So the structure is: CoreOS is my physical machine. Jenkins server is running in a docker container in the CoreOS. And I want to launch more docker containers to run my application. How can I achieve this? The hardest part I think is to launch a docker container in CoreOS from Jenkins JOB. I want to start a new docker container ever time for a build.
I'm not familiar with Jenkins, but I would suggest that you take a look at the docker-machine and docker-compose utilities.
You should be able to have Jenkins use one of those to have the host start your build container.
I wanted to be able to start multiple linked containers on demand, with a restrict where this build run tag like I do with docker plugin for one single container.
I'm currently running Jenkins inside a docker container and configured a slave cloud using docker plugin to provide a single slave container per job, this provisioning is done on demand by the plugin.
But now I have some new requirements, example:
Starting nodejs application container linked to selenium grid container for protractor e2e testing
Starting a container with a nodejs application linked to a redis server in another container.
Currently, docker plugin does not support linked containers so how should I approach those scenarios?
I know how to start multiple linked containers with docker-compose but there are currently no Jenkins plugins for compose.
I was able to get docker-in-docker working, and thought about having a DIND job with using compose in a pre-setup, but I'm finding this a quite inelegant solution.
Is there a plugin-wise solution?
Docker Slaves Plugin new version's side container feature solves that problem now!