What is the official / recommended way to install Docker on GitHub Actions?
In particular I need to run a custom script on GitHub Actions that, among other tasks, calls docker build and docker push.
I don't want pre-made actions that build/push, I want to have access to the docker command.
What should I do?
The only official action that I can find uses Docker Buildx and not the normal docker: https://github.com/marketplace/actions/build-and-push-docker-images
Beside that I can find this action (https://github.com/marketplace/actions/setup-docker) but I don't know if I can trust that source, since it's not official.
Any recommendations? How do you install the docker command on GitHub Actions?
Related
Github recommending running their runner as a non-root user gives rise to some issues surrounding mixing docker and non-docker actions. This is quite annoying because it results in the checkout action not being able to run because it can't access the files created by actions run in docker containers.
Can this be solved by running the actions runner with rootless docker?
This problem can be solved by running the github actions runner as root, which somewhat reduces security.
A better solution is using rootless docker:
Remove docker from your system if you have previously installed it through Ubuntu's default repositories.
install docker from Docker's repositories as directed here (I also recommend
enabling cgroupsV2, as described here) & reboot. This will give you the script in /usr/bin needed to setup rootless docker in the next step.
setup rootless docker as described here.
don't forget to run the following, so docker remains running after you logout (as described in the guide)
systemctl --user enable docker
systemctl --user start docker
sudo loginctl enable-linger $(whoami)
Also make sure to create the rootless context as described on that same page. This will make your own docker commands and the github actions runner automatically use rootless docker.
install the self hosted runner: https://docs.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners (skip if already installed)
Add the DOCKER_HOST env var to the .env file in the runner directory. The file might already be created by default. The line you add should look as follows (change the 1000 if your UID is not 1000):
DOCKER_HOST=unix:///run/user/1000/docker.sock
re(start) the actions runner. This can by done by restarting its systemd service. Your runner should now work with rootless docker
If you're having issues with the new docker build github action using buildx, also see How to solve error with rootless docker in github actions self hosted runner: write /proc/sys/net/ipv4/ping_group_range: invalid argument: unknown
Why?
I'm trying to create a general purpose solution for running docker-compose on Heroku. I want to make a one click deployment solution through the use of Heroku Button deployment. This way, a user does not need any knowledge of git, Heroku cli and docker.
The problem.
Docker and the docker daemon are only available when I set the stack to container. There are buildpacks that give you docker and docker-compose CLI but without the docker daemon you cannot run the docker image. So buildpacks won't work.
With the stack set to container I can use the file heroku.yml (article). In there I define my processes. (It replaces Procfile. If I still add a Procfile to my project it will do nothing.)
I can also define a Dockerfile there to build my docker image.
When I however run the docker image the following error pops up:
2019-02-28T15:32:48.462101+00:00 app[worker.1]: Couldn't connect to Docker daemon at http+docker://localhost - is it running?
2019-02-28T15:32:48.462119+00:00 app[worker.1]:
2019-02-28T15:32:48.462122+00:00 app[worker.1]: If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
The problem is inside the Docker container the Docker daemon is not running. The solution to this is to mount it:
docker run -v /var/run/docker.sock:/var/run/docker.sock ...
And since you cannot use Procfile I cannot run that command. (See above heroku.yml replaces Procfile.) And if I was using a buildpack I could use Procfile but the docker daemon wouldn't be running.....
I tried defining a VOLUME within the Dockerfile and the problem persists. Furthermore a heroku article says "Volume mounting is not supported. The filesystem of the dyno is ephemeral."
On Heroku it is possible to run a docker image. What I am struggling at is running a docker in docker image.
Running a docker in docker image works fine on my VPS by mounting /var/run/docker.sock but this cannot(?) be done on Heroku.
Last words:
I'm trying to make this work so that other people can easily deploy software solution even though they are not comfortable with git, heroku cli and docker.
Unfortunately the answer to your question is: not yet.
For securiy reasons Heroku does not provide to the users the ability to run priviledged containers because the container could access to host capabilities.
The documentation is pretty clear about your limitations, e.g: No --priviledged container and no root user either, no VOLUMES and disk is ephemeral.
After playing with DinD images for your concern, I came to the conclusion that trying to run Docker inside a Heroku container is not the right choice and design.
I am pretty sure what you are trying to achieve is close to what Heroku is offering to the users. Offering a platform or an application where non-developper can push and deploy applications with just a button can be very interesting in various ways. And it can be done with an application using their Platform API.
In this situation a Web application (running on Heroku) may not (up to my knowledge) be able to do what you want. Instead you need to embed in a Desktop application: git, docker, and your app for parsing, verifying, building and pushing your applications/components to Heroku's container registry.
In the end, if you still think what you need a DinD solution, well, your primary solution to use a VPS is the only solution for the moment. But be aware that it may open security vulnerabilities to your system and you may arrive to offer something similar to Heroku's offer when trying to limit those security doors.
I don't think you can run a service on Heroku that can use the docker command to start some docker container.
I want to make a one click deployment solution through the use of Heroku Button deployment.
I think you can update the reference of the Deploy button to some of your automation servers (ex: an instance Jenkins already deploy on Heroku/another cloud) to trigger the deploy pipeline and don't let people interact with git/docker, etc.
But yes, you have to deal with a lot of problems like security, parameter. when not using popular solutions like Jenkins/CircleCI login and then deploy...
What I did was install it in my dockerfile like this:
RUN curl -fsSLO https://get.docker.com/builds/Linux/x86_64/docker-17.04.0-ce.tgz \
&& tar xzvf docker-17.04.0-ce.tgz \
&& mv docker/docker /usr/local/bin \
&& rm -r docker docker-17.04.0-ce.tgz
Then in the args section for running the docker I added this:
args '--user root -v /var/run/docker.sock:/var/run/docker.sock'
For further explanation on why this works see: stackoverflow.com/q/27879713/354577 This works well for me though.
I use following module:
https://docs.ansible.com/ansible/latest/modules/docker_service_module.html?highlight=ansible%20doc
I can create and start docker container using this module. However, is it possible to execute tasks (and preserve changes) on this container?
I mean:
install some yum package
insert some bash script into container.
Could you give me some clues?
As a general rule, you don't install software on a running container. If you need a container with some software installed in it, you should build a custom image that has the software you need, and set it up so that it can do everything it needs on its own once you start it up once. (As an even broader rule, you shouldn't need to docker exec into a running container except to debug things; it definitely isn't part of the core "how to do things with containers" workflow.)
I would recommend following a standard Docker tutorial, such as Docker's official tutorial on building and running custom images. Once you have a working Docker image workflow, you'd use the Ansible docker_container module in place of the docker run command.
Can images in docker be installed by source code. What I mean is I want to build my environment with several components and their dependencies. I want to build the components by executing the source code. Does docker allow me to do something like that ?
Sounds like you want a dynamic docker build process. For this you need docker 1.9 up , use --build-args to pass argument variables . You can build multiple images from a single docker file passing in different argument values each time.
Obviously suffers the reproducibility issue discussed.
Yes, it allows you to do that. You need to start with a base image. For example Ubuntu:
docker pull ubuntu
docker run -t -i ubuntu /bin/bash
After that you will have a bash running inside of your container. Then you can apt-get stuff, run code, change configurations, clone repos and whatever else you want. After that to convert your container into an image you need to commit the container.
Be aware that this is not the Docker way of building infrastructure. The correct way is to create a recipe for building you images by using other base images and standard Docker instructions. This will allow your infrastructure to be stateless, faster to build and will provide more reproducibility.
I forked whilp/ssh-agent and created a feature enhancement and submitted a pull request.
I want to reference/use my branch until it is accepted. On my CI agents, and I don't want to go locally to each one to build a local image.
github.com/rosskevin/ssh-agent branch: feature-known-hosts is what I'd like to use with the run command, is this possible? I can't find references to using github (not to mention a branch) with run, only build.
i.e.
docker run -d --name=ssh-agent whilp/ssh-agent \
github.com/rosskevin/ssh-agent -b feature-known-hosts
Any other advice on docker project patches/workflow/best practices? This is really easy with Bundler, looking for an analog here.
You can't run a docker image directly from GitHub, because GitHub is made to store only the code itself.
When you run the following command:
docker run -d --name=ssh-agent whilp/ssh-agent
Docker is looking for whilp/ssh-agent on Docker Hub, and not on GitHub.
Docker Hub is the equivalent of GitHub for Docker images.
To use your pull request the same way you are using whilp/ssh-agent, you need to create an account on Docker Hub, and create an automated build based on your ssh-agent fork (tutorial here).
Finally, you will be able to use your version with:
docker run -d --name=ssh-agent <username>/ssh-agent