I am running buildbot which is a CI tool on an EC2 machine. It's currently running as docker containers one for buildbot master and one for buildbot worker. Inside buildbot worker, I have to again run docker for building images and running containers.
After doing some research on how to best do this, I have mounted the docker sock file from the host machine to the buildbot worker container. Now from inside the buildbot worker, I am able to connect to the host docker daemon and use the build cache.
Main problem now is that inside the buildbot worker, I have a docker compose file in which for one service, I am mounting a file like this
./configs/my.cnf:/etc/my.cnf
but it is failing. And doing some more research, it's because the configs/my.cnf is relative to the buildbot worker directory and since I am using the host docker daemon which resolves the files using the host paths, it is not able to find the file.
I am not able to figure out on how to best do this. There were some suggestions on using the data volumes for this, but I am not sure on how best to use those.
Any idea on how we can do this?
Do you have any control over the creation of the buildbot worker? Can you control the buildbot worker directory.
export BUILD_BOT_DIR=$(mktemp -d) &&
docker container create -v /var/run/docker.sock:/var/run/docker.sock -v ${BUILD_BOT_DIR}:${BUILD_BOT_DIR} -e BUILD_BOT_DIR ...
In this scenario, the path './configs/my:conf' points to the same file on both the container and the host.
Related
Question 1: I am new to docker swarm, I created a docker swarm cluster on my local machine and SSH in to it. To my surprise docker-compose was NOT installed inside the manager node. Is that normal ? Is there any workaround to get the docker compose up and running on swarm manager node ?
Question 2: how do I manage to get all my code inside manager node. Let’s say I have my source code on a director. If I want to move that inside my docker swarm manager node. How can I do that ?
It is common for docker-compose to not be installed on servers compared to docker-desktop-clients which come bundled with docker-compose and other tools.
You have to install it to use it on your local machine. https://docs.docker.com/compose/install/
Although you can use your installation of docker-compose to work against the docker-daemon on your local machine by setting DOCKER_HOST
https://docs.docker.com/engine/reference/commandline/cli/#environment-variables
You can copy your source-code onto your local-machine via scp https://linuxize.com/post/how-to-use-scp-command-to-securely-transfer-files/
But you would rather build images and deploy onto your local-machine.
Is it possible to use a cloud-init configuration file to define commands to be executed when a docker container is started?
I'd like to test the provisioning of an Ubuntu virtual machine using a docker container.
My idea is to provide the same cloud-init config file to an Ubuntu docker container.
No. If you want to test a VM setup, you need to use actual virtualization technology. The VM and Docker runtime environments are extremely different and you can't just substitute one technology for the other. A normal Linux VM startup will run a raft of daemons and startup scripts – systemd, crond, sshd, ifconfig, cloud-init, ... – but a Docker container will start none of these and will only run the single process in the container.
If your cloud-init script is ultimately running a docker run command, you can provide an alternate command to that container the same way you could docker run on your development system. But a Docker container won't look to places like the EC2 metadata service to find its own configuration usually, and it'd be unusual for a container to run cloud-init at all.
Lets say I want to edit a config file for an NGINX Docker service that is replicated across 3 nodes.
Currently I list the services using docker service ls.
Then get the details to find a node running a container for that service using docker serivce ps servicename.
Then ssh to a node where one of the containers is running.
Finally, docker exec -it containername bash. Then I edit the config file.
Two questions:
Is there a better way to do this rather than ssh to a node running a container? Maybe there is a swarm or service command to do so?
If I were to edit that config file on one container would that change be replicated to the other 2 containers in the swarm?
The purpose of this exercise would be to edit configuration without shutting down a service.
You should not be exec'ing into containers to change their configuration, and so docker has not created an easy way to do this within Swarm Mode. You could use classic swarm to avoid the need to ssh into the other host, but I still don't recommend this.
The correct way to do this is to migrate your configuration file into a docker config entry. Version your config name. Then when you want to update it, you create a new version with the desired changes, and do a rolling update of your service to use that new configuration.
Unless the config is mounted from an external source like NFS, changes to one config in one container will not apply to other containers running on other nodes. If that config is stored locally inside your container as part of it's internal copy-on-write filesystem, then no changes from one container will be visible in any other container.
Does anyone know how (if possible) to run docker-compose commands against a swarm using the new docker 1.12 'swarm mode' swarm?
I know with the previous 'Docker Swarm' you could run docker-compose commands directly against the swarm by updating the DOCKER_HOST to point to the swarm master :
export DOCKER_HOST="tcp://123.123.123.123:3375"
and then simply execute commands as if you were running them against a single instance of Docker engine.
OR is this functionality something that docker-compose bundle is replacing?
I realized my question was vaguely worded and actually has two parts to it. Eventually however, I was able to figure out solutions to both issues.
1) Can you run commands directly 'against' a swarm / swarm-mode in Docker 1.12 running on a remote machine?
While you can't really run commands 'against' a swarm you CAN run docker service commands on the master node of a swarm in order to run services on that swarm.
You can also configure the Docker daemon (the docker daemon that is the master node of the swarm) to listen on TCP ports in order to externally expose the Docker API.
2) Can you still use docker-compose files to start services in Docker 1.12 swarm-mode?
Yes, although these features are currently part of Docker's "experimental" features. This means you must download/install the version that includes the experimental features (check the github).
You essentially follow these instructions https://github.com/docker/docker/blob/master/experimental/docker-stacks-and-bundles.md
to go from the docker-compose.yml file to a distributed application bundle and then to an application stack (this is when your services are actually run).
$ docker-compose bundle
$ docker deploy [OPTIONS] STACK
Here's what I did:
On my remote swarm manager node I started docker with the following options:
docker daemon -D -H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375 &
This configures Docker daemon to listen on the standard docker socket unix:///var/run/docker.sock AND on localhost:2375.
WARNING : I'm not enabling TLS here just for simplicity
On my local machine I update the docker host environment variable to point at my swarm master node.
$ export DOCKER_HOST="tcp://XX.XX.XX.XX:2377" (populate with your IP)
Navigate to the directory of my docker-compose.yml file
Create a bundle file from my docker-compose.yml file. Make sure to include the .dab extension.
docker-compose bundle --fetch-digests -o myNewBundleFile.dab
Create an application stack from the bundle file. Do not specify the .dab extension here.
$ docker deploy myNewBundleFile
Now I'm still experiencing some networking related issues but I have successfully gotten my service up and running from my unmodified docker-compose.yml files. The network issues I'm experiencing is documented here : https://github.com/docker/docker/issues/23901
While the official support for Swarm mode in Docker Compose is still in progress, I've created a simple script that takes docker-compose.yml file and runs docker service commands for you. See https://github.com/ddrozdov/docker-compose-swarm-mode for details.
It is not possible. Compose uses containers to create a client-side concept of a service. Docker 1.12 Swarm mode introduces a new server-side concept of a service.
You are correct that docker-compose bundle; docker stack deploy is the way to get a Compose file running in Swarm Mode.
In my dev machine I have app container with mounted code directory f.e -v /host/code:/app/code
What is the best practice of deploying such containers to production?
How should I pack this mount binding inside container in a way that in prod I would only execute "docker load"... and get everything work.
The best practice is to use a data volume container (a container which is only docker create'd, not docker run, because it does not run any process)
See "Creating and mounting a data volume container"
That way, you can easily export and deploy that container alongside the other instead of relying on a local host path which might not be available once on the production host.