Question 1: I am new to docker swarm, I created a docker swarm cluster on my local machine and SSH in to it. To my surprise docker-compose was NOT installed inside the manager node. Is that normal ? Is there any workaround to get the docker compose up and running on swarm manager node ?
Question 2: how do I manage to get all my code inside manager node. Let’s say I have my source code on a director. If I want to move that inside my docker swarm manager node. How can I do that ?
It is common for docker-compose to not be installed on servers compared to docker-desktop-clients which come bundled with docker-compose and other tools.
You have to install it to use it on your local machine. https://docs.docker.com/compose/install/
Although you can use your installation of docker-compose to work against the docker-daemon on your local machine by setting DOCKER_HOST
https://docs.docker.com/engine/reference/commandline/cli/#environment-variables
You can copy your source-code onto your local-machine via scp https://linuxize.com/post/how-to-use-scp-command-to-securely-transfer-files/
But you would rather build images and deploy onto your local-machine.
Related
I am fairly new to docker and kubernetes and going through kubernetes docs it says,
When using a single VM for Kubernetes, it’s useful to reuse Minikube’s built-in Docker daemon. Reusing the built-in daemon means you don’t have to build a Docker registry on your host machine and push the image into it. Instead, you can build inside the same Docker daemon as Minikube, which speeds up local experiments.
So, my understanding is there are two instances are running in my local machine, one in macOS and the other one in the VM.
Suppose I created a image using the docker instance on my macOS, and then I want to use it on Kubernetes then,
Question 1: Do I strictly need to create a local registry and then pull it from within Kubernetes cluster ?
It further says,
To work with the Docker daemon on your Mac/Linux host, use the docker-env command in your shell: eval $(minikube docker-env)
Running this creates a few environment variables in the current shell.
Question 2: Will this be able to pull images that I build from within the docker in my macOS without creating the local registry.
Minikube runs a single-node Kubernetes cluster inside a Virtual Machine (VM) on your laptop.
So it will create a k8s setup on a VM running on your macOS.
eval $(minikube docker-env)
this command on you macOS will help you to switch context to docker, so that you can run docker commands from your macOS.
Question 1: Do I strictly need to create a local registry and then pull it from within Kubernetes cluster ?
No you don't need to explicitly create a local registry as everthing runs on a single VM in minikube.
Question 2: Will this be able to pull images that I build from within the docker in my macOS without creating the local registry?
By switching docker env context on your host machine you can able to pull the images you don't need create a registry for that. Remember your macOS is not the part of you k8s cluster. Your k8s cluster is running on the single VM created by minikube.
I am running buildbot which is a CI tool on an EC2 machine. It's currently running as docker containers one for buildbot master and one for buildbot worker. Inside buildbot worker, I have to again run docker for building images and running containers.
After doing some research on how to best do this, I have mounted the docker sock file from the host machine to the buildbot worker container. Now from inside the buildbot worker, I am able to connect to the host docker daemon and use the build cache.
Main problem now is that inside the buildbot worker, I have a docker compose file in which for one service, I am mounting a file like this
./configs/my.cnf:/etc/my.cnf
but it is failing. And doing some more research, it's because the configs/my.cnf is relative to the buildbot worker directory and since I am using the host docker daemon which resolves the files using the host paths, it is not able to find the file.
I am not able to figure out on how to best do this. There were some suggestions on using the data volumes for this, but I am not sure on how best to use those.
Any idea on how we can do this?
Do you have any control over the creation of the buildbot worker? Can you control the buildbot worker directory.
export BUILD_BOT_DIR=$(mktemp -d) &&
docker container create -v /var/run/docker.sock:/var/run/docker.sock -v ${BUILD_BOT_DIR}:${BUILD_BOT_DIR} -e BUILD_BOT_DIR ...
In this scenario, the path './configs/my:conf' points to the same file on both the container and the host.
Lets say I want to edit a config file for an NGINX Docker service that is replicated across 3 nodes.
Currently I list the services using docker service ls.
Then get the details to find a node running a container for that service using docker serivce ps servicename.
Then ssh to a node where one of the containers is running.
Finally, docker exec -it containername bash. Then I edit the config file.
Two questions:
Is there a better way to do this rather than ssh to a node running a container? Maybe there is a swarm or service command to do so?
If I were to edit that config file on one container would that change be replicated to the other 2 containers in the swarm?
The purpose of this exercise would be to edit configuration without shutting down a service.
You should not be exec'ing into containers to change their configuration, and so docker has not created an easy way to do this within Swarm Mode. You could use classic swarm to avoid the need to ssh into the other host, but I still don't recommend this.
The correct way to do this is to migrate your configuration file into a docker config entry. Version your config name. Then when you want to update it, you create a new version with the desired changes, and do a rolling update of your service to use that new configuration.
Unless the config is mounted from an external source like NFS, changes to one config in one container will not apply to other containers running on other nodes. If that config is stored locally inside your container as part of it's internal copy-on-write filesystem, then no changes from one container will be visible in any other container.
After looking through docker official swarm explanations, github issues and stackoverflow answers im still at a loss on why i am having the problem that i have.
Issue at hand: docker-compose up starts services not in the swarm even though swarm is active and has 2 nodes.
Im using 1.12.1 docker version.
Looking at swarm tutorial i was able to start and scale my swarm using docker service create without any issues.
running docker-compose up with version 2 docker-compose.yml results in services starting outside of swarm, i can see them through docker ps but not docker service ls
I can see that docker-machine as the tool that solves this problems, but then again it needs virtual box to be installed.
so my questions would be
Can i use docker-compose with docker-swarm (NOT docker-engine) without docker-machine and without experimental build bundle functionality?
If docker service create can start a service on any nodes is it an indication that network configuration of the swarm is correct ?
What is the advantages/disadvantages of docker-machine versus experimental build functionality
1) No. Docker Compose isn't integrated with the new Swarm Mode yet. Issue 3656 in GitHub is tracking that. If you start containers on a swarm with Docker Compose at the moment, it uses docker run to start containers, which is why you see them all on one node.
2) Yes. Actually you can use docker node ls on the manager to confirm all the nodes are up and active, and docker node inspect to check a particular node, you don't need to create a service to validate the swarm.
3) Docker Machine is also behind the 1.12 release, so if you start a swarm with Docker Machine it will be the 'old' type of swarm. The old Docker Swarm product needed a whole lot of extra setup for a key-value store, TLS etc. which Swarm Mode does for free.
1) You can't start services using docker-compose on the new Docker "Swarm Mode". There's a feature to convert a docker-compose file to the new dab format which is understood by the new swarm mode but that's incomplete and experimental at this point. You basically need to use bash scripts to start services at the moment.
2) The nodes in a swarm (swarm mode) interact using their own overlay network. It's the one named ingress when you do docker network ls. You need to setup your own overlay network to run services in. eg:
docker network create -d overlay mynet
docker service create --name serv1 --network mynet nginx
3) I'm not sure what feature you mean by "experimental build'. docker-machine is just a way to create hosts (the nodes). It facilitates the setting up of the docker daemon on each host, the certificates and allows some basic maintenance (renewing the certs, stopping/starting a host if you're the one who created it). It doesn't create services, volumes, networks or manages them. That's the job of the docker api.
Does anyone know how (if possible) to run docker-compose commands against a swarm using the new docker 1.12 'swarm mode' swarm?
I know with the previous 'Docker Swarm' you could run docker-compose commands directly against the swarm by updating the DOCKER_HOST to point to the swarm master :
export DOCKER_HOST="tcp://123.123.123.123:3375"
and then simply execute commands as if you were running them against a single instance of Docker engine.
OR is this functionality something that docker-compose bundle is replacing?
I realized my question was vaguely worded and actually has two parts to it. Eventually however, I was able to figure out solutions to both issues.
1) Can you run commands directly 'against' a swarm / swarm-mode in Docker 1.12 running on a remote machine?
While you can't really run commands 'against' a swarm you CAN run docker service commands on the master node of a swarm in order to run services on that swarm.
You can also configure the Docker daemon (the docker daemon that is the master node of the swarm) to listen on TCP ports in order to externally expose the Docker API.
2) Can you still use docker-compose files to start services in Docker 1.12 swarm-mode?
Yes, although these features are currently part of Docker's "experimental" features. This means you must download/install the version that includes the experimental features (check the github).
You essentially follow these instructions https://github.com/docker/docker/blob/master/experimental/docker-stacks-and-bundles.md
to go from the docker-compose.yml file to a distributed application bundle and then to an application stack (this is when your services are actually run).
$ docker-compose bundle
$ docker deploy [OPTIONS] STACK
Here's what I did:
On my remote swarm manager node I started docker with the following options:
docker daemon -D -H unix:///var/run/docker.sock -H tcp://0.0.0.0:2375 &
This configures Docker daemon to listen on the standard docker socket unix:///var/run/docker.sock AND on localhost:2375.
WARNING : I'm not enabling TLS here just for simplicity
On my local machine I update the docker host environment variable to point at my swarm master node.
$ export DOCKER_HOST="tcp://XX.XX.XX.XX:2377" (populate with your IP)
Navigate to the directory of my docker-compose.yml file
Create a bundle file from my docker-compose.yml file. Make sure to include the .dab extension.
docker-compose bundle --fetch-digests -o myNewBundleFile.dab
Create an application stack from the bundle file. Do not specify the .dab extension here.
$ docker deploy myNewBundleFile
Now I'm still experiencing some networking related issues but I have successfully gotten my service up and running from my unmodified docker-compose.yml files. The network issues I'm experiencing is documented here : https://github.com/docker/docker/issues/23901
While the official support for Swarm mode in Docker Compose is still in progress, I've created a simple script that takes docker-compose.yml file and runs docker service commands for you. See https://github.com/ddrozdov/docker-compose-swarm-mode for details.
It is not possible. Compose uses containers to create a client-side concept of a service. Docker 1.12 Swarm mode introduces a new server-side concept of a service.
You are correct that docker-compose bundle; docker stack deploy is the way to get a Compose file running in Swarm Mode.