The new Docker 1.13.0 release introduced docker stack deploy.
docker stack deploy adds support of deploying services from a docker-compose.yml file directly.
Are the command docker service create and docker deploy stack creating the same service, just i two different ways?
Are there any special features added the service by creating it with docker deploy stack?
Im a bit confused, and can't find any information about this subject.
Thanks in advance
A stack is a collection of services. This is similar to how a compose file defines a group of containers to run. So if you only need to run a single service, you can do so with a docker service create. If you need to deploy a collection of services, or use a definition from a compose file as your definition for the service(s), then you would deploy them with docker stack deploy.
Related
I have a Docker Swarm running (spread over three nodes), and pulling images from my private dockerhub repo.
So I can have multiple web applications spread over the same swarm, I'm using Traefik as a proxy.
For each web application, I have a docker-compose.yml and launch it to the stack:
docker stack deploy -c example.yml example
docker stack deploy -c exampletwo.yml exampletwo
The actual *.yml files are all stored in a home folder of one specific swarm manager node.
I don't feel like this is best practice, as I'm going to end up with:
/home/user/example.yml
/home/user/exampletwo.yml
/home/user/examplethree.yml
etc
which seems messy in that they are all together and open to mistakes and loss. And all documenation I read always refers to just using docker-compose.yml; but that doesn't work when I have multiple (totally separate) web applications (I don't want to put all services into one main yml as they are all fundamentally totally unrelated)
Is there a best practise way of doing this? I wonder if I'm missing something fundamental with how to get the yml file available on the node to docker stack deploy
I've researched around before asking here, but all answers lead me to the same conclusion:
Build your Docker Compose stack locally
Tag and push the images to a registry (either a private or public one like Docker Hub)
Push the stack to the swarm using docker stack deploy --compose-file docker-compose.yml stackdemo
From here, the stack picks up and "pulls" the images from the registry and runs the containers
Is there no straightforward way to make the following (I think common sense) scenario work seamlessly?
Docker swarm manager has access (SSH keys) to pull the project from Git.
It periodically pulls the project and builds it "locally" using docker-compose up
When the build succeeds (containers are ready), it pushes the stack to the swarm using docker stack deploy, propagating the images to all worker nodes.
In that way, the original "source code" is only known by the Manager Node and only it has direct access to the Git repository.
Maintaining a registry (or paying for a cloud one), seems like a huge disadvantage for using Docker in Swarm Mode.
Side note: I've tried the approach of deploying a registry as a service within the stack and tagging + pushing the images to 127.0.0.1/myimage but that led to a different set of problems of its own - e.g. the fact that worker nodes that do not have an instance of the Registry container running, have no access to pull the image (the registry needs to be replicated to all nodes).
Use the docker save and docker load commands to transfer images from your dev machine to all of your swarm machines.
Docker swarm is orchestration & used or intended for managing docker node cluster. When any service is deployed, docker engine can start it on any of the node in the cluster (node that satisfy placement constraints if provided). Now, if one dont have registry, & images are available on node locally, docker cant verify if all nodes will point to same version of the image. Hence, swarm pulls image from registry & then deploys it to nodes.
Having registry also helps in keeping copy of images & registry keeps all version even if images are prune on one or all of docker nodes. One can enable backup of registry & hence there's no chance for loss of any image built & pushed to a registry.
Starting a registry (at least on localhost) is very easy - but that's not what your question.
Coming to your question, you can keep the compose stack file in the same directory where you have Dockerfile & then in the stack compose file you can write service which will get build at the time you deploy the stack:
version: "3.9"
services:
web:
image: 127.0.0.1:5000/stackdemo
build: .
ports:
- "8000:8000"
Here,
build: .
this line builds the image with given name tagging to registry on localhost - but is not pushed which needs to be done manually.
So, in Dockerfile you can write git pull or git clone & then run command to build your code & so on.
Here's a link which provides simple steps to start a registry & build image on the fly while deploying the stack:
https://docs.docker.com/engine/swarm/stack-deploy/
Also, swarm does not works without registry & hence it's not possible to just save & load image & use with swarm orchestration.
I have a swarm cluster wherein different technology dockers are deployed. (Zookeeper, Kafka, Elastic, Storm and custom web application)
Web application goes under tremendous changes and have to update the stack everytime web docker changes. Once in a while there will be updates to Elasticsearch image.
When i run docker stack deploy, it goes and restarts all existing docker services which are not even changed. This hampers whole stack and there is a downtime for whole application. Docker stack does not have option of update.
Someone has solution for this?
docker service update --image does the trick.
Check the docker service update docs.
Redeploying the stack with changed configuration(docker-compose.yml file) solves the problem see https://docs.docker.com/engine/reference/commandline/stack_deploy/#extended-description.
There they stated "Create and update a stack from a compose or a dab file on the swarm." Also i dont see any command like 'docker stack update
'. So this can solve the problem.
If you have docker stack created from compose.yml and you need to re-deploy only one service from stack, just do this:
docker service rm <your-service>
and then:
docker stack deploy -c compose.yml <stack-name>
And you just will update your stack, not recreate all services.
I have a doubt in using docker swarm mode commands to update existing services after having deployed a set of services using docker stack deploy.
As far I understood every service is pinned to the SHA256 digest of the image at the time of creation, so if you rebuild and push an image (with same tag) and you try to run a docker service update, service image is not updated (even if SHA256 is different). On the contrary, if you run a docker stack deploy again, all the services are updated with the new images.
I managed to update the service image also by using docker service update --image repository/image:tag <service>. Is this the normal behavior of these commands or is there something I didn't understood?
I'm using Docker 17.03.1-ce
Docker stack deploy documentation says:
"Create and update a stack from a compose or a dab file on the swarm. This command has to be run targeting a manager node."
So the behaviour you described is as expected.
Docker service update documentation is not so clear but you yourself said it only runs with --image repository/image:tag <service> so the flag is necessary to update the image.
You have two ways to accomplish what you want.
It is normal and expected behavior for docker stack deploy to update images of existing services to whatever hash the specified tag is linked.
If no tag is present, latest is assumed - which can be problematic at times, since the latest tag is not well understood by most persons, and thus lead to some unexpected results.
I'm learning about using Docker Compose to deploy applications in multiple containers, across multiple hosts. And I have come across two configuration files - stack file, and Compose file.
From the Cloud stack file YAML reference, it states a stack file is a file in YAML format that defines one or more services, similar to a docker-compose.yml file but with a few extensions.
And from this post, it states that stacks are very similar to docker-compose except they define services while docker-compose defines containers.
They look very similar, so I am wondering when I would use the stack file, and when to use the Compose file?
Conceptually, both files serve the same purpose - deployment and configuration of your containers on docker engines.
Docker-compose tool was created first and its purpose is "for defining and running multi-container Docker applications" on a single docker engine. (see docker compose overview )
You use docker-compose up to create/update your containers, networks, volumes and so on.
Where Docker Stack is used in Docker Swarm (Docker's orchestration and scheduling tool) and, therefore, it has additional configuration parameters (i.e. replicas, deploy, roles) that are not needed on a single docker engine.
The stack file is interpreted by docker stack command. This command can be invoked from a docker swarm manager only.
You can convert docker-compose.yml to docker-cloud.yml and back. However, as stated in your question, you must pay attention to the differences. Also, you need to keep in mind that there're different versions for docker-compose. Presently, the latest version is version 3. (https://docs.docker.com/compose/compose-file/)
Edit: An interesting blog, that might help to understand the differences, can be found here https://blog.nimbleci.com/2016/09/14/docker-stacks-and-why-we-need-them/
Note: The question guesses that the Docker Cloud reference is the go-to for understanding stack, and it is useful, but that isn't the authoritative source on stack vs compose -- instead that is a guide that is specific to Docker's hosted service: "Docker Cloud provides a hosted registry service with build and testing facilities." For file documentation, see the Compose file version 3 format -- although it is named "Compose", this is the authoritative place for which features work with both compose and swarm/stack, and how.
You can specify a group of Docker containers to configure and deploy in two ways:
Docker compose (docker-compose up)
Docker swarm (docker swarm init; docker stack deploy --compose-file docker-stack.yml mystack)
Both take a YAML file written in the Docker Compose file version 3 format. That reference is the primary source documenting both docker-compose and docker swarm/stack configuration.
However, there are specific differences between what you can do in the two yml files -- specific options, and specific naming conventions:
Options
The available service configuration options are documented on the Compose file reference page -- usually with a note at the bottom of an option entry describing it as ignored either by docker stack deploy or by docker-compose up.
For example, the following options are ignored when deploying a stack in swarm mode with a (version 3) Compose file:
build, cap_add, cap_drop, cgroup_parent, container_name, depends_on, devices, external_links, links, network_mode, restart, security_opt, stop_signal, sysctls, tmpfs (version 3-3.5), userns_mode
...while some options are ignored by docker-compose, yet work with docker stack deploy, such as:
deploy, restart_policy
When run from the command line, docker stack deploy will print warnings about which options it is ignoring:
Ignoring unsupported options: links
File naming
For docker-compose up the default file name is docker-compose.yml if no alternate file name is specified using -f (see the compose reference). It is common to use this default name and run the command without an argument.
For docker stack deploy there is no default file given in the docker stack deploy reference. You can use whatever name you want, however here are three conventions:
use docker-stack.yml, as used in the official Docker for Beginners Ch.3: Deploying an app to a Swarm.
use docker-cloud.yml, as used in the Docker Cloud Stack YML reference for the Docker Cloud service.
use docker-compose.yml -- the old default name for the Compose file format.