I was wondering if anyone could differentiate between these two.Both of them have similar naming.
Docker Swarm is a separate product which you can use to cluster multiple Docker hosts. Prior to Docker version 1.12 it was the only native Docker option for clustering hosts, and it needed a lot of additional setup for distributed state, service discovery and security.
With Docker 1.12, Swarm Mode is built into Docker Engine. To run a cluster you just need to install Docker on multiple machines, run docker swarm init to switch to Swarm Mode and docker swarm join to add more nodes to the cluster. State, discovery and security are all included with zero setup.
Swarm Mode is optional, but if you want to run several Docker hosts it's the preferred way. You get reliability, load-balancing, scaling, and rolling service upgrades in 1.12, and it's likely that the bulk of new features will go into Swarm Mode. The original Docker Swarm product will probably only have maintenance updates in the future (although Swarm is open source, just like Docker Engine).
Docker Swarm (also Swarm classic) is fundamentally different from Swarm Mode. Native Swarm functionality will continue to be supported in Docker 1.12 release, this is done to preserve backward compatibility.
Docker Swarm (classic):
Separate from Docker Engine and can run as Container
Needs external KV store like Consul, etcd, Zookeeper
Usage example:
docker run swarm manage <consul-ip>
docker -H <worker-ip> run swarm join --advertise=<worker-ip> <consul-ip>
Swarm Mode (new, preferable):
Integrated inside Docker engine
No need of separate external KV store
Usage example:
docker swarm init --advertise-addr <manager-ip>
docker -H <worker-ip> swarm join --token <worker-token>
Docker Swarm:
Docker swarm is a service which allows users to create and manage a cluster of docker nodes and schedule container. Each node in docker swarm is a docker daemon and docker daemon interact using docker API.
Swarm Mode:
When we create a cluster of one or more Docker Engines its called a swarm mode. Swarm mode was introduced in Docker Engine 1.12. A swarm consists of one or more nodes physical or virtual machines running Docker Engine.
Related
I've managed to run Docker Swam mode with multiple hosts with Docker Toolbox, but I am unable to create a swarm with Docker Desktop since it apparently only offers single node swarm.
Is there any way to get this working with Docker Desktop or is it not supported?
No. But yes. But actually no. But technically yes.
No. Docker Desktop does not support this. It manages a single docker node in a vm and has no capability to manage multiple dockers.
But yes. docker:dind is an image you can easily use to deploy multiple docker nodes as containers, and then swarm init / swarm join to create a swarm cluster hosted on docker. You can even swarm join the docker-desktop node to be the swarm manager which means you can communicate with your local docker desktop node to control the swarm.
But actually, no.
Unless your use case is a very limited hello-world on swarm tutorial, there is no support for exposing ports from the dind-swarm to the host. Even if the host docker acts as the manager, overlay networking that is required for ingress will require communications over :2377, :4789/udp, and :7946, and as the host is not part of its own overlay networks, this will never work.
So, communicating with tasks running on the swarm is basically impossible.
But technically yes. play-with-docker apparently runs docker swarms using dind. They do some heavy lifting to expose a restricted set of ports via l7 loadbalancers. Pretty cool. but not at all easy to do at home. If you have a spare Dell PowerEdge or equivalent blade server with 120+ cores just laying around, and want to expose it as a docker swarm rather than split it into VMS... perhaps this is a viable approach.
How do I start a docker swarm cluster with consul back-end?
I can't see any discovery param in the docker swarm init command? or the docker swarm join command? I successfully ran
docker swarm init ....
and than
docker swarm join
to start a cluster on the internal swarm discovery mechanism, but it's not recommended for production. So what am I missing?
You are running the newer Swarm Mode commands but asking about the usage of the classic Swarm that runs as a container, these are two very different things.
Swarm Mode uses a raft implementation for the manager state that is not swappable with an external key/value store. You run swarm mode with the commands you listed (docker swarm init and docker swarm join). The join command eliminates the need for an external node discovery database. https://docs.docker.com/engine/swarm/how-swarm-mode-works/nodes/
Classic swarm used an external node discovery, the default using a docker hub token that was not recommended for production. To implement classic Swarm you run the docker run swarm manage with the options to publish the port to access the manager and option to discover the nodes in the swarm. Classic Swarm has more in common with a reverse proxy to the docker api than an orchestration tool like Swarm Mode or Kubernetes. https://docs.docker.com/swarm/reference/manage/
So the answer to your question is to either not use Swarm Mode commands and instead run the classic Swarm containers, or if you want Swarm Mode, to not try to implement your own external node discovery database because that's not an option. I'd recommend the latter unless you have a specific need for classic Swarm.
I am using the latest docker for mac (17.09.0-ce), when I run docker-machine create --help it says
...
--swarm-master Configure Machine to be a Swarm master
But the document here said "run docker swarm init to enable swarm mode and make your current machine a swarm manager"
So what is the difference between swarm master and swarm manager?
------ update -------
I found this so post explained it all, Need for service discovery for docker engine swarm mode
These are two different implementations of Swarm, with slightly different terminology.
The docker-machine command is configuring the classic container-based Swarm. You can find it on docker hub. This version of swarm is more like a reverse proxy to the various docker engines.
The docker swarm init command enables Swarm Mode that is now included with the docker engine. Swarm Mode is closer in features to Kubernetes and provides orchestration. You will most likely want Swarm Mode, classic Swarm is being preserved for historical reasons and special use cases.
i'm learning about docker swarm, and got confused about the swarm discovery option, i see that lots of tutorials on internet use this option to create containers with docker-machine, but when i enter the documentation on docker swarm doc it says:
You are viewing docs for legacy standalone Swarm. These topics describe standalone Docker Swarm. In Docker 1.12 and higher, Swarm mode is integrated with Docker Engine. Most users should use integrated Swarm mode.
So, what are the use cases for the discovery options? All the tutorials use the docker-machine to create a swarm, i always need it or can just install the docker on machines in my cluster, join them in swarm and use normal?
I saw some names like Docker Swarm and Docker Swarm Mode, are there any difference or just different ways to call the same feature?
Q. Docker Swarm discovery is still relevant?
A: No, if you use docker Swarm Mode and an overlay network (see below)
Q. Are there any difference between Docker Swarm and Docker Swarm Mode?
A: Yes, TL;DR Docker Swarm is deprecated and should not be used anymore, Docker Swarm Mode (we should just say Swarm Mode) is the recommended way of clustering containers and have reliability, load-balancing, scaling, and rolling service upgrades.
Docker Swarm (official doc) :
is the old fashioned way (<1.12) of clustering containers
uses a dedicated container for building a Docker Swarm cluster
needs a discovery service like Consul to reference containers in cluster
Swarm Mode (official doc):
is the new and recommended way (>=1.12) of clustering containers on host nodes (called managers / workers)
is built-in in Docker engine, you don't need an additional container
has a built-in discovery service if you use an overlay network (DNS resolution is done within this network), you don't need an additional container
You can have a look to this SO thread on same topic.
Q. Do i always need docker-machine to create a swarm?
A: No, docker-machine is a helper to create virtual hosts in the cloud like amazon ec2, azure, digitalocean, google, openstack..., or your own network with virtual box.
To create a Swarm Mode, you need :
a multiple hosts cluster with docker engine installed on each host (called node) (that is what docker-machine facilitates)
run docker swarm init to switch to Swarm Mode on your first manager node
run docker swarm join on worker nodes to add them in the cluster
There are some subtle adjustments to Swarm mode to increase high availability (recommended number of managers in the swarm, node placement in multiple availability zones in the cloud)
Hope this helps!
After looking through docker official swarm explanations, github issues and stackoverflow answers im still at a loss on why i am having the problem that i have.
Issue at hand: docker-compose up starts services not in the swarm even though swarm is active and has 2 nodes.
Im using 1.12.1 docker version.
Looking at swarm tutorial i was able to start and scale my swarm using docker service create without any issues.
running docker-compose up with version 2 docker-compose.yml results in services starting outside of swarm, i can see them through docker ps but not docker service ls
I can see that docker-machine as the tool that solves this problems, but then again it needs virtual box to be installed.
so my questions would be
Can i use docker-compose with docker-swarm (NOT docker-engine) without docker-machine and without experimental build bundle functionality?
If docker service create can start a service on any nodes is it an indication that network configuration of the swarm is correct ?
What is the advantages/disadvantages of docker-machine versus experimental build functionality
1) No. Docker Compose isn't integrated with the new Swarm Mode yet. Issue 3656 in GitHub is tracking that. If you start containers on a swarm with Docker Compose at the moment, it uses docker run to start containers, which is why you see them all on one node.
2) Yes. Actually you can use docker node ls on the manager to confirm all the nodes are up and active, and docker node inspect to check a particular node, you don't need to create a service to validate the swarm.
3) Docker Machine is also behind the 1.12 release, so if you start a swarm with Docker Machine it will be the 'old' type of swarm. The old Docker Swarm product needed a whole lot of extra setup for a key-value store, TLS etc. which Swarm Mode does for free.
1) You can't start services using docker-compose on the new Docker "Swarm Mode". There's a feature to convert a docker-compose file to the new dab format which is understood by the new swarm mode but that's incomplete and experimental at this point. You basically need to use bash scripts to start services at the moment.
2) The nodes in a swarm (swarm mode) interact using their own overlay network. It's the one named ingress when you do docker network ls. You need to setup your own overlay network to run services in. eg:
docker network create -d overlay mynet
docker service create --name serv1 --network mynet nginx
3) I'm not sure what feature you mean by "experimental build'. docker-machine is just a way to create hosts (the nodes). It facilitates the setting up of the docker daemon on each host, the certificates and allows some basic maintenance (renewing the certs, stopping/starting a host if you're the one who created it). It doesn't create services, volumes, networks or manages them. That's the job of the docker api.