Docker Swarm - strategy doesn't seem to have any effect - docker

I have provisioned a swarm master and swarm nodes with Docker Machine (as described here). Everything is working fine; all the machines are created and running, they have all been discovered and accepts containers.
The output from 'docker-machine ls' is:
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
default - virtualbox Stopped Unknown
local - virtualbox Stopped Unknown
my-swarm * (swarm) digitalocean Running tcp://104.131.161.197:2376 my-swarm (master) v1.11.1
node0 - digitalocean Running tcp://104.236.29.169:2376 my-swarm v1.11.1
node1 - digitalocean Running tcp://104.236.216.164:2376 my-swarm v1.11.1
The problem I'm having is with the distribution of containers. No matter which strategy I set for the swarm, it only seems to distribute containers to one of the nodes at a time. I.e. I run a bunch of containers, and the are all started on the same node, as shown below (with strategy Spread):
$ docker ps
5c075d7ccddc stress "/bin/sh -c /stress.s" 32 seconds ago Up 31 seconds node0/elated_goldstine
5bae22a15829 stress "/bin/sh -c /stress.s" 46 seconds ago Up 44 seconds node0/cocky_booth
dc52b3dfa0e6 stress "/bin/sh -c /stress.s" About a minute ago Up About a minute node0/goofy_kalam
3b9e69c694da stress "/bin/sh -c /stress.s" About a minute ago Up About a minute node0/focused_fermat
ef0e006ff3e0 stress "/bin/sh -c /stress.s" About a minute ago Up About a minute node0/stoic_engelbart
53e46b19ab33 stress "/bin/sh -c /stress.s" About a minute ago Up About a minute node0/condescending_rosalind
e9e126c7f4c6 stress "/bin/sh -c /stress.s" About a minute ago Up About a minute node0/sleepy_jang
f9c0003d509d stress "/bin/sh -c /stress.s" About a minute ago Up About a minute node0/amazing_bhaskara
What I would expect here is for the containers to be distributed roughly evenly on the 3 nodes, especially as the script I'm running in the containers is designed to take as much CPU as possible. But instead all of them are on node0 (which I would only expect with Binpack). The Random strategy has the exact same behaviour.
The output from 'docker info' with the swarm master set as active seems correct:
$ docker info
Containers: 15
Running: 4
Paused: 0
Stopped: 11
Images: 5
Server Version: swarm/1.2.1
Role: primary
Strategy: spread
Filters: health, port, containerslots, dependency, affinity, constraint
Nodes: 3
my-swarm: 104.131.161.197:2376
└ ID: L2HK:F6S3:WWIM:BHNI:M4XL:KLEA:4U22:J6CE:ZHZI:OGGT:76KF:MTQU
└ Status: Healthy
└ Containers: 2
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 513.4 MiB
└ Labels: executiondriver=, kernelversion=4.2.0-27-generic, operatingsystem=Ubuntu 15.10, provider=digitalocean, storagedriver=aufs
└ Error: (none)
└ UpdatedAt: 2016-05-09T10:25:24Z
└ ServerVersion: 1.11.1
node0: 104.236.29.169:2376
└ ID: I3TQ:5BMS:TM2P:GLL4:64OH:BDMY:SWBU:3QG4:TOZ2:LEDW:A6SQ:X34H
└ Status: Healthy
└ Containers: 12
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 513.4 MiB
└ Labels: executiondriver=, kernelversion=4.2.0-27-generic, operatingsystem=Ubuntu 15.10, provider=digitalocean, storagedriver=aufs
└ Error: (none)
└ UpdatedAt: 2016-05-09T10:25:02Z
└ ServerVersion: 1.11.1
node1: 104.236.216.164:2376
└ ID: OTQH:UBSV:2HKE:ZVHL:2K7Z:BYGC:ZX25:Y6BQ:BN5J:UWEB:65KE:DABM
└ Status: Healthy
└ Containers: 1
└ Reserved CPUs: 0 / 1
└ Reserved Memory: 0 B / 513.4 MiB
└ Labels: executiondriver=, kernelversion=4.2.0-27-generic, operatingsystem=Ubuntu 15.10, provider=digitalocean, storagedriver=aufs
└ Error: (none)
└ UpdatedAt: 2016-05-09T10:25:10Z
└ ServerVersion: 1.11.1
Plugins:
Volume:
Network:
Kernel Version: 4.2.0-27-generic
Operating System: linux
Architecture: amd64
CPUs: 3
Total Memory: 1.504 GiB
Name: my-swarm
Docker Root Dir:
Debug mode (client): false
Debug mode (server): false
WARNING: No kernel memory limit support
Is there some piece I have missed that is necessary for this type of automatic distribution to work the way I'm expecting?

Issue
It is most likely that you Docker Agent does not have the image you try to run on them.
Note that swarm build and swarm pull will not build, or pull, on each Docker agent.
Solution A
Run the pull manually on each Docker Agent
Solution B (not tested)
Run a local Docker registry wherein the Docker agent will pull the images from.
Have fun with Swarm! Also, if I may, I suggest you to read this answer that details a whole step-by-step tutorial on Swarm.

Related

i/o timeout ImagePullBackOff issue on minikube docker image for pod on fedora32

I am getting ImagePullBackOff error when I pull images from docker fro kubernates pods. I am using Fedora32 and running a minikube on docker.
'''
[prem#localhost simplek8s]$ **docker ps --all**
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
070fba4347b7 **gcr.io/k8s-minikube/kicbase:v0.0.13** "/usr/local/bin/entr…" 3 days ago Up About an hour 127.0.0.1:32771->22/tcp, 127.0.0.1:32770->2376/tcp, 127.0.0.1:32769->5000/tcp, 127.0.0.1:32768->8443/tcp minikube
85a476d91b28 hello-world "/hello" 3 days ago Exited (0) 3 days ago musing_fermat
dbf5151bc72e frontend_tests "docker-entrypoint.s…" 6 days ago Exited (137) 3 days ago frontend_tests_1
e47486560719 frontend_web "docker-entrypoint.s…" 6 days ago Exited (137) 3 days ago frontend_web_1
75933fdf45c4 274c1d9065e6 "/bin/sh -c 'npm ins…" 6 days ago Created romantic_sinoussi
ab0d87295579 274c1d9065e6 "/bin/sh -c 'npm ins…" 6 days ago Created
'''
In the above containers you can see minikube running.
This is what is showing when I see the pods.
[prem#localhost simplek8s]$ **kubectl get pods**
NAME READY STATUS RESTARTS AGE
busybox 0/1 ImagePullBackOff 0 47h
client-pod 0/1 ImagePullBackOff 0 2d
dnsutils 0/1 ImagePullBackOff 0 21m
nginx 0/1 ImagePullBackOff 0 47h
[prem#localhost simplek8s]$ **kubectl describe pods**
Name: busybox
Namespace: default
Priority: 0
Node: minikube/192.168.49.2
Start Time: Wed, 28 Oct 2020 02:05:29 +0530
Labels: run=busybox
Annotations: <none>
Status: Pending
IP: 172.17.0.2
IPs:
IP: 172.17.0.2
Containers:
busybox:
Container ID:
Image: busybox
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-zr7nz (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-zr7nz:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-zr7nz
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
**Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute** op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Pulling 60m (x4 over 63m) kubelet Pulling image "busybox"
Warning Failed 60m **kubelet Failed to pull image "busybox": rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.49.1:53: read udp 192.168.49.2:38852->192.168.49.1:53: i/o timeout**
Warning Failed 13m (x13 over 62m) kubelet Error: ErrImagePull
Warning Failed 8m17s (x226 over 62m) kubelet Error: ImagePullBackOff
Normal BackOff 3m12s (x246 over 62m) kubelet Back-off pulling image "busybox"
[myname#localhost simplek8s]$ **minikube ssh**
docker#minikube:~$ cat /etc/resolv.conf
nameserver 192.168.49.1
options ndots:0
docker#minikube:~$ **curl google.com**
curl: (6) **Could not resolve host: google.com**
Clearly the pods are not able to access the internet to pull images. looks like I am missing some dns config to make the minikube pull images from internet.Help me resolve this.
Due to the changes in Fedora 32, the install experience is slightly more involved than usual, and currently requires some extra manual steps to be performed, depending on your machine's configuration.
See: docker-for-linux-fedora32.
Take a look also on: cannot-pull-image, could-not-resolve-host, curl-6-could-not-resolve-host, err-docker-daemon.
Remember that the current state of Docker on Fedora 32 is not ideal. The lack of an official package might bother some, and there is an issue upstream where this is discussed. See: docker-fedora32-example.
Take a look on this installation Docker on Fedora32 - docker-on-fedora32.
To install Docker Engine, you need the 64-bit version of one of these
Fedora versions:
Fedora 30
Fedora 31
I advice you to use different drivers, example kvm or virtualbox - minikube-drivers. Look at this guide - minikube-kubernetes.

Getting ImageInspectError when trying to run an OpenFaas function on Raspberry Pi 3B+

I'm trying to deploy a function with OpenFaas project and a kubernetes cluster running on 2 Raspberry Pi 3B+.
Unfortunately, the pod that should handle the function is going to ImageInspectError state...
I tried to run the function with Docker directly and which is contain in a Docker image, and everything is working fine.
I opened an issue on the OpenFaas github and the maintainer told me to ask directly the Kubernetes community to get some clues.
My first question is : What does ImageInspectError mean and where it comes from ?
And here is all the informations I have :
Expected Behaviour
Pod should run.
Current Behaviour
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system etcd-masternode 1/1 Running 1 1d
kube-system kube-apiserver-masternode 1/1 Running 1 1d
kube-system kube-controller-manager-masternode 1/1 Running 1 1d
kube-system kube-dns-7f9b64f644-x42sr 3/3 Running 3 1d
kube-system kube-proxy-wrp6f 1/1 Running 1 1d
kube-system kube-proxy-x6pvq 1/1 Running 1 1d
kube-system kube-scheduler-masternode 1/1 Running 1 1d
kube-system weave-net-4995q 2/2 Running 3 1d
kube-system weave-net-5g7pd 2/2 Running 3 1d
openfaas-fn figlet-7f556fcd87-wrtf4 1/1 Running 0 4h
openfaas-fn testfaceraspi-7f6fcb5897-rs4cq 0/1 ImageInspectError 0 2h
openfaas alertmanager-66b98dd4d4-kcsq4 1/1 Running 1 1d
openfaas faas-netesd-5b5d6d5648-mqftl 1/1 Running 1 1d
openfaas gateway-846f8b5686-724q8 1/1 Running 2 1d
openfaas nats-86955fb749-7vsbm 1/1 Running 1 1d
openfaas prometheus-6ffc57bb8f-fpk6r 1/1 Running 1 1d
openfaas queue-worker-567bcf4d47-ngsgv 1/1 Running 2 1d
The testfaceraspi doesn't run.
Logs from the pod :
$ kubectl logs testfaceraspi-7f6fcb5897-rs4cq -n openfaas-fn
Error from server (BadRequest): container "testfaceraspi" in pod "testfaceraspi-7f6fcb5897-rs4cq" is waiting to start: ImageInspectError
Pod describe :
$ kubectl describe pod -n openfaas-fn testfaceraspi-7f6fcb5897-rs4cq
Name: testfaceraspi-7f6fcb5897-rs4cq
Namespace: openfaas-fn
Node: workernode/10.192.79.198
Start Time: Thu, 12 Jul 2018 11:39:05 +0200
Labels: faas_function=testfaceraspi
pod-template-hash=3929761453
Annotations: prometheus.io.scrape=false
Status: Pending
IP: 10.40.0.16
Controlled By: ReplicaSet/testfaceraspi-7f6fcb5897
Containers:
testfaceraspi:
Container ID:
Image: gallouche/testfaceraspi
Image ID:
Port: 8080/TCP
Host Port: 0/TCP
State: Waiting
Reason: ImageInspectError
Ready: False
Restart Count: 0
Liveness: exec [cat /tmp/.lock] delay=3s timeout=1s period=10s #success=1 #failure=3
Readiness: exec [cat /tmp/.lock] delay=3s timeout=1s period=10s #success=1 #failure=3
Environment:
fprocess: python3 index.py
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-5qhnn (ro)
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
Volumes:
default-token-5qhnn:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-5qhnn
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning DNSConfigForming 2m (x1019 over 3h) kubelet, workernode Search Line limits were exceeded, some search paths have been omitted, the applied search line is: openfaas-fn.svc.cluster.local svc.cluster.local cluster.local heig-vd.ch einet.ad.eivd.ch web.ad.eivd.ch
And the event logs :
$ kubectl get events --sort-by=.metadata.creationTimestamp -n openfaas-fn
LAST SEEN FIRST SEEN COUNT NAME KIND SUBOBJECT TYPE REASON SOURCE MESSAGE
14m 1h 347 testfaceraspi-7f6fcb5897-rs4cq.1540db41e89d4c52 Pod Warning DNSConfigForming kubelet, workernode Search Line limits were exceeded, some search paths have been omitted, the applied search line is: openfaas-fn.svc.cluster.local svc.cluster.local cluster.local heig-vd.ch einet.ad.eivd.ch web.ad.eivd.ch
4m 1h 75 figlet-7f556fcd87-wrtf4.1540db421002b49e Pod Warning DNSConfigForming kubelet, workernode Search Line limits were exceeded, some search paths have been omitted, the applied search line is: openfaas-fn.svc.cluster.local svc.cluster.local cluster.local heig-vd.ch einet.ad.eivd.ch web.ad.eivd.ch
10m 10m 1 testfaceraspi-7f6fcb5897-d6z78.1540df9ed8b91865 Pod Normal Scheduled default-scheduler Successfully assigned testfaceraspi-7f6fcb5897-d6z78 to workernode
10m 10m 1 testfaceraspi-7f6fcb5897.1540df9ed6eee11f ReplicaSet Normal SuccessfulCreate replicaset-controller Created pod: testfaceraspi-7f6fcb5897-d6z78
10m 10m 1 testfaceraspi-7f6fcb5897-d6z78.1540df9eef3ef504 Pod Normal SuccessfulMountVolume kubelet, workernode MountVolume.SetUp succeeded for volume "default-token-5qhnn"
4m 10m 27 testfaceraspi-7f6fcb5897-d6z78.1540df9eef5445c0 Pod Warning DNSConfigForming kubelet, workernode Search Line limits were exceeded, some search paths have been omitted, the applied search line is: openfaas-fn.svc.cluster.local svc.cluster.local cluster.local heig-vd.ch einet.ad.eivd.ch web.ad.eivd.ch
8m 9m 8 testfaceraspi-7f6fcb5897-d6z78.1540df9f670d0dad Pod spec.containers{testfaceraspi} Warning InspectFailed kubelet, workernode Failed to inspect image "gallouche/testfaceraspi": rpc error: code = Unknown desc = Error response from daemon: readlink /var/lib/docker/overlay2/l: invalid argument
9m 9m 7 testfaceraspi-7f6fcb5897-d6z78.1540df9f670fcf3e Pod spec.containers{testfaceraspi} Warning Failed kubelet, workernode Error: ImageInspectError
Steps to Reproduce (for bugs)
Deploy OpenFaas on a 2 node k8s cluster
Create function with faas new testfaceraspi --lang python3-armhf
Add the following code in the handler.py :
import json
def handle(req):
jsonl = json.loads(req)
return ("Found " + str(jsonl["nbFaces"]) + " faces in OpenFaas Function on raspi !")
Change gateway and image in the .yml
provider:
name: faas
gateway: http://127.0.0.1:31112
functions:
testfaceraspi:
lang: python3-armhf
handler: ./testfaceraspi
image: gallouche/testfaceraspi
Run faas build -f testfacepi.yml
Login in DockerHub with docker login
Run faas push -f testfacepi.yml
Run faas deploy -f testfacepi.yml
Your Environment
FaaS-CLI version ( Full output from: faas-cli version ):
Commit: 3995a8197f1df1ecdf524844477cffa04e4690ea
Version: 0.6.11
Docker version ( Full output from: docker version ):
Client:
Version: 18.04.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 3d479c0
Built: Tue Apr 10 18:25:24 2018
OS/Arch: linux/arm
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.04.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.4
Git commit: 3d479c0
Built: Tue Apr 10 18:21:25 2018
OS/Arch: linux/arm
Experimental: false
Operating System and version (e.g. Linux, Windows, MacOS):
Distributor ID: Raspbian
Description: Raspbian GNU/Linux 9.4 (stretch)
Release: 9.4
Codename: stretch
Thanks by advance and tell me if you need any more informations.
Gallouche
I've seen this error because the docker version wasn't supported by Kubernetes. As of Kubernetes version 1.11, the supported versions are 1.11.2 to 1.13.1 and 17.03.x.
I couldn't test the solution with OpenFaaS.

docker-compose swarm: force containers to run on specific hosts

Trying to run cluster application on different virtual machines with use of Swarm stand alone and docker-compose version '2'. Overlay network is set. But want to force certain containers to run on specific hosts.
In documentation there is following advice, but with this parameter I was not able to start any container at all:
environment:
- "constraint:node==node-1"
ERROR: for elasticsearch1 Cannot create container for service elasticsearch1: Unable to find a node that satisfies the following conditions
[available container slots]
[node==node-1]
Should we register hosts as node-1 node-2... or it is done by default.
[root#ux-test14 ~]# docker node ls
Error response from daemon: 404 page not found
[root#ux-test14 ~]# docker run swarm list
[root#ux-test14 ~]#
[root#ux-test14 ~]# docker info
Containers: 8
Running: 6
Paused: 0
Stopped: 2
Images: 8
Server Version: swarm/1.2.5
Role: primary
Strategy: spread
Filters: health, port, containerslots, dependency, affinity, constraint
Nodes: 2
ux-test16.rs: 10.212.212.2:2375
â ID: JQPG:GKFF:KJZJ:AY3N:NHPZ:HD6J:SH36:KEZR:2SSH:XF65:YW3N:W4DG
â Status: Healthy
â Containers: 4 (4 Running, 0 Paused, 0 Stopped)
â Reserved CPUs: 0 / 2
â Reserved Memory: 0 B / 3.888 GiB
â Labels: kernelversion=3.10.0-327.28.3.el7.x86_64, operatingsystem=CentOS Linux 7 (Core), storagedriver=devicemapper
â UpdatedAt: 2016-09-05T11:11:31Z
â ServerVersion: 1.12.1
ux-test17.rs: 10.212.212.3:2375
â ID: Z27V:T5NU:QKSH:DLNK:JA4M:V7UX:XYGH:UIL6:WFQU:FB5U:J426:7XIR
â Status: Healthy
â Containers: 4 (2 Running, 0 Paused, 2 Stopped)
â Reserved CPUs: 0 / 2
â Reserved Memory: 0 B / 3.888 GiB
â Labels: kernelversion=3.10.0-327.28.3.el7.x86_64, operatingsystem=CentOS Linux 7 (Core), storagedriver=devicemapper
â UpdatedAt: 2016-09-05T11:11:17Z
â ServerVersion: 1.12.1
Plugins:
Volume:
Network:
Swarm:
NodeID:
Is Manager: false
Node Address:
Security Options:
Kernel Version: 3.10.0-327.28.3.el7.x86_64
Operating System: linux
Architecture: amd64
CPUs: 4
Total Memory: 7.775 GiB
Name: 858ac2fdd225
Docker Root Dir:
Debug Mode (client): false
Debug Mode (server): false
WARNING: No kernel memory limit support
My first answer is about "swarm mode". You'd since clarified that you're using legacy Swarm and added more info, so here:
The constraint you list assumes that you have a host named node-1. Your hosts are named ux-test16.rs and ux-test17.rs. Just use that instead of node-1 in your constraint. Eg:
environment:
- "constraint:node==ux-test16.rs"
The environment variable constraint is only valid for the legacy (stand alone) version of Swarm. The newer "Swarm Mode" uses either mode or constraints options (not environment variables).
To enforce one and only one task (container) per node, use mode=global.
docker service create --name proxy --mode global nginx
The default mode is replicated which means that the swarm manager will create tasks (containers) across all available nodes to meet the number specified in the --replicas option. Eg:
docker service create --name proxy --replicas 5 nginx
To enforce other constraints based on hostname (node), label, role, id's use the --constraint option. Eg:
docker service create --name proxy --constraint "node.hostname!=node01" nginx
See https://docs.docker.com/engine/reference/commandline/service_create/#/specify-service-constraints
EDIT sept 2016:
Something else. docker-compose is not currently supported in "swarm mode". Swarm mode understands the new dab format instead. There is a way to convert docker-compose files to dab but it's experimental and not to be relied on at this point. It's better to create a bash script that calls all the docker service create ... directly.
EDIT March 2017:
As of docker 1.13 (17.03), docker-compose can now be used to provision swarm environments directly without having to deal with the dab step.
Related issue - I had a recent Swarm project with a mixture of worker nodes (3 x Linux + 4 x Windows). My containers needed to run on a specific OS, but not on any specific node. Swarm mode now supports specifying an OS under "constraints" in docker-compose files. No need to create labels for each node:
version: '3'
services:
service_1:
restart: on-failure
image: 'service_1'
deploy:
placement:
constraints:
- node.platform.os == windows
junittestsuite:
restart: on-failure
image: 'junit_test_suite:1.0'
command: ant test ...
deploy:
placement:
constraints:
- node.platform.os == linux

How to run docker-compose against docker swarm (without docker-machine)

I have managed to manually set up a docker swarm (e.g.: without using docker-machine) following the official tutorial
I am able to run containers on the swarm successfully using docker engine:
docker -H :4000 run redis
I would like to use docker-compose to run containers on the swarm, however I cannot seem to get this right.
The first thing I had to work out was how to get compose to talk on port :4000. I achieved this by specifying: export DOCKER_HOST=":4000".
However, now, when I run docker-compose I get the following error:
$docker-compose up
Creating network "root_default" with the default driver
ERROR: Error response from daemon: failed to parse pool request for address space "GlobalDefault" pool "" subpool "": cannot find address space GlobalDefault (most likely the backing datastore is not configured)
It feels like this issue has to do with either TLS or network, but I'm pretty stumped as to how to fix it, or even how to go about investigating it further.
I'm using Docker engine: 1.10, Compose 1.6. Swarm:latest
In case it's useful, here is my docker info:
$docker -H :4000 info
Containers: 7
Running: 5
Paused: 0
Stopped: 2
Images: 7
Server Version: swarm/1.2.0
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 2
node02: 10.129.5.211:2375
└ Status: Healthy
└ Containers: 3
└ Reserved CPUs: 0 / 2
└ Reserved Memory: 0 B / 2.053 GiB
└ Labels: executiondriver=, kernelversion=3.13.0-79-generic, operatingsystem=Ubuntu 14.04.4 LTS, storagedriver=aufs
└ Error: (none)
└ UpdatedAt: 2016-04-15T08:28:20Z
└ ServerVersion: 1.11.0
node03: 10.129.6.21:2375
└ Status: Healthy
└ Containers: 4
└ Reserved CPUs: 0 / 2
└ Reserved Memory: 0 B / 2.053 GiB
└ Labels: executiondriver=, kernelversion=3.13.0-79-generic, operatingsystem=Ubuntu 14.04.4 LTS, storagedriver=aufs
└ Error: (none)
└ UpdatedAt: 2016-04-15T08:28:43Z
└ ServerVersion: 1.11.0
Plugins:
Volume:
Network:
Kernel Version: 3.13.0-79-generic
Operating System: linux
Architecture: amd64
CPUs: 4
Total Memory: 4.105 GiB
Name: b156985db557
Docker Root Dir:
Debug mode (client): false
Debug mode (server): false
WARNING: No kernel memory limit support
I am using Docker 1.10, Docker Swarm, Docker Compose on our production.
I have met the same issue in the past. And I solved it by steps as below:
Step 1: export DOCKER_HOST=tcp://localhost:4000
Step 2: Verify Docker Swarm by command docker info (without -H). If it not OK, make sure that Swarm Manager is working your host.
Step 3: If Step 2 is OK. Run your application on Docker Compose docker-compose up
I ran into the same problem and found the answer here: https://groups.google.com/a/weave.works/forum/m/#!topic/weave-users/Mf6fv9OEd-E
What fixed it was:
To make it work with docker-compose 2, you should add:
network_mode: "bridge"
to all service definitions.
to remove the above error, you need to run docker deamon on every node like this
docker run -d -p 4000:4000 swarm manage -H :4000 --replication --advertise 192.168.56.103:4000 consul://192.168.56.101:8500

Docker Swarm: Does it let two containers in different hosts to communictae?

I set up swarm cluster with two machine. It work as desired. I'm able to launch container on desired node based on constraint filter. However when I'm trying to ping one container in one node from container in another node, it fail. It does not recognize it. Is it as expected or I did something wrong in setting swarm cluster?
Other Details:
machine 1(10.0.0.4) as both host/node
machine 2(10.0.0.21) as node
Swarm agent is 10.0.0.4:2374(ip :port)
The output of info command it's:
docker -H tcp://10.0.0.4:2374 info
Containers: 11
strategy: spread
Filters: affinity, health, constraint, port, dependency
Nodes: 2
machine1: 10.0.0.4:2375
└ Containers: 6
└ Reserved CPUs: 0 / 25
└ Reserved Memory: 0 B / 24.76 GiB
machine2: 10.0.0.21:2375
└ Containers: 5
└ Reserved CPUs: 0 / 25
└ Reserved Memory: 0 B / 24.76 GiB
Overlay networks was introduced in Docker 1.9 (Nov 2015). It allows containers in different nodes (hosts) to be part of the same network and communicate.
Yes, from the docs "Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single, virtual host." https://docs.docker.com/swarm/
It looks docker swarm is more like managing tool and scheduler. I have to use some other tool like weave or ambassador to connect two container in different host . Anyhow, docker swarm is good clustering tool and help me setup thing as i desire.

Resources