Is it possible to create swarm with the following nodes setup:
Host machine(docker desktop in Windows containers mode)- Manager, Worker
VM on HyperV, same machine- Worker
I tried following:
docker swarm init --advertise-addr 172.18.69.65
docker-machine create --driver hyperv --hyperv-virtual-switch ‘Default Switch’ --hyperv-disable-dynamic-memory --hyperv-memory 2048 --hyperv-boot2docker-url https://github.com/boot2docker/boot2docker/releases/download/v19.03.2/boot2docker.iso docker-worker-linux
docker-machine ssh docker-worker-linux
docker#docker-worker-linux:~$ ping 172.18.69.65
PING 172.18.69.65 (172.18.69.65): 56 data bytes
64 bytes from 172.18.69.65: seq=0 ttl=128 time=0.294 ms
64 bytes from 172.18.69.65: seq=1 ttl=128 time=0.264 ms
docker#docker-worker-linux:~ docker swarm join --token bla-bla-bla 172.18.69.65:2377 Error response from daemon: Timeout was reached before node joined. The attempt to join the swarm will continue in the background. Use the "docker info" command to see the current swarm status of your node.
docker#docker-worker-linux:~ docker info
Swarm: error
NodeID:
Error: rpc error: code = DeadlineExceeded desc = context deadline exceeded
Is Manager: false
Node Address: 172.18.69.66
Should it be possible at all- joining host windows machine running docker desktop with linux vm as swarm nodes?
Related
I intend to test a non-trivial Kubernetes setup as part of CI and wish to run the full system before CD. I cannot run --privileged containers and am running the docker container as a sibling to the host using docker run -v /var/run/docker.sock:/var/run/docker.sock
The basic docker setup seems to be working on the container:
linuxbrew#03091f71a10b:~$ docker run hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.
However, minikube fails to start inside the docker container, reporting connection issues:
linuxbrew#03091f71a10b:~$ minikube start --alsologtostderr -v=7
I1029 15:07:41.274378 2183 out.go:298] Setting OutFile to fd 1 ...
I1029 15:07:41.274538 2183 out.go:345] TERM=xterm,COLORTERM=, which probably does not support color
...
...
...
I1029 15:20:27.040213 197 main.go:130] libmachine: Using SSH client type: native
I1029 15:20:27.040541 197 main.go:130] libmachine: &{{{<nil> 0 [] [] []} docker [0x7a1e20] 0x7a4f00 <nil> [] 0s} 127.0.0.1 49350 <nil> <nil>}
I1029 15:20:27.040593 197 main.go:130] libmachine: About to run SSH command:
sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname
I1029 15:20:27.040992 197 main.go:130] libmachine: Error dialing TCP: dial tcp 127.0.0.1:49350: connect: connection refused
This is despite the network being linked and the port being properly forwarded:
linuxbrew#51fbce78731e:~$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
93c35cec7e6f gcr.io/k8s-minikube/kicbase:v0.0.27 "/usr/local/bin/entr…" 2 minutes ago Up 2 minutes 127.0.0.1:49350->22/tcp, 127.0.0.1:49351->2376/tcp, 127.0.0.1:49348->5000/tcp, 127.0.0.1:49349->8443/tcp, 127.0.0.1:49347->32443/tcp minikube
51fbce78731e 7f7ba6fd30dd "/bin/bash" 8 minutes ago Up 8 minutes bpt-ci
linuxbrew#51fbce78731e:~$ docker network ls
NETWORK ID NAME DRIVER SCOPE
1e800987d562 bridge bridge local
aa6b2909aa87 host host local
d4db150f928b kind bridge local
a781cb9345f4 minikube bridge local
0a8c35a505fb none null local
linuxbrew#51fbce78731e:~$ docker network connect a781cb9345f4 93c35cec7e6f
Error response from daemon: endpoint with name minikube already exists in network minikube
The minikube container seems to be alive and well when trying to curl from the host and even sshis responding:
mastercook#linuxkitchen:~$ curl https://127.0.0.1:49350
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 127.0.0.1:49350
mastercook#linuxkitchen:~$ ssh root#127.0.0.1 -p 49350
The authenticity of host '[127.0.0.1]:49350 ([127.0.0.1]:49350)' can't be established.
ED25519 key fingerprint is SHA256:0E41lExrrezFK1QXULaGHgk9gMM7uCQpLbNPVQcR2Ec.
This key is not known by any other names
What am I missing and how can I make minikube properly discover the correctly working minikube container?
Because minikube does not complete the cluster creation, running Kubernetes in a (sibling) Docker container favours kind.
Given that the (sibling) container does not know enough about its setup, the networking connections are a bit flawed. Specifically, a loopback IP is selected by kind (and minikube) upon cluster creation even though the actual container sits on a different IP in the host docker.
To correct the networking, the (sibling) container needs to be connected to the network actually hosting the Kubernetes image. To accomplish this, the procedure is illustrated below:
Create a kubernetes cluster:
linuxbrew#324ba0f819d7:~$ kind create cluster --name acluster
Creating cluster "acluster" ...
✓ Ensuring node image (kindest/node:v1.21.1) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-acluster"
You can now use your cluster with:
kubectl cluster-info --context kind-acluster
Thanks for using kind! 😊
Verify if the cluster is accessible:
linuxbrew#324ba0f819d7:~$ kubectl cluster-info --context kind-acluster
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
The connection to the server 127.0.0.1:36779 was refused - did you specify the right host or port?
3.) Since the cluster cannot be reached, retrieve the control planes master IP. Note the "-control-plane" addition to the cluster name:
linuxbrew#324ba0f819d7:~$ export MASTER_IP=$(docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' acluster-control-plane)
4.) Update the kube config with the actual master IP:
linuxbrew#324ba0f819d7:~$ sed -i "s/^ server:.*/ server: https:\/\/$MASTER_IP:6443/" $HOME/.kube/config
5.) This IP is still not accessible by the (sibling) container and to connect the container with the correct network retrieve the docker network ID:
linuxbrew#324ba0f819d7:~$ export MASTER_NET=$(docker inspect --format='{{range .NetworkSettings.Networks}}{{.NetworkID}}{{end}}' acluster-control-plane)
6.) Finally connect the (sibling) container ID (which should be stored in the $HOSTNAME environment variable) with the cluster docker network:
linuxbrew#324ba0f819d7:~$ docker network connect $MASTER_NET $HOSTNAME
7.) Verify whether the control plane accessible after the changes:
linuxbrew#324ba0f819d7:~$ kubectl cluster-info --context kind-acluster
Kubernetes control plane is running at https://172.18.0.4:6443
CoreDNS is running at https://172.18.0.4:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
If kubectl returns Kubernetes control plane and CoreDNS URL, as shown in the last step above, the configuration has succeeded.
You can run minikube in docker in docker container. It will use docker driver.
docker run --name dind -d --privileged docker:20.10.17-dind
docker exec -it dind sh
/ # wget https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
/ # mv minikube-linux-amd64 minikube
/ # chmod +x minikube
/ # ./minikube start --force
...
* Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
/ # ./minikube kubectl -- run --image=hello-world
/ # ./minikube kubectl -- logs pod/hello
Hello from Docker!
...
Also, note that --force is for running minikube using docker driver as root which we shouldn't do according minikube instructions.
After sudo docker swarm join --token XXXXX YYY.YYY.YYY.YYY:2377 I can attach to swarm as worker successfully. Than I leave this swarm from secondary/slave node and try again with management token. And receive:
Error response from daemon: manager stopped: can't initialize raft node: rpc error: code = Unknown desc = could not connect to prospective new cluster member using its advertised address: rpc error: code = DeadlineExceeded desc = context deadline exceeded
Both nodes directly connected one to another. Firewall in both node is not working. What can be a reason of this issue?
You can add a node as worker and promote it to manager role
docker swarm join --token XXXXX YYY.YYY.YYY.YYY:2377
And on the manager node:
docker node promote SECOND_MANAGER_HOSTNAME
For me port 2377 was not open as a TCP destination in the subnet ingress rule.
I am using VMs from different VPCs.
I am a beginner to docker.Please correct me if anything wrong.
As shown in this docker swarm tutorial https://www.youtube.com/watch?v=nGSNULpHHZc , i am trying to setup multhost setup for my hyperledger fabric application.
I am using two oracle linux servers namely server 1 and server 2.
I connected both the servers using the docker swarm as managers and created overlay network called my-net.
I followed the same syntax given in the above mentioned tutorial and created the service using the beolw mentioned syntax.
docker service create --name myservice --network my-net --replicas 2 alpine sleep 1d
As expected it created one conatianer in each the server.
Say for example server 1 coantainer IP is 10.0.0.4 and server 2 container IP 10.0.0.5.
Now, i am trying to ping from the second servers container to first server's container as shown below and it is pinging.
# docker exec -it ContainerID sh
/ # ping 10.0.0.4
PING 10.0.0.4 (10.0.0.4): 56 data bytes
64 bytes from 10.0.0.4: seq=0 ttl=64 time=0.082 ms
64 bytes from 10.0.0.4: seq=1 ttl=64 time=0.062 ms
64 bytes from 10.0.0.4: seq=2 ttl=64 time=0.067 ms
^C
--- 10.0.0.4 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.062/0.070/0.082 ms
Now, I am trying to create my service(1) using the beolw mentioned syntax.
docker service create --name myservice1 --network my-net --replicas 2 hyperledger/fabric-peer sleep 1d
As expected this also created one conatianer in each the server.
Say for example server 1 coantainer IP is 10.0.0.6 and server 2 container IP 10.0.0.7.
Now, I am trying to ping from the second servers container to first server's container as shown below.
This time i am getting ping not found error,
# docker exec -it ContainerID sh
# ping 10.0.0.6
sh: 1: ping: not found
Can anyone please help what is the problem with the second myservice1.
The Fabric Docker images are based on a bare bones base Ubuntu image and do not include utilities like ping. Once you "exec" into the peer containers, you use "apt" to install ping:
apt-get update
apt-get install inetutils-ping
Added -ping at the end
Expanding on Gari Singh's answer, on a Fabric network I've spun this week, the inetutils has been split in different packages:
# apt-cache search inetutils
inetutils-ftp - File Transfer Protocol client
inetutils-ftpd - File Transfer Protocol server
inetutils-inetd - internet super server
inetutils-ping - ICMP echo tool
inetutils-syslogd - system logging daemon
inetutils-talk - talk to another user
inetutils-talkd - remote user communication server
inetutils-telnet - telnet client
inetutils-telnetd - telnet server
inetutils-tools - base networking utilities (experimental pac
so to install e.g. ping the correct command has become:
# apt-get install inetutils-ping
The Ubuntu version of the peer is:
# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.5 LTS"
I use windows container and try to create docker swarm ，I create three virtual machine use hyper-v , and each OS is windows server 2016.There machines ip is :
windocker211 192.168.1.211
windocker212 192.168.1.212
windocker219 192.168.1.219
The docker swarm node is :
PS C:\ConsoleZ> docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
4c0g0o0uognheugw4do1a1h7y windocker212 Ready Active
bbxot0c8zijq7xw4lm86svgwp * windocker219 Ready Active Leader
wftwpiqpqpbqfdvgenn787psj windocker211 Ready Active
I create use command:
docker service create --name=demo5 -p 5005:5005 --replicas 6 192.168.1.245/cqgis/wintestcore:0.6
The docker image is asp.net core app , the Dockerfile is:
FROM 192.168.1.245/win/aspnetcore-runtime:1.1.2
COPY . /app
WORKDIR /app
ENV ASPNETCORE_URLS http://*:5005
EXPOSE 5005/tcp
ENTRYPOINT ["dotnet", "dotnetcore.dll"]
then it create success:
PS C:\ConsoleZ> docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
omhu7e0vo96s demo5 replicated 6/6 192.168.1.245/cqgis/wintestcore:0.6 *:5005->5005/tcp
PS C:\ConsoleZ> docker service ps demo5
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
8pihnak9a2ei demo5.1 192.168.1.245/cqgis/wintestcore:0.6 windocker212 Running Running 59 seconds ago
ut3f3b9giu4w demo5.2 192.168.1.245/cqgis/wintestcore:0.6 windocker219 Running Running 47 seconds ago
iy1xjevt67yl demo5.3 192.168.1.245/cqgis/wintestcore:0.6 windocker211 Running Running about a minute ago
q7f1gnbwslr3 demo5.4 192.168.1.245/cqgis/wintestcore:0.6 windocker212 Running Running about a minute ago
8zewaktcu32h demo5.5 192.168.1.245/cqgis/wintestcore:0.6 windocker219 Running Running about a minute ago
xq820kqwf3v9 demo5.6 192.168.1.245/cqgis/wintestcore:0.6 windocker211 Running Running 55 seconds ago
but my question is I cann't accessing The Site each by
http://192.168.1.219:5005/
http://192.168.1.219:5005/
http://192.168.1.219:5005/
When I use command
docker run -it -p 5010:5005 192.168.1.245/cqgis/wintestcore:0.6
I can use http://192.168.1.219:5010/ get the right result
my docker info is
PS C:\ConsoleZ> docker info
Containers: 4
Running: 3
Paused: 0
Stopped: 1
Images: 5
Server Version: 17.06.0-ce-rc1
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: l2bridge l2tunnel nat null overlay transparent
Log: awslogs etwlogs fluentd json-file logentries splunk syslog
Swarm: active
NodeID: bbxot0c8zijq7xw4lm86svgwp
Is Manager: true
ClusterID: 32vsgwrbn6ihvpevly71gkgxk
Managers: 1
Nodes: 3
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Root Rotation In Progress: false
Node Address: 192.168.1.219
Manager Addresses:
192.168.1.219:2377
Default Isolation: process
Kernel Version: 10.0 14393 (14393.1198.amd64fre.rs1_release_sec.170427-1353)
Operating System: Windows Server 2016 Datacenter
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 2.89GiB
Name: windock219
ID: 7AOY:OT6V:BTJV:NCHA:3OF5:5WR5:K2YR:CFG3:VXLD:QTMD:GA3D:ZFJ2
Docker Root Dir: C:\ProgramData\docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: -1
Goroutines: 297
System Time: 2017-06-04T19:58:20.7582294+08:00
EventsListeners: 2
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
192.168.1.245
127.0.0.0/8
Live Restore Enabled: false
I beleive you need to publish port in "host" mode (learn.microsoft.com/en-us/virtualization/windowscontainers/…). Also it will be one to one port mapping between running container and host and hence you will not be able to run several containers on the same port. Routing mesh is not working on Windows yet.
There are some differences in the network between Docker for windows container and Docker for Linux. Windows Containers uses the HyperV Network technologies to provide the virtual networking features that docker uses. From there are some restrictions that are not work like you would expect or maybe found in standard Docker Documentation.
First you cannot access the web side running inside your container by
using the lookback address (127.0.0.1) or the host address (192.168.1.xxx) You have to call it
always from a remote machine.
I saw you are using the expose command in your Dockerfile. It is not
so self-explaining but expose is to expose a port in any other
network then the host or ingress network. It’s not a problem if you
do that in a non swarm configuration but it does not work in a swarm.
I Suggest to remove the Expose command.
There are some unsolved problems with windows networking. Sometimes the port stays in use after the container gets restarted.
For example, after a reboot of the host system.
[https://github.com/moby/moby/issues/21558][1]
With this scrip you can remove the all virtual network settings:
Stop-Service docker
Get-ContainerNetwork | Remove-ContainerNetwork
Get-NetNat | Remove-NetNat
Get-VMSwitch | Remove-VMSwitch
Start-Service docker
You cannot reach a container's published port from the same machine because of a limitation of the WinNAT networking. But you can reach the required port using an external request.
In your example, from a machine other than 192.168.1.219, accessing using the url http://192.168.1.219:5005/ will succeed. The url's http://192.168.1.211:5005/ and http://192.168.1.212:5005/ will also succeed provided the requests originate from outside those machines.
Using the 'host' mode will succeed: however, you are not getting the advantage of the 'routing mesh' feature which allows the service to be reachable from any of the services' nodes - only from that one single node.
When running docker containers a swarm cluster do the containers have access to all the IPs of the cluster nodes via ENV variables or otherwise?
I want to run an Elasticsearch instance on each node in my swarm the cluster. And they will discovery each other in unicast mode. Therefore each Elasticsearch instanc needs to be configured with the list of IPs in the cluster.
If you mean that container of one node can access container's IP of other node , then it is not possible . You have to use weave tool to connect container across different node or other tool .
If you are using latest Docker (1.13+) with a swam overlay network, you should be able to get all the cluster's node IPs through DNS round robin.(--endpoint-mode dnsrr)
1) Create an overlay network.
https://docs.docker.com/engine/swarm/networking/
docker network create \
--driver overlay \
my-network
2) Verify swam nodes:
docker#node1:~$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
5l07yt2itiee60xfq7g6c01e4 * node1 Ready Active Leader
pckn7qo3xpbxvs89ni6whyql3 node2 Ready Active
3) Create an alpine container on each nodes using "global" mode:
docker service create --mode global --endpoint-mode dnsrr --name testservice --detach=true --network my-network alpine ash -c "apk update;apk add drill; ping docker.com"
4) verify service is running:
docker#node1:~$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
lmy5s3flw763 testservice global 2/2 alpine:latest
5) Verify that containers were deployed on individual nodes:
$ docker-machine ssh node1 "docker ps"
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4c7055b01479 alpine:latest "ash -c 'apk updat..." 2 minutes ago Up 2 minutes testservice.5l07yt2itiee60xfq7g6c01e4.atvascigh3rvxvlzttaotkrua
$ docker-machine ssh node2 "docker ps"
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
28da546aa0d5 alpine:latest "ash -c 'apk updat..." 2 minutes ago Up 2 minutes testservice.pckn7qo3xpbxvs89ni6whyql3.ebjz4asni4w1f0srna0p3vj4a
6) Confirm individual virtual IP of each containers on node1 and node2:
| => docker-machine ssh node1 "docker exec 4c7055b01479 ash -c 'ip addr'|grep eth0"
349: eth0#if350: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP
inet 10.0.0.2/24 scope global eth0
| => docker-machine ssh node2 "docker exec 28da546aa0d5 ash -c 'ip addr'|grep eth0"
319: eth0#if320: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP
inet 10.0.0.3/24 scope global eth0
7) Get the container IP addresses for all containers in cluster using Drill dns tool :
| => docker-machine ssh node1 "docker exec 4c7055b01479 ash -c 'drill testservice'"
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 60920
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;; testservice. IN A
;; ANSWER SECTION:
testservice. 600 IN A 10.0.0.3
testservice. 600 IN A 10.0.0.2
;; AUTHORITY SECTION:
;; ADDITIONAL SECTION:
;; Query time: 0 msec
;; SERVER: 127.0.0.11
;; WHEN: Thu Jul 20 19:20:49 2017
;; MSG SIZE rcvd: 83
8) Verify that containers can ping each other:
docker-machine ssh node1 "docker exec 4c7055b01479 ash -c 'ping -c2 10.0.0.3'"
PING 10.0.0.3 (10.0.0.3): 56 data bytes
64 bytes from 10.0.0.3: seq=0 ttl=64 time=0.539 ms
64 bytes from 10.0.0.3: seq=1 ttl=64 time=0.731 ms
--- 10.0.0.3 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.539/0.635/0.731 ms
docker-machine ssh node2 "docker exec 28da546aa0d5 ash -c 'ping -c2 10.0.0.2'"
PING 10.0.0.2 (10.0.0.2): 56 data bytes
64 bytes from 10.0.0.2: seq=0 ttl=64 time=0.579 ms
64 bytes from 10.0.0.2: seq=1 ttl=64 time=0.736 ms
--- 10.0.0.2 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.579/0.657/0.736 ms