When I create a node with docker-machine
docker-machine create -d virtualbox node1
it is created with tls verification enabled for docker deamon which made things a bit more of a hassle than normal for swarm.
I want to create a node with docker-machine without tls verification for testing purpose.
I tried with:
docker-machine create -d virtualbox --engine-tls false node1
and
docker-machine create -d virtualbox --engine-tls-verify false node1
and
docker-machine create -d virtualbox --engine-opt-tls false node1
I use commands below:
docker-machine create -d virtualbox --engine-env DOCKER_TLS=no node1
And then ssh to the node to execute docker commands:
docker-machine ssh node1
$ docker info
try:
docker-machine create -d virtualbox --engine-opt tlsverify=false node1
and after running:
eval "$(docker-machine env node1)"
run:
unset DOCKER_TLS_VERIFY
This worked best for me:
docker-machine create -d virtualbox --engine-env DOCKER_TLS=no --engine-opt host=tcp://0.0.0.0:2375 node1
This way it binds to 2375 in addition to 2376. 2375 is the tradition for non-tls daemons.
Related
I was trying to set up a cluster with docker swarm. However, I am a bit confused about how is docker-machine with swarm options different from initialising a swarm manager on one host and joining as workers from other hosts.
Here is an example for my question:
docker-machine with swarm options
docker-machine create --driver virtualbox --swarm --swarm-master --virtualbox-hostonly-cidr "10.0.0.1/24" node1
docker-machine create --driver virtualbox --swarm --swarm-discovery "token://..." --virtualbox-hostonly-cidr "10.0.0.1/24" node2
docker-machine create --driver virtualbox --swarm --swarm-discovery "token://..." --virtualbox-hostonly-cidr "10.0.0.1/24" node3
join a manager node as worker nodes
docker-machine create --driver virtualbox --virtualbox-hostonly-cidr "10.0.0.1/24" node1
docker-machine create --driver virtualbox --virtualbox-hostonly-cidr "10.0.0.1/24" node2
docker-machine create --driver virtualbox --virtualbox-hostonly-cidr "10.0.0.1/24" node3
eval $(docker-machine env node1)
docker swarm init
eval $(docker-machine env node2)
docker swarm join --token <token> <node1_IP>
eval $(docker-machine env node3)
docker swarm join --token <token> <node1_IP>
The first method you used is from old docker version when SWARM need to have a discovery key/value store setup
docker-machine create --driver virtualbox --swarm --swarm-master --virtualbox-hostonly-cidr "10.0.0.1/24" node1
docker-machine create --driver virtualbox --swarm --swarm-discovery "token://..." --virtualbox-hostonly-cidr "10.0.0.1/24" node2
docker-machine create --driver virtualbox --swarm --swarm-discovery "token://..." --virtualbox-hostonly-cidr "10.0.0.1/24" node3
The second method you had used is the Swarm mode
To use Docker in swarm mode, install Docker 1.12.0 or later
This was introduced in Docker 1.12.0. This is the method you should be using now as all new commands like docker service, docker stack require swarm mode. They won't work on the old one
I create sucsessfully a swarm, with two nodes. However when I use docker-compose build && docker-compose up in order to start my project it crashes erroring out this:
ERROR: Error response from daemon: datastore for scope "global" is not initialized
It's a very very simple process:
docker run swarm create
swarm hash:
1477bcd7778d083e02a80c352d4f1b87
docker-machine create -d virtualbox --swarm --swarm-master --swarm-discovery token://1477bcd7778d083e02a80c352d4f1b87 myswarmmaster
docker-machine create -d virtualbox --swarm --swarm-discovery token://1477bcd7778d083e02a80c352d4f1b87 myremotenode1
eval $(docker-machine env --swarm myswarmmaster)
docker-compose build && docker-compose up
And then I get the error:
ERROR: Error response from daemon: datastore for scope "global" is not initialized
I'm running docker on Fedora 25.
I had the same error when I did docker swarm init on an Ubuntu machine. What I found was that swarm tries to access port 2377 so first open up the port 2377 sudo ufw allow 2377
And now docker swarm init worked and showed a message like this
Swarm initialized: current node (sdf23fsd3f24fr3f2f) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join \
--token SW3Wwww-1-0dfsdffsdfdsfsdfdsfdfdsfdsf-dsfsdfdsfdsfdsfd \
52.15.91.31:2377
The key is make sure that appropriate ports are available.
Hope this helps
I follow these steps to create Docker swarm cluster.
First: Create Cunsol
docker-machine create -d virtualbox mh-keystore
eval "$(docker-machine env mh-keystore)"
docker run -d \
-p "8500:8500" \
-h "consul" \
progrium/consul -server -bootstrap
Second: Create swarm manager
docker-machine create -d virtualbox node1
docker run -d -p 4000:4000 swarm manage -H :4000 --replication -- advertise $(docker-machine ip node1):4000 consul://$(docker-machine ip mh-keystore):8500
Third: Create swarm node
docker-machine create -d virtualbox node2
docker run -d swarm join --advertise=$(docker-machine ip node2):2375 consul://$(docker-machine ip mh-keystore):8500
Fourth: Login to node1
docker-machine ssh node1
docker -H :4000 info
But this instruction output
(unknown): 192.168.99.106:2375(node2 ip)
└ ID:
└ Status: Pending
└ Containers: 0
└ Reserved CPUs: 0 / 0
└ Reserved Memory: 0 B / 0 B
└ Labels:
└ Error: Cannot connect to the Docker daemon. Is the docker daemon running on this host?....
How can I fix this ?
I have already checked node2 and it runs well.
[Update] I follow this page and it works well. But I still wan't to know how set up swarm cluster without docker-machine.
[Update] Another approach doen't work either.
docker-machine create -d virtualbox \
--swarm \
--swarm-discovery="consul://$(docker-machine ip mh-keystore):8500" \
--engine-opt="cluster-store=consul://$(docker-machine ip mh-keystore):8500" \
--engine-opt="cluster-advertise=eth1:2376" \
mhs-demo1
Node1 docker info appear mhs-demo1 ip but info still unknown..
[Update]
When I type eval docker-machine env --swarm node1 It shows
Error checking TLS connection: "node1" is not a swarm master. The
--swarm flag is intended for use with swarm masters Does this cause error ? Why using swarm manager instruction to set up is not swarm
master?
It's so strange. How can I get the same result as
docker-machine create \ -d virtualbox \ --swarm --swarm-master \
--swarm-discovery="consul://$(docker-machine ip mh-keystore):8500" \
--engine-opt="cluster-store=consul://$(docker-machine ip mh-keystore):8500" \
--engine-opt="cluster-advertise=eth1:2376" \
mhs-demo0
using swarm instruction?
I want to use swarm instruction because I don't want to declare swarm master when I create it.
Why are you using docker-machine just to start a node? You can use docker machine to setup your node with swarm ready to go..
You can follow this tutorial
https://docs.docker.com/engine/userguide/networking/get-started-overlay/
Try deleting this file with:
sudo rm /etc/docker/key.json
Then restart docker with:
sudo service docker restart
At this point docker will make a new key.json file and your master should be able to find your workers. This happens sometimes when you use the same image for all your worker nodes, but its an easy fix.
In docker 1.12 swarm mode is directly available. There is no need for a key value store for the cluster.
just follow the this : https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/
I am building my docker swarm cluster in a sandbox.
I have 1 zookeeper on a machine for discovery, 1 swarm master and 2 swarm nodes.
I try to connect them but when I try to run my docker run commands on the swarm master, it does not distribute the work to the nodes.
Also when I do docker info on the swarm master I can see that the nodes are not connected.
I do not know what I am doing wrong.
Here are the step to reproduce my problem:
I have an empty pwd/data folder and a pwd/config folder with my zoo.cfg:
tickTime=2000
dataDir=/tmp/zookeeper
clientPort=2181
initLimit=5
-
#---- CREATE ZOO ---
docker-machine create --driver virtualbox zoo1
docker-machine start zoo1
eval $(docker-machine env zoo1)
docker pull jplock/zookeeper
docker run -p 2181:2181 -v `pwd`/conf:/opt/zookeeper/conf -v `pwd`/data:/tmp/zookeeper jplock/zookeeper
docker-machine ip zoo1 #############192.168.99.100
-
#--- CREATE CLUSTER ---
docker-machine create --driver virtualbox --swarm --swarm-master machine-smaster
docker-machine create --driver virtualbox --swarm machine-s01
docker-machine create --driver virtualbox --swarm machine-s02
-
eval "$(docker-machine env machine-smaster)"
docker run -p 2375:2375 -d -t swarm manage -H 0.0.0.0:2375 --advertise $(docker-machine ip machine-smaster):2375 zk://192.168.99.100:2181/swarm
docker run swarm list zk://192.168.99.100:2181/swarm
sleep 10
eval "$(docker-machine env machine-s01)"
docker run -d swarm join --advertise $(docker-machine ip machine-s01):2375 zk://192.168.99.100:2181/swarm
docker run swarm list zk://192.168.99.100:2181/swarm
eval "$(docker-machine env machine-s02)"
docker run -d swarm join --advertise $(docker-machine ip machine-s02):2375 zk://192.168.99.100:2181/swarm
docker run swarm list zk://192.168.99.100:2181/swarm
If I run some containers:
eval "$(docker-machine env machine-smaster)"
docker run hello-world
The work is not dispatched to nodes (it is run by the master).
If I run docker info:
eval "$(docker-machine env machine-smaster)"
docker info
I do not see the swarm nodes.
Can you verify that the addresses you're advertising are actually reachable from the manager instance? i.e., does docker -H $(docker-machine ip machine-s01):2375 info return a valid result?
(Note that this subshell won't work inside the manager VM, just on your original client.)
Maybe your problem is that the started Docker Machine instances are listening on :2376 with TLS, but your started Swarm containers are trying to advertise and connect to :2375 without any TLS settings specified?
What do the docker logs for the Swarm containers say?
It looks like you're connecting to the "Swarm master" machine through the Docker API, not the Swarm API. Because of this, Docker will always deploy containers on the host you're connected to, and does not take advantage of Swarm scheduling the containers on the right host.
To connect to the Swarm API, add the --swarm option when running docker-machine env, so in your case:
eval "$(docker-machine env --swarm machine-smaster)"
After update to docker-machine, I have no idea how to update DOCKER_OPTS in a local boot2docker docker-machine and save it permanently
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM
dev * virtualbox Running tcp://192.168.99.100:2376
I tried to ssh into vm with docker-machine ssh dev, and update /etc/docker/default. But the changes I made are dropped after restart VM with docker-machine restart dev.
If you'd simply like to pass arguments to docker, add them to EXTRA_ARGS in /var/lib/boot2docker/profile. For example:
EXTRA_ARGS='
--label provider=virtualbox
--insecure-registry=10.0.0.1:5000
'
The docker process is started via /etc/init.d/docker which sources /var/lib/boot2docker/profile. $EXTRA_ARGS is passed transparently to the docker process towards the end of start().
As of docker-machine 0.5.0, the mirror can be provisioned with --engine-registry-mirror
docker-machine create -d virtualbox --engine-registry-mirror http://mirror.dockerhub.com dev
--engine-registry-mirror [--engine-registry-mirror option --engine-registry-mirror option] Specify registry mirrors to use