I have Docker Desktop installed on my Mac (not Docker Toolkit) and I installed docker-machine according to the official documentation
I'm triying to add my localhost Docker engine like a docker node under docker machine with no success.
The steps that I made were:
Enable sshd in localhost (ssh localhost works)
Add localhost Docker to Docker Machine:
docker-machine create --driver generic --generic-ip-address 127.0.0.1 --generic-ssh-user <"ssh_username"> <node_name>
Running pre-create checks...
Creating machine...
(localhost) No SSH key specified. Assuming an existing key at the default location.
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Password:
Detecting the provisioner...
Password:
Error creating machine: Error detecting OS: Error getting SSH command: ssh command error:
command : cat /etc/os-release
err : exit status 1
output : cat: /etc/os-release: No such file or directory
Output of docker-machine ls
docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
localhost - generic Running tcp://127.0.0.1:2376 Unknown Unable to query docker version: Cannot connect to the docker engine endpoint
Sorry for my English, I'm not native.
docker-machine is dangerous. I wouldn't recommend it for managing production servers as it requires passwordless sudo and makes it very easy to damage your Docker installation. I managed to completely remove all containers an images from a server, not realizing the command I ran was not merely connecting to the server, but initializing it from scratch.
If you want to control multiple Docker daemons from single CLI try Docker Contexts.
Edit:
docker-machine's purpose is provisioning and managing machines with Docker daemon.
It can be used both with local VM's and with various cloud providers. With a single command it can create and start a VM, then install and configure Docker on that new VM (including generating TLS certificates).
It can create an entire Docker Swarm cluster.
It can also install Docker on a physical machine, given SSH access with passwordless sudo (that is what generic driver you tried to use is for).
Once a machine is fully provisioned with Docker it also can set environment variables that configure Docker CLI to send commands to a remote Docker daemon installed on that machine - see here for details.
Finally, one can also add machines with Docker manually configured by not using any driver - as described here. The only purpose of that is to allow for a unified workflow when switching between various remote machines.
However, as I stated before docker-machine is dangerous - it can also remove existing VMs and in case of physical machines reprovsion them, thereby removing all existing images, containers, etc. A simple mistake can wipe a server clean. Not to mention it requires both key-based SSH and passwordless sudo, so if an unauthorized person gets their hands on an SSH key for a production server, then that's it - they have full root access to everything.
It is possible to use docker-machine with preexisting Docker installations safely - you need to add them without using any driver as described here. In this scenario, however, most docker-machine commands won't work, so the only benefit is easy generation of those environment variables for Docker CLI I mentioned before.
Docker Contexts are a new way of telling Docker CLI which Docker daemon it's supposed to communicate with. They essentially are meant to replace all those environment variables docker-machine generates.
Since Docker CLI only communicates with Docker daemon, there is no risk of accidentally deleting a VM or reprovisioning already configured physical machine. And since they are a part of Docker CLI, there is no need to install additional software.
On the other hand, Docker contexts cannot be used to create or provision new machines - one needs to either do that manually or use some other mechanism or tool (like Vagrant or some kind of template provided by the cloud provider).
So if you really need a tool that'll let you easily create, provision and remove docker-enabled machines then use docker-machine. If, however, all you wan is to have a list of all your Docker-enabled machines in one place and a way to easily set up which one your local Docker CLI is supposed to talk to, Docker Contexts are a much safer alternative.
Related
can i run docker socket over ssh?
i'm trying to run unix:///var/run/docker.sock but i'm getting the error "Is daemon service running?, Cannnot connect to daemon service"
Jenkins master and the ubuntu machine a very isolated they might as well just be on different machines not even in the same room. Unix domain sockets, the ones that are identified by unix://* are made for communicating within a single local OS kernel, trying to bridge them into remote machine will lead to disaster.
how can i use Docker sock over ssh?
stephen proposed a solution but i find this one more adequate to your use case.
you can simply use
ssh xxx "docker run yyy"
or you can use env variables :
be sure that you have ssh key authentification active
and call all your docker commands with this env variable defined :
DOCKER_HOST=remoteservername
docker will use ssh connection to run commands
you can also use -H (works the same)
see more here
https://betterprogramming.pub/docker-tips-access-the-docker-daemon-via-ssh-97cd6b44a53
When I run my docker container using Docker Desktop for Windows I am able to connect to it using
docker run -p 5051:5000 my_app
http://0.0.0.0:5051
However when I open another terminal and do this
minikube docker-env | Invoke-Expression
and build and run the same container using the same run command as above
I cannot connect to the running instance.
Should I be running and testing the containers using Docker Desktop, then using minikube to store the images only (for Kubernetes)? Or can you run them and test them as well through minikube?
That's because on your second attempt, the container is not running on the host but on the minikube VM. You'll be able to access it using the minikube VM IP.
To get the minikube ip you can run minikube ip
Why ?
Invoking minikube docker-env sets all the docker env variable on your host to match the minikube environment. This means that when you run a container after that, it is run with the docker daemon on the minikube VM.
I asked you if there are any specific reasons to use Docker Desktop and Minikube together on a single machine as these are two competitive solutions which basically enable you to perform similar tasks and achieve same goals.
This article nicely explains differences between these two tools.
Docker-for-windows uses Type-1 hypervisor, such as Hyper-V, which are
better compared to Type-2 hypervisors, such as VirtualBox, while
Minikube supports both hypervisors. Unfortunately, there are a couple
of limitations in which technology you are using, since you cannot
have Type-1 or Type-2 hypervisors running at the same time on your
machine
If you use Docker Desktop and Minikube at the same time I assume you're using Type-1 hypervisor, such as mentioned Hyper-V, but keep in mind that even if they use the same hypervisor, both tools create their own instances of virtual machine. Basically you are not supposed to use those two tools together expecting that they will work as a kind of hybrid that lets you manage single container environment.
First check what hypervisor you are using exactly. If you're using Hyper-V, simple Get-VM command in Powershell (more details in this article) should tell you what you currently have.
#mario no, I didn't know minikube had a docker daemon until recently
which is why I have both
Yes, Minikube has built in docker environment (in fact it sets everything up, but yes, it also sets up container runtime) so basically you don't need to install docker additionally, and as #Marc ABOUCHACRA already suggested in his answer, Minikube runs the whole environment (single node k8s cluster with docker runtime) on a separate VM. Linux version has an option --vm-driver=none which allows you to use your host container runtime and set-up k8s components on it, but this is not the case with Windows version - here you can only use one of two currently supported hypervisors: Hyper-V or VirtualBox (ref).
I wouldn't say that Docker Destkop runs everything on your host. It also uses Type-1 hypervisor to run the container runtime environment. Please check the Get-VM command on your computer and it should be clear what VMs you have and created by which tool.
I'm getting in habit with rancher and docker and I'm now trying to figure out if it is possible to create multiple local custom hosts on the same physical machine. I'm running RancherOS in a local computer. Through the Rancher Web UI I'm able to create a local custom host and add containers to it.
When I try to add another local custom host copying the given command to the terminal (SSH into the rancher machine) it stars the process but nothing happen. The new host doesn't appear in the hosts list of the web interface and I don't receive any error from the terminal.
I couldn't get any useful information from the Rancher documentation about this possible issue.
I was wondering if it's not possible to have more than one custom virtual host on the same physical machine or if the command fails for some reason that I would like to know how to debug.
sudo docker run -e -d --privileged \
-v /var/run/docker.sock:/var/run/docker.sock rancher/agent:v0.8.2 \
http://192.168.1.150:8080/v1/projects/1a5/scripts/<registrationToken>
where registrationToken is replaced by the one provided by rancher.
There is nothing "virtual" about them. The agent talks to docker and manages one docker daemon, which is the entire machine. Running multiple does not make sense for a variety of reasons, such as when you type "docker run ..." on the machine, which agent is supposed to pick up that container? And they are not really isolated from each other regardless, because any of them can run privileged containers which can then do whatever they want that affects the others.
The only way to do what you're asking is to have actual virtual machines running on the physical machine, each with their own OS and docker daemon.
Another option might be to use linux containers to create separated environments, each having it's own ip address and running it's own docker daemon.
I have a linux on cloud with a installed docker service on it. How can I use my VS on cloud instead of docker-machine on my OSX? it means instead of install VirtualBox and create a VM on it by docker-machine, I use my server on cloud as docker server.
To access a remote Docker daemon simply pass the -H flag to your docker commands:
docker -H=tcp://192.168.0.100:2375 images
You need to ensure that the remote Docker daemon is listening on the appropriate network interface. Be aware though that doing this on an external server is highly insecure, anyone that can reach the port has effectively root access on the server. At the very least read this article on securing the Docker daemon.
Personally I would only recommend using a port binding via ssh tunnel to access the remote Docker daemon.
You might get a solution from docker-machine's generic driver. Just start the virtual server in cloud, set up proper SSH keys and get started :) It should work just the same as with a VM within VirtualBox.
I'm not sure how to get VS auto-started if it is shut down though. Via a could-vendor specific command line program?
Edit: I should have read the docs better, the first cloud example actually shows the usage of digital ocean driver. If it is already running then just use the generic driver.
I'm new to a docker and tried to create docker host with docker-machine.
Currently, I use VirutalBox for trial environment.
When I created docker host with docker-mahine, it created VM with Boot2Docker on VirtualBox by default. But I want to create a docker host with Ubuntu 15.10 on Virtualbox.
Is it possible to use docker-machine for creating Ubuntu based docker host on VirtualBox?
OP didn't describe how they used the generic driver to solve their problem, so here's how I did it in case anyone's interested:
Get Ubuntu Server ISO
Create a machine in VirtualBox. I called mine "Ubuntu template" because I want to learn Swarm locally, so I want a machine that I'll be able to duplicate and get subsequent machines quickly after the longer initial setup.
Enable bridged networking instead of NAT for the machine in the settings
Start the machine and install Ubuntu using the ISO. During installation it'll give you an option to install OpenSSH, select that option. It'll also ask you to create a new user. I called mine "ubuntu" with password "ubuntu". You'll use this user a few times, so set the credentials to something easy to remember
After installation, switch to root: sudo su
Change root's password to something easy to remember using passwd
Generate keys: ssh-keygen
Make the keys you just generated authorized: cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys
Edit the file /etc/ssh/sshd_config and change the line with "PermitRootLogin" so it reads PermitRootLogin yes
Restart SSH to activate the changes: service ssh restart
Run ifconfig and take note of the machine's IP
Open terminal on your host computer
Run (with your machine's IP substituted):
ssh root#10.10.10.89 'cat ~/.ssh/id_rsa' > ~/.ssh/docker_test
Run:
ssh root#10.10.10.89 'cat ~/.ssh/id_rsa.pub' > ~/.ssh/docker_test.pub
Run (back in the VM) shutdown now
In VirtualBox, clone the template machine (check the checkbox to reinitialize MAC address). I named mine ubuntu-1
Start the new virtual machine and run echo 'ubuntu-1' > /etc/hostname and then reboot. That's only necessary if you're going to create more machines from the same template, then you'd name them ubuntu-1, ubuntu-2 and so on
Run ifconfig to find out the IP of the cloned machine
On your host machine run:
docker-machine create --driver generic --generic-ip-address 10.10.10.90 --generic-ssh-key ~/.ssh/docker_test ubuntu-1
It might take a few minutes to complete (mostly on the "Installing docker" step) but you should then have a working Ubuntu-based docker machine. You can verify that it works by running docker-machine use ubuntu-1 and then docker run hello-world
It's more involved than using Boot2Docker, but after the initial setup it should be quite workable. I haven't done too much with it yet, I just verified that it seems to work by running hello-world, so there might be more gotchas down the road like there often are with Docker.
Extra tip: VirtualBox allows you to run machines in headless mode. After the initial setup and allowing root access via SSH it'll probably be more convenient to run the machines headless and connect to them via SSH if necessary and you can close VB's GUI and the machines are now running like services in the background.
Is it possible to use docker-machine for creating Ubuntu based docker host on VirtualBox?
Yes, but not with docker-machine directly, which relies on a TinyCore-based linux distribution of 30 Mo only.
You can try and launch a full-fledge Ubuntu VM, and in it follows the regular docker installation for Ubuntu.