When I use Cisco VPN Anyconnect to join to my corporate network, I cannot get docker-machine to connect to my virtualbox VM. It has something to do with Cisco Anyconnect taking over all 192.168.. routes. I also tried using a totally different cidr range (25.0.1.100/24) but still cannot get docker-machine to talk to VM. When I check the routes table, route gets added to utun0 instead of vboxnet0. I'm assuming utun0 is VPN's host network interface. Here the docker-machine output:
docker-machine create -d virtualbox dev
Running pre-create checks...
Creating machine...
Waiting for machine to be running, this may take a few minutes...
Machine is running, waiting for SSH to be available...
Detecting operating system of created instance...
Provisioning created instance...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
WARNING >>>
This machine has been allocated an IP address, but Docker Machine could not
reach it successfully.
SSH for the machine should still work, but connecting to exposed ports, such as
the Docker daemon port (usually <ip>:2376), may not work properly.
You may need to add the route manually, or use another related workaround.
This could be due to a VPN, proxy, or host file configuration issue.
You also might want to clear any VirtualBox host only interfaces you are not using.
To see how to connect Docker to this machine, run: docker-machine env dev
I had a similar problem with IP conflicts on 192.168.x.x I solved it changing the subnet of the VirtualBox host-only network.
1) run docker-machine rm dev
2) Go into the VirtualBox preferences and remove the host-only network
3) run docker-machine create --driver virtualbox --virtualbox-hostonly-cidr "25.0.1.100/24" dev
There is also a discussion on Github here: https://github.com/docker/kitematic/issues/1029#issuecomment-156219462
I have the same issue and this post on docker at github.com solves it.
sudo ifconfig vboxnet0 down && sudo ifconfig vboxnet0 up
You also may want to use port 2377 as discussed here
If you have the option to run Cisco VPN in Split Tunnel (instead of Full Tunnel) mode, that seems to work well, while still allowing you to access your corporate network.
Related
I'm seeing problems mounting local volumes when running docker on Windows 10. The problems only appear when I have my company VPN enabled.
C:\Users\matt> docker run --rm -v d:/tmp:/data alpine ls /data
my_local_test_file.txt
When connected to VPN, I get this:
C:\Users\matt> docker run --rm -v d:/tmp:/data alpine ls /data
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: error while creating mount source path '/host_mnt/d/tmp': mkdir /host_mnt/d: file exists.
Docker version is 17.12.0-ce-win47
I believe the problem is that docker uses the network when mounting local volumes, and the VPN routes ALL network traffic via the VPN gateway, so docker can't see the local drive.
Is there a workaround for this?
I'm aware I could run docker within a linux VM, or use docker toolbox. Neither of those are particularly good.
Is there another possible workaround?
the VPN routes ALL network traffic via the VPN gateway
You're probably right, in which case all traffic routed from Docker client to Docker daemon will also be through the VPN. When you use Docker CLI on Windows, it will connect to the Docker daemon which is accessible through the network. Using a VPN may disrupt this mechanism.
I think what's happening is:
When VPN is disabled, you use the Docker daemon on your machine and everything works
When VPN is enabled, another Docker daemon is used either because your VPN redirect traffic addressed to your Docker host (127.0.0.1 by default or set via -H flag or DOCKER_HOST env variable). This means that somehow this IP or host exists on your VPN network and there is a Docker daemon listening on it (which is kind of odd admittedly, it may be risky to use that daemon)
If that's really happening, you'll certainly see different output from docker ps -a, docker images, etc. because you are connecting to different daemons. (the daemon accessible through your VPN is actually being owned by someone else, you'd better not use it!)
What you can do:
Do not route 127.0.0.1 (or whatever is configured as Docker host) through your VPN
Action to take will depend on the VPN software you are using, or you can add route directly on your windows machine (here is a good article on the subject)
Find out your IP when VPN is enabled and configure Daemon to listen to this IP
When your VPN is enabled, run ipconfig /all and find the interface used by your VPN and it's IP address, for example 10.142.0.12 (you can compare output before/after enabling VPN to identify which one it is)
Configure your Docker daemon to listen this IP address and restart it. Either use the UI, or on Windows config file is located at %programdata%\docker\config\daemon.json by default, you need to specify "hosts": ["10.142.0.12", "127.0.0.1"] for example (see docs for details)
Configure Docker host to 10.142.0.12 when VPN is enabled, either by setting environment variable DOCKER_HOST=10.142.0.12 or with client docker -H 10.142.0.12 <cmd>
/!\ Security note: this may present a security issue as anyone knowing your IP on the VPN network will be able to use the Daemon on your machine
Hope this helps. I am not a Windows expert so I was not able to give details on Windows-related issues, but feel free to ask details if needed.
I have an issue where I'm unable to start any docker machines whilst connected to a the WiFi network in my local Starbucks, receiving the following output;
$ docker-machine start
Starting "default"...
(default) Check network to re-create if needed...
Error setting up host only network on machine start: host-only cidr conflicts with the network address of a host interface
This does not happen when connected to my home network, or whilst using my mobile hotspot. Is there any workaround for this?
There is a collision between the docker machine and the network that is being set up for wifi. Try creating a new docker machine with other ip:
docker-machine create --driver virtualbox --virtualbox-hostonly-cidr "192.168.123.99/24" mymachine
Use it:
docker-machine env mymachine
This is a new machine in addition to the 'default' one. You might see that it won't have your previous work (images, etc).
Your docker-machine start failed.
So either you create new VM or repair existing VM.
For new VM option, use --virtualbox-hostonly-cidr "10.10.10.1/24"
(replace 10.10.10.1/24 with whatever subnet you want VM to use.)
For already created VM(start failed), bring up the virtual box UI and change it in the network preferences.
Use docker-machine ls to list existing VMs.
I have installed docker machine and it shows up running when I check using the docker-machine ls command. But it does not ends up creating a default VM on my virtual box.
The logs generated are as under:
Running pre-create checks...
Creating machine...
(default) Copying /root/.docker/machine/cache/boot2docker.iso to /root/.docker/machine/machines/default/boot2docker.iso...
(default) Creating VirtualBox VM...
(default) Creating SSH key...
(default) Starting the VM...
(default) Check network to re-create if needed...
(default) Waiting for an IP...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with boot2docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
This machine has been allocated an IP address, but Docker Machine could not
reach it successfully.
SSH for the machine should still work, but connecting to exposed ports, such as
the Docker daemon port (usually <ip>:2376), may not work properly.
You may need to add the route manually, or use another related workaround.
This could be due to a VPN, proxy, or host file configuration issue.
You also might want to clear any VirtualBox host only interfaces you are not using.
Checking connection to Docker...
Docker is up and running!
To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env default
root#varun:~# docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
default - virtualbox Running tcp://192.168.99.100:2376 v17.09.0-ce
What changes should I do so that the docker machine gets created as a VM on Virtual Box also?
Try creating the Hyper-V virtual switch and reference it
docker-machine create -d hyperv --hyperv-virtual-switch "NameOfVirtualSwitch" nameOfNode
Please refer the documenation.
<https://docs.docker.com/machine/drivers/hyper-v/#4-create-the-nodes-with-docker-machine-and-the-microsoft-hyper-v-driver>
There is some websource "http://vpnaccessible.com" where I need to download some RPM package via wget. And this web-source is accessible only from VPN. So I'm using Cisco AnyConnect VPN client to enter VPN, then I want to build image using Dockerfile where this wget command is listed.
The problem is: Docker can't access to that domain within container. So I tried to pass dns options in /etc/docker/daemon.json, but not sure what DNS IP I should pass, because in my local there are default DNS 192.168.0.1, 8.8.8.8. I tried to pass in that array IP addresses of docker0 interface, e.g. 172.17.0.1 -- didn't work.
$ cat /etc/docker/daemon.json
{
"insecure-registry": "http://my-insecure-registry.com",
"dns": ["192.168.0.1", "172.17.0.1", "8.8.8.8"]
}
I also tried to add this websource to /etc/resolf.conf but when I run docker to build image -- it's edited to the prev state (changes are not persisted there), and I guess, it's my Cisco VPN client behavior -- didn't work.
Also tried to add IP address of interface created by Cisco VPN client to that dns -- didn't work
I also commented out dns=dnsmasq in /etc/NetworkManager/NetworkManager.conf -- didnt work
For sure, I'm restarting docker and NetworkManager services after these changes.
Question: Should I create some bridge between Docker container and my VPN? How to solve this issue?
You can try using your host network instead of the default bridge one. Just add the following argument:
--network host
or
--net host
Depending of your docker version.
I've got the following setup:
OSX running MySQL listening on all network adaptors at port 3306
XDEBUG enabled IDE listening on port 9000 on the base OSX system.
docker-machine host running on the OSX system with the host ip 192.168.99.100
A debian based docker container with a mysql client running on the docker host and HHVM running with xdebug looking to connect to some lucky remote host on port 9000.
The ip addresses change frequently on the OSX system due to being assigned via DHCP, so I want the docker container to be able to somehow be able to hit the mysql server regardless of what IP the native OSX network adaptors get assigned (without manually updating it). Also, I need a stable ip I can provide my HHVM server.ini file a remotes host for Xdebug.
With running a base system of linux this isn't an issue as the docker host and the actual native machine running docker are one-and-the-same. Also, there are several ways for a container to learn of the host's ip so the issue isn't hitting the docker host.
However, in OSX running docker-machine, the host ain't the native OSX system, but instead is a VM running in virtual box (assuming you're using the vb driver, and who the sam hill blazes isn't?).
The only thing I could think of was to port forward request on 3306 to the docker-machine host (192.168.99.100 which never changes) to the OSX's port 3306. Then have the container hit the docker-machine host for Mysql requests. IF this works, I could rinse and repeat for any port I port I need to link like xdebug on port 9000.
Does anyone know how to accomplish this or have another suggestion?
Figured a way out without needing to make any changes that provides a consistent ip to connect to on the base OSX system. Docker machine sets things up in such a way to make this possible.
Docker machine creates a virtualbox VM with 2 network adaptors, one set up as host-only, the other set as NAT. Don't know why it creates 2, but
The host-only adaptor provides the OSX an ip of 192.168.99.1 and the various VM's using it get addresses starting with 192.168.99.100. However, inside the VM network, you can't use the address 192.168.99.1 to hit ports on the parent OSX system (not sure why, but guessing host only intends to be only communication between the VMs).
The NAT network adaptor is set so the OSX get's the ip 10.0.2.2 and the VM get's 10.0.2.15. With a NAT, you can route to the OSX system at 10.0.2.2 from both the docker host VM and containers running on the host.
Since this 10.0.2.2 address for the OSX machine doesn't change (unless you screw with the virtual box networking settings) bingo, got what I need.