I have two hosts HostA and HostB which currently uses memcached installed on HostA. I am containerizing memcached on HostA and the services that uses this container from HostA can connect to it by using hostname as 'memcached' and port '11211'. What is the hostname that I should use when I connect to this container from HostB? Should I use HostA's IP address and port 11211?
One option would be to use the HostA's IP Address and port 11211, as you mentioned, as long as the port is exposed and firewall rules set appropriately.
Another option is to create an overlay network and keep using the name of the container (e.g. memcached). It's a bit more difficult to set up and requires an extra piece of software for service discovery (e.g. Consul, ZK, etcd). On the bright side, it also makes the whole thing more dynamic. A good reference in this blog post.
I have a Python 3 application deployed in Google App Engine, flexible environment.
I'm using psycopg2 to connect to a PostgreSQL instance hosted in Google cloud SQL.
I'm having trouble connecting to PostgreSQL from Google App Engine.
Cloud SQL Proxy seems to initialize ok, but it binds to 0.0.0.0
Listening on 0.0.0.0:5432 for projectID:us-central1:my-db
Trying to connect on 127.0.0.1 or localhost doesn't work. Connection is refused.
What does work is using the docker (app engine flexible environment uses docker underneath) default IP 172.17.0.1 (from the docker0 adapter)
Using that IP address to connect to Cloud SQL seems like it would bite me in the ass if someone decides to change it.
Why is this happening?
Is using the default docker0 adapter's IP address a viable long term solution?
Is there an alternative other than switching to a socket based connection instead of the tcp approach.
It sounds like you are running the Cloud SQL proxy on your host machine, while you are attempting to run your application from inside a container. The reason it can't connect to the proxy is because 127.0.0.1 refers to docker's loopback interface, while the proxy is bound to the host machine's interface. The 172.17.0.1 is the address the container can use to can reach the host interface.
One alternative is to use host networking (https://docs.docker.com/network/host/), by passing in --network host. This will cause the host's interface to be used for the application.
I've switched from using TCP as the connection method and to using a Unix Socket.
The TCP issue seems to be a bug in the app engine flexible environment. But it's a beta feature (it is under the name beta_settings in app.yaml) and I'm not holding out for Google to fix it.
I also don't want to commit to an IP address that could be changed sometime in the future as a workaround.
I'm trying to make a docker machine available to my Windows by a host name. After creating it like
docker-machine create -d virtualbox mymachine
and setting up a docker container that exposes the port 80, how can I give that docker machine a host name such that I can enter "http://mymachine/" into my browser to load the website? When I change "mymachine" to the actual IP address then it works.
There is an answer to this question but I would like to achieve it without an entry in the hosts file. Is that possible?
You might want to refer to docker documentaion:
https://docs.docker.com/engine/userguide/networking/#exposing-and-publishing-ports
You expose ports using the EXPOSE keyword in the Dockerfile or the
--expose flag to docker run. Exposing ports is a way of documenting which ports are used, but does not actually map or open any ports.
Exposing ports is optional.
You publish ports using the --publish or --publish-all flag to docker
run. This tells Docker which ports to open on the container’s network
interface. When a port is published, it is mapped to an available
high-order port (higher than 30000) on the host machine, unless you
specify the port to map to on the host machine at runtime. You cannot
specify the port to map to on the host machine when you build the
image (in the Dockerfile), because there is no way to guarantee that
the port will be available on the host machine where you run the
image.
I also suggest reviewing the -P flag as it differs from the -p one.
Also i suggest you try "Kitematic" for Windows or Mac, https://kitematic.com/ . It's much simpler (but dont forget to commit after any changes!)
Now concerning the network in your company, it has nothing to do with docker, as long as you're using docker locally on your computer it wont matter what configuration your company set. Even you dont have to change any VM network config in order to expose things to your local host, all comes by default if you're using Vbox ( adapter 1 ==> NAT & adapter 2 ==> host only )
hope this is what you're looking for
If the goal is to keep it as simple as possible for multiple developers, localhost will be your best bet. As long as the ports you're exposing and publishing are available on host, you can just use http://localhost in the browser. If it's a port other than 80/443, just append it like http://localhost:8080.
If you really don't want to go the /etc/hosts or localhost route, you could also purchase a domain and have it route to 127.0.0.1. This article lays out the details a little bit more.
Example:
dave-mbp:~ dave$ traceroute yoogle.com
traceroute to yoogle.com (127.0.0.1), 64 hops max, 52 byte packets
1 localhost (127.0.0.1) 0.742 ms 0.056 ms 0.046 ms
Alternatively, if you don't want to purchase your own domain and all developers are on the same network and you are able to control DHCP/DNS, you can setup your own DNS server to include a private route back to 127.0.0.1. Similar concept to the Public DNS option, but a little more brittle since you might allow your devs to work remote, outside of a controlled network.
Connecting by hostname requires that you go through hostname to IP resolution. That's handled by the hosts file and falls back to DNS. This all happens before you ever touch the docker container, and docker machine itself does not have any external hooks to go out and configure your hosts file or DNS servers.
With newer versions of Docker on windows, you run containers with HyperV and networking automatically maps ports to localhost so you can connect to http://localhost. This won't work with docker-machine since it's spinning up virtualbox VM's without the localhost mapping.
If you don't want to configure your hosts file, DNS, and can't use a newer version of docker, you're left with connecting by IP. What you can do is use a free wildcard DNS service like http://xip.io/ that maps any name you want, along with your IP address, back to that same IP address. This lets you use things like a hostname based reverse proxy to connect to multiple containers inside of docker behind the same port.
One last option is to run your docker host VM with a static IP. Docker-machine doesn't support this directly yet, so you can either rely on luck to keep the same IP from a given range, or use another tool like Vagrant to spin up the docker host VM with a static IP on the laptop. Once you have a static IP, you can modify the host file once, create a DNS entry for every dev, or use the same xip.io URL, to access the containers each time.
If you're on a machine with Multicasting DNS (that's Bonjour on a Mac), then the approach that's worked for me is to fire up an Avahi container in the Docker Machine vbox. This lets me refer to VM services at <docker-machine-vm-name>.local. No editing /etc/hosts, no crazy networking settings.
I use different Virtualbox VMs for different projects for my work, which keeps a nice separation of concerns (prevents port collisions, lets me blow away all the containers and images without affecting my other projects, etc.)
Using docker-compose, I just put an Avahi instance at the top of each project:
version: '2'
services:
avahi:
image: 'enernoclabs/avahi:latest'
network_mode: 'host'
Then if I run a webserver in the VM with a docker container forwarding to port 80, it's just http://machine-name.local in the browser.
You can add a domain name entry in your hosts file :
X.X.X.X mymachine # Replace X.X.X.X by the IP of your docker machine
You could also set up a DNS server on your local network if your app is meant to be reachable from your coworkers at your workplace and if your windows machine is meant to remain up as a server.
that would require to make your VM accessible from local network though, but port forwarding could then be a simple solution if your app is the only webservice running on your windows host. (Note that you could as well set up a linux server to avoid using docker-machine on windows, but you would still have to set up a static IP for this server to ensure that your domain name resolution works).
You could also buy your own domain name (or get a free one) and assign it your docker-machine's IP if you don't have rights to write in your hosts file.
But these solution may not work anymore after some time if app host doesn't have a static IP and if your docker-machine IP changes). Not setting up a static IP doesn't imply it will automatically change though, there should be some persistence if you don't erase the machine to create a new one, but that wouldn't be guaranteed either.
Also note that if you set up a DNS server, you'd have to host it on a device with a static IP as well. Your coworkers would then have to configure their machine to use this one.
I suggest nginx-proxy. This is what I use all the time. It comes in especially handy when you are running different containers that are all supposed to answer to the same port (e.g. multiple web-services).
nginx-proxy runs seperately from your service and listens to docker-events to update it's own configuration. After you spun up your service and query the port nginx-proxy is listening to, you will be redirected to your service. Therefore you either need to start nginx-proxy with the DEFAULT_HOST flag or send the desired host as header param with the request.
As I am running this only with plain docker, I don't know if it works with docker-machine, though.
If you go for this option, you can decide for a certain domain (e.g. .docker) to be completely resolved to localhost. This can be either done company-wide by DNS, locally with hosts file or an intermediate resolver (the specific solution depends on your OS, of course). If you then try to reach http://service1.docker nginx-proxy will route to the container that has then ENV VIRTUAL_HOST=service1.docker. This is really convenient, because it only needs one-time setup and is from then on dynamic.
I've got the following setup:
OSX running MySQL listening on all network adaptors at port 3306
XDEBUG enabled IDE listening on port 9000 on the base OSX system.
docker-machine host running on the OSX system with the host ip 192.168.99.100
A debian based docker container with a mysql client running on the docker host and HHVM running with xdebug looking to connect to some lucky remote host on port 9000.
The ip addresses change frequently on the OSX system due to being assigned via DHCP, so I want the docker container to be able to somehow be able to hit the mysql server regardless of what IP the native OSX network adaptors get assigned (without manually updating it). Also, I need a stable ip I can provide my HHVM server.ini file a remotes host for Xdebug.
With running a base system of linux this isn't an issue as the docker host and the actual native machine running docker are one-and-the-same. Also, there are several ways for a container to learn of the host's ip so the issue isn't hitting the docker host.
However, in OSX running docker-machine, the host ain't the native OSX system, but instead is a VM running in virtual box (assuming you're using the vb driver, and who the sam hill blazes isn't?).
The only thing I could think of was to port forward request on 3306 to the docker-machine host (192.168.99.100 which never changes) to the OSX's port 3306. Then have the container hit the docker-machine host for Mysql requests. IF this works, I could rinse and repeat for any port I port I need to link like xdebug on port 9000.
Does anyone know how to accomplish this or have another suggestion?
Figured a way out without needing to make any changes that provides a consistent ip to connect to on the base OSX system. Docker machine sets things up in such a way to make this possible.
Docker machine creates a virtualbox VM with 2 network adaptors, one set up as host-only, the other set as NAT. Don't know why it creates 2, but
The host-only adaptor provides the OSX an ip of 192.168.99.1 and the various VM's using it get addresses starting with 192.168.99.100. However, inside the VM network, you can't use the address 192.168.99.1 to hit ports on the parent OSX system (not sure why, but guessing host only intends to be only communication between the VMs).
The NAT network adaptor is set so the OSX get's the ip 10.0.2.2 and the VM get's 10.0.2.15. With a NAT, you can route to the OSX system at 10.0.2.2 from both the docker host VM and containers running on the host.
Since this 10.0.2.2 address for the OSX machine doesn't change (unless you screw with the virtual box networking settings) bingo, got what I need.
I am running a Debian VMware virtual machine on Windows7. I am developing a Rails web app on the virtual machine, I would like to access the app (localhost:3000) from outside the vm, i.e from windows.
Is there a way to do that?
Thanks.
This really depends on how your virtual network adapters are setup between the guest and host. Assuming you have not changed anything, they will be bridged by default. In your VM from a terminal run:
ifconfig
This will give you the IPv4 address (generally a 10.x.x.x) of the virtual interface which you should be able to access from your Windows host. You should be able to put that into your browser on the host with the :3000 port and access it.
If you have modified the network adapters, things may be different. For instance, setting it up as a NAT will give it an address on your physical network; i.e. 192.x.x.x, but accessing it the same. A host only netowork, would also most likely provide a 10.x.x.x address, but should also provide access.