I have docker container (lets say A) running on a host (lets say X). I would like the container to be able to have outbound connectivity to a DNS (ie ELB) that host X already has access to.
ie
container A ---> want to be able to curl an external ELB DNS endpoint...problem is container unable to resolve it!
host X ---> is able to curl the external ELB DNS endpoint already
Do either of these work for you?
$ docker run --dns 10.0.0.2 busybox nslookup google.com
Or edit your /etc/docker/daemon.json similar to:
{
"dns": ["10.0.0.2", "8.8.8.8"]
}
and restart the docker service
$ sudo systemctl docker restart
For more information, please see Fix Docker's networking DNS config and Container networking.
The symptom is: the host machine has proper network access, but programs running within containers can't resolve DNS names (which may appear to be "can't access the network" before investigating more).
$ sudo docker run -ti mmoy/ubuntu-netutils /bin/bash
root#082bd4ead733:/# ping www.example.com
... nothing happens (timeout) ... ^C
root#082bd4ead733:/# host www.example.com
... nothing happens (timeout) ... ^C
(The docker image mmoy/ubuntu-netutils is a simple image based on Ubuntu with ping and host included, convenient here since the network is broken and we can't apt install these tools)
The issue comes from the fact that docker automatically configured Google's public DNS as DNS server within the container:
root#082bd4ead733:/# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
nameserver 8.8.4.4
This just works in many configurations, but obviously doesn't when the host runs on a network where Google's public DNS are filtered by some firewall rules.
The reason this happened is:
Docker first tries configuring the same DNS server(s) on the host and within the container.
The host runs dnsmasq, a DNS caching service. dnsmasq acts as a proxy for DNS requests, hence the apparent DNS server in the host's /etc/resolve.conf is nameserver 127.0.1.1, i.e. localhost.
The host's dnsmasq listens only for requests comming from localhost and blocks requests coming from the docker container.
Since using 127.0.1.1 within docker doesn't work, docker falls back to Google's public DNS, which do not work either.
There may be several reasons why DNS is broken within docker containers. This question (and answers) covers the case where:
dnsmasq is used. To check whether this is the case:
Run ps -e | grep dnsmasq on the host. If the output is empty, you're not running dnsmasq.
Check the host's resolv.conf, it probably contains an entry like nameserver 127.0.1.1. If it contains nameserver 127.0.0.53, you're probably running systemd-resolved instead of dnsmasq. If so, you won't be able to use the solution forwading DNS requests to dnsmasq (the one using listen-address=172.17.0.1). systemd-resolved versions earlier than 247 hardcoded the fact that it listens only on the 'lo' interface hence there's no easy way to adapt this solution with these versions. Other answers below will work with systemd-resolved.
Google's public DNS is filtered. Run host www.example.com 8.8.8.8. If it fails or times out, then you are in this situation.
What are the solutions to get a proper DNS configuration in this configuration?
A clean solution is to configure docker+dnsmasq so than DNS requests from the docker container are forwarded to the dnsmasq daemon running on the host.
For that, you need to configure dnsmasq to listen to the network interface used by docker, by adding a file /etc/NetworkManager/dnsmasq.d/docker-bridge.conf:
$ cat /etc/NetworkManager/dnsmasq.d/docker-bridge.conf
listen-address=172.17.0.1
Then restart network manager to have the configuration file taken into account:
sudo service network-manager restart
Once this is done, you can add 172.17.0.1, i.e. the host's IP address from within docker, to the list of DNS servers. This can be done either using the command-line:
$ sudo docker run -ti --dns 172.17.0.1 mmoy/ubuntu-netutils bash
root#7805c7d153cc:/# ping www.example.com
PING www.example.com (93.184.216.34) 56(84) bytes of data.
64 bytes from 93.184.216.34: icmp_seq=1 ttl=54 time=86.6 ms
... or through docker's configuration file /etc/docker/daemon.json (create it if it doesn't exist):
$ cat /etc/docker/daemon.json
{
"dns": [
"172.17.0.1",
"8.8.8.8",
"8.8.4.4"
]
}
(this will fall back to Google's public DNS if dnsmasq fails)
You need to restart docker to have the configuration file taken into account:
sudo service docker restart
Then you can use docker as usual:
$ sudo docker run -ti mmoy/ubuntu-netutils bash
root#344a983908cb:/# ping www.example.com
PING www.example.com (93.184.216.34) 56(84) bytes of data.
64 bytes from 93.184.216.34: icmp_seq=1 ttl=54 time=86.3 ms
A brutal and unsafe solution is to avoid containerization of the network, and use the same network on the host and on the container. This is unsafe because this gives access to all the network resources of the host to the container, but if you do not need this isolation this may be acceptable.
To do so, just add --network host to the command-line, e.g.
$ sudo docker run -ti --network host mmoy/ubuntu-netutils /bin/bash
root#ubuntu1604:/# ping www.example.com
PING www.example.com (93.184.216.34) 56(84) bytes of data.
64 bytes from 93.184.216.34: icmp_seq=1 ttl=55 time=86.5 ms
64 bytes from 93.184.216.34: icmp_seq=2 ttl=55 time=86.5 ms
One way is to use a user defined network for your container. In that case the container's /etc/resolv.conf will have the nameserver 127.0.0.11 (a.k.a. the Docker's embedded DNS server), which can forward DNS requests to the host's loopback address properly.
$ cat /etc/resolv.conf
nameserver 127.0.0.1
$ docker run --rm alpine cat /etc/resolv.conf
nameserver 8.8.8.8
nameserver 8.8.4.4
$ docker network create demo
557079c79ddf6be7d6def935fa0c1c3c8290a0db4649c4679b84f6363e3dd9a0
$ docker run --rm --net demo alpine cat /etc/resolv.conf
nameserver 127.0.0.11
options ndots:0
If you use docker-compose, it will set up a custom network for your services automatically (with a file format v2+). Note, however, that while docker-compose runs containers in a user-defined network, it still builds them in the default network. To use a custom network for builds you can specify the network parameter in the build configuration (requires file format v3.4+).
I just had to deal with this last night and eventually remembered that docker run has a set of options for handling it. I used --dns to specify the DNS server I want the container to use. Works like a champ and no need to hack my docker host. There are other options for the domain name and search suffixes.
Since the automatic DNS discovery is guilty here, you may override the default setting in docker's configuration.
First, get the IP of the DNS server dnsmasq is using with e.g.:
$ sudo kill -USR1 `pidof dnsmasq`
$ sudo tail /var/log/syslog
[...]
Apr 24 13:20:19 host dnsmasq[2537]: server xx.yy.zz.tt1#53: queries sent 0, retried or failed 0
Apr 24 13:20:19 host dnsmasq[2537]: server xx.yy.zz.tt2#53: queries sent 0, retried or failed 0
The IP addresses correspond to the xx.yy.zz.tt placeholders above.
Alternatively, if your system is using systemd-resolve instead of dnsmasq, run:
$ resolvectl status | grep 'Current DNS'
Current DNS Server: xx.yy.zz.tt
You can set the DNS at docker run time with the --dns option:
$ sudo docker run --dns xx.yy.zz.tt1 --dns xx.yy.zz.tt2 -ti mmoy/ubuntu-netutils bash
root#6c5d08df5dfd:/# ping www.example.com
PING www.example.com (93.184.216.34) 56(84) bytes of data.
64 bytes from 93.184.216.34: icmp_seq=1 ttl=54 time=86.6 ms
64 bytes from 93.184.216.34: icmp_seq=2 ttl=54 time=86.6 ms
One advantage of this solution is that there is no configuration file involved, hence no risk of forgetting about the configuration and running into troubles later because of a specific config: you're getting this DNS configuration if and only if you type the --dns option.
A drawback is that you won't get any DNS caching in the containers, hence DNS resolution will be slower.
Alternatively you may set it permanently in Docker's configuration file, /etc/docker/daemon.json (create it, on the host, if it doesn't exist):
$ cat /etc/docker/daemon.json
{
"dns": ["xx.yy.zz.tt1", "xx.yy.zz.tt2"]
}
You need to restart the docker daemon to take the daemon.json file into account:
sudo service docker restart
Then you can check the configuration:
$ sudo docker run -ti mmoy/ubuntu-netutils bash
root#56c74d3bd94b:/# cat /etc/resolv.conf
nameserver xx.yy.zz.tt1
nameserver xx.yy.zz.tt2
root#56c74d3bd94b:/# ping www.example.com
PING www.example.com (93.184.216.34) 56(84) bytes of data.
64 bytes from 93.184.216.34: icmp_seq=1 ttl=54 time=86.5 ms
Note that this hardcodes the DNS IP in your configuration files. This is strongly discouraged if your machine is a laptop that connects to different networks, and may be problematic if your internet service provider changes the IP of the DNS servers.
Since dnsmasq is the issue, one option is to disable it on the host. This works, but will disable DNS caching for all applications running on the host, hence is a really bad idea if the host is used for applications other than docker.
If you're sure you want to go this way, uninstall dnsmasq, e.g. on Debian-based systems like Ubuntu, run apt remove dnsmasq.
You may then check that /etc/resolv.conf within the container points to the DNS server used by the host.
I had problems with the DNS resolver in our docker containers. I tried a lot of different things, and in the end, I just figured that my VPS in Hostgator didn't have installed by default NetworkManager-tui (nmtui), I just installed and reboot it.
sudo yum install NetworkManager-tui
And reconfigured my resolv.conf with default DNS as 8.8.8.8.
nano /etc/resolv.conf
how is it possible to resolve names defined in Docker host's /etc/hosts in containers?
Containers running in my Docker host can resolve public names (e.g. www.ibm.com) so Docker dns is working fine.
I would like to resolve names from Docker hosts's (e.g. 127.17.0.1 smtp) from containers.
My final goal is to connect to services running in Docker host (e.g. smtp server) from containers. I know I can use the Docker Host IP (127.17.0.1) from containers, but I thought that Docker would have used the Docker host /etc/hosts to build containers's resolve files as well.
I am even quite sure I have seen this working a while ago... but I could be wrong.
Any thoughts?
Giovanni
Check out the --add-host flag for the docker command: https://docs.docker.com/engine/reference/run/#managing-etchosts
$ docker run --add-host="smtp:127.17.0.1" container command
In Docker, /etc/hosts cannot be overwritten or modified at runtime (security feature). You need to use Docker's API, in this case --add-host to modify the file.
For docker-compose, use the extra_hosts option.
For the whole "connect to services running in host" problem, see the discussion in this GitHub issue: https://github.com/docker/docker/issues/1143.
The common approach for this problem is to use --add-host with Docker's gateway address for the host, e.g. --add-host="dockerhost:172.17.42.1". Check the issue above for some scripts that find the correct IP and start your containers.
You can setup on host simple DNS server, and in container setup /etc/resolve.conf to Docker host DNS server.
For example in dnsmasq you can add addn-hosts=/etc/hosts to config file. So container, by using Docker host DNS server, will be able to resolve hosts /etc/hosts.
I am trying to run Ubuntu inside of Docker, but my network has several restrictions:
must use network's DNS, when I set it, I do not get a response, I suspect it is because the return address is the docker IP 172 ...
must use network's Proxy, I can set an env, but it is useless without DNS working
I am trying this:
docker run --dns=10.69.114.61 -it ubuntu
docker run --dns=172.17.0.1 -it ubuntu (docker0 interface IP)
Docker containers will use docker host's dns settings by default.
In order to specify DNS server:
docker run --dns=10.69.114.61 -it debian:jessie /bin/bash
root#299849f13f21:/# cat /etc/resolv.conf
nameserver 10.69.114.61
Needless to say you must supply your network's DNS server IP address to the --dns flag.
docker run --dns=<your DNS server IP> -it debian:jessie /bin/bash
As you pointed out you should specify HTTP_PROXY variable to use your proxy server.
I have started to use docker and liking it mostly because Docker containers are kind of light-weight VMs. But I am unable to figure out, how docker containers may be able resolve each-other's hostnames. They can connect to each other using there IPs, but not using their hostnames, I cannot even edit /etc/hosts in the containers to make up for that somehow. When I restart the containers, they get different IPs and hence I want to use the hostnames in place of IPs to communicate with each other. Let us say, I want to run Zookeeper instances of a Zookeeper cluster in the containers and I want to put the hostnames of the Zookeeper servers in the config (zoo.cfg) files.
As of Docker 1.10, if you create a distinct docker network, Docker will resolve hostnames intra-container-wise using an internal DNS server [1][2][3].
You can change the network hostname by specifying one with --name within the docker run. Otherwise the hostname will refer to the container id (12 char long hash, shown by docker container ls ).
See also:
Docker doesn't resolve hostname
When to use --hostname in docker?
Sources:
[1] = docker docs - Embedded DNS server in user-defined networks
[2] = Docker Engine release notes
- 1.10.0 (2016-02-04) - Networking
[3] = Docker pull requests - Vendoring libnetwork
It may be worth checking out Docker links (https://docs.docker.com/userguide/dockerlinks/). When you link to a running container, a host entry is added for the container you wish to connect to.
In their example they show
$ sudo docker run -t -i --rm --link db:db training/webapp /bin/bash
root#aed84ee21bde:/opt/webapp# cat /etc/hosts
172.17.0.7 aed84ee21bde
. . .
172.17.0.5 db
As you see here, they link the application they're in bash with to the container named db, and subsequently a host entry is added for db with the IP address of that container.
So in the instance of having zookeeper running, you could simply make the containers you start just link to zookeeper. I hope this helps!
Can depend on OS of container, but supposing that container runs Linux
you can check your DNS configuration this way:
cat /etc/resolv.conf
It can return something like:
nameserver 127.0.0.11
options ndots:0
/etc/resolv.conf is standard configuration file for DNS in UNIX-like OS-es.
In this particular case container is configured to use 127.0.0.11 as DNS server.
So container can query it to determine IP address of another container using it's host name.
You can check whether that host actually works by using nslookup command, e.g.:
nslookup redis 127.0.0.11
, which will contact DNS server 127.0.0.11 and ask to resolve host name "redis".
It can return something like:
Server: 127.0.0.11
Address 1: 127.0.0.11
Name: redis
Address 1: 172.21.0.3 counter-app_redis_1.counter-app_counter-net
, which would mean that host name resolved to ip 172.21.0.3.
In this specific case nameserver entry was added by using the following entry in the docker-compose.yml configuration file:
...
networks:
counter-net:
This root entry configured common bridge network shared by several docker containers.