i'm trying to set up pihole in a docker container (on a raspberry pi) and as such, have my DNS on my ip: 192.160.170.10. The docker container runs the dns and exposes its port 53, where the dns is available
when running iplookup google.com on the host, i get the correct output:
Server: 192.160.170.10
Address: 192.160.170.10#53
Non-authoritative answer:
Name: google.com
Address: 172.217.16.78
My resolv.conf also contains this address.
when running a docker container, i am unable to do this nslookup however:
docker run busybox nslookup google.com
outputs:
;; connection timed out; no servers could be reached
Following this tutorial i've tried specifying the dns with the following command:
docker run --dns 192.160.170.10 busybox nslookup google.com
but this also does not solve the problem. I've also tried adding the dns to /etc/docker/daemon.json, which also does nothing.
the docker container's resolv.conf output is: nameserver 192.160.170.10
What is wrong with my configuration / How can i further debug this DNS issue?
edit:
output from docker run --rm --net=host busybox nslookup google.com:
Server: 192.160.170.10
Address: 192.160.170.10:53
Non-authoritative answer:
Name: google.com
Address: 172.217.16.78
*** Can't find google.com: No answer
Related
On my Windows 10 host machine with Docker 4.9.1 I want to ssh into a docker container.
I followed a bunch of tutorials just like this one:
https://phoenixnap.com/kb/how-to-ssh-into-docker-container
From within the container I can ssh into the container using its IP of 172.17.0.2, but from my host machine I cannot.
Confirmation of the IP address:
docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' interesting_meitner
'172.17.0.2'
Ping without response:
ping 172.17.0.2
Ping wird ausgeführt für 172.17.0.2 mit 32 Bytes Daten:
Zeitüberschreitung der Anforderung.
Ping-Statistik für 172.17.0.2:
Pakete: Gesendet = 1, Empfangen = 0, Verloren = 1
(100% Verlust),
SSH with connection timeout:
ssh root#172.17.0.2
ssh: connect to host 172.17.0.2 port 22: Connection timed out
Starting the container (obviously done before trying to connect to it):
docker run -ti with_ssh:new /bin/bash
I have also tried this with options for remapping ports i.e. -p 22:666 or -p 666:22 .
Starting ssh server:
/etc/init.d/ssh start
* Starting OpenBSD Secure Shell server sshd
Checking status:
/etc/init.d/ssh status
* sshd is running
Ssh from container into container:
ssh root#172.17.0.2
The authenticity of host '172.17.0.2 (172.17.0.2)' can't be established.
ECDSA key fingerprint is SHA256:471dnz1q83owB/Nu0Qnnyz/Sct4Kwry9Sa9L9pwQeZo.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.17.0.2' (ECDSA) to the list of known hosts.
root#172.17.0.2's password:
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 5.10.16.3-microsoft-standard-WSL2 x86_64)
[...]
Again from the Docker host I get a connection timeout. What do?
Your Docker container runs in a virtual network you cannot reach from the host (because it is isolated), which is why you cannot ping the containers IP from the host (but your docker container can, because it is attending the same network). You can expose the port like you already did with -p 666:22, but then you have to SSH to localhost not to the IP of the container: ssh -p 666 root#127.0.0.1.
You could also configure a correct routing from your hosts network to the virtual network and then you can reach the IP directly.
I did not reproduce your setup but this might work i guess. Hope it helps.
TL;DR: How do I make containers to use dnsmasq running on the host machine as a DNS?
Details of what I tried and where I am rn are below.
In my docker-compose.yml I set up a custom network:
networks:
mycustomnet:
driver: bridge
and all containers are on it:
services:
mycontainer:
networks:
- mycustomnet
As per the docs and some answers here on SO, in a configuration like this, docker will setup /etc/resolv.conf in the container to point to 127.0.0.11, which will then forward DNS requests to whatever the host DNS resolver is set to. That's my understanding, and indeed it seems to do set that correctly:
root#717f2c8ce87e:/# cat /etc/resolv.conf
nameserver 127.0.0.11
options ndots:0
My host is configured with a dnsmasq service that resolves our internal TLDs (say, .example) against a cloud provider (DO) DNS server, and everything else is resolved against Google and Cloudflare DNS. From /etc/dnsmasq.conf:
server=/example/173.245.58.51
server=/example/173.245.59.41
server=/example/198.41.222.173
server=8.8.8.8
server=8.8.4.4
server=1.1.1.1
server=1.0.0.1
On my host I can do nslookup test.example and it is getting correctly resolved:
-> # nslookup test.example
Server: ::1
Address: ::1#53
Name: test.example
Address: 10.104.0.4
But if I do the same in the container, it doesn't resolve hostnames with internal TLDs:
-> # docker exec -it mycontainer_1 bash
root#717f2c8ce87e:/# nslookup test.example
Server: 127.0.0.11
Address: 127.0.0.11#53
** server can't find test.example: NXDOMAIN
But it does resolve addresses such as google.com or github.com:
root#717f2c8ce87e:/# nslookup google.com
Server: 127.0.0.11
Address: 127.0.0.11#53
Non-authoritative answer:
Name: google.com
Address: 172.217.194.100
Name: google.com
Address: 172.217.194.102
Name: google.com
Address: 172.217.194.101
Name: google.com
Address: 172.217.194.139
Name: google.com
Address: 172.217.194.113
Name: google.com
Address: 172.217.194.138
Name: google.com
Address: 2404:6800:4003:c04::8b
Name: google.com
Address: 2404:6800:4003:c04::71
Name: google.com
Address: 2404:6800:4003:c04::64
Name: google.com
Address: 2404:6800:4003:c04::66
This is quite confusing because I don't quite understand what DNS server it is using. Clearly, not the host machine's dnsmasq, or otherwise it should have resolved the .example domains too – I guess?
What am I missing? How do I resolve .example domains from within the container? And how can I check which DNS server my containers currently use?
Suddenly when I deployed some new containers with docker-compose the internal hostname resolution didn't work.
When I tried to ping one container from the other using the service name from the docker-compose.yaml file I got ping: bad address 'myhostname'
I checked that the /etc/resolv.conf was correct and it was using 127.0.0.11
When I tried to manually resolve my hostname with either nslookup myhostname. or nslookup myhostname.docker.internal I got error
nslookup: write to '127.0.0.11': Connection refused
;; connection timed out; no servers could be reached
Okay so the issue is that the docker DNS server has stopped working. All already started containers still function, but any new ones started has this issue.
I am running Docker version 19.03.6-ce, build 369ce74
I could of course just restart docker to see if it solves it, but I am also keen on understanding why this issue happened and how to avoid it in the future.
I have a lot of containers started on the server and a total of 25 docker networks currently.
Any ideas on what can be done to troubleshoot? Any known issues that could explain this?
The docker-compose.yaml file I use has worked before and no changes has been done to it.
Edit: No DNS names at all can be resolved. 127.0.0.11 refuses all connections. I can ping any external IP addresses, as well as the IP of other containers on the same docker network. It is only the 127.0.0.11 DNS server that is not working. 127.0.0.11 still replies to ping from within the container.
Make sure you're using a custom bridge network, NOT the default one. As per the Docker docs (https://docs.docker.com/network/bridge/), the default bridge network does not allow automatic DNS resolution:
Containers on the default bridge network can only access each other by IP addresses, unless you use the --link option, which is considered legacy. On a user-defined bridge network, containers can resolve each other by name or alias.
I have the same problem. I am using the pihole/pihole docker container as the sole dns server on my network. Docker containers on the same host as the pihole server could not resolve domain names.
I resolved the issue based on "hmario"'s response to this forum post.
In brief, modify the pihole docker-compose.yml from:
---
version: '3.7'
services:
unbound:
image: mvance/unbound-rpi:1.13.0
hostname: unbound
restart: unless-stopped
ports:
- 53:53/udp
- 53:53/tcp
volumes: [...]
to
---
version: '3.7'
services:
unbound:
image: mvance/unbound-rpi:1.13.0
hostname: unbound
restart: unless-stopped
ports:
- 192.168.1.30:53:53/udp
- 192.168.1.30:53:53/tcp
volumes: [...]
Where 192.168.1.30 is a ip address of the docker host.
I'm having exactly the same problem. According to the comment here I could reproduce the setting without docker-compose, only using docker:
docker network create alpine_net
docker run -it --network alpine_net alpine /bin/sh -c "cat /etc/resolv.conf; ping -c 4 www.google.com"
stopping docker (systemctl stop docker) and enabling debug output it gives
> dockerd --debug
[...]
[resolver] read from DNS server failed, read udp 172.19.0.2:40868->192.168.177.1:53: i/o timeout
[...]
where 192.168.177.1 is my local network ip for the host that docker runs on and where also pi-hole as dns server is running and working for all of my systems.
I played around with fixing iptables configuration. but even switching them off completely and opening everything did not help.
The solution I found, without fully understanding the root case, was to move the dns to another server. I installed dnsmasq on a second system with ip 192.168.177.2 that nothing else than forwarding all dns queries back to my pi-hole server on 192.168.177.1
starting docker on 192.168.177.1 again with dns configured to use 192.168.177.2 everything was working again
with this in one terminal
dockerd --debug --dns 192.168.177.2
and the command from above in another it worked again.
> docker run -it --network alpine_net alpine /bin/sh -c "cat /etc/resolv.conf; ping -c 4 www.google.com"
search mydomain.local
nameserver 127.0.0.11
options ndots:0
PING www.google.com (172.217.23.4): 56 data bytes
64 bytes from 172.217.23.4: seq=0 ttl=118 time=8.201 ms
--- www.google.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 8.201/8.201/8.201 ms
So moving the the dns server to another host and adding "dns" : ["192.168.177.2"] to my /etc/docker/daemon.json fixed it for me
Maybe someone else can help me to explain the root cause behind the problem with running the dns server on the same host as docker.
First, make sure your container is connected to a custom bridged network. I suppose by default in a custom network DNS request inside the container will be sent to 127.0.0.11#53 and forwarded to the DNS server of the host machine.
Second, check iptables -L to see if there are docker-related rules. If there is not, probably that's because iptables are restarted/reset. You'll need to restart docker demon to re-add the rules to make DNS request forwarding working.
I had same problem, the problem was host machine's hostname. I have checked hostnamectl result and it was ok but problem solved with stupid reboot. before reboot result of cat /etc/hosts was like this:
# The following lines are desirable for IPv4 capable hosts
127.0.0.1 localhost HostnameSetupByISP
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4
# The following lines are desirable for IPv6 capable hosts
::1 localhost HostnameSetupByISP
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
and after reboot, I've got this result:
# The following lines are desirable for IPv4 capable hosts
127.0.0.1 hostnameIHaveSetuped HostnameSetupByISP
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4
# The following lines are desirable for IPv6 capable hosts
::1 hostnameIHaveSetuped HostnameSetupByISP
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
I have many containers on my server, I need to access all containers by the name(not IP).
and of course, I can ping containers by IP address.
problem is I can't ping them by hostname or name.
also, I don't using docker-compose. I use docker file and docker restapi
Option A: run a DNS proxy server container
Here is a DNS proxy server project that can do this: https://github.com/mageddo/dns-proxy-server
First, you need to start the DNS proxy server:
docker run --rm --hostname dns.mageddo -v /var/run/docker.sock:/var/run/docker.sock -v /etc/resolv.conf:/etc/resolv.conf defreitas/dns-proxy-server
Then, run a dummy container and assign it a --hostname for testing purpose:
docker run -d --hostname=this-can-be-resolved-from-host nginx
Finally, try to resolve/ping/curl the name you assigned to the nginx container in the previous step, from your host machine:
neo#neo-desktop:~$ nslookup this-can-be-resolved-from-host
Server: 172.17.0.4
Address: 172.17.0.4#53
Non-authoritative answer:
Name: this-can-be-resolved-from-host
Address: 172.17.0.3
Name: this-can-be-resolved-from-host
Address: 172.17.0.3
neo#neo-desktop:~$ ping this-can-be-resolved-from-host
PING this-can-be-resolved-from-host (172.17.0.3) 56(84) bytes of data.
64 bytes from 172.17.0.3 (172.17.0.3): icmp_seq=1 ttl=64 time=0.032 ms
neo#neo-desktop:~$ curl this-can-be-resolved-from-host
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
[...]
Option B: Run an injector that adds the container names directly in the hosts file, on the docker host:
(solution found by the OP #Tokyo Developer)
Here is a simple "etc/hosts" file injection tool: https://github.com/dvddarias/docker-hoster
Run the injector container:
docker run -d \
-v /var/run/docker.sock:/tmp/docker.sock \
-v /etc/hosts:/tmp/hosts \
dvdarias/docker-hoster
Run a dummy container and assign it a --hostname for testing purpose:
docker run -d --hostname=this-can-be-resolved-from-host nginx
Try to resolve the hostname AND the container name assigned to the nginx container in the previous step, from your host machine:
nslookup this-can-be-resolved-from-host
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
Name: this-can-be-resolved-from-host
Address: 172.17.0.3
nslookup keen_lamarr
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
Name: keen_lamarr
Address: 172.17.0.3
I just installed Docker on CentOS 8. Everything went fine, I’ve run the hello_world test container and the result was successful. The issue is that inside the containers I cannot resolve any DNS queries.
For example when I type ping 8.8.8.8 (on any kind of base container such as Ubuntu or CentOS) I get the correct answer but when I try running ping google.com I don’t get any answer, meaning that the container is unable to resolve the DNS for any URL.
I’ve also tried running a base container like this:
docker run busybox ping -c 1 192.203.230.10
and I’m getting this (correct) output:
[server#localhost ~]$ docker run busybox ping -c 1 192.203.230.10
PING 192.203.230.10 (192.203.230.10): 56 data bytes
64 bytes from 192.203.230.10: seq=0 ttl=51 time=32.413 ms
--- 192.203.230.10 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 32.413/32.413/32.413 ms
But now trying resolving the domain google.com with the command:
docker run busybox nslookup google.com
I get this output:
[server#localhost ~]$ docker run busybox nslookup google.com
nslookup: write to '8.8.8.8': No route to host
nslookup: write to '8.8.4.4': No route to host
;; connection timed out; no servers could be reached
I’m finding this problem really frustrating since I’ve not found a way to solve it. I'm attaching for completeness a cat /etc/resolv.conf of both the machine running Docker and the container itself
On the machine with CentOS 8:
[server#localhost ~]$ cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 8.8.8.8
nameserver 8.8.4.4
Inside any Docker container:
bash-4.4# cat /etc/resolv.conf
# Generated by NetworkManager
nameserver 8.8.8.8
nameserver 8.8.4.4
Which looks like a normal and working configuration, but when I try to ping google.com (inside the containers) I get:
bash-4.4# ping google.com
ping: google.com: Try again
What new things can I try?
The solution to the problem seems to be adding docker as a trusted interface on firewalld using the command:
firewall-cmd --permanent --zone=trusted --add-interface=docker0
and then reloading firewalld with:
firewall-cmd --reload
After it should work correctly (remember to reboot)