DNS resolution failing inside docker container when host network changes without rebooting device - docker

Question

Expected behavior
DNS resolution failing inside docker container when host network changes ( network1-192.168.1., network2-192.168.2. ) without rebooting device
docker container running on bridge network
Actual behavior
DNS resolution should be successful in case of host network changes
Steps to reproduce the behavior
Create network container (lets say.. "abc_network.ctr") without specifying network_mode as it takes bridge by default (and it used docker's embedded nameserver inside docker container)
Create another container (xyz.ctr) whose network mode will be "container" ("abc_network.ctr")
/etc/resolv.conf from inside the xyz.ctr container. which is able to resolve nameserver
root#d3e0dc71f370:~# cat /etc/resolv.conf
nameserver 127.0.0.11
options ndots:0
/etc/resolv.conf from host.. which is also able to resolve nameserver
root#RP4-2711-c03112-1xb0e6f3b8:~# cat /etc/resolv.conf
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients directly to
# all known uplink DNS servers. This file lists all configured search domains.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.
nameserver 192.168.1.1
Change network subnet of the machine (remove LAN cable and connect to another subnet network1 -> network2) without reboot.
And try to resolve nameserver inside docker container using following commands:
docker exec -it xyz.ctr bash
apt-get update
result-
root#d3e0dc71f370:~# apt-get update
Err:1 http://security.debian.org/debian-security buster/updates InRelease
Temporary failure resolving 'security.debian.org'
Err:2 http://deb.debian.org/debian buster InRelease
Temporary failure resolving 'deb.debian.org'
Err:3 http://deb.debian.org/debian buster-updates InRelease
Temporary failure resolving 'deb.debian.org'
Reading package lists... Done
W: Failed to fetch http://deb.debian.org/debian/dists/buster/InRelease Temporary failure resolving 'deb.debian.org'
W: Failed to fetch http://security.debian.org/debian-security/dists/buster/updates/InRelease Temporary failure resolving 'security.debian.org'
W: Failed to fetch http://deb.debian.org/debian/dists/buster-updates/InRelease Temporary failure resolving 'deb.debian.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.
dig output
root#15c396b06f57:~# dig
; <<>> DiG 9.11.5-P4-5.1+deb10u5-Debian <<>>
;; global options: +cmd
;; connection timed out; no servers could be reached
/etc/resolve.conf from host machine which is able to resolve nameserver
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients directly to
# all known uplink DNS servers. This file lists all configured search domains.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.
nameserver 192.168.2.1
apt-get working on host and dig command shows NO ERROR
Output of docker version:
Docker version 20.10.6, build 370c289
Output of docker info:
root#RP4-2711-c03112-1xb0e6f3b8:~# docker info
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
Server:
Containers: 22
Running: 18
Paused: 0
Stopped: 4
Images: 98
Server Version: 20.10.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.17-v7l+
Operating System: Raspbian GNU/Linux 10 (buster)
OSType: linux
Architecture: armv7l
CPUs: 4
Total Memory: 3.44GiB
Name: RP4-2711-c03112-1xb0e6f3b8
ID: C66C:3DAT:6DLH:PHJA:JR2N:KURB:PTMQ:BHUP:XRIH:FE3H:VSEZ:B7K3
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Default Address Pools:
Base: 172.128.0.1/10, Size: 16
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory TCP limit support
WARNING: No oom kill disable support
Additional environment details (AWS, VirtualBox, physical, etc.)
Device- Raspberry Pi-4B, Intel NUC (This is not an environment specific issue)