Docker | docker0 has IP (is accessable from external) only at first run - docker

When I start a simple docker container (e.g. Portainer) with
docker run -d --name portainer -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock portainer/portainer
the container is accessable from the internet as expected.
When I stop (docker stop portainer) and start (docker start portainer) the container, the port 9000 is open again (verified with nmap), but the web interface of portainer loads forever.
# first run
networkctl
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier configured
2 enp35s0 ether routable configured
3 enp36s0 ether no-carrier configuring
5 br-1815f2210327 bridge no-carrier configuring
6 br-7f9b2f2637a1 bridge no-carrier configuring
7 br-a9ae27884558 bridge no-carrier configuring
6552 br-39aac8ad8ef3 bridge routable configuring
6559 docker0 bridge no-carrier configuring
# next run
networkctl
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier configured
2 enp35s0 ether routable configured
3 enp36s0 ether no-carrier configuring
5 br-1815f2210327 bridge no-carrier configuring
6 br-7f9b2f2637a1 bridge no-carrier configuring
7 br-a9ae27884558 bridge no-carrier configuring
6552 br-39aac8ad8ef3 bridge no-carrier configuring
6559 docker0 bridge no-carrier configuring
I already tried different workarounds that I found on the internet, like
nano /etc/docker/daemon.json
{ "debug": true, "bip": "172.20.0.1/16" }
and this config file in various configurations
nano /etc/systemd/network/docker0.network
#[Match]
#Name=docker0
#[Network]
#IPForward=yes
#[Network]
#Address=172.17.0.1/16
#[Link]
#Unmanaged=yes
(Currently everythings is commented out.)
When I restart the docker daemon with
systemctl restart docker
and then start the docker container
docker start portainer
it's working fine again.
My system is a linux root server hosted by strato.de:
docker -v
Docker version 20.10.6, build 370c289
cat /etc/issue
Ubuntu 20.04.2 LTS
uname -r
5.4.0-73-generic
The problem occurs with all of my docker containers on that server.
I would be very grateful for any further tips.
UPDATE
Docker on Ubuntu doesn't connect to localhost
The mentioned solution seems not work on my server with Ubuntu 20.04.
Yesterday I installed the same OS and docker version in a VM. Everything is working fine there.
Kind regards,
K1LLUM1N471

My problem was that i could ping google only once in a container (docker run --rm alpine ping google.com), after exit it would not ping the next time i ran the same command. In ifconfig docker0 the inet address was gone after exiting the container the inet6 was still there after running the command once.
when running networkctl status the docker0 link is at configuring.
This might do the trick:
the default Netplan config (/etc/netplan/01-netcfg.yaml)in my Ubuntu 22.04 server from Strato (dedicated server) is:
network:
version: 2
ethernets:
mainif:
match:
name: '*'
dhcp4: yes
replace it with something like this:
network:
version: 2
ethernets:
enp3s0:
dhcp4: yes
dhcp6: no
enp2s0:
dhcp4: yes
dhcp6: no
apply netplan config
sudo netplan try or sudo netplan apply
restart Docker
sudo systemctl restart docker
when now running networkctl the docker0 link should be unmanaged

For your interrest:
I know that is not the best answer, but in my case I solved the problem by downgrading the OS on the root server :(
cat /etc/issue
Ubuntu 18.04.4 LTS
docker -v
Docker version 20.10.7, build f0df350

Related

Establish conversation between hello-world apps in Docker containers

I'm trying to run my hello-world apps inside Docker: frontend need to consume REST from backend.
I run
docker run -p 1337:1337 --net=bridge me/p-dockerfile-advanced-backend:latest
docker run -p 1338:1338 --net=bridge me/p-dockerfile-advanced-frontend:latest http://127.0.0.1:1337
I am able to connect to both of them using a browser from the host OS (My desktop Windows 10 x64) :
The http://127.0.0.1:1337 parameter needed for the frontend application to know where the restful services reside. But the app cannot connect to them. I cannot connect too.
Windows PowerShell
Copyright (C) 2016 Microsoft Corporation. All rights reserved.
PS C:\Users\user1> docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4b0852253b8a me/p-dockerfile-advanced-frontend:latest "/usr/bin/java -ja..." 24 minutes ago Up 24 minutes 0.0.0.0:1338->1338/tcp laughing_noyce
e73f8a6efa24 me/p-dockerfile-advanced-backend:latest "/usr/bin/java -ja..." 26 minutes ago Up 26 minutes youthful_chandrasekhar
PS C:\Users\user1> docker exec -it 4b0852253b8a bash
root#4b0852253b8a:/# apt-get install telnet
<...>
root#4b0852253b8a:/# telnet localhost 1337
Trying 127.0.0.1...
Trying ::1...
telnet: Unable to connect to remote host: Cannot assign requested address
root#4b0852253b8a:/#
Unable to connect, but it should because I specified --net=bridge on both containers and backend listen the port 1337 :
root#e73f8a6efa24:/# netstat -lntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:1337 0.0.0.0:* LISTEN
root#e73f8a6efa24:/#
PS: I spent almost all day trying to make it work before asking here.
The problem is the 127.0.0.1 address.
Each container is assigned, by default, 2 interfaces: eth0 and lo (the loopback interface with the 127.0.0.1 address).
You need to specify the name or address of the previous container. For this simple application you may use the --link option.
docker run -p 1337:1337 --name backend me/p-dockerfile-advanced-backend:latest
docker run -p 1338:1338 --link backend:backend me/p-dockerfile-advanced-frontend:latest http://backend:1337
Note that the --link option is deprecated as stated in:
https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/
Since these are different containers, you have to expose ports on both of them. Run the first with:
docker run -p 1337:1337 --net=bridge me/p-dockerfile-advanced-backend:latest
Note that bridge is the default network so you it is extra. Both containers will be on the same bridge network by default anyway.

Hit a service running on localhost from inside a docker image

I'm on mac os, I have a service running on my machine on localhost:8000
now I want to launch a docker image and hit this service from here.
I did a docker bridge and I use it from inside, but it is not working.
Here are my steps:
My host ip:
ifconfig
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ether 98:01:a7:b0:2b:41
inet 192.168.0.70 netmask 0xffffff00 broadcast 192.168.0.255
media: autoselect
status: active
I hit my service from host:
curl localhost:8000 #this is working!
I create a bridge and I use it:
docker network create -d bridge --subnet 192.168.0.0/24 --gateway 192.168.0.1 dockernet
docker run --rm -it -v "$(pwd):/src" --network=dockernet qatests /bin/bash
from inside docker, I do a curl but it is not working:
curl 192.168.0.1:8000 #it's not working :-(
any ideas?
You dont need to create a new network. You can use the default one (bridge).
Just check which ip is associated to the docker0 interface in your host with ip or ifconfig (in my case is 172.17.42.1), and use that ip from inside the container:
$ curl 172.17.42.1:8000
In the end, I've discovered that if I ping my pc ip, I can see it even from the docker image.
For convenience, I did a lounch script witch get my current IP and launch the docker image making my ip accessible under "mymac" address
So what i did is lunching
MY_IP=$(ifconfig en0 | grep inet | grep -v inet6 | awk '{print $2}')
docker run --rm -it -v "$(pwd):/src" --add-host=mymac:$MY_IP qatests
/bin/bash
inside docker I can now lunch:
curl mymac:8000 #it works! now mymac is my pc outside docker

Can't ping docker IPv6 container

I ran docker daemon for using it with global IPv6 for containers:
docker daemon --ipv6 --fixed-cidr-v6="xxxx:xxxx:xxxx:xxxx::/64"
After it I ran docker container:
docker run -d --name my-container some-image
It successfully got Global IPv6 address( I checked by docker inspect my-container). But I can't to ping my container by this ip:
Destination unreachable: Address unreachable
But I can successfully ping docker0 bridge by it's IPv6 address.
Output of route -n -6 contains next lines:
Destination Next Hop Flag Met Ref Use If
xxxx:xxxx:xxxx:xxxx::/64 :: U 256 0 0 docker0
xxxx:xxxx:xxxx:xxxx::/64 :: U 1024 0 0 docker0
fe80::/64 :: U 256 0 0 docker0
docker0 interface has global IPv6 address:
inet6 addr: xxxx:xxxx:xxxx:xxxx::1/64 Scope:Global
xxxx:xxxx:xxxx:xxxx:: everywhere is the same, and it's global IPv6 address of my eth0 interface
Does docker required something additional configs for accessing my containers via IPv6?
Assuming IPv6 in your guest OS is properly configured probably you are pinging the container not from host OS, but outside and network discovery protocol is not configured. Other hosts does not know if your container is behind of your host. I'm doing this after start of container with IPv6 (in host OS) (in ExecStartPost clauses of Systemd .service file)
/usr/sbin/sysctl net.ipv6.conf.interface_name.proxy_ndp=1
/usr/bin/ip -6 neigh add proxy $(docker inspect --format {{.NetworkSettings.GlobalIPv6Address}} container_name) dev interface_name"
Beware of IPv6: docker developers say in replies to bug reports they do not have enough time to make IPv6 production-ready in version 1.10 and say nothing about 1.11.
Mb you use wrong ping command. For ipv6 is ping6.
$ ping6 2607:f0d0:1002:51::4

Docker 1.9.0 "bridge" versus a custom bridge network results in difference in hosts file and SSH_CLIENT env variable

Let me first explain what I'm trying to do, as there may be multiple ways to solve this. I have two containers in docker 1.9.0:
node001 (172.17.0.2) (sudo docker run --net=<<bridge or test>> --name=node001 -h node001 --privileged -t -i -v /sys/fs/cgroup:/sys/fs/cgroup <<image>>)
node002 (172.17.0.3) (,,)
When I launch them with --net=bridge I get the correct value for SSH_CLIENT when I ssh from one to the other:
[root#node001 ~]# ssh root#172.17.0.3
root#172.17.0.3's password:
[root#node002 ~]# env | grep SSH_CLIENT
SSH_CLIENT=172.17.0.3 56194 22
[root#node001 ~]# ping -c 1 node002
ping: unknown host node002
In docker 1.8.3 I could also use the hostnames I supply when I start them, in 1.8.3 that last ping statement works!
In docker 1.9.0 I don't see anything being added in /etc/hosts, and the ping statement fails. This is a problem for me. So I tried creating a custom network...
docker network create --driver bridge test
When I launch the two containers with --net=test I get a different value for SSH_CLIENT:
[root#node001 ~]# ssh root#172.18.0.3
root#172.18.0.3's password:
[root#node002 ~]# env | grep SSH_CLIENT
SSH_CLIENT=172.18.0.1 57388 22
[root#node001 ~]# ping -c 1 node002
PING node002 (172.18.0.3) 56(84) bytes of data.
64 bytes from node002 (172.18.0.3): icmp_seq=1 ttl=64 time=0.041 ms
Note that the ip address is not node001's, it seems to represent the docker host itself. The hosts file is correct though, containing:
172.18.0.2 node001
172.18.0.2 node001.test
172.18.0.3 node002
172.18.0.3 node002.test
My current workaround is using docker 1.8.3 with the default bridge network, but I want this to work with future docker versions.
Is there any way I can customize the test network to make it behave similarly to the default bridge network?
Alternatively:
Maybe make the default bridge network write out the /etc/hosts file in docker 1.9.0?
Any help or pointers towards different solutions will be greatly appreciated..
Edit: 21-01-2016
Apparently the problem is fixed in 1.9.1, with bridge in docker 1.8 and with a custom (--net=test) in 1.9.1, now the behaviour is correct:
[root#node001 tmp]# ip route
default via 172.17.0.1 dev eth0
172.17.0.0/16 dev eth0 proto kernel scope link src 172.17.0.5
[root#node002 ~]# env | grep SSH_CLIENT
SSH_CLIENT=172.18.0.3 52162 22
Retried in 1.9.0 to see if I wasn't crazy, and yeah there the problem occurs:
[root#node001 tmp]# ip route
default via 172.18.0.1 dev eth0
172.18.0.0/16 dev eth0 proto kernel scope link src 172.18.0.3
[root#node002 ~]# env|grep SSH_CLI
SSH_CLIENT=172.18.0.1 53734 22
So after remove/stop/start-ing the instances the IP-addresses were not exactly the same, but it can be easily seen that the ssh_client source ip is not correct in the last code block. Thanks #sourcejedi for making me re-check.
Firstly, I don't think it's possible to change any settings on the default network, i.e. to write /etc/hosts. You apparently can't delete the default networks, so you can't recreate them with different options.
Secondly
Docker is careful that its host-wide iptables rules fully expose containers to each other’s raw IP addresses, so connections from one container to another should always appear to be originating from the first container’s own IP address. docs.docker.com
I tried reproducing your issue with the random containers I've been playing with. Running wireshark on the bridge interface for the network, I didn't see my ping packets. From this I conclude my containers are indeed talking directly to each other; the host was not doing routing and NAT.
You need to check the routes on your client container ip route. Do you have a route for 172.18.0.2/16? If you only have a default route, it could try to send everything through the docker host. And it might get confused and do masquerading as if it was talking with the outside world.
This might happen if you're running some network configuration in your privileged container. I don't know what's happening if you're just booting it with bash though.

Docker inside Linux VM cannot connect to web application

My setup is the following:
Host: Win10
Guest: Ubuntu 15.10 (clean install, only docker and nodejs are added)
Base image: https://hub.docker.com/r/microsoft/aspnet/ 1.0.0-beta8-coreclr
Inside the guest I have installed Docker and created image (added sample webapp using yeoman to the image above). When I run the image inside container I can ping the container IP sucessfuly using the container IP from the linux (e.g. 172.17.0.2).
$sudo docker run -d -p 80:5000 --name web myapp
$sudo docker inspect --format '{{ .NetworkSettings.IPAddress }}' "web"
172.17.0.2
$ping 172.17.0.2
PING 172.17.0.2 (172.17.0.2) 56(84) bytes of data.
64 bytes from 172.17.0.2: icmp_seq=1 ttl=64 time=0.060 ms
1 packets transmitted, 1 received, 0% packet loss, time 999ms
$curl 172.17.0.2:80
curl: (7) Failed to connect to 172.17.0.2 port 80: Connection refused
I can also connect to the container and execute commands like ping, however from the linux machine (guest in VirtualBox, host for docker) I cannot access the web app that is hosted inside the container as seen above. I tried several approaches like mapping to the host IP addresses etc, but none of them worked. Did anyone have ideas where to start from ? Is the issue comes from that the docker is installed inside VirtualBox machine?
Thank you in advance.
Edit: Here are the logs from the container:
Could not open /etc/lsb_release. OS version will default to the empty string.
Hosting environment: Production
Now listening on: http://localhost:5000
Application started. Press Ctrl+C to shut down.
Your command tells Docker to essentially proxy requests from port 80 of the Linux guest to port 5000 of the container. So the curl command you tried doesn't work because you're trying on port 80 on the container, while the container itself has a service listening on port 5000.
To connect to the container directly, you would use (on the Linux guest):
curl 172.17.0.2:5000
To access via the published port on the Linux guest (from your host):
curl (Linux guest IP)
Or (from the Linux guest):
curl localhost
Edit: This will also prove to be problematic:
Now listening on: http://localhost:5000
You'll want your app inside the container to bind to all interfaces (0.0.0.0) so it listens on the container's assigned IP. With localhost it won't be accessible.
You might find this example useful:
https://github.com/aspnet/Home/blob/dev/samples/1.0.0-beta8/HelloWeb/project.json
This line specifies that the app bind to all interfaces (using "*") on port 5004:
21 "kestrel": "Microsoft.AspNet.Hosting --server Microsoft.AspNet.Server.Kestrel --server.urls http://*:5004"
You'll need similar configuration.

Resources