I have a bare metal server on Hetzner with IP 5.6.7.8 and 8 additional IPs reserved for me.
IPs: 1.2.3.144 to 1.2.3.151
subnet: 1.2.3.144/29
netmask: 255.255.255.248
broadcast: 1.2.3.151
gateway: 5.6.7.8
Now I want to create Docker network with type macvlan
docker network create macvlan --subnet=1.2.3.144/29 --gateway=5.6.7.8 -o parent=enp0s31f6 macvlan1
But this command causes an error
no matching subnet for gateway 5.6.7.8
Note when I set for example IP 1.2.3.150 and gateway 5.6.7.8 on a virtual machine on the host, it works correctly! but I can't set this none matching gateway in Docker network create command.
Related
Instead of listening to a single IP address like e.g. localhost:
ports:
- "127.0.0.1:80:80"
I want the container to only listen to a local network, i.e. e.g.:
ports:
- "10.0.0.0/16:80:80"
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.SERVICE.ports contains an invalid type, it should be a number, or an object
Is this possible?
I don't want to use things like swarm mode etc., yet.
If IP range is not supported, maybe at least multiple IP addresses like 10.0.0.2 and 10.0.0.3?
ERROR: for CONTAINER Cannot start service SERVICE: driver failed programming external connectivity on endpoint CONTAINER (...): Error starting userland proxy: listen tcp 10.0.0.3:80: bind: cannot assign requested address
ERROR: for SERVICE Cannot start service SERVICE: driver failed programming external connectivity on endpoint CONTAINER (...): Error starting userland proxy: listen tcp 10.0.0.3:80: bind: cannot assign requested address
Or is it not even supported to listen to 10.0.0.3 ?
The host machine is connected to 10.0.0.0/16:
> ifconfig
ens10: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet 10.0.0.2 netmask 255.255.255.255 broadcast 10.0.0.2
inet6 f**0::8**0:ff:f**9:b**7 prefixlen 64 scopeid 0x20<link>
ether **:00:00:**:**:** txqueuelen 1000 (Ethernet)
Listening to a single IP address seems not correct. The service is listening at an IP address.
Let's say your VM has two network interfaces (ethernet cards):
Network 1 → subnet: 10.0.0.0/24 and IP 10.0.0.100
Network 2 → subnet: 10.0.1.0/24 and IP 10.0.1.200
If you set 127.0.0.1:80:80 that means that your service listening at 127.0.0.1's (localhost) port 80.
If you want to access service from 10.0.0.0/24 subnet you should set 10.0.0.100:80:80 and use http://10.0.0.100:80 address to be able connect your container from external hosts
If you want to access service from multiple networks simultaneously you can bind the container port to multiple ports, where the IP is the connection source IP):
ports:
- 10.0.0.100:80:80
- 10.0.1.200:80:80
- 127.0.0.1:80:80
And don't forget to open 80 port at VM's firewall, if a firewall exists and restricts that network
I think you misunderstood this field.
When you map 127.0.0.1:80:80 you will map interface 127.0.0.1 from your host to your container.
In the case of the 127.0.0.1 you can only access it from inside your host.
When you map 10.0.0.3:80:80 you will map interface 10.0.0.3 from your host to your container. And all ip who can access 10.0.0.3 will have acces to your docker container mapping.
But in anycase this field will not do any filtering about who access this container
EDIT: After your modification i've seen my misunderstood about your question.
You want docker to create "bridge interface" to not share the ip of your host.
I don't think this is possible when using the port mapping
If you give Compose ports: (or docker run -p) an IP address, it must be a specific known IP address of a host interface, or 0.0.0.0 for "all interfaces". The Docker daemon gives this specific IP address to a bind(2) call, which takes an address and not a network, and follows the rules in ip(7) for IPv4.
With the output you show, you can only bind containers to 10.0.0.2. If you want to use other IP addresses on the same network, you also need to assign them to the host; see for example How can I (from CLI) assign multiple IP addresses to one interface? on Ask Ubuntu, and then you can bind a container to the newly-added address.
If your system is on multiple physical networks, you can have any number of ports: so long as the host address and host port are unique. In particular you can have multiple ports: that all forward to the same container port.
ports:
# make this visible to the external load balancer on port 80
- '192.168.17.2:80:3000'
# also make this visible to the internal network also on port 80
- '10.0.0.2:80:3000'
# and the management network but on port 3000
- '10.99.0.36:3000:3000'
Again, the host must already have these IP addresses in the ifconfig output.
I am wanting to have a worker node on a server I have that is behind a NAT (i.e can't expose ports publicly) I thought this wasn't a problem but it turns out to be one:
On this server behind the NAT I run:
docker swarm join --token SWMTKN-1... X.X.X.X:2377
Which in turn adds the server to the swarm. I am not sure where the "internal" IP address comes from but on traefik I then have a new server http://10.0.1.126:8080 (10.0.1.126 is definitely not the public IP) if I exec inside the traefik container:
docker exec -it 80f9cb33e24c sh
I can ping every server/node/worker in the list on traefik apart from the new one. Why?
When joining the swarm like this on the worker behind the vpn:
docker swarm join --advertise-addr=tun0 --token SWMTKN-1-... X.X.X.X:2377
I can see a new peer on my network from the manager:
$ docker network inspect traefik
...
"Peers": [
...
{
"Name": "c2f01f1f1452",
"IP": "12.0.0.2"
}
]
where 12.0.0.2 and tun0 is the vpn interface from the manager to the server behind the NAT. Unfortunately when I then run:
$ nmap -p 2377,2376,4789,7946 12.0.0.2
Starting Nmap 7.70 ( https://nmap.org ) at 2020-05-04 11:01 EDT
Nmap scan report for 12.0.0.2
Host is up (0.017s latency).
PORT STATE SERVICE
2376/tcp closed docker
2377/tcp closed swarm
4789/tcp closed vxlan
7946/tcp open unknown
I can see that the ports are closed for the docker worker which is weird?
Also if I use nmap -p 8080 10.0.1.0/24 inside the traefik container on the manager I get:
Nmap scan report for app.6ysph32io2l9q74g6g263wed3.mbnlnxusxv2wz0pa2njpqg2u1.traefik (10.0.1.62)
Host is up (0.00033s latency).
PORT STATE SERVICE
8080/tcp open http-proxy
on a succesfull swarm worker which has the network internal ip 10.0.1.62
but I get:
Nmap scan report for app.y7odtja923ix60fg7madydia3.jcfbe2ke7lzllbvb13dojmxzq.traefik (10.0.1.126)
Host is up (0.00065s latency).
PORT STATE SERVICE
8080/tcp filtered http-proxy
on the new swarm node. Why is it filtered? What am I doing wrong?
I'm adding this here as it's a bit longer.
I don't think it's enough for only the manager and the remote node to be able to communicate; nodes need to be able to communicate between themselves.
Try to configure the manager (who is connected to the VPN) to route packets to and from the remote worker through the VPN and add the needed routes on all nodes (including the remote one).
Something like:
# Manager
sysctl -w net.ipv4.ip_forward=1 # if you use systemd you might need extra steps
# Remote node
ip route add LOCAL_NODES_SUBNET via MANAGER_TUN_IP dev tun0
#Local nodes
ip route add REMOTE_NODE_TUN_IP/32 via MANAGER_IP dev eth0
If the above works correctly you need to make the routing changes above permanent.
To find the IP addresses for all your nodes run this command on the manager:
for NODE in $(docker node ls --format '{{.Hostname}}'); do echo -e "${NODE} - $(docker node inspect --format '{{.Status.Addr}}' "${NODE}")"; done
I have a macvlan network created with the following command:
docker network create -d macvlan --subnet=192.168.1.0/24 --gateway=192.168.1.2 -o parent=wlp2s0 pub_ne
Where wlp2s0 is the name of the wireless interface of my laptop.
gateway is 192.168.1.1 and subnet 192.168.1.0/24
Then I have created and attached a container to this network:
docker run --rm -itd --network pub_ne --name myAlpine alpine:latest sh
In addition I have created a virtual machine using, virtualbox provider, with bridged network interface.
if I use ping command:
- docker container -> vm ubuntu (ip of vm: 192.168.1.200) : ping works
but if I use ping command:
- docker container -> gateway 192.168.1.1
or
- docker container -> external world (google.com): ping not works
suggestions?
edit 1:
On docker host if i run tcpdump ( tcpdump -i icmp ) i see:
14:53:30.015822 IP 192.168.1.56 > 216.58.205.142: ICMP echo request, id 5376, seq 29, length 64
14:53:31.016143 IP 192.168.1.56 > 216.58.205.142: ICMP echo request, id 5376, seq 30, length 64
14:53:32.016426 IP 192.168.1.56 > 216.58.205.142: ICMP echo request, id 5376, seq 31, length 64
14:53:33.016722 IP 192.168.1.56 > 216.58.205.142: ICMP echo request, id 5376, seq 32, length 64
Where 192.168.1.56 is my docker container and 216.58.205.142 should be google ip address. No echo reply is received.
Macvlan is unlikely to work with IEEE 802.11.
Your wifi access point, and/or your host network stack, are not going to be thrilled.
You might want to try ipvlan instead: add -o ipvlan_mode=l2 to your network creation call and see if that helps.
That might very well still not work... (for eg, if you rely on DHCP and your DHCP server uses macaddresses and not client id)
And your only (reasonable) solution might be to drop the wifi entirely and wire the device up instead... (or move away from macvlan and use host / bridge - whichever is the most convenient)
I know I can't ping the macvlan interface from the same host, but I can't ping my container's macvlan interface from hosts on a different subnet (even though they're connected via a router).
Host IP: 10.8.2.132/22
Macvlan container IP: 10.8.2.250/22
Other host IP: 10.4.16.141/22
Ping FROM 10.8.2.132 TO 10.4.16.141 is successful
Ping FROM 10.8.2.250 TO 10.4.16.141 is successful
Ping FROM 10.4.16.141 TO 10.8.2.132 is successful
Ping FROM 10.4.16.141 TO 10.8.2.250 fails with 100% packet loss
ip route get 10.8.2.250 shows that there is a known route:
10.8.2.250 via 10.4.16.1 dev eth0 src 10.4.16.141
cache mtu 1500 hoplimit 64
How can I go about debugging this?
The docker macvlan network was created with:
docker network create -d macvlan --subnet=10.8.0.0/22 --gateway=10.8.0.1 -o parent=em1 macnet
and when I run the container I specifically add "--ip=10.8.2.250"
I have setup a docker swarm with 3 nodes (docker 18.03). These nodes use an overlay network to communicate.
node1:
laptop
host tun0 172.16.0.6 --> openvpn -> nat gateway
container n1
ip = 192.169.1.10
node2:
aws ec2
host eth2 10.0.30.62
container n2
ip = 192.169.1.9
node3:
aws ec2
host eth2 10.0.140.122
container n3
ip = 192.169.1.12
nat-gateway:
aws ec2
tun0 172.16.0.1 --> openvpn --> laptop
eth0 10.0.30.198
The scheme is partly working:
1. Containers can ping eachother using name (n1,n2,n3)
2. Docker swarm commands are working, services can be deployed
The overlay is partly working. Some nodes cannot communicate with each other either using tcp/ip or udp. I tried all combinations of the 3 nodes with udp and tcp/ip:
I did a tcpdump on the nat gateway to monitor overlay vxlan network activity (port 4789):
tcpdump -l -n -i eth0 "port 4789"
tcpdump -l -n -i tun0 "port 4789"
Then I tried tcp/ip communication from node2 to node3. On node3:
nc -l -s 0.0.0.0 -p 8999
On node1:
telnet 192.169.1.12 8999
Node1 will then try to connect to node3. I see packets coming in on the nat-gateway over the tun0 interface:
on the nat-gateway eth0 interface:
it seems that the nat-gateway is not sending replies back over the tun0 interface.
The iptables configuration the nat-gateway
The routing of the nat-gateway
Can you help me solve this issue?
I have been able to fix the issue using the following configuration on the NAT gateway:
and
No masquerading of 172.16.0.0/22 is needed. All the workers and managers will route their traffic for 172.16.0.0/22 via the NAT gateway, and it knows how to send the packets over tun0.
Masquerading of eth0 was just wrong...
All the containers can now ping and establish tcp/ip connections to each other.