I have a docker container which needs to add some iptables rules into the host. From searching, it seems like this is supposed to work either in privileged mode or by adding CAP_NET_ADMIN and CAP_NET_RAW and in host networking mode.
However, I tried both of these and no matter what I do the docker container seems to have it's own set of iptables rules. Here's an example:
on the host machine iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER-USER all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
# Warning: iptables-legacy tables present, use iptables-legacy to see them
(note that I ran docker with iptables set to false to try to debug this so it's a minimal set of rules, that setting doesn't seem to make a difference)
Next in an Ubuntu container: docker run -it --privileged --net=host ubuntu:18.04 /bin/bash same command (iptables -L)
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
so it's a totally different filter table, like it has it's own copy. Similar behavior for other tables and I've confirmed that adding rules in the container does not add them on the host even though the container is privileged and in host networking mode.
The host is a raspberry pi running Raspbian buster. Is there something else I need to do to make this work?
I should have thought of this earlier but I checked the raspbian kernel version and it was 4.19.something which is ancient at this point. So I re-installed with Ubuntu 20.04 server (which provides an arm64 distribution for the raspberry pi) and it seem to work as expected now. So likely it was something to do with the out-of-date kernel.
Related
I'm having an issue with a Ubuntu 22.10 host on which I installed Docker and have several services running (docker-elk).
I'm also running a simple "Hello world" HTTP server on port 3000:
GET http://localhost:3000 -> "Hello world"
I can access my Docker services for another computer on the same network using:
http://192.168.3.10:5601 -> Loads Kibana UI running in Docker on Host
However, I cannot access my simple HTTP server from that same computer:
http://192.168.3.10:3000 -> Refused to connect
From reading here looks like Docker makes some changes to iptables which probably has something to do with this:
I tried running that command, no success. Now iptables --list gives me:
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy DROP)
target prot opt source destination
DOCKER-USER all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (1 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere
RETURN all -- anywhere anywhere
I have previously "allowed" those ports using ufw but when that didnt work I also tried completely disabling the firewall using:
sudo ufw disable
sudo iptables -F
Even then I still cant reach my http server from the other computer on the same network.
I am using a really simple docker-compose file from here:
https://github.com/brandonserna/flask-docker-compose
this is the docker compose file:
version: '3.5'
services:
flask-app-service:
build: ./app
volumes:
- ./app:/usr/src/app
- .:/user/src
ports:
- 5555:9999
However I can only reach the app from outside network when I am using port 80.
ports:
- 80:9999
When I am using for example port 8000. I cant reach the container from outside network.
From the local machine I can reach the app. (Tested with wget localhost:8000)
ports:
- 8000:9999
iptables -L gives me this:
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER-USER all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (2 references)
target prot opt source destination
ACCEPT tcp -- anywhere 172.18.0.2 tcp dpt:9999
ACCEPT tcp -- anywhere 172.17.0.2 tcp dpt:http
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (2 references)
target prot opt source destination
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Not enough for comment so this is why:
From what it seems it could be either firewall rule in your host running the container or one between the host to your house.
To test which on between the two I'd try to use nmap with --reason and --tracerout options, since we have connectivity in another port it's unlikely that there is a complete block between your home and the container so the traceroute wouldn't give much info but just in case.
Also if you have root access to the host machine or just to the iptables service try to stop it to check if that's the root cause for the block.
also check with docker ps if the port is bound to the port on the machine, should look something like this:
0.0.0.0:port --> tcp\port
where instead of port you have the port number
If it doesn't maybe it's due to some problem with the docker-compose up command so try to run the service with a simple docker run command
Bear with me, I'm new to Docker...
I'm trying to get a Docker environment going on a Red Hat Linux server (7.6) and am having trouble accessing containers from a computer other than the host.
I got Docker installed no problem. Then, the first container I installed was Portainer and the Portainer Agent:
docker run -d -p 8000:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer
docker run -d -p 9001:9001 --name portainer_agent --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker/volumes:/var/lib/docker/volumes portainer/agent
Seems peachy:
# docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
973a685cfbe1 portainer/portainer "/portainer" 19 hours ago Up 2 minutes 0.0.0.0:8000->8000/tcp, 0.0.0.0:9000->9000/tcp portainer
602537dc21ec portainer/agent "./agent" 45 hours ago Up 19 hours 0.0.0.0:9001->9001/tcp portainer_agent
And using # curl http://localhost:9000 connects just fine. However, the connection gets dropped when attempting to connect from another computer on the same network (in a different subnet, if that matters). I can connect to the server just fine (I'm managing it via SSH, and even tested netcat on port 9002 for good measure).
The iptables, if this helps:
# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy DROP)
target prot opt source destination
DOCKER-USER all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (2 references)
target prot opt source destination
ACCEPT tcp -- anywhere 172.17.0.2 tcp dpt:etlservicemgr
ACCEPT tcp -- anywhere 172.17.0.3 tcp dpt:cslistener
ACCEPT tcp -- anywhere 172.17.0.3 tcp dpt:irdmi
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (2 references)
target prot opt source destination
DROP all -- anywhere anywhere
DROP all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
I've searched around a bit but keep finding conflicting answers (some suggesting that it should just work, and others suggesting that there's a lot more I've got left to learn and configure). I'm afraid that I'm fumbling in the dark. I gather that I need a route configured to forward host traffic to the container? Or an iptables rule? What exactly am I missing?
...Nevermind.
On a lark, I tried connecting to the server from a device that's on-premises; rather than my computer which is connected via VPN. The on-prem device connected fine.
I have enabled the privileged mode in the container and add a rule to it,
iptables -N udp2rawDwrW_191630ce_C0
iptables -F udp2rawDwrW_191630ce_C0
iptables -I udp2rawDwrW_191630ce_C0 -j DROP
iptables -I INPUT -p tcp -m tcp --dport 4096 -j udp2rawDwrW_191630ce_C0
and kt exec into the container and use iptables --table filter -L, I can see the added rules.
/ # iptables --table filter -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
udp2rawDwrW_191630ce_C0 tcp -- anywhere anywhere tcp dpt:4096
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain udp2rawDwrW_191630ce_C0 (1 references)
target prot opt source destination
DROP all -- anywhere anywhere
While when I logged into the node where the container lives, and run sudo iptalbes --table filter -L, I cannot see the same result.
I was thinking by default the previleged is dropped because the container might leverage it to change something like the iptables in the node, however it looks not like that.
So my question is "what is the relationship between K8S iptables and the one of a container inside a pod" and "why we stop user from modifying the container's iptables without the privileged field"?
if you want to manipulate node's iptables then you definitely need to put the pod on host's network (hostNetwork: true within pod's spec). After that granting to the container NET_ADMIN and NET_RAW capabilities (in containers[i].securityContext.capabilities.add) is sufficient.
example json slice:
"spec": {
"hostNetwork": true,
"containers": [{
"name": "netadmin",
"securityContext": {"capabilities": { "add": ["NET_ADMIN", "NET_RAW"] } }
I'm not sure if privileged mode has anything to do with manipulating host's iptables these days.
Looking at the iptables of my docker host i get something like this:
sudo iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp -- anywhere 172.17.0.2 tcp dpt:http-alt
I was able to assign a durable IP address like this:
sudo ip addr add 10.0.0.99/8 dev eth0
docker run -d -p 10.0.0.99:8888:8080 tomcat:8
but that address is only available on this machine, as in I need to ssh into it and ping it from this box.
Reading through this, it looks like i need to add a custom bridge:
Custom Docker Bridge
Is there a way to make the bridge hand fresh ips from the DHCP server? For example if my DHCP server assigned addresses from 10.1.1.x - I want to assign those addresses to Docker containers.
Would this involve a generic *nix way of pushing my iptable /etc/hosts ip addresses and dns names to a DNS server so other machines outside of the Docker cluster can ping those machines?
I have port forwarding working but need to do the same with the ip addresses as Zookeeper only tracks the internal Docker ip addresses and hostnames as I've defined them in my docker-compose.yml.