I'm having an issue with docker swarm with it modifying iptables when i need control over the ports. I am trying to use UFW to make my own rules.
My setup has an nginx proxy that routes all traffic on designated ports to containers on different nodes to specific ports on a local network interface. So all of the node servers i want to block every single port on the public interface so all traffic has to come via the proxy server.
The problem is docker opens the ports on the public interface, let's say i have a container on 80:80 it opens port 80 on the public interface, when i don't want that server to be directly accessed, it needs to come in via the reverse proxy and down through the private network interface only.
I read with docker compose you can bind the port to the 127.0.0.1 ip address instead of letting docker bind to 0.0.0.0 like this:
"127.0.0.1:80:80"
However this doesn't work with docker swarm yaml config, when i try it gives this error:
error decoding 'Ports': Invalid hostPort: 127.0.0.1
This is causing me a headache, i don't want docker touching my iptables rules at all but i can't find a solid answer on how to stop it.
At the moment i am using OVH firewall directly on the ip addresses, however this isn't an ideal solution as OVH is basic and doesn't allow me to set port ranges which i need to do;
Related
From the Docker documentation:
--publish or -p flag. Publish a container's port(s) to the host.
--expose. Expose a port or a range of ports.
--link. Add link to another container. Is a legacy feature of Docker. It may eventually be removed.
I am using docker-compose with several networks. I do not want to publish any ports to the host, yet when I use expose, the port is then exposed to all the networks that container is connected to. It seems that after a lot of testing and reading I cannot figure out how to limit this to a specific network.
For example in this docker-compose file with where container1 joins the following three networks: internet, email and database.
services:
container1:
networks:
- internet
- email
- database
Now what if I have one specific port that I want to expose to ONLY the database network, so NOT to the host machine and also NOT to the email and internet networks in this example? If I would use ports: on container1 it is exposed to the host or I can bind it to a specific IP address of the host. *I also tried making a custom overlay network, giving the container a static IPv4 address and trying to set the ports in that format in ports: like - '10.8.0.3:80:80', but that also did not work because I think the binding can only happen to a HOST IP address. If i use expose: on container1 the port will be exposed to all three networks: internet, email and database.
I am aware I can make custom firewall ruling but it annoys me that I cannot write such simple config in my docker-compose file. Also, maybe something like 80:10.8.0.3:80 (HOST_IP:HOST_PORT:CONTAINER_IP:CONTAINER_PORT) would make perfect sense here (did not test it).*
Am I missing something or is this really not possible in Docker and Docker-compose?
Also posted here: https://github.com/docker/compose/issues/8795
No, container to container networking in docker is one-size-fits-many. When two containers are on the same network, and ICC has not been disabled, container-to-container communication is unrestricted. Given Docker's push into the developer workflow, I don't expect much development effort to change this.
This is handled by other projects like Kubernetes by offloading the networking to a CNI where various vendors support networking policies. This may be iptables rules, eBPF code, some kind of sidecar proxy, etc to implement it. But it has to be done as the container networking is setup, and docker doesn't have the hooks for you to implement anything there.
Perhaps you could hook into docker events and run various iptables commands for containers after they've been created. The application could also be configured to listen on the specific IP address for the network it trusts, but this requires injecting the subnet you trust and then looking up your container IP in your entrypoint, non-trivial to script up, and I'm not even sure it would work. Otherwise, this is solved by either restructuring the application so components that need to be on a less secure network are minimized, by hardening the sensitive ports, or switching the runtime over to something like Kubernetes with a network policy.
Things that won't help:
Removing exposed ports: this won't help since expose is just documentation. Changing exposed ports doesn't change networking between containers, or between the container and host.
Links: links are a legacy feature that adds entries to the host file when the container is created. This was replaced by creating networks with DNS resolution of other containers.
Removing published ports on the host: This doesn't impact container to container communication. The published port with -p creates a port forward from the host to the container, which you do want to limit, but containers can still communicate over a shared network without that published port.
The answer to this for me was to remove the -p command as that binds the container to the host and makes it available outside the host.
If you don't specify -p options. The container is available on all the networks it is connected to. On whichever port or ports the application is listening on.
It seems the -P forces the container on to the host and binds it to the port specified.
In your example if you don't use -p when staring "container1". "container1" would be available to the networks: internet, email, database with all its ports but not outside the host.
I know that the title might be confusing so let me explain.
The is my current situation:
Server A - 127.0.0.1
Server B - 1.2.3.4.5
Server B opens a reverse tunnel to Server A. This gives me a random port on Server A to communicate with the Server B. Let's assume the port is 1337.
As I mentioned to access Server B I am sending packets to 127.0.0.1:1337.
Our client needs a Telnet connection. Since Telnet is insecure but a requirement, we decided to use telnet OVER the ssh reverse tunnel.
Moreover, we created an alpine container with busybox inside of it to eliminate any access to the host. And here is our problem.
The tunnel is created on the host, yet the telnet client is inside a docker container. Those are two separate systems.
I can share my host network with the docker with -network=host but it eliminates the encapsulation idea of the docker container.
Also binding the docker to host like that -p 127.0.0.1:1337:1337 screams that the port is already in use and it can't bind to that (duh ssh is using it)
Mapping ports from host to the container are also not working since the telnet client isn't forwarding the traffic to a specific port so we can't just "sniff" it out.
Does anyone have an idea how to overcome this?
I thought about sharing my host network and trying to configure iptables rules to limit the docker functionality over the network but my iptables skills aren't really great.
The port forward does not work, because that is basically the wrong direction. -p 127.0.0.1:1337:1337 means "take everything thats coming in on that host-port, and forward it into the container". But you want to connect from the container to that port on the host.
Thats basically three steps:
The following steps require atleast Docker v20.04
On the host: Bind your tunnel to the docker0 interface on the host (might require that you figure out the ip of that interface first). In other words, referring to your example, ensure that the local side of the tunnel does not end at 127.0.0.1:1337 but <ip of host interface docker0>:1337
On the host: Add --add-host host.docker.internal:host-gateway to your docker run command
Inside your container: telnet to host.docker.internal (magic DNS name) on the port you bound in step 2 (i.e. 1337)
I have some docker containers talking together through docker bridge networks. They cannot be accessed from outside (I was said) as they are launched from a script with a default command which does not include 'expose' nor '-p' option. I cannot change that script.
I would like to connect to one of this containers which runs a server and listens for requests on port 8080. I tried connecting that bridge to a newly created docker bridge network, but i did not succede.
Now I am thinking of creating a new container and letting it talk to the server one (through bridge networks). As it is a new contaienr I can use the 'expose' or '-p' options, so it would be able to talk to the host machine.
Is it a good idea? How can I forward every request made to that container to the server one and get responses back to the host machine then?
Thanks
Within the default docker network, all ports are exposed. So you only need a container that exposes a port to the host machine and is in the same network as the other containers you have already created.
This is a relatively normal pattern. You can use a reverse proxy like nginx to achieve something like this.
There are some containers that automate this process:
https://github.com/jwilder/nginx-proxy
If you have no control over the other containers though, you will need to write the proxy config by hand.
If the container to which you are trying to connect is an http server, you may be able to use a ready-made container image that can work as an http forwarder (e.g., nginx - it is relatively easy to configure it as an http forwarder).
If you need plain tcp forwarding, you could make a container running 'socat' (socat can work as a tcp forwarder).
NOTE: in either case, you will be exposing a listener that wasn't meant to be on a public address. Do take measures not to allow unauthorized connections.
I am deploying a eureka server on a VM(say host external IP is a.b.c.d) as a docker image. Trying this in 2 ways.
1.I am running the docker image without explicit port mapping : docker run -p 8671 test/eureka-server
Then running docker ps command shows the port mapping as : 0.0.0.0:32769->8761/tcp
Try accessing the eureka server from outside of the VM with http://a.b.c.d:32769 , its not available.
2.I am running the docker image with explicit port mapping : docker run -p 8761:8761 test/eureka-server
Then running docker ps command shows the port mapping as : 0.0.0.0:8761->8761/tcp
Try accessing the eureka server from outside of the VM with http://a.b.c.d:8761 , its available.
Why in the first case the eureka server is not available from out side the host machine even if there is a random port(32769) assigned by docker.
Is it necessary to have explicit port mapping to have docker app available from external network ?
Since you're looking for access from the outside world to the host via the mapped port you'll need to ensure that the source traffic is allowed to reach that port on the host and given protocol. I'm not a network security specialist, but I'd suggest that opening up an entire range of ports simply because you don't know which port docker will pick would be a bad idea. If you can, I'd say pick a port and explicitly map it and ensure the firewall allows access to that port from the appropriate source address(es) e.g. ALLOW TCP/8671 in from 10.0.1.2/32 as an example - obviously your specific address range will vary on your network configuration. Docker compose may help you keep this consistent (as will other orchestration technologies like Kubernetes). In addition if you use cloud hosting services like AWS you may be able to leverage VPC security groups to help you whitelist source traffic to the port without knowing all possible source IP addresses ahead of time.
You either have the firewall blocking this port, or from wherever you are making the requests, for certain ports your outgoing traffic is disabled, so your requests never leave your machine.
Some companies do this. They leave port 80, 443, and couple of more for their intranet, and disable all other destination ports.
I am using Docker on OSX and have created a host with a bridged network and a couple of containers that share a virtual IP (so 172.19.0.50 points to 172.19.0.1 and if that container goes down, 172.19.0.50 is pointed at 172.19.0.2 and so on). Other containers within this network can access 172.19.0.50 and see either of the boxes as planned, so great so far.
Where I am confused is quite how to point my public 192.168.99.100 IP such that it goes to 172.19.0.50 rather than a specific container IP.
This is an area I'm not too familiar with so any advice much appreciated.
After some research, it seems to me that there's no "docker solution", for the moment. I have the exact same problem : I have a galera cluster on 3 containers on 1 docker host. Let's say you need to access the port 3306 on your containers, like me.
I manage a virtual IP on those galera nodes, and it works great. But I can't tell Docker to match the port's host (3306 in my case for mysql) to the virtual_ip:3306. The port from your host has to be mapped to a container and port. Not an IP address.
If you run the container that have the virtual ip, with port mapping like this :
docker run -d -p 3306:3306 docker_image /bin/bash
The requests to the public host IP address on 3306 will be redirected to the port 3306 of your container network interface (which have in theory 2 addresses, one static and the virtual one). But if your virtual ip moves to another docker, it won't change anything to your port mapping. The requests from the outside will be redirected to the first container anyway. (and, by the way, you can't map multiple containers to the same docker host's port. Actually, it wouldn't help)
In my opinion, you could use a HAProxy or Nginx reverse proxy in a container : so you can map the 3306's host port to the HAProxy's 3306 port, and HAProxy can redirect your requests to the cluster (with or without loadbalancing). So now you don't even need virtual IP.
BUT, now you have a nice single point of failure. It would be nice to add another reverse proxy as a backup, but then you would need a virtual IP for the failover, and you would be stuck with the problem of the beginning.
If somebody has a better solution to this...