Docker Swarm Routing Mesh - docker

Assuming that I have 2 nodes in the swarm (Node 1 is a manager, node2 is a worker), and using the following compose to launch
version: "3.9"
services:
app1:
image: app1image
ports:
- 8080:8080
deploy:
mode: global
app2:
image: app2image
ports:
- 9080:9080
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- "node.role==manager"
My questions are:
If I try to access app1 through node1 could I be routed to the app1 container in node2?
Since the app2 only deploys to node1, if I try to access it through node2 on port 9080 will I be able to?
Besides ports referenced by the docker documentation(TCP port 2377 for cluster management communications
TCP and UDP port 7946 for communication among nodes
UDP port 4789 for overlay network traffic) are there any other ports that need to be opened? Like in case app1 wants to call app2

So, to understand whats actually going on:
version: "3.9"
networks:
default:
driver: overlay
ingress:
external: true
services:
app1:
image: app1image
ports:
- 8080:5000
deploy:
mode: global
networks:
- default
app2:
image: app2image
ports:
- 9080:5000
networks:
- default
deploy:
placement:
constraints:
- node.role==manager
In this configuration my expectation is that the app is listening on 0.0.0.0:5000.
So, what docker has done is created two networks: an ingress network that is used to bridge ports on each host, to each container:
node1:8080 node2:8080 will be routed and loadbalanced to app1 containers.
and
node1:9080, nod2:9080 will be routed and loadbalanced to app2 containers.
The service containers, or tasks, also have been attached to an implicit default network for the compose stack. Its an overlay - or software defined - network so each container has an ip on that network that is unrelated to the node its on. I have decided that the actual listen port is port :5000 for both services, so any services attached to {stack}_default will be able to use the servicename, and the actual port address:
app1:5000 will route via a vip to loadbalance traffic to instances of app1, and app1.tasks is a dnsrr record that will return each container ip.
Likewise app2:5000 will route to the app2 container on the manager node.
The app1 and app2 dns names are entirely private to services that are part of the stack / attached to the {stack_default} network so the app1:5000 names are not available external to the swarm, or even to other stacks or containers that are not explicitly attached.
So:
yes.
yes.
no but:
If you ports: to publish ports, those ports are external to docker and do not go through the overlay network. You would need to add every port published to the firewalls if required for node to node comms. e.g. 8080 and 9080 need to be open.
However, because overlay network allows connections, uses 4789 at the physical link layer, the traffic goint to app1, and app2 ips (the :5000 traffic) on the overlay is tunneled and does not need to be opened.

Related

docker-compose - How to specify which network for listening port?

I'm using:
docker-compose 3.7
docker engine 18.09.7
In a docker-compose file, how do I specify which network I want a specific listening port bound to?
For example:
version: "3.7"
services:
service-a:
image: service-a:0.1.0
networks:
- network1
service-b:
image: service-b:0.1.0
networks:
- network1
- network2
expose:
- "8000"
- "9000"
ports:
- target: 8000
published: 8000
protocol: tcp
mode: host
- target: 9000
published: 9000
protocol: tcp
mode: host
service-c:
image: service-c:0.1.0
networks:
- network2
networks:
network1:
network2:
In this contrived example service-b is listening on port 8000 and 9000.
Is there a way to specify that port 8000 is only accessible on network1 while 9000 is only accessible on network2?
This would be most helpful in the case where a server listens on, say 0.0.0.0 as the host.
So if I get this right what you want to achive is to grant service-a access to port 8000 of service-b but block any access from service-a to port 9000 of service-b. And the same for service-c but the other way around?
For this you first need to know how the networking with docker-compose works: for each network under the networks section docker-compose (in this case) creates a virtual network connecting a virtual network device of the host machine to it as well as a virtual network device of each container contected to the network. Each of these virtual devices can communicate directly with each other in the same virtual network while the different virtual networks are usally isolated from each other.
The expose keyword now does not actually expose any ports but instead only documents the intent that a process will listen on that port(s). You can examine this information about a container using docker inspect. Besides the added meta-data expose does not actually do much more, see the documentation. So in this case it has no real use.
The ports keyword on the other hand does expose the listed ports to ports on the host machine - see the docs. Since the containers communicate directly via their share networks this is again not of real use for your scenario.
There are also no other configuration options which are intended to limit the communication of containers within the same network, i.e. there is no officially supported way to do this nicely.
One way to do this would be to modify the application itself to not listen on 0.0.0.0 with each port but only bind to the address of the respective network (network1/network2). But this requires application-specific changes and to somehow detect the correct address for each port.
Another way would be to inject your own iptables rules to block undesired access between containers, see the docs on this. The downside of this is that it has to be done completely outside of docker and docker-compose.
And lastly there is this hackish solution: instead of blocking undesired access only allow for explicitly whitelisted ports:
version: "3.7"
services:
service-a:
image: service-a:0.1.0
networks:
- network1
service-b:
image: service-b:0.1.0
networks:
- network2
ports:
- 172.101.0.1:8000:8000
- 172.103.0.1:9000:9000
service-c:
image: service-c:0.1.0
networks:
- network3
networks:
network1:
ipam:
config:
- subnet: 172.101.0.0/24
network2:
network3:
ipam:
config:
- subnet: 172.103.0.0/24
This works by assigning each container to its very own network completely isolating them from each other. But for network1/network3 we explicitly configure the subnet so we know the gatway IPs (172.101.0.1/172.103.0.1) of them which are assigned to the virtual network devices of the host.
Now we can "expose" the ports 8000/9000 of the service-b container to these host IP addresses, i.e. port 8000 on 172.101.0.1 will be forwarded to port 8000 of the service-b container. 172.101.0.1 belongs to the host but is part of network1 and thus can be accessed by service-a allowing it to only access that one port of service-b.

Docker Per Network Port Mapping

I'm looking for a way to map the same port into 2 different ports, each for another container in a different network.
consider the below docker-compose scenario:
services:
web:
build: .
ports:
- "8080:8080"
networks:
Net1:
Net2:
serv1:
image: tomcat:7.0.92-jre8
networks:
Net1:
serv2:
image: tomcat:7.0.92-jre8
networks:
Net2:
Now what I would really like to do is to actually map the "web" service port 8080 so that serv1 could consume it as 8081 and serv2 will be using it as 8082.
Is that even possible?
Thanks
Ports are published to the host, not to docker networks, and not to other docker containers. So the above "8080:8080" maps port 8080 on the docker host into that container's port 8080.
For container-to-container communication, that happens using docker's internal DNS for service discovery, and the container port. So both serv1 and serv2 can connect to http://web:8080 to reach the web service on its container port. That in no way prevents serv1 and serv2 from listening within their own container on any ports they wish.

Can't access container IP from host docker for windows stack

I'm using the following docker compose file to build my docker swarm stack that have windows containers deployed in a Windows 10:
version: '3.2'
services:
service1:
image: myrepository/dotnet-framework:3.5-windowsservercore
environment:
- my_path="C:/app/build/app.exe"
- my_arg= 1
deploy:
replicas: 1
placement:
constraints:
- node.id == asdfasdgasgasg
volumes:
- service1:C:/app
service1:
image: myrepository/dotnet-framework:3.5-windowsservercore
ports:
- target: 7878
published: 7878
mode: host
environment:
- my_path="C:/app/app.exe"
- my_arg= 2
deploy:
replicas: 1
placement:
constraints:
- node.id == asdfasdgasgasg
volumes:
- service1:C:/app
volumes:
service2:
external:
name: service1
service1:
external:
name: service1
As you can see service2 is listening in port 7878. I know, as is shown in this post, that I can't reach this port using localhost:7878. Thus I run the command docker inspect containerID to figure out the IP address of the container.
If I ping the container service2 from service1, it responds. But If I try to access the port 10.0.3.18:7878 from the host, there's no response. How could I reach port 7878from the hots? On the other hand, I have Linux containers that must reach the 'service2' windows container.
Each of the docker containers in the service can communicate with each other by default as they are started up on their own private network. That is why you can ping between the service containers.
The port 7878 you opened up will also be accessible to the host windows 10 os via the host machine’s ip address not the container ip address. The container’s IP address is private even to the host os.
Ping may not work as you have not opened up the ping port in the service and there may not be a ping service in the image to respond to your ping request. I may be wrong on this last point. Ping is not a good method to verify if a container is working or not.
Windows updated and everything works as expected.

Docker compose v3 specify network to be ingress?

I have docker-compose file for my docker swarm stack.I want my stack services to be deployed on specific network and that network i want to be ingress network so that i can use DNSRR of docker.
version: "3"
services:
a:
image: xyz/a:dev
ports:
- "80:80"
- "443:443"
networks:
-my_network
b:
image: xyz/b:dev
ports:
- "5000:5000"
networks:
-my_network
networks:
my_network:
driver:overlay
ipam:
driver: default
config:
-subnet: 10.0.1.0/24
here, where can i specify that this network should be ingress network? plus how can i specify ip-range same as specified here in cli : here
Apparently, these options are not exposed in the composefile. The documentation doesn't mention them and to be sure you can check the source code, in particular the latest compose schema.
The only option is to create the ingress network on the command line and reference it from the compose file as an external network.
The ingress network is only for manager/worker nodes.
The routing mesh routes all incoming requests to published ports on available nodes to an active container.
https://docs.docker.com/engine/swarm/ingress/
What you want is an addition overlay network. All services which are assigned to the same overlay network which is not ingress, can talk to each other over the name you've given them.
E.g.
docker service create --name A --network dev ...
docker service create --name B --network dev ...
Service A can simply do ping B.
Example of how to specify port ranges:
ports:
- "9090-9091:8080-8081"
The long form allows specifying the mode as either host or ingress.
ports:
- target: 80
host_ip: 127.0.0.1
published: 8080
protocol: tcp
mode: ingress
See the Ports section of the docker-compose spec for more details.

Traefik as a proxy for Docker container with host machines network

I would like to set up the following scenario:
One physical machine with Docker containers
traefik in a container with network backend
another container which is using the host machines network (network_mode: host)
Traefik successfully finds the container and adds it with the IP address 127.0.0.1 which obviously not accessible from the traefik container (different network/bridge).
docker-compose.yml:
version: '3'
services:
traefik:
image: traefik
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik.toml:/etc/traefik/traefik.toml
networks:
- backend
app:
image: my_app
labels:
- "traefik.enable=true"
- "traefik.frontend.rule=Host:myapp.example"
- "traefik.port=8080"
network_mode: host
networks:
backend:
driver: bridge
The app container is added with
Server URL Weight
server-app http://127.0.0.1:8080 0
Load Balancer: wrr
Of course I can access app with http://127.0.0.1:8080 on the host machine or with http://$HOST_IP:8080 from the traefik container.
Can I somehow convince traefik to use another IP for the container?
Thanks!
Without a common docker network, traefik won't be able to route to your container. Since you're using host networking, there's little need for traefik to proxy the container, just access it directly. Or if you need to only access it through the proxy, then place it on the backend network. If you need some ports published on the host and others proxied through traefik, then place it on the backend network and publish the ports you need to publish, rather than using the host network directly.

Resources