all. I'm learning Docker. But still cannot find any documentations about how Docker ingress network connect several separated hosts.
I have 2 VMs in different datacenters and want create swarm cluster on them.
Is it possible that default installed ingress network makes containers on vm1 visible for containers on vm2 inside some overlay network? Or both vm1 and vm2 should be in same local network?
In general, it's not recommended to span datacenters within a Swarm. You can span availability zones (datacenters in same geo area that are ~10ms or less latency) but between regions should be their own Swarms. This is 100% a latency issue of inter-virtual-network traffic (overlay driver) and the Raft consensus traffic between Swarm managers. There is no hard limit on latency, but you likely don't want the complexity in a single Swarm of trying to prevent traffic in your apps from hopping back and forth between datacenters... unless the datacenters are very low latency.
For more data on this look at the Docker Success site (search swarm overlay and filter to reference), as the Docker EE requirements for Swarm are the same as Docker CE generally.
The other requirement between nodes in a Swarm is that they have ports open between each other's public IP's. Ideally, there is no NAT between nodes.
If both hosts are part of the same docker swarm cluster then from perspective of docker it does not matter that they are in different data centers. Routing between services will just work. For example service1 on host1 will be able to access service2 in another data center. You might however need to account for any possibly large latencies that would occur because of physical distance of hosts.
It is also the same story with the ingress network. It does not care that there are 2 data centers. Any swarm cluster node will participate in it and route incoming requests to the correct service/host.
Related
We have two docker swarm clusters in production in different virtual local area networks (for example LAN and DMZ) on own server infrastructures (not cloud).
And we need to monitor them (swarm's machines and services) from one dedicated monitoring solution to consolidate all metrics in one dash board (yeah, we need dash board, out of the box)
It need to be fault tolerance and need to be isolated from both swarms, of course.
Because one of networks is DMZ, we can't do any outgoing connection from DMZ to LAN for monitoring-agents (agents on DMZ's Swarm cluster) - it seems, we need outgoing connection from main monitoring service to agents.
Could u help us to find best and maybe most popular monitoring solutions for Docker Swarm that fits to described requirenments?
Try Portainer. It has two types of agents: one (edge) needs to connect to server instance (which you don't want) but the other doesn't, see https://portainer.readthedocs.io/en/stable/agent.html
Portainer agents need to have access to docker socket on machines so when installed on swarm managers, you can manage as many swarms need.
It's free and opensource so if you need something that's missing, you can always build a custom fork :)
Am trying to implement a cluster of containerised applications in the production using docker in the swarm mode.
Let me describe a very minimalist scenario.
All i have is just 5 aws-ec2 instances.
None of these nodes have a public IP assigned and all have private IPs assigned part of a subnet.
For example,
Manager Nodes
172.16.50.1
172.16.50.2
Worker Nodes
172.16.50.3
172.16.50.4
172.16.50.5
With the above infrastructure, have created a docker swarm with the first node's IP (172.16.50.1) as the --advertise-addr so that the other 4 nodes join the swarm as manager or worker with their respective tokens.
I didn't want to overload the Manager Nodes by making them doing the role of worker nodes too. (Is this a good idea or resource under-utilization?).
Being the nodes are 4 core each, am hosting 9 replicas of my web application which are distributed in the 3 worker nodes each running 3 containers hosting my web app.
Now with this setup in hand, how should i go about exposing the entire docker swarm cluster with a VIP (virtual IP) to the external world for consumption?
please validate my below thoughts:
1. Should I have a classic load-balancer setup like keeping a httpd or nginx or haproxy based reverse proxy which has a public IP assigned
and make it balance the load to the above 5 nodes where our
docker-swarm is deployed?
One downside I see here is that the above reverse-proxy would be Single Point of Failure? Any ideas how this could be made fault-tolerant/hightly available? should I try a AnyCast solution?
2. Going for a AWS ALB/ELB which would route the traffic to the above 5 nodes where our swarm is.
3. If keeping a separate Load Balancer is the way to go, then what does really docker-swarm load-balancing and service discovery is all
about?
what is docker swarm's answer to expose 1 virtual IP or host name to the external clients to access services in the swarm cluster?
Docker-swarm touts a lot about overlay networks but not sure how it
relates to my issue of exposing the cluster via VIP to clients in the
internet. Should we always keep the load balancer aware of the IP
addresses of the nodes that join the docker swarm later?
please shed some light!
On further reading, I understand that the Overlay Network we are creating in the swarm manager node only serves inter container communication.
The only difference from the other networking modes like bridge, host, macvlan is that the others enables communication among containers with in a single host and while the Overlay network facilitates communication among containers deployed in different subnets too. i.e., multi-host container communication.
with this knowledge as the headsup, to expose the swarm to the world via a single public IP assigned to a loadbalancer which would distribute requests to all the swarm nodes. This is just my understanding at a high level.
This is where i need your inputs and thoughts please...explaining the industry standard on how this is handled?
I see a lot of examples running multiple Docker nodes in swarm mode, but they all mention that the nodes shares a local/private network. I was wondering, is it possible to connect two hosts on a swarm that are not on a private network but can still reach each others over IP and having the correct ports setup ?
This would not be for a production setup.
Are there any Swarm mechanisms that prevent such architecture ?
Thank you for your time !
You can connect swarm nodes over the public internet. What's needed is:
Routeable IP addresses for each node, this may require a VPN between nodes
Firewall rules to allow 2376/tcp, 7946/tcp+udp, 4789/udp between each node
Low latency, if the heartbeat timeout is exceeded, nodes will be flagged as down and workload will migrate
Because of the last requirement, typically people will install nodes in the same region but multiple AZ's. And when you get to multiple regions, you typically see multiple clusters to keep the latency down within a cluster.
Running this command helped me have all nodes available across all continents:
sudo docker swarm update --dispatcher-heartbeat 120s
Quite possibly a very trivial question but I can't find anything in the documentation about a feature like this. As we know from the routing mesh documentation:
All nodes participate in an ingress routing mesh. The routing mesh enables each node in the swarm to accept connections on published ports for any service running in the swarm, even if there’s no task running on the node. The routing mesh routes all incoming requests to published ports on available nodes to an active container.
However, I do not wish some nodes to participate in the routing mesh, but I still want them to participate in hosting the service.
The configuration I'm trying to achieve looks a bit like this:
I have a single service, hello-world, with three instances, one on each node.
I would like, in this example, only node-1 and node-2 to participate in externalising the ingress network. However, when I visit 10.0.0.3, it still exposes port 80 and 443 as it still has to have the ingress network on it to be able to run the container hello-world, and I would like this not to be the case.
In essence, I'd like to be able to run containers for a service that hosts port 80 & 443 on 10.0.0.3 without being to access it by visiting 10.0.0.3 in a web browser. Is there any way to configure this? Even if there's no container running on the node, it'll still forward traffic to a container that is running.
Thank you!
The short answer to your specific question is no, there is no supported way to selectively enable/disable the ingress network on specific nodes for specific overlay networks.
But based on what you're asking to do, the expected model for using only specific nodes for incoming traffic is to control which nodes receive the traffic, not shutoff ports on specific nodes...
In a typical 6-node swarm where you've separated out your managers to be protected in a different subnet from the DMZ (e.g. a subnet behind the workers). You'd use placement constraints to ensure your app workloads were only assigned to worker nodes, and those nodes were the only ones in the VLAN/Security Group/etc. for being accessible from user/client traffic.
Most prod designs of Swarm recommend protecting your managers (which manage the orchestration and scheduling of containers, store secrets, etc.) from external traffic.
Why not put your proxies on the workers in a client-accessible network, and have those nodes the only in DMZ/external LB.
Note that if you only allow firewall/LB access to some nodes (e.g. just 3 workers) then the other nodes that don't receive external incoming traffic are effectively not using their ingress networks, which achieves your desired result. The node that receives the external connection uses its VIP to route the traffic directly to the node that runs the published container port.
Is there any known solution for enabling an external machine to connect to the containers on a docker swarm overlay network?
The question is legitimate, see example below, however I do not know of a simple solution for it, I'll propose offhand a possible solution and would test later and update.
Suppose you have a docker overlay network of many Kafkas running on a
couple of nodes (container hosts). All Kafka brokers communicate with
each other beautifully.
When a Kafka client needs to access a Kafka broker it connects
to it (say somehow, supposedly even through Swarm's service external
port), but then that broken may reply that that data is in another
broker with and here is that other broker's IP (on the overlay
network)... meaning Kafka client must be able to access to all Kafka
brokers (overlay network).
You can do this easily if everything is containerized, but what if not?
You can do this with SDN, or an offhand solution:
A container with two networks serving as a router with one "leg" on the overlay network and the other l2bridged to where that other VM or host is and route through it, you'd have to Swarm "constrain" it to run where the network from which you want overlay network access is available. that should work!
If someone has another clean/clear solution I'm very interested too