I have a manager and a worker node in a docker swarm and two domains. One domain is pointing to the manager node (and is accepting traffic and working as intended) the other domain is pointed at the worker node but the traffic is not being routed through the swarm. What needs to be configured to get the worker nodes domain traffic into the swarm?
I was thinking it had to do with the ingress network but that doesn’t seem to be the case. The ingress network just allows for inter-node communication from what I can gather.
The swarm is setup with a traefik proxy that routes based on domain. The domain for the worker node is setup in the traefik proxy but it is not getting any of its traffic.
Thank you!
The problem I was running into was the swarm was receiving the traffic except the traefik proxy wasn't routing the traffic to the correct service.
I was able to debug it by looking at the logs in the main proxy then worked backwards.
Related
Am trying to implement a cluster of containerised applications in the production using docker in the swarm mode.
Let me describe a very minimalist scenario.
All i have is just 5 aws-ec2 instances.
None of these nodes have a public IP assigned and all have private IPs assigned part of a subnet.
For example,
Manager Nodes
172.16.50.1
172.16.50.2
Worker Nodes
172.16.50.3
172.16.50.4
172.16.50.5
With the above infrastructure, have created a docker swarm with the first node's IP (172.16.50.1) as the --advertise-addr so that the other 4 nodes join the swarm as manager or worker with their respective tokens.
I didn't want to overload the Manager Nodes by making them doing the role of worker nodes too. (Is this a good idea or resource under-utilization?).
Being the nodes are 4 core each, am hosting 9 replicas of my web application which are distributed in the 3 worker nodes each running 3 containers hosting my web app.
Now with this setup in hand, how should i go about exposing the entire docker swarm cluster with a VIP (virtual IP) to the external world for consumption?
please validate my below thoughts:
1. Should I have a classic load-balancer setup like keeping a httpd or nginx or haproxy based reverse proxy which has a public IP assigned
and make it balance the load to the above 5 nodes where our
docker-swarm is deployed?
One downside I see here is that the above reverse-proxy would be Single Point of Failure? Any ideas how this could be made fault-tolerant/hightly available? should I try a AnyCast solution?
2. Going for a AWS ALB/ELB which would route the traffic to the above 5 nodes where our swarm is.
3. If keeping a separate Load Balancer is the way to go, then what does really docker-swarm load-balancing and service discovery is all
about?
what is docker swarm's answer to expose 1 virtual IP or host name to the external clients to access services in the swarm cluster?
Docker-swarm touts a lot about overlay networks but not sure how it
relates to my issue of exposing the cluster via VIP to clients in the
internet. Should we always keep the load balancer aware of the IP
addresses of the nodes that join the docker swarm later?
please shed some light!
On further reading, I understand that the Overlay Network we are creating in the swarm manager node only serves inter container communication.
The only difference from the other networking modes like bridge, host, macvlan is that the others enables communication among containers with in a single host and while the Overlay network facilitates communication among containers deployed in different subnets too. i.e., multi-host container communication.
with this knowledge as the headsup, to expose the swarm to the world via a single public IP assigned to a loadbalancer which would distribute requests to all the swarm nodes. This is just my understanding at a high level.
This is where i need your inputs and thoughts please...explaining the industry standard on how this is handled?
How does Traefik work in Docker Swarm Mode? From what I'm reading here and here, Traefik will be installed as part of the managers.
If there's 3 managers and each Docker Swarm manager is setup on a different host with different IP, what is the IP address of Traefik that domain DNS records need to point to? Is it the leader of all the managers? If so, if the leader fails and another manager gets appointed leader, then the IP address of Traefik needs to be updated in the domain DNS records?
The DNS may point to any of the nodes in the swarm and docker mesh will forward the requests to the node that runs the container/service in that moment.
You can read more about docker routing mesh here.
Quite possibly a very trivial question but I can't find anything in the documentation about a feature like this. As we know from the routing mesh documentation:
All nodes participate in an ingress routing mesh. The routing mesh enables each node in the swarm to accept connections on published ports for any service running in the swarm, even if there’s no task running on the node. The routing mesh routes all incoming requests to published ports on available nodes to an active container.
However, I do not wish some nodes to participate in the routing mesh, but I still want them to participate in hosting the service.
The configuration I'm trying to achieve looks a bit like this:
I have a single service, hello-world, with three instances, one on each node.
I would like, in this example, only node-1 and node-2 to participate in externalising the ingress network. However, when I visit 10.0.0.3, it still exposes port 80 and 443 as it still has to have the ingress network on it to be able to run the container hello-world, and I would like this not to be the case.
In essence, I'd like to be able to run containers for a service that hosts port 80 & 443 on 10.0.0.3 without being to access it by visiting 10.0.0.3 in a web browser. Is there any way to configure this? Even if there's no container running on the node, it'll still forward traffic to a container that is running.
Thank you!
The short answer to your specific question is no, there is no supported way to selectively enable/disable the ingress network on specific nodes for specific overlay networks.
But based on what you're asking to do, the expected model for using only specific nodes for incoming traffic is to control which nodes receive the traffic, not shutoff ports on specific nodes...
In a typical 6-node swarm where you've separated out your managers to be protected in a different subnet from the DMZ (e.g. a subnet behind the workers). You'd use placement constraints to ensure your app workloads were only assigned to worker nodes, and those nodes were the only ones in the VLAN/Security Group/etc. for being accessible from user/client traffic.
Most prod designs of Swarm recommend protecting your managers (which manage the orchestration and scheduling of containers, store secrets, etc.) from external traffic.
Why not put your proxies on the workers in a client-accessible network, and have those nodes the only in DMZ/external LB.
Note that if you only allow firewall/LB access to some nodes (e.g. just 3 workers) then the other nodes that don't receive external incoming traffic are effectively not using their ingress networks, which achieves your desired result. The node that receives the external connection uses its VIP to route the traffic directly to the node that runs the published container port.
I want to setup a failover for a webservice I've written.
Everywhere i read that docker swarm 1.12 does automatic failover but I think only for failed containers.
How should i configure public DNS for my service?
How does failover work if a host is down?
With normal dns round robin and IPs of the nodes it won't work. every nth request will fail for n servers. The docker route mesh doesn't help if one host is down. Or do i miss something here?
FYI: Currently I'm not using docker swarm but I'm planing to do so.
You got a couple of options:
If you are using a cloud provider like amazonaws, you can use ELB or what ever loadbalancer your cloudprovider provides, They usually support mode TCP and healthchecks. Then you will point your dns to load balancer. This way when ever a node fails, it will be removed from load balancer via healthchecks and will be back when it is healthy again.
You can use your own load balancer. HAproxy and Nginx support TCP mode and you can set healthcheck rules. when ever a node fails, it will be removed from load balancer via healthchecks and will be back when it is healthy again. Make sure HighAvailablity is provided via tools like keepalived so the loadbalancer wont become a point of failure.
Please note this applies to TCP traffic as many loadbalancers dont support UDP traffic.
So I've got a Plex server running on my Docker swarm!! If I kill a node magically it'll start Plex somewhere else. This is great! Now comes the fun part...
With old-school containers I would just port forward port 32400 on my router to the server that was running Plex and it would work find. Now that Plex can run in multiple different places I need to figure out how to forward the port to some static resource. I could use HAProxy to bind some bridge interface and run it on every node to provide failover...but I'd like to see if there's an easier way to accomplish this.
What's the best way to forward ports to services in Docker Swarm?
Port forwarding is built into the new swarm mode. There's a section on load balancing in the documentation:
The swarm manager uses ingress load balancing to expose the services
you want to make available externally to the swarm. The swarm manager
can automatically assign the service a PublishedPort or you can
configure a PublishedPort for the service in the 30000-32767 range.
External components, such as cloud load balancers, can access the
service on the PublishedPort of any node in the cluster whether or not
the node is currently running the task for the service. All nodes in
the swarm cluster route ingress connections to a running task
instance.
Swarm mode has an internal DNS component that automatically assigns
each service in the swarm a DNS entry. The swarm manager uses internal
load balancing to distribute requests among services within the
cluster based upon the DNS name of the service.
Update
The following article discusses how to integrate a proxy load balancer into the docker engine
https://technologyconversations.com/2016/08/01/integrating-proxy-with-docker-swarm-tour-around-docker-1-12-series/