I want to move all of my operations over to K8S for so long, but am still hesitant to that. This question will likely be broad, but bear with me. Let me first describe the existing system.
I hosts a lot of different websites (>30). A lot of that for my own experimentation, but some are for actual clients. I have 1 VM in New York (I'm using DigitalOcean), with multiple Docker containers, frequently managed using docker-compose. There is 1 container for every site. The request first comes in to front container running HAProxy. This strips away SSL, then forwards the request to 2 proxy container running Nginx. These 2 container then forwards the request to all the other containers for their service. All of my certificates come from LetsEncrypt, and have to be renewed every 3 months. To do so, I stop front, run certbot --apache so it binds to port 80. It gets the certificates, then I stop apache, then recreate front container.
There are several reasons to why I do it this way:
I change site configs a lot, and how all of them are wired together. So front is expected to run forever, unless I'm getting certificates, and proxys are expected to change a lot. I change the proxy image, then stops and recreates the 1st container, then stops and recreates the 2nd container, so that there will be no downtime at all.
I really don't know how to get certificates when there are multiple nodes. In fact, I'm a total noob at the whole certificate thing and LetsEncrypt is pretty much the only way I know of to do this.
I want to directly edit files on the remote server. I have a bad practice of editing production code directly, mainly because I get impatient with setting up dev, staging and production environments. It takes too much time, and the gains feels small. And for clients, they are typically small businesses, with <10 employees, and regularly, they want to have some aesthetic changes to the websites. I can have a video call with them, they tell me exactly what they want, I code that in, it gets uploaded to the server immediately, and they see changes right away. Then they can critique the design, and we can iterate back and forth. If I were to setup different environments, they can't see it right away, and there has to be this long process of committing to git, deploy to staging, then production. This takes a long time, and I don't think is justified.
I realize that my systems are not that well maintained. Images are not getting security updates, I don't know if they are still running or not unless I check for them manually, which is tedious, so I don't do them at all. Furthermore, I have an Asian background, that means I have clients from both the US and Asia, pretty much the farthest place possible from each other, which increases latency by a lot. That means client in Asia has to wait for around 1-2 second for the page to actually load, which is eternal. I have also moved to Asia in the past week, so now, accessing the New York server via ssh is incredibly slow, and my productivity just plummets. So now it might be the best time to revamp everything, and move to K8S once and forever. However, there are major problems in the planning process and currently, K8S seems to lack a lot of stuff that are just deal breakers for me. So please criticize my plans, and improve them however you see fit.
What I plan to do now is this:
There will be 2 servers, 1 at New York, 1 at Singapore. These 2 severs will have 2 different ip addresses. Those 2 will be running K8S Pods. Preferably, they should have exactly the same configs, website containers, database containers, etc. Then for each website DNS record, I will modify A and AAAA records so that they contain 2 ip addresses for the 2 servers.
My question is:
Will DNS always route to Singapore if user is in China, and always route to New York if user is in England?
How to actually get certificates for 2 nodes? My understanding is that when certbot issues a certificate, it associates the domain name with the node ip address. That means 2 nodes can't have the same certificate for the same domain name. Is this correct? If you can get certificates for 2 nodes then how to do that?
How to keep files in sync between servers? Say I edit the file tree in Singapore server, I want that file to also be modified in New York several seconds later. For databases, I can have a master database at either Singapore or New York, then have slave databases at both locations that updates whenever the master updates, and the slaves can serve as a low latency database for each server.
How to actually route requests from servers to containers inside. I initially plan to use NodePort, to direct the request to front Pods, then that can distribute requests to other Pods, but I was heartbroken when NodePort can't attach to ports below 30000. The only other option that I am aware of is to have an external load balancing service that directs traffic to the 2 servers. But that costs like $15/site/month, and because I have >30 sites, doing so will bankrupt me. I can also have 4 servers in total, 2 for the K8S cluster, and 2 serves as a load balancer that will forward to NodePort. Will this plan works? How will automatic renewing of certificates even work here?
Please note that may be my questions are the wrong questions to ask (like, may be I shouldn't use A and AAAA records for directing traffic), and there's a different way to do this entirely, so feel free to ask the right questions.
read your question hats off to write down the whole stuff but half of the stuff is useless.
Answers of your question :
Can we add the same or multiple entries in DNS? example.com with A record multiple times possible?
You might require to set up a regional K8s cluster with regional ingress support. you can use certmanager with letsencrypt which will manage your cert at LB level and terminate it at the front.
If you are looking forward to use two VMs put one LB in front of both and set SSL over there.
if you are using K8s with stateless PODs editing direct file inside container is not a option. better you manage the Github update inside and container get deployed on to both cluster at a same time for that you can setup CI/CD. You are right in case of database server setup with master slave concept you can use read replicas.
To route the traffic from server to internal application of K8s you can an internal LB or exposing services with node ports(above 30000 but change target port in SVC) and route the port if you want to redirect requests on a specific port using the target port.
still, i am not getting "I can also have 4 servers in total, 2 for the K8S cluster, and 2 serves as a load balancer that will forward to NodePort. Will this plan works? How will automatic renewing of certificates even work here?" which server will be in front and which one in the backend.
If all your services are websites (run over http) you could use k8s ingress to route traffic to pods based on Host header (domain name) and use only one LB with one IP address. The most popular ingress controller seems to be the Nginx Ingress Controller
If you don't want to use LB you can use hostPort to expose nginx ingress but as soon as you have k8s cluster with more than one node, use LB because hostPort is generally not advised to use unless you have a very good reason to do so.
Speaking of DNS, you can use sth like AWS route53 routing policies for location routing. You don't necessarily need to use AWS. I just want to show you that there are solutions to this problem, but use whatever you like.
For certificates use cetrmanager with DNS-01 challenge.
From letsencrypt docs about DNS-01 challenge:
It works well even if you have multiple web servers.
cetrmanager will also handle certificate renewal for you.
About keeping files in sync between servers; It depends on files, but for static content it might be best to use CDN that will replicate content from one source to other locations.
For simultanous deploys to 2 separate clusters you can use some CI/CD pipeline like e.g. github actions.
I am new to docker-swarm and very much interested in knowing inner workings of how docker-swarm distributes the incoming requests for one single service.
For example, I deployed stack with one single service and 10 replicas across 2 nodes. when I brought up the node 5 containers did show up on node1 and other 5 on node2. Now, I make 10 http requests to the same service from 10 different browser instances, does each container end up with one request each? If it was a round-robin, I would think so. However, I am not observing the same behavior from the stack that I have just deployed.
I brought up the stack with configuration above and made 10 requests. When I did that load was mire distributed than concentrated but only 7 of the containers got 10 requests and 3 were free.
This tells me that it is not even-distribution round robin. If not, what algorithm does docker services api follow to determine which container will serve the next request?
When I searched for inner workings of docker swarm, I ended up with article here : https://docs.docker.com/engine/swarm/ingress/
which is interesting look into ingress and routing mesh but still does not answer my original question.
Anyone?
I’m doing on-prem deployments using docker swarm and I need application and DB high availability.
As far as application HA is concerned, it works great within docker (service discovery and load balancing), but I’m not sure how to use it on my network. I mean how can I assign a virtual IP to all of my docker managers so that if any of them goes down, that virtual IP automatically points to the other docker manager in the cluster. I don’t want to have a single point of failure in my architecture, that’s why I’m not inclined to use any (single) reverse proxy solution in front of my swarm cluster (because to my understanding, if nginx/HAProxy goes down, the whole system goes into abyss. I would love to know that I’m wrong).
Secondly, I use WebSockets in my application for push notifications which doesn’t behave normally with all the load balancing stuff because socket handshakes get distorted.
I want a solution to these problems without writing anything in code (HA-specific and non-generic like hard coding IPs etc). Any suggestions? I hope I explained my problem correctly.
Docker Flow Proxy or Traefik can be placed on a set of swarm nodes that you want to receive traffic for incoming connections, and use DNS routing to get packets to the correct containers. Both have sticky sessions option (I know Docker Flow does, not sure about Traefik).
Then you can either:
If your incoming connections are just client HTTP/S requests, you can use DNS Round Robin with multiple A records, which works great, or
By an expensive hardware fault tolerant reverse proxy like F5
Use some network-layer IP failover that is at the OS and physical network level (not related to Docker really), but I'm not sure how well that would work with Swarm.
Number 2 is the typical solution in private datacenters that need full HA at all layers.
All,
We recently had an issue with ELB HealthCheck in covering up a certain use-case or scenario which caused an application impact.
Can anyone suggest a fault-tolerant approach to handle this?
We have a nodeJS app running in a port - 80
We have 3 instances in the Target Group & that is enrolled in ELB.
ELB HealthCheck was configured to hit root path on port 80 and return success if it gets HTTP 200
Recently one of the node had 100% disk filled on application mount and root mount was still having space.
Though the HealthCheck was succeeding as per ELB the server didn't respond for any other services and it was ideally unhealthy. This means that there are some requests that got
succeeded but some of them failed (that was routed to this disk-filled server).
We did received notifications from other monitoring systems on disk filling but due to overwhelming emails & limited resources it got missed out.
Is there any other way we can improvise the HealthCheck strategy to just have these scenarios intimated to AutoScaling Group or ELB
so that we can target these nodes to be removed and replace them automatically?
Rather than just checking that the index.htm page is returning a 200 response, you can configure Elastic Load Balancing to point to a customer Health Check page (eg healthcheck.php).
You could run some code on that page to test the general health of the application (database connectivity, disk space, free memory). If everything checks out OK, return a 200 response. If something is wrong, return a 500 response. This will cause the Load Balancer to treat the instance as Unhealthy and it will stop serving traffic to the instance.
If Auto Scaling is configured to use the ELB Health Check, then Auto Scaling will terminate the unhealthy instance and automatically replace it with a new instance.
My question is related to microservices & service discovery of a service which is spread between several hosts.
The setup is as follows:
2 docker hosts (host A & host B)
a Consul server (service discovery)
Let’s say that I have 2 services:
service A
service B
Service B is deployed 10 times (with random ports): 5 times on host A and 5 times on host B.
When service A communicates with service B, for example, it sends a request to serviceB.example.com (hard coded).
In order to get an IP and a port, service A should query the Consul server for an SRV record.
It will get 10 ip:port pairs, for which the client should apply some load-balancing logic.
Is there a simpler way to handle this without me developing a client resolver (+LB) library for that matter ?
Is there anything like that already implemented somewhere ?
Am I doing it all wrong ?
There are a few options:
Load balance on client as you suggest for which you'll either need to find a ready-build service discovery library that works with SRV records and handles load balancing and circuit breaking. Another answer suggested Netflix' ribbon which I have not used and will only be interesting if you are on JVM. Note that if you are building your own, you might find it simpler to just use Consul's HTTP API for discovering services than DNS SRV records. That way you can "watch" for changes too rather than caching the list and letting it get stale.
If you don't want to reinvent that particular wheel, another popular and simple option is to use a HAProxy instance as the load balancer. You can integrate it with consul via consul-template which will automatically watch for new/failed instances of your services and update LB config. HAProxy then provides robust load balancing and health checking with a lot of options (http/tcp, different balancing algorithms, etc). One possible setup is to have a local HAProxy instance on each docker host and a fixed port assigned statically to each logical service (can store it in Consul KV) so you connect to localhost:1234 for service A for example and localhost:2345 for service B. Local instance means you don't pay for extra round trip to loadbalancer instance then to the actual service instance but this might not be an issue for you.
I suggest you to check out Kontena. It will solve this kind of problem out of the box. Every service will have an internal DNS that you can use in communication between services. Kontena has also built-in load balancer that is very easy to use making it very easy to create and scale micro services.
There are also lot's of built-in features that will help developing containerized applications, like private image registry, VPN access to running services, secrets management, stateful services etc.
Kontena is open source project and the code is visible on Github
If you look for a minimal setup, you can wrap the values you receive from Consul via ribbon, Netflix' client based load balancer.
You will find it as a module for Spring Cloud.
I didn't find an up-to-date standalone example, only this link to chrisgray's dropwizard-consul implementation that is using it in a Dropwizard context. But it might serve as a starting point for you.