Sorry if my subject was already handled elsewhere but I don't see where to start the search.
I have an app separated in 3 containers : front (Angular) / back (Node.js) / mysql. This is a demo that will be available on a website.
The app will be provided by another server and I want to launch the app in a separated window with an http request from my website (button). As the user will test his own data (video file and management of fictive users), I want to erase all after he leaves.
Question 1: is it possible to launch the set of containers on a http request (and how to?)
Question 2: How to erase the datas? (destroying the container by a timeout?)
Question 3: Is launching a set of containers for each user a good solution to handle several users at the same time? I looked at Kubernetes but didn't find metric to scale up on http request. Moreover, how to redirect each user on his set of containers?
lunching on http request - not on kubernetes. you ususally deploy
there with kubectl command. and it takes few min to start all the
pods you need and services to be accesable.
Destroy pod (running container in kubernetes called pod) is usually
kubectl.... command as well.
creating pod per user is not what kubernetes designed for sure.
Kubernetes has autoscaling on load, but it is based on load balancer, and all pods in autoscaler should be able to do any request. So kubernetes is more like constantly monitoring automatic DevOps guy which also autoscale if necessary (most often cpu usage, but not limited by cpu)
Related
I want to move all of my operations over to K8S for so long, but am still hesitant to that. This question will likely be broad, but bear with me. Let me first describe the existing system.
I hosts a lot of different websites (>30). A lot of that for my own experimentation, but some are for actual clients. I have 1 VM in New York (I'm using DigitalOcean), with multiple Docker containers, frequently managed using docker-compose. There is 1 container for every site. The request first comes in to front container running HAProxy. This strips away SSL, then forwards the request to 2 proxy container running Nginx. These 2 container then forwards the request to all the other containers for their service. All of my certificates come from LetsEncrypt, and have to be renewed every 3 months. To do so, I stop front, run certbot --apache so it binds to port 80. It gets the certificates, then I stop apache, then recreate front container.
There are several reasons to why I do it this way:
I change site configs a lot, and how all of them are wired together. So front is expected to run forever, unless I'm getting certificates, and proxys are expected to change a lot. I change the proxy image, then stops and recreates the 1st container, then stops and recreates the 2nd container, so that there will be no downtime at all.
I really don't know how to get certificates when there are multiple nodes. In fact, I'm a total noob at the whole certificate thing and LetsEncrypt is pretty much the only way I know of to do this.
I want to directly edit files on the remote server. I have a bad practice of editing production code directly, mainly because I get impatient with setting up dev, staging and production environments. It takes too much time, and the gains feels small. And for clients, they are typically small businesses, with <10 employees, and regularly, they want to have some aesthetic changes to the websites. I can have a video call with them, they tell me exactly what they want, I code that in, it gets uploaded to the server immediately, and they see changes right away. Then they can critique the design, and we can iterate back and forth. If I were to setup different environments, they can't see it right away, and there has to be this long process of committing to git, deploy to staging, then production. This takes a long time, and I don't think is justified.
I realize that my systems are not that well maintained. Images are not getting security updates, I don't know if they are still running or not unless I check for them manually, which is tedious, so I don't do them at all. Furthermore, I have an Asian background, that means I have clients from both the US and Asia, pretty much the farthest place possible from each other, which increases latency by a lot. That means client in Asia has to wait for around 1-2 second for the page to actually load, which is eternal. I have also moved to Asia in the past week, so now, accessing the New York server via ssh is incredibly slow, and my productivity just plummets. So now it might be the best time to revamp everything, and move to K8S once and forever. However, there are major problems in the planning process and currently, K8S seems to lack a lot of stuff that are just deal breakers for me. So please criticize my plans, and improve them however you see fit.
What I plan to do now is this:
There will be 2 servers, 1 at New York, 1 at Singapore. These 2 severs will have 2 different ip addresses. Those 2 will be running K8S Pods. Preferably, they should have exactly the same configs, website containers, database containers, etc. Then for each website DNS record, I will modify A and AAAA records so that they contain 2 ip addresses for the 2 servers.
My question is:
Will DNS always route to Singapore if user is in China, and always route to New York if user is in England?
How to actually get certificates for 2 nodes? My understanding is that when certbot issues a certificate, it associates the domain name with the node ip address. That means 2 nodes can't have the same certificate for the same domain name. Is this correct? If you can get certificates for 2 nodes then how to do that?
How to keep files in sync between servers? Say I edit the file tree in Singapore server, I want that file to also be modified in New York several seconds later. For databases, I can have a master database at either Singapore or New York, then have slave databases at both locations that updates whenever the master updates, and the slaves can serve as a low latency database for each server.
How to actually route requests from servers to containers inside. I initially plan to use NodePort, to direct the request to front Pods, then that can distribute requests to other Pods, but I was heartbroken when NodePort can't attach to ports below 30000. The only other option that I am aware of is to have an external load balancing service that directs traffic to the 2 servers. But that costs like $15/site/month, and because I have >30 sites, doing so will bankrupt me. I can also have 4 servers in total, 2 for the K8S cluster, and 2 serves as a load balancer that will forward to NodePort. Will this plan works? How will automatic renewing of certificates even work here?
Please note that may be my questions are the wrong questions to ask (like, may be I shouldn't use A and AAAA records for directing traffic), and there's a different way to do this entirely, so feel free to ask the right questions.
read your question hats off to write down the whole stuff but half of the stuff is useless.
Answers of your question :
Can we add the same or multiple entries in DNS? example.com with A record multiple times possible?
You might require to set up a regional K8s cluster with regional ingress support. you can use certmanager with letsencrypt which will manage your cert at LB level and terminate it at the front.
If you are looking forward to use two VMs put one LB in front of both and set SSL over there.
if you are using K8s with stateless PODs editing direct file inside container is not a option. better you manage the Github update inside and container get deployed on to both cluster at a same time for that you can setup CI/CD. You are right in case of database server setup with master slave concept you can use read replicas.
To route the traffic from server to internal application of K8s you can an internal LB or exposing services with node ports(above 30000 but change target port in SVC) and route the port if you want to redirect requests on a specific port using the target port.
still, i am not getting "I can also have 4 servers in total, 2 for the K8S cluster, and 2 serves as a load balancer that will forward to NodePort. Will this plan works? How will automatic renewing of certificates even work here?" which server will be in front and which one in the backend.
If all your services are websites (run over http) you could use k8s ingress to route traffic to pods based on Host header (domain name) and use only one LB with one IP address. The most popular ingress controller seems to be the Nginx Ingress Controller
If you don't want to use LB you can use hostPort to expose nginx ingress but as soon as you have k8s cluster with more than one node, use LB because hostPort is generally not advised to use unless you have a very good reason to do so.
Speaking of DNS, you can use sth like AWS route53 routing policies for location routing. You don't necessarily need to use AWS. I just want to show you that there are solutions to this problem, but use whatever you like.
For certificates use cetrmanager with DNS-01 challenge.
From letsencrypt docs about DNS-01 challenge:
It works well even if you have multiple web servers.
cetrmanager will also handle certificate renewal for you.
About keeping files in sync between servers; It depends on files, but for static content it might be best to use CDN that will replicate content from one source to other locations.
For simultanous deploys to 2 separate clusters you can use some CI/CD pipeline like e.g. github actions.
I have a system of apps/services in docker containers that, when I bring them up using docker-compose, talk to each other using a bridge network.
Workers start up and register themselves with a manager. The manager assigns the workers work to do. In order to do this, the workers need to know where the manager is, and the manager needs to know where the workers are.
I want to deploy them all to Google Cloud Run.
At the moment, in docker via docker-compose, they talk to each other using their container names. For example the worker might call: http://manager:5000/register?name=worker1&port=5000 to register on startup, and then the manager can call http://worker1:5000 to send work. All thanks to the fact that they're connected to the same bridge network.
How does this work with Google Cloud Run? As far as I can see, when you create a service linked with a container, you get a permanent URL to communicate with your app once it has started. The app in the container doesn't know what the URL is.
Can I use the service names to communicate with each other in the same way as a docker bridge network?
Cloud Run currently does not support hostname based service discovery for other services in your project.
Currently, your best bet is to configure service URLs that your app depends on using environment variables or something like that.
In fact, you can't orchestrate in the same way the workers. Indeed, the Cloud Run services reply to an HTTP request. When an instance is spawn, there is no registration to a manager.
If you want to perform several task in parallel, perform several HTTP requests.
If you want a strong isolation between the different instances of a same service, set the concurrency param to 1 (only 1 HTTP request is processed in the same time by an instance of the service).
For information, you can have up to 100 instances for a same service.
So, deploy a manager service, and a worker service. The manager service perform HTTP request to worker with the right param for doing the right job.
Take care of the job duration. For now, the timeout can be set up to 900 seconds (15min) maximum
About the naming, the pattern is the following: https://<service-name>-<project-hash>.run.app/
Our cloud application consists of 3 tightly coupled Docker containers, Nginx, Web and Mongo. Currently we run these containers on a single machine. However as our users are increasing we are looking for a solution to scale. Using Kubernetes we would form a multi container pod. If we are to replicate we need to replicate all 3 containers as a unit. Our cloud application is consumed by mobile app users. Our app can only handle approx 30000 users per Worker node and we intend to place a single pod on a single worker node. Once a mobile device is connected to worker node it must continue to only use that machine ( unique IP address )
We plan on using Kubernetes to manage the containers. Load balancing doesn't work for our use case as a mobile device needs to be tied to a single machine once assigned and each Pod works independently with its own persistent volume. However we need a way of spinning up new Pods on worker nodes if the number of users goes over 30000 and so on.
The idea is we have some sort of custom scheduler which assigns a mobile device a Worker Node ( domain/ IPaddress) depending on the number of users on that node.
Is Kubernetes a good fit for this design and how could we implement a custom pod scale algorithm.
Thanks
Piggy-Backing on the answer of Jonah Benton:
While this is technically possible - your problem is not with Kubernetes it's with your Application! Let me point you the problem:
Our cloud application consists of 3 tightly coupled Docker containers, Nginx, Web, and Mongo.
Here is your first problem: Is you can only deploy these three containers together and not independently - you cannot scale one or the other!
While MongoDB can be scaled to insane loads - if it's bundled with your web server and web application it won't be able to...
So the first step for you is to break up these three components so they can be managed independently of each other. Next:
Currently we run these containers on a single machine.
While not strictly a problem - I have serious doubt's what it would mean to scale your application and what the challenges that come with scalability!
Once a mobile device is connected to worker node it must continue to only use that machine ( unique IP address )
Now, this IS a problem. You're looking to run an application on Kubernetes but I do not think you understand the consequences of doing that: Kubernetes orchestrates your resources. This means it will move pods (by killing and recreating) between nodes (and if necessary to the same node). It does this fully autonomous (which is awesome and gives you a good night sleep) If you're relying on clients sticking to a single nodes IP, you're going to get up in the middle of the night because Kubernetes tried to correct for a node failure and moved your pod which is now gone and your users can't connect anymore. You need to leverage the load-balancing features (services) in Kubernetes. Only they are able to handle the dynamic changes that happen in Kubernetes clusters.
Using Kubernetes we would form a multi container pod.
And we have another winner - No! You're trying to treat Kubernetes as if it were your on-premise infrastructure! If you keep doing so you're going to fail and curse Kubernetes in the process!
Now that I told you some of the things you're thinking wrong - what a person would I be if I did not offer some advice on how to make this work:
In Kubernetes your three applications should not run in one pod! They should run in separate pods:
your webservers work should be done by Ingress and since you're already familiar with nginx, this is probably the ingress you are looking for!
Your web application should be a simple Deployment and be exposed to ingress through a Service
your database should be a separate deployment which you can either do manually through a statefullset or (more advanced) through an operator and also exposed to the web application trough a Service
Feel free to ask if you have any more questions!
Building a custom scheduler and running multiple schedulers at the same time is supported:
https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
That said, to the question of whether kubernetes is a good fit for this design- my answer is: not really.
K8s can be difficult to operate, with the payoff being the level of automation and resiliency that it provides out of the box for whole classes of workloads.
This workload is not one of those. In order to gain any benefit you would have to write a scheduler to handle the edge failure and error cases this application has (what happens when you lose a node for a short period of time...) in a way that makes sense for k8s. And you would have to come up to speed with normal k8s operations.
With the information provided, hard pressed to see why one would use k8s for this workload over just running docker on some VMs and scripting some of the automation.
I have a Docker container that is a REST API webserver. I want to use this webserver in a Docker Swarm. A couple of the REST API calls are used in an asynchronous pattern. That is, the first call provides data for processing, and is returned a request identifier. The second call uses the request identifier to check on the processing and get the results when processing is done. Since there is no connection between any of the webservers in the Docker Swarm, how can I force the second REST API call back to the Docker instance that was used in the first REST API call? Is there anyway to ensure webserver affinity for these two REST API calls in a Docker Swarm?
With Docker Swarm Mode and Ingress networking, connections are processed with round robin load balancing, and this isn't configurable. If the connection remains open, which is the case for most web browsers, you'll find that requests go back to the same instance.
You can use a reverse proxy in front of your application that is aware of each instance of the service. Docker has this with their HRM tool in the EE offering, and many of the other reverse proxies, like traefik, offer similar sticky session options.
If you can, a better design would be to utilize an external cache for any persistence, e.g. redis. This way you can perform a rolling update of your application without breaking all the sessions.
All,
We recently had an issue with ELB HealthCheck in covering up a certain use-case or scenario which caused an application impact.
Can anyone suggest a fault-tolerant approach to handle this?
We have a nodeJS app running in a port - 80
We have 3 instances in the Target Group & that is enrolled in ELB.
ELB HealthCheck was configured to hit root path on port 80 and return success if it gets HTTP 200
Recently one of the node had 100% disk filled on application mount and root mount was still having space.
Though the HealthCheck was succeeding as per ELB the server didn't respond for any other services and it was ideally unhealthy. This means that there are some requests that got
succeeded but some of them failed (that was routed to this disk-filled server).
We did received notifications from other monitoring systems on disk filling but due to overwhelming emails & limited resources it got missed out.
Is there any other way we can improvise the HealthCheck strategy to just have these scenarios intimated to AutoScaling Group or ELB
so that we can target these nodes to be removed and replace them automatically?
Rather than just checking that the index.htm page is returning a 200 response, you can configure Elastic Load Balancing to point to a customer Health Check page (eg healthcheck.php).
You could run some code on that page to test the general health of the application (database connectivity, disk space, free memory). If everything checks out OK, return a 200 response. If something is wrong, return a 500 response. This will cause the Load Balancer to treat the instance as Unhealthy and it will stop serving traffic to the instance.
If Auto Scaling is configured to use the ELB Health Check, then Auto Scaling will terminate the unhealthy instance and automatically replace it with a new instance.