I’m using Amazon ECS to auto deploy my containers on uat/production.
What is the best way to do that?
I have a REST api with a several front-end clients
Should I package my api container with nginx in the same container?
And do the same thing with the others front end clients.
Or I have to write a big task definition to bring together all my containers(db, nginx, php, api, clients) :(, but that's mean that I should redeploy all my infrastructure at each push uat/prod
I'm very confusing.
I would avoid including too much in a single container. Try and distill your containers down to one process doing one thing. If all you're doing is serving up a REST API for consumption by your front end, just put the essential pieces in for that and no more.
In my experience you also want your ECS tasks to be able to handle failure gracefully and restart, and the more complicated your containers are the harder this is to get right.
Depending on your requirements I would look into using ELB instead of nginx, you can have your ECS cluster point at an ELB and not have to deal with that piece at all.
Do not use ECS - it's too crude. I was using it as a platform for our staging/production environments and had odd problems during deployments - sometimes it worked well, sometimes - not (with the same Docker images). ECS provides not clear model of container deployment and maintenance.
There is another good, stable and predictive option - Docker Cloud service. It's new tool (a.k.a. Tutum) that was acquired by Docker. I switched the CI/CD to use it and we're happy with it.
Bind Amazon user credentials to Docker Cloud account. Docker Cloud uses AWS (or other provider) API for creating appropriate computer instances.
Create Node. Select Amazon EC2 instance type and parameters of storage, security group and so on. New instance will contain installed docker software and managing container that handles messages from Docker Cloud (deploy, destroy and others).
Create Stackfile, see https://docs.docker.com/docker-cloud/apps/stack-yaml-reference/. Stackfile is a definition of container group you required. You can define different scaling/distribution models for your containers using specific Stackfile options like deployment strategy, see https://docs.docker.com/docker-cloud/apps/stack-yaml-reference/#deployment-strategy-1.
Define ELB configurations in AWS for your new instances.
P.S. I'm not a member of Docker team and I like other AWS services :).
Here is my two cents on the topic, the question is not really related to ecs, it applies to any body deploying their apps on docker.
I would suggest separating the containers, one for nginx and one for API.
if they need to be co-located on the same instance, on ECS you can define them as part of the same task and on kubernetes you can make them part of same pod.
Define a docker link between the nginx and the api container. This will allow the nginx process to talk to api container without the api container exposing its ports to the host.
One advantage of using the container running platforms such as kubernetes and ecs is that they ensure each of the container run all the time and dynamically restart if one of the processes/containers go down.
Separating the containers will allow these platforms to monitor both the processes separately. When you combine the two into one container the docker container can only run with one of the processes in foreground, so you will loose the advantage of auto-healing for one of the processes.
Also moving from nginx to ELB is not a straightforward solution, you may have redirections and other things configured on the nginx, which are not available on ELB(As of date).
If you also need the ELB, there is no harm in forwarding the requests from the ELB to the nginx port.
Related
I a want to run a web page similar like kubernetes dashboard.The web page takes input from the user and generates a small file but i want the web page to be loaded without using any server. kubernetes is deploying a pod and bringing up the web page i want to do the same.If kubernetes is also using a server how is it using it(is it directly downloading it with the OS in the pod or how is kubernetes doing it).
Overview I want to know how kubernetes dashboard is getting deployed is it using a server if so how is it getting the server installed in the kubernetes pod else how is it bring up the UI.
Actually, Kubernetes plays the role as an orchestrator and provides sufficient way for building communication channels between containers in the cluster and uses Docker by default as a container runtime.
Containers represent run-time environment for images, however images consist with OS layer and application binaries, a good explanation you can find here. In order to build own image you might consider two ways to afford this: create an image from existing one in Docker Hub or compose image from Dockerfile.To store the customized image might be the option to push it into Docker Hub repository or stand for some private isolated repo by deploying a Registry server.
When you are ready with an image, and you plan to implement application in Kubernetes cluster, that's a good time to create first microservice. Although, there are tons of materials about Kubernetes cluster and its run-time engine architecture in the globe, I would focus on the application deployment lifecycle.
Deployment is the main mechanism which defines how are Pods should to be implemented within a cluster and provides specific configuration for further application run-time workflow.
Service describes a way how the particular Pod will communicate with other resources within a cluster, providing endpoint IP address and port where your application will respond.
In general scenario with Kubernetes Dashboard, the method in use kubectl proxy will expose the application by proxying gateway between host and Kubernetes API, which is more like for testing purposes and not secure, in comparison with Nodeport type which brings more convenient way to make application accessible outside the cluster, as described in this Stack thread.
I encourage you to get some more learning stuff in the official Kubernetes documentation.
I'm using Docker I have implemented a system to deploy environments (on a single server) based on Git branches using Traefik (*.dev.domain.com) and Docker Compose templates.
I like Kubernetes and I've never switched to it since I'm limited to one single server for my infrastructure. I've only used it using local installations (Docker for Windows).
So, my question is: does it make sense to run a Kubernetes "cluster" (master and nodes) on a single server to orchestrate and route containers (in place of Traefik/Rancher/Docker Compose)?
This use is for development and staging only for the moment, so high availability is not a prerequisite.
Thanks.
If it is not a production environment, it doesn't matter how many nodes you are using. So yes, it should be just fine in this case. But make sure all the k8s features you will need in production are available in test/dev, to keep things similar and portable.
AFAIU,
I do not see a requirement for kubernetes unless we are doing below at least for single host using native docker run or docker-compose or docker engine swarm mode -
Make sure there are enough(>=2) replicas of your app in a single server and you are balancing the load across those apps docker containers.
If you want to go bit advanced, we should be able to scale up & down dynamically (docker swarm mode supports this out of the box else use jwilder nginx proxy).
Your deployment should not cause a downtime. Make sure a single container is always healthy at any instant of time while deploying.
Container should auto heal(restart automatically) in case your HTTP or TCP health check fails.
Doing all of the above will certainly put you in a better place but single host is still a single source of failure which you got to deal with at regular intervals.
Preferred : if possible try to start with docker engine swarm mode or kubernetes single master or minikube. This will automatically take care of all the above scenarios out of the box and will also allow you to further scale up anytime by adding more nodes without changing much in your YML files for docker swarm or kubernetes.
Ref -
https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
https://docs.docker.com/engine/swarm/
I would use single host k8s only if I managed clusters with the same project that I would like to deploy to the said host. This enables you to reuse manifests and all the automation you've created for your clusters.
Have I had single host environments only, I would probably stick to docker-compose.
If you're looking to try it out your easiest options are probably minikube (easy to run single-node cluster locally but without some features) or using one of the free trial accounts for a managed Kubernetes service from one of the big cloud providers (fully-featured and multi-node but limited use before you have to pay).
I've been learning how to use Docker to setup dev environments, however, I'm
curious how these ideas translate to a production stack. As an example, I have a Laravel (Php) app, which uses MySQL, Redis, and Nginx
So in production, let's say I would normally have 2 application ec2 instances behind a load balancer on AWS. When setting up a similar production situation using Docker...
1) because I'd be using RDS and Elasticache, there would be no need for containers for those. So basically, id only need containers for PHP-Fpm and Nginx?
2) to have high availability, I would still have 2 (or least more than 1) ec2 instances behind the ELB. So I suppose each instance would run the above containers (PHP and Nginx). But that sounds no different than my previous VM setup, where each server runs what it needs to serve the application. Is that accurate?
3) with VMs, I would traditionally bake the code into an AMI and add those AMIs to a Launch Configuration and an Auto Scaling group, and that group would spin up instances as needed. So for deployment, I would tear down the old ec2 instances and spin up new ones. With Docker, Since these containers would be running on ec2 instances, wouldn't i still have to spin up / tear down the VMs, or would I just replace the containers and keep the VMs running?
Its reasonable to keep RDS, Elasticache and other fully managed services, outside of docker environment. Yes for high availability you need multiple EC2 instances having docker daemon running.
The real advantage is not coming with having two EC2 instances running two web server docker containers on each of them. Real advantages comes when you break down your application to microservices, where multiple containers in combination construct your web application providing the benefits of microservices.
Apart from that the DevOps flow would be different compared to traditional web application deployment in EC2 with autoscaling and load balancing and have many benefits. For example your source code will contain the container code as well, which will guarantee, the environment will work uniformly in your staging and production. Also you will be having images pointing to branches/tags in your source control, which allows to get new updates(delta downloads) for new releases.
If you are going to setup docker in AWS, its recommended to go with AWS ECS to reduce management overhead.
You're right, you will only need to run your code in a container and it will simply access the remote services. The only thing you'll have to consider is to ensure connectivity to them.
You're right again, you'll need to have everything you previously had in your VMs in the Docker container so that your code works as before. Anyway, with Docker containers it is possible to run multiple instances of your app on the same EC2 instance. Of course, your app will try to run on the same port, so some extra networking layer is needed for managing ports is necessary, but it's possible. All the EC2 instances needs to have is docker installed.
Instead of creating AMIs and closing and spinning up EC2 instances, you'll only have to pull the new Docker image and restart the container with the new image. This means just a few seconds compared to minutes in the EC2 instances flow. This is means you have a really quick way of reverting buggy deploys and opens the doors for a setup in which 0% downtime can be reached.
I'm pretty new to Docker orchestration and managing a fleet of containers. I'm wanting to build an app that would give the user a container when they ran a command. What is the best tool and best way to accomplish this?
I plan on having a pool of CoreOS servers to run the containers on and I'm imagining the scheduler to have an API that I can just call to create the container.
Most of what I have seen with Nomad, Kubernetes, Docker Swarm, etc is how to provision multiple clusters of containers all doing the same thing. I'm wanting to be able to create a single container based on a users command and then be able to communicate with an API on that container. Anyone have experience with this?
I'd look at Kubernetes + the Jobs API (short lived) or Deployments (long lived)
I'm not sure exactly what you mean by command, but I'll assume its some sort of dev env triggered by a CLI, make-dev.
User triggers make-dev, which sends a webhook to your app sitting in front of the Jobs API, ideally doing rate-limiting and/or auth.
Your app takes the command, sanity checks it, then fires off a Job/Deployment request + an Ingress rule + Service
Kubernetes will schedule it out across your fleet of machines
Your app waits for the pod to start, then returns back the address of the API with a unique identifier (the same thing in the ingress rule) like devclusters.com/foobar123
User now accesses their service at that address. Internally Kubernetes uses the ingress and service to route the requests to your pod
This should scale well, and if your different environments use the same base container image, they should start really fast.
Plug: If you want an easy CoreOS + Kubernetes cluster plus a UI try https://coreos.com/tectonic
I plan on having a pool of CoreOS servers to run the containers on and I'm imagining the scheduler to have an API that I can just call to create the container
kubernetes comes with a RESTful API that you can use to directly create pods (the unit of work in kubernetes which contains one or more containers) within your cluster.
The command line utility kubectl also interacts with the cluster in the exact same way, via the api. There are client libraries written in golang, Java, and Python at the moment with others on the way to help communicate with the cluster.
If you later want a higher level abstraction to manage pods, update them and manage their lifetimes, looking at one of the controllers (replicaset, replication controller, deployment, statefulset) should help.
I am new to both Docker and Consul, and am trying to get a feel for how containerized apps could use Consul for both service registry and KV pair config management ("configuration").
My understanding was that I could:
Create an image that runs Consul server, so something like this; then
Spin up three of these Docker-Consul containers (thus forming a cluster/quorum) on myvm01.example.com (an Ubuntu VM); then
Refactor my app to use Consul and create a Docker image that runs my app and Consul agent, with the agent configured to join the 3-node quorum at startup. On startup, my app uses the local Consul agent to pull down all of its configurations, stored as KV pairs. It also pulls in registered/healthy services, and uses a local load balancing tool to balance the services it integrates with.
Run my app's containers on, say, myvm02.example.com (another Ubuntu VM).
So to begin with, if any of this seems like I am misunderstanding the normal/proper uses of Docker and Consul (sans Registrator), please begin by correcting me!
Assuming I'm more or less correct, I recently stumbled across Registrator and am now even more confused. Registrator seems to be some middleman between your app containers and your Consul (or whatever registry you use) servers.
After reading their Quickstart tutorial, it sounds like what you're supposed to do is:
Deploy my Consul cluster/quorum containers to myvm01.example.com like before
Instead of "Dockerizing" my app to use Consul directly, I simply integrate it with Registrator
Then I deploy a Registrator container somewhere, and configure it to integrate with Consul
Then I deploy my app containers. They integrate with Registrator, and Registrator in turn integrates with Consul.
My concerns:
Is my understanding here correct or way off base? If so, how?
What is actually gained by the addition of Registrator. It doesn't seem (to the untrained eye at least) like anything more than a layer of indirection between the app and the service registry.
Will I still be able to leverage Consul's KV config service through Registrator?
Is my understanding here correct or way off base? If so, how?
It seems to me, that it's not a good solution, to have all cluster/quorum members running inside the same VM. It's not so bad if you use it for development or tetsing or something, where you don't care much about reliability, but not for production.
Once your VM dies, you'll loose all the advantages you have by creating a cluster. And even more, you can loose all the data you have in K/V store, because you are running Consul servers inside a docker containers, which should be additionaly configured to share the configuration between runs.
As for the rest, I see it the same as you.
What is actually gained by the addition of Registrator.
From my point of view, the main thing is, that you don't have to provide an instance of Consul Agent in every container you run. And the container with the image you run is responsible only for their main functions, not for registering itself somewhere. You may simply pull an image and just run a container with it, to make it's service available, without making additional work.
Will I still be able to leverage Consul's KV config service through Registrator?
Unfortunately, no. At least, we didn't find a solution to use it this way, when we were looking for something to make service discovering and configuration management. We came to conclusion, that Registrator is not a proxy for K/V store and is used only to automate service discovery. So you have to use some other logic to access consul's K/V store.
Update: furthermore, here is 2 articles: "Automatic Docker Service Announcement with Registrator" and "Automatic container registration with Consul and Registrator", I found usefull to understand Registrator role in service discovery process.