I'm a huge fan of the docker philosophy (or at least I think I am). Even so, I'm still quite novice in the sense that I don't seem to grasp the intended way of using docker.
As I see it currently, there are two ways of using docker.
Create a container with everything I need for the app in it.
For example, I would like something like a Drupal site. I would then put nginx, php, mysql and code into a container. I could run this as a service in swarm mode and scale it as needed. If I need another Drupal site, I would then run a second container/service that holds nginx, php and mysql and (slightly) different code. I would now need 2 images to run a container or service off.
Pro's - Easy, everything I need in a single container
Con's - Cannot run each container on port 80 (so need a reverse proxy or something). (Not sure at but I could imagine) Server load is higher since there are multiple containers/services running nginx, php and mysql.
Create 4 separate containers. 1 nginx container, 1 php container, 1 mysql container and 1 code/data container.
For example, I would like the same Drupal site. I could now run them all as a separate service and scale them across my servers as the amount of code containers (Drupal sites or other sites) increases. I would only need 1 image per container/service instead of a separate image for each site.
Pro's - Modular, single responsibility per service (1 for database, 1 for webserver etc), easy to scale only the area that needs scaling (scale database if requests increase, nginx if traffic increases etc).
Con's - I don't know how to make this work :).
Personally I would opt to make a setup according to the second option. Have a database container/service, nginx container/service etc. This seems much more flexible to me and makes more sense.
I am struggling however on how to make this work. How would I make the nginx service look at the php service and point the nginx config to the code folder in the data service etc. I have read some stuff about an overlay network but that does not make clear to me how nginx would look for php in a separate container/service.
I therefore have 2 (and a half) questions:
How is docker meant to be used (option 1 or 2 above or totally different)?
How can I link services together (make nginx look for php in a different service)?
(half) I know I am a beginner trying to grasp the concept but setting up a simple webserver and running websites seems like a basic task (at least, it is for me in conventional ways) but I can't seem to find my answers online anywhere. Am I totally off par in the way I think I would like to use docker or have I not been looking well enough?
How is docker meant to be used (option 1 or 2 above or totally different)?
Upto you, I prefer using Option #2, but i have at times used mix of Option #1 and options #2 also. So it all depends on the use case and which options looks better for the use case. At one of our client it was needed to have SSH and Nginx, PHP all in same container. So we mixed #1 and #2. Mysql, redis on their own container and app on one container
How can I link services together (make nginx look for php in a different service)?
Use docker-compose to define your services and docker stack to deploy them. You won't have to worry about the names of the services
version: '3'
services:
web:
image: nginx
db:
image: mysql
environment:
- "MYSQL_ROOT_PASSWORD=root"
Now deploy using
docker stack deploy --compose-file docker-compose.yml myapp
In your nginx container you can reach mysql by using it's service name db. So linking happens automatically and you need not worry.
I know I am a beginner trying to grasp the concept but setting up a simple webserver and running websites seems like a basic task (at least, it is for me in conventional ways) but I can't seem to find my answers online anywhere. Am I totally off par in the way I think I would like to use docker or have I not been looking well enough
There are lot of good resources available in forms of articles, you just need to look
Related
I’ve spent months building an application and now I’m looking to deploy it, but I’m new to Docker and I seem to have brain block when it comes to actually containerizing my application. I need to run the following technologies:
php 7.2
mysql 5.7
apache 2.4
phpMyAdmin 4.7
My application will need to be available exclusively through https and I’m assuming the connection between my application and the mysql container will also need to be through a secure port.
In addition to that I have a wordpress site that will serve as the pre-login experience for my application that I’d like to dockerize, but should not share the same DB. When I move this to a prod environment, I will not include the phpMyAdmin container.
How many containers do I need? I was thinking that I would need at least 5:
apache
php
mysql (my application)
mysql (wordpress)
phpmyAdmin
Should my application and the worpress site live in the php container? or should I create separate containers for each.
What should my docker-compose.yml file and dockerfiles look like to achieve this feat?
The driving idea here is that a container should contain a single "service". You don't break things into containers by software component (php, apache, etc.) but rather by whatever needs to be combined to create a single service. So if your application is a PHP application hosted by Apache, then you'd want a container for your application that contained PHP, Apache and your application code. That would provide your application as a service.
Same goes for Wordpress. If Wordpress is running behind Apache and needs PHP, you'd create a second container containing PHP, Apache, WordPress, and your WordPress content, producing your "Wordpress service".
Each of your individual databases can be seen as a service, so you might want two containers running MySQL, one serving each of your databases. You could choose to consider the database server as a whole to be a service, and have it serve both of your databases. Then you could get away with a single MySQL container. Which way you go with this is a minor issue. Having a single database server will likely save a little bit of resources by avoiding some duplication.
If all of your services need to talk to each other, the easiest way to do this with Docker is to use Docker Compose. This lets you create multiple containers that know about each other and can communicate very easily between each other by way of some simple DNS logic that Docker Compose provides. With Compose, you give each of your containers a simple name, and then that name can be looked up via DNS to provide the IP address of each container. So for example, if your MySql container was named "mysql", your app container could connect to it via the DNS address "mysql" with no additional work on your part.
Following Docker best practices - a container should be created with a "single responsibility" principle in mind - it makes a good idea to have nginx and some-my-custom service in separate docker containers. But the question is - if i run in an autoscaled/loadbalanced environment where i have at least 2-3 running copies of the same container - should I have a separate nginx container cluster for my every custom services cluster or should I have one nginx containers cluster for the whole infrastructure (but simply more instances). Please find an illustrated example.
Does it even make any difference?
I don't think there is an absolute answer to this question, so I'm just throwing my current (random) thoughts.
First, it depends on what you use nginx for. If it's used to host webpacked web site, then sure you need one nginx for each of your frontend service, combined with worker_processes in nginx.conf, you have an easy load balancing solution for the frontend. If you want to use it a reverse proxy for load balancing, one instance is enough for a small cluster (when it grows bigger, you can add more to form a multi-tier load balancer).
Second swarm actually has a native load balancer that sort of works out of the box.
Third, the complexity of setting up nginx is also to be considered. Personally I don't think nginx is easy enough to setup and debug. Modern solutions like traefik.io are easier to use and built with clustering in mind, and come with extra features like automatic https configuration via lets'encrypt.
So, here is the problem, I need to do some development and for that I need following packages:
MongoDb
NodeJs
Nginx
RabbitMq
Redis
One option is that I take a Ubuntu image, create a container and start installing them one by one and done, start my server, and expose the ports.
But this can easily be done in a virtual box also, and it will not going to use the power of Docker. So for that I have to start building my own image with these packages. Now here is the question if I start writing my Dockerfile and if place the commands to download the Node js (and others) inside of it, this again becomes the same thing like virtualization.
What I need is that I start from Ubuntu and keep on adding the references of MongoDb, NodeJs, RabbitMq, Nginx and Redis inside the Dockerfile and finally expose the respective ports out.
Here are the queries I have:
Is this possible? Like adding the refrences of other images inside the Dockerfile when you are starting FROM one base image.
If yes then how?
Also is this the correct practice or not?
How to do these kind of things in Docker ?
Thanks in advance.
Keep images light. Run one service per container. Use the official images on docker hub for mongodb, nodejs, rabbitmq, nginx etc. Extend them if needed. If you want to run everything in a fat container you might as well just use a VM.
You can of course do crazy stuff in a dev setup, but why spend time setting up something that has zero value in a production environment? What if you need to scale up one of the services? How do set memory and cpu constraints on each service? .. and the list goes on.
Don't make monolithic containers.
A good start is to use docker-compose to configure a set of services that can talk to each other. You can make a prod and dev version of your docker-compose.yml file.
Getting into the right frame of mind
In a perfect world you would run your containers in clustered environment in production to be able to scale your system and have concurrency, but that might be overkill depending on what you are running. It's at least good to have this in the back of your head because it can help you to make the right decisions.
Some points to think about if you want to be a purist :
How do you have persistent volume storage across multiple hosts?
Reverse proxy / load balancer should probably be the entry point into the system that talks to the containers using the internal network.
Is my service even able run in a clustered environment (multiple instances of the container)
You can of course do dirty things in dev such as mapping in host volumes for persistent storage (and many people who use docker standalone in prod do that as well).
Ideally we should separate docker in dev and docker i prod. Docker is a fantastic tool during development as you can have redis, memcached, postgres, mongodb, rabbitmq, node or whatnot up and running in minutes sharing that compose setup with the rest of the team. Docker in prod can be a completely different beast.
I would also like to add that I'm generally against the fanaticism that "everything should be running in docker" in prod. Run services in docker when it makes sense. It's also not uncommon for larger companies to make their own base images. This can be a lot of work and will require maintenance to keep up with security fixes etc. It's not necessarily the first thing you jump on when starting with docker.
I started to use rancher recently for a project.
Within few days I set up a standard microservice architecture with 4 basic services (hosted on Digital Ocean), trying to make it as production ready as possible
Services:
Api Gateway
GraphQL Api
OAuth2 Server
Frontend
it also includes Loadbalancers, Health checks etc...
I'm amazed at how good it is, as such I heavily used all the features provided by rancher in my configs, for example, the DNS conventions <service>.<stack>, sidekicks, rancher-compose etc...
The above services lives in their own repository and they have their
own Dockerfile , docker-compose.yml and rancher-compose.yml for production, so that they can be deployed independently.
Now that I proved myself that rancher will be my new "friend", I need a strategy to run the same application on my local environment and being able to develop my services, just like I would do with Vagrant.
I'm wondering what's the best approach to port an application that runs on rancher to a development environment.
I had some ideas on how to tackle this, however, none of them seemed to allow me to achieve it without re-configuring the whole services for development.
1 - Rancher on local machine
This is the first approach I took, install a rancher-server and a rancher-client locally and deploy the whole stack just like in production. It seemed the most logical idea to me. However, this wouldn't allow me to change the code of the services and being reflected into the containers live. Maybe using shared volumes might work but it looks trivial to me if you have any idea please let me know. For me, This solution is gone :(
2 - Docker compose
My second attempt was to use plainly docker compose and shared volumes, omitting load balancers and all the features of rancher :( However, this might work, I would need to change all the configurations of all my services where they point to a rancher specific DNS domain <service>.<stack> to use just <service> over the bridge network. But this means maintaining 2 different configurations for different environments, which is weird and not fun to do.
3 - Vagrant
As the second solution is already messy (double docker-compose and double configuration for the services) why not just re-create the whole environment in vagrant (without rancher features, maybe with ansible) where one nginx does reverse proxy and resolve requests between services. However, this require also quite a lot work and double effort again :(
Is there any other approach which will make rancher suitable for a development environment in a non-painfull way? How companies which rely on rancher or any other platform tools solved this issue?
Rancher on the local machine is a common pattern. If you run Rancher on a VM, or locally on a Linux box, when you launch your stacks the subtle change is that you add volumes to the host..
services:
myapp:
volumes:
- /Users/myhome/code:/src
...
You could now use templating features in the compose files and the Rancher CLI. Something like:
services:
myapp:
{{ if dev-local == "true"}}
volumes:
- /Users/blah:/src
{{end}}
...
Then you could have an answers file that just has
dev-local="false"
Do I need use separate Docker container for my complex web application or I can put all required services in one container?
Could anyone explain me why I should divide my app to many containers (for example php-fpm container, mysql container, mongo container) when I have ability to install and launch all stuff in one container?
Something to think about when working with Docker is how it works inside. Docker replaces your PID 1 with the command you specify in the CMD (and ENTRYPOINT, which is slightly more complex) directive in your Dockerfile. PID 1 is normally where your init system lives (sysvinit, runit, systemd, whatever). Your container lives and dies by whatever process is started there. When the process dies, your container dies. Stdout and stderr for that process in the container is what you are given on the host machine when you type docker logs myContainer. Incidentally, this is why you need to jump through hoops to start services and run cronjobs (things normally done by your init system). This is very important in understanding the motivation for doing things a certain way.
Now, you can do whatever you want. There are many opinions about the "right" way to do this, but you can throw all that away and do what you want. So you COULD figure out how to run all of those services in one container. But now that you know how docker replaces PID 1 with whatever command you specify in CMD (and ENTRYPOINT) in your Dockerfiles, you might think it prudent to try and keep your apps running each in their own containers, and let them work with each other via container linking. (Update -- 27 April 2017: Container linking has been deprecated in favor of regular ole container networking, which is much more robust, the idea being that you simply join your separate application containers to the same network so they can talk to one another).
If you want a little help deciding, I can tell you from my own experience that it ends up being much cleaner and easier to maintain when you separate your apps into individual containers and then link them together. Just now I am building a Wordpress installation from HHVM, and I am installing Nginx and HHVM/php-fpm with the Wordpress installation in one container, and the MariaDB stuff in another container. In the future, this will let me drop in a replacement Wordpress installation directly in front of my MariaDB data with almost no hassle. It is worth it to containerize per app. Good luck!
When you divide your web application to many containers, you don't need to restart all the services when you deploy your application. Like traditionally you don't restart your mysql server when you update your web layer.
Also if you want to scale your application, it is easier if your application is divided separate containers. Then you can just scale those parts of your application that are needed to solve your bottlenecks.
Some will tell you that you should run only 1 process per container. Others will say 1 application per container. Those advices are based on principles of microservices.
I don't believe microservices is the right solution for all cases, so I would not follow those advices blindly just for that reason. If it makes sense to have multiples processes in one container for your case, then do so. (See Supervisor and Phusion baseimage for that matter)
But there is also another reason to separate containers: In most cases, it is less work for you to do.
On the Docker Hub, there are plenty of ready to use Docker images. Just pull the ones you need.
What's remaining for you to do is then:
read the doc for those docker images (what environnement variable to set, etc)
create a docker-compose.yml file to ease operating those containers
It is probably better to have your webapp in a single container and your supporting services like databases etc. in a separate containers. By doing this if you need to do rolling updates or restarts you can keep your database online while your application nodes are doing individual restarts so you wont experience downtime. If you have caching with something like Redis etc this is also useful for the same reason. It will also allow you to more easily add nodes to scale in a loosely coupled fashion. It will also allow you to manage the containers in a manner more suitable to a specific purpose. For the type of application you are describing I see very few arguments for running all services on a single container.
It depends on the vision and road map you have for your application. Putting all components of an application in one tier in this case docker container is like putting all eggs in one basket.
Whenever your application would require security, performance related issues then separating those three components in their own containers would be an ideal solution. It's needless to mention that this division of labor across containers would come at some cost and which would be related to wiring up those containers together for communication and security etc.