I can't run docker (linux) containers on my PC - it is too slow for that. Is there other ways of running/developing/testing docker containers in a similar way to doing it on my PC? Maybe some browser app? Or the only option is to simply host a VM somewhere like DigitalOcean or AWS?
Use AWS ECS. AWS gives you a free trial for 12 months where they give you some free stuff like free server hours and so on. After you create a new AWS account, go to AWS ECS service and then go to the repository section and create a new repository item by uploading your Docker image. Now go to the tasks section and create a task configuration for your docker image like memory, port and so on. After then, create a new service and assign it to the task that you just created and run it.
This process is straightforward and it will take you around an hour to finish all.
Follow the steps in this video: https://www.youtube.com/watch?v=1wLMLwjCqN4
Related
I have installed and configured a system in EC2s using Ansible. It is 1 EC2 master with a few EC2 workers. Sometimes when I use ansible to update or reinstall configuration, it fails because either some package has been removed from open-source repositories, or the package is updated so not compatible with some other packages. And I learned that using docker-container can resolve these kind of configuration problems.
However, according to what I learned, each docker image will create image of one application (I guess one application means one process). But mine is a system which has airflow master webserver, airflow worker webserver, flower webserver, rabbitmq, airflow celery, several configuration files, etc. how can I create docker images for that? Should I create one docker image for each process? How do I know which linux folder should I go to create each docker image? How do I know which applications/processes I need to create? And how to combine these images to make them work together as a system?
Or maybe in my case I should not use docker image, Instead I should just create an EC2 image?
Use docker-compose.
Compose is a tool for defining and running multi-container Docker applications
https://docs.docker.com/compose/
each docker image will create image of one application (I guess one application means one process)
That is basically correct. You should create one docker-container per application. In theory you can have multiple process per container, but that doesn't matter in this case.
how can I create docker images for that?
In your case you should make one docker-container for airflow master webserver, one for airflow worker webserver, one for flower webserver, etc. And the you use a docker-compose.yml to link them all together.
Should I create one docker image for each process?
generally yes. (It may depend on your exact setup though)
And how to combine these images to make them work together as a system?
docker-compose.
How do I know which linux folder should I go to create each docker image?
I don't understand that question
How do I know which applications/processes I need to create?
You could create a deployment-diagram and then start from there.
I'm trying to setup the deployment of docker images to Linux server (Debian 10).
I looked over the internet to find an easy solution to deploy images from docker repository onto a server automatically.
I know that Docker Hub has webhooks.
Also, there is an option to use Kubernetes, but it seems to be a bit too much for a simple application running on one server.
What I am looking for is a way for server to detect that docker image has been updated, so that it downloads it and runs the newest version.
Currently, I have setup automatic build of docker images on Azure DevOps that are pushed to private repository on Docker Hub (I will most likely move to privately hosted Nexus repository).
I am looking for suggestions on how to do it with relatively low complexity (e.g. should I use docker-compose for it or some sort of bash script on a server).
The closest thing to what I am looking for is this solution: How to auto deploy Docker Image on own server with GitLab?
I would like to know if this is the recommended way to do or are there any other, possibly easier ways to approach it.
I found this project that looks good as a solution for my case.
https://containrrr.github.io/watchtower/
I'm setting up a CI/CD workflow for my organization but I'm missing the final piece of the puzzle. Surely this is a solved problem, or do I have to write my own?
The full picture.
I'm running a few EC2 instances on AWS, each running docker in its native swarm mode. A few services are running here which I've started manually via docker service create ....
When a developer commits source code a trigger is sent to jenkins to pull the new code and build a new docker image which is then pushed to my private registry.
All is well and good up to here, but how do I get the new image onto my docker hosts and the running container automatically updated to the new version?
Docker documentation states (here) that the registry can send events to configurable endpoints when a new image gets pushed onto it. This is what I want to automatically react to by having my docker hosts then pull the new image and stop, destroy and restart the service using that new version (with the same env flags, labels, etc etc), but I'm not finding any solution to this that fits my use case.
I've found v2tec/watchtower but it's not swarm-aware nor can it pull from a private registry at the time of writing this question.
Preferably I want a docker image I can deploy on my docker manager which listens to registry events (after pointing the registry config at it) and does the magic I need.
Cost is an issue, but time is less so, so I'm more inclined writing my own solution than I am adopting a fee-based service for this.
One option you have is to SSH to swarm master from Jenkins using SSH plugin and pull the new image and update the service when new image is pushed to the registry.
I'm very new to Amazon ECS, and I've written a task definition with 3 containers. One for my php application (main-server), second for node application (pubsub-server) and a redis container.
main-server and pubsub-server link to redis container. (Is this the best way to arrange the containers?)
The cluster runs well. However, I have an update to make in my main-server. I am able to push the updated image to Amazon ECR but my changes don't reflect on the cluster. Is there any additional step to perform to run the updated container on push?
I have tried deregistering the tasks and activating them back. But it doesn't seem to work.
Please let me know if I need to provide anymore details.
You need to force a new deployment.
From the AWS console, update the service definition and check the force new deployment checkbox on the first page, then skip to confirmation page.
From CLI:
aws ecs update-service --cluster [cluster arn] --service [service arn] --force-new-deployment
I have an aws ec2 account, where I am running couple of web apps on nginx. I don't know much about docker, except it is a container that takes snapshot of filesystem. Now, for some reason I am forced to switch accounts. I have opened a new aws ec2 account. Can I use docker to set up a container in my old virtual system, then get an image and deploy in my new system? This way I can remove the headache of having to install many components, configure nginx and all applications in my new system. Can I do that? If so, how?
According to the best practices of Docker and its CaaS, images are not supposed to "virtualize" a whole lot of services, on the contrary. Docker does not aim at taking a snapshot of the system (it uses FS overlay to create images, but theses are not snapshots).
So basically, if your (yet unclear) question is: "Can I virtualize my whole system into one image" the answer is: "No".
What you can do is using an image for each of your service (you'll find everything you need on the hub.docker) to keep a clean system on your new one.
Another solution would be to list all the installed Linux packages on your old system, and installed them on the new one and copy all the configuration files.