I am building a docker container with only data (there is no service running). The data is mapped to the host.
What is the proper way to start a container from such an image?
My idea is to finish my Dockerfile with a
CMD /usr/bin/sleep 1000d
This will have no impact on the server and will be good for 3 years or so (which is plenty enough).
UPDATE: I was reluctant to give the whole context of why a docker container but since there are comments and by popular demand, here it goes :)
I self-host a myriad of services, all on docker. Some are pulled from Docker Hub (or similar) and some are my own code. They (= the ones I wrote) are deployed via a CI/CD served via a Caddy reverse proxy (all of them).
One of the services is a static site. I could copy the build artifact somewhere and have my Caddy serve it. It would work.
It would be very different from the other pipelines, though, and I would vastly prefer to be consistent because this is a home (amateur) environment and the more I streamline the production, the better.
Thus the idea of a docker container that runs a gimmick process (that does not grab resources) and that allows me to maintain a coherent CI/CD pipeline and final environment (with specific docker-compose and caddy configurations, per container in a neatly organized nest)
Related
Is there a reason to use docker-swarm instead of docker-compose for deploying a single host in production?
I'm currently rewriting an existing application. My predecessors set up the application using docker-swarm. But I do not understand why: the application will only consist of a single host running a couple of services. These services will only supply some local information on the customer network via a REST-Api to a kubernetes cluster (so no real load or reason to add additional hosts).
I looked through the Docker website and could not find a reason to use docker-swarm to deploy a single host, apart from testing a deployment on a single host dev environment.
Are there benefits of using docker-swarm compared to docker-compose regarding deployment, networking, etc...?
Docker Swarm and Docker Compose are fundamentally different animals. Compose is a build tool that lets you define and configure a group of related containers, whereas swarm is an orchestration tool that manages multiple docker engines in a way that lets you treat them (somewhat) as a single unit. Swarm exposes an API that is mostly compatible with the Docker Remote API, which allows existing applications to use Swarm to scale horizontally without having to completely overhaul the existing interface to the container engine.
That said, much of the functionality in Docker Compose that overlaps with Docker Swarm has been added incrementally. Compose has grown over time, and the distinction between the two has narrowed a bit. Swarm was eventually integrated into the Docker engine, and Docker Stack was introduced, allowing compose.yml files to be read directly by Docker, without using Compose.
So the real question might be: what is the difference between docker compose and docker stack? Not a whole lot. Compose is actually a separate project, written in Python that uses the Docker API under the hood. Stack does much of the same things as Compose, but is integrated into Docker. Stack also wants pre-built images, while compose will handle those image builds for you, which makes compose very handy for development.
What you are dealing with might be a product of a time when these 2 tools were a lot more distinct. Docker Swarm is part of Docker, and it allows for easy scaling if needed (even if you don't need it now, it might be good down the road). On the other hand, Compose (in my opinion anyway) is much more useful for development situations where you are making frequent tweaks to your images, and rebuilding.
I have a couple of compose files (docker-compose.yml) describing a simple Django application (five containers, three images).
I want to run this stack in production - to have the whole stack begin on boot, and for containers to restart or be recreated if they crash. There aren't any volumes I care about and the containers won't hold any important state and can be recycled at will.
I haven't found much information on using specifically docker-compose in production in such a way. The documentation is helpful but doesn't mention anything about starting on boot, and I am using Amazon Linux so don't (currently) have access to Docker Machine. I'm used to using supervisord to babysit processes and ensure they start on boot up, but I don't think this is the way to do it with Docker containers, as they end up being ultimately supervised by the Docker daemon?
As a simple start I am thinking to just put restart: always on all my services and make an init script to do docker-compose up -d on boot. Is there a recommended way to manage a docker-compose stack in production in a robust way?
EDIT: I'm looking for a 'simple' way to run the equivalent of docker-compose up for my container stack in a robust way. I know upfront that all the containers declared in the stack can reside on the same machine; in this case I don't have need to orchestrate containers from the same stack across multiple instances, but that would be helpful to know as well.
Compose is a client tool, but when you run docker-compose up -d all the container options are sent to the Engine and stored. If you specify restart as always (or preferably unless-stopped to give you more flexibility) then you don't need run docker-compose up every time your host boots.
When the host starts, provided you have configured the Docker daemon to start on boot, Docker will start all the containers that are flagged to be restarted. So you only need to run docker-compose up -d once and Docker takes care of the rest.
As to orchestrating containers across multiple nodes in a Swarm - the preferred approach will be to use Distributed Application Bundles, but that's currently (as of Docker 1.12) experimental. You'll basically create a bundle from a local Compose file which represents your distributed system, and then deploy that remotely to a Swarm. Docker moves fast, so I would expect that functionality to be available soon.
You can find in their documentation more information about using docker-compose in production. But, as they mention, compose is primarily aimed at development and testing environments.
If you want to use your containers in production, I would suggest you to use a suitable tool to orchestrate containers, as Kubernetes.
If you can organize your Django application as a swarmkit service (docker 1.11+), you can orchestrate the execution of your application with Task.
Swarmkit has a restart policy (see swarmctl flags)
Restart Policies: The orchestration layer monitors tasks and reacts to failures based on the specified policy.
The operator can define restart conditions, delays and limits (maximum number of attempts in a given time window). SwarmKit can decide to restart a task on a different machine. This means that faulty nodes will gradually be drained of their tasks.
Even if your "cluster" has only one node, the orchestration layer will make sure your containers are always up and running.
You say that you use AWS so why don't you use ECS which is built for what you ask. You create an application which is the pack of your 5 containers. You will configure which and how many instances EC2 you want in your cluster.
You just have to convert your docker-compose.yml to the specific Dockerrun.aws.json which is not hard.
AWS will start your containers when you deploy and also restart them in case of crash
As mentioned in the title, I thinking about a dockerized jenkins. I have a running container that run all tests but now I want to run some deployment job.
The files (.py, .conf, .sh) will be copied into folders which are mounted by other container (app container). As I seen some recommend do not use docker as well.
Now I'm wondering if I should continue to use jenkins in a container (so i must find a way to run the deployment script) or prefer to install it on the server ?
If you are running dockerized Jenkins for production, It is good practice to have its volume mounted on Docker host.
I personally do not prefer dockerized Jenkins for production due to non static IP for Jenkins, and reliability issues with docker networking. For non-production use, i dockerize Jenkins.
We're experimenting with containerizing Jenkins in production - the flexibility of being able to easily set up or move instances offsets the learning pain, and that pain is :
1 - Some build jobs are themselves containerized, requiring that you run docker-in-docker. This is possible by passing the host docker.sock into the Jenkins' container. (more reading : https://getintodevops.com/blog/the-simple-way-to-run-docker-in-docker-for-ci). It requires that the host and Jenkins container are running identical versions of Docker, but I can live with that.
2 - SSH keys are a bigger issue. ssh agent forwarding in Docker is notorious for its unreliability, and we've always copied keys into containers (ignoring security questions for the context of this question). In an on-the-host Jenkins instance we put our ssh keys in Jenkins' home folder and everything works seamlessly. But, dockerized Jenkins has its home folder inside a Docker volume, which is owned by the host system, so keys are too open. We got around this by copying the keys to a folder outside Jenkins' home, chown/chmod'ing those keys to the Jenkins container user, then adding the key path to the container's /etc/ssh/ssh_config.
I am having hard time figuring how I should combine Chef and Docker to get the best of them.
Right now I am using Chef to automatically pull a docker image and create a container.
But things get messy when I want to change the configuration inside the container.
I read about knife container but I didn't understand how one can bootstrap a container and a new vm (on Amazon for example) all together.
I would suggest that if all you want to do is manage Docker images/containers, that you don't really need Chef.
Docker provides tools like:
Fig (http://www.fig.sh/), which brings up multiple containers as one logical unit.
Swarm (https://github.com/docker/swarm/), which allows you to abstract away the machines you have for deployments. For example, "My app needs 2GB of RAM, 1 CPU, 10GB of HD, which machine has available resources?"
Machine (https://github.com/docker/machine), which allows you to create VMs in the cloud in pretty much any provider.
A REST API (https://docs.docker.com/reference/api/docker_remote_api/), which allows you to remotely start/stop containers etc.
In my opinion those suite of tools replace the need for Chef if all you're going to do is manage Docker images and containers.
As someone already noted, don't change configs after a container has started. Better to make a new image or restart the container. You could also mount the configs external to the container and modify them there, then restart the container.
Consider the following scenario:
There are three independent web applications A, B and C that require an apache server to be run on.
A linux server runs web application A and may act as a Docker host. It is required that the applications B and C are isolated from the linux server and each other. They are therefore realized as two Docker containers, initially created from the same image.
My question updates, and security updates in special.
Do the two Docker containers require a full OS installation image? Can they share the host's apache binary, so security updates of the host's apache could be propagated to the containers automatically?
If this doesn't work: Would I need to install updates for apache on both containers independently, or could I benefit from the fact, that they are based on the same image, and somehow simplify the update process?
Docker will not share binaries from the host server that would be missing the point. What docker does have is a layered file system which means two docker images that share a common base will be more space efficient when sharing the same host.
As for patching. If you update the base image, you'll need to rebuild the container images that derive from it. Not a big deal, if you're using a registry to store and distribute images. to update the host server you perform a "pull" operation to update the local images, followed by a container restart.