I'm currently trying to bridge the gap between persistent, but unique volumes while scaling containers with Rancher (alternatively Docker Compose, since this is more of an abstract question).
Take as an example a Minecraft server, I have a Service defined in Rancher/Compose which uses a named volume as its data/world directory (e.g. -v minecraft_data:/data where the Minecraft image loads its world files from this /data directory). The reason I'm using such a named volume, is that I want it to persist between service upgrades (e.g. I'm changing the image version, or want to change some environment variables), which would not be possible with an anonymous volume.
Now when trying to scale up my service, I'm either getting multiple containers accessing the same data (not good for many use cases), or losing the service upgradeability when using anonymous volumes.
Are there any tools, best practices or patterns that might help with this issue?
In current versions of rancher (v1.4 at this time) storage drivers can be plugged in at the environment infrastructure level. This allows you to create volumes that are scoped at the environment, stack, or container.
For your use case, it sounds like per-container scope is what you need. Using rancher-compose you do something like:
version: '2'
services:
foo:
image: busybox
volumes:
- bar:/var/lib/storage
command: /bin/sh -c 'while true; do sleep 500; done'
volumes:
bar:
per_container: true
Then, rancher-compose up -d will create the stack and service with one container and a unique volume. rancher scale foo=2 will create another container with its own volume, etc. You can also specify volume storage drivers for each volume like rancher-ebs or rancher-nfs with their respective options.
I think what you want is to have difference instances of the entire project. scale implies identical clones, but if they have different data, they are not identical.
Instead of using scale, I would start different instances with different project names: https://docs.docker.com/compose/overview/#multiple-isolated-environments-on-a-single-host
Related
I have a task that I already solved, but where I'm not satisfied with the solution. Basically, I have a webserver container (Nginx) and a fast-CGI container (PHP-FPM). The webserver container is built on an off-the-shelf image, the FCGI container is based on a custom image and contains the application files. Now, since not everything is sourcecode and processed on the FCGI container, I need to make the application files available inside the webserver container as well.
Here's the docker-compose.yml that does the job:
version: '3.3'
services:
nginx:
image: nginx:1-alpine
volumes:
- # customize just the Nginx configuration file
type: bind
source: ./nginx.conf
target: /etc/nginx/nginx.conf
- # mount application files from PHP-FPM container
type: volume
source: www-data
target: /var/www/my-service
read_only: true
volume:
nocopy: true
ports:
- "80:80"
depends_on:
- php-fpm
php-fpm:
image: my-service:latest
command: ["/usr/sbin/php-fpm7.3", "--nodaemonize", "--force-stderr"]
volumes:
- # create volume from application files
# This one populates the content of the volume.
type: volume
source: www-data
target: /var/www/my-service
volumes:
# volume with application files shared between nginx and php-fpm
www-data:
What I don't like here is mostly reflected by the comments concerning the volumes. Who creates and stores data should be obvious from the code and not from comments. Also, what I really dislike is that docker actually creates a place where it stores data for this volume. Not only does this use up disk space and increase startup time, it also requires me to never forget to use docker-compose down --volumes in order to refresh the content on next start. Imagine my anger when I found out that down didn't tear down what up created and that I was hunting ghosts from previous runs.
My questions concerning this:
Can I express in code that one container contains data that should be made available to other containers more clearly? The above code works, but it fails utterly to express the intent.
Can I avoid anything persistent being created to avoid above mentioned downsides?
I would have liked to investigate into things like tmpfs volumes or other volume options. My problem is that I can't find documentation for available volume drivers or even explore which volume drivers exist. Maybe I have missed some CLI for that, I'd really appreciate a nudge in the right direction here.
You can use the local driver with option type=tmpfs, for example:
volumes:
www-data:
driver: local
driver_opts:
type: tmpfs
device: tmpfs
Which will follow your requirements:
Data will be shared between containers at runtime
Data should not be persisted, i.e. volumes will be emptied when container are stopped, restarted or destroyed
This is CLI equivalent of
docker volume create --driver local --opt type=tmpfs --opt device=tmpfs www-data
Important note: this is NOT a Docker tmpfs mount, but a Docker volume using a tmpfs option. As stated in the local volume driver documentation it uses the Linux mount options, in our case --types to specify a tmpfs filesystem. Contrary to a simple tmpfs mount, it will allow you to share the volume between container while retaining classic behavior of a temporary filesystem
I can't find documentation for available volume drivers or even explore which volume drivers exist
Volume doc, including tmpfs, Bind mount and Volumes
local driver options are found in the docker volume create doc
Volume driver plugins - some are still updated regularly or seem maintained, but most of them have not been updated for a long time or are deprecated. The list does not seem exhaustive though, for instance vieux/sshfs is not mentioned.
Can I express in code that one container contains data that should be made available to other containers more clearly? The above code works, but it fails utterly to express the intent.
I don't think so, your code is are already quite clear as to the intent of this volume:
That's what volume are for: sharing data between containers. As stated in the doc Volumes can be more safely shared among multiple container, furthermore only containers are supposed to write into volumes.
nocopy and read_only clearly express that nginx relies on data written by another container as it will only be able to read from this volume
Given your volume is not external, it is safe to assume only another container from the same stack can use it
A bit of logic and experience with Docker allow to quickly come to the previous point, but even for less experienced Docker users your comments gives clear indications, and your comments are part of the code ;)
You also can make custom nginx image with copy of static from your php image
here is Dockerfile for nginx
FROM my-service:latest AS src-files
FROM nginx
COPY --from=src-files /path-to-static-in-my-service-image /path-to-static-in-nginx
This will allow you to use no volumes with source code
Also can use TAG from env variables in Dockerfile
FROM my-service:${TAG} AS src-files
...
It depends of the usecase of your setup. If it's only for local dev or if you want the same thing on production. On dev, having a volume populated manually or by one container for others could be OK.
But if you want something that will run the same way in production, you may need something else. For exemple, in production, I don't want to have my code in a volume but in my image in an immutable way and I just need to redeploy it.
For me a volume is not for storing application code but for storing data like cache, user uploaded content, etc. Something we want to keep between deployments.
So, if we want to have 2 images with the same files not in a volume, I will build 2 images with the application code and static content, one for php, one for nginx.
But the deployment is usually not synchrone for the 2 images. We solve this issue by deploying the PHP application first and the nginx application after. On nginx, we add a config that try to serve static content from it first and if the file doesn't exist, to ask it to PHP.
For the dev environment, we will reuse the same image but use a volume to mount the current code inside the container (an host bind mount).
But in some case, the bind mount could have some issues:
- On Mac the file sharing is slow but it should be better with the latest version of Docker Desktop in the Edge channel (2.3.1.0)
- On Windows, the file sharing is slow too
- On Linux, we need to be careful about the file permission and the user used inside the container
If you try to solve one/many of this issues with the volume solution, we could find some solution for that. Ex, on Mac, I will try to Edge release of docker first, on Windows, if possible, I will use WSL2 and Docker set to use the WSL2 backend.
I am new to docker and volumes and is confused about how named volumes are working. I have two scenarios in which I want to know how the named volumes will work
First Scenario
I have to setup two projects with docker and both have separate databases. Now how the database volumes will be mapped with /var/lib/mysql? Does it maintain separate data based on db name?
Second Scenario
I have two services using same named volume. In both the services, the path of container mapped to the named volume is different. How this will work?
services:
s1:
volume:
- vol:/var/lib/s1
s2:
volume:
- vol:/var/lib/s2
volumes:
vol:
Since you are using docker-compose, it does some things for you. If your composed "project name" is project_a, the docker-compose vol volume will be named project_a_vol. Verify this by running docker volume ls. By a "composed project name" I mean the name of the project which usually equals to the name of the directory in which the docker-compose was run, or custom one if the --project-name parameter was set (eg. docker-compose --project-name xxx up)
I assume you're using the default docker volume filesystem storage driver. A named volume is nothing more than a directory inside the /var/lib/docker/volumes folder (try it sudo ls -l /var/lib/docker/volumes). By mounting a volume using vol:/var/lib/s1 you tell docker to synchronize directories:
Local /var/lib/docker/volumes/project_a_vol with container directory /var/lib/s1.
If you compose your services this way:
services:
s1:
volumes:
- vol:/var/lib/s1
s2:
volumes:
- vol:/var/lib/s2
The same directory will be mounted to 2 services: s1 and s2 and you most probably will have a problem because 2 services will try to read & write to the same directory at the same time. Unless those services can handle such case.
It's better to have separate volumes though. In such case a volume for one service can be purged leaving the other one intact.
Some hints to your questions.
First Scenario : Two docker containers with a DB each.
- For this scenario since the databases are different, they run on their own container space.
- You can create a Docker Volume or use Docker bind mounts to attach disk to your Database `/var/lib/mysql'
- If you use volumes, you create one volume per database and the data are isolated.
- If you use bind mounts, make sure you mount different disk locations, if you use same location, the second database container data will overwrite the first database data.
Second Scenario : As per this scenario, since the Volume label is same, the second coming up service data would replace the already running Service data every time the services start.
What's the difference between declaring in the docker-compose.yml file a volume section and just using the volumes keyword under a service?
For example, I map a volume this way for a container:
services:
mysqldb:
volumes:
- ./data:/var/lib/mysql
This will map to the folder called data from my working directory.
But I could also map a volume by declaring a volume section and use its alias for the container:
services:
mysqldb:
volumes:
- data_volume:/var/lib/mysql
volumes:
data_volume:
driver: local
In this method, the actual location of where the mapped files are stored appears to be somewhat managed by docker compose.
What are the differences between these 2 methods or are they the same? Which one should I really use?
Are there any benefits of using one method over the other?
The difference between the methods you've described is that first method is a bind mount, and the other is a volume. These are more of Docker functions (rather than Docker Compose), and there are several benefits volumes provide over mounting a path from your host's filesystem. As described in the documentation, they:
are easier to back up or migrate
can be managed with docker volumes or the API (as opposed to the raw filesystem)
work on both Linux and Windows containers
can be safely shared among multiple containers
can have content pre-populated by a container (with bind mounts sometimes you have to copy data out, then restart the container)
Another massive benefit to using volumes are the volume drivers, which you'd specify in place of local. They allow you to store volumes remotely (i.e. cloud, etc) or add other features like encryption. This is core to the concept of containers, because if the running container is stateless and uses remote volumes, then you can move the container across hosts and it can be run without being reconfigured.
Therefore, the recommendation is to use Docker volumes. Another good example is the following:
services:
webserver_a:
volumes:
- ./serving/prod:/var/www
webserver_b:
volumes:
- ./serving/prod:/var/www
cache_server:
volumes:
- ./serving/prod:/cache_root
If you move the ./serving directory somewhere else, the bind mount breaks because it's a relative path. As you noted, volumes have aliases and have their path managed by Docker, so:
you wouldn't need to find and replace the path 3 times
the volume using local stores data somewhere else on your system and would continue mounting just fine
TL;DR: try and use volumes. They're portable, and encourage practices that reduce dependencies on your host machine.
I want to use Docker MySQL.
docker run mysql
But I don't want to save data on the host machine. I want all the information to be protected inside the container. By default, this image created an unnamed volume, and attach it to the container.
Is it possible, to use the same container, (I don't want to create a new MySQL image from ground), but disable the volume?
In other words: Many Docker images in docker hub are using volumes by default. What is the easiest way to save all the data inside the container (so push, and commit will contain the data)? There is a command to stop a container, change it's Mounts settings, and start again?
I know that it is not best practice, my question is if it is possible.
EDIT: There is a tool mentioned in the comments of the below thread that can edit docker image metadata, allowing you to remove a volume.
This is currently an open issue awaiting someone with the bandwidth to code it. You can track the progress here, with this link going directly to the applicable comment:
#veqryn since reopening this issue, nobody started working on a pull-request; the existing pull request did no longer apply cleanly on the code-base so a new one has to be opened; if anyone is interested in working on this, then things can get going again.
I too would like this feature! Mounting /var/lib/mysql/ on windows hosts with NTFS gives the volume root:root permissions which can't be chown'd; I don't want to add mysql user to the root group. I would like to UNVOLUME the /var/lib/mysql directory and replace it with a symlink that does have mysql:mysql permissions, pointed at /host/ntfs/mnt which is root:root 🤷♀️
As shown in this question, you can create, name and associate a container volume easily enough to the default unnamed one of mysql
version: '2'
services:
db:
image: mysql
volumes:
- dbdata:/var/lib/mysql
volumes:
dbdata:
driver: local
See "Mount a shared-storage volume as a data volume": you can uise other drivers, like flocker, and benefit from a multi-host portable volume.
I have the following example
version: '2'
services:
proxy:
container_name: proxy
hostname: proxy
image: nginx
ports:
- 80:80
- 443:443
volumes:
- proxy_conf:/etc/nginx
- proxy_htdocs:/usr/share/nginx/html
volumes:
proxy_conf: {}
proxy_htdocs: {}
which works fine. When I run docker-compose up it creates those named volumes in /var/lib/docker/volumes and all is good. However, from the host, I can only access /var/lib/docker as root, because it's root:root (makes sense). I was wondering if there is a way of chowning the host's directories to something more sensible/safe (like, my relatively unprivileged user that I use to do most things on the host) or if I just have to suck it up and chown them manually. I'm starting to have a number of scripts already to work around other issues, so having an extra couple of lines won't be much of a problem, but I'd really like to keep my self-written automation minimal, if I can -- fewer chances for stupid mistakes.
By the way, no: if I mount host directories instead of creating volumes, they get overlaid, meaning that if they start empty, they stay empty, and I don't get the default configuration (or whatever) from inside the container.
Extra points: can I just move the volumes to a more convenient location? Say, /home/myuser/myserverstuff/volumes?
It's best to not try to access files inside /var/lib/docker directly. Those directories are meant to be managed by the docker daemon, and not to be messed with.
To access the data inside a volume, there's a number of options;
use a bind-mounted directory (you considered that, but didn't fit your use case).
use a "service" container that uses the same volume and makes it accessible through that container, for example a container running ssh (to use scp) or a SAMBA container (such as svendowideit/samba)
use a volume-driver plugin. there's various plugins around that offer all kind of options. For example, the local persist plugin is a really simple plug-in that allows you to specify where docker should store the volume data (so outside of /var/lib/docker)