I am looking at this example
docker run --rm --volumes-from myredis -v $(pwd)/backup:/backup debian cp /data/dump.rdb /backup/
from Using Docker book.
Why do we need --rm flag?
Why do we have --volumes-from?
The idea here is that
you have a redis container named myredis which has some volumes for persistent storage (that you'd like to backup).
you run a temporary debian container that will save the backup to your_current_dir/backup and get removed.
docker run --rm ... debian runs the container and removes it after it exits
--volumes-from myredis this way the debian container will have access to the database
-v $(pwd)/backup:/backup this second volume is used to put the backup at your current dir $(pwd)/backup. If it wasn't used, the backup would have only been copied to /backup (inside the container) and later been removed together with the container. This way the backup persists.
cp /data/dump.rdb /backup/ copies the actual files
The --rm flag tells Docker Engine to remove the container once it exits. Without this flag, you need to manually remove the container after you stop it.
The --volumes-from flag mounts all the defined volumes from the referenced containers, it ensures the two containers mounts same volumes.
Related
I want to make a backup of all my containers and volumes, so the easiest way would be to copy /var/lib/docker to another location.
However this directory also includes all the images, and I don't want to include them since they all can easily be re-downloaded from public sources.
So how can I copy this directory while excluding the images?
You have to differentiate between container backup and vol backup:
Backing up a container, that is, its configurations like labels, envs, etc.
You do that by committing the container as an image:
$ docker container commit <container-name/id> <name-of-new-image>
Better give it also some metainfo:
$ docker container ... -m "very important container config state" -a "John Doe"
Backing up a volume
Let's say the volume of interest <my-vol> is bound to a container <other-container> - which may have been created like: docker container run -v /my-vol other-container ...
So first you have to bind the volume also to a newly created temporary container with the --volumes-from flag. With the -v option you mount a local path (of the host) into the container:
$ docker container run -rm --volumes-from <other-container> \
-v <dir/on/host>:<mountpath/in/container> \
<ubuntu/centos/whatever-base-image> tar cvf <mountpath/in/container>/backup.tar /<my-vol>
After completing the command the container stops and with that it will also be deleted because of the -rm option.
Whith all that the steps are:
bind the volume to a temp container
mount a hostpath into the container
make a tarbal (or whatever kind of backup)
of the volume in the container
container stops and is deleted after the backup command has finished
the backup tarbal is left on the mounted dir of the container host.
see also: https://docs.docker.com/storage/volumes/
Shell Command
.. the other - not recommended - way would be to do it just with os level commands:
shopt -s extglob
cp -r var/lib/docker/!(image) your/path/backup
For that you have to stop all involved containers to prevent read/write issues.
Problem:
I'd like to create regular backups of a docker volume used by a running docker container.
Partial solutions:
If I shut down the container using the volume, I found these solutions:
Save to .tar like:
docker run --rm -v some_volume:/volume -v /tmp:/backup alpine tar -cjf
/backup/some_archive.tar.bz2 -C /volume ./
(Source: https://medium.com/#loomchild/backup-restore-docker-named-volumes-350397b8e362)
Stackoverflow solution:
How can I backup a Docker-container with its data-volumes?
Question:
Using either docker or also docker-compose, how do I backup a volume, without any downtime of the app container using the volume?
I am trying to create a wrapper container to build and run a set of containers using a docker-compose I cannot modify. The docker-compose mounts several volumes, but when starting the docker-compose from inside of the wrapper docker, the volumes are still mounted from the host since the docker .sock is volume mounted to be the host's docker.sock.
I would like to not have to use full docker-in-docker due to all the problems associated with it outlined in jpetazzo's article.
I would also like to avoid volume-from since I cannot edit the docker-compose file mentioned previously.
Is there a way to get this snippet to correctly use the parent docker's file instead of going to the host filesystem and mounting it from there?
FROM docker:latest
RUN mkdir -p /tmp/parent/ && echo "This is from the parent docker" > /tmp/parent/parent.txt
CMD docker run -v /tmp/parent/parent.txt:/root/parent.txt --rm ubuntu:18.04 bash -c "cat /root/parent.txt"
when run with a command akin to this:
docker build -t parent . && docker run --rm -v /var/run/docker.sock:/var/run/docker.sock parent
Make your paths the same on the host and inside of the docker image, e.g.
docker run -v /var/run/docker.sock:/var/run/docker.sock \
-v /home/user:/home/user -w /home/user/project parent_image ...
By mounting the volume as /home/user in the same location inside the image, a command like docker-compose up with relative bind mounts will use the container path names when talking to the docker socket, which will match the paths on the host.
I have been trying to setup a graph database using orientdb. So I tried using volumes by the following command
docker run -d -p 2424:2424 -p 2480:2480 -v config:/orientdb/config -v database:/orientdb/databases -v backup:/orientdb/backup -e ORIENTDB_ROOT_PASSWORD=mypasswdhere orientdb:latest
My prime motive behind using volumes was to store data in database after I kill the container.
But I used this command frequently to start the server.
Now it has hogged my disk space so I guess it creates a new copy each time this command is executed.
Can someone indicate a correct way to use existing volumes to use stored data in docker and to clean up the redundant data recreated by frequent execution of this command?
You can create named volumes with docker volume create
$ docker volume create --name hello
$ docker run -d -v hello:/world busybox ls /world
That way, only one volume in /var/lib/docker/volumes will be used each time you launch that container.
See also "Mount a shared-storage volume as a data volume".
In the meantime, to remove dangling volumes:
docker volume ls -qf "dangling=true" | xargs docker volume rm
As far as I understand, you aren't re-using the container, instead you start a new one each time.
After the first run, you can stop and the restart it with docker stop/start commands.
I would like to persist some configuration data from a container and am following the tutorial on data volumes.
I'm successfully running the app with:
docker run -it --privileged -v /app/config -p 8083:8083 myapp-ubuntu:2.2.2
Where -v /app/config is the directory inside the container that contains the config that should survive a container restart.
Also the result of running the container creates a volume in /var/lib/docker/volumes.
# ls /var/lib/docker/volumes
5e60d70dc15bcc53aa13cfd84507b5758842c7743d43da2bfa2fc121b2f32479
However, if I kill the container and rerun it no data is persisted and a new volume is created in /var/lib/docker/volumes:
# ls /var/lib/docker/volumes
5e60d70dc15bcc53aa13cfd84507b5758842c7743d43da2bfa2fc121b2f32479 (FIRST RUN)
82de3aa910bc38157a6dc20a516b770bd0264860ae83093d471212f69960d02a (SECOND RUN)
I would expect that these would be the steps for persisting, am I missing something here?
I think you can solve it with named volumes:
docker run -it --privileged -v some_named_volume:/app/config -p 8083:8083 myapp-ubuntu:2.2.2
Or you can use Dockerfile
with directive COPY