I have two volumes: repo and cache. I want to mount repo read-only at /repo and I want to mount cache as read-write at /repo/cache. When the app writes to /repo/cache, I expect the writes to go to the cache volume, while the /repo volume is not changed; however, when I run docker run -it -v repo:/repo:ro -v cache:/repo/cache alpine, I get docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:430: container init caused \"rootfs_linux.go:58: mounting \\\"/var/lib/docker/volumes/cache/_data\\\" to rootfs \\\"/var/lib/docker/overlay2/9f1dfff7f943921fd2da278d357bec88007d7fcb41eb9bc34fc0e728c459ad73/merged\\\" at \\\"/var/lib/docker/overlay2/9f1dfff7f943921fd2da278d357bec88007d7fcb41eb9bc34fc0e728c459ad73/merged/repo/cache\\\" caused \\\"mkdir /var/lib/docker/overlay2/9f1dfff7f943921fd2da278d357bec88007d7fcb41eb9bc34fc0e728c459ad73/merged/repo/cache: read-only file system\\\"\"": unknown.
Is there a better way to achieve this result? I need the repo volume to be ro so the app doesn't write to it, and I need cache to be writable and mounted "inside" of /repo because that's where the app expects it (and I can't modify the app to look elsewhere).
To mount your cache volume on /repo/cache, Docker would need to be able to create a directory /repo/cache inside the container. Since you have mounted the repo volume read-only, this isn't possible.
If you first create the mountpoint yourself:
docker run --rm -v repo:/repo alpine mkdir /repo/cache
Then you will be able to successfully mount /repo/cache despite repo being read-only:
bash-5.0$ docker run --rm -it -v repo:/repo:ro -v cache:/repo/cache alpine
/ # ls /repo
cache
Related
I would like to add a file system view of Arvados Keep to a Docker container. I created this file system view with arv-mount and it is based on File System in Userspace (FUSE).
Approach 1
$ docker run --rm -it -v /home/test/arv:/opt ubuntu:hirsute bash
docker: Error response from daemon: error while creating mount source path '/home/test/arv': mkdir /home/test/arv: file exists.
Approach 2
I also tried bind mounts
$ docker run --rm -it --mount type=bind,source=/home/test/arv,target=/opt,bind-propagation=rshared ubuntu:hirsute bash
docker: Error response from daemon: invalid mount config for type "bind": stat /home/test/arv: permission denied.
Both approaches I tried as non-root user (I configured Docker for non-root users) and root user.
I made it working by
Adding user_allow_other to /etc/fuse.conf
Creating FUSE mount with arv-mount --allow-other /home/test/arv
First create a volume sample_vol
docker volume create sample_vol
My Dockerfile
FROM archlinux/base
RUN touch /root/testing [**edited** find note at RUN below]
# VOLUME sample_vol:/root [**edited** this will not work, because VOLUME will not accpet named volumes. So this will not mount at /root, it will mount at sample_vol:/root which does not exist]
VOLUME "/root" or VOLUME ["/root"] [**edited** this will create a local mount volume only till the time the container is running. I tried to use named volumes like VOLUME ["name:/root"] but didnt work ]
# RUN touch /root/testing [**edited** this will not work because volume when mounted will only copy files till it got declared]
build the image
docker build -t archlinux/sample_vol .
checking whether testing file is created in sample_vol
docker run --rm -it -v=sample_vol:/tmp/myvolume archlinux/base ls /tmp/myvolume
It does not show any file testing created
while
$ docker run --rm -it --name sample_vol archlinux/sample_vol ls /root/testing
It shows the file testing is created in the /root/ of build image
So why sample_vol is not mounted at /root and testing is created inside it.
Update: Reason i found can be due to
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#volume
Changing the volume from within the Dockerfile: If any build steps
change the data within the volume after it has been declared, those
changes will be discarded.
You are misunderstanding docker-volume.
Docker-Image are more about build time.
Docker-Volume is useful only in runtime.
Try running following commands to get an idea:
docker run --rm -it -v=sample_vol:/tmp/myvolume archlinux/base touch /tmp/myvolume/1.txt
docker run --rm -it -v=sample_vol:/tmp/myvolume archlinux/base touch /tmp/myvolume/2.txt
docker run --rm -it -v=sample_vol:/tmp/myvolume archlinux/base touch /tmp/myvolume/3.txt
docker run --rm -it -v=sample_vol:/tmp/myvolume archlinux/base ls -altr /tmp/myvolume/
1st container create a file 1.txt in docker volume mounted at /tmp/myvolume and then container gets deleted after this operation.
2nd container create a file 2.txt in docker volume mounted at /tmp/myvolume and then container gets deleted after this operation.
3rd container create a file 3.txt in docker volume mounted at /tmp/myvolume and then container gets deleted after this operation.
4th container list files in docker volume mounted at /tmp/myvolume and then container gets deleted after this operation.
Docker volume is to store persistent data outside of the lifecycle of container.That means when you remove container , you still have data outside of the container living inside volume.
So next time if you create a container and attach that docker volume - you will automatically get all the data with new container.
Consider an example of database image where you want to have data in volume so that when you change the container to the higher version - you will get the old data in the new database.
I am looking at this example
docker run --rm --volumes-from myredis -v $(pwd)/backup:/backup debian cp /data/dump.rdb /backup/
from Using Docker book.
Why do we need --rm flag?
Why do we have --volumes-from?
The idea here is that
you have a redis container named myredis which has some volumes for persistent storage (that you'd like to backup).
you run a temporary debian container that will save the backup to your_current_dir/backup and get removed.
docker run --rm ... debian runs the container and removes it after it exits
--volumes-from myredis this way the debian container will have access to the database
-v $(pwd)/backup:/backup this second volume is used to put the backup at your current dir $(pwd)/backup. If it wasn't used, the backup would have only been copied to /backup (inside the container) and later been removed together with the container. This way the backup persists.
cp /data/dump.rdb /backup/ copies the actual files
The --rm flag tells Docker Engine to remove the container once it exits. Without this flag, you need to manually remove the container after you stop it.
The --volumes-from flag mounts all the defined volumes from the referenced containers, it ensures the two containers mounts same volumes.
I have Jenkins running in a Docker container. The home directory is in a host volume, in order to ensure that the build history is preserved when updates to the container are actioned.
I have updated the container, to create an additional file in the home directory. When the new container is pulled, I cannot see the changed file.
ENV JENKINS_HOME=/var/jenkins_home
RUN mkdir -p ${JENKINS_HOME}/.m2
COPY settings.xml ${JENKINS_HOME}/.m2/settings.xml
RUN chown -R jenkins:jenkins ${JENKINS_HOME}/.m2
VOLUME ["/var/jenkins_home"]
I am running the container like this:
docker run -v /host/directory:/var/jenkins_home -p 80:8080 jenkins
I had previous run Jenkins and so the home directory already exists on the host. When I pull the new container and run it, I see that the file .m2/settings.xml is not created. Why is this please?
Basically when you run:
docker run -v /host-src-dir:/container-dest-dir my_image
You will overlay your /container-dest-dir with what is in /host-src-dir
From Docs
$ docker run -d -P --name web -v /src/webapp:/webapp training/webapp python app.py
This command mounts the host directory, /src/webapp, into the
container at /webapp. If the path /webapp already exists inside the
container’s image, the /src/webapp mount overlays but does not remove
the pre-existing content. Once the mount is removed, the content is
accessible again. This is consistent with the expected behavior of the
mount command.
This SO question is also relevant docker mounting volumes on host
It seems you want it the other way around (i.e. the container is source and the host is destination).
Here is a workaround:
Create the volume in your Dockerfile
Run it without -v i.e.: docker run --name=my_container my_image
Run docker inspect --format='{{json .Mounts}}' my_container
This will give you output similar to:
[{"Name":"5e2d41896b9b1b0d7bc0b4ad6dfe3f926c73","Source":"/var/lib/docker/volumes/5e2d41896b9b1b0d7bc0b4ad6dfe3f926c73/_data","Destination":"/var/jenkins_home","Driver":"local","Mode":"","RW":true,"Propagation":""}]
Which means your dir as it is on container was mounted into the host directory /var/lib/docker/volumes/5e2d41896b9b1b0d7bc0b4ad6dfe3f926c73/_data
Unfortunately, I do not know a way to make it mount on a specific host directory instead.
when I run nodered with
docker run -v D:/mydir:/data
the content of /data is copied in my volume at first run, thats what I've expected.
If I make
docker run -v D:/mydir:/usr/src/node-red/node_modules nodered
Then the volume is empty
I was expecting to get the content of node_modules being copied in the volume at start time... what am I missing ?
I can illustrate that a little bit more :
docker run --rm -v d:/VM:/data nodered/node-red-docker ls /data
--> list files
docker run --rm ls /usr/src/node-red/node_modules
--> list content of node_modules
docker run --rm -v d:/VM:/usr/src/node-red/node_modules nodered/node-red-docker ls /usr/src/node-red/node_modules
--> is empty !
You're mounting host directories as volumes, so there isn't any copying going on - the mount path inside the container is being mapped to the path on the host, so you're seeing the contents of the host directory.
Volumes sit outside the Union File System when you mount them, so you don't get an overlay which merges the contents of the image and the contents of the host directory. Instead you're effectively bypassing the contents of the image for that volume, and repointing it to your host.
Samples:
touch /docker/nodered-modules/sample.txt
docker run --rm -v /docker/nodered-modules:/usr/src/node-red/node_modules nodered/node-red-docker ls /usr/src/node-red/node_modules
sample.txt
touch /docker/nodered-data/sample.txt
docker run --rm -v /docker/nodered-data:/data nodered/node-red-docker ls /data
sample.txt
The reason you're seeing a difference is because the /data volume is defined in the Dockerfile and empty in the image, so you see the contents of your host directory as expected. The modules directory isn't empty in the image, but you're repointing it to an empty directory on your host.
Docker does not support copying data from the base image into host directories that are mounted as container volumes.