Why docker in docker (dind) containers mount volumes with host path? - docker

I have a setup with docker in docker and try to mount folders.
Let's say I have those folders that I wish to share with his parent. On the host, I created a file in /tmp/dind called foo. Host starts container 1, which starts container 2. This is the result I want to have.
Host | Container 1 | Container 2
/tmp/dind | /tmp/dind2 | /tmp/dind3
<-------> <------->
Instead, I get
Host | Container 1 | Container 2
/tmp/dind | /tmp/dind2 | /tmp/dind3
<------->
<----------------------->
Code here:
docker run --rm -it \
-v /tmp/dind:/tmp/dind2 \
-v /var/run/docker.sock:/var/run/docker.sock docker sh -c \
"docker run --rm -it \
-v /tmp/dind2:/tmp/dind3 \
-v /var/run/docker.sock:/var/run/docker.sock \
docker ls /tmp/dind3"
This outputs nothing, while the next command gives foo as result. I changed the mounted volume:
docker run --rm -it \
-v /tmp/dind:/tmp/dind2 \
-v /var/run/docker.sock:/var/run/docker.sock docker sh -c \
"docker run --rm -it \
-v /tmp/dind:/tmp/dind3 \
-v /var/run/docker.sock:/var/run/docker.sock \
docker ls /tmp/dind3"
The question is, what do I need to do in order to use Container 1 path and not host? Or do I misunderstand something about docker here?

For all that you say “Docker-in-Docker” and “dind”, this setup isn’t actually Docker-in-Docker: your container1 is giving instructions to the host’s Docker daemon that affect container2.
Host Container1
/-----
(Docker)
| Container2
\---->
(NB: this is generally the recommended path for CI-type setups. “Docker-in-Docker” generally means container1 is running its own, separate, Docker daemon, which tends to not be recommended.)
Since container1 is giving instructions to the host’s Docker, and the host’s Docker is launching container2, any docker run -v paths are always the host’s paths. Unless you know that some specific directory has already been mounted into your container, it’s hard to share files with “sub-containers”.
One way to get around this is to assert that there is a shared path of some sort:
docker run \
-v $PWD/exchange:/exchange \
-v /var/run/docker.sock:/var/run/docker.sock \
-e EXCHANGE_PATH=$PWD/exchange \
--name container1
...
# from within container1
mkdir $EXCHANGE_PATH/container2
echo hello world > $EXCHANGE_PATH/container2/file.txt
docker run \
-v $EXCHANGE_PATH/container2:/data
--name container2
...
When I’ve done this in the past (for a test setup that wanted to launch helper containers) I’ve used a painstaking docker create, docker cp, docker start, docker cp, Docker rm sequence. That’s extremely manual, but it has the advantage that the “local” side of a docker cp is always the current filesystem context even if you’re talking to the host’s Docker daemon from within a container.

It does not matter if container 2 binds the host path, because the changes to files in container 1 directly affect everything on the host path. So they all work on the same files.
So your setup is correct and will function the same as if they referenced in the way you described.
Update
If you want to make sure that the process do not modify the host files you could do the following:
Build a custom docker images which copies all data from folder a to folder b, where you execute the script on folder b. And then mount the files with ./:/a. This way you maintain flexibility on which files you bind to the container without letting the container modify the host files.
I hope this answers your question :)

Related

Docker inside Docker via -v /var/run/docker.sock:/var/run/docker.sock and volume inside the Docker [duplicate]

I have a setup with docker in docker and try to mount folders.
Let's say I have those folders that I wish to share with his parent. On the host, I created a file in /tmp/dind called foo. Host starts container 1, which starts container 2. This is the result I want to have.
Host | Container 1 | Container 2
/tmp/dind | /tmp/dind2 | /tmp/dind3
<-------> <------->
Instead, I get
Host | Container 1 | Container 2
/tmp/dind | /tmp/dind2 | /tmp/dind3
<------->
<----------------------->
Code here:
docker run --rm -it \
-v /tmp/dind:/tmp/dind2 \
-v /var/run/docker.sock:/var/run/docker.sock docker sh -c \
"docker run --rm -it \
-v /tmp/dind2:/tmp/dind3 \
-v /var/run/docker.sock:/var/run/docker.sock \
docker ls /tmp/dind3"
This outputs nothing, while the next command gives foo as result. I changed the mounted volume:
docker run --rm -it \
-v /tmp/dind:/tmp/dind2 \
-v /var/run/docker.sock:/var/run/docker.sock docker sh -c \
"docker run --rm -it \
-v /tmp/dind:/tmp/dind3 \
-v /var/run/docker.sock:/var/run/docker.sock \
docker ls /tmp/dind3"
The question is, what do I need to do in order to use Container 1 path and not host? Or do I misunderstand something about docker here?
For all that you say “Docker-in-Docker” and “dind”, this setup isn’t actually Docker-in-Docker: your container1 is giving instructions to the host’s Docker daemon that affect container2.
Host Container1
/-----
(Docker)
| Container2
\---->
(NB: this is generally the recommended path for CI-type setups. “Docker-in-Docker” generally means container1 is running its own, separate, Docker daemon, which tends to not be recommended.)
Since container1 is giving instructions to the host’s Docker, and the host’s Docker is launching container2, any docker run -v paths are always the host’s paths. Unless you know that some specific directory has already been mounted into your container, it’s hard to share files with “sub-containers”.
One way to get around this is to assert that there is a shared path of some sort:
docker run \
-v $PWD/exchange:/exchange \
-v /var/run/docker.sock:/var/run/docker.sock \
-e EXCHANGE_PATH=$PWD/exchange \
--name container1
...
# from within container1
mkdir $EXCHANGE_PATH/container2
echo hello world > $EXCHANGE_PATH/container2/file.txt
docker run \
-v $EXCHANGE_PATH/container2:/data
--name container2
...
When I’ve done this in the past (for a test setup that wanted to launch helper containers) I’ve used a painstaking docker create, docker cp, docker start, docker cp, Docker rm sequence. That’s extremely manual, but it has the advantage that the “local” side of a docker cp is always the current filesystem context even if you’re talking to the host’s Docker daemon from within a container.
It does not matter if container 2 binds the host path, because the changes to files in container 1 directly affect everything on the host path. So they all work on the same files.
So your setup is correct and will function the same as if they referenced in the way you described.
Update
If you want to make sure that the process do not modify the host files you could do the following:
Build a custom docker images which copies all data from folder a to folder b, where you execute the script on folder b. And then mount the files with ./:/a. This way you maintain flexibility on which files you bind to the container without letting the container modify the host files.
I hope this answers your question :)

Docker mount host directory not container when using Docker inside Docker by docker.sock [duplicate]

I have a setup with docker in docker and try to mount folders.
Let's say I have those folders that I wish to share with his parent. On the host, I created a file in /tmp/dind called foo. Host starts container 1, which starts container 2. This is the result I want to have.
Host | Container 1 | Container 2
/tmp/dind | /tmp/dind2 | /tmp/dind3
<-------> <------->
Instead, I get
Host | Container 1 | Container 2
/tmp/dind | /tmp/dind2 | /tmp/dind3
<------->
<----------------------->
Code here:
docker run --rm -it \
-v /tmp/dind:/tmp/dind2 \
-v /var/run/docker.sock:/var/run/docker.sock docker sh -c \
"docker run --rm -it \
-v /tmp/dind2:/tmp/dind3 \
-v /var/run/docker.sock:/var/run/docker.sock \
docker ls /tmp/dind3"
This outputs nothing, while the next command gives foo as result. I changed the mounted volume:
docker run --rm -it \
-v /tmp/dind:/tmp/dind2 \
-v /var/run/docker.sock:/var/run/docker.sock docker sh -c \
"docker run --rm -it \
-v /tmp/dind:/tmp/dind3 \
-v /var/run/docker.sock:/var/run/docker.sock \
docker ls /tmp/dind3"
The question is, what do I need to do in order to use Container 1 path and not host? Or do I misunderstand something about docker here?
For all that you say “Docker-in-Docker” and “dind”, this setup isn’t actually Docker-in-Docker: your container1 is giving instructions to the host’s Docker daemon that affect container2.
Host Container1
/-----
(Docker)
| Container2
\---->
(NB: this is generally the recommended path for CI-type setups. “Docker-in-Docker” generally means container1 is running its own, separate, Docker daemon, which tends to not be recommended.)
Since container1 is giving instructions to the host’s Docker, and the host’s Docker is launching container2, any docker run -v paths are always the host’s paths. Unless you know that some specific directory has already been mounted into your container, it’s hard to share files with “sub-containers”.
One way to get around this is to assert that there is a shared path of some sort:
docker run \
-v $PWD/exchange:/exchange \
-v /var/run/docker.sock:/var/run/docker.sock \
-e EXCHANGE_PATH=$PWD/exchange \
--name container1
...
# from within container1
mkdir $EXCHANGE_PATH/container2
echo hello world > $EXCHANGE_PATH/container2/file.txt
docker run \
-v $EXCHANGE_PATH/container2:/data
--name container2
...
When I’ve done this in the past (for a test setup that wanted to launch helper containers) I’ve used a painstaking docker create, docker cp, docker start, docker cp, Docker rm sequence. That’s extremely manual, but it has the advantage that the “local” side of a docker cp is always the current filesystem context even if you’re talking to the host’s Docker daemon from within a container.
It does not matter if container 2 binds the host path, because the changes to files in container 1 directly affect everything on the host path. So they all work on the same files.
So your setup is correct and will function the same as if they referenced in the way you described.
Update
If you want to make sure that the process do not modify the host files you could do the following:
Build a custom docker images which copies all data from folder a to folder b, where you execute the script on folder b. And then mount the files with ./:/a. This way you maintain flexibility on which files you bind to the container without letting the container modify the host files.
I hope this answers your question :)

How to re-mount a docker volume without overriding existing files?

When running Docker, you can mount files and directories using the --volume option. E.g.:
docker run --volume /remote ./local myimage
I'm running a docker image that defines VOLUMESs in the Dockerfile. I need to access a config file that happens to be inside one of the defined volumes. I'd like to have that file "synced" on the host so that I can edit it. I know I could run docker exec ..., but I hope to circumvent that overhead for only editing one file. I found out that the volumes created by the VOLUMES line are stored in /var/lib/docker/volumes/<HASH>/_data.
Using docker inspect I was able to find the directory that is mounted:
docker inspect gitlab-runner | grep -B 1 '"Destination": "/etc/gitlab-runner"' | head -n 1 | cut -d '"' -f 4
Output:
/var/lib/docker/volumes/9c233c085c36380c6c33035222c16e5d061368c5060cc81dda2a9a713a2b2b3b/_data
So the question is:
Is there a way to re-mount volumes defined in an image? OR to somehow get the directory easier than my oneliner above?
EDIT after comments by zeppelin I've tried rebinding the volume with no success:
$ mkdir etc
$ docker run -d --name test1 gitlab/gitlab-runner
$ docker run -d --name test2 -v ~/etc:/etc/gitlab-runner gitlab/gitlab-runner
$ docker exec test1 ls /etc/gitlab-runner/
certs
config.toml
$ docker exec test2 ls /etc/gitlab-runner/
# empty. no files
$ ls etc
# also empty
docker inspect shows correctly that the volume is bound to ~/etc, but the files inside the container at /etc/gitlab-runner/ seem lost.
$ docker run -d --name test1 gitlab/gitlab-runner
$ docker run -d --name test2 -v ~/etc:/etc/gitlab-runner gitlab/gitlab-runner
You've got two different volume types there. One I call an anonymous volume (a very long uuid visible when you run docker volume ls). The second is a host volume or bind mount that maps a directory on the host directly into the container. So each container you spun up is looking at different places.
Anonymous volumes and named volumes (docker run -d -v mydata:/etc/gitlab-runner gitlab/gitlab-runner) get initialized to the contents of the image at that directory location. This initialization only happens when the volume is empty and is mounted into a new container. Host volumes, as you've seen, only get the contents of the host filesystem, even if it's empty at that location.
With that background, the short answer to your question is no, you cannot mount a file inside the container back out to your host. But you can copy the file out with several methods, assuming you don't overlay the source of the file with a host volume mount. With a running container, there's the docker cp command. Personally, I like:
docker run --rm -v ~/etc:/target gitlab/gitlab-runner \
cp -av /etc/gitlab-runner/. /target/.
If you have a named volume with data you want to copy in or out, you can use any image with the tools you need to do the copy:
docker run --rm -v mydata:/source -v ~/etc:/target busybox \
cp -av /source/. /target/.
Try to avoid modifying data inside a container from the host directly, much nicer is when you wrap your task into another container that you then start with "--volumes-from" option when possible in your case.
Not sure I understood your problem, anyway, as for the documentation you mention,
The VOLUME instruction creates a mount point with the specified name
and marks it as holding externally mounted volumes from native host or
other containers. [...] The docker run command initializes the newly
created volume with any data that exists at the specified location
within the base image.
So, following the example Dockerfile , after having built the image
docker build -t mytest .
and having the container running
docker run -d -ti --name mytestcontainer mytest /bin/bash
you can access it from the container itself, e.g.
docker exec -ti mytestcontainer ls -l /myvol/greeting
docker exec -ti mytestcontainer cat /myvol/greeting
Hope it helps.

what is 'z' flag in docker container's volumes-from option?

While going through the docker docs, I came across volumes-from (https://docs.docker.com/engine/reference/commandline/run/) option for docker run command.
I didn't understand the differences between ro, rw, and z option provided as-
$ docker run --volumes-from ba8c0c54f0f2:ro -i -t ubuntu pwd
In the above command the ro option is replaced with z. I will be thankful if anyone explores on differences of using these options.
Two suffixes :z or :Z can be added to the volume mount. These suffixes tell Docker to relabel file objects on the shared volumes. The 'z' option tells Docker that the volume content will be shared between containers. Docker will label the content with a shared content label. Shared volumes labels allow all containers to read/write content. The 'Z' option tells Docker to label the content with a private unshared label.
https://github.com/rhatdan/docker/blob/e6473011583967df4aa5a62f173fb421cae2bb1e/docs/sources/reference/commandline/cli.md
If you use selinux you can add the z or Z options to modify the selinux label of the host file or directory being mounted into the container. This affects the file or directory on the host machine itself and can have consequences outside of the scope of Docker.
The z option indicates that the bind mount content is shared among multiple containers.
The Z option indicates that the bind mount content is private and unshared.
Use extreme caution with these options. Bind-mounting a system directory such as /home or /usr with the Z option renders your host machine inoperable and you may need to relabel the host machine files by hand.
$ docker run -d \
-it \
--name devtest \
-v "$(pwd)"/target:/app:z \
nginx:latest
https://docs.docker.com/storage/bind-mounts/#configure-bind-propagation
From tests here in my machine, -z lets you share content from one container with another. Suppose this image:
FROM alpine
RUN mkdir -p /var/www/html \
&& echo "foo" > /var/www/html/index.html
Let's build it and tag as test-z:
$ docker build . -t test-z
Now create and run test-z container with the name testing-z, mapping the volume test-vol to /var/www/html and adding the z modifier
$ docker run \
--name testing-z \
--volume test-vol:/var/www/html:z \
-d test-z tail -f /dev/null
The contents of /var/www/html from testing-z can be accessed from others containers by using the --volumes-from flag, like below:
$ docker run --rm --volumes-from testing-z -it nginx sh
# cat /var/www/html/index.html
foo
Obs.: I'm running Docker version 19.03.5-ce, build 633a0ea838
docker run --volumes-from a64f10cd5f0e:z -i -t rhel6 bin/bash
I have tested it, i have mounted in one container and from that container to another newly container. IT goes with rw option
I've done the following observation:
# docker run --rm -ti -v /host/path/to/flyway/scripts:/flyway/sql:z --entrypoint '' flyway/flyway ls -l /flyway/sql
total 0
# docker run --rm -ti -v /host/path/to/flyway/scripts:/flyway/sql --entrypoint '' flyway/flyway ls -l /flyway/sql
ls: cannot open directory '/flyway/sql': Permission denied
So, in this case, the container works only if :z is set. On this host, SELinux is installed. If this is not the case, the :z doesn't have a recognizable effect to me.
Alternatively to :z, one could use chcon on the host folder to change this permission:
# chcon -t svirt_sandbox_file_t /host/path/to/flyway/scripts
# docker run --rm -ti -v /host/path/to/flyway/scripts:/flyway/sql:z --entrypoint '' flyway/flyway ls -l /flyway/sql
total 0
# docker run --rm -ti -v /host/path/to/flyway/scripts:/flyway/sql --entrypoint '' flyway/flyway ls -l /flyway/sql
total 0

How can I add a volume to an existing Docker container?

I have a Docker container that I've created simply by installing Docker on Ubuntu and doing:
sudo docker run -i -t ubuntu /bin/bash
I immediately started installing Java and some other tools, spent some time with it, and stopped the container by
exit
Then I wanted to add a volume and realised that this is not as straightforward as I thought it would be. If I use sudo docker -v /somedir run ... then I end up with a fresh new container, so I'd have to install Java and do what I've already done before just to arrive at a container with a mounted volume.
All the documentation about mounting a folder from the host seems to imply that mounting a volume is something that can be done when creating a container. So the only option I have to avoid reconfiguring a new container from scratch is to commit the existing container to a repository and use that as the basis of a new one whilst mounting the volume.
Is this indeed the only way to add a volume to an existing container?
You can commit your existing container (that is create a new image from container’s changes) and then run it with your new mounts.
Example:
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5a8f89adeead ubuntu:14.04 "/bin/bash" About a minute ago Exited (0) About a minute ago agitated_newton
$ docker commit 5a8f89adeead newimagename
$ docker run -ti -v "$PWD/somedir":/somedir newimagename /bin/bash
If it's all OK, stop your old container, and use this new one.
You can also commit a container using its name, for example:
docker commit agitated_newton newimagename
That's it :)
We don't have any way to add volume in running container, but to achieve this objective you may use the below commands:
Copy files/folders between a container and the local filesystem:
docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH
docker cp [OPTIONS] SRC_PATH CONTAINER:DEST_PATH
For reference see:
https://docs.docker.com/engine/reference/commandline/cp/
I've successfully mount /home/<user-name> folder of my host to the /mnt folder of the existing (not running) container. You can do it in the following way:
Open configuration file corresponding to the stopped container, which can be found at /var/lib/docker/containers/99d...1fb/config.v2.json (may be config.json for older versions of docker).
Find MountPoints section, which was empty in my case: "MountPoints":{}. Next replace the contents with something like this (you can copy proper contents from another container with proper settings):
"MountPoints":{"/mnt":{"Source":"/home/<user-name>","Destination":"/mnt","RW":true,"Name":"","Driver":"","Type":"bind","Propagation":"rprivate","Spec":{"Type":"bind","Source":"/home/<user-name>","Target":"/mnt"},"SkipMountpointCreation":false}}
or the same (formatted):
"MountPoints": {
"/mnt": {
"Source": "/home/<user-name>",
"Destination": "/mnt",
"RW": true,
"Name": "",
"Driver": "",
"Type": "bind",
"Propagation": "rprivate",
"Spec": {
"Type": "bind",
"Source": "/home/<user-name>",
"Target": "/mnt"
},
"SkipMountpointCreation": false
}
}
Restart the docker service: service docker restart
This works for me with Ubuntu 18.04.1 and Docker 18.09.0
Jérôme Petazzoni has a pretty interesting blog post on how to Attach a volume to a container while it is running. This isn't something that's built into Docker out of the box, but possible to accomplish.
As he also points out
This will not work on filesystems which are not based on block devices.
It will only work if /proc/mounts correctly lists the block device node (which, as we saw above, is not necessarily true).
Also, I only tested this on my local environment; I didn’t even try on a cloud instance or anything like that
YMMV
Unfortunately the switch option to mount a volume is only found in the run command.
docker run --help
-v, --volume list Bind mount a volume (default [])
There is a way you can work around this though so you won't have to reinstall the applications you've already set up on your container.
Export your container
docker container export -o ./myimage.docker mycontainer
Import as an image
docker import ./myimage.docker myimage
Then docker run -i -t -v /somedir --name mycontainer myimage /bin/bash
A note for using Docker Windows containers after I had to look for this problem for a long time!
Condiditions:
Windows 10
Docker Desktop (latest version)
using Docker Windows Container for image microsoft/mssql-server-windows-developer
Problem:
I wanted to mount a host dictionary into my windows container.
Solution as partially discripted here:
create docker container
docker run -d -p 1433:1433 -e sa_password=<STRONG_PASSWORD> -e ACCEPT_EULA=Y microsoft/mssql-server-windows-developer
go to command shell in container
docker exec -it <CONTAINERID> cmd.exe
create DIR
mkdir DirForMount
stop container
docker container stop <CONTAINERID>
commit container
docker commit <CONTAINERID> <NEWIMAGENAME>
delete old container
docker container rm <CONTAINERID>
create new container with new image and volume mounting
docker run -d -p 1433:1433 -e sa_password=<STRONG_PASSWORD> -e ACCEPT_EULA=Y -v C:\DirToMount:C:\DirForMount <NEWIMAGENAME>
After this i solved this problem on docker windows containers.
My answer will be little different. You can stop your container, add the volume and restart it. How to do it, follow the steps.
docker volume create ubuntu-volume
docker stop <container-name>
sudo docker run -i -t --mount source=ubuntu-volume,target=<target-path-in-container> ubuntu /bin/bash
You can stop and remove the container, append the existing volume in a startup script, and restart from the image. If the already existing existing partitions do keep the data, you shouldn't experience any loss of information. This should also work the same way with Dockerfile and Docker composer.
eg (solr image).
(initial script)
#!/bin/sh
docker pull solr:8.5
docker stop my_solr
docker rm solr:8.5
docker create \
--name my_solr \
-v "/XXXX/docker/solr/solrdata":/var/solr \
-p 8983:8983 \
--restart unless-stopped \
--user 1000:1000 \
-e SOLR_HEAP=1g \
--log-opt max-size=10m \
--log-opt max-file=3 \
solr:8.5
docker cp /home/XXXX/docker/solr/XXXXXXXX.jar my_solr:/opt/solr/contrib/dataimporthandler-extras/lib
docker start my_solr
file with the second volume
#!/bin/sh
docker pull solr:8.5
docker stop my_solr
docker rm solr:8.5
docker create \
--name my_solr \
-v "/XXXX/docker/solr/solrdata":/var/solr \
-v "/XXXX/backups/solr_snapshot_folder":/var/solr_snapshots \
-p 8983:8983 \
--restart unless-stopped \
--user 1000:1000 \
-e SOLR_HEAP=1g \
--log-opt max-size=10m \
--log-opt max-file=3 \
solr:8.5
docker cp /home/XXXX/docker/solr/XXXXXXXX.jar my_solr:/opt/solr/contrib/dataimporthandler-extras/lib
docker start my_solr
Use symlink to the already mounted drive:
ln -s Source_path targer_path_which_is_already_mounted_on_the_running_docker
The best way is to copy all the files and folders inside a directory on your local file system by: docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH
SRC_PATH is on container
DEST_PATH is on localhost
Then do docker-compose down attach a volume to the same DEST_PATH and run Docker containers by using docker-compose up -d
Add volume by following in docker-compose.yml
volumes:
- DEST_PATH:SRC_PATH

Resources