I have a test data which I build as a Docker image (docker build with copy to /db/data) and pushed to docker hub.
I want to run a db instance that will use that data.
I would expect to be able to:
run a "docker create" and create a container from image and map it to a volume (maybe named volume) which will practically will copy the data to that volume.
run a "docker run" with volumes-from and map that data from first container to the second.
When I tried it out I always see that in the second directory there is a folder mapping but I can't reach any pre-populated data from the data-container.
--volumes-from will add "Volume" from a docker container to another.
In your case you created an image that contains your data, you didnt create a volume that contains your data.
Docker is currently supporting the following:
a host bind mount that is listed as a bind mount in both services
a named volume that is used by both services
volumes_from another service
for example if you have your data in /my/data on docker host you can use the following:
sudo docker run -d --name firstcontainer -v /my/data:/db/data <image name>
sudo docker run -d --name secondcontainer -v /my/data:/db/data <image name>
If you want to use named volumes follow these steps:
create the named volume
sudo docker volume create <volumename>
Mount it to a transitional container and copy your data in it with docker cp
sudo docker cp /path/to/your/file <containername>:/destination/path
Mount your named volume on multiple containers.
sudo docker run -d --name firstcontainer -v <volumename>:/db/data <image name>
sudo docker run -d --name secondcontainer -v <volumename>:/db/data <image name>
Related
What is the exact difference between the two flags used in docker volume commands -v and --volumes-from. It seems to me that they are doing the same work, consider the following scenario.
First lets create a volume named myvol using command:
$ docker volume create myvol
Now create and run a container named c1 that uses myvol and get into his bash:
$ docker run -it --name c1 -v myvol:/data nginx bash
Lets create a file test.txt in the mounted directory of the container as:
root#766f90ebcf37:/# touch /data/test.txt
root#766f90ebcf37:/# ls /data
test.txt
Using -volume flag:
Now create another container named c2 that also uses myvol:
$ docker run -it --name c2 -v myvol:/data nginx bash
As expected, the new generated container c2 also have access to all the files of myvol
root#393418742e2c:/# ls /data
test.txt
Now doing the same thing with --volumes-from
Creating a container named c3 using volumes from container c1
$ docker run -it --name c3 --volumes-from c1 nginx bash
This will result the same thing in c3:
root#27eacbe25f92:/# ls /data
test.txt
The point is if -v and --volumes-from are working the same way i.e. to share data between containers then why they are different flags and what --volumes-from can do that -v cannot do?
The point is if -v and --volumes-from are working the same way i.e. to share data between containers
-v and --volumes-from are not working the same way, but with both of them you can share data between containers.
what --volumes-from can do that -v cannot do?
E.g. it can connect to another containers volumes without knowing how volumes are named and you do not specify path. You are able to add suffixes to containers ID's, with permissions like :ro or :rw.
More details here - Mount volumes from container (--volumes-from) section
The other question is what -v can do that --volumes-from cannot do?
E.g. it can mount named volumes, you can mount host directories or tmpfs (this one you cannot share between containers). In --volume-from you cannot share data with host directly.
The conclusion is: the purpose of --volume is to share data with host. (here you can find more use cases).
The purpose of --volumes-from is to share the data between containers.
They both works nice together.
I want to know how I can allow a "child" (sibling) docker container to access some subdirectory of an already mounted volume. As an explanation, this is a simple setup:
I have the following Dockerfile, which just installs Docker in a Docker container:
FROM ubuntu
RUN apt-get update && apt-get install -y curl
RUN curl -fsSL https://get.docker.com/ | sh
I have the following data directory on my host machine
/home/user/data/
data1.txt
subdir/
data2.txt
Build the parent image:
[host]$> docker build -t parent .
Then run the parent container:
[host]$> docker run --rm --name parent -it -v /home/user/data/:/data/ -v /var/run/docker.sock:/var/run/docker.sock parent
Now I have a running container, and am "inside" the new container. Since I have the docker socket bound to the parent, I am able to run docker commands to create "child" containers, which are actually sibling containers. The data volume has been successfully mapped:
[parent]$> ls /data/
subdir data1.txt
Now I want to create a sibling container that can only see the subdir directory:
[parent]$> docker run --rm --name child -it -v /data/subdir/:/data/ ubuntu
This creates a sibling container, and I am successfully "inside" the container, however the new data directory is empty. My assumption is because the volume I tell it to use "/data/" is mapped by the host to a directory that doesn't exist on the host, rather than using the volume defined when running the parent.
[child]$> ls /data/
<nothing>
What can I do to allow this mapping to work, so that the child can create files in the subdirectory, and that the parent container can see and access these files? The child is not allowed to see data1.txt (or anything else above the subdirectory).
"Sibling" container is the correct term, there is no direct relationship between what you have labeled the "parent" and "child" containers, even though you ran the docker command in one of the containers.
The container with the docker socket mounted still controls the dockerd running on the host, so any paths sent to dockerd via the API will be in the hosts scope.
There are docker commands where using the containers filesystem does change things. This is when the docker utility itself is accessing the local file system. docker build docker cp docker import docker export are examples where docker interacts with the local file system.
Mount
Use -v /home/user/data/subdir:/data for the second container
docker run --name parent_volume \
-it --rm -v /home/user/data:/data ubuntu
docker run --name child_volume \
-it --rm -v /home/user/data/subdir:/data ubuntu
The processes you run need to be careful with what is writing to data mounted into multiple containers so data doesn't get clobbered.
I have Docker installed in top of a CentOS system.
I tried to use volume but each new container is deleting (or hidding) the content of the folder to share.
My volume is always empty after a Docker run.
In order to create my containers, I use
docker run -dit --name $CONTAINER_NAME -p $PORT:8080 \
-v $VOLUME_PATH:/opt/conf/ \
$IMAGE_NAME
I aim at sharing a folder from the host between few Docker containers (to READ) AND I want also to write into this folder from the host.
What is an elegant way to do that ?
One solution is to use Data Volume Containers.
First, create a data volume container
docker run -d --name <data-volume-name> -v /<data-volume-name> ubuntu
You can add any data you want in this container.
Create your containers that will share by using the option volume-from
Let's create container foo and container bar using the shared datacontainer :
docker run -it --name foo --volumes-from=<data-volume-name> ubuntu
docker run -it --name bar --volumes-from=<data-volume-name> centos
Enjoy yourself
Each container in my example is mapped to the root folder.
From either bar or foo you can see /, in the filesytem.
You can also use volume field.
Create a volume
docker volume create --name <volume-name>
Create containers foo and bar that witl be mapped to the volume
docker run -dit --name foo -v test-volume:/path/in/container/ <image-name>
docker run -dit --name bar -v test-volume:/path/in/container/ <image-name>
Each container that will write in the volume will be visible by other.
I have made a data container in Docker with the directory /tmp:
sudo docker create -v /tmp --name datacontainer ubuntu
I will add another directory to this existing data container like /opt.
How can i do this?
You cannot add a new data volume to an existing (created or running) container.
With docker 1.9+, you would use instead docker volume create:
docker volume create --name my-tmp
docker volume create --name my-opt
Then you can mount those volumes to any container you want (when you run those containers, not when they are already running)
docker run -d -P \
-v my-tmp:/tmp \
-v my-opt:/opt \
--name mycontainer myimage
I am now learning docker and according to the documentation a shared data volume shall solely be destroyed when the last container holding a link to the shared volume is removed with the -v flag. Nevertheless, in my initial tests this is not the behaviour that I saw.
From the documentation:
Managing Data in Containers
If you remove containers that mount volumes, including the initial dbdata container, or the subsequent containers db1 and db2, the volumes will not be deleted. To delete the volume from disk, you must explicitly call docker rm -v against the last container with a reference to the volume. This allows you to upgrade, or effectively migrate data volumes between containers.
I did the following:
docker run -d -v /dbdata --name dbdata ubuntu:14.04 echo Data-only container for postgres
docker run -d --volumes-from dbdata --name db1 ubuntu:14.04 /bin/bash
Created some files on the /dbdata directory
Exited the db1 container
docker run -d --volumes-from dbdata --name db2 ubuntu:14.04 /bin/bash
I could access the files created on item 3 and create some new files
Exited the db2 container
docker run -d --volumes-from dbdata --name db3 ubuntu:14.04 /bin/bash
I could access the files created on item 3 and 6 and create some new files
Exited the db3 container
Removed all containers without the -v flag
Created the db container again, but the data was not there.
As stated in the user manual:
This allows you to upgrade, or effectively migrate data volumes between containers.
I wonder what I am doing wrong.
You are doing nothing wrong. In step 12, you are creating a new container with the same name. It has a different volume, which initially is empty.
Maybe the following example can illustrate what is happening (ids and paths will/may vary on your system or in other docker versions):
$ docker run -d -v /dbdata --name dbdata ubuntu:14.04 echo Data-only container for postgres
7c23cc1e6637e29f36c6cdd4c1461f6e1742b201e05227279ac3db55328da674
Run a container that has a volume /dbdata and give it the name dbdata. The Id is returned (your Id will be different).
Now lets inspect the container and print the "Volumes" information:
$ docker inspect --format "{{ .Volumes }}" dbdata
map[/dbdata:/var/lib/docker/vfs/dir/248641a5f51a80b5004f72f622a7329835e93881e9915a01b3c7112189d0b55e]
We can see that your /dbdata volume is located at /var/lib/docker/vfs/dir/248641...
Let's create some new data inside the container's volume:
$ docker run --rm --volumes-from dbdata ubuntu:14.04 /bin/bash -c "echo fuu >> /dbdata/test"
And check if it is available
$ docker run --rm --volumes-from dbdata -it ubuntu:14.04 cat /dbdata/test
fuu
Afterwards you delete the containers, without the -v flag.
$ docker rm dbdata
The dbdata container (with id 7c23cc1e6637) is gone, however is still present on your filesystem, as you can see if you inspect the folder:
$ cat /var/lib/docker/vfs/dir/248641a5f51a80b5004f72f622a7329835e93881e9915a01b3c7112189d0b55e/test
fuu
(Please note: if you use the -v flag and delete the container with docker rm -v dbdata the files of the volume on your host filesystem will be deleted and the above cat command would result in a No such file or directory message or similar)
Finally, in step 12. you start a new container with a different volume and give it the same name: dbdata.
docker run -d -v /dbdata --name dbdata ubuntu:14.04 echo Data-only container for postgres
2500731848fd6f2093243da3be064db79e76d731904e6f5349c3f00f054e5f8c
Inspection yields a different volume, which is initially empty.
docker inspect --format "{{ .Volumes }}" dbdata
map[/dbdata:/var/lib/docker/vfs/dir/faffba00358060024026412203a1562125f73d2bdd69a2202483e858dda04740]
If you want to re-use the volume, you have to create a new container and import/restore the data from the filesystem into the data container. In your case, you should not delete the data container in the first place, as you want to reuse the volume from it.