I have a directory(maybe later volume), that I would like to share with all my interactive containers. I know, that native Docker volumes are stored under /var/lib/docker/volumes and docker run -v seems the easiest way, but I think Data Volume Container is a much more standardized way. I don't know, how to create this volume container from a directory or an existing another volume. Maybe is it wrong method?
There are two ways to create and share volumes: 1. using the VOLUME instruction on the Dockerfile. 2 Specifying the -v <volume_name> option during container runtime and later using --volumes-from=<container> with every subsequent container which need to share the data. Here is an ex with the later:
Start your first container with -v, then add a test file under the directory of the shared volume.
docker run -it -v /test-volume --name=testimage1 ubuntu:14.04 /bin/bash
root#ca30f0f99401:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys test-volume ===> test-volume dir got created here
root#ca30f0f99401:/# touch test-volume/1
root#ca30f0f99401:/# cat > test-volume/1
Test Message!
From the host OS, you can get details of the volume by inspecting your container:
docker inspect ca30f0f99401 | grep -i --color -E '^|Vol'
"Mounts":
{
"Name": "025835b8b47d282ec5f27c53b3165aee83ecdb626dc36b3b18b2e128595d9134",
"Source": "/var/lib/docker/volumes/025835b8b47d282ec5f27c53b3165aee83ecdb626dc36b3b18b2e128595d9134/_data",
"Destination": "/test-volume",
"Driver": "local",
"Mode": "",
"RW": true
"Image": "ubuntu:14.04",
"Volumes": {
"/test-volume": {} }
Start another container with a shared volume and check if the shared folder/files exists.
$ docker run -it --name=testimage2 --volumes-from=testimage1 ubuntu:14.04 /bin/bash
root#60ff1dcebc44:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys test-volume tmp usr var
root#60ff1dcebc44:/# cat test-volume/1
Test Message!
Goto step-3 to share volume with a new container.
Make a data volume container by writing a dedicated Dockerfile in which you would:
COPY your folder in it
declare that copied local container path folder as a VOLUME
Then docker create <imagename> and you get a (created) container, that you can mount in all your other containers, provided you run them with the --volumes-from <containername> option.
Related
I run the docker for Windows and ubuntu in WSL.
When I run the following command
docker volume create test
docker volume inspect test
I get the following output
[
{
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/test/_data",
"Name": "test",
"Options": {},
"Scope": "local"
}
]
when I access the location, I get
bash: cd: /var/lib/docker/volumes: No such file or directory
So how should I access the folder?
The directory is is protected so you can cd into it, however you can ls the contents:
sudo ls /wsl/docker-desktop-data/data/docker/volumes/test/_data
I've modified my WSL set up as per this article so you may find your path is different. I think the default path is probably /mnt/wsl/docker-desktop-data/data/docker/volumes/test/_data
You might find it more useful to mount a directory in your Windows user folder which can be done by changing the WSL mount point as per the article linked to above and then running:
docker volume create --driver local --name test --opt device=/run/desktop/mnt/host/c/Users/<username>/test --opt type=none --opt o=bind
(assuming you've got a folder called test at the root of your Windows user directory)
based on #nick's answer
From linux: sudo ls /mnt/wsl/docker-desktop-data/data/docker/volumes
From windows: \\wsl$\docker-desktop-data\mnt\wsl\docker-desktop-data\data\docker\volumes
For me the volumes data are at /mnt/wsl/docker-desktop-data/version-pack-data/community/docker/volumes, each distro may mount wsl at different location, this is the default path for Ubuntu at least.
I can cd into it with root permission.
I want to dynamically get symlinks to devices created by udev running on Host in a docker container
I was able to bind the symlink to the container but it's not dynamically recreated if the device is removed (e.g: usb is disconnected)
Udev rules example:
SUBSYSTEM=="tty", ATTRS{idVendor}=="12d1", ATTRS{idProduct}=="140c", MODE="0666", SYMLINK+="my_dir/gsm-modem0"
docker run example:
sudo docker run -v /dev/my_dir/gsm-modem0:/dev/my_dir/gsm-modem0 my_image my_script.sh
Answer:
Udev rule should symlink to a new directory:
SUBSYSTEM=="tty", ATTRS{idVendor}=="12d1", ATTRS{idProduct}=="140c", MODE="0666", SYMLINK+="my_dir/gsm-modem0"
Running docker must contain --privileged:
sudo docker run --privileged -v /dev/my_dir:/dev/my_dir my_image my_script.sh
and my_script.sh should start by creating a new file in the created symlink directory:
mkdir -p /dev/my_dir
touch /dev/my_dir/keep
Explanation:
For some reason udev may delete the link directory if the directory is empty, and since usually /dev is a tmpfs creating new file won't survive restart. Touching a file on every run will keep the link containing directory on host and if a new link is created it will appear on the container
I was running a docker container process with:
host$ docker run -it <image> /etc/bootstrap.sh -bash
Then inside of the container I created a file /a.txt:
container$ echo "abc" > /a.txt
container$ cat a.txt
abc
I noticed the filesystem type for / is none:
container$ df -h
Filesystem Size Used Avail Use% Mounted on
none 355G 19G 318G 6% /
tmpfs 1.9G 0 1.9G 0% /dev
...
The inspect command shows the volumes is null.
host$ docker inspect <image>
...
"Volumes": null,
...
After I exited the container process and restarted, the file disappeared. I wanted to understand:
1) what the root filesystem of the container process actually is;
2) how can I persist the newly created file?
Q: After I exited the container process and restarted, the file disappeared.
A: Data in a docker container is not persisted. That is you lost everything when that container gets restarted.
Q: What the root filesystem of the container process actually is?
A: Don't really understand this question but I assume you are asking about where is the root user's home directory? If it is, then root's home is at /root.
Q: How can I persist the newly created file?
A: If you are intending to keep the changes even after you restart the container then you will need to use docker's data volume.
See:
https://docs.docker.com/engine/tutorials/dockervolumes/
Essentially when you start the container, you can pass in the -v option to tell the container that you would like to map the directory from the host's file system to the container's directory.
That is by doing the example command below,
$ docker run -d -P --name web -v $(pwd):/root
You will you would like to map your current working directory to the container's /root directory. So everything gets written to the container's /root/ area gets reflected to your host's file system and it is persisted.
When creating volumes through the volume API, that is, as the container volume pattern is now not necessarily the best practice anymore:
# docker volume inspect test-data
[
{
"Name": "test-data",
"Driver": "local",
"Mountpoint": "/var/lib/docker/volumes/test-data/_data"
}
]
I would like to, for example, have docker volumes exist in /data (which is mounted in a different physical volume).
This is not possible to do with symbolic links, it is possible to do with bind mounts, but would I'm wondering if there is some configuration in Docker to change the default location for each separate volume.
You can change where Docker stores its files including volumes by changing one of its startup parameters called --data-root.
If you're using systemd for service management, the file is usually located at /lib/systemd/system/docker.service. Edit the file as such:
# Old - taken from the generated docker.service file in Ubuntu 16.04's docker.io package
ExecStart=/usr/bin/dockerd -H fd:// $DOCKER_OPTS
# New
ExecStart=/usr/bin/dockerd --data-root /new_location/ -H fd:// $DOCKER_OPTS
Alternatively, you can edit the Docker daemon configuration file which defaults to /etc/docker/daemon.json.
Restart the Docker daemon and your volumes will be under /new_location/volumes/{volume_name}/_data
Note: be careful in production and also locally! You also have to move the existing data from /var/lib/docker/ to the new location for your docker install to work as expected.
You can use symlinks from the new location if you want specific folders to be in specific place.
2017: with 17.05.0-ce (2017-05-04), the PR 28696 deprecates --graph flag in favor or --data-root: commit 1ecaed0
The name "graph" is a legacy term from long ago when there used to be a directory at the default location /var/lib/docker/graph.
However, the flag would indicate the path of the parent directory of the "graph" directory which contains not only image data but also data for volumes, containers, and networks.
In the most recent version of docker, this directory also contains swarm cluster state and node certificates.
With issue 5922 and PR 5978, the documentation has been updated.
Example:
ExecStart=/usr/bin/dockerd -H fd:// --data-root=/mnt/ssd/lib/docker
2016 (now deprecated)
I only know of a docker option to change /var/lib/docker itself, not its subfolders (part of its "graph" used by a docker daemon storage driver)
See docker daemon "Miscellaneous options":
Docker supports softlinks for the Docker data directory (/var/lib/docker) and for /var/lib/docker/tmp.
The DOCKER_TMPDIR and the data directory can be set like this:
DOCKER_TMPDIR=/mnt/disk2/tmp /usr/local/bin/docker daemon -D -g /var/lib/docker -H unix:// > /var/lib/docker-machine/docker.log 2>&1
# or
export DOCKER_TMPDIR=/mnt/disk2/tmp
/usr/local/bin/docker daemon -D -g /var/lib/docker -H unix:// > /var/lib/docker-machine/docker.log
As mentioned in "Where are docker images stored on the host machine?" (and that would apply also for containers/volumes):
The contents of the /var/lib/docker directory vary depending on the driver Docker is using for storage.
I successfully moved the storage location of docker by moving the content of /var/lib/docker to a new location and then place a symlink pointing to the new location (I took this solution from here https://askubuntu.com/questions/631450/change-data-directory-of-docker):
Caution - These steps depend on your current /var/lib/docker being an
actual directory (not a symlink to another location).
1) Stop docker: service docker stop. Verify no docker process is running:
ps aux | grep -i [d]ocker
2) Double check docker really isn't running. Take a look at the current docker directory:
ls /var/lib/docker/
2b) Make a backup - tar -zcC /var/lib docker >
/mnt/pd0/var_lib_docker-backup-$(date +%s).tar.gz
3) Move the /var/lib/docker directory to your new partition:
mv /var/lib/docker /mnt/pd0/docker
4) Make a symlink: ln -s /mnt/pd0/docker /var/lib/docker
5) Take a peek at the directory structure to make sure it looks like
it did before the mv: ls /var/lib/docker/ (note the trailing slash)
6) Start docker back up service docker start
7) restart your containers (resolve the symlink)
Worked for me on Ubuntu 18.04.1 LTS on an Azure VM with Docker 18.09.2
If you're on Fedora (tested on 32) just change or add the --data-root flag with your desired path to the OPTIONS variable on /etc/sysconfig/docker, this is the environment file used by systemd to start the dockerd service.
I'm new to docker, and I'm trying mount the root directory of docker container as a NFS mount point.
for example, I had a NFS mount point test:/home/user/3243, and I'm trying:
docker run -it -v "test:/home/user/3243":/ centos7 /bin/bash
absolutely, it's failed. So I tried this:
mount -t nfs test:/home/user/3243 /mnt/nfs/3243
docker run -it -v /mnt/nfs/3243:/ centos7 /bin/bash
but failed again, so how to do this? Could it be worked out?
A couple of issues here:
You cannot mount to the root directory of a container. So docker run -v /foo:/ will never work.
With the syntax of your first attempt, -v test:/foo:bar, Docker would see this as wanting to create a "named" volume called "test".
You should be able to first do the NFS mount, then do docker run -v /mnt/nfs/3243:/foo to have the nfs path mounted to /foo.
But again, you can't mount to /.
That is currently discussed (since mid 2014) in issue 4213.
One recent workaround by Jeroen van Bemmel (jbemmel) was:
It appears that NFS functionality depends on the underlying storage driver ( aufs, devicemapper, etc. ), as well as the sharing of file handles between processes ( see blog post "docker: devicemapper fix for “device or resource busy” (EBUSY)") i.e. 'unshare' may have an impact on NFS mounts.
I've moved away from using the 'MOUNTPOINT=/vm/nfs' as I am not sure if that event is even emitted.
Instead I created an upstart file like this:
cat > /etc/init/ecdn.conf << EOF
description "eCDN container"
author "Jeroen van Bemmel"
# mounted MOUNTPOINT=/vm/nfs doesn't seem to work, at least not the first time
start on started docker and virtual-filesystems
stop on starting rc RUNLEVEL=[016]
respawn
script
exec /usr/bin/docker start -a ecdn
end script
pre-stop script
/usr/bin/docker stop ecdn
# dont /usr/bin/docker rm ecdn
end script
EOF
and then create the container like this:
script -c "docker create -it --name='ecdn' --volume /vm:/usr/share/nginx/html/vm:ro image/name"