According to the documentation here:
https://docs.docker.com/storage/volumes/
If you start a container which creates a new volume, as above, and the container has files or directories in the directory to be mounted (such as /app/ above), the directory’s contents are copied into the volume. The container then mounts and uses the volume, and other containers which use the volume also have access to the pre-populated content.
In other words, the expectation is that, if i have files like so in /var/lib/mysql
root#d8fa9a8b305a:/var/lib/mysql# ls
auto.cnf xtz ib_logfile0 ibdata1 mysql sys
debian-5.7.flag ib_buffer_pool ib_logfile1 ibtmp1 performance_schema
Then, when I mount a volume into /var/lib/mysql then all the files from the container should be copied into my volume.
But I find this is not happening:
/var/lib/mysql/mysql # ls
auto.cnf ib_buffer_pool ib_logfile0 ib_logfile1 ibdata1
This is content of the volume that I mounted into /var/lib/mysql, and as you can see the data is not the same as the one present in /var/lib/mysql of the docker image itself. So, as a result, there's a failure on startup.
Note: The Volume in question is actually mounted by kubernetes. So, I'm making a major assumption here that
volumeMounts:
- name: xtz-persistent-storage
mountPath: "/var/lib/mysql/"
is the equivalent of doing this : docker run -p 443:443 --rm -v mysql:/var/lib/mysql <image>
Kubernetes Volumes is not the same thing as Docker Volumes.
From the Kubernetes documentation:
Docker also has a concept of volumes, though it is somewhat looser and less managed. In Docker, a volume is simply a directory on disk or in another container. Lifetimes are not managed and until very recently there were only local-disk-backed volumes. Docker now provides volume drivers, but the functionality is very limited for now (e.g. as of Docker 1.7 only one volume driver is allowed per container and there is no way to pass parameters to volumes).
A Kubernetes volume, on the other hand, has an explicit lifetime - the same as the pod that encloses it. Consequently, a volume outlives any containers that run within the Pod, and data is preserved across Container restarts. Of course, when a Pod ceases to exist, the volume will cease to exist, too. Perhaps more importantly than this, Kubernetes supports many types of volumes, and a Pod can use any number of them simultaneously.
At its core, a volume is just a directory, possibly with some data in it, which is accessible to the containers in a pod. How that directory comes to be, the medium that backs it, and the contents of it are determined by the particular volume type used.
So, although the name of the concept is the same, that is a different volume.
On the mount, Kubernetes overlays a destination directory.
Unfortunately, for now, there are no ways to merge the content in the mounted volume and in the container. Here is one of discussions about it.
Related
docker run -ti --rm -v DataVolume3:/var ubuntu
Lets say I have a volume DataVolume 3 which pulls the contents of /var in the ubuntu container
even after killing this ubuntu container the volume remains and I can use this volume DataVolume3 to mount it to other containers.
This means with the deletion of container the volume mounts are not deleted.
How does this work ?
Does that volume mount mean that it copies the contents of /var into some local directory because this does not look like a symbolic link ?
If I have the container running and I create a file in the container then the same file gets copied to the host path ?
How does this whole process of volume mount from container to host and host to container work ?
Volumes are used for persistent storage and the volumes persists independent of the lifecycle of the container.
We can go through a demo to understand it clearly.
First, let's create a container using the named volumes approach as:
docker run -ti --rm -v DataVolume3:/var ubuntu
This will create a docker volume named DataVolume3 and it can be viewed in the output of docker volume ls:
docker volume ls
DRIVER VOLUME NAME
local DataVolume3
Docker stores the information about these named volumes in the directory /var/lib/docker/volumes/ (*):
ls /var/lib/docker/volumes/
1617af4bce3a647a0b93ed980d64d97746878564b141f30b6110d0818bf32b76 DataVolume3
Next, let's write some data from the ubuntu container at the mounted path var:
echo "hello" > var/file1
root#2b67a89a0050:/# cat /var/file1
hello
We can see this data with cat even after deleting the container:
cat /var/lib/docker/volumes/DataVolume3/_data/file1
hello
Note: Although, we are able to access the volumes like shown above but it not a recommended practice to access volumes data like this.
Now, next time when another container uses the same volume then the data from the volume gets mounted at the container directory specified as part of -v flag.
(*) The location may vary based on OS as pointed by David and probably can be seen by the docker volume inspect command.
Docker has a concept of a named volume. By default the storage for this lives somewhere on your host system and you can't directly access it from outside Docker (*). A named volume has its own lifecycle, it can be independently docker volume rm'd, and if you start another container mounting the same volume, it will have the same persistent content.
The docker run -v option takes some unit of storage, either a named volume or a specific host directory, and mounts it (as in the mount(8) command) in a specific place in the container filesystem. This will hide what was originally in the image and replace it with the volume content.
As you note, if the thing you mount is an empty named volume, it will get populated from the image content at container initialization time. There are some really important caveats on this functionality:
Named volume initialization happens only if the volume is totally empty.
The contents of the named volume never automatically update.
If the volume isn't empty, the volume contents completely replace what's in the image, even if it's changed.
The initialization happens only on native Docker, and not for example in Kubernetes.
The initialization happens only on named volumes, and not for bind-mounted host directories.
With all of these caveats, I'd avoid relying on this functionality.
If you need to mount a volume into a container, assume it will be empty when your entrypoint or the main container command starts. If you need a particular directory layout or file structure there, an entrypoint script can create it; if you're expecting it to hold particular data, keep a copy of it somewhere else in your image and copy it in if it's not already there (or, perhaps, always).
(*) On native Linux you can find a filesystem location for it, but accessing this isn't a best practice. On other OSes this will be hidden inside a virtual machine or other opaque storage. If you need to directly access the data (or inject config files, or read log files) a docker run -v /host/path:/container/path bind mount is a better choice.
Volumes are part of neither the container nor the host. Well, technically everything resides in the host machine. But the docker directories are only accessible by users in "docker" group. The files in these directories are separately managed by docker.
"Volumes are stored in a part of the host filesystem which is managed by Docker (/var/lib/docker/volumes/ on Linux)."
Hence volumes are like the union of files under the docker container and the host itself. Any addition on either end will be added to the volume(/var/lib/docker/volumes), not hard copy, rather something like symbol link
As volumes can be shared across different containers, deleting a container does not cascade to the volumes associated with it.
To remove unused volumes:
docker volume prune .
In docker, where is the container data, apart from the mounted volume, is stored till the container is running.
So lets say /usr/local is volume mounted, so it would be shared between the host and the container. Where is everything else stored?
You should inspect your docker container
docker inspect [ID_CONTAINER]
and check for the fields MergedDir, LowerDir and UpperDir. Docker uses OverlayFS file system to store the data.
OverlayFS layers two directories on a single Linux host and presents them as a single directory. These directories are called layers and the unification process is referred to a a union mount. OverlayFS refers to the lower directory as lowerdir and the upper directory a upperdir. The unified view is exposed through its own directory called merged.
Check the doc here.
You can run the following command to get container written data
ll `podman inspect ContainerName --format "{{ .GraphDriver.Data.MergedDir }}"`
I want to know why we have two different options to do the same thing, What are the differences between the two.
We basically have 3 types of volumes or mounts for persistent data:
Bind mounts
Named volumes
Volumes in dockerfiles
Bind mounts are basically just binding a certain directory or file from the host inside the container (docker run -v /hostdir:/containerdir IMAGE_NAME)
Named volumes are volumes which you create manually with docker volume create VOLUME_NAME. They are created in /var/lib/docker/volumes and can be referenced to by only their name. Let's say you create a volume called "mysql_data", you can just reference to it like this docker run -v mysql_data:/containerdir IMAGE_NAME.
And then there's volumes in dockerfiles, which are created by the VOLUME instruction. These volumes are also created under /var/lib/docker/volumes but don't have a certain name. Their "name" is just some kind of hash. The volume gets created when running the container and are handy to save persistent data, whether you start the container with -v or not. The developer gets to say where the important data is and what should be persistent.
What should I use?
What you want to use comes mostly down to either preference or your management. If you want to keep everything in the "docker area" (/var/lib/docker) you can use volumes. If you want to keep your own directory-structure, you can use binds.
Docker recommends the use of volumes over the use of binds, as volumes are created and managed by docker and binds have a lot more potential of failure (also due to layer 8 problems).
If you use binds and want to transfer your containers/applications on another host, you have to rebuild your directory-structure, where as volumes are more uniform on every host.
Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. While bind mounts are dependent on the directory structure of the host machine, volumes are completely managed by Docker. Volumes are often a better choice than persisting data in a container’s writable layer, because a volume does not increase the size of the containers using it, and the volume’s contents exist outside the lifecycle of a given container. More on
Differences between -v and --mount behavior
Because the -v and --volume flags have been a part of Docker for a long time, their behavior cannot be changed. This means that there is one behavior that is different between -v and --mount.
If you use -v or --volume to bind-mount a file or directory that does not yet exist on the Docker host, -v creates the endpoint for you. It is always created as a directory.
If you use --mount to bind-mount a file or directory that does not yet exist on the Docker host, Docker does not automatically create it for you, but generates an error. More on
Docker for Windows shared folders limitation
Docker for Windows does make much of the VM transparent to the Windows host, but it is still a virtual machine. For instance, when using –v with a mongo container, MongoDB needs something else supported by the file system. There is also this issue about volume mounts being extremely slow.
More on
Bind mounts are like a superset of Volumes (named or unnamed).
Bind mounts are created by binding an existing folder in the host system (host system is native linux machine or vm (in windows or mac)) to a path in the container.
Volume command results in a new folder, created in the host system under /var/lib/docker
Volumes are recommended because they are managed by docker engine (prune, rm, etc).
A good use case for bind mount is linking development folders to a path in the container. Any change in host folder will be reflected in the container.
Another use case for bind mount is keeping the application log which is not crucial like a database.
Command syntax is almost the same for both cases:
bind mount:
note that the host path should start with '/'. Use $(pwd) for convenience.
docker container run -v /host-path:/container-path image-name
unnamed volume:
creates a folder in the host with an arbitrary name
docker container run -v /container-path image-name
named volume:
should not start with '/' as this is reserved for bind mount.
'volume-name' is not a full path here. the command will cause a folder to be created with path "/var/lib/docker/volumes/volume-name" in the host.
docker container run -v volume-name:/container-path image-name
A named volume can also be created beforehand a container is run (docker volume create). But this is almost never needed.
As a developer, we always need to do comparison among the options provided by tools or technology. For Volume & Bind mounts, I would suggest to list down what kind of application you are trying to containerize.
Following are the parameters that I would consider before choosing Volume over Bind Mounts:
Docker provide various CLI commands to Volumes easily outside containers.
For backup & restore, Volume is far easier than Bind as it depends upon the underlying host OS.
Volumes are platform-agnostic so they can work on Linux as well as on Window containers.
With Bind, you have 2 technologies to take care of. Your host machine directory structure as well as Docker.
Migration of Volumes are easier not only on local machines but on cloud machines as well.
Volumes can be easily shared among multiple containers.
We can have a data volume in docker:
$ docker run -v /path/to/data/in/container --name test_container debian
$ docker inspect test_container
...
Mounts": [
{
"Name": "fac362...80535",
"Source": "/var/lib/docker/volumes/fac362...80535/_data",
"Destination": "/path/to/data/in/container",
"Driver": "local",
"Mode": "",
"RW": true
}
]
...
But if the data volume lives in /var/lib/docker/volumes/fac362...80535/_data, is it any different from having the data in a folder mounted using -v /path/to/data/in/container:/home/user/a_good_place_to_have_data?
Although using volumes and bind mounts feels the same (with the only change being the location of the directory), there are differences in behavior.
Volumes vs Bind Mounts
With Bind Mount, a file or directory on the host machine is mounted into a container. The file or directory is referenced by its full or relative path on the host machine.
With Volume, a new directory is created within Docker's storage directory on the host machine, and Docker manages that directory's content.
Volumes advantages over bind mounts:
Volumes are easier to back up or migrate than bind mounts.
You can manage volumes using Docker CLI commands or the Docker API.
Volumes work on both Linux and Windows containers.
Volumes can be more safely shared among multiple containers.
Volume drivers allow you to store volumes on remote hosts or cloud providers, to encrypt the contents of volumes, or to add other functionality.
A new volume’s contents can be pre-populated by a container.
EDIT (9.9.2019):
According to #Sebi2020 comment, Bind mounts are much easier to backup. Docker doesn't provide any command to backup volumes. You have to use temporary containers with a bind mount to create backups.
Volumes
Created and managed by Docker. You can create a volume explicitly
using the docker volume create command, or Docker can create a volume
during container or service creation.
When you create a volume, it is stored within a directory on the
Docker host. When you mount the volume into a container, this
directory is what is mounted into the container. This is similar to
the way that bind mounts work, except that volumes are managed by
Docker and are isolated from the core functionality of the host
machine.
A given volume can be mounted into multiple containers simultaneously.
When no running container is using a volume, the volume is still
available to Docker and is not removed automatically. You can remove
unused volumes using docker volume prune.
When you mount a volume, it may be named or anonymous. Anonymous
volumes are not given an explicit name when they are first mounted
into a container, so Docker gives them a random name that is
guaranteed to be unique within a given Docker host. Besides the name,
named and anonymous volumes behave in the same ways.
Volumes also support the use of volume drivers, which allow you to
store your data on remote hosts or cloud providers, among other
possibilities.
Bind mounts
Available since the early days of Docker. Bind mounts have limited
functionality compared to volumes. When you use a bind mount, a file
or directory on the host machine is mounted into a container. The file
or directory is referenced by its full path on the host machine. The
file or directory does not need to exist on the Docker host already.
It is created on demand if it does not yet exist. Bind mounts are very
performant, but they rely on the host machine’s filesystem having a
specific directory structure available. If you are developing new
Docker applications, consider using named volumes instead. You can’t
use Docker CLI commands to directly manage bind mounts.
There is also tmpfs mounts.
tmpfs mounts
A tmpfs mount is not persisted on disk, either on the Docker host or
within a container. It can be used by a container during the lifetime
of the container, to store non-persistent state or sensitive
information. For instance, internally, swarm services use tmpfs mounts
to mount secrets into a service’s containers.
Reference:
https://docs.docker.com/storage/
is it any different from having the data in a folder mounted using -v /path/to/data/in/container:/home/user/a_good_place_to_have_data?
It is because, as mentioned in "Mount a host directory as a data volume"
The host directory is, by its nature, host-dependent. For this reason, you can’t mount a host directory from Dockerfile because built images should be portable. A host directory wouldn’t be available on all potential hosts.
If you have some persistent data that you want to share between containers, or want to use from non-persistent containers, it’s best to create a named Data Volume Container, and then to mount the data from it.
You can combine both approaches:
docker run --volumes-from dbdata -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /dbdata
Here we’ve launched a new container and mounted the volume from the dbdata container.
We’ve then mounted a local host directory as /backup.
Finally, we’ve passed a command that uses tar to backup the contents of the dbdata volume to a backup.tar file inside our /backup directory. When the command completes and the container stops we’ll be left with a backup of our dbdata volume.
Yes, this is quite different from a few perspectives. Like you wrote in the question's title, it is about understanding why we need data volumes vs bind mount to host.
Part 1 - Basic scenarios with examples
Lets take 2 scenarios.
Case 1: Web server
We want to provide our web server a configuration file that might change frequently. For example: exposing ports according to the current environment.
We can rebuild the image each time with the relevant setup or create 2 different images for each environment. Both of this solutions aren’t very efficient.
With Bind mounts Docker mounts the given source directory into a location inside the container.
(The original directory / file in the read-only layer inside the union file system will simply be overridden).
For example - binding a dynamic port to nginx:
version: "3.7"
services:
web:
image: nginx:alpine
volumes:
- type: bind #<-----Notice the type
source: ./mysite.template
target: /etc/nginx/conf.d/mysite.template
ports:
- "9090:8080"
environment:
- PORT=8080
command: /bin/sh -c "envsubst < /etc/nginx/conf.d/mysite.template >
/etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
(*) Notice that this example could also be solved using Volumes.
Case 2 : Databases
Docker containers do not store persistent data -- any data that will be written to the writable layer in container’s union file system will be lost once the container stops running.
But what if we have a database running on a container, and the container stops - that means that all the data will be lost?
Volumes to the rescue.
Those are named file system trees which are managed for us by Docker.
For example - persisting Postgres SQL data:
services:
db:
image: postgres:latest
volumes:
- "dbdata:/var/lib/postgresql/data"
volumes:
- type: volume #<-----Notice the type
source: dbdata
target: /var/lib/postgresql/data
volumes:
dbdata:
Notice that in this case, for named volumes, the source is the name of the volume
(for anonymous volumes, this field is omitted).
Part 2 - Comparison
Differences in management and isolation on the host
Bind mounts exist on the host file system and being managed by the host maintainer. Applications / processes outside of Docker can also modify it.
Volumes can also be implemented on the host, but Docker will manage them for us and they can not be accessed outside of Docker.
Volumes are a much wider solution
Although both solutions help us to separate the data lifecycle from containers,
by using Volumes you gain much more power and flexibility over your system.
With Volumes we can design our data effectively and decouple it from other parts of the system by storing it in dedicated remote locations (e.g., in the cloud) and integrate it with external services like backups, monitoring, encryption and hardware management.
The difference between host directory and a data volume is in that that Docker manages the latter by placing it into the $DOCKER-DATA-DIR/volumes directory and attaching a reference to it (names or randomly generated ids). That is you get a little bit of convenience.
Both host directories and data volumes are directories on the host. Both are host dependent. You can't reference either of them in a Dockerfile; the VOLUME directive creates a new nameless (with randomly generated id) volume every time you launch a new container and cannot reference an existing volume.
* $DOCKER-DATA-DIR is /var/lib/docker here unless you changed the defaults.
I want to list all container directories that are mounted volumes.
I.e. to be able to get similar info I get from
docker inspect --format "{{ .Volumes }}" <self>
But from within the container and without having docker installed in there.
I tried cat /proc/mounts, but I couldn't find a proper filter for it.
(EDIT - this may no longer work on Mac) If your Docker host is OS X, the mounted volumes will be type osxfs (or fuse.osxfs). You can run a
mount | grep osxfs | awk '{print $3}'
and get a list of all the mounted volumes.
If your Docker host is Linux (at least Ubuntu 14+, maybe others), the volumes appear to all be on /dev, but not on a device that is in your container's /dev filesystem. The volumes will be alongside /etc/resolv.conf, /etc/hostname, and /etc/hosts. If you do a mount | grep ^/dev to start, then filter out any of the files in ls /dev/*, then filter out the three files listed above, you should be left with host volumes.
mount | grep ^/dev/ | grep -v /etc | awk '{print $3}'
My guess is the specifics may vary from Linux to Linux. Not ideal, but at least possible to figure out.
Assuming you want to check what volumes are mounted from inside a linux based container you can look up entries beginning with "/dev" in /etc/mtab, removing the /etc entries
$ grep "^/dev" /etc/mtab | grep -v " \/etc/"
/dev/nvme0n1p1 /var/www/site1 ext4 rw,relatime,discard,data=ordered 0 0
/dev/nvme0n1p1 /var/www/site2 ext4 rw,relatime,discard,data=ordered 0 0
As you can read from many of the comments you had, a container is initially nothing but a restricted, reserved part of resources that is totally cut away from the rest of your machine. It is not aware of being a Docker, and inside the container everything behaves as if it were a separate machine. Sort of like the matrix, I guess ;)
You get access to the host machine's kernel and its resources, but yet again restricted as just a filtered out set. This is done with the awesome "cgroups" functionality that comes with Unix/Linux kernels.
Now the good news: There are multiple ways for you to provide the information to your Docker, but that is something that you are going to have to provide and build yourself.
The easiest ad most powerful way is to mount the Unix socket located on your host at /var/run/docker.sock to the inside of your container at the same location. That way, when you use the Docker client inside your container you are directly talking to the docker engine on your host.
However, with great power comes great responsibility. This is a nice setup, but it is not very secure. Once someone manages to get into your docker it has root access to your host system this way.
A better way would be to provide a list of mounts through the environment settings, or clinging on to some made-up conventions to be able to predict the mounts.
(Do you realize that there is a parameter for mounting, to give mounts an alias for inside your Docker?)
The docker exec command is probably what you are looking for.
This will let you run arbitrary commands inside an existing container.
For example:
docker exec -it <mycontainer> bash
Of course, whatever command you are running must exist in the container filesystem.
#docker cp >>>> Copy files/folders between a container and the local filesystem
docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH
docker cp [OPTIONS] SRC_PATH CONTAINER:DEST_PATH
to copy full folder:
docker cp ./src/build b081dbbb679b:/usr/share/nginx/html
Note – This will copy build directory in container’s …/nginx/html/ directory to copy only files present in folder:
docker cp ./src/build/ b081dbbb679b:/usr/share/nginx/html
Note – This will copy contents of build directory in container’s …./nginx/html/ directory
Docker Storage options:
Volumes are stored in a part of the host filesystem which is managed by Docker(/var/lib/docker/volumes/ on Linux). Non-Docker processes should not modify this part of the filesystem. Volumes are the best way to persist data in Docker.
When you create a volume, it is stored within a directory on the Docker host. When you mount the volume into a container, this directory is what is mounted into the container. This is similar to the way that bind mounts work, except that volumes are managed by Docker and are isolated from the core functionality of the host machine.
A given volume can be mounted into multiple containers simultaneously. When no running container is using a volume, the volume is still available to Docker and is not removed automatically. You can remove unused volumes using docker volume prune.
When you mount a volume, it may be named or anonymous. Anonymous volumes are not given an explicit name when they are first mounted into a container, so Docker gives them a random name that is guaranteed to be unique within a given Docker host. Besides the name, named and anonymous volumes behave in the same ways.
Volumes also support the use of volume drivers, which allow you to store your data on remote hosts or cloud providers, among other possibilities.
Bind mounts may be stored anywhere on the host system. They may even be important system files or directories. Non-Docker processes on the Docker host or a Docker container can modify them at any time.
Available since the early days of Docker. Bind mounts have limited functionality compared to volumes. When you use a bind mount, a file or directory on the host machine is mounted into a container. The file or directory is referenced by its full path on the host machine. The file or directory does not need to exist on the Docker host already. It is created on demand if it does not yet exist. Bind mounts are very performant, but they rely on the host machine’s filesystem having a specific directory structure available. If you are developing new Docker applications, consider using named volumes instead. You can’t use Docker CLI commands to directly manage bind mounts.
One side effect of using bind mounts, for better or for worse, is that you can change the host filesystem via processes running in a container, including creating, modifying, or deleting important system files or directories. This is a powerful ability which can have security implications, including impacting non-Docker processes on the host system.
tmpfs mounts are stored in the host system’s memory only, and are never written to the host system’s filesystem.
A tmpfs mount is not persisted on disk, either on the Docker host or within a container. It can be used by a container during the lifetime of the container, to store non-persistent state or sensitive information. For instance, internally, swarm services use tmpfs mounts to mount secrets into a service’s containers.
If you need to specify volume driver options, you must use --mount.
-v or --volume: Consists of three fields, separated by colon characters (:). The fields must be in the correct order, and the meaning of each field is not immediately obvious.
o In the case of named volumes, the first field is the name of the volume, and is unique on a given host machine. For anonymous volumes, the first field is omitted.
o The second field is the path where the file or directory will be mounted in the container.
o The third field is optional, and is a comma-separated list of options, such as ro. These options are discussed below.
• --mount: Consists of multiple key-value pairs, separated by commas and each consisting of a = tuple. The --mount syntax is more verbose than -v or --volume, but the order of the keys is not significant, and the value of the flag is easier to understand.
o The type of the mount, which can be bind, volume, or tmpfs. This topic discusses volumes, so the type will always be volume.
o The source of the mount. For named volumes, this is the name of the volume. For anonymous volumes, this field is omitted. May be specified as source or src.
o The destination takes as its value the path where the file or directory will be mounted in the container. May be specified as destination, dst, or target.
o The readonly option, if present, causes the bind mount to be mounted into the container as read-only.
o The volume-opt option, which can be specified more than once, takes a key-value pair consisting of the option name and its value.