Is it possible to use VDO on Kubernetes(with Docker containers)?
As far as I know, block devices are mountable - the problem here(I think) would be loading the VDO modules into the Docker container. I assume it's not possible to do it within the Docker container, so the responsibility lies on the host.
Correct, it's not directly supported by Kubernetes but you can always manage your VDO modules and volumes at the host level. For example, mount the volumes under /mnt/vdo0 and then use them in a container with the HostPath volume option.
You can also, for example, specify a VDO volume as the main graph directory for your docker daemon with something like /usr/bin/dockerd -g /mnt/vdo0 That will make your images and your non-external volume container storage stored in that directory.
Related
docker run -ti --rm -v DataVolume3:/var ubuntu
Lets say I have a volume DataVolume 3 which pulls the contents of /var in the ubuntu container
even after killing this ubuntu container the volume remains and I can use this volume DataVolume3 to mount it to other containers.
This means with the deletion of container the volume mounts are not deleted.
How does this work ?
Does that volume mount mean that it copies the contents of /var into some local directory because this does not look like a symbolic link ?
If I have the container running and I create a file in the container then the same file gets copied to the host path ?
How does this whole process of volume mount from container to host and host to container work ?
Volumes are used for persistent storage and the volumes persists independent of the lifecycle of the container.
We can go through a demo to understand it clearly.
First, let's create a container using the named volumes approach as:
docker run -ti --rm -v DataVolume3:/var ubuntu
This will create a docker volume named DataVolume3 and it can be viewed in the output of docker volume ls:
docker volume ls
DRIVER VOLUME NAME
local DataVolume3
Docker stores the information about these named volumes in the directory /var/lib/docker/volumes/ (*):
ls /var/lib/docker/volumes/
1617af4bce3a647a0b93ed980d64d97746878564b141f30b6110d0818bf32b76 DataVolume3
Next, let's write some data from the ubuntu container at the mounted path var:
echo "hello" > var/file1
root#2b67a89a0050:/# cat /var/file1
hello
We can see this data with cat even after deleting the container:
cat /var/lib/docker/volumes/DataVolume3/_data/file1
hello
Note: Although, we are able to access the volumes like shown above but it not a recommended practice to access volumes data like this.
Now, next time when another container uses the same volume then the data from the volume gets mounted at the container directory specified as part of -v flag.
(*) The location may vary based on OS as pointed by David and probably can be seen by the docker volume inspect command.
Docker has a concept of a named volume. By default the storage for this lives somewhere on your host system and you can't directly access it from outside Docker (*). A named volume has its own lifecycle, it can be independently docker volume rm'd, and if you start another container mounting the same volume, it will have the same persistent content.
The docker run -v option takes some unit of storage, either a named volume or a specific host directory, and mounts it (as in the mount(8) command) in a specific place in the container filesystem. This will hide what was originally in the image and replace it with the volume content.
As you note, if the thing you mount is an empty named volume, it will get populated from the image content at container initialization time. There are some really important caveats on this functionality:
Named volume initialization happens only if the volume is totally empty.
The contents of the named volume never automatically update.
If the volume isn't empty, the volume contents completely replace what's in the image, even if it's changed.
The initialization happens only on native Docker, and not for example in Kubernetes.
The initialization happens only on named volumes, and not for bind-mounted host directories.
With all of these caveats, I'd avoid relying on this functionality.
If you need to mount a volume into a container, assume it will be empty when your entrypoint or the main container command starts. If you need a particular directory layout or file structure there, an entrypoint script can create it; if you're expecting it to hold particular data, keep a copy of it somewhere else in your image and copy it in if it's not already there (or, perhaps, always).
(*) On native Linux you can find a filesystem location for it, but accessing this isn't a best practice. On other OSes this will be hidden inside a virtual machine or other opaque storage. If you need to directly access the data (or inject config files, or read log files) a docker run -v /host/path:/container/path bind mount is a better choice.
Volumes are part of neither the container nor the host. Well, technically everything resides in the host machine. But the docker directories are only accessible by users in "docker" group. The files in these directories are separately managed by docker.
"Volumes are stored in a part of the host filesystem which is managed by Docker (/var/lib/docker/volumes/ on Linux)."
Hence volumes are like the union of files under the docker container and the host itself. Any addition on either end will be added to the volume(/var/lib/docker/volumes), not hard copy, rather something like symbol link
As volumes can be shared across different containers, deleting a container does not cascade to the volumes associated with it.
To remove unused volumes:
docker volume prune .
We use containers to provision storage on our storage nodes but I can't for the life of my figure out a way to mount a device to the bare metal OS from a container. Both bare metal and containers are running oracle linux 7.5.
We cannot use ssh in any form for this. This is an isolated compute environment and the only access is thru the orchestration we use to manage containers.
I'm mainly a solaris guy so wondering if there is any linux magic I can work here.
I can mount any bare metal devices or filesystems into the container and I can run the container in privileged more.
Thx for any help
* clarification *
This is not about mounting a volume into a container.
This container is a temporary provisioning container, ie: it does stuff like mount iscisi volumes, create volume groups, create logical volumes and make filesystems.
This part is all working fine.
The last step this container needs to do is somehow tell the BARE METAL OPERATING SYSTEM TO MOUNT A DEVICE INTO IT'S FILESYSTEM. NOT IN THE CONTAINER.
Simplistic example: I need this container to somehow tell the OS to "mount /dev/sdg /data".
This mount does not need to be available to the container. The container is being destroyed once it allocate the storage and mounts it.
And we can't use SSH for this.
There are several problems you need to overcome.
By default, Docker does not have access to block devices on the
host.
A docker container is unable to modify its own mount namespace.
A docker container runs in a private mount namespace, so even after
solving (1) and (2), any mounts you make inside the container will
not be visible from the host.
Fortunately, there are solutions to all of the above!
We can solve (1) and (2) by passing the --privileged flag to
docker run. This removes all the restrictions that Docker normally
places on a container.
For solving (3), we need to use the --mount option instead of the
-v option, since we need to modify the style of mount propagation
used. Reading through the documentation on
bind-mounts,
we see that the --mount option supports the following options:
The type of the mount, which can be bind, volume, or tmpfs. This topic discusses bind mounts, so the type will always be bind.
The source of the mount. For bind mounts, this is the path to the file or directory on the Docker daemon host. May be specified as source or src.
The destination takes as its value the path where the file or directory will be mounted in the container. May be specified as destination, dst, or target.
The readonly option, if present, causes the bind mount to be mounted into the container as read-only.
The bind-propagation option, if present, changes the bind propagation. May be one of rprivate, private, rshared, shared, rslave, slave.
The consistency option, if present, may be one of consistent, delegated, or cached. This setting only applies to Docker for Mac, and is ignored on all other platforms.
The one we care about is the bind-propagation option. The values for
that are described later on in the same
document.
Reading through them, we probably want rshared.
Armed with this knowledge, I can run:
docker run -it \
--mount type=bind,source=/,dst=/host,bind-propagation=rshared \
--privileged alpine sh
Then inside the container I can run, for example:
mount /dev/sdd1 /host/mnt
And on the host I see the contents of /dev/sdd1 mounted on /mnt. The mount will persist after the container exits.
I want to know why we have two different options to do the same thing, What are the differences between the two.
We basically have 3 types of volumes or mounts for persistent data:
Bind mounts
Named volumes
Volumes in dockerfiles
Bind mounts are basically just binding a certain directory or file from the host inside the container (docker run -v /hostdir:/containerdir IMAGE_NAME)
Named volumes are volumes which you create manually with docker volume create VOLUME_NAME. They are created in /var/lib/docker/volumes and can be referenced to by only their name. Let's say you create a volume called "mysql_data", you can just reference to it like this docker run -v mysql_data:/containerdir IMAGE_NAME.
And then there's volumes in dockerfiles, which are created by the VOLUME instruction. These volumes are also created under /var/lib/docker/volumes but don't have a certain name. Their "name" is just some kind of hash. The volume gets created when running the container and are handy to save persistent data, whether you start the container with -v or not. The developer gets to say where the important data is and what should be persistent.
What should I use?
What you want to use comes mostly down to either preference or your management. If you want to keep everything in the "docker area" (/var/lib/docker) you can use volumes. If you want to keep your own directory-structure, you can use binds.
Docker recommends the use of volumes over the use of binds, as volumes are created and managed by docker and binds have a lot more potential of failure (also due to layer 8 problems).
If you use binds and want to transfer your containers/applications on another host, you have to rebuild your directory-structure, where as volumes are more uniform on every host.
Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. While bind mounts are dependent on the directory structure of the host machine, volumes are completely managed by Docker. Volumes are often a better choice than persisting data in a container’s writable layer, because a volume does not increase the size of the containers using it, and the volume’s contents exist outside the lifecycle of a given container. More on
Differences between -v and --mount behavior
Because the -v and --volume flags have been a part of Docker for a long time, their behavior cannot be changed. This means that there is one behavior that is different between -v and --mount.
If you use -v or --volume to bind-mount a file or directory that does not yet exist on the Docker host, -v creates the endpoint for you. It is always created as a directory.
If you use --mount to bind-mount a file or directory that does not yet exist on the Docker host, Docker does not automatically create it for you, but generates an error. More on
Docker for Windows shared folders limitation
Docker for Windows does make much of the VM transparent to the Windows host, but it is still a virtual machine. For instance, when using –v with a mongo container, MongoDB needs something else supported by the file system. There is also this issue about volume mounts being extremely slow.
More on
Bind mounts are like a superset of Volumes (named or unnamed).
Bind mounts are created by binding an existing folder in the host system (host system is native linux machine or vm (in windows or mac)) to a path in the container.
Volume command results in a new folder, created in the host system under /var/lib/docker
Volumes are recommended because they are managed by docker engine (prune, rm, etc).
A good use case for bind mount is linking development folders to a path in the container. Any change in host folder will be reflected in the container.
Another use case for bind mount is keeping the application log which is not crucial like a database.
Command syntax is almost the same for both cases:
bind mount:
note that the host path should start with '/'. Use $(pwd) for convenience.
docker container run -v /host-path:/container-path image-name
unnamed volume:
creates a folder in the host with an arbitrary name
docker container run -v /container-path image-name
named volume:
should not start with '/' as this is reserved for bind mount.
'volume-name' is not a full path here. the command will cause a folder to be created with path "/var/lib/docker/volumes/volume-name" in the host.
docker container run -v volume-name:/container-path image-name
A named volume can also be created beforehand a container is run (docker volume create). But this is almost never needed.
As a developer, we always need to do comparison among the options provided by tools or technology. For Volume & Bind mounts, I would suggest to list down what kind of application you are trying to containerize.
Following are the parameters that I would consider before choosing Volume over Bind Mounts:
Docker provide various CLI commands to Volumes easily outside containers.
For backup & restore, Volume is far easier than Bind as it depends upon the underlying host OS.
Volumes are platform-agnostic so they can work on Linux as well as on Window containers.
With Bind, you have 2 technologies to take care of. Your host machine directory structure as well as Docker.
Migration of Volumes are easier not only on local machines but on cloud machines as well.
Volumes can be easily shared among multiple containers.
I have a Docker container and I am trying to make it so that all of the files in /var/www/ on the container will be saved on the host system at a location (/home/me), and vise-versa. Is it possible to have this shared space between the two?
Would you accomplish this with mount points, or is there a better method?
Thanks
You can use volumes for sharing between container and host.
docker run -v /home/me:/var/www <image>
If you have a fixed files/data, you can add to the image using dockerfile or committing after copying into container. If you want to share rw dir between host and container, you need to use the volumes. Your data will also be persisted even if you remove and recreate a new container.
There are three ways that you can do this
Use volumes. Official docs
Burn the files in your image. Basically include the creation of the files inside the Dockerfile. This means every container container from that image will have an initial state of sorts.
Use data-only containers. These are containers without a running process that contain the data that you need. This also uses volumes. But instead of mounting to the host, your containers mount on the data-only container (which in turn mounts on the host if you want to). This answer will be useful
Normally we would run container by using the following command:
Docker run -it ubuntu /bin/bash
Is there any option to specify where to run the container (like on which disk or partition)?
Do you mean where the container data/layers will be stored?
The layers are all inside /var/lib/docker/(aufs)
It's possible for you to mount a different larger/faster partition into this folder, but this is for the entire docker platform. if you are careful, you can mount the partition for a particular docker container.
It would be better if you would use "docker run -v folder:mount point" flag, since you can mount specific host folders as external volumes inside the container.
Both these can help you spread data over different partitions/disks.
I am not aware of a container specific option.
However, you can bind-mount (or create a symlink) a particular disk or partition to '/var/lib/docker'. This will make all the container storage to be on that partition.
If you want the container storage to be on multiple partitions, LVM is an option.
You can setup a volume group that spans multiple partitions. You can then ask the Docker daemon to create a thinly provisioned logical volume in one of these volume groups to be used as storage.
The following link provides more information : https://access.redhat.com/documentation/en/red-hat-enterprise-linux-atomic-host/7/getting-started-with-containers/chapter-7-managing-storage-with-docker-formatted-containers
Also, using a union mount like OverlayFS could be another solution : https://askubuntu.com/questions/109413/how-do-i-use-overlayfs