I'm new to docker. Been reading some but still don't get it much. Basically if I mount a volume it will be persistent on my hard drive so that my container service can write down and save datas. Seen a lot of container services that is mounted with volumes, like jenkins. But If I have mariadb, do I need to mount a volume to it? What happens if I stop and start the mariadb container?
If you just stop and start the data will not be lost.
All data will be deleted when you delete and recreate the container.
Answering your question. It is not strictly necessary to mount a volume for a persistent service (e.g: database). But it is a recommended practice, it will ease scenarios like changing the container (upgrading).
Related
I'm new to the area, sorry if my question sounds dumb.
What I'm looking for: I have a containers pod, where one of the containers (alpine based) should read/write from/to the customer's provided file. I don't want to limit customer on how to provide file (or at least to support most common ways).
And file's size might be huge sometimes (not sure if that requirement makes any difference).
The more flexibility here the better.
From the initial search I found there are multiple ways to bind the volume/directory to docker's container:
Docker bind mount - sharing dir between Host and container (nice to have)
Add a docker volume to the pod (must have)
Mount AWS S3 bucket to docker's container (must have)
any other ways of supplying file access to the container? Let's say from the remote machine via sftp access?
But main question - is it all possible to configure via Kubernetes?
Ideally in the same yaml file that starts the containers?
Any hints/examples are very welcome!
It surely is possible!
Like there are volume mount for a docker container, there are volume mounts in Kubernetes as well.
This is achieved using Persistent Volume Claim, PVC. These are Pod lifecycle independent storage classes to store the data in the volume mount.
Understand more about the concept here: https://kubernetes.io/docs/concepts/storage/persistent-volumes/
I have seen the terms "bind mount" and "host volume" being used in various articles but none of them mention whether they are the same thing or not. But looking at their function, it looks like they are pretty much the same thing. Can anyone answer whether it is the same thing or not? If not, what is the difference?
Ref:
Docker Docs - Use bind mounts
https://blog.logrocket.com/docker-volumes-vs-bind-mounts/
They are different concepts.
As mentioned in bind mounts:
Bind mounts have been around since the early days of Docker. Bind mounts have limited functionality compared to volumes. When you use a bind mount, a file or directory on the host machine is mounted into a container. The file or directory is referenced by its absolute path on the host machine. By contrast, when you use a volume, a new directory is created within Docker’s storage directory on the host machine, and Docker manages that directory’s contents.
And as mentioned in volumes:
Volumes are the preferred mechanism for persisting data generated by
and used by Docker containers. While bind mounts are dependent on the
directory structure and OS of the host machine, volumes are completely
managed by Docker. Volumes have several advantages over bind mounts:
Volumes are easier to back up or migrate than bind mounts.
You can manage volumes using Docker CLI commands or the Docker API.
Volumes work on both Linux and Windows containers.
Volumes can be more safely shared among multiple containers.
Volume drivers let you store volumes on remote hosts or cloud providers, to encrypt the contents of volumes, or to add other functionality.
New volumes can have their content pre-populated by a container.
Volumes on Docker Desktop have much higher performance than bind mounts from Mac and Windows hosts.
A "bind mount" is when you let your container see and use a normal directory in a normal filesystem on your host. Changes made by programs running in the container will be visible in your host's filesystem.
A "volume" is a single file on your host that acts like a whole filesystem visible to the container. You can't normally see what's inside it from the host.
I was able to figure it out.
There are 3 types of storage in Docker.
1. Bind mounts-also known as host volumes.
2. Anonymous volumes.
3. Named volumes.
So bind mount = host volume. They are the same thing. "Host volume" must be a deprecating term though, as I cannot see it in Docker docs. But it can be seen in various articles published 1-2 years ago.
Examples for where it is referred to as "host volume":
https://docs.drone.io/pipeline/docker/syntax/volumes/host/
https://spin.atomicobject.com/2019/07/11/docker-volumes-explained/
This docs page here Manage data in Docker is quite helpful
Volumes are stored in a part of the host filesystem which is managed by Docker (/var/lib/docker/volumes/ on Linux). Non-Docker processes should not modify this part of the filesystem. Volumes are the best way to persist data in Docker.
Bind mounts may be stored anywhere on the host system. They may even be important system files or directories. Non-Docker processes on the Docker host or a Docker container can modify them at any time.
On the host side, should all the mount points be located in the same location? Or should they reflect the locations which are inside the containers?
For example, what is the best place to mount /var/jenkins_home on the host side in order to be consistent with its Unix filesystem?
/var/jenkins_home
/srv/jenkins_home
/opt/docker-volumes/jenkins/var/jenkins_home
Other location ?
It absolutely depends on you where you want to mount the volume on the host. Just don't map it to any system file locations.
In my opinion the volumes reflecting the locations inside the container is not a great idea since you will have many containers, and all will have similar file system structure, so you will never be able to isolate container writes.
With jenkins, since the official Jenkins docker image runs with user "jenkins", it will be not a bad idea for you to create jenkins user on the host and map /home/jenkins on the host to /var/jenkins_home on the container.
Rather than using explicit host:container mounts, consider using named volumes. This has several benefits:
They can be shared easily into other containers
They are host-agnostic (if you don't have the specific mount on that machine, it will fail)
They can be managed as first-class citizens in the Docker world (docker volume)
You don't have to worry about where to put them on your host ;)
I know how to create and mount a data volume container to multiple other containers using --volumes-from, but I do have a few questions regarding it's usage and limitations:
Situation: I am looking to use a data volume container to store user uploaded images in for my web application. This data volume container will be used/mounted by many other containers running the web frontend.
Questions:
Can data volume containers be used/mounted in containers residing on other hosts within a docker swarm?
How is the performance? is it recommended to structure things this way?
Is there a better way to handle sharing of persistent resources across containers and hosts such as NFS?
Can data volume containers be used/mounted in containers residing on other hosts within a docker swarm?
Docker, by itself, does not provide any facility for either migrating data or sharing data between hosts in a cluster.
How is the performance? is it recommended to structure things this way?
Docker volumes impose very little performance overhead in general, because they're just bind mounts.
Is there a better way to handle sharing of persistent resources across containers and hosts such as NFS?
NFS or any cluster filesystem, such as gluster, can be used to create data volumes that are available to all hosts in a cluster.
There is a technology called as Flocker which will help you to move your containers across the hosts with the attached volume. Flocker is open source. Docker in itself doesn't provide any functionality for Docker container migration. There is a small article on container migration. Hope it will be helpful.
Is it possible to mount Cinder volumes on docker containers in openstack?
And if it is, is there a way to encrypt data leaving the container to the cinder volume?
I was thinking of mounting the volume as a loopback device and encrypt the data as it was being flushed to the disk. Ist this possible?
Kind regards
It is not currently possible to mount Cinder volumes inside a Docker container in OpenStack.
A fundamental problem is that Docker is filesystem-based, rather than block-device-based. Any block device -- like a Cinder volume -- would need to be formatted with a filesystem and mounted prior to starting the container. While it might be technically feasible, the necessary support for this does not yet exist.
The Manila project may be a better solution for adding storage to containers, but I haven't looked into that and I don't know if (a) the project works at all yet and (b) it it works with nova-docker.
If you're not using the nova-docker driver but are instead using the Heat plugin for Docker, you can mount host volumes in a container similar to docker run -v ..., but making this work seamlessly across multiple nodes in a multi-tenant setting may be difficult or impossible.