I want to share a volume across multiple containers , in docker swarm .
I need it to be such that all the containers have R/W access to this volume at any point of time .
If not rexray ,is there any other docker volume plugin which would enable me to do the same thing ?
The rexray documentation doesn't state the fact clearly.
REX-Ray has to use some backend storage driver. This is more about the storage, which has to support multiple read/write connections to the same volume. If you truly need multiple read/write, some options include:
REX-Ray with AWS EFS driver. EFS supports multiple NFS r/w connections.
https://portworx.com which will replicate data between nodes.
REX-Ray with any custom NFS storage.
Maybe a custom storage solution with drivers from Docker Store.
Related
I'm studying about Docker and I couldn't understand very well the difference between what storage drivers are used for and what volume drivers are used for.
My theory (please correct me if I'm wrong) is the storage drivers manage the way Docker deals underneath with the writable layer, can use overlay, overlay2, aufs, zfs, btrfs and so on.
Volume drivers, however, deal with volumes underneath, like it can be a local volume (in this case I think it will use a storage driver) or can be a remote one (like EBS).
Am I right?
Docker uses storage drivers to store image layers, and to store data in the writable layer of a container. Docker uses Volume drivers for write-intensive data, data that must persist beyond the container’s lifespan, and data that must be shared between containers. So, I understand Storage Drivers are used with image and container layers while Volume Drivers are used for persistent container application data. See the first three paragraphs of this Docker documentation: https://docs.docker.com/storage/storagedriver/
Docker engine volume enable engine deployments to be integrated with external storage systems such as Amazon EBS, and enable data volumes to persist beyond the lifetime of a single Docker host. Here the term 'local' in concept of Docker volume driver means the volumes esdata1 and esdata2 are created on the same Docker host where you run your container. By using other Volume plugins, e.g.,-driver=flocker. You are able to create a volume on a external host and mount it to the local host, say, /data-path.
Looking at multiple options to implement a shared storage for a Docker Swarm, I can see most of them require a special Docker plugin:
sshFs
CephFS
glusterFS
S3
and others
... but one thing that is not mentioned anywhere is just mounting a typical block storage to all VPS nodes running the Docker Swarm. Is this option impractical and thus not mentioned on the Internet? Am I missing something?
My idea is as follows:
Create a typical Block Storage (like e.g. one offered by DigitalOcean or Vultr).
Mount it to your VPS filesystem.
Mount a folder from that Block Storage as a volume in the Docker Container / Docker Worker with using a "local" driver.
Sounds the simplest and most obvious to me. Why people are using more complicated setups like sshFs, CephFS etc? And most importantly, is the implementation I described viable, and if so, what are the drawbacks of doing it this way?
The principal advantage of using a volume plugin over mounted storage comes down to the ability to create storage volumes dynamically, and associate them with namespaces.
i.e. with docker managing storage for a volume via a "volumes: data:" directive in a compose file, a volume will be created for each named stack that is deployed.
Using the local driver and mounts, you the swarm admin now need to ensure that no two stacks are trying to use /mnt/data.
Once you pass that hurdle, some platforms have limitations to the number of hosts a block storage can be mounted on to.
Theres also the security angle to consider - with the volume mapped like that a compromose to any service on any host can potentially expose all your data to an attacker, where a volume plugin will expose just exactly the data mounted on that container.
All that said - docker swarm is awesome and the current plugin space is lacking - if mounting block storage is what it takes to get a workable storage solution I say do it. Hopefully the CSI support will be ready before year end however.
I'm able to bind a docker volume to a specific container in a swarm thanks to flocker, but now i would have multiple replicas of my server (to do load balancing) and so i'm searching something to bind the same data volume to multiple replicas of a docker service.
In flocker documentaiton i have found that
Can more than one container access the same volume? Flocker works by
creating a 1 to 1 relationship of a volume and a container. This means
you can have multiple volumes for one container, and those volumes
will always follow that container.
Flocker attaches volumes to the individual agent host (docker host)
and this can only be one host at a time because Flocker attaches
Block-based storage. Nodes on different hosts cannot access the same
volume, because it can only be attached to one node at a time.
If multiple containers on the same host want to use the same volume,
they can, but be careful because multiple containers accessing the
same storage volume can cause corruption.
Can I attach a single volume to multiple hosts? Not currently, support
from multi-attach backends like GCE in Read Only mode, or NFS-like
backends like storage, or distributed filesystems like GlusterFS would
need to be integrated. Flocker focuses mainly on block-storage uses
cases that attach a volume to a single node at a time.
So i think is no possible to do what i want with flocker.
I could use a different orchestrator (k8s) if that could help me, even if i have no experience with that.
I would not use NAS/NFS or anything distribuited filesystems.
Any suggestions?
Thanks in advance.
In k8s, you can mount volume to different Pods at the same time if technology that backs the volume supports shared access.
As mentioned in Kubernetes Persistent Volumes:
Access Modes A PersistentVolume can be mounted on a host in any way
supported by the resource provider. As shown below, providers will
have different capabilities and each PV’s access modes are set to the
specific modes supported by that particular volume. For example, NFS
can support multiple read/write clients, but a specific NFS PV might
be exported on the server as read-only. Each PV gets its own set of
access modes describing that specific PV’s capabilities.
The access modes are:
ReadWriteOnce – the volume can be mounted as read-write by a single node
ReadOnlyMany – the volume can be mounted read-only by many nodes
ReadWriteMany – the volume can be mounted as read-write by many nodes
Types of volumes that supports ReadOnlyMany mode:
AzureFile
CephFS
FC
FlexVolume
GCEPersistentDisk
Glusterfs
iSCSI
Quobyte
NFS
RBD
ScaleIO
Types of volumes that supports ReadWriteMany mode:
AzureFile
CephFS
Glusterfs
Quobyte
RBD
PortworxVolume
VsphereVolume(works when pods are collocated)
I need to use Postgres database in Rancher stack (Cattle).
I made environment for my application, that contains api, frontend and database services (in one stack). I want to use multiple hosts for this environment, but if I will add some hosts, database can't properly work, because volume with data will only exists on one host.
I think, I cannot use NFS (or some other network storage) for databases because of IO speed. Am I right? Is there any workflow to use databases in Rancher?
I thought, that I can bind service only to one host of environment, but I didn't find this setting.
Rancher has Storage Services to manage Volumes.
NFS (rancher-nfs) will not work very well for a read/write Postgres database. If you're running on AWS Rancher also supports EBS volumes via convoy
For vanilla Docker local is the only volume type that will work in most places but these are obviously not shared volumes, it's local to the Docker host. If you create an environment scoped local volume for the database then the database container will always be scheduled to the node holding the storage.
Other shared storage volume types require the storage backend and a Docker volume plugin that can manage the movements of external storage. REX-Ray supports a number of storage providers. Flocker also supports a number of storage providers.
So with the introduction of volumes we do not longer use data-only containers! Nice. But right now I have this nice home-grown ZFS appliance and I want to use it as a backend for my docker volumes (of course Docker is running on other hosts).
I can export ZFS as NFS relatively easy, what are proven (i.e. battle-tested) options for using NFS as a volume backend for docker?
A google search shows me the following possibilities:
using Flocker, I could use the flocker-agent-thingie on the zfs appliance. However with Flocker being scrapped, I am concerned...
using the local volume backend and simply mount the nfs export on the docker host -> does not scale, but might do the job.
using a specialized volume plugin like https://github.com/ContainX/docker-volume-netshare to utilize nfs
something alike from Rancher: https://github.com/rancher/convoy
or going big and use Kubernetes and NFS as persistent storage https://kubernetes.io/docs/concepts/storage/volumes/#nfs
I have pretty extensive single-host Docker knowledge - which option is a stable one? Performance is not that important to me (alas the use case is a dockerized OwnCloud/NextCloud stack, throughput is limited by the internet connection)