I'm trying to build a Container Optimized VM in Google Cloud to host a Docker container. This Docker container needs storage but the optimized container VM images have almost no writeable storage. I then created a persistent disk to attach to the VM to mount in the container, but the VMs /etc is also read only, so I'm unable to write to fstab, OR mount the disk anywhere in the filesystem. How is this supposed to be accomplished in a VM that is designed specifically to host Docker containers?
The storage space in the instances is independent of the image used.
You can change the boot disk size on creation time or later. This will allow you to have more storage space in the instance.
If you want to use Kubernetes Engine it is also possible to change the boot disk size on creation time.
Related
I'm studying about Docker and I couldn't understand very well the difference between what storage drivers are used for and what volume drivers are used for.
My theory (please correct me if I'm wrong) is the storage drivers manage the way Docker deals underneath with the writable layer, can use overlay, overlay2, aufs, zfs, btrfs and so on.
Volume drivers, however, deal with volumes underneath, like it can be a local volume (in this case I think it will use a storage driver) or can be a remote one (like EBS).
Am I right?
Docker uses storage drivers to store image layers, and to store data in the writable layer of a container. Docker uses Volume drivers for write-intensive data, data that must persist beyond the container’s lifespan, and data that must be shared between containers. So, I understand Storage Drivers are used with image and container layers while Volume Drivers are used for persistent container application data. See the first three paragraphs of this Docker documentation: https://docs.docker.com/storage/storagedriver/
Docker engine volume enable engine deployments to be integrated with external storage systems such as Amazon EBS, and enable data volumes to persist beyond the lifetime of a single Docker host. Here the term 'local' in concept of Docker volume driver means the volumes esdata1 and esdata2 are created on the same Docker host where you run your container. By using other Volume plugins, e.g.,-driver=flocker. You are able to create a volume on a external host and mount it to the local host, say, /data-path.
As someone new to docker world and coming from a virtual machine mindset. I downloaded a docker image for elastic search from docker hub.I am thinking about any configurations I need to do because a lot of data will be forwarded to this image. I need to be considerate about the available disk space. In a Virtual machine world, I can always add additional vhds to increase disk size etc. What's similar operation in a docker world?
A Docker container uses copy-on-write storage drivers such as aufs, btrfs, ... to manage the container layers. All writes done by a container are persisted in the top read-write layer. These layers (especially the write layer) are what determine a container's size.
There is a size limit to the Docker container known as base device size. The default value is 10GB. This value can be changed to allow the container to have more size using dockerd --storage-opt dm.basesize=50G. This will give the container a rootFS size of 50GB.
However, this is not the recommended way to handle heavy write operations that increase the container size. The recommnded way to do that is using Docker volumes. Volumes are not persisted within the Docker local storage area for container layers (i.e /var/lib/docker/<storage-driver>/...), and are thus independent from the container's storage driver. Therefore, they do not contribute to the size of the container.
There is no limits on the number of volumes a container can have. It is recommended to map directories inside the container that will grow in size into volumes.
For more info about storage drivers check About storage drivers
Note: for write-heavy applications, you should not store the data in
the container. Instead, use Docker volumes, which are independent of
the running container and are designed to be efficient for I/O. In
addition, volumes can be shared among containers and do not increase
the size of your container’s writable layer.
Use volumes mounted to the container for storage. That way the size of your containers does not become too big.
https://docs.docker.com/storage/volumes/
I have a Linux (Ubuntu) machine with a partition on an SSD raid and a partition on an HDD raid. I want to put my docker containers with high traffic (like a database) on the SSD part and the other containers on the cheaper HDD part. I can't find an answer here or on other sides. Is there a possibility?
Docker itself doesn't provide that level of control over Docker storage on a per container basis.
You can use the devicemapper storage driver and use a specific raid logical volume for the container file systems. There's no way to choose between multiple storage devices at container run time, or via some policy.
Docker does have volumes that can be added to a container and volume plugins to use different storage backends for volumes. These can controlled on a per container basis.
There is an LVM volume plugin. You could assign SSD's to a lvm volume
group and mount data volumes from that in any container you want the extra write performance in.
Another option would be to run multiple Docker daemons, one with each storage configuration, that would be difficult to maintain.
I've read through so much documentation, and I'm still not sure how this really works. It's a bit of a Docker vs. VM question.
If I start a VM with a 2GB hard drive and fill its disk with files, I know it runs out after 2GB of files.
Does Docker work the same way? I would assume so. But from what I've read about "UnionFS" it seems like it does not run out of space.
So then why do Docker "volumes" exist? Is that automagically expanding Docker disk space transient in some way? Will the files I've saved inside of my Docker container disappear after a reboot? How about after restarting the container?
Docker's usage (1.12+) depends on the Docker storage driver and possibly the physical file system in use.
TL;DR Storage will be shared between all containers and local volumes unless you are using the devicemapper storage driver or have set a limit via docker run --storage-opt size=X when running on the zfs or btrfs drivers. Docker 1.13+ also supports a quota size with overlay2 on an xfs backed file system.
Containers
For all storage drivers, except devicemapper, the container and local volume storage is limited by the underlying file system hosting /var/lib/docker and it's subdirectories. A container can fill the shared file system and then other containers can't write any more.
When using the devicemapper driver, a default volume size of 100G is "thin allocated" for each container. The default size can be overridden with the daemon option --storage-opt dm.basesize option or set on a per container basis with docker run --storage-opt size=2G.
The same per container quota support is available for the zfs and btrfs drivers as both file systems provide simple built in support for creating volumes with a size or quota.
The overlay2 storage driver on xfs supporta per container quotas as of Docker 1.13. This will probably be extended to ext4 when new 4.5+ kernels become standard/common and ext4 and xfs quotas share a common API.
Volumes
Docker volumes are separate from a container and can be viewed as a persistant storage area for an ephemeral container.
Volumes are stored separately from Docker storage, and have their own plugins for different backends. local is the default backend, which writes data to /var/lib/docker/volumes so is held outside of the containers storage and possible quota system.
Other volume plugins could be used if you wanted to set per volume limits on a local file system that supports it.
Containers will keep their own file state over a container restart and reboot, until you docker rm the container. Files in a volume will survive a container removal and can be mounted on creation of the new container.
Suppose I use a Cloud SDK Docker container, which lets me run various gcloud commands, including gcloud compute disks create which creates a Google persistent disk. However I cannot then attach to this disk within the container, since gcloud compute instances attach-disk only works on GCE instances and not Docker containers.
Is there a way for the container to attach or even access the persistent disk? Can I in fact attach persistent disks to arbitrary Linux machines, not just GCE instances?
I know I can use either Docker or Kubernetes to attach persistent disks fixed and determined before the container is launched, but what I need is the container itself to attach to arbitrary persistent disks as determined by container code.
Can I in fact attach persistent disks to arbitrary Linux machines, not just GCE instances?
No, you can only attach GCE persistent disks to GCE VMs.
I cannot then attach to this disk within the container, since gcloud compute instances attach-disk only works on GCE instances and not Docker containers.
If the container is running inside a GCE VM, you should be able to attach the persistent disk to the VM that hosts the container.
I need is the container itself to attach to arbitrary persistent disks as determined by container code.
If you run your container in privileged mode, then you should be able to run the appropriate mount commands to mount the disk after you've attached it to the VM. You can try mapping in a volume to the container that is initially empty and then mounting the PD to that path, but I'm not sure whether it will work.