How can I use different filesystems in a container?
I have tried to make an img with XFS filesystem.
mkfs.xfs proba.img
It was succesful, but when tried to mount it, get following error. Already tried to use chmod, but same error.
mount: block device /root/proba.img is write-protected, mounting read-only
mount: cannot mount block device /root/proba.img read-only
xfs_info's output: proba.img is not a mounted XFS filesystem
Is XFS(or another) filesystem usage possible in a single conatiner?
As per this question too, it's not possible to mount a filesystem inside a container.
You'll have to mount the new image in the Docker host and expose it to the container as a volume. See question above for examples.
Related
How do multiple containers(create from same image) access the directory(packaged in the image) in other container? There are some ways not elegant:
Mount directory into a shared volume or host path. When a container changed files in the directory, other containers will perceive it. Is there any idea to make shared volume working like copy-on-write?
Create a volume for each container, and copy directory into the volume. In this way, the directory have to save multiple copies in disk.
I've browsed this problem [https://stackoverflow.com/questions/29550736/can-i-mount-docker-host-directory-as-copy-on-write-overlay], I don't want containers to run in privileged mode. Is there a better idea?
Per design one container cannot access the files inside another container.
You will need to store the data outside of a container, then mount it via volume mount or bind mount. Alternatively you could try to perform some NFS/SMB/... mount of shared storage inside the container.
This is completely independent from running a container in host mode.
docker run -ti --rm -v DataVolume3:/var ubuntu
Lets say I have a volume DataVolume 3 which pulls the contents of /var in the ubuntu container
even after killing this ubuntu container the volume remains and I can use this volume DataVolume3 to mount it to other containers.
This means with the deletion of container the volume mounts are not deleted.
How does this work ?
Does that volume mount mean that it copies the contents of /var into some local directory because this does not look like a symbolic link ?
If I have the container running and I create a file in the container then the same file gets copied to the host path ?
How does this whole process of volume mount from container to host and host to container work ?
Volumes are used for persistent storage and the volumes persists independent of the lifecycle of the container.
We can go through a demo to understand it clearly.
First, let's create a container using the named volumes approach as:
docker run -ti --rm -v DataVolume3:/var ubuntu
This will create a docker volume named DataVolume3 and it can be viewed in the output of docker volume ls:
docker volume ls
DRIVER VOLUME NAME
local DataVolume3
Docker stores the information about these named volumes in the directory /var/lib/docker/volumes/ (*):
ls /var/lib/docker/volumes/
1617af4bce3a647a0b93ed980d64d97746878564b141f30b6110d0818bf32b76 DataVolume3
Next, let's write some data from the ubuntu container at the mounted path var:
echo "hello" > var/file1
root#2b67a89a0050:/# cat /var/file1
hello
We can see this data with cat even after deleting the container:
cat /var/lib/docker/volumes/DataVolume3/_data/file1
hello
Note: Although, we are able to access the volumes like shown above but it not a recommended practice to access volumes data like this.
Now, next time when another container uses the same volume then the data from the volume gets mounted at the container directory specified as part of -v flag.
(*) The location may vary based on OS as pointed by David and probably can be seen by the docker volume inspect command.
Docker has a concept of a named volume. By default the storage for this lives somewhere on your host system and you can't directly access it from outside Docker (*). A named volume has its own lifecycle, it can be independently docker volume rm'd, and if you start another container mounting the same volume, it will have the same persistent content.
The docker run -v option takes some unit of storage, either a named volume or a specific host directory, and mounts it (as in the mount(8) command) in a specific place in the container filesystem. This will hide what was originally in the image and replace it with the volume content.
As you note, if the thing you mount is an empty named volume, it will get populated from the image content at container initialization time. There are some really important caveats on this functionality:
Named volume initialization happens only if the volume is totally empty.
The contents of the named volume never automatically update.
If the volume isn't empty, the volume contents completely replace what's in the image, even if it's changed.
The initialization happens only on native Docker, and not for example in Kubernetes.
The initialization happens only on named volumes, and not for bind-mounted host directories.
With all of these caveats, I'd avoid relying on this functionality.
If you need to mount a volume into a container, assume it will be empty when your entrypoint or the main container command starts. If you need a particular directory layout or file structure there, an entrypoint script can create it; if you're expecting it to hold particular data, keep a copy of it somewhere else in your image and copy it in if it's not already there (or, perhaps, always).
(*) On native Linux you can find a filesystem location for it, but accessing this isn't a best practice. On other OSes this will be hidden inside a virtual machine or other opaque storage. If you need to directly access the data (or inject config files, or read log files) a docker run -v /host/path:/container/path bind mount is a better choice.
Volumes are part of neither the container nor the host. Well, technically everything resides in the host machine. But the docker directories are only accessible by users in "docker" group. The files in these directories are separately managed by docker.
"Volumes are stored in a part of the host filesystem which is managed by Docker (/var/lib/docker/volumes/ on Linux)."
Hence volumes are like the union of files under the docker container and the host itself. Any addition on either end will be added to the volume(/var/lib/docker/volumes), not hard copy, rather something like symbol link
As volumes can be shared across different containers, deleting a container does not cascade to the volumes associated with it.
To remove unused volumes:
docker volume prune .
I have a volume mount that mounts my /media directory to the container. It sees the mount points that exist there, but can only see the contents of them if the disk is mounted before the container is started. If a drive is disconnected then reconnected, or was connected after the container has started, it can't view the contents.
How can I make it so that it automatically updates the volume mount when I mount a drive on the host?
The requirement for mounts is that your host folder must exist when starting the container.
I think your problem is due to your /media in host disappear when you unmount. So, if the container doesn't find volume folder, it loses reference.
The only thing I can recommend you is:
Create folder /media in host.
Mount your fs in /media/...
Build docker image where /media doesn't exist.
Create container sharing /media volume.
I hope this will be useful.
In docker, where is the container data, apart from the mounted volume, is stored till the container is running.
So lets say /usr/local is volume mounted, so it would be shared between the host and the container. Where is everything else stored?
You should inspect your docker container
docker inspect [ID_CONTAINER]
and check for the fields MergedDir, LowerDir and UpperDir. Docker uses OverlayFS file system to store the data.
OverlayFS layers two directories on a single Linux host and presents them as a single directory. These directories are called layers and the unification process is referred to a a union mount. OverlayFS refers to the lower directory as lowerdir and the upper directory a upperdir. The unified view is exposed through its own directory called merged.
Check the doc here.
You can run the following command to get container written data
ll `podman inspect ContainerName --format "{{ .GraphDriver.Data.MergedDir }}"`
In the context of docker I understand what layers of an image is and what's the uppermost writable layer. If I run docker container without any -v arguments will any volumes get created? If I do specify a volume (which is not a mount to some location in host) does the volume become this container's writable layer?
Volumes and image layers are separate concepts in Docker. To your first question, no, no volumes are created without -v being used (ignoring for the moment the fact that the Dockerfile format does have a VOLUME verb).
If you specify a volume that is provided by any volume driver (the default being a local directory which will bind-mount into the target location in your container filesystem), this volume is unrelated to the image layers, including the 'writable' topmost layer provided by the back-end storage driver in use in your Docker engine.
Specifically, the storage driver in use is what controls how the layer stack, and that top "r/w" layer is managed. For example, the overlay or devicemapper or btrfs driver handles the mount of the image layers and creating and managing the top layer.
Volumes come into play outside of this by being bind-mounted in to existing (or created) paths within the filesystem image. When you exit the container, these volumes are preserved in their source location (e.g. for a local/default volume driver, in /var/lib/docker/volumes/<name>) and the unmounted layer stack will have, under management of the storage driver in use, the modified "upper layer" of the layer stack, unless you had your container removed on exit (--rm). These are two separately managed concepts and the volume system has no interplay with the storage backend driver.
Volume is for mounting data between container and host
If I run docker container without any -v arguments will any volumes get created?
-v is not for creating the volume, just to mount the data from and to
-v [host_path]:[container_path],
here host_path, your server directory path where you want to mout the data to
container_path, container directory path mount date from
If I do specify a volume (which is not a mount to some location in host) does the volume become this container's writable layer?
When the path doesn't exist in the host. But you have mountaed it to container, there are two cases here.
CASE 1: If you want to mount some config files from your machine to container if the container expects that file while running then it throws error in the logs
CASE 2: If you want to mount the volume from container to host machine. When cotainer starts running it creates a folder in the host machine and writes the data to the host machine mounted directory.