I understand that in Kubernetes you don't want to "tie" a pod to a host, but in certain cases you might need to.
In my particular case I have a DB that lives on blockstorage which is mounted to a specific host.
What I am trying to accomplish with Kubernetes is the equivalent of a bind-mount in Docker. I want to specify the directory on the host that I need mounted in the pod, similar to this:
/mnt/BTC_2:/root/.bitcoin:rw
How do I specify the location of where I want my persistent storage to be on the node/host? Would this be a hostPath volume like the following:
volumeMounts:
- mountPath: /root/.bitcoin
name: test-volume
volumes:
- name: test-volume
hostPath:
# directory location on host
path: /mnt/BTC_2
I want to specify the directory on the host that I need mounted in the pod
That should be documented here
A hostPath volume mounts a file or directory from the host node’s filesystem into your pod. This is not something that most Pods will need, but it offers a powerful escape hatch for some applications.
Warning:
The files or directories created on the underlying hosts are only writable by root. You either need to run your process as root in a privileged container or modify the file permissions on the host to be able to write to a hostPath volume
volumes:
- name: test-volume
hostPath:
# directory location on host
path: /data
# this field is optional
type: Directory
Related
Currently, some of my pods are using ephemeral volumes (those created by Docker at the container level). I want some form of persistent data stored in specific directories on the host (like bind mounts in Docker) such that my pods can restart without losing data. Persistent Volume Claims seem to be the best way to do this. How should I go about transitioning my existing pods to use PVCs that copy across data from the existing ephemeral volumes?
On the host, create the directory to hold your data. Eg. mkdir /local_data
Copy the data to the local directory. Eg. kubectl cp <namespace>/<pod>:/path/in/the/container /local_data
Check and check again all your data is intact in /local_data
Create a new pod with the following spec.
Example:
kind: Pod
...
spec:
...
nodeName: <name> # <-- if you have multiple nodes this ensure your pod always run on the host that hold the data
containers:
- name: ...
...
volumeMounts:
- name: local_data
mountPath: /path/in/the/container
...
volumes:
- name: local_data
hostPath: /local_data
type: Directory
Apply and check if your pod runs as expected
I'm try to get my local development environment converted from docker-compose to a flavour of lightweight Kubernetes, since our production environments are all Kubernetes (like). With the help of Kompose I could convert my settings. The part however I struggle with is the conversion of Docker volumes to a local developer machine Volume and Volume claim. Most examples I found cover drivers for S3 or NFS, which isn't what my development laptop offers.
So when I have in my docker-compose.yml a setting like this, what's the local development equivalent in k8s?
version: "3.6"
services:
my_test:
image: "myregistry.com/my_test"
container_name: "demo"
hostname: "demo.acme.local"
stop_grace_period: 120s
cap_add:
- SYS_PTRACE
ports:
- 80:80
volumes:
- my_vol:/local/my_app
volumes:
my_vol:
name: "/local/containers/my_app"
external: false
I'm "only" looking for the volume part. The rest seems clear.
Help is appreciated
Both solutions (hostpath and emptydir) you found looks good.
Here you have documentations to both ones:
A hostPath volume mounts a file or directory from the host node's filesystem into your Pod.
To the required path property, you can also specify a type for a hostPath volume.
NOTE:
HostPath volumes present many security risks, and it is a best practice to avoid the use of HostPaths when possible. When a HostPath volume must be used, it should be scoped to only the required file or directory, and mounted as ReadOnly.
hostPath configuration example
An emptyDir volume is first created when a Pod is assigned to a node, and exists as long as that Pod is running on that node.
EmptyDir volume is initially empty. All containers in the pod have permissions in the emptyDir volume to read and write the same files, despite possibility of diffrent localisations of that volume (it can be mounted at the same or different paths in each container).
In case the Pod is removed from a node for any reason, the data in the emptyDir is deleted permanently.
Depending on your environment, emptyDir volumes are stored on whatever medium that backs the node such as disk or SSD, or network storage.
emptyDir configuration example
Yes you can use the host path or you can use the emptydir that's on you
Probably hostpath would be a better option.
If you are planning to use the minikube you can also check out the PV and PVC option which is mostly for the persistence storage in Kubernetes.
Hostpath : https://kubernetes.io/docs/concepts/storage/volumes/#hostpath-configuration-example
emptydir : https://kubernetes.io/docs/concepts/storage/volumes/#emptydir
When configuring a docker-compose.yml, I can easily mount a volume that maps a folder from the host machine to the container:
...
volumes:
- "/host/folder/:/container/folder/"
...
However, if I try to use the long syntax, that doesn't work anymore:
...
volumes:
- type: volume
source: /host/folder
target: /container/folder/
...
According to the docs
source: the source of the mount, a path on the host for a bind mount,
or the name of a volume defined in the top-level volumes key. Not
applicable for a tmpfs mount.
So, it seems that in the long syntax I have to use the bind type to mount a host path. Does this make sense, different features according to syntax?
Furthermore, again, according to the docs,
Volumes are the preferred mechanism for persisting data generated by
and used by Docker containers. While bind mounts are dependent on the
directory structure and OS of the host machine, volumes are completely
managed by Docker.
So, I much prefer having volumes instead of binds. Do I have to use the short syntax for that?
The type: field says whether it's a named volume, a bind mount, or a couple of other things. Since you're mounting a host directory, you need to specify type: bind in the extended syntax.
volumes:
- type: bind # <-- not "volume"
source: /host/folder
target: /container/folder/
according to the docs, "volumes are [...] preferred...."
IMHO the Docker documentation is very enthusiastic about named volumes and glosses over their downsides. Since you can't access the contents of a named volume from outside of Docker, they're harder to back up and manage, and a poor match for tasks like injecting config files and reviewing logs. I would not automatically reach for a named volume because the Docker documentation suggests it's preferred.
version: '3.8'
services:
some-application:
volumes:
# Use a bind mount to inject configuration files; you need to
# directly edit them on the host
- type: bind
source: ./config
target: /app/config
# Use a bind mount to read back log files; you need to read them
# on the host
- type: bind
source: ./log
target: /app/data
# Use a named volume for opaque application data; you do not
# need to manipulate files on the host, and on MacOS/Windows
# a named volume will be faster
- type: volume
source: app-data # when type: volume, this is a volume name
target: /app/data
# Do not mount anything over /app or the code in the image.
volumes:
app-data:
Try this:
...
services:
some-service:
image: some-image
volumes:
- my-volume-name:/some-path-in-container
volumes:
my-volume-name:
driver: local
driver_opts:
type: 'none'
o: 'bind'
device: '/my-host-folder'
...
I have a docker image that contains data in directors /opt/myfiles, Lets say the following:
/opt/myfiles/file1.txt
/opt/myfiles/file2.dat
I want to deploy that image to kubernetes and mount an NFS volume to that directory so that the changes to these files are persisted when I delete the pod.
When I do it in docker swarm I simply mount an empty NFS volume to /opt/myfiles/ and then my docker swarm service is started, the volume is populated with the files from the image and I can then work with my service and when I delete the service, I still have the files on my NFS server, so on next start of the service, I have my previous state back.
In kubernetes, when I mount an empty NFS volume to /opt/myfiles/, the pod is started and /opt/myfiles/ is overwritten with an empty directory, so my pod does not see the files from the image anymore.
My volme mount and volume definition:
[...]
volumeMounts:
- name: myvol
mountPath: /opt/myfiles
[...]
volumes:
- name: myvol
nfs:
server: nfs-server.mydomain.org
path: /srv/shares/myfiles
I read some threads about similar problems (for example K8s doesn't mount files on Persistent Volume) and tried some stuff using subPath and subPathExpr as in the documentation (https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath) but none of my changes does, what docker swarm does by default.
The behaviour of kubernetes seems strange to my as I have worked with docker swarm for quite a while now and I am familiar with the way docker swarm handles that. But I am sure that there is a reason why kubernetes handles that in another way and that there are some possibilities to get, what I need.
So please, can someone have a look at my problem and help me find a way to get the following behaviour?
I have files in my image in some directory
I want to mount an NFS volume to that directory, because I need to have the data persisted, when for example my pod crashes and moves to another host or when my pod is temporarily shut down for whatever reason.
When my pod starts, I want the volume to be populated with the files from the image
And of course I would be really happy if someone could explain me, why kubernetes and docker swarm behave so different.
Thanks a lot in advance
This can be usually achieved in Kubernetes with init-containers.
Let's have an image with files stored in /mypath folder. If you are able to reconfigure the container to use a different path (like /persistent) you can use init container to copy files from /mypath to /persistent on pod's startup.
containers:
- name: myapp-container
image: myimage
env:
- name: path
value: /persistent
volumeMounts:
- name: myvolume
path: /persistent
initContainers:
- name: copy-files
image: myimage
volumeMounts:
- name: myvolume
path: /persistent
command: ['sh', '-c', 'cp /mypath/*.* /persistent']
In this case you have a main container myapp-container using files from /persistent folder from the NFS volume and each time when container starts the files from /mypath will be copied into that folder by init container copy-files.
I am using this config to mount drive that is read-only to docker image:
volumes:
- name: drive-c
hostPath:
path: /media/sf_C_DRIVE
containers:
...
volumeMounts:
- name: drive-c
mountPath: /c
When I try to write (mkdir) to this folder (which is virtual-box read only mounted folder) inside docker image, I get:
mkdir: can't create directory 'somedir': Read-only file system
Which is OK since this is read-only filesystem (C_DRIVE on /c type vboxsf (rw,nodev,relatime))
Is it possible to somehow make this folder writable inside docker container? Changes does not need to be propagated to hostPath. I read something about overlayfs, but I'm not sure how to specify this in Kubernetes yaml.
You can't do this via Kubernetes volume mounts, as they simply work on volumes. What you need to do is create a writable overlay filesystem on your host machine, then map that into Kubernetes with a different path that is writable.
You want to create something like /mnt/writable_sf_C_drive by mounting your /media/sf_C_DRIVE with an overlay on top of it, then your volume mount in Kubernetes would mount /mnt/writable_sf_C_DRIVE.
There are some instructions here