I have a builder image / container which is supposed to run tests on a directory with tests sources.
The container is run in a Kubernetes pod, in AWS EKS, through helm test. I.e. not docker, so I can't simply use -v volume mount.
I am struggling to find the right way to bring this directory to the container, in a simple way. This is a Helm template I have. All works except for the volume.
apiVersion: v1
kind: Pod
metadata:
name: "{{ .Release.Name }}-gatling-test"
annotations:
"helm.sh/hook": test-success
spec:
restartPolicy: Never
containers:
- name: {{ .Release.Name }}-gatling-test
image: {{ .Values.builderImage }}
command: ["sh", "-c", 'mvn -B gatling:test -pl csa-testing -DCSA_SERVER={{ template "project.fullname" . }} -DCSA_PORT={{ .Values.service.appPort }}']
## TODO: The builder image also counts with having /tmp/build, so it needs a mount: -v '${job.WORKDIR}:/tmp/build'
volumeMounts:
- name: mavenRepoToBuild
mountPath: /tmp/build
volumes:
- name: mavenRepoToBuild
hostPath:
path: {{.Values.fromJenkins.WORKDIR}}
I've read on few places that it can't be done directly. So what's the easy way to do it indirectly? Zip and upload to S3 and download? Or add it to the image as a layer? Or should I create a Kubernetes volume resource?
The hostPath directory or file must be existing on all your cluster nodes.
You can attach some types on the hostPath to determine whether its files or directories.
List of types you can use on hostpath can be found in kubernetes documentation.
https://kubernetes.io/docs/concepts/storage/volumes/
Btw, what error do you get? Permission denied? You can do helm dry-run to see the rendered template.
Related
In docker world, we can use pure docker image and use --volumes-from to connect it with container, how this works in the kubernetes
Docker image could be html files like
FROM scratch
COPY html /www
How can I mount it to the nginx pod?
BTW: Surely I can turn docker pure data image to use busybox as base image, then copy the data out using initContainers, which make the image 1M bigger, here try to see whether it is possible in k8s world
Unlike Docker's named volumes, volume mounts in Kubernetes never copy anything into the volume. You occasionally see tricks with Docker Compose setups where a Docker named volume is mounted over two containers, with the expectation that static files will be copied into the volume from one of them to be served by the other; this just doesn't work in Kubernetes unless you copy the files yourself.
For the setup you show, you have a collection of files you want to serve, and you want to have the standard nginx image serve them. Instead of trying to copy files between images, you can have a unified image that starts FROM nginx and contains your files:
FROM nginx
COPY html /usr/share/nginx/html
# Base image provides a suitable default CMD and other setup
You don't need any sort of volume to run this. Just specify it as the image: in your Deployment spec, and all of the files to be served are already compiled into the image.
You would use the volumes and volumeMounts.
UPD:
html file to mount:
$ cat index.html
<h1>HELLO</h1>
Create configMap with the content of the file:
$ kubectl create configmap nginx-index-html-configmap --from-file=index.html
configmap/nginx-index-html-configmap created
nginx pod file:
$ cat nginx-with-config.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
volumeMounts:
- mountPath: /usr/share/nginx/html/index.html
name: nginx-config
subPath: index.html
volumes:
- name: nginx-config
configMap:
name: nginx-index-html-configmap
Creating the pod:
$ kubectl create -f nginx-with-config.yaml
pod/nginx created
Checking nginx serves the file:
$ kubectl exec -it nginx -- curl 127.0.0.1
<h1>HELLO</h1>
UPD2:
You can have everything in one big happy file, no need to prep anything in advance:
$ cat nginx-with-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-index-html-configmap-2
data:
index.html: |
<h1>HELLO 2!</h1>
---
apiVersion: v1
kind: Pod
metadata:
name: nginx2
spec:
containers:
- name: nginx
image: nginx
volumeMounts:
- mountPath: /usr/share/nginx/html/index.html
name: nginx-config
subPath: index.html
volumes:
- name: nginx-config
configMap:
name: nginx-index-html-configmap-2
$ kubectl apply -f nginx-with-config.yaml
configmap/nginx-index-html-configmap-2 created
pod/nginx2 created
$ kubectl exec -it nginx2 -- curl 127.0.0.1
<h1>HELLO 2!</h1>
I'm creating an application that is using helm(v3.3.0) + k3s. A program in a container uses different configuration files. As of now there are just few config files (that I added manually before building the image) but I'd like to add the possibility to add them dynamically when the container is running and not to lose them once the container/pod is dead. In docker I'd do that by exposing a folder like this:
docker run [image] -v /host/path:/container/path
Is there an equivalent for helm?
If not how would you suggest to solve this issue without stopping using helm/k3s?
In Kubernetes (Helm is just a tool for it) you need to do two things to mount host path inside container:
spec:
volumes:
# 1. Declare a 'hostPath' volume under pod's 'volumes' key:
- name: name-me
hostPath:
path: /path/on/host
containers:
- name: foo
image: bar
# 2. Mount the declared volume inside container using volume name
volumeMounts:
- name: name-me
mountPath: /path/in/container
Lots of other volumes types and examples in Kubernetes documentation.
Kubernetes has a dedicated construct for holding configuration files, ConfigMaps. Helm in turn has support for Accessing Files Inside Templates which can help you copy them into ConfigMap objects. A minimal setup here would look like:
# templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: my-config
data:
config.ini: |
{{ .Files.Get "config.ini" | indent 4 }}
# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment:
metadata: { ... }
spec:
template:
spec:
volumes:
- name: config-data
configMap:
name: my-config # matches ConfigMap metadata: { name: }
containers:
- volumeMounts:
- name: config-data # matches volume name: in this file
mountPath: /container/path
You can use Helm's templating constructs in various ways here: to dynamically construct the contents of the ConfigMap, to set an environment variable saying which file to use, and so on.
Do not use hostPath volumes here. Since Kubernetes is designed as a clustered environment, you do not have much control over which node a given pod will run on; you would have to copy these config files to every node in the cluster and try to update them all when a file changed. That's a huge maintenance problem, especially if you don't have direct filesystem access to the nodes.
I have a docker image that creates few folders and extract files into it like below
RUN mkdir -p /home/myapp/myappv4 \
/home/myapp/myappv4/files \
/home/myapp/myappv4/files/logs \
/home/myapp/myappv4/myappentries
WORKDIR /home/myapp
RUN chown -R myapp:myapp /home/myapp
ADD /myapp-v4-files/*.zip /home/myapp/myappv4/files/
ADD /myapp-v4-files/init.txt /home/myapp/myappv4/myappentries/
ADD /myapp-v4-files/pro.json /home/myapp/myappv4/myappentries/
These folders and files needs to be accessed by other containers in a pod in kubernetes. Should i create persistentvolume in kubernetes and have these locations in them and copy the content from this container to this volume? In that way they would not get deleted right?. Since i am new to kubernetes i am not sure on how to achieve this. Transition from docker container to kubernetes deployment seems to be a confusing part for me,any help on this would be appreciated.
If you want to share a set of directories between multiple containers in a single pod, using only EmptyDir volume will suffice. You don't need to use PersistentVolumes (unless you want persistence, meaning you want the data to survive pod restarts).
However note that adding a volume (a kubernetes construct) will overwrite the files already present in your container at the path where you are mounting the volume, kind of what happens with a layered filesystem that docker uses.
For your usecase, I think you can move the file fetching logic from the Dockerfile to a script that the pod will run, that will fix the above mentioned issue.
apiVersion: v1
kind: Pod
metadata:
name: test-pd
spec:
containers:
- image: k8s.gcr.io/test-webserver
name: test-container
volumeMounts:
- mountPath: /cache
name: cache-volume
- image: k8s.gcr.io/test-webserver
name: test-container-2
volumeMounts:
- mountPath: /cache
name: cache-volume
volumes:
- name: cache-volume
emptyDir: {}
Read more about volumes here.
I have a command to run docker,
docker run --name pre-core -itdp 8086:80 -v /opt/docker/datalook-pre-core:/usr/application app
In above command, /opt/docker/datalook-pre-core is host directory, /usr/application is container directory. The purpose is that container directory maps to host directory. So when container crashes, the directory functions as storage and data on it would be saved.
When I am going to use kubernetes to create a pod for this containter, how to write pod.yaml file?
I guess it is something like following:
apiVersion: v1
kind: Pod
metadata:
name: app-ykt
labels:
app: app-ykt
purpose: ykt_production
spec:
containers:
- name: app-ykt
image: app
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
volumnMounts:
- name: volumn-app-ykt
mountPath: /usr/application
volumns:
- name: volumn-app-ykt
？？？？
I do not know what's the exact properties in yaml I shall write in my case?
This would be a hostPath volume: https://kubernetes.io/docs/concepts/storage/volumes/
volumes:
- name: volumn-app-ykt
hostPath:
# directory location on host
path: /opt/docker/datalook-pre-core
# this field is optional
type: Directory
However remember that while a container crash won't move things, other events can cause a pod to move to a different host so you need to be prepared to both deal with cold caches and to clean up orphaned caches.
How can I inject code/files directly into a container in Kubernetes on Google Cloud Engine, similar to the way that you can mount host files / directories with Docker, e.g.
docker run -d --name nginx -p 443:443 -v "/nginx.ssl.conf:/etc/nginx/conf.d/default.conf"
Thanks
It is possible to use ConfigMaps to achieve that goal:
The following example mounts a mariadb configuration file into a mariadb POD:
ConfigMap
apiVersion: v1
data:
charset.cnf: |
[client]
# Default is Latin1, if you need UTF-8 set this (also in server section)
default-character-set = utf8
[mysqld]
#
# * Character sets
#
# Default is Latin1, if you need UTF-8 set all this (also in client section)
#
character-set-server = utf8
collation-server = utf8_unicode_ci
kind: ConfigMap
metadata:
name: mariadb-configmap
MariaDB deployment
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: mariadb
labels:
app: mariadb
spec:
replicas: 1
template:
metadata:
labels:
app: mariadb
version: 10.1.16
spec:
containers:
- name: mariadb
image: mariadb:10.1.16
ports:
- containerPort: 3306
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mariadb
key: rootpassword
volumeMounts:
- name: mariadb-data
mountPath: /var/lib/mysql
- name: mariadb-config-file
mountPath: /etc/mysql/conf.d
volumes:
- name: mariadb-data
hostPath:
path: /var/lib/data/mariadb
- name: mariadb-config-file
configMap:
name: mariadb-configmap
It is also possible to use subPath feature that is available in kubernetes from version 1.3, as stated here.
I'm not sure you can do that exactly. Kubernetes does things quite differently than docker, and isn't really ideal for interacting with the 'host' you are probably used to with docker.
A few alternative possibilities come to mind. First, and probably least ideal but closest to what you are asking, would be to add the file after the container is running, either by adding commands or args to the pod spec, or using kubectl exec and echo'ing the contents into the file. Second would be to create a volume where that file already exists, e.g. create a GCE or EBS disk, add that file, and then mount the file location (read-only) in the container's spec. Third, would be to create a new docker image where that file or other code already exists.
For the first option, the kubectl exec would be for one-off jobs, it isn't very scalable/repeatable. Any creation/fetching at runtime adds that much overhead to the start time for the container, so I normally go with the third option, building a new docker image whenever the file or code changes. The more you change it, the more you'll probably want a CI system (like drone) to help automate the process.
Add a comment if I should expand any of these options with more details.
Kubernetes allows you to mount volumes into your pod. One such volume type is hostPath (link) which allows you to mount a directory from the host into the pod.