kubernetes volumes for nestjs app for hot reloading data - docker

I'm having an issue with volumes on Kubernetes when I'm trying to mount hostPath volumes. (i also tried with PVC, but no success)
Dockerfile:
FROM node:16
WORKDIR /usr/src/app
COPY package.json /usr/src/app
RUN yarn install
COPY . /usr/src/app
EXPOSE 3000
ENTRYPOINT ["yarn", "start:dev"]
docker-compose.yml:
version: '3.8'
services:
api:
container_name: api
build:
context: .
dockerfile: Dockerfile
volumes:
- .:/usr/src/app
- /usr/src/app/node_modules
ports:
- 3000:3000
restart: always
labels:
kompose.volume.type: 'hostPath'
database:
container_name: database
image: postgres:latest
ports:
- 5432:5432
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: task-management
api-development.yml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
kompose.cmd: kompose -f docker-compose.yml convert
kompose.version: 1.26.1 (HEAD)
kompose.volume.type: hostPath
creationTimestamp: null
labels:
io.kompose.service: api
name: api
spec:
replicas: 1
selector:
matchLabels:
io.kompose.service: api
strategy:
type: Recreate
template:
metadata:
annotations:
kompose.cmd: kompose -f docker-compose.yml convert
kompose.version: 1.26.1 (HEAD)
kompose.volume.type: hostPath
creationTimestamp: null
labels:
io.kompose.service: api
spec:
containers:
- image: task-management_api
name: api
imagePullPolicy: Never
ports:
- containerPort: 3000
resources: {}
volumeMounts:
- mountPath: /usr/src/app
name: api-hostpath0
- mountPath: /usr/src/app/node_modules
name: api-hostpath1
restartPolicy: Always
volumes:
- hostPath:
path: /Users/handrei/workspace/devs/nest-ws/task-management
name: api-hostpath0
- hostPath:
name: api-hostpath1
status: {}
the error I received from the pod is the next one:
kubectl logs api-84b56776c5-v86c7
yarn run v1.22.17
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Couldn't find a package.json file in "/usr/src/app"
I assume that's something wrong with volumes because applying the deployment and service without volumes it's working

A hostPath volume mounts a file or directory from the host node's filesystem into your Pod.
To the required path property, you can also specify a type for a hostPath volume.
NOTE: HostPath volumes present many security risks, and it is a best practice to avoid the use of HostPaths when possible. When a HostPath volume must be used, it should be scoped to only the required file or directory, and mounted as ReadOnly.
As #David Maze mentioned before, It's better idea to
use Node locally for day-to-day development and use a self-contained image (without any volume mounts at all) in Kubernetes. (...)
The node_modules directory is empty and nothing in Kubernetes will every copy data there. You'll need to delete all of the volume declarations from your Deployment spec for this to run.
This quide will help you to translate a Docker Compose File to Kubernetes Resources.
See also this questions on StackOverflow:
Why node_modules is empty after docker build?
Kubernetes volume for node_modules

Related

Why I cannot read files from a shared PersistentVolumeClaim between containers in Kubernetes?

I have a docker image felipeogutierrez/tpch-dbgen that I build using docker-compose and I push it to docker-hub registry using travis-CI.
version: "3.7"
services:
other-images: ....
tpch-dbgen:
build: ../docker/tpch-dbgen
image: felipeogutierrez/tpch-dbgen
volumes:
- tpch-dbgen-data:/opt/tpch-dbgen/data/
- datarate:/tmp/
stdin_open: true
and this is the Dockerfile to build this image:
FROM gcc AS builder
RUN mkdir -p /opt
COPY ./generate-tpch-dbgen.sh /opt/generate-tpch-dbgen.sh
WORKDIR /opt
RUN chmod +x generate-tpch-dbgen.sh && ./generate-tpch-dbgen.sh
In the end, this scripts creates a directory /opt/tpch-dbgen/data/ with some files that I would like to read from another docker image that I am running on Kubernetes. Then I have a Flink image that I create to run into Kubernetes. This image starts 3 Flink Task Managers and one stream application that reads files from the image tpch-dbgen-data. I think that the right approach is to create a PersistentVolumeClaim so I can share the directory /opt/tpch-dbgen/data/ from image felipeogutierrez/tpch-dbgen to my flink image in Kubernetes. So, first I have this file to create the PersistentVolumeClaim:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: tpch-dbgen-data-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 200Mi
Then, I am creating an initContainers to launch the image felipeogutierrez/tpch-dbgen and after that launch my image felipeogutierrez/explore-flink:1.11.1-scala_2.12:
apiVersion: apps/v1
kind: Deployment
metadata:
name: flink-taskmanager
spec:
replicas: 3
selector:
matchLabels:
app: flink
component: taskmanager
template:
metadata:
labels:
app: flink
component: taskmanager
spec:
initContainers:
- name: tpch-dbgen
image: felipeogutierrez/tpch-dbgen
#imagePullPolicy: Always
env:
command: ["ls"]
# command: ['sh', '-c', 'for i in 1 2 3; do echo "job-1 `date`" && sleep 5s; done;', 'ls']
volumeMounts:
- name: tpch-dbgen-data
mountPath: /opt/tpch-dbgen/data
containers:
- name: taskmanager
image: felipeogutierrez/explore-flink:1.11.1-scala_2.12
#imagePullPolicy: Always
env:
args: ["taskmanager"]
ports:
- containerPort: 6122
name: rpc
- containerPort: 6125
name: query-state
livenessProbe:
tcpSocket:
port: 6122
initialDelaySeconds: 30
periodSeconds: 60
volumeMounts:
- name: flink-config-volume
mountPath: /opt/flink/conf/
- name: tpch-dbgen-data
mountPath: /opt/tpch-dbgen/data
securityContext:
runAsUser: 9999 # refers to user _flink_ from official flink image, change if necessary
volumes:
- name: flink-config-volume
configMap:
name: flink-config
items:
- key: flink-conf.yaml
path: flink-conf.yaml
- key: log4j-console.properties
path: log4j-console.properties
- name: tpch-dbgen-data
persistentVolumeClaim:
claimName: tpch-dbgen-data-pvc
The Flink stream application is starting but it cannot read the files on the directory /opt/tpch-dbgen/data of the image felipeogutierrez/tpch-dbgen. I am getting the error: java.io.FileNotFoundException: /opt/tpch-dbgen/data/orders.tbl (No such file or directory). It is strange because when I try to go into the container felipeogutierrez/tpch-dbgen I can list the files. So I suppose there is something wrong on my Kubernetes configuration. Does anyone know to point what I am missing on the Kubernetes configuration files?
$ docker run -i -t felipeogutierrez/tpch-dbgen /bin/bash
root#10c0944a95f8:/opt# pwd
/opt
root#10c0944a95f8:/opt# ls tpch-dbgen/data/
customer.tbl dbgen dists.dss lineitem.tbl nation.tbl orders.tbl part.tbl partsupp.tbl region.tbl supplier.tbl
Also, when I list the logs of the container tpch-dbgen I can see the directory tpch-dbgen that I want to read. Although I cannot execute the command command: ["ls tpch-dbgen"] inside my Kubernetes config file.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
flink-jobmanager-n9nws 1/1 Running 2 17m
flink-taskmanager-777cb5bf77-ncdl4 1/1 Running 0 4m54s
flink-taskmanager-777cb5bf77-npmrx 1/1 Running 0 4m54s
flink-taskmanager-777cb5bf77-zc2nw 1/1 Running 0 4m54s
$ kubectl logs flink-taskmanager-777cb5bf77-ncdl4 tpch-dbgen
generate-tpch-dbgen.sh
tpch-dbgen
Docker has an unusual feature where, under some specific circumstances, it will populate a newly created volume from the image. You should not rely on this functionality, since it completely ignores updates in the underlying images and it doesn't work on Kubernetes.
In your Kubernetes setup, you create a new empty PersistentVolumeClaim, and then mount this over your actual data in both the init and main containers. As with all Unix mounts, this hides the data that was previously in that directory. Nothing causes data to get copied into that volume. This works the same way as every other kind of mount, except the Docker named-volume mount: you'll see the same behavior if you change your Compose setup to do a host bind mount, or if you play around with your local development system using a USB drive as a "volume".
You need to make your init container (or something else) explicitly copy data into the directory. For example:
initContainers:
- name: tpch-dbgen
image: felipeogutierrez/tpch-dbgen
command:
- /bin/cp
- -a
- /opt/tpch-dbgen/data
- /data
volumeMounts:
- name: tpch-dbgen-data
mountPath: /data # NOT the same path as in the image
If the main process modifies these files in place, you can make the command be more intelligent, or write a script into your image that only copies the individual files in if they don't exist yet.
It could potentially make more sense to have your image generate the data files at startup time, rather than at image-build time. That could look like:
FROM gcc
COPY ./generate-tpch-dbgen.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/generate-tpch-dbgen.sh
CMD ["generate-tpch-dbgen.sh"]
Then in your init container, you can run the default command (the generate script) with the working directory set to the volume directory
initContainers:
- name: tpch-dbgen
image: felipeogutierrez/tpch-dbgen
volumeMounts:
- name: tpch-dbgen-data
mountPath: /opt/tpch-dbgen/data # or anywhere really
workingDir: /opt/tpch-dbgen/data # matching mountPath
I got to run the PersistentVolumeClaim and share it between pods. Basically I had to use a subPath property which I learned from this answer https://stackoverflow.com/a/43404857/2096986 and I am using a simple Job that I learned from this answer https://stackoverflow.com/a/64023672/2096986. The final results is below:
The Dockerfile:
FROM gcc AS builder
RUN mkdir -p /opt
COPY ./generate-tpch-dbgen.sh /opt/generate-tpch-dbgen.sh
WORKDIR /opt
RUN chmod +x /opt/generate-tpch-dbgen.sh
ENTRYPOINT ["/bin/sh","/opt/generate-tpch-dbgen.sh"]
and the script generate-tpch-dbgen.sh has to have this line in the end sleep infinity & wait to not finalize. The PersistentVolumeClaim is the same of the question. Then I create a Job with the subPath property.
apiVersion: batch/v1
kind: Job
metadata:
name: tpch-dbgen-job
spec:
template:
metadata:
labels:
app: flink
component: tpch-dbgen
spec:
restartPolicy: OnFailure
volumes:
- name: tpch-dbgen-data
persistentVolumeClaim:
claimName: tpch-dbgen-data-pvc
containers:
- name: tpch-dbgen
image: felipeogutierrez/tpch-dbgen
imagePullPolicy: Always
volumeMounts:
- mountPath: /opt/tpch-dbgen/data
name: tpch-dbgen-data
subPath: data
and I use it on the other deployment also with the subPath property.
apiVersion: apps/v1
kind: Deployment
metadata:
name: flink-taskmanager
spec:
replicas: 3
selector:
matchLabels:
app: flink
component: taskmanager
template:
metadata:
labels:
app: flink
component: taskmanager
spec:
volumes:
- name: flink-config-volume
configMap:
name: flink-config
items:
- key: flink-conf.yaml
path: flink-conf.yaml
- key: log4j-console.properties
path: log4j-console.properties
- name: tpch-dbgen-data
persistentVolumeClaim:
claimName: tpch-dbgen-data-pvc
containers:
- name: taskmanager
image: felipeogutierrez/explore-flink:1.11.1-scala_2.12
imagePullPolicy: Always
env:
args: ["taskmanager"]
ports:
- containerPort: 6122
name: rpc
- containerPort: 6125
name: query-state
livenessProbe:
tcpSocket:
port: 6122
initialDelaySeconds: 30
periodSeconds: 60
volumeMounts:
- name: flink-config-volume
mountPath: /opt/flink/conf/
- name: tpch-dbgen-data
mountPath: /opt/tpch-dbgen/data
subPath: data
securityContext:
runAsUser: 9999 # refers to user _flink_ from official flink image, change if necessary
Maybe the issue is the accessMode you set on your PVC. ReadWriteOnce means it can only be mounted by one POD.
See here for Details.
You could try to use ReadWriteMany.
Your generate-tpch-dbgen.sh script is executed while building the docker image resulting those files in /opt/tpch-dbgen/data directory. So, when you run the image, you can see those files.
But the problem with k8s pvc, when you mount the volume (initially empty) to your containers, it replaces the /opt/tpch-dbgen/data directory along with the files in it.
Solution:
Don't execute the generate-tpch-dbgen.sh while building the docker image, rather execute it in the runtime. Then, the files will be created in the shared pv from the init container.
Something like below:
FROM gcc AS builder
RUN mkdir -p /opt
COPY ./generate-tpch-dbgen.sh /opt/generate-tpch-dbgen.sh
RUN chmod +x /opt/generate-tpch-dbgen.sh
ENTRYPOINT ["/bin/sh","/opt/generate-tpch-dbgen.sh"]

How to mount volume from my container in kubenetes

I want to mount directory from SourceContaner to ServerContainer.
ServerContainer:
FROM php:7.2-apache
RUN a2enmod rewrite
# /var/www/html is apache document root.
SourceContaner:
FROM alpine:3.7
# Copy local code to the container image.
COPY ./my_src /var/www/html/my_src
VOLUME /var/www/html/my_src
And, yaml is below.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: test
spec:
...snip...
spec:
containers:
- name: server-container
image: "Server Container image"
ports:
...snip...
volumeMounts:
- name: src-volume
mountPath: /var/www/html/my_src
- name: src-container
image: "Source Container Image"
volumes:
- name: src-volume
hostPath:
path: /var/www/html/my_src
But Source Container "CrashLoopBackOff" occured.
and nothing log is output.
This is not a feature of Kubernetes. There is an old FlexVolume plugin that implements the same behavior as Docker, but it isn’t recommended. You can use an initContainer to copy from the data container into a volume like an emptyDir.

env vars in postgres deployment not working

First off, I'm pretty sure I know why this isn't working: I'm pulling the Docker postgres:11-alpine image, modifying it, but then trying to change the env: in the k8s deployment.yaml on a custom image. I think that is the issue.
Basically, I'm trying to accomplish this per the Docker postgres docs:
docker run --name some-postgres -e POSTGRES_PASSWORD='foo' POSTGRES_USER='bar'
This is what I have:
Dockerfile.dev
FROM postgres:11-alpine
EXPOSE 5432
COPY ./db/*.sql /docker-entrypoint-initdb.d/
postgres.yaml (secrets will be moved after I'm done playing with this)
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres-deployment
spec:
replicas: 1
selector:
matchLabels:
component: postgres
template:
metadata:
labels:
component: postgres
spec:
containers:
- name: postgres
image: testproject/postgres
ports:
- containerPort: 5432
env:
- name: POSTGRES_DB
value: "test_dev"
- name: POSTGRES_USER
value: "bar"
- name: POSTGRES_PASSWORD
value: "foo"
volumeMounts:
- name: postgres-storage
mountPath: /var/lib/postgresql/data
subPath: postgres
volumes:
- name: postgres-storage
persistentVolumeClaim:
claimName: postgres-storage
---
apiVersion: v1
kind: Service
metadata:
name: postgres-cluster-ip-service
spec:
type: ClusterIP
selector:
component: postgres
ports:
- port: 5432
targetPort: 5432
When I use Skaffold to spin the cluster up locally, however, the env: "don't take" as I can still access the DB using the defaults POSTGRES_USER=postgres and POSTGRES_PASSWORD=''.
I bet if I did image: postgres then the env: would work, but then I'm not sure how to do the equivalent of this that is in the Dockerfile:
COPY ./db/*.sql /docker-entrypoint-initdb.d/
Any suggestions?
Here is the skaffold.yaml if that is helpful too:
apiVersion: skaffold/v1beta15
kind: Config
build:
local:
push: false
artifacts:
- image: testproject/postgres
docker:
dockerfile: ./db/Dockerfile.dev
sync:
manual:
- src: "***/*.sql"
dest: .
- image: testproject/server
docker:
dockerfile: ./server/Dockerfile.dev
sync:
manual:
- src: "***/*.py"
dest: .
deploy:
kubectl:
manifests:
- k8s/ingress.yaml
- k8s/postgres.yaml
- k8s/server.yaml
The Docker postgres docs mention the following:
Warning: the Docker specific variables will only have an effect if you start the container with a data directory that is empty; any pre-existing database will be left untouched on container startup.
Are you sure that you're starting your deployment with an empty data directory? Could it be that PostgreSQL starts and allows you to login using the credentials that were specified in the environment variables during the first time your started it with that persistent volume?
If that's not it, have a look at the environment variables of the running pod. kubectl describe POD should tell you which environment variables are actually passed through to the pod. Maybe something in your Skaffold setup overwrites the environment variables? You could have a look in the pod by running env when execing into the pod. Also don't forget the logs, the PostgreSQL container should log which user account it creates during startup.

How to mount volumes on kubernetes using emptyDir

I am trying to create a deployment out of my kompose file, but whenever I try:
kompose convert -f docker-compose.yaml
I get the error:
Volume mount on the host "[file directory]" isn't supported - ignoring path on the host
I have tried a few different solutions to my issue, firstly trying to add hostPath to my kompose convert as well as using persistent volumes, however both do not work.
my kompose files looks like this:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
kompose.cmd: kompose convert -f docker-compose.yaml --volumes emptyDir
kompose.version: 1.7.0 (HEAD)
creationTimestamp: null
labels:
io.kompose.service: es01
name: es01
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
creationTimestamp: null
labels:
io.kompose.service: es01
spec:
containers:
- env:
- name: COMPOSE_CONVERT_WINDOWS_PATHS
value: "1"
- name: COMPOSE_PROJECT_NAME
value: elastic_search_container
- name: ES_JAVA_OPTS
value: -Xms7g -Xmx7g
- name: discovery.type
value: single-node
- name: node.name
value: es01
image: docker.elastic.co/elasticsearch/elasticsearch:7.2.1
name: es01
ports:
- containerPort: 9200
resources: {}
volumeMounts:
- mountPath: /usr/share/elasticsearch/data
name: es01-empty0
restartPolicy: Always
volumes:
- emptyDir: {}
name: es01-empty0
status: {}
I am using kompose version 1.7.0
My Docker Compose version:
version: '3'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.2.1
container_name: es01
environment:
- node.name=es01
- COMPOSE_PROJECT_NAME=elastic_search_container
- discovery.type=single-node
- "ES_JAVA_OPTS=-Xms7g -Xmx7g"
- COMPOSE_CONVERT_WINDOWS_PATHS=1
ulimits:
nproc: 3000
nofile: 65536
memlock: -1
volumes:
- /home/centos/Sprint0Demo/Servers/elasticsearch:/usr/share/elasticsearch/data
ports:
- 9200:9200
networks:
- kafka_demo
You need to take a look on warning you get:
Volume mount on the host "[file directory]" isn't supported - ignoring path on the host
It happens when volume in docker-compose.yaml is configured with direct path.
Example below:
version: '3'
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- "./storage1:/test1"
- "./storage2:/test2"
redis:
image: "redis:alpine"
volumes:
storage1:
storage2:
Persistent Volume Claim
Take a look on this link: Conversion matrix.
It describes how kompose converts Docker's volumes to Kubernetes ones.
Executing the conversion command without --volumes parameter:
$ kompose convert -f docker-compose.yml
With kompose 1.19 output will yield:
WARN Volume mount on the host "SOME_PATH" isn't supported - ignoring path on the host
WARN Volume mount on the host "SOME_PATH" isn't supported - ignoring path on the host
INFO Kubernetes file "web-service.yaml" created
INFO Kubernetes file "redis-deployment.yaml" created
INFO Kubernetes file "web-deployment.yaml" created
INFO Kubernetes file "web-claim0-persistentvolumeclaim.yaml" created
INFO Kubernetes file "web-claim1-persistentvolumeclaim.yaml" created
Warning message means that you are explicitly telling docker-compose to create volumes with direct path. By default kompose will convert Docker's volume to Persistent Volume Claim.
emptyDir
Executing the conversion command with --volumes emptyDir parameter:
$ kompose convert -f docker-compose.yml --volumes emptyDir
Will yield effect:
WARN Volume mount on the host "SOME_PATH" isn't supported - ignoring path on the host
WARN Volume mount on the host "SOME_PATH" isn't supported - ignoring path on the host
INFO Kubernetes file "web-service.yaml" created
INFO Kubernetes file "redis-deployment.yaml" created
INFO Kubernetes file "web-deployment.yaml" created
kompose will create emptyDir declaration inside web-deployment.yaml instead of creating separate definitions of PVC as it has in default.
hostPath
Executing the conversion command with --volumes hostPath parameter:
$ kompose convert -f docker-compose.yml --volumes hostPath
Will yield effect:
INFO Kubernetes file "web-service.yaml" created
INFO Kubernetes file "redis-deployment.yaml" created
INFO Kubernetes file "web-deployment.yaml" created
As you can see there is no warning about not supported path. There is no warning because it created hostPath explicitly using your own provided paths from docker-compose.yml.
Take a look on web-deployment.yaml volume section:
volumes:
- hostPath:
path: /LOCAL_PATH-/POD_PATH/storage1
name: web-hostpath0
- hostPath:
path: /LOCAL_PATH-/POD_PATH/storage2
name: web-hostpath1

How to mimic '--volumes-from' in Kubernetes

I'm looking for a pattern that allows to share volumes between two containers running on the same pod in Kubernetes.
My use case is:
I have a Ruby on Rails application running inside a docker container.
The docker image contains static assets in /app/<app-name>/public directory, and I need to access those assets from the nginx container running alongside in the same pod.
In 'vanilla' docker I would have used --volumes-from flag to share this directory:
docker run --name app -v /app/<app-dir>/public <app-image>
docker run --volumes-from app nginx
After reading this doc: https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/volumes.md
I tried this (only relevant entries presented):
spec:
containers:
- image: <app-image>
name: <app-name>
volumeMounts:
- mountPath: /app/<app-name>/public
name: assets
- image: nginx
name: nginx
volumeMounts:
- mountPath: /var/www/html
name: assets
readOnly: true
volumes:
- name: assets
hostPath:
path: /tmp/assets
But:
Even though /tmp/assets on the node exists, it's empty
/app/<app-name>/public inside the app container is also empty
As a workaround I'm gonna try to populate the shared directory when the application container is up (simply cp /app/<app-name>/public/* to shared directory), but I really dislike this idea.
Question: how to mimic --volumes-from in Kubernetes, or if there is no direct counterpart, how can I share files from one container to other running in the same pod ?
apiVersion: v1beta3
Client Version: version.Info{Major:"0", Minor:"17", GitVersion:"v0.17.0", GitCommit:"82f8bdac06ddfacf493a9ed0fedc85f5ea62ebd5", GitTreeState:"clean"}
Server Version: version.Info{Major:"0", Minor:"17", GitVersion:"v0.17.0", GitCommit:"82f8bdac06ddfacf493a9ed0fedc85f5ea62ebd5", GitTreeState:"clean"}
[update-2016-8] In latest Kubernetes release, you can use a very nice feature named init-container to replace the postStart part in my answer below, which will make sure the container order.
apiVersion: v1
kind: Pod
metadata:
name: javaweb-2
spec:
initContainers:
- name: war
image: resouer/sample:v2
command: ["cp", "/sample.war", "/app"]
volumeMounts:
- mountPath: /app
name: app-volume
containers:
- name: tomcat
image: resouer/mytomcat:7.0
command: ["sh","-c","/root/apache-tomcat-7.0.42-v2/bin/start.sh"]
volumeMounts:
- mountPath: /root/apache-tomcat-7.0.42-v2/webapps
name: app-volume
ports:
- containerPort: 8080
hostPort: 8001
volumes:
- name: app-volume
emptyDir: {}
NOTE: initContainer is still a beta feature so the work version of this yaml is actually like: http://kubernetes.io/docs/user-guide/production-pods/#handling-initialization, please notice the pod.beta.kubernetes.io/init-containers part.
---original answer begin---
Actually, you can. You need to use container life cycle handler to control what files/dirs you want to share with other containers. Like:
---
apiVersion: v1
kind: Pod
metadata:
name: server
spec:
restartPolicy: OnFailure
containers:
- image: resouer/sample:v2
name: war
lifecycle:
postStart:
exec:
command:
- "cp"
- "/sample.war"
- "/app"
volumeMounts:
- mountPath: /app
name: hostv1
- name: peer
image: busybox
command: ["tail", "-f", "/dev/null"]
volumeMounts:
- name: hostv2
mountPath: /app/sample.war
volumes:
- name: hostv1
hostPath:
path: /tmp
- name: hostv2
hostPath:
path: /tmp/sample.war
Please check my gist for more details:
https://gist.github.com/resouer/378bcdaef1d9601ed6aa
And of course you can use emptyDir. Thus, war container can share its /sample.war to peer container without mess peer's /app directory.
If we can tolerate /app been overridden, it will be much simpler:
---
apiVersion: v1
kind: Pod
metadata:
name: javaweb-2
spec:
restartPolicy: OnFailure
containers:
- image: resouer/sample:v2
name: war
lifecycle:
postStart:
exec:
command:
- "cp"
- "/sample.war"
- "/app"
volumeMounts:
- mountPath: /app
name: app-volume
- image: resouer/mytomcat:7.0
name: tomcat
command: ["sh","-c","/root/apache-tomcat-7.0.42-v2/bin/start.sh"]
volumeMounts:
- mountPath: /root/apache-tomcat-7.0.42-v2/webapps
name: app-volume
ports:
- containerPort: 8080
hostPort: 8001
volumes:
- name: app-volume
emptyDir: {}
The answer is - for now - you can't. Here's a couple of discussion threads from the Kubernetes issues:
https://github.com/GoogleCloudPlatform/kubernetes/issues/6120
https://github.com/GoogleCloudPlatform/kubernetes/issues/831
However, may I suggest that you have an alternate design that might work better?
If your assets are locked at the point of the container going live,
you could use something like gitRepo
volume which would copy it to an emptyDir at the point of going live, and would mean you wouldn't have to move the content around at
all, just download it directly to the shared directory.
If your assets are locked at the point of the container
being built, it's probably best to copy them in at that point, using
the Docker COPY command.
If you really want to stick with the way you're doing it, you would have to copy the content to the emptyDir volume, which is designed for exactly what you're looking for (minus the lack of having to copy it in).
NFS[1] volumes also could solve your problem, but may be overly complex.
Additionally, I'd recommend that these two services exist in different pods, so you can scale each separately. You can create a service endpoint to communicate between them if you need to.
[1] https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples/nfs/nfs-web-pod.yaml
Further update from the future:
There is now a FlexVol plugin for Docker volumes: https://github.com/dims/docker-flexvol
At the time of writing, FlexVol is still an alpha feature though, so caveat emptor.
Kubernetes has its own volume types and these are most used volume type:
emptyDir
secret
gitRepo
hostPath (similar to --volumes-from)
config Maps
persistent storage (storage disks provided by cloud platforms)
You can find more about kubernets volumes here -https://kubernetes.io/docs/concepts/storage/volumes/
an example of hostpath volume :
apiVersion: v1
kind: Pod
metadata:
name: test-pd
spec:
containers:
- image: k8s.gcr.io/test-webserver
name: test-container
volumeMounts:
- mountPath: /test-pd
name: test-volume
volumes:
- name: test-volume
hostPath:
# directory location on host
path: /data
# this field is optional
type: Directory
hostpath will mount host/node directory to container directory.Multiple containers inside a pod can use different or same volumes.You need to mention it in each container.
hostPath volumes are independent of pod lifecycle but it create tight coupling between node and pod , you should avoid using hostPath.
If you are using Docker v17.0.5 or greater you can use a multi-stage build to copy files from one of your containers to the other during build time. This is a great primer on the advanced features at https://medium.com/#tonistiigi/advanced-multi-stage-build-patterns-6f741b852fae
The way I used it to copy static assets from my backend container into Nginx proxy is
ARG API_BACKEND_CONTAINER="api:backend"
FROM $API_BACKEND_CONTAINER as source
FROM nginx:mainline-alpine
ARG NGINX_ROOT=/usr/share/nginx/html/
COPY --from=source /var/share/api/static/ ${NGINX_ROOT}
The great thing is that because the API_BACKEND_CONTAINER is a build arg I'm able to pass in the tag of the latest API build.

Resources