I have run into a Kubernetes related issue. I just moved from a Pod configuration to a ReplicationController for a Ruby on Rails app and I'm using persistent disks for the Rails pod. When I try apply the ReplicationController it gives the following error:
The ReplicationController "cartelhouse-ror" is invalid.
spec.template.spec.volumes[0].gcePersistentDisk.readOnly: Invalid
value: false: must be true for replicated pods > 1; GCE PD can only be
mounted on multiple machines if it is read-only
Does this mean there is no way to use persistent disks (R/W) when using ReplicationControllers or is there another way?
If not, how can I scale and/or apply rolling updates to the Pod configuration?
Pod configuration:
apiVersion: v1
kind: Pod
metadata:
name: appname
labels:
name: appname
spec:
containers:
- image: gcr.io/proj/appname:tag
name: appname
env:
- name: POSTGRES_PASSWORD
# Change this - must match postgres.yaml password.
value: pazzzzwd
- name: POSTGRES_USER
value: rails
ports:
- containerPort: 80
name: appname
volumeMounts:
# Name must match the volume name below.
- name: appname-disk-per-sto
# Mount path within the container.
mountPath: /var/www/html
volumes:
- name: appname-disk-per-sto
gcePersistentDisk:
# This GCE persistent disk must already exist.
pdName: appname-disk-per-sto
fsType: ext4
ReplicationController configuration:
apiVersion: v1
kind: ReplicationController
metadata:
labels:
name: appname
name: appname
spec:
replicas: 2
selector:
name: appname
template:
metadata:
labels:
name: appname
spec:
containers:
- image: gcr.io/proj/app:tag
name: appname
env:
- name: POSTGRES_PASSWORD
# Change this - must match postgres.yaml password.
value: pazzzzwd
- name: POSTGRES_USER
value: rails
ports:
- containerPort: 80
name: appname
volumeMounts:
# Name must match the volume name below.
- name: appname-disk-per-sto
# Mount path within the container.
mountPath: /var/www/html
volumes:
- name: appname-disk-per-sto
gcePersistentDisk:
# This GCE persistent disk must already exist.
pdName: appname-disk-per-sto
fsType: ext4
You can't achieve this with current Kubernetes - see Independent storage for replicated pods. This will be covered by the implementation of PetSets due in v1.3.
The problem is not with Kubernetes, but with shared block device and filesystem that can not be mounted at the same time to more than one host.
https://unix.stackexchange.com/questions/68790/can-the-same-ext4-disk-be-mounted-from-two-hosts-one-readonly
You can try to use Claims: http://kubernetes.io/docs/user-guide/persistent-volumes/
Or another filesystem, e.g. nfs: http://kubernetes.io/docs/user-guide/volumes/
Related
I have the postgres container running in a Pod on GKE and a PersistentVolume set up to store the data. However, all of the data in the database is lost if the cluster reboots or if the Pod is deleted.
If I run kubectl delete <postgres_pod> to delete the existing Pod and check the newly created (by kubernetes) Pod to replace the deleted one, the respective database has not the data that it had before the Pod being deleted.
Here are the yaml files I used to deploy postgres.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: custom-storage
parameters:
type: pd-standard
provisioner: kubernetes.io/gce-pd
reclaimPolicy: Retain
volumeBindingMode: Immediate
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: postgres-volume-claim
spec:
storageClassName: custom-storage
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
spec:
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:11.5
resources: {}
ports:
- containerPort: 5432
env:
- name: POSTGRES_DB
value: "dbname"
- name: POSTGRES_USER
value: "user"
- name: POSTGRES_PASSWORD
value: "password"
volumeMounts:
- mountPath: /var/lib/postgresql/
name: postgresdb
volumes:
- name: postgresdb
persistentVolumeClaim:
claimName: postgres-volume-claim
I double checked that the persistentVolumeReclaimPolicy has value Retain.
What am I missing?
Is the cluster creating a new volume each time you delete a pod? Check with kubectl get pv.
Is this a multi-zone cluster? Your storage class is not provisionig regional disks, so you might be getting a new disk when the pod moves from one zone to another.
Possibly related to your problem, the postgres container reference recommends mounting at /var/lib/postgresql/data/pgdata and setting the PGDATA env variable:
https://hub.docker.com/_/postgres#pgdata
I am trying to deploy a redis sentinel deployment in Kubernetes. I have accomplished that but want to use ConfigMaps to allow us to change the IP address of the master in the sentinel.conf file. I started this but redis cant write to the config file because the mount point for configMaps are readOnly.
I was hoping to run an init container and copy the redis conf to a different dir just in the pod. But the init container couldn't find the conf file.
What are my options? Init Container? Something other than ConfigMap?
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: redis-sentinel
spec:
replicas: 3
template:
metadata:
labels:
app: redis-sentinel
spec:
hostNetwork: true
containers:
- name: redis-sentinel
image: IP/redis-sentinel
ports:
- containerPort: 63790
- containerPort: 26379
volumeMounts:
- mountPath: /redis-master-data
name: data
- mountPath: /usr/local/etc/redis/conf
name: config
volumes:
- name: data
emptyDir: {}
- name: config
configMap:
name: sentinel-redis-config
items:
- key: redis-config-sentinel
path: sentinel.conf
According to #P Ekambaram proposal, you can try this one:
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: redis-sentinel
spec:
replicas: 3
template:
metadata:
labels:
app: redis-sentinel
spec:
hostNetwork: true
containers:
- name: redis-sentinel
image: redis:5.0.4
ports:
- containerPort: 63790
- containerPort: 26379
volumeMounts:
- mountPath: /redis-master-data
name: data
- mountPath: /usr/local/etc/redis/conf
name: config
initContainers:
- name: copy
image: redis:5.0.4
command: ["bash", "-c", "cp /redis-master/redis.conf /redis-master-data/"]
volumeMounts:
- mountPath: /redis-master
name: config
- mountPath: /redis-master-data
name: data
volumes:
- name: data
emptyDir: {}
- name: config
configMap:
name: example-redis-config
items:
- key: redis-config
path: redis.conf
In this example initContainer copy the file from ConfigMap into writable dir.
Note:
An emptyDir volume is first created when a Pod is assigned to a Node, and exists as long as that Pod is running on that node. As the name says, it is initially empty. Containers in the Pod can all read and write the same files in the emptyDir volume, though that volume can be mounted at the same or different paths in each Container. When a Pod is removed from a node for any reason, the data in the emptyDir is deleted forever.
Create a startup script. In that copy the configMap file that is mounted in a volume to writable location. Then run the container process.
I have the following deployment yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: iam-mysql
labels:
app: iam
tier: mysql
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: iam
tier: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: iam
tier: mysql
spec:
containers:
- image: mysql:5.6
name: iam-mysql
envFrom:
- configMapRef:
name: iam-mysql-conf-dev
- secretRef:
name: iam-mysql-pass-dev
ports:
- containerPort: 3306
name: iam-mysql
volumeMounts:
- name: iam-mysql-persistent-storage
mountPath: /var/lib/mysql
- name: mysql-initdb
mountPath: /docker-entrypoint-initdb.d
restartPolicy: Always
volumes:
- name: iam-mysql-persistent-storage
persistentVolumeClaim:
claimName: iam-mysql-pv-claim
- name: mysql-initdb
configMap:
name: iam-mysql-initdb-dev
I cannot reload "iam-mysql-initdb-dev" with the new schema once it was create. In fact, I deleted a table (user) inside of the pod and when I created the deployment again the table (user) wasn't there. That means kubernetes is not reload the schema once the deployment has benn recreated.
That's expected behavior. Init files under /docker-entrypoint-initdb.d/ directory, are ran only when the data directory is empty. That means only 1st time.
If you look into the entry-point script of MySQL 5.6, you can see this process.
In line 98, it checks if the data directory is empty.
If it is empty, The script will run the init files of /docker-entrypoint-initdb.d/ directory. See in line 190-192
If it is not empty, the whole entry point script will be ignored, From line 98 - 202
In kubernetes, when you are using a persistent volume, the volumes persists regarding deletion and recreation of pods. So, when pod restarts, the data directory is not empty. So, MySQL is skipping the init part, From line 98 - 202
I fiddle around with this example: https://cloud.google.com/container-engine/docs/tutorials/persistent-disk/
My modifications:
- I'm using my own Wordpress image [x] Works
Service starts (it needed more CPU 0.8 instead of 0.5, but now it works)
I want to use mariadb instead of mysql [ ] Fails!
I can't figure out how two pods link together!!!! ~5h + still failing
Here are my .yaml-Files
apiVersion: v1
kind: Pod
metadata:
name: wpsite
labels:
name: wpsite
spec:
containers:
- image: <my image on gcr.io>
name: wpsite
env:
- name: WORDPRESS_DB_PASSWORD
# Change this - must match mysql.yaml password.
value: example
ports:
- containerPort: 80
name: wpsite
volumeMounts:
# Name must match the volume name below.
- name: wpsite-disk
# Mount path within the container.
mountPath: /var/www/html
volumes:
- name: wpsite-disk
gcePersistentDisk:
# This GCE persistent disk must already exist.
pdName: wpsite-disk
fsType: ext4
service:
apiVersion: v1
kind: Service
metadata:
labels:
name: wpsite
name: wpsite
spec:
type: LoadBalancer
ports:
# The port that this service should serve on.
- port: 80
targetPort: 80
protocol: TCP
# Label keys and values that must match in order to receive traffic for this service.
selector:
name: wpsite
mariadb:
apiVersion: v1
kind: Pod
metadata:
name: mariadb
labels:
name: mariadb
spec:
containers:
- resources:
limits:
# 0.5 hat nicht funktioniert
# Fehlermeldung in: kubectl describe pod mariadb
cpu: 0.8
image: mariadb:10.1
name: mariadb
env:
- name: MYSQL_ROOT_PASSWORD
# Change this password!
value: example
ports:
- containerPort: 3306
name: mariadb
volumeMounts:
# This name must match the volumes.name below.
- name: mariadb-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mariadb-persistent-storage
gcePersistentDisk:
# This disk must already exist.
pdName: mariadb-disk
fsType: ext4
maria-db-service:
apiVersion: v1
kind: Service
metadata:
labels:
name: mariadb
name: mariadb
spec:
ports:
# The port that this service should serve on.
- port: 3306
# Label keys and values that must match in
# order to receive traffic for this service.
selector:
name: mysql
kubectl logs wpsite shows error messages like this: Warning: mysqli::mysqli(): php_network_getaddresses: getaddrinfo failed: Name or service not known in - on line 10
OK - found it out!
It's the name in mariadb-service.yaml
metadata.name must be mysql and not mariadb, the selector in mariadb-service must point to mariadb (the pod)
Here are the working files:
mariadb.yaml
apiVersion: v1
kind: Pod
metadata:
name: mariadb
labels:
name: mariadb
spec:
containers:
- resources:
limits:
# 0.5 hat nicht funktioniert
# Fehlermeldung in: kubectl describe pod mariadb
cpu: 0.8
image: mariadb:10.1
name: mariadb
env:
- name: MYSQL_ROOT_PASSWORD
# Change this password!
value: example
ports:
- containerPort: 3306
name: mariadb
volumeMounts:
# This name must match the volumes.name below.
- name: mariadb-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mariadb-persistent-storage
gcePersistentDisk:
# This disk must already exist.
pdName: mariadb-disk
fsType: ext4
mariadb-service.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql
labels:
name: mysql
spec:
ports:
# The port that this service should serve on.
- port: 3306
# Label keys and values that must match in
# order to receive traffic for this service.
selector:
name: mariadb
wpsite.yaml
apiVersion: v1
kind: Pod
metadata:
name: wpsite
labels:
name: wpsite
spec:
containers:
- image: <change this to your imagename on gcr.io>
name: wpsite
env:
- name: WORDPRESS_DB_PASSWORD
# Change this - must match mysql.yaml password.
value: example
ports:
- containerPort: 80
name: wpsite
volumeMounts:
# Name must match the volume name below.
- name: wpsite-disk
# Mount path within the container.
mountPath: /var/www/html
volumes:
- name: wpsite-disk
gcePersistentDisk:
# This GCE persistent disk must already exist.
pdName: wpsite-disk
fsType: ext4
wpsite-service.yaml
apiVersion: v1
kind: Service
metadata:
name: wpsite
labels:
name: wpsite
spec:
type: LoadBalancer
ports:
# The port that this service should serve on.
- port: 80
targetPort: 80
protocol: TCP
# Label keys and values that must match in order to receive traffic for this service.
selector:
name: wpsite
With these settings I run: (my yaml-files are under gke)
$ kubectl create -f gke/mariadb.yaml
# Check
$ kubectl get pod
$ kubectl create -f gke/mariadb-service.yaml
# Check
$ kubectl get service mysql!!!! (name in mariadb = mysql)
$ kubectl create -f gke/wpsite.yaml
# Check
$ kubectl get pod
$ kubectl create -f gke/wpsite-service.yaml
# Check
$ kubectl describe service wpsite
Hope this helps someone...
Found this example for Kubernetes EmptyDir volume
apiVersion: v1
kind: Pod
metadata:
name: www
spec:
containers:
- name: nginx
image: nginx
volumeMounts:
- mountPath: /srv/www
name: www-data
readOnly: true
- name: git-monitor
image: kubernetes/git-monitor
env:
- name: GIT_REPO
value: http://github.com/some/repo.git
volumeMounts:
- mountPath: /data
name: www-data
volumes:
- name: www-data
emptyDir: {}
I want to volume mount between 2 pods. I am creating these pods using 2 different Replication Controllers. The replication controllers looks like this
Replication Controller 1:
apiVersion: v1
kind: ReplicationController
metadata:
name: node-worker
labels:
name: node-worker
spec:
replicas: 1
selector:
name: node-worker
template:
metadata:
labels:
name: node-worker
spec:
containers:
-
name: node-worker
image: image/node-worker
volumeMounts:
- mountPath: /mnt/test
name: deployment-volume
volumes:
- name: deployment-volume
emptyDir: {}
Replication Controller 2:
apiVersion: v1
kind: ReplicationController
metadata:
name: node-manager
labels:
name: node-manager
spec:
replicas: 1
selector:
name: node-manager
template:
metadata:
labels:
name: node-manager
spec:
containers:
-
name: node-manager
image: image/node-manager
volumeMounts:
- mountPath: /mnt/test
name: deployment-volume
volumes:
- name: deployment-volume
emptyDir: {}
Can Kubernetes emptyDir volume be used for this scenario?
EmptyDir volumes are inherently bound to the lifecycle of a single pod and can't be shared amongst pods in replication controllers or otherwise. If you want to share volumes amongst pods, the best choices right now are NFS or gluster, in a persistent volume. See an example here: https://github.com/kubernetes/examples/blob/master/staging/volumes/nfs/README.md
Why do you want to share the volume mount between pods? This will not work reliably because you aren't guaranteed to have a 1:1 mapping between where pods in replication controller 1 and replication controller 2 are scheduled in your cluster.
If you want to share local storage between containers, you should put both of the containers into the same pod, and have each container mount the emptyDir volume.
You require three things to get this working. More info can be found here and some documentation here, but it's a little confusing at first.
This example mounts a NFS volume.
1. Create a PersistentVolume pointing to your NFS server
file : mynfssharename-pv.yaml
(update server to point to your server)
apiVersion: v1
kind: PersistentVolume
metadata:
name: mynfssharename
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteMany
nfs:
server: yourservernotmine.yourcompany.com
path: "/yournfspath"
kubectl create -f mynfssharename-pv.yaml
2. Create a PersistentVolumeClaim to points to PersistentVolume mynfssharename
file : mynfssharename-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mynfssharename
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
kubectl create -f mynfssharename-pvc.yaml
3. Add the claim to your ReplicationController or Deployment
spec:
containers:
- name: sample-pipeline
image: yourimage
imagePullPolicy: Always
ports:
- containerPort: 8080
name: http
volumeMounts:
# name must match the volume name below
- name: mynfssharename
mountPath: "/mnt"
volumes:
- name: mynfssharename
persistentVolumeClaim:
claimName: mynfssharename