How to update Kubernetes secrets for all namespaces - docker

Question

I am using docker and kubernetes on Google Cloud Platform, with the Kubernetes Engine.
I have secrets configurated in a app.yaml file like so :
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: app
namespace: $CI_COMMIT_REF_SLUG
labels:
app: app
spec:
replicas: 1
template:
metadata:
labels:
app: app
spec:
containers:
- name: app
image: gcr.io/engagement-org/app:$CI_COMMIT_SHA
imagePullPolicy: Always
ports:
- containerPort: 9000
env:
- name: MAILJET_APIKEY_PUBLIC
valueFrom:
secretKeyRef:
name: mailjet
key: apikey_public
- name: MAILJET_APIKEY_PRIVATE
valueFrom:
secretKeyRef:
name: mailjet
key: apikey_private
Each time I push on a new branch, a new namespace is created through a deploy in my gitlab-ci file. Secrets are created like so :
- kubectl create secret generic mailjet --namespace=$CI_COMMIT_REF_SLUG --from-literal=apikey_public=$MAILJET_APIKEY_PUBLIC --from-literal=apikey_private=$MAILJET_APIKEY_PRIVATE || echo 'Secret already exist';
Now, I have updated my mailjet api keys and want to make the change to all namespaces.
I can edit the secret on each namespace by getting a shell on the pods and running kubectl edit secret mailjet --namespace=<namespace_name>
What I want is to send the new secret values to the new pods that will be created in the future. When I deploy a new one, it still uses the old values.
From what I understand, the gitlab-ci file uses the app.yaml file to replace the environment variables with values. But I don't understand where app.yaml finds the original values.
Thank you for your help.

Answer 1

In general, Kubernetes namespaces are designed to provide isolation for components running inside them. For this reason, the Kubernetes API is not really designed to perform update operations across namespaces, or make secrets usable across namespaces.
That being said, there are a few things to solve this issue.
1. Use a single namespace & Helm releases instead of separate namespaces
From the looks of it, you are using Gitlab CI to deploy individual branches to review environments (presumably using Gitlab's Review App feature?). The same outcome can be achieved by deploying all Review Apps into the same namespace, and using Helm to manage multiple deployments ("releases" in Helm-speak) of the same application within a single namespace.
Within the gitlab-ci.yml, creating a Helm release for a new branch might look similar to this:
script:
- helm upgrade --namespace default --install review-$CI_COMMIT_REF_SLUG ./path/to/chart
Of course, this requires that you have defined a Helm chart for your application (which, in essence is just a set of YAML templates with a set of default variables that can then be overridden for individual releases). Refer to the documentation (linked above) for more information on creating Helm charts.
2. Keep secrets in sync across namespaces
We have had a similar issue a while ago and resorted to writing a custom Kubernetes controller that keeps secrets in sync across namespaces. It's open source and you can find it on GitHub (use with caution, though). It is based on annotations and provides unidirectional propagation of changes from a single, authoritative parent secret:
apiVersion: v1
kind: Secret
metadata:
name: mailjet
namespace: some-kubernetes-namespace
annotations:
replicator.v1.mittwald.de/replicate-from: default/mailjet
With the secret replicator deployed in your cluster, using this annotation will propagate all changes made to the mailjet secret in the default namespace to all secrets in any namespaced annotated like show above.

Answer 2

Now there is a way to share or sync secret across namespaces and its by using the ClusterSecret operator:
https://github.com/zakkg3/ClusterSecret