I have a simple Kubernetes job (based on the http://kubernetes.io/docs/user-guide/jobs/work-queue-2/ example) which uses a Docker image that I have placed as a public image on my dockerhub account. It all loks like this:
job.yaml:
apiVersion: batch/v1
kind: Job
metadata:
name: job-wq-2
spec:
parallelism: 2
template:
metadata:
name: job-wq-2
spec:
containers:
- name: c
image: jonalv/job-wq-2
restartPolicy: OnFailure
Now I want to try to instead use a private Docker registry which requires authentication as in:
docker login https://myregistry.com
But I can't find anything about how I add username and password to my job.yaml file. How is it done?
You need to use ImagePullSecrets.
Once you create a secret object, you can refer to it in your pod spec (the spec value that is the parent of containers:
apiVersion: batch/v1
kind: Job
metadata:
name: job-wq-2
spec:
parallelism: 2
template:
metadata:
name: job-wq-2
spec:
imagePullSecrets:
- name: myregistrykey
containers:
- name: c
image: jonalv/job-wq-2
restartPolicy: OnFailure
Ofcourse, you'll have to create the secret (as per the docs). This is what this will look like:
apiVersion: v1
kind: Secret
metadata:
name: myregistrykey
namespace: mynamespace
data:
.dockerconfigjson: UmVhbGx5IHJlYWxseSByZWVlZWVlZWVlZWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWxsbGxsbGxsbGxsbGxsbGxsbGxsbGxsbGxsbGxsbGx5eXl5eXl5eXl5eXl5eXl5eXl5eSBsbGxsbGxsbGxsbGxsbG9vb29vb29vb29vb29vb29vb29vb29vb29vb25ubm5ubm5ubm5ubm5ubm5ubm5ubm5ubmdnZ2dnZ2dnZ2dnZ2dnZ2dnZ2cgYXV0aCBrZXlzCg==
type: kubernetes.io/dockerconfigjson
The value of .dockerconfigjson is a base64 encoding of this file: .docker/config.json.
The key point: A job spec contains a pod spec. So whatever knowledge you gain about pod specs can be applied to jobs as well.
Related
I hope it's ok to ask for your advice.
The problem in a nutshell: my pipeline cannot pull private images from GHCR.IO into Okteto Kubernetes, but public images from the same private repo work.
I'm on Windows 10 and use WSL2-Ubuntu 20.04 LTS with kinD for development and tried minikube too.
I get an error in Okteto which says that the image pull is “unauthorized” -> “imagePullBackOff”.
Things I did:browsed Stack Overflow, RTFM, Okteto FAQ, download the Okteto kubeconfig, pulled my hair out and spent more hours than I would like to admit – still no success yet.
For whatever reason I cannot create a “kubectl secret” that works. When logged-in to ghcr.io via “docker login --username” I can pull private images locally.
No matter what I’ve tried I still get the error “unauthorized” when trying to pull a private image in Okteto.
My Setup with latest updates:
Windows 10 Pro
JetBrains Rider IDE
WSL2-Ubuntu 20.04 LTS
ASP.NET Core MVC app
.NET 6 SDK
Docker
kinD
minikube
Chocolatey
Homebrew
Setup kinD
kind create cluster --name my-name
kubectl create my-namespace
// create a secret to pull images from ghcr.io
kubectl create secret docker-registry my-secret -n my-namespace --docker-username="my-username" --docker-password="my-password" --docker-email="my-email" --docker-server="https://ghcr.io"
// patch local service account
kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "my-secret"}]}'
kubernetes.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: okteto-repo
namespace: my-namespace
spec:
replicas: 1
selector:
matchLabels:
app: okteto-repo
template:
metadata:
labels:
app: okteto-repo
spec:
containers:
- name: okteto-repo
image: ghcr.io/user/okteto-repo:latest
ports:
- containerPort: 80
imagePullSecrets:
- name: my-secret
---
apiVersion: v1
kind: Service
metadata:
name: okteto-repo
annotations:
dev.okteto.com/auto-ingress: "true"
spec:
type: ClusterIP
selector:
app: okteto-repo
ports:
- protocol: TCP
port: 8080
targetPort: 80
Do you have an idea why it doesn't work and what I could do?
Thanks a lot my dear friends, every input is highly appreciated!
Hope you guys have great holidays.
Cheers,
Michael
I was able to pull a private image by doing the following:
Create a personal token in GitHub with repo access.
Build and push the image to GitHub's Container registry (I used okteto build -t ghcr.io/rberrelleza/go-getting-started:0.0.1)
Download my kubeconfig credentials from Okteto Cloud by running okteto context update-kubeconfig.
Create a secret with my credentials: kubectl create secret docker-registry gh-regcred --docker-server=ghcr.io --docker-username=rberrelleza --docker-password=ghp_XXXXXX
Patched the default account to include the secret as an image pull secret: kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": "gh-regcred"}]}'
Updated the image name in the kubernetes manifest
Created the deployment (kubectl apply -f k8s.yaml)
These is what my kubernetes resources looks like, in case it helps:
# k8s.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-world
spec:
replicas: 1
selector:
matchLabels:
app: hello-world
template:
metadata:
labels:
app: hello-world
spec:
containers:
- image: ghcr.io/rberrelleza/go-getting-started:0.0.1
name: hello-world
---
apiVersion: v1
kind: Service
metadata:
name: hello-world
annotations:
dev.okteto.com/auto-ingress: "true"
spec:
type: ClusterIP
ports:
- name: "hello-world"
port: 8080
selector:
app: hello-world
# default SA
apiVersion: v1
imagePullSecrets:
- name: gh-regcred
- name: okteto-regcred
kind: ServiceAccount
metadata:
creationTimestamp: "2021-05-21T22:26:38Z"
name: default
namespace: rberrelleza
resourceVersion: "405042662"
uid: 2b6a6eef-2ce7-40d3-841a-c0a5497279f7
secrets:
- name: default-token-7tm42
I created customize Docker Image and stored in my local system Now I want use that Docker Image via kubectl .
Docker image:-
1:- docker build -t backend:v1 .
Then Kubernetes file:-
apiVersion: apps/v1beta1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: backend
namespace: web-console
spec:
replicas: 1
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: backend
spec:
containers:
- env:
- name: mail_auth_pass
- name: mail_auth_user
- name: mail_from
- name: mail_greeting
- name: mail_service
- name: mail_sign
- name: mongodb_url
value: mongodb://mongodb.mongodb.svc.cluster.local/console
- name: server_host
value: "0.0.0.0"
- name: server_port
value: "3000"
- name: server_sessionSecret
value: "1234"
image: backend
imagePullPolicy: Never
name: backend
resources: {}
restartPolicy: Always
status: {}```
Command to run kubectl:- kubectl create -f backend-deployment.yaml
**getting Error:-**
error: error validating "backend-deployment.yaml": error validating data: [ValidationError(Deployment.spec.template.spec.containers[0].env[9]): unknown field "image" in io.k8s.api.core.v1.EnvVar, ValidationError(Deployment.spec.template.spec.containers[0].env[9]): unknown field "imagePullPolicy" in io.k8s.api.core.v1.EnvVar]; if you choose to ignore these errors, turn validation off with --validate=false
Local Registry
Set the local registry first using this command
docker run -d -p 5000:5000 --restart=always --name registry registry:2
Image Tag
Given a Dockerfile, the image could be built and tagged this easy way:
docker build -t localhost:5000/my-image
Image Pull Policy
the field imagePullPolicy should then be changed to Never get the right image from the right repo.
given this sample pod template
apiVersion: v1
kind: Pod
metadata:
name: my-pod
labels:
app: my-app
spec:
containers:
- name: app
image: localhost:5000/my-image
imagePullPolicy: Never
Deploy Pod
The pod can be deployed using:
kubectl create -f pod.yml
Hope this comes in handy :)
As the error specifies unknown field "image" and unknown field "imagePullPolicy"
There is syntax error in your kubernetes deployment file.
Make these changes in your yaml file.
apiVersion: apps/v1beta1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: backend
namespace: web-console
spec:
replicas: 1
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: backend
spec:
containers:
- name: backend
image: backend
imagePullPolicy: Never
env:
- name: mail_auth_pass
- name: mail_auth_user
- name: mail_from
- name: mail_greeting
- name: mail_service
- name: mail_sign
- name: mongodb_url
value: mongodb://mongodb.mongodb.svc.cluster.local/console
- name: server_host
value: "0.0.0.0"
- name: server_port
value: "3000"
- name: server_sessionSecret
value: "1234"
resources: {}
restartPolicy: Always
status: {}
Validate your kubernetes yaml file online using https://kubeyaml.com/
Or with kubectl apply --validate=true --dry-run=true -f deployment.yaml
Hope this helps.
To use a docker container from a private docker repo, kubernetes recommends creating a secret of type 'docker-registry' and referencing it in your deployment.
https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
kubectl create secret docker-registry regcred --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>
Then in your helm chart or kubernetes deployment file, use imagePullSecrets
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: foo
spec:
replicas: {{ .Values.replicaCount }}
template:
spec:
imagePullSecrets:
- name: regcred
containers:
- name: foo
image: foo.example.com
This works, but requires that all containers be sourced from the same registry.
How would you pull 2 containers from 2 registries (e.g. when using a sidecar that is stored separate from the primary container) ?
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: foo
spec:
replicas: {{ .Values.replicaCount }}
template:
spec:
containers:
- name: foo
image: foo.example.com
imagePullSecrets:
- name: foo-secret
- name: bar
image: bar.example.com
imagePullSecrets:
- name: bar-secret
I've tried creating 2 secrets foo-secret and bar-secret and referencing each appropriately, but I find it fails to pull both containers.
You have to include imagePullSecrets: directly at the pod level, but you can have multiple secrets there.
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: foo
spec:
replicas: {{ .Values.replicaCount }}
template:
spec:
imagePullSecrets:
- name: foo-secret
- name: bar-secret
containers:
- name: foo
image: foo.example.com/foo-image
- name: bar
image: bar.example.com/bar-image
The Kubernetes documentation on this notes:
If you need access to multiple registries, you can create one secret for each registry. Kubelet will merge any imagePullSecrets into a single virtual .docker/config.json when pulling images for your Pods.
I have my controller.yaml that looks like this:
apiVersion: v1
kind: ReplicationController
metadata:
name: hmrcaction
labels:
name: hmrcaction
spec:
replicas: 1
selector:
name: hmrcaction
template:
metadata:
labels:
name: hmrcaction
version: 0.1.4
spec:
containers:
- name: hmrcaction
image: ccc-docker-docker-release.someartifactory.com/hmrcaction:0.1.4
ports:
- containerPort: 9000
imagePullSecrets:
- name: fff-artifactory
and service yaml that looks like this:
apiVersion: v1
kind: Service
metadata:
name: hmrcaction
labels:
name: hmrcaction
spec:
# if your cluster supports it, uncomment the following to automatically create
# an external load-balanced IP for the frontend service.
type: LoadBalancer
ports:
# the port that this service should serve on
- port: 9000
selector:
name: hmrcaction
and I have a kubernetes cluster, so I wanted to use this rc to upload my docker to the cluster and I did it like this:
kubectl create -f controller.yaml
but I get some weird status, when I run the command kubectl get pods I get:
NAME READY STATUS RESTARTS AGE
hmrcaction-k9bb6 0/1 ImagePullBackOff 0 40s
what is this?? before the status was ErrImagePull...
please help :)
thanks!
kubectl describe pods -l name=hmrcaction should give you more useful information.
I am using kubernetes to deploy a rails app to google container engine.
Follow the kubernetes secrets document: http://kubernetes.io/v1.1/docs/user-guide/secrets.html
I created a web controller file:
# web-controller.yml
apiVersion: v1
kind: ReplicationController
metadata:
labels:
name: web
name: web-controller
spec:
replicas: 2
selector:
name: web
template:
metadata:
labels:
name: web
spec:
containers:
- name: web
image: gcr.io/my-project-id/myapp:v1
ports:
- containerPort: 3000
name: http-server
env:
secret:
- secretName: mysecret
And created a secret file:
# secret.yml
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
RAILS_ENV: production
When I run:
kubectl create -f web-controller.yml
It showed:
error: could not read an encoded object from web-controller.yml: unable to load "web-controller.yml": json: cannot unmarshal object into Go value of type []v1.EnvVar
error: no objects passed to create
Maybe the yaml format is wrong in the web-controller.yml file. Then how to write?
secret.yml
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
stringData:
RAILS_ENV: production
stringData is the easymode version of what you're after, one thing though.
you'll see the cleartext original yaml used to create the secret in the annotation (and if you used the above method that means you'll have a human readable secret in your annotation, if you use the below method you'll have the base64'd secret in your annotation), unless you follow up with the erase annotation command like so:
kubectl apply -f secret.yml
kubectl annotate secret mysecret kubectl.kubernetes.io/last-applied-configuration-
(the - at the end is what says to erase it)
kubectl get secret mysecret -n=api -o yaml
(to confirm)
Alternatively you'd do
Bash# echo production | base64
cHJvZHVjdGlvbgo=
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
RAILS_ENV: cHJvZHVjdGlvbgo=
You need to base64 encode the value and your key must be a valid DNS label, that is, replace RAILS_ENV with, for example, rails-env. See also this end-to-end example I put together here for more details and concrete steps.
We do not currently support secrets exposed as env vars.
Lets Say we adding imagepull secrets in deployment now follow the steps,
kubectl create secret docker-registry secret-name --docker-server=<registry-server-url> --docker-username=<Username> --docker-password=<password> --docker-email=<your-email>
Now refer this in deployment yaml file,
apiVersion: v1
kind: Deployment
metadata:
name: test-deployment
spec:
containers:
- name: test-app
image: <Image-name-private>
imagePullSecrets:
- name: secret-name
OR
Lets say you have some api key for access the application.
kubectl create secret generic secret-name --from-literal api-key="<your_api-key"
Now refer this in deployment like this.
env:
- name: API_KEY
valueFrom:
secretKeyRef:
name: secret-name
key: api-key