I'm learning Kubernetes and I've just installed minikube on my mac.
I have a docker image that I'd like to deploy. I created a deployment yaml file which looks like this:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: sonarqube
spec:
template:
metadata:
labels:
app: sonarqube
spec:
containers:
- image: docker-sonarqube-developer:latest
args:
- -Dsonar.web.context=/
name: sonarqube
env:
- name: SONARQUBE_JDBC_USERNAME
value: sonarqube
- name: SONARQUBE_JDBC_PASSWORD
value: sonarqube
ports:
- containerPort: 9000
name: sonarqube
I am trying to deploy my docker image on minikube with the following command:
kubectl create -f deployment.yaml
But I'm getting an error and I'm not sure what's going on.
W0628 09:18:45.550812 64359 factory_object_mapping.go:423] Failed to download OpenAPI (the server could not find the requested resource), falling back to swagger
error: error validating "k8s/deployment.yaml": error validating data: the server could not find the requested resource; if you choose to ignore these errors, turn validation off with --validate=false
Minikube is running and I can access the dashboard.
❯ kubectl get nodes
NAME STATUS ROLES AGE VERSION
minikube Ready master 17h v1.15.0
The docker image is available locally
❯ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED
6fcfdad92d16 docker-sonarqube-developer "./bin/run.sh" 16 hours
Any idea what's wrong?
Thanks!
First Check the kubectl version
Check whether the Minor for both client and server version are same
$Kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.20.2",
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0",
If not,then follow the below steps-
$curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl
$chmod +x ./kubectl
$sudo mv ./kubectl /usr/local/bin/kubectl
Now check the version again
$kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2",
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0",
$kubectl create -f deployment.yaml
Related
Errors while setting up CI/CD environment in Ubuntu 18.04 in Parallel Desktop environment:
There is an issue connecting with proxy and running nginx image.
I am trying to setup kubernetes CI CD environment on Ubuntu, but I am getting few errors related to apiserver proxy and kubectl get pods command is failing with unable to connect message.
$sudo minikube start --memory 8000 --cpus 2 --kubernetes-version v1.11.10 --vm-driver none
Wait failed: waiting for k8s-app=kube-proxy: timed out waiting for the condition
$kubectl run nginx --image nginx --port 80
error: failed to discover supported resources: Get https://192.168.64.19:8443/apis/apps/v1?timeout=32s: net/http: TLS handshake timeout
Below are docker, kubectl & minikube version version used:
$ docker --version
Docker version 18.09.7, build 2d0083d
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:18:23Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
$ minikube version
minikube version: v1.5.0
commit: d1151d93385a70c5a03775e166e94067791fe2d9
Content of ~/.kube/config content
apiVersion: v1
clusters:
- cluster:
certificate-authority: /home/parallels/.minikube/ca.crt
server: https://192.168.64.19:8443
name: minikube
contexts:
- context:
cluster: minikube
user: minikube
name: minikube
current-context: ""
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /home/parallels/.minikube/client.crt
client-key: /home/parallels/.minikube/client.key
The following deployment file is working if I'm uploading it from my local machine.
kind: Deployment
apiVersion: apps/v1
metadata:
name: api
namespace: app
spec:
replicas: 2
selector:
matchLabels:
run: api
template:
metadata:
labels:
run: api
spec:
containers:
- name: api
image: gcr.io/myproject/api:1535462260754
ports:
- containerPort: 8080
readinessProbe:
httpGet:
path: /_ah/health
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
The same one is on remote Compute Engine machine which running Jenkins. On this machine, with ssh I'm also able to apply this config. Under the Jenkins shell execute it's always throws
error: unable to recognize "./dist/cluster/api.deployment.yaml": no matches for kind "Deployment" in version "apps/v1"
I tried to change apiVersion to apps/v1beta1 and to extensions/v1beta1 as well.
Don't know what to try else.
Update 1
kubectl version on Compute Engine:
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff0 88eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:17:28Z", GoVersion:"go1.10.3", Compiler:"gc", Pla tform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"9+", GitVersion:"v1.9.7-gke.5", GitCommit:"9b635efce81582e1da13b3 5a7aa539c0ccb32987", GitTreeState:"clean", BuildDate:"2018-08-02T23:42:40Z", GoVersion:"go1.9.3b4", Compiler:"gc ", Platform:"linux/amd64"}
Update 2
Run inside Jenkins job shown this.
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:17:28Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Error from server (Forbidden): <html><head><meta http-equiv='refresh' content='1;url=/securityRealm/commenceLogin?from=%2Fversion%3Ftimeout%3D32s'/><script>window.location.replace('/securityRealm/commenceLogin?from=%2Fversion%3Ftimeout%3D32s');</script></head><body style='background-color:white; color:white;'>
Authentication required
<!--
You are authenticated as: anonymous
Groups that you are in:
Permission you need to have (but didn't): hudson.model.Hudson.Read
... which is implied by: hudson.security.Permission.GenericRead
... which is implied by: hudson.model.Hudson.Administer
-->
</body></html>
Probably the kubectl version in your Jenkins server or agent is old. Try running kubectl version from the Jenkins job to check for mismatches.
Thanks to #csanchez I figured out that I was needed to get credentials under jenkins user. For that I just ran this command:
gcloud container clusters get-credentials cluster-1 --zone=my-cluster-zone --project myproject
I have stacked in this phase:
Have local docker insecure registry and some images in it, e.g. 192.168.1.161:5000/kafka:latest
Have kubernetes cloud cluster, for which I can access only via ~/.kube/config file, e,g. token.
Need to deploy below deployment, but kubernetes cannot pull images, error message:
Failed to pull image "192.168.1.161:5000/kafka:latest": rpc error:
code = Unknown desc = Error response from daemon: Get
https://192.168.1.161:5000/v2/: http: server gave HTTP response to
HTTPS client
apiVersion: v1
kind: Service
metadata:
name: kafka
labels:
app: kafka
spec:
type: NodePort
ports:
- name: port9094
port: 9094
targetPort: 9094
selector:
app: kafka
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: kafka
spec:
replicas: 1
template:
metadata:
labels:
app: kafka
spec:
hostname: kafka
containers:
- name: redis
image: 192.168.1.161:5000/kafka:latest
imagePullPolicy: Always
ports:
- name: port9094
containerPort: 9094
- envFrom:
- configMapRef:
name: env
imagePullSecrets:
- name: regsec
ON Kubernetes cluster I have created secret file "regsec" with this command:
kubectl create secret docker-registry regsec --docker-server=192.168.1.161 --docker-username=<name from config file> --docker-password=<token value from config file>
cat ~/.docker/config.json
{
"auths": {},
"HttpHeaders": {
"User-Agent": "Docker-Client/18.06.0-ce (linux)"
}
cat /etc/docker/daemon.json
{
"insecure-registries":["192.168.1.161:5000"]
}
kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:17:28Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-21T09:05:37Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
docker version
Client:
Version: 18.06.0-ce
API version: 1.38
Go version: go1.10.3
Git commit: 0ffa825
Built: Wed Jul 18 19:09:54 2018
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.06.0-ce
API version: 1.38 (minimum version 1.12)
Go version: go1.10.3
Git commit: 0ffa825
Built: Wed Jul 18 19:07:56 2018
OS/Arch: linux/amd64
Experimental: false
You need to go to each of your nodes, edit the file /etc/default/docker.json and add the following in it:
{
"insecure-registries": ["192.168.1.161:5000"]
}
I used minikube for my Kubernetes cluster.
When I tried to apply a Pod with an image from my private docker registry (that is local, without authentication), the Pod didn't run and describe had a message indicating the repository wasn't reached (paraphrasing).
To fix this, I had to configure insecure-registry for the Docker daemon. According to the Docker docs, this can be done in two ways: as a flag passed to the dockerd command, or by modifying /etc/docker/daemon.json (on Linux).
However, as I used minikube to create and configure the cluster and daemon, I instead followed the minikube docs to set the flag --insecure-registry. The complete command is:
minikube start --insecure-registry "DOMAIN_DOCKER_REGISTRY:PORT_DOCKER_REGISTRY"
I have come to this thread over and over again trying to find the correct answer to get rid of certificates issues, without much success.
I finally solved the problem by installing the self signed certificate root on the system for all the kubernetes machines. That finally fixes the issue. On Ubuntu, you can import via:
sudo mv internal-ca.cert /usr/local/share/ca-certificates/internal-ca.crt
sudo update-ca-certificates
Keep in mind that if you have a certificate chain, it will require the root certificate, not the immediate certficate. You can check if the import worked by running:
openssl s_client -connect <YOUR REGISTRY HERE> -showcerts < /dev/null
You should see something like:
CONNECTED(00000005)
as the response.
I have an ubuntu installed on my laptop.
I started a private docker registry (ssl enabled + htpasswd secured) and added it on overlay network (so it can be accessed from other hosts/vms)
here is the code (docker-compose.yaml):
version: "3"
services:
registry:
restart: always
image: registry:2
ports:
- 5000:5000
environment:
REGISTRY_HTTP_TLS_CERTIFICATE: /certs/domain.crt
REGISTRY_HTTP_TLS_KEY: /certs/domain.key
REGISTRY_AUTH: htpasswd
REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
volumes:
- /certs:/certs
- ~/caas_rd/workspace/ci_cd_pipeline/registry_setup:/auth
networks:
- overlaynetwork
networks:
overlaynetwork:
so my registry is running in the following link (with dns, i can verify it in browser) : https://home-thinkpad-t420s:5000/v2/_catalog
Now I installed "Minikube" on my laptop. && ssh it by "minikube ssh".
I created a folder "/etc/docker/certs.d" on minikube vm & added certificates as per instructions :
https://docs.docker.com/engine/security/certificates/#understanding-the-configuration
I also modified /etc/hosts && appended ca.cert on /etc/ssl/certs/ca-certificates.crt.
and restarted docker service on minikube vm by : sudo systemctl restart docker.service
after this I am able to pull the images on minikube vm by "docker login & docker pull" & also by "curl with (cacert + username/password)"
above is working perfectly fine, means I can successfully access/pull private registry images inside minikube vm.
Then I tried to create a secret (on my laptop with kubectl create -f ) defined as below:
apiVersion: "v1"
kind: "Secret"
metadata:
name: "ssl-proxy-secret"
namespace: "default"
data:
proxycert: "LS0..."
proxykey: "LS0t..."
htpasswd: "YWRt..."
and created a pod (on my laptop with kubectl create -f ) defined as below:
apiVersion: v1
kind: Pod
metadata:
name: private-jenkins
spec:
containers:
- name: private-jenkins-container
image: home-thinkpad-t420s:5000/my-jenkins
volumeMounts:
- name: secrets
mountPath: /etc/secrets
volumes:
- name: secrets
secret:
secretName: ssl-proxy-secret
but when I try to run this pod, it throws error :
Failed to pull image "home-thinkpad-t420s:5000/my-jenkins": rpc error: code = 2 desc = Error: image my-jenkins not found
Error syncing pod, skipping: failed to "StartContainer" for "private-jenkins-container" with ErrImagePull: "rpc error: code = 2 desc = Error: image my-jenkins not found"
If I am able to pull images inside the minikube vm successfully by curl & docker login/pull......then when why pod creation is failing with above error ?
You need to create a separated kubernetes registry secret instead. Could use this command:
kubectl create secret docker-registry <secret-name> \
--docker-email=<your-email> --docker-username=<registry-user> \
--docker-password=<registry-password> --docker-server=<registry-server-domain>
After that you could update the pod configuration as follow:
apiVersion: v1
kind: Pod
metadata:
name: private-jenkins
spec:
containers:
- name: private-jenkins-container
image: home-thinkpad-t420s:5000/my-jenkins
imagePullSecrets:
- name: <secret-name>
Reference: link
Hope it helps!
Can you remove those double quotes escpecially from the credentials in the secret.yml file and try spinning the pod again ?
Hope doing good all.
Env: centos 7.3.1611, kubernetes : 1.5, docker 1.12
Problem 1 : Extended jboss docker not working but docker image created successfully
POD gets an error see below, step 7.
Problem 2 : Once problem #1 fixed then i wish to upload to docker hub: https://hub.docker.com/
how can i upload steps please if possible.
1) pull
docker pull jboss/wildfly
2) vi Dockerfile
FROM jboss/wildfly
RUN /opt/jboss/wildfly/bin/add-user.sh admin admin123$ --silent
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]
3) Extend docker image
docker build --tag=nbasetty/wildfly-server .
4) [root#centos7 custom-jboss]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nbasetty/wildfly-server latest c1fbb87faffd 43 minutes ago 583.8 MB
docker.io/httpd latest e0645af13ada 2 weeks ago 177.5 MB
5)vi jboss-wildfly-rc-service-custom.yaml
apiVersion: v1
kind: Service
metadata:
name: wildfly-service
spec:
externalIPs:
- 10.0.2.15
selector:
app: wildfly-rc-pod
ports:
- name: web
port: 8080
#- name: admin-console
# port: 9990
type: LoadBalancer
---
apiVersion: v1
kind: ReplicationController
metadata:
name: wildfly-rc
spec:
replicas: 2
template:
metadata:
labels:
app: wildfly-rc-pod
spec:
containers:
- name: wildfly
image: nbasetty/wildfly-server
ports:
- containerPort: 8080
#- containerPort: 9990
6) kubectl create -f jboss-wildfly-rc-service-custom.yaml
7) [root#centos7 jboss]# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-pvc-pod 1/1 Running 6 2d
wildfly-rc-d0k3h 0/1 ImagePullBackOff 0 23m
wildfly-rc-hgsfj 0/1 ImagePullBackOff 0 23m
[root#centos7 jboss]# kubectl logs wildfly-rc-d0k3h
Error from server (BadRequest): container "wildfly" in pod
"wildfly-rc-d0k3h" is waiting to start:
trying and failing to pull image
Glad you have found a way to make it working. here is step I followed.
I labeled node-01 as 'dbserver: mysql'
create the docker image in node-01
created this pod, it worked.
apiVersion: v1 kind: ReplicationController metadata: name: wildfly-rc spec: replicas: 2 template:
metadata:
labels:
app: wildfly-rc-pod
spec:
containers:
- name: wildfly
image: nbasetty/wildfly-server
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
nodeSelector:
dbserver: mysql
Re-creating the issue:
docker pull jboss/wildfly
mkdir jw
cd jw
echo 'FROM jboss/wildfly
RUN /opt/jboss/wildfly/bin/add-user.sh admin admin123$ --silent
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]' | tee Dockerfile
docker build --tag=docker.io/surajd/wildfly-server .
See the images available:
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/surajd/wildfly-server latest 10e96902ea12 11 seconds ago 583.8 MB
Create a config that works:
echo '
apiVersion: v1
kind: Service
metadata:
name: wildfly
spec:
selector:
app: wildfly
ports:
- name: web
port: 8080
type: LoadBalancer
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: wildfly
spec:
replicas: 2
template:
metadata:
labels:
app: wildfly
spec:
containers:
- name: wildfly
image: docker.io/surajd/wildfly-server
imagePullPolicy: Never
ports:
- containerPort: 8080
' | tee config.yaml
kubectl create -f config.yaml
Notice the field imagePullPolicy: Never, this helps you use the image available on the node(the image we built using docker build). This works on single node cluster but may or may not work on multiple node cluster. So not recommended to put that value, but since we are doing experiment on single node cluster we can set it to Never. Always set it to imagePullPolicy: Always. So that whenever the pod is scheduled the image will be pulled from registry. Read about imagePullPolicy and some config related tips.
Now to pull the image from registry the image should be on registry, so to answer your question of pushing it to docker hub run command:
docker push docker.io/surajd/wildfly-server
So in the above example replace surajd with your docker registry username.
Here are steps I used to do setup of single node cluster on CentOS:
My machine version:
$ cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core)
Here is what I have done:
Setup single node k8s cluster on CentOS as follows (src1 & src2):
yum update -y
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
setenforce 0
yum install -y docker kubelet kubeadm kubectl kubernetes-cni
systemctl enable docker && systemctl start docker
systemctl enable kubelet && systemctl start kubelet
sysctl net.bridge.bridge-nf-call-iptables=1
sysctl net.bridge.bridge-nf-call-ip6tables=1
kubeadm init
cp /etc/kubernetes/admin.conf $HOME/
chown $(id -u):$(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf
kubectl taint nodes --all node-role.kubernetes.io/master-
Now k8s version:
# kubectl version
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae", GitTreeState:"clean", BuildDate:"2017-05-19T18:44:27Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae", GitTreeState:"clean", BuildDate:"2017-05-19T18:33:17Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}