Dockerfile
FROM ubuntu
MAINTAINER user#gmail.com
RUN apt-get update
RUN apt-get install -y openjdk-8-jdk
ADD build/libs/micro-service-gradle-0.0.1-SNAPSHOT.jar /var/local/
ENTRYPOINT exec java $JAVA_OPTS \
-jar /var/local/micro-service-gradle-0.0.1-SNAPSHOT.jar
EXPOSE 8080
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: micro-service-gradle
labels:
app: micro-service-gradle
spec:
replicas: 1
selector:
matchLabels:
app: micro-service-gradle
template:
metadata:
labels:
app: micro-service-gradle
spec:
containers:
- name: micro-service-gradle
image: micro-service-gradle:latest
ports:
- containerPort: 8080
Deploying spring boot application in Kubernetes . Pod is not getting created. When i check kubectl get pods. it says CrashLoopBackOff.
NAME READY STATUS RESTARTS AGE
micro-service-gradle-fc97c97b-8hwhg 0/1 CrashLoopBackOff 6 6m23s
I tried to check logs for the same container. Logs are empty
kubectl logs -p micro-service-gradle-fc97c97b-8hwhg
I created the container manually using docker run. There is no issues in image and containers works fine.
How to verify the logs for why the pods in crash status.
You need to use
kubectl describe pod micro-service-gradle-fc97c97b-8hwhg
to get the relevant logs. This should guide you to your problem.
I ran into a similar issue. When I run
kubectl describe pod <podname>
and read the events, Though the image was pulled, message outputted was 'restarting failed container'
The pod was crashing because it was not performing any task. To keep it running, I add a sleep command based on similar example in the docs
command: ['sh', '-c', 'echo The app is running! && sleep 3600']
This SO answer also mentions running an infinite could also solve the problem
https://stackoverflow.com/a/55610769/7128032
You deployment resource looks ok. As you are able to create the container manually using, problem is with the connection to the image repository. Setup the impage pull secret and you should be able to create the pod
I faced similar issue. Just verify if your container is able to run continuously. You have to run the process in foreground to keep container running.
The possible reasons of such error are:
the application inside your pod is not starting due to an error;
the image your pod is based on is not present in the registry, or the node where your pod has been scheduled cannot pull from the registry;
some parameters of the pod has not been configured correctly
You can view what is happening by checking the events:
kubectl get events
or checking pod status:
kubectl describe po mypod-390jo50wn3-sp40r
Full explanation here: https://pillsfromtheweb.blogspot.com/2020/05/troubleshooting-kubernetes.html
I had the same problem, but using this worked for me:
image: Image:latest
command: [ "sleep" ]
args: [ "infinity" ]
Related
I want to delete a specific file from a cronJob to the following container, the problem is that when I run exec I got error, how can I exec to distroless container (k8s v1.22.5) and delte the file from a cronJob, which option do we have?
this is the deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: distro
labels:
app: distro
spec:
replicas: 1
selector:
matchLabels:
app: distro
template:
metadata:
labels:
app: distro
spec:
containers:
- name: edistro
image: timberio/vector:0.21.X-distroless-libc
ports:
- containerPort: 80
what I tried is
kubectl exec -i -t -n apits aor-agent-zz -c tor "--" sh -c "clear; (bash || ash || sh)"
The error is:
error: Internal error occurred: error executing command in container: failed to exec in container: failed to start exec
I tried it out like following
kubectl debug -it distro-d49b456cf-t85cm --image=ubuntu --target=edistro --share-processes -n default
And got error:
Targeting container "edistro". If you don't see processes from this container it may be because the container runtime doesn't support this feature. Defaulting debug container name to debugger-fvfxs. error: ephemeral containers are disabled for this cluster (error from server: "the server could not find the requested resource").
As I guess (not sure) that our the container runtime doesnt support it which option do we have?
The answer below doesn't solve the issue, I need a way to access from outside the distroless pod and delete specific file there, how can I do this?
The point of using distro-less is to have a minimal amount of tools/software packaged in the image. This means the removal of unnecessary tools like shell from the image.
You may work around using, however it may depend on your objective:
kubectl debug -it <POD_TO_DEBUG> --image=<helper-image> --target=<CONTAINER_TO_DEBUG> --share-processes
Eg:
kubectl debug -it distro-less-pod --image=ubuntu --target=edistro --share-processes
Not a great option but it is the only option I can think of.
If you are able to enter the nodes where the pods are running and you have permissions to execute commands (most likely as root) in there, you can try nsenter or any other way to enter the container mount namespace directly.
We want to deploy using ArgoCD from our Jenkinsfile (which is slightly not how this is intended to be done but close enough), and after done some experiments want to try using the official container with the CLI, so we have added this snippet to our our pipeline kubernetes yaml:
- name: argocdcli
image: argoproj/argocli
command:
- argo
args:
- version
tty: true
Unfortunately the usual way to keep these containers alive is to invoke cat in the container, which isn't there, so it fails miserably. Actually the only command in there is the "argo" command which doesn't have a way to sleep infinitely. (We are going to report this upstream so it will be fixed, but while we wait for that....)
My question therefore is, is there a way to indicate to Kubernetes that we know that this pod cannot keep itself up on its own, and therefore not tear it down immediately?
Unfortunately it's not possible since as you stated, argo is the only command available on this image.
It can be confirmed here:
####################################################################################################
# argocli
####################################################################################################
FROM scratch as argocli
COPY --from=argo-build /go/src/github.com/argoproj/argo/dist/argo-linux-amd64 /bin/argo
ENTRYPOINT [ "argo" ]
As we can see on this output, running argo is all this container is doing:
$ kubectl run -i --tty --image argoproj/argocli argoproj1 --restart=Never
argo is the command line interface to Argo
Usage:
argo [flags]
argo [command]
...
You can optionally create you own image based on that and include sleep, so it'll be possible to keep it running as in this example:
apiVersion: v1
kind: Pod
metadata:
name: busybox
namespace: default
spec:
containers:
- name: busybox
image: busybox:1.28
command:
- sleep
- "3600"
imagePullPolicy: IfNotPresent
restartPolicy: Always
I have one docker image and I am using following command to run it.
docker run -it -p 1976:1976 --name demo demo.docker.cloud.com/demo/runtime:latest
I want to run the same in Kubernetes. This is my current yaml file.
apiVersion: v1
kind: Deployment
metadata:
name: demo-deployment
labels:
app: demo
spec:
replicas: 1
selector:
matchLabels:
app: demo
template:
metadata:
labels:
app: demo
spec:
containers:
- name: demo
image: demo.docker.cloud.com/demo/runtime:latest
ports:
- containerPort: 1976
imagePullPolicy: Never
This yaml file covers everything except flag "-it". I am not able to find its Kubernetes equivalent. Please help me out with this. Thanks
I assume you are trying to connect a shell to your running container. Following the guide at https://kubernetes.io/docs/tasks/debug-application-cluster/get-shell-running-container/ - You would need the following commands. To apply your above configuration:
Create the pod: kubectl apply -f ./demo-deployment.yaml
Verify the Container is running: kubectl get pod demo-deployment
Get a shell to the running Container: kubectl exec -it demo-deployment -- /bin/bash
Looking at the Container definition in the API reference, the equivalent options are stdin: true and tty: true.
(None of the applications I work on have ever needed this; the documentation for stdin: talks about "reads from stdin in the container" and the typical sort of server-type processes you'd run in a Deployment don't read from stdin at all.)
kubectl run is the close match to docker run based on the requested scenario.
Some examples from Kubernetes documentation and it's purpose :
kubectl run -i --tty busybox --image=busybox -- sh # Run pod as interactive shell
kubectl run nginx --image=nginx -n
mynamespace # Run pod nginx in a specific namespace
kubectl run nginx --image=nginx # Run pod nginx and write its spec into a file called pod.yaml
--dry-run=client -o yaml > pod.yaml
I have successfully built Docker images and ran them in a Docker swarm. When I attempt to build an image and run it with Docker Desktop's Kubernetes cluster:
docker build -t myimage -f myDockerFile .
(the above successfully creates an image in the docker local registry)
kubectl run myapp --image=myimage:latest
(as far as I understand, this is the same as using the kubectl create deployment command)
The above command successfully creates a deployment, but when it makes a pod, the pod status always shows:
NAME READY STATUS RESTARTS AGE
myapp-<a random alphanumeric string> 0/1 ImagePullBackoff 0 <age>
I am not sure why it is having trouble pulling the image - does it maybe not know where the docker local images are?
I just had the exact same problem. Boils down to the imagePullPolicy:
PC:~$ kubectl explain deployment.spec.template.spec.containers.imagePullPolicy
KIND: Deployment
VERSION: extensions/v1beta1
FIELD: imagePullPolicy <string>
DESCRIPTION:
Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always
if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated.
More info:
https://kubernetes.io/docs/concepts/containers/images#updating-images
Specifically, the part that says: Defaults to Always if :latest tag is specified.
That means, you created a local image, but, because you use the :latest it will try to find it in whatever remote repository you configured (by default docker hub) rather than using your local. Simply change your command to:
kubectl run myapp --image=myimage:latest --image-pull-policy Never
or
kubectl run myapp --image=myimage:latest --image-pull-policy IfNotPresent
I had this same ImagePullBack error while running a pod deployment with a YAML file, also on Docker Desktop.
For anyone else that finds this via Google (like I did), the imagePullPolicy that Lucas mentions above can also be set in the deployment yaml file. See the spec.templage.spec.containers.imagePullPolicy in the yaml snippet below (3 lines from the bottom).
I added that and my app deployed successfully into my local kube cluser, using the kubectl yaml deploy command: kubectl apply -f .\Deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app-deployment
labels:
app: web-app
spec:
replicas: 3
selector:
matchLabels:
app: web-app
template:
metadata:
labels:
app: web-app
spec:
containers:
- name: web-app
image: node-web-app:latest
imagePullPolicy: Never
ports:
- containerPort: 3000
You didn't specify where myimage:latest is hosted, but essentially ImagePullBackoff means that I cannot pull the image because either:
You don't have networking setup in your Docker VM that can get to your Docker registry (Docker Hub?)
myimage:latest doesn't exist in your registry or is misspelled.
myimage:latest requires credentials (you are pulling from a private registry). You can take a look at this to configure container credentials in a Pod.
I am really having trouble debugging this and can use some help. I am successfully staring a kubernetes service and deployment using a working docker image.
My service file:
apiVersion: v1
kind: Service
metadata:
name: auth-svc
labels:
app: auth_v1
spec:
type: NodePort
ports:
- port: 3000
nodePort: 30000
protocol: TCP
selector:
app: auth_v1
Deploy File:
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-deploy
labels:
app: auth_v1
spec:
revisionHistoryLimit: 5
minReadySeconds: 10
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
replicas: 3
selector:
matchLabels:
app: auth_v1
template:
metadata:
labels:
app: auth_v1
spec:
containers:
- name: auth-pod
image: index.docker.io/XXX/auth
command: [ "yarn", "start-staging" ]
imagePullPolicy: Always
ports:
- containerPort: 3000
imagePullSecrets:
- name: myregistrykey
kubectl get pods shows that the pods are up and running. I have tested jumping into the pod/conatiner with shell and tried running my application and it works. When I run kubectl describe auth-deploy I am seeing a container listed as auth-pod. However, I am not seeing any containers when I run docker ps or docker ps -a. Also, the logs for my pods show nothing. Is there something I am doing wrong?
For reference, here is my Dockerfile:
FROM node:8.11.2-alpine AS build
LABEL maintainer="info#XXX.com"
# Copy Root Dir & Set Working Dir
COPY . /src
WORKDIR /src
# Build & Start Our App
RUN apk update
RUN apk add --update alpine-sdk
RUN apk add --update python
RUN yarn install
RUN yarn build-staging
# Build Production Image Using Node Container
FROM node:8.11.2-alpine AS production
# Copy Build to Image
COPY --from=build /src/.next /src/.next/
COPY --from=build /src/production-server /src/production-server/
COPY --from=build /src/static /src/static/
COPY --from=build /src/package.json /src
WORKDIR /src
# Install Essential Pacakges & Start App
RUN apk update
RUN apk add --update alpine-sdk
RUN apk add --update python
RUN yarn install
# Expose Ports Needed
EXPOSE 3000
VOLUME [ "/src/log" ]
# Start App
CMD [ "yarn", "start-staging" ]
Is it possible that you are running docker ps on the K8s-master instead of where the pods are located?
You can find out where your pods are running by running the command below:
$ kubectl describe pod auth-deploy
It should return something similar to below (in my case it's a percona workload):
$ kubectl describe pod percona
Name: percona-b98f87dbd-svq64
Namespace: default
Node: ip-xxx-xx-x-xxx.us-west-2.compute.internal/xxx.xx.x.xxx
Get the IP, SSH into the node, and run docker ps locally from the node your container is located.
$ docker ps | grep percona
010f3d529c55 percona "docker-entrypoint.s…" 7 minutes ago Up 7 minutes k8s_percona_percona-b98f87dbd-svq64_default_4aa2fe83-861a-11e8-9d5f-061181005f56_0
616d70e010bc k8s.gcr.io/pause-amd64:3.1 "/pause" 8 minutes ago Up 7 minutes k8s_POD_percona-b98f87dbd-svq64_default_4aa2fe83-861a-11e8-9d5f-061181005f56_0
Another possibility is that you might be using different container runtime such as rkt, containerd, and lxd instead of docker.
Kubernetes pods are made of grouped containers and running on the dedicated node.
Kubernetes are managing directions where to create pods and
their lifecycle.
Kubernetes configuration consists of worker nodes and the master server.
The master server is able to connect to nodes, create containers,
and bond them into pods. The master node is designed to run only managing commands like kubectl, cluster state database etcd,
and others daemons required to keep cluster up and running.
docker ps
shows nothing in this case.
To get list of running pods:
kubectl get pods
You can then connect to pod already running on node:
kubectl attach -i <podname>
Back to your question.
If you are interested in how Kubernetes are working with containers including your application image and Kubernetes infrastructure,
you have to obtain node’s IP address first:
kubectl describe pod <podname> | grep ^Node:
or by:
kubectl get pods -o wide
Next connect to the node via ssh and then:
docker ps
You will see there are containers including the one you are looking for.