I have 2 init containers and 1 container and want to deploy into azure iot edge.
Requirement is these 2 init containers should run in order first before main container starts.
I was able to do this in kubernetes pod and also docker compose(using depends on)
Not sure how to do this in deployment manifest for azure iot edge.
Is init containers supported in azure iot-edge?
apiVersion: v1
kind: Pod
metadata:
name: <<sample>>
spec:
containers:
- env:
image: <<image_path>>
imagePullPolicy: IfNotPresent
name: <<image name>>
ports:
- containerPort: 443
hostPort: 443
name: https
protocol: TCP
resources: {}
volumeMounts:
- mountPath: "/shared/path"
name: data
subPath: v1/data
imagePullSecrets:
- name: <<imagePull_name>>
initContainers:
- image: <<image-init-path1>>
imagePullPolicy: IfNotPresent
name: <<image_name>>
volumeMounts:
- mountPath: "/shared/path"
name: data
subPath: v1/data
- image: <<image-init-path2>>
imagePullPolicy: IfNotPresent
name: <<image_name>>
volumeMounts:
- mountPath: "/shared/path"
name: data
subPath: v1/data
nodeName: test-name
volumes:
- name: data
persistentVolumeClaim:
claimName: pvc-data
While you can specify the startup order of a container on IoT Edge, the runtime will not wait for the module to be up or running before starting the next. Azure IoT Edge doesn't offer any functionality for your scenario.
A way around this would be to write the main container to check if the other two are done running. Possible ways to achieve this:
Send a Direct Method from the main container to the init containers, and have them respond with a certain code when done.
Send a message to the main container from the init containers when they finish the job.
The containers could communicate with each other over HTTP do achieve the same.
Related
I am using pushgateway to exposes metrics coming from short-lived batch jobs.
At the moment the pushgateway instance is launched on a baremetal machine, where I have a docker volume mounted to allow survival of metrics in case of a container restart (in conjunction with the --persistence.file parameter).
Here an extract of the docker-compose.yml file used to run the container:
pushgateway:
image: prom/pushgateway:v1.2.0
restart: unless-stopped
volumes:
- pushgw-data:/data
ports:
- "${PUSHGW_PORT:-9091}:9091"
command: --persistence.file="/data/metric.store"
I am moving to a (private) kubernetes cluster without persistent volumes, but equipped with an s3-compatible object storage.
From this issue on github it seems possible to target s3 for the checkpointing, but without further input I am not sure how to achieve this, and that's the best I could find by searching the Web for information.
Can anyone point me in the right direction?
So finally https://serverfault.com/questions/976764/kubernetes-run-aws-s3-sync-rsync-against-persistent-volume-on-demand pointed me in the right direction.
This is an extract of the deployment.yaml descriptor which works as expected:
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: {{K8S_NAMESPACE}}
name: {{K8S_DEPLOYMENT_NAME}}
spec:
selector:
matchLabels:
name: {{K8S_DEPLOYMENT_NAME}}
strategy:
type: Recreate
template:
metadata:
labels:
name: {{K8S_DEPLOYMENT_NAME}}
version: v1
spec:
containers:
- name: {{AWSCLI_NAME}}
image: {{IMAGE_AWSCLI}}
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: {{SECRET_NAME}}
key: accesskey
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: {{SECRET_NAME}}
key: secretkey
command: [ "/bin/bash",
"-c",
"aws --endpoint-url {{ENDPOINT_URL}} s3 sync s3://{{BUCKET}} /data; while true; do aws --endpoint-url {{ENDPOINT_URL}} s3 sync /data s3://{{BUCKET}}; sleep 60; done" ]
volumeMounts:
- name: pushgw-data
mountPath: /data
- name: {{PUSHGATEWAY_NAME}}
image: {{IMAGE_PUSHGATEWAY}}
command: [ '/bin/sh', '-c' ]
args: [ 'sleep 10; /bin/pushgateway --persistence.file=/data/metric.store' ]
ports:
- containerPort: 9091
volumeMounts:
- name: pushgw-data
mountPath: /data
volumes:
- name: pushgw-data
emptyDir: {}
- name: config-volume
configMap:
name: {{K8S_DEPLOYMENT_NAME}}
imagePullSecrets:
- name: harbor-bot
restartPolicy: Always
Note the override of entrypoint for the docker image of the pushgateway. In my case I have put 10 seconds delay to start, you might need to tune the delay to suits your needs. This delay is needed because the pushgateway container will boot faster than the sidecar (also due to the network exchange with s3, I suppose).
If the pushgateway starts when not metric store file is already present, it won't be used/considered. But it gets worse, when you first send data to the pushgateway, it will override the file. At that point, the "sync" from the sidecar container will also override the original "copy", so please pay attention and be sure you have a backup of the metrics file before experimenting with this delay value.
I have containerized microservice built with Java. This application uses the default /config-volume directory when it searches for property files.
Previously I manually deployed via Dockerfile, and now I'm looking to automate this process with Kubernetes.
The container image starts the microservice immediately so I need to add properties to the config-volume folder immediately. I accomplished this in Docker with this simple Dockerfile:
FROM ########.amazon.ecr.url.us-north-1.amazonaws.com/company/image-name:1.0.0
RUN mkdir /config-volume
COPY path/to/my.properties /config-volume
I'm trying to replicate this type of behavior in a kubernetes deployment.yaml but I have found no way to do it.
I've tried performing a kubectl cp command immediately after applying the deployment and it sometimes works, but it can result in a race condition which cause the microservice to fail at startup.
(I've redacted unnecessary parts)
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-service
spec:
replicas: 1
template:
spec:
containers:
- env:
image: ########.amazon.ecr.url.us-north-1.amazonaws.com/company/image-name:1.0.0
name: my-service
ports:
- containerPort: 8080
volumeMounts:
- mountPath: /config-volume
name: config-volume
volumes:
- name: config-volume
emptyDir: {}
status: {}
Is there a way to copy files into a volume inside the deployment.yaml?
You are trying to emulate a ConfigMap using volumes. Instead, put your configuration into a ConfigMap, and mount that to your deployments. The documentation is there:
https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/
Once you have your configuration as a ConfigMap, mount it using something like this:
...
containers:
- name: mycontainer
volumeMounts:
- name: config-volume
mountPath: /config-volume
volumes:
- name: config-volume
configMap:
name: nameOfConfigMap
I'm setting up a kubernetes deployment with an image that will execute docker commands (docker ps etc.).
My yaml looks as the following:
kind: Deployment
apiVersion: apps/v1
metadata:
name: discovery
namespace: kube-system
labels:
discovery-app: kubernetes-discovery
spec:
selector:
matchLabels:
discovery-app: kubernetes-discovery
strategy:
type: Recreate
template:
metadata:
labels:
discovery-app: kubernetes-discovery
spec:
containers:
- image: docker:dind
name: discover
ports:
- containerPort: 8080
name: my-awesome-port
imagePullSecrets:
- name: regcred3
volumes:
- name: some-volume
emptyDir: {}
serviceAccountName: kubernetes-discovery
Normally I will run a docker container as following:
docker run -v /var/run/docker.sock:/var/run/docker.sock docker:dind
Now, kubernetes yaml supports commands and args but for some reason does not support options.
What is the right thing to do?
Perhaps I should configure a volume, but then, is it volumeMount or just a volume?
I am new with kubernetes so it is important for me to do it the right way.
Thank you
You want to add the volume to the container.
spec:
containers:
- name: discover
image: docker:dind
volumeMounts:
- name: dockersock
mountPath: "/var/run/docker.sock"
volumes:
- name: dockersock
hostPath:
path: /var/run/docker.sock
It seems like a bad idea to interact directly with containers on any nodes in Kubernetes. The whole point of Kubernetes is to orchestrate. If you add containers outside of the Pod construct, then Kubernetes will not be aware the processes running on the nodes. This will affect resource allocation.
It also needs to be said that directly working with containers bypasses security.
I am new to Docker and Kubernetes.
Technologies used:
Dotnet Core 2.2
Asp.NET Core WebAPI 2.2
Docker for windows(Edge) with Kubernetes support enabled
Code
I am having two services hosted into two docker containers container1 and container2.
Below is my deploy.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: webapi-dockerkube
spec:
replicas: 1
template:
metadata:
labels:
app: webapi-dockerkube
spec:
containers:
- name: webapi-dockerkube
image: "webapidocker:latest"
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
livenessProbe:
httpGet:
path: /api/values
port: 80
readinessProbe:
httpGet:
path: /api/values
port: 80
- name: webapi-dockerkube2
image: "webapidocker2:latest"
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
livenessProbe:
httpGet:
path: /api/other/values
port: 80
readinessProbe:
httpGet:
path: /api/other/values
port: 80
When I am running command:
kubectl create -f .\deploy.yaml
I am getting status as CrashLoopBackOff.
But same is running fine when i have only one container configured.
When checking logs I am getting following error:
Error from server (BadRequest): a container name must be specified for pod webapi-dockerkube-8658586998-9f8mk, choose one of: [webapi-dockerkube webapi-dockerkube2]
You are running two containers in the same pod which bind both to port 80. This is not possible within the same pod.
Think of a pod like a 'server' and you can't have two processes bind to the same port.
Solution in your situation: Use different ports inside the pod or use separate pods. From your deployment there seems to be no shared resources like filesystem, so it would be easy to split the containers to separate pods.
Note that it will not suffice to change the pod definition if you want to have both containers running in the same pod with different ports. The application in the container must bind to a different port as well.
apiVersion: v1
kind: Pod
metadata:
name: two-containers
spec:
restartPolicy: Never
volumes:
- name: shared-data
emptyDir: {}
containers:
- name: nginx-container
image: nginx
volumeMounts:
- name: shared-data
mountPath: /usr/share/nginx/html
- name: debian-container
image: debian
volumeMounts:
- name: shared-data
mountPath: /pod-data
command: ["/bin/sh"]
args: ["-c", "echo Hello from the debian container > /pod-data/index.html"]
here sharing example for multi container you can use this template
Also you can check for logs of using
Kubectl logs
Check reason for crashloop back
I've dockerized a python project that requires the use of several CSVs (~2gb). In order to keep image size down I didn't include the CSVs in the build, instead opting to give the running container the data from a directory outside the container through a volume. Locally, when running through docker, I can just do
docker run -v ~/local/path/:/container/path my-image:latest
This works, but I'm not sure how to go about doing this in Kubernetes. I've been reading the documentation and am confused by the number of volume types, where the actual CSVs should be stored, etc.
Based on the information about the project that I've provided, is there an obvious solution?
If you'd like to replicate that exact behavior from Docker the most common way to do it is to use hostPath. Something like this:
apiVersion: v1
kind: Pod
metadata:
name: test-pd
spec:
containers:
- image: my-image:latest
name: my-container
volumeMounts:
- mountPath: /container/path
name: test-volume
volumes:
- name: test-volume
hostPath:
path: /usr/local/path
type: Directory
Here is a typical example of sharing between containers. You can keep your data in a separate container and code in a different container.
https://kubernetes.io/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-volume/
apiVersion: v1
kind: Pod
metadata:
name: two-containers
spec:
restartPolicy: Never
volumes:
- name: shared-data
emptyDir: {}
containers:
- name: nginx-container
image: nginx
volumeMounts:
- name: shared-data
mountPath: /usr/share/nginx/html
- name: debian-container
image: debian
volumeMounts:
- name: shared-data
mountPath: /pod-data
command: ["/bin/sh"]
args: ["-c", "echo Hello from the debian container > /pod-data/index.html"]
Hope it helps.