I have this docker-compose.yml file which I am using to run up three microservices and one api gateway
version: '3'
services:
serviceone:
container_name: serviceone
restart: always
build: serviceone/
ports:
- '3000:3000'
servicetwo:
container_name: servicetwo
restart: always
build: servicetwo/
ports:
- '3001:3001'
servicethree:
container_name: servicethree
restart: always
build: servicethree/
ports:
- '3002:3003'
apigateway:
container_name: timezoneapigateway
restart: always
build: timezone/
ports:
- '8080:8080'
links:
- serviceone
- servicetwo
- servicethree
Now I want to deploy these dockerimages in one pod in kubernetes so that the api gateway can connect with all the three microservices ,current version of api gateway is working but I am really not getting even a slightest hint of doing this in kubernetes. I am really new to kubernetes can anyone tell me how to design a kubernetes yml file to achieve this
You don't have to run all your service in the same pod. The standard in Kubernetes is to have separate deployments and services for all apps. Here is a deployment manifest for serviceone but you can easily modify it for servicetwo, servicethree and apigateway
apiVersion: apps/v1
kind: Deployment
metadata:
name: serviceone
labels:
app: serviceone
spec:
replicas: 1
selector:
matchLabels:
app: serviceone
template:
metadata:
labels:
app: serviceone
spec:
containers:
- name: serviceone
image: serviceone:latest
ports:
- containerPort: 3001
And the same goes for the service manifest
apiVersion: v1
kind: Service
metadata:
name: serviceone
spec:
selector:
app: serviceone
ports:
- protocol: TCP
port: 3001
targetPort: 3001
Your services will be accessible within the cluster like this:
serviceone:3001
servicetwo:3002
servicethree:3003
timezoneapigateway:8080
Related
I am learning Kubernetes and I am starting to convert an existing docker-compose.yml to Kuberbetes manifest one component at a time. Here is the component I am currently trying to convert
version: '3'
services:
mongodb:
container_name: mongodb
restart: always
hostname: mongo-db
image: mongo:latest
networks:
- mynetwork
volumes:
- c:/MongoDb:c:/data/db
ports:
- 27017:27017
networks:
mynetwork:
name: mynetwork
I am able to log into the Mongo instance when the container is running in Docker but I cannot get this working in Kubernetes. Here is the Kubernetes manifests I have tried so far
apiVersion: apps/v1
kind: Deployment
metadata:
name: mongodb
spec:
selector:
matchLabels:
app: mongodb
template:
metadata:
labels:
app: mongodb
spec:
containers:
- name: mongodb
image: mongo:latest
ports:
- containerPort: 27017
volumeMounts:
- name: mongodb-data
mountPath: /data/db
ports:
- containerPort: 27017
hostPort: 27017
volumes:
- name: mongodb-data
hostPath:
path: "C:\\MongoDb"
With this manifest I see the error Error: Error response from daemon: invalid mode: /data/db when I do a kubectl describe pods.
I understand the mapping of volumes from Docker to Kubernetes isn't 1-to-1, so is this reasonable to do in the Kubernetes space?
Thank you and apologies if this feels like a silly question.
docker-compose.yml
version: '3'
services:
zookeeper:
image: confluentinc/cp-zookeeper:6.0.0
hostname: zookeeper
container_name: zookeeper
restart: unless-stopped
ports:
- "2181:2181"
volumes:
- ./data/zoo/zk-data:/var/lib/zookeeper/data
- ./data/zoo/zk-txn-logs:/var/lib/zookeeper/log
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
broker:
image: confluentinc/cp-server:6.0.0
hostname: broker
container_name: broker
restart: unless-stopped
depends_on:
- zookeeper
volumes:
- ./data/kafka:/var/lib/kafka/data
ports:
- "9092:9092"
- "9101:9101"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_CONFLUENT_LICENSE_TOPIC_REPLICATION_FACTOR: 1
KAFKA_CONFLUENT_BALANCER_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_JMX_PORT: 9101
KAFKA_CONFLUENT_SCHEMA_REGISTRY_URL: http://schema-registry:8081
CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: broker:29092
CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
CONFLUENT_METRICS_ENABLE: 'true'
CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous'
schema-registry:
image: confluentinc/cp-schema-registry:6.0.0
hostname: schema-registry
container_name: schema-registry
restart: unless-stopped
depends_on:
- broker
ports:
- "8081:8081"
environment:
SCHEMA_REGISTRY_HOST_NAME: schema-registry
SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: 'broker:29092'
dev.yml
apiVersion: v1
kind: Service
metadata:
name: auth-server
labels:
app: auth-server
spec:
type: LoadBalancer
ports:
- port: 8882
protocol: TCP
targetPort: 8882
nodePort: 30080
selector:
app: auth-server
status:
loadBalancer:
ingress:
- hostname: localhost
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-server
spec:
selector:
matchLabels:
app: auth-server
replicas: 1
template:
metadata:
labels:
app: auth-server
spec:
containers:
- name: auth-server
image: auth-server:latest
imagePullPolicy: Never
ports:
- containerPort: 8080
volumeMounts:
- name: auth-server
mountPath: "/config"
volumes:
- name: auth-server
configMap:
name: auth-server
I need connect from my app on kubernetes cluser (docker) connect to docker-compose kafka.
I tried change kafka url in properties: http://localhost:9092, http://broker:9092 and
THIS tutorial but won't work too.
In docker ps I see: kafka and auth-server from kubernetes.
Can someone explain me. How can I do it ?
Since I don't know your flavor of local K8s, I can only speculate. But the problem here could be that Kubernetes and docker compose run their containers in different virtual networks.
What you have to do to create connectivity is:
Expose the Kafka port in your compose file with port mappings to the host:
ports:
- 9092:9092
Connect to it from Kubernetes via the IP address of your docker0 interface (typically 172.17.0.1).
I'm running a kubernetes cluster with minikube on ubuntu server 20.04.1 LTS. When I launch the deployments I can access the Tomcat server from the cluster, but I can't open my application, I got the error message :
HTTP Status 404 – Not Found
Here is my docker-compose file :
version: "3"
#NETWORK
networks:
network1:
driver: bridge
volumes:
dir-site:
driver_opts:
device: /smartclass_docker/site/
type: bind
o: bind
# Declare services
services:
# service name
server:
container_name: server
# Container image name
image: repo/myimage
## Image used to build Dockerfile
build:
#dockerfile: Dockerfile
context: ./smart/
# Container ports and host matching
ports:
- "3306:3306"
- "8080:8080"
## Startup Policy
restart driver: bridge
volumes:
dir-site:
driver_opts:
device: /smart/site/
type: bind
o: bind
# Declaration of services
services:
# service name
server:
container_name: server
# name of the image
image: repo/myimage
## Use IMAGE BUILD DOCKERFILE
build:
#dockerfile: Dockerfile
context: ./smart/
# CONTAINER ports
ports:
- "3306:3306"
- "8080:8080"
## Startup policy
restart: always
# connect to network1
networks:
- network1
volumes:
- /data/dockerStore/data/db_server_1:/var/lib/mysql/
restart: always
# connect to network1
networks:
- network1
volumes:
- /data/dockerStore/data/db_server_1:/var/lib/mysql/
Here is my kubernetes deployment file :
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 3
selector:
matchLabels:
app: myapp
# type: LoadBalancer
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: myapp
image: jksun12/vdsaipro
command: ["/run.sh"]
ports:
- containerPort: 80
- containerPort: 3306
- containerPort: 9001
- containerPort: 9002
# volumeMounts:
# - name: myapp-pv-claim
# mountPath: /var/lib/mysql
# volumes:
# - name: myapp-pv-claim
# persistentVolumeClaim:
# claimName: myapp-pv-claim
---
apiVersion: apps/v1
kind: PersistentVolumeClaim
metadata:
name: myapp-pv-claim
labels:
app: myapp
spec:
accesModes:
- ReadWriteMany
resources:
requests:
storage: 4Gi
My application files are in the /smart directory, which is a subdirectory of the directory where I actually run docker-compose and build my main Dockerfile.
How can I solve it ?
Thanks
I checked the logs inside the containers, and I found that the errors come from the extraction of the .war file inside the container.
There was also an error in the version of the tomcat server used for the application. So I changed the version of tomcat to another version.
It's solved now. Thanks
I have a docker swarm configured to use Traefik as reverse proxy. One of the containers in my swarm is running an Nginx server but I am getting a 502 Bad Gateway error when I navigate to that particular endpoint. Traefik is setup as follows:
version: '3.5'
services:
traefik:
image: traefik:alpine
command: |-
--entryPoints="Name:http Address::80 Redirect.EntryPoint:https"
--entryPoints="Name:https Address::443 TLS"
--defaultentrypoints="http,https"
--accesslogsfile="/var/log/access.log"
--acme
--acme.acmelogging="true"
--acme.domains="${SERVER},${SANS1}"
--acme.email="${ACME_EMAIL}"
--acme.entrypoint="https"
--acme.httpchallenge
--acme.httpchallenge.entrypoint="http"
--acme.storage="/opt/traefik/acme.json"
--acme.onhostrule="true"
--docker
--docker.swarmmode
--docker.domain="${SERVER}"
--docker.network="frontend"
--docker.watch
--api
networks:
- frontend
ports:
- target: 80
published: 80
mode: host
- target: 443
published: 443
mode: host
- target: 8080
published: 8080
mode: host
volumes:
- traefik_acme:/opt/traefik
- traefik_logs:/var/log/access.log
- /var/run/docker.sock:/var/run/docker.sock:ro
deploy:
replicas: 1
placement:
constraints: [node.role == manager]
networks:
frontend:
name: "frontend"
driver: overlay
volumes:
traefik_acme:
traefik_logs:
This compose file provides the overlay network and the Traefik service. The rest of my swarm is defined in the following compose file:
version: "3.5"
services:
test:
image: emilevauge/whoami
deploy:
labels:
traefik.enable: "true"
traefik.frontend.rule: "Host:${SERVER};PathPrefixStrip:/test"
traefik.port: 80
networks:
- frontend
octeditor:
image: ${DOCKER_OCTEDITOR_IMAGE_TAG}
deploy:
replicas: 1
labels:
traefik.enable: "true"
traefik.frontend.rule: "Host:${SERVER}"
traefik.port: 3000
networks:
- frontend
ports:
- "3000:80"
octserver:
image: ${DOCKER_OCTSERVER_IMAGE_TAG}
deploy:
replicas: 1
labels:
traefik.enable: "true"
traefik.frontend.rule: "Host:${SERVER};PathPrefixStrip:/api"
traefik.port: 4000
networks:
- frontend
ports:
- "4000:4000"
visualizer:
image: dockersamples/visualizer:stable
deploy:
placement:
constraints:
- 'node.role == manager'
labels:
traefik.enable: "true"
traefik.frontend.rule: "Host:${SERVER};PathPrefixStrip:/visualizer"
traefik.port: 8001
networks:
- frontend
ports:
- "8001:8080"
stop_grace_period: 1m30s
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
networks:
frontend:
external: true
The relevant configuration is for the octeditor service:
octeditor:
image: ${DOCKER_OCTEDITOR_IMAGE_TAG}
deploy:
replicas: 1
labels:
traefik.enable: "true"
traefik.frontend.rule: "Host:${SERVER}"
traefik.port: 3000
networks:
- frontend
ports:
- "3000:80"
I'm mapping port 80 (which Nginx listens to by default) to port 3000 where Traefik is configured to locate this service. This is the Dockerfile for the service running Nginx:
FROM node:latest as builder
WORKDIR /usr/src/app
COPY package.json .
RUN npm install
COPY . .
RUN npm run build
FROM nginx
COPY --from=builder /usr/src/app/build /usr/share/nginx/html
I simply build a react app and copy the build folder to the /usr/share/nginx/html folder. I've tried building and running this Dockerfile as a standalone container and it works, also I've checked the contents of the html folder and everything looks correct. The other services, apart from the visualizer service, are running correctly. Only this octedtior service and the visualizer service are giving me 502 errors. Can anyone suggest a solution or even how to check the traffic being sent to the nginx container? I've tried docker ps servicename but I can't see any errors coming from the service.
Edit:
If I change the configuration of the octeditor to this:
octeditor:
image: ${DOCKER_OCTEDITOR_IMAGE_TAG}
deploy:
replicas: 1
labels:
traefik.enable: "true"
traefik.frontend.rule: "Host:${SERVER}"
traefik.port: 80
networks:
- frontend
ports:
- "80:80"
And remove the test service that was previously listening on port 80 it seems to work. I don't understand what was wrong with the previous configuration however? I thought I was mapping traffic from port 3000 to port 80 of the container before, whereas now I'm mapping from port 80 to 80, but nothing should have changed from the perspective of the container, right?
The relevant configuration is for the octeditor service:
octeditor:
image: ${DOCKER_OCTEDITOR_IMAGE_TAG}
deploy:
replicas: 1
labels:
traefik.enable: "true"
traefik.frontend.rule: "Host:${SERVER}"
traefik.port: 3000
networks:
- frontend
ports:
- "3000:80"
I'm mapping port 80 (which Nginx listens to by default) to port 3000
where Traefik is configured to locate this service.
The traefik port needs to be 80, not 3000. The port mapping will create a forward from the host on 3000 to the container on 80. However traefik talks directly to the container over a shared network (frontend) and you need to provide it the container port.
There is no need to publish a host port for services accessed through traefik or any other reverse proxy unless you need to access them directly without the proxy (which brings into question whether you need a reverse proxy in those scenarios). In other words, this could be written without the ports:
octeditor:
image: ${DOCKER_OCTEDITOR_IMAGE_TAG}
deploy:
replicas: 1
labels:
traefik.enable: "true"
traefik.frontend.rule: "Host:${SERVER}"
traefik.port: 80
networks:
- frontend
I have created docker images using docker-compose.yml as below
version: '2'
services:
djangoapp:
build: .
volumes:
- .:/sig_app
- static_volume:/sig_app
networks:
- nginx_network
nginx:
image: nginx:1.13
ports:
- "80:80"
volumes:
- ./config/nginx/conf.d:/etc/nginx/conf.d
- static_volume:/sig_app
depends_on:
- djangoapp
networks:
- nginx_network
networks:
nginx_network:
driver: bridge
volumes:
static_volume:
I have used docker-compose build and docker-compose up. The three images are created as below
kubernetes_djangoapp
docker.io/python
docker.io/nginx
I want to deploy the application into kubernetes using YAML file.
I am new to kubernetes.
Django application is running with port 8000
and Nginx with port 80
This should work:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: my-deploy
spec:
replicas: 1
template:
metadata:
labels:
app: my-app
spec:
volumes:
- name: django-nginx
emptyDir: {}
- name: nginx-host
hostPath:
path: /config/nginx/conf.d
containers:
- name: djangoapp
image: kubernetes_djangoapp
volumeMounts:
- name: django-nginx
mountPath: /sig_app
- name: nginx
image: nginx:1.13
ports:
- containerPort: 80
volumeMounts:
- name: django-nginx
mountPath: /sig_app
- name: nginx-host
mountPath: /etc/nginx/conf.d
Note that you will have to modify some things to make it custom yours. I am missing where the image is. You should upload it to docker hub, or any registry of your choice.
About the volumes, here both containers share a non-persistent volume (django-nginx), which maps /sig_app directory in each container with each other. And another one that shares the container nginx (etc/nginx/conf.d) with your host (/config/nginx/conf.d), to pass the config file. A better way would be to use a ConfigMap. Check on that.
So, yeah, set the image for django and let me know if it doesn't work, and we will see whats failing.
Cheers
Take a look at Kompose. It will allow you to simply run the command
kompose up
to immediately deploy your docker-compose configuration to your cluster.
If you want to create a .yaml from your docker-compose file first to inspect and edit, you can run
kompose convert