Running Multiple Microservices with multiple docker compose with NATS - docker

I am new to microservices and I have a project to setup multiple microservies, The project is setup like this.
Every nest js application has
API Application exposed to a port
database
docker-compose file responsible which creates the containers for each microservice.
Now what I am doing is to have
Nest JS MICROSERVICE APP 1
- API exposed to port 5000
- Postgres database working on 5432
- NATS running on 4222
NEST JS APP MICROSERVICE 2
- API exposed to port 5001
- Postgres database working on 5433
- NATS not running on 4222 as it is already occupied. If I change the port how I am gonna use the same message broker on both services.
The problem is I wanted to use the same NATS message broker on the second microservice and all the newly created microservice. my docker-compose file for NEST JS APP 1 is as follows.
version: '3.9'
services:
api:
container_name: nest_app_1
image: nest_app_1
build:
dockerfile: Dockerfile
context: .
ports:
- 127.0.0.1:5000:5000
env_file:
- .env
depends_on:
- db
- nats
networks:
- main
db:
container_name: postgres
image: postgres:latest
ports:
- 127.0.0.1:5432:5432
volumes:
- ./data:/var/lib/postgresql/data
env_file:
- .env
networks:
- main
pgadmin:
container_name: pgadmin
image: dpage/pgadmin4
ports:
- 127.0.0.1:8080:80
env_file:
- .env
networks:
- main
nats:
image: nats-streaming:latest
entrypoint:
- /nats-streaming-server
- -cid
- main_cluster
ports:
- "127.0.0.1:4222:4222"
- "127.0.0.1:6222:6222"
- "127.0.0.1:8222:8222"
restart: always
tty: true
networks:
- main
networks:
main:
driver: bridge
Second NEST JS microservice docker-compose is as follows
version: '3.9'
services:
api:
container_name: nest_app_2
image: nest_app_2
build:
dockerfile: Dockerfile
context: .
ports:
- 127.0.0.1:5001:5001
env_file:
- .env
depends_on:
- app_db_2
networks:
- main
app_db_2:
container_name: postgres_2
image: postgres:latest
ports:
- 127.0.0.1:5433:5432
volumes:
- ./data:/var/lib/postgresql/data
env_file:
- .env
networks:
- main
pgadmin:
container_name: pgadmin_2
image: dpage/pgadmin4
ports:
- 127.0.0.1:8081:80
env_file:
- .env
networks:
- main
nats:
image: nats-streaming:latest
entrypoint:
- /nats-streaming-server
- -cid
- main_cluster
ports:
- "127.0.0.1:4222:4222"
restart: always
tty: true
networks:
- main
networks:
main:
driver: bridge
Now I want to use NATS to communicate between both apps. So if I publish message from microservice 1 and I subscribe that message to microservice 2 and so on.

yes, sure the host ports are occupied if you link it through the Host network stack. You can only have one service linked to ip:port
It looks like you trying to start two NATS instances and let them join the same NATS cluster. But maybe you need two instances for development. You just want to see messages passing through it.
Option 1: just put everything in one compose and use depends_on and the same NATS node for both services
Option 2: Use a separate compose stack to provision your NATS infrastructure and use
extrnal_links.
Option 3: Define custom network for NATS cluster where every NATS container get's own iP.
But I would start with 1.

Related

Traefik 2 network between 2 containers results in Gateway Timeout errors

I'm trying to set up 2 docker containers with docker-compose, 1 is a Traefik proxy and the other is a Vikunja kanban board container.
They both have their own docker-compose file. I can start the containers and the Traefik dashboard doesn't show any issues but when I open the URL in a browser I only get a Gateway Timeout error.
I have been looking at similar questions on here and different platforms and in nearly all other cases the issue was that they were placed on 2 different networks. However, I added a networks directive to the Traefik docker-compose.yml and still have this problem, unless I'm using them wrong.
The docker-compose file for the Vikunja container
(adapted from https://vikunja.io/docs/full-docker-example/)
version: '3'
services:
api:
image: vikunja/api
environment:
VIKUNJA_DATABASE_HOST: db
VIKUNJA_DATABASE_PASSWORD: REDACTED
VIKUNJA_DATABASE_TYPE: mysql
VIKUNJA_DATABASE_USER: vikunja
VIKUNJA_DATABASE_DATABASE: vikunja
VIKUNJA_SERVICE_JWTSECRET: REDACTED
VIKUNJA_SERVICE_FRONTENDURL: REDACTED
volumes:
- ./files:/app/vikunja/files
networks:
- web
- default
depends_on:
- db
restart: unless-stopped
labels:
- "traefik.enable=true"
- "traefik.http.routers.vikunja-api.rule=Host(`subdomain.domain.de`) && PathPrefix(`/api/v1`, `/dav/`, `/.well-known/`)"
- "traefik.http.routers.vikunja-api.entrypoints=websecure"
- "traefik.http.routers.vikunja-api.tls.certResolver=myresolver"
frontend:
image: vikunja/frontend
labels:
- "traefik.enable=true"
- "traefik.http.routers.vikunja-frontend.rule=Host(`subdomain.domain.de`)"
- "traefik.http.routers.vikunja-frontend.entrypoints=websecure"
- "traefik.http.routers.vikunja-frontend.tls.certResolver=myresolver"
networks:
- web
- default
restart: unless-stopped
db:
image: mariadb:10
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
environment:
MYSQL_ROOT_PASSWORD: REDACTED
MYSQL_USER: vikunja
MYSQL_PASSWORD: REDACTED
MYSQL_DATABASE: vikunja
volumes:
- ./db:/var/lib/mysql
restart: unless-stopped
command: --max-connections=1000
networks:
- web
networks:
web:
external: true
The network directives for the api and frontend services in the Vikunja docker-compose.yml were present in the template (I added one for the db service for testing but it didn't have any effect).
networks:
- web
After getting a docker error about the network not being found I created it via docker network create web
The docker-compose file for the Traefik container
version: '3'
services:
traefik:
image: traefik:v2.8
ports:
- "80:80"
- "443:443"
- "8080:8080" # dashboard
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./letsencrypt:/letsencrypt
- ./traefik.http.yml:/etc/traefik/traefik.yml
networks:
- web
networks:
web:
external: true
I've tried adding the Traefik service to the Vikunja docker-compose.yml in one file but that didn't have any effect either.
I'm thankful for any pointers.
For debugging you could try to configure all container to use the host network to enusre they are realy on the same netwok.
i had a similar issue trying to run two different dockers and getting a
"Gateway Timeout". My issue was solved after changing the mapping in the second docker for traefik and accessing the site with :84 at the end (http://sitename:84)
traefik:
image: traefik:v2.0
container_name: "${PROJECT_NAME}_traefik"
command: --api.insecure=true --providers.docker
ports:
- '84:80'
- '8084:8080'

Write to Neo4j standalone instance from another container within Kubernetes network

I have created a docker network consisting of a Neo4j container and a python script that writes to the Neo4j database in another container. The python script uses the Neo4j service name to create the Neo4j URI bolt://db:7687 (stored in the .env file), which is accessible from within the docker network.
My question is, how do I translate the following docker-compose.yml to Kubernetes? My attempt to translate it with Kompose failed because Service "daily_pe_update" won't be created because 'ports' is not specified.
version: "3.8"
services:
db:
container_name: db
image: neo4j:4.4.3-community
ports:
- 7888:7474
- 7999:7687
restart: unless-stopped
volumes:
- .db/data:/data
- .db/conf:/conf
- .db/logs:/logs
- .db/plugins:/plugins
networks:
- data_service
env_file:
- ./.env
daily_pe_update:
depends_on:
- "db"
container_name: daily_pe_update
build: ./pe_update
command: "python daily_pe_update.py"
restart: on-failure:5
volumes:
- ./pe_update/:/usr/app/src/
env_file:
- ./.env
networks:
- data_service
networks:
data_service:

How to create 2 different running app with the same docker-compose.yml file?

I already have a docker-compose.yml file like this:
version: "3.1"
services:
memcached:
image: memcached:alpine
container_name: dl-memcached
redis:
image: redis:alpine
container_name: dl-redis
mysql:
image: mysql:5.7.21
container_name: dl-mysql
restart: unless-stopped
working_dir: /application
environment:
- MYSQL_DATABASE=dldl
- MYSQL_USER=docker
- MYSQL_PASSWORD=docker
- MYSQL_ROOT_PASSWORD=docker
volumes:
- ./../:/application
ports:
- "8007:3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: dl-phpmyadmin
environment:
- PMA_ARBITRARY=1
- PMA_HOST=dl-mysql
- PMA_PORT=3306
- MYSQL_USER=docker
- MYSQL_PASSWORD=docker
- MYSQL_ROOT_PASSWORD=docker
restart: always
ports:
- 8002:80
volumes:
- /application
links:
- mysql
elasticsearch:
build: phpdocker/elasticsearch
container_name: dl-es
volumes:
- ./phpdocker/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
ports:
- "8003:9200"
webserver:
image: nginx:alpine
container_name: dl-webserver
working_dir: /application
volumes:
- ./../:/application:delegated
- ./phpdocker/nginx/nginx.conf:/etc/nginx/conf.d/default.conf
- ./logs:/var/log/nginx:delegated
ports:
- "9003:80"
php-fpm:
build: phpdocker/php-fpm
container_name: dl-php-fpm
working_dir: /application
volumes:
- ./../:/application:delegated
- ./phpdocker/php-fpm/php-ini-overrides.ini:/etc/php/7.2/fpm/conf.d/99-overrides.ini
- ./../docker/php-fpm/certs/store_stock/:/usr/local/share/ca-certificates/
- ./logs:/var/log:delegated # nginx logs
- /application/var/cache
environment:
XDEBUG_CONFIG: remote_host=host.docker.internal
PHP_IDE_CONFIG: "serverName=dl"
node:
build:
dockerfile: dl/phpdocker/node/Dockerfile
context: ./../
container_name: dl-node
working_dir: /application
ports:
- "8008:3000"
volumes:
- ./../:/application:cached
tty: true
My goal is to have 2 isolate environments working at the same time in the same server with the same docker-compose file? I wonder if it's possible?
I want to be able to stop and update one env. while the other one is still running and getting the traffic.
Maybe I need another approach in my case?
There are a couple of problems with what you're trying to do. If your goal is to put things behind a load balancer, I think that rather than trying to start multiple instances of your project, a better solution would be to use the scaling features available to docker-compose. In particular, if your goal is to put some services behind a load balancer, you probably don't want multiple instances of things like your database.
If you combine this with a dynamic front-end proxy like Traefik, you can make the configuration largely automatic.
Consider a very simple example consisting of a backend container running a simple webserver and a traefik frontend:
---
version: "3"
services:
webserver:
build:
context: web
labels:
traefik.enable: true
traefik.port: 80
traefik.frontend.rule: "PathPrefix:/"
frontend:
image: traefik
command:
- --api
- --docker
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
ports:
- "80:80"
- "127.0.0.1:8080:8080"
If I start it like this, I get a single backend and a single frontend:
docker-compose up
But I can also ask docker-compose to scale out the backend:
docker-compose up --scale webserver=3
In this case, I get a single frontend and three backend servers. Traefik will automatically discover the backends and will round-robin connections between them. You can download this example and try it out.
Caveats
There are a few aspects of your configuration that would need to change in order to make this work (and in fact, you would need to change them even if you were to create multiple instances of your project as you have proposed in your question).
Conflicting paths
Take for example the configuration of your webserver container:
volumes:
- ./logs:/var/log/nginx:delegated
If you start two instances of this service, both containers will mount ./logs on /var/log/nginx. If they both attempt to write to /var/log/nginx/access.log, you're going to have problems.
The easiest solution here is to avoid bind mounts for things like log directories (and any other directories to which you will be writing), and instead use named docker volumes.
Hardcoding container names
In some places, you are hardcoding the container name, like this:
mysql:
image: mysql:5.7.21
container_name: dl-mysql
This will cause problems if you attempt to start multiple instances of this project or multiple instances of the mysql container. Don't statically set the container name.
Deprecated links syntax
Your configuration is using the deprecated links syntax:
links:
- mysql
Don't do that. In modern docker, containers on the same network can simply refer to each other by name. In other words, if your compose configuration has:
mysql:
image: mysql:5.7.21
restart: unless-stopped
working_dir: /application
environment:
- MYSQL_DATABASE=dldl
- MYSQL_USER=docker
- MYSQL_PASSWORD=docker
- MYSQL_ROOT_PASSWORD=docker
volumes:
- ./../:/application
ports:
- "8007:3306"
Other containers in your compose stack can simply use the hostname mysql to refer to this service.
You won't be able to run same compose file on a host without changing the port mappings because that will cause port conflict. I'd recommend creating a base compose file and using extends to override port mappings for different environments.

Docker mis-forwarding ports

I have several domains sharing one public IP (EC2 instance). My setup is like this:
/home/ubuntu contains docker-compose.yml:
version: '3'
services:
nginx-proxy:
image: "jwilder/nginx-proxy"
container_name: nginx-proxy
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
ports:
- "80:80"
restart: "always"
This creates a network named ubuntu_default which will allow other compose instances to join. The nginx-proxy image creates reverse proxies for these other compose instances so that you can visit example.com and be routed to the appropriate UI within the appropriate compose instance.
/home/ubuntu/example.com/project-1 contains a docker-compose.yml like:
version: '3'
services:
db:
build: "./db" # mongo
volumes:
- "./data:/data/db"
restart: "always"
api:
build: "./api" # a node backend
ports:
- "9005:9005"
restart: "always"
depends_on:
- db
ui:
build: "./ui" # a react front end
ports:
- "8005:8005"
restart: "always"
environment:
- VIRTUAL_HOST=project-1.example.com # this tells nginx-proxy which domain to proxy
- VIRTUAL_PORT=8005 # this tells nginx-proxy which port to proxy
networks:
default:
external:
name: ubuntu_default
/home/ubuntu/testing.com/project-2 contains a docker-compose.yml like:
version: '3'
services:
db:
build: "./db" # postgres
volumes:
- "./data:/var/lib/postgresql/data"
restart: "always"
api:
build: "./api" # a python backend
ports:
- "9000:9000"
restart: "always"
depends_on:
- db
ui:
build: "./ui" # a react front end
ports:
- "8000:8000"
restart: "always"
environment:
- VIRTUAL_HOST=testing.com,www.testing.com # tells nginx-proxy which domains to proxy
- VIRTUAL_PORT=8000 # tells nginx-proxy which port to proxy
networks:
default:
external:
name: ubuntu_default
So basically:
project-1.example.com:80 forwards to the UI running on :8005
project-1.example.com:80/api forwards to the API running on :9005
testing.com forwards to the UI running on :8000
testing.com/api forwards to the API running on :9000
...and that all works perfectly as long as I only run one at a time. The moment I start both Compose instances, the /api urls start clashing. I can sit on one of them and refresh repeatedly and sometimes I'll see the one for example.com/api and sometimes I'll see the one for testing.com/api.
I have no idea whats going on at this point. Maybe the premise I'm working against is fundamentally flawed but it seems like an intended use of Docker/Compose. I'm open to suggestions to accomplish the same otherwise.
Docker containers communicate using DNS lookups on their network. If multiple containers have the same alias on the same network, it will round robin load balance between the containers with each network connection. If you don't want containers to talk to each other, then you don't want them on the same docker network. The good news is you solve this by using more than one network, and not putting the api and db server on the frontend proxy network:
version: '3'
services:
db:
build: "./db" # postgres
volumes:
- "./data:/var/lib/postgresql/data"
restart: "always"
api:
build: "./api" # a python backend
ports:
- "9000:9000"
restart: "always"
depends_on:
- db
ui:
build: "./ui" # a react front end
ports:
- "8000:8000"
restart: "always"
networks:
- default
- proxy
environment:
- VIRTUAL_HOST=testing.com,www.testing.com # tells nginx-proxy which domains to proxy
- VIRTUAL_PORT=8000 # tells nginx-proxy which port to proxy
networks:
proxy:
external:
name: ubuntu_default
If you do not override the default network, docker will create one for your compose project and use it for any containers not assigned to another network.

Docker Compose Nginx Link containers

I have a docker compose container that runs Nginx. The site hosted is just a .test domain, like example.test.
Also in the container Nginx runs a location proxy and redirects it to example.test:8000. But it's not able to connect to that because that's actually being hosted from a different container on the same system (all bridged networks).
How can I let the containers communicate using example.test domain?
Or if I can't get them to communicate via example.test then how can I link them so they can use their docker-compose service name such as api or frontend?
Docker compose:
version: '3'
services:
db:
image: postgres
ports:
- "5432:5432"
django:
build: ./api
command: ["./docker_up.sh"]
restart: always
volumes:
- ./api:/app/api
- api-static:/app/api/staticfiles
ports:
- "8000:8000"
depends_on:
- db
environment:
- MODE=DEV
volumes:
frontend-build:
api-static:
certificates:
2nd compose file (run together):
version: '3'
services:
django:
environment:
- MODE=PROD
#links:
# - hosting
hosting:
build: ./hosting
restart: always
network_mode: bridge
volumes:
- frontend-build:/var/www
ports:
- "80:80"
- "443:443"
environment:
- MODE=PROD
#links:
# - django
volumes:
frontend-build:
With these current settings I get an error when I run it
ERROR: for 92b89f848637_opensrd_hosting_1 Cannot start service hosting: Cannot link to /opensrd_django_1, as it does not belong to the default network
Edit: Altered docker-compose.prod.yml:
networks:
app_net:
driver: bridge
ipam:
driver: default
config:
-
subnet: 172.16.238.0/24
services:
django:
environment:
- MODE=PROD
networks:
app_net:
ipv4_address: 172.16.238.10
But this gives me an error.
ERROR: The Compose file './docker-compose.prod.yml' is invalid because:
networks.app_net value Additional properties are not allowed ('config' was unexpected)
networks.app_net.ipam contains an invalid type, it should be an object
So I tried the options given by #trust512 and #DimaL, and those didn't work.
However after deleting the network and links from my compose files, and removing the existing default network and built containers, it worked, and I can not refer between container using db, django, and hosting.
The only thing different is I changed the composer version from 3 to 3.5.
These are the final files for anyone interested:
version: '3.5'
services:
db:
image: postgres
ports:
- "5432:5432"
django:
build: ./api
command: ["./docker_up.sh"]
restart: always
volumes:
- ./api:/app/api
- api-static:/app/api/staticfiles
ports:
- "8000:8000"
depends_on:
- db
environment:
- MODE=DEV
volumes:
frontend-build:
api-static:
docker-compose.prod.yml:
version: '3.5'
services:
django:
environment:
- MODE=PROD
hosting:
build: ./hosting
restart: always
volumes:
- frontend-build:/var/www
ports:
- "80:80"
- "443:443"
environment:
- MODE=PROD
volumes:
frontend-build:
You can use external_links (https://docs.docker.com/compose/compose-file/#external_links) or try to put all containers on the same virtual network.
As far as I understand you just want them (django and nginx) to be linked across composes?
Then a native solution would be to use external_links exampled here
And use it like this:
services:
[...]
hosting:
[...]
external_links:
- django_1:example
[...]
Where django_1 stands for the container name created by the compose you provided and example is the alias that the container will be visible inside Django container.
Other way round you can just point a example.test domain to a specific address by editing your /etc/hosts (provided you work on linux/mac)
for example by adding a record like
172.16.238.10 example.test
Where the address above would point to your django application (container).
The above can be achieved without altering your /etc/hosts by using native solution from compose (extra_hosts) documented here
Additionally if you prefer a static ip address for your django/nginx containers in case you stick to the /etc/hosts od extra_hosts solution you can utilize another native solution provided by compose that sets up a static ip for a chosen services, properly exampled here
A adjusted listing from the linked documentation:
services:
[...]
django:
[...]
networks:
app_net:
ipv4_address: 172.16.238.10
networks:
app_net:
driver: bridge
ipam:
driver: default
config:
-
subnet: 172.16.238.0/24

Resources