I've been trying to connect two docker containers. My flask backend and my react frontend, when I use localhost in the request the request goes through, but when i use the docker container name ie http://backend-service:5000/endpoint , the name can't be resolved. The documentation states that the containers connect to the same networking automatically and that accessing services from one should be as simple as that. I've tried adding links to the docker compose file as well with no luck.
Here is my docker-compose file:
version: '3'
services:
backend-service:
build: ./api
expose:
- 5000
ports:
- "5000:5000"
volumes:
- ./api:/usr/src/app
environment:
- FLASK_ENV=development
- FLASK_APP=app.py
- FLASK_DEBUG=1
client-service:
build: ./clientside
expose:
- 3000
ports:
- "3000:3000"
volumes:
- ./clientside/src:/usr/src/app/src
- ./clientside/public:/usr/src/app/public
links:
- "backend-service:backend"
Related
I am developing a workflow service as a training project. Abstracting from the details, everything you need to know for this question is in the image. For deployment, I rented a server and ran docker-compose on it. Everything works well, but what I'm worried about is that ports 8000 and 5432 are open.
The first question is, is it worth worrying? And if so, how to get rid of it?
Docker-compose file content below
version: "3"
services:
db:
container_name: 'emkk-db'
image: postgres
volumes:
- ./backend/data:/var/lib/postgresql/data
env_file:
- ./backend/db.env
ports:
- "5432:5432"
backend:
container_name: 'emkk-backend'
image: emkk_backend
build: ./backend
volumes:
- ./backend:/emkk/backend
env_file:
- ./backend/.env
ports:
- "8000:8000"
depends_on:
- db
frontend:
container_name: 'emkk-frontend'
image: emkk_frontend
build: ./frontend
command: npm run start
env_file:
- ./frontend/.env
volumes:
- /emkk/frontend/node_modules
- ./frontend:/emkk/frontend
ports:
- "80:80"
depends_on:
- backend
I also want to configure HTTPS protocol. I tried installing nginx and putting a certificate on it using a certbot, and then proxying requests to containers. I sat with this for several hours and I still did not manage to achieve anything better than a HTTPS for the nginx start page.
Maybe I'm doing completely wrong things, but I'm new to this, I haven't had to deal with deployments before. I would be grateful for your answers, which will contain an idea or an example of how you can do this.
If you don't have a connection to 8000 (probably WAS) or 5432 (database) from an external server, you can change docker-compose.yml to:
you have to expose only necessary ports for external clients.
when you connect to backend from web, you should use service name like backend:8000
when you connect to db from backend, you should use service name like db:5432
version: "3"
services:
db:
container_name: 'emkk-db'
image: postgres
volumes:
- ./backend/data:/var/lib/postgresql/data
env_file:
- ./backend/db.env
backend:
container_name: 'emkk-backend'
image: emkk_backend
build: ./backend
volumes:
- ./backend:/emkk/backend
env_file:
- ./backend/.env
depends_on:
- db
frontend:
container_name: 'emkk-frontend'
image: emkk_frontend
build: ./frontend
command: npm run start
env_file:
- ./frontend/.env
volumes:
- /emkk/frontend/node_modules
- ./frontend:/emkk/frontend
ports:
- "80:80"
depends_on:
- backend
And, you can use nginx proxy manager to service with HTTPS and a certificate from the certbot.
I am new to microservices and I have a project to setup multiple microservies, The project is setup like this.
Every nest js application has
API Application exposed to a port
database
docker-compose file responsible which creates the containers for each microservice.
Now what I am doing is to have
Nest JS MICROSERVICE APP 1
- API exposed to port 5000
- Postgres database working on 5432
- NATS running on 4222
NEST JS APP MICROSERVICE 2
- API exposed to port 5001
- Postgres database working on 5433
- NATS not running on 4222 as it is already occupied. If I change the port how I am gonna use the same message broker on both services.
The problem is I wanted to use the same NATS message broker on the second microservice and all the newly created microservice. my docker-compose file for NEST JS APP 1 is as follows.
version: '3.9'
services:
api:
container_name: nest_app_1
image: nest_app_1
build:
dockerfile: Dockerfile
context: .
ports:
- 127.0.0.1:5000:5000
env_file:
- .env
depends_on:
- db
- nats
networks:
- main
db:
container_name: postgres
image: postgres:latest
ports:
- 127.0.0.1:5432:5432
volumes:
- ./data:/var/lib/postgresql/data
env_file:
- .env
networks:
- main
pgadmin:
container_name: pgadmin
image: dpage/pgadmin4
ports:
- 127.0.0.1:8080:80
env_file:
- .env
networks:
- main
nats:
image: nats-streaming:latest
entrypoint:
- /nats-streaming-server
- -cid
- main_cluster
ports:
- "127.0.0.1:4222:4222"
- "127.0.0.1:6222:6222"
- "127.0.0.1:8222:8222"
restart: always
tty: true
networks:
- main
networks:
main:
driver: bridge
Second NEST JS microservice docker-compose is as follows
version: '3.9'
services:
api:
container_name: nest_app_2
image: nest_app_2
build:
dockerfile: Dockerfile
context: .
ports:
- 127.0.0.1:5001:5001
env_file:
- .env
depends_on:
- app_db_2
networks:
- main
app_db_2:
container_name: postgres_2
image: postgres:latest
ports:
- 127.0.0.1:5433:5432
volumes:
- ./data:/var/lib/postgresql/data
env_file:
- .env
networks:
- main
pgadmin:
container_name: pgadmin_2
image: dpage/pgadmin4
ports:
- 127.0.0.1:8081:80
env_file:
- .env
networks:
- main
nats:
image: nats-streaming:latest
entrypoint:
- /nats-streaming-server
- -cid
- main_cluster
ports:
- "127.0.0.1:4222:4222"
restart: always
tty: true
networks:
- main
networks:
main:
driver: bridge
Now I want to use NATS to communicate between both apps. So if I publish message from microservice 1 and I subscribe that message to microservice 2 and so on.
yes, sure the host ports are occupied if you link it through the Host network stack. You can only have one service linked to ip:port
It looks like you trying to start two NATS instances and let them join the same NATS cluster. But maybe you need two instances for development. You just want to see messages passing through it.
Option 1: just put everything in one compose and use depends_on and the same NATS node for both services
Option 2: Use a separate compose stack to provision your NATS infrastructure and use
extrnal_links.
Option 3: Define custom network for NATS cluster where every NATS container get's own iP.
But I would start with 1.
net core application. I have webapp and api application.
Below is my docker-compose file.
version: '3.4'
services:
enrichment.webapi:
container_name: enrichment.webapi
ports:
- 8000:80
- 8001:443
environment:
- "ASPNETCORE_URLS=https://+;http://+"
- ASPNETCORE_Kestrel__Certificates__Default__Path=/https/aspnetapp.pfx
- ASPNETCORE_Kestrel__Certificates__Default__Password=mypassword
volumes:
- ./conf.d/https/:/https/
build:
context: .
dockerfile: Enrichment.WebApi/Dockerfile
enrichment.webapp:
image: enrichment.web
ports:
- 7000:80
- 7001:443
environment:
- "ASPNETCORE_URLS=https://+;http://+"
- ASPNETCORE_Kestrel__Certificates__Default__Path=/https/aspnetapp.pfx
- ASPNETCORE_Kestrel__Certificates__Default__Password=mypassword
volumes:
- ./conf.d/https/:/https/
build:
context: .
dockerfile: Enrichment.Web/Dockerfile
depends_on:
- enrichment.webapi
When I do docker-compose up I see below when I do docker ps
0.0.0.0:8000->80/tcp, 0.0.0.0:32835->80/tcp, 0.0.0.0:8001->443/tcp, 0.0.0.0:32834->443/tcp
for the api app. When I do https://localhost:32834/index.html it works fine. But this port 32834 dynamically assigned. I want to set some static port so that I can handle my CORS in my web app. Each time this port is keep on getting changed when I rebuild docker-compose file. So Is there any way to handle this. Any help would be appreciated. Thanks
I already have a docker-compose.yml file like this:
version: "3.1"
services:
memcached:
image: memcached:alpine
container_name: dl-memcached
redis:
image: redis:alpine
container_name: dl-redis
mysql:
image: mysql:5.7.21
container_name: dl-mysql
restart: unless-stopped
working_dir: /application
environment:
- MYSQL_DATABASE=dldl
- MYSQL_USER=docker
- MYSQL_PASSWORD=docker
- MYSQL_ROOT_PASSWORD=docker
volumes:
- ./../:/application
ports:
- "8007:3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: dl-phpmyadmin
environment:
- PMA_ARBITRARY=1
- PMA_HOST=dl-mysql
- PMA_PORT=3306
- MYSQL_USER=docker
- MYSQL_PASSWORD=docker
- MYSQL_ROOT_PASSWORD=docker
restart: always
ports:
- 8002:80
volumes:
- /application
links:
- mysql
elasticsearch:
build: phpdocker/elasticsearch
container_name: dl-es
volumes:
- ./phpdocker/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
ports:
- "8003:9200"
webserver:
image: nginx:alpine
container_name: dl-webserver
working_dir: /application
volumes:
- ./../:/application:delegated
- ./phpdocker/nginx/nginx.conf:/etc/nginx/conf.d/default.conf
- ./logs:/var/log/nginx:delegated
ports:
- "9003:80"
php-fpm:
build: phpdocker/php-fpm
container_name: dl-php-fpm
working_dir: /application
volumes:
- ./../:/application:delegated
- ./phpdocker/php-fpm/php-ini-overrides.ini:/etc/php/7.2/fpm/conf.d/99-overrides.ini
- ./../docker/php-fpm/certs/store_stock/:/usr/local/share/ca-certificates/
- ./logs:/var/log:delegated # nginx logs
- /application/var/cache
environment:
XDEBUG_CONFIG: remote_host=host.docker.internal
PHP_IDE_CONFIG: "serverName=dl"
node:
build:
dockerfile: dl/phpdocker/node/Dockerfile
context: ./../
container_name: dl-node
working_dir: /application
ports:
- "8008:3000"
volumes:
- ./../:/application:cached
tty: true
My goal is to have 2 isolate environments working at the same time in the same server with the same docker-compose file? I wonder if it's possible?
I want to be able to stop and update one env. while the other one is still running and getting the traffic.
Maybe I need another approach in my case?
There are a couple of problems with what you're trying to do. If your goal is to put things behind a load balancer, I think that rather than trying to start multiple instances of your project, a better solution would be to use the scaling features available to docker-compose. In particular, if your goal is to put some services behind a load balancer, you probably don't want multiple instances of things like your database.
If you combine this with a dynamic front-end proxy like Traefik, you can make the configuration largely automatic.
Consider a very simple example consisting of a backend container running a simple webserver and a traefik frontend:
---
version: "3"
services:
webserver:
build:
context: web
labels:
traefik.enable: true
traefik.port: 80
traefik.frontend.rule: "PathPrefix:/"
frontend:
image: traefik
command:
- --api
- --docker
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
ports:
- "80:80"
- "127.0.0.1:8080:8080"
If I start it like this, I get a single backend and a single frontend:
docker-compose up
But I can also ask docker-compose to scale out the backend:
docker-compose up --scale webserver=3
In this case, I get a single frontend and three backend servers. Traefik will automatically discover the backends and will round-robin connections between them. You can download this example and try it out.
Caveats
There are a few aspects of your configuration that would need to change in order to make this work (and in fact, you would need to change them even if you were to create multiple instances of your project as you have proposed in your question).
Conflicting paths
Take for example the configuration of your webserver container:
volumes:
- ./logs:/var/log/nginx:delegated
If you start two instances of this service, both containers will mount ./logs on /var/log/nginx. If they both attempt to write to /var/log/nginx/access.log, you're going to have problems.
The easiest solution here is to avoid bind mounts for things like log directories (and any other directories to which you will be writing), and instead use named docker volumes.
Hardcoding container names
In some places, you are hardcoding the container name, like this:
mysql:
image: mysql:5.7.21
container_name: dl-mysql
This will cause problems if you attempt to start multiple instances of this project or multiple instances of the mysql container. Don't statically set the container name.
Deprecated links syntax
Your configuration is using the deprecated links syntax:
links:
- mysql
Don't do that. In modern docker, containers on the same network can simply refer to each other by name. In other words, if your compose configuration has:
mysql:
image: mysql:5.7.21
restart: unless-stopped
working_dir: /application
environment:
- MYSQL_DATABASE=dldl
- MYSQL_USER=docker
- MYSQL_PASSWORD=docker
- MYSQL_ROOT_PASSWORD=docker
volumes:
- ./../:/application
ports:
- "8007:3306"
Other containers in your compose stack can simply use the hostname mysql to refer to this service.
You won't be able to run same compose file on a host without changing the port mappings because that will cause port conflict. I'd recommend creating a base compose file and using extends to override port mappings for different environments.
Today I switched from "Docker Toolbox" to "Docker for Mac", because Docker now has finally write-access to my User directory (which doesn't worked with "Docker Toolbox") - Yay!
But this change also includes that all containers now running under my localhost and not under Docker's IP as before (e.g. 192.168.99.100).
Since my localhost listens to various ports by default (80, 443, ...) and I don't want to always add new created ports, that doesn't conflict with the standard one's, to my local dev domains (e.g. example.dev:8443), I wonder how to run my containers as before.
I read about network configs and tried a lot of things (creating a new host network, exposing ports with an IP in front of it, ...), but didn't got it working.
What kind of config do I need to run my app container with the IP 192.168.99.100? Thats my docker-compose.yml so far.
version: '2'
services:
app:
build:
context: .
dockerfile: Dockerfile
depends_on:
- mysql
- redis
- memcached
ports:
- 80:80
- 443:443
- 22:22
- 3000:3000
- 3001:3001
volumes:
- ./app/:/app/
- /tmp/debug/:/tmp/debug/
- ./:/docker/
volumes_from:
- storage
# cap and privileged needed for slowlog
cap_add:
- SYS_PTRACE
privileged: true
env_file:
- etc/environment.yml
- etc/environment.development.yml
mysql:
build:
context: docker/mysql/
dockerfile: MariaDB-10
ports:
- 3306:3306
volumes_from:
- storage
volumes:
- ./data/mysql:/var/lib/mysql
- /tmp/debug/:/tmp/debug/
env_file:
- etc/environment.yml
- etc/environment.development.yml
redis:
build: docker/redis/
volumes_from:
- storage
env_file:
- etc/environment.yml
- etc/environment.development.yml
memcached:
build: docker/memcached/
volumes_from:
- storage
env_file:
- etc/environment.yml
- etc/environment.development.yml
storage:
build: docker/storage/
volumes:
- /storage
You need to declare "networks:" for each of your services:
e.g.
version: '2'
services:
app:
image: xxxx:xxx
ports:
- "80:80"
networks:
- my-network
mysql:
image: xxxx:xxx
networks:
- my-network
networks:
my-network:
driver: bridge
Then from side your app configuration, you can use "mysql" as the hostname of database server.
You can define a network in your compose file, then add any services to the network.
https://docs.docker.com/compose/networking/
But I would suggest you just use different ports now that you are running natively. I.e. 8080:80