in docker-compose.yml,
What is the difference between in following ports notations?
ports:
- "5000:5000"
resp:
ports:
- "8080"
or no ports at all.
For example in following docker-compose.yml, the mongodb service must be exposing a port to communicate with node service, but no port is specified
services:
node:
build:
context: .
dockerfile: node.dockerfile
ports:
- "3000:3000"
networks:
- nodeapp-network
depends_on:
- mongodb
mongodb:
image: mongo
networks:
- nodeapp-network
networks:
nodeapp-network:
driver: bridge
source: https://github.com/DanWahlin/NodeExpressMongoDBDockerApp
However in these docker-compose.yml, there are ports awlays specified with either 27017:27017 or 8080 notation.
services:
nginx:
container_name: nginx
image: ${DOCKER_ACCT}/nginx
build:
context: .
dockerfile: .docker/nginx.${APP_ENV}.dockerfile
links:
- node1:node1
- node2:node2
- node3:node3
ports:
- "80:80"
- "443:443"
networks:
- codewithdan-network
node1:
container_name: node-codewithdan-1
image: ${DOCKER_ACCT}/node-codewithdan
build:
context: .
dockerfile: .docker/node-codewithdan.${APP_ENV}.dockerfile
ports:
- "8080"
volumes:
- .:/var/www/codewithdan
working_dir: /var/www/codewithdan
env_file:
- ./.docker/env/app.${APP_ENV}.env
depends_on:
- mongo
- redis
networks:
- codewithdan-network
node2:
container_name: node-codewithdan-2
image: ${DOCKER_ACCT}/node-codewithdan
build:
context: .
dockerfile: .docker/node-codewithdan.${APP_ENV}.dockerfile
ports:
- "8080"
volumes:
- .:/var/www/codewithdan
working_dir: /var/www/codewithdan
env_file:
- ./.docker/env/app.${APP_ENV}.env
depends_on:
- mongo
- redis
networks:
- codewithdan-network
node3:
container_name: node-codewithdan-3
image: ${DOCKER_ACCT}/node-codewithdan
build:
context: .
dockerfile: .docker/node-codewithdan.${APP_ENV}.dockerfile
ports:
- "8080"
volumes:
- .:/var/www/codewithdan
working_dir: /var/www/codewithdan
env_file:
- ./.docker/env/app.${APP_ENV}.env
depends_on:
- mongo
- redis
networks:
- codewithdan-network
mongo:
container_name: mongo
image: ${DOCKER_ACCT}/mongo
build:
context: .
dockerfile: .docker/mongo.dockerfile
ports:
- "27017:27017"
env_file:
- ./.docker/env/mongo.${APP_ENV}.env
networks:
- codewithdan-network
redis:
container_name: redis
image: ${DOCKER_ACCT}/redis
build:
context: .
dockerfile: .docker/redis.${APP_ENV}.dockerfile
ports:
- "6379"
networks:
- codewithdan-network
networks:
codewithdan-network:
driver: bridge
source: https://github.com/DanWahlin/CodeWithDanDockerServices
Can you explain the difference?
Typical Docker containers run a long-running server listening on some TCP port. Other containers on the same Docker network can reach that container using the container’s name (docker run --name, container_name: directive) as a DNS name and the port the server is running on. In Docker Compose, Compose creates a Docker network per Compose YAML file, and also makes services available under their key in the YAML file. This works even if no ports: are specified.
So, for instance, if your docker-compose.yml file says
services:
mongo:
image: mongo
others:
env:
MONGODB_HOST: mongo
MONGODB_PORT: 27017
then the MongoDB container will be reachable on that host name and (default) port, even though it doesn’t explicitly have a ports:.
If you do declare a ports: then the container will be reachable from outside Docker space. If you only have one port it’s the port number of the server, and Docker picks the host port; this isn’t useful in most cases (but it’s guaranteed to not hit a port conflict). If you have two ports they’re the host port and internal service port. You can also specify a host IP address to bind(2) to.
Presence or absence of ports: doesn’t affect inter-dontainer communication. Always use the container’s name (or Docker-compose.yml service name) and the “internal” port number the server is listening on.
Either specify both ports (HOST:CONTAINER), or just the container port (an ephemeral host port is chosen). So in your case 8080 is container port
ports:
- "3000"
- "3000-3005"
- "8000:8000"
- "9090-9091:8080-8081"
- "49100:22"
- "127.0.0.1:8001:8001"
- "127.0.0.1:5000-5010:5000-5010"
- "6060:6060/udp"
From here
The ephemeral port range is configured by /proc/sys/net/ipv4/ip_local_port_range kernel parameter, typically ranging from 32768 to 61000.
Either way, you should be able to peek at what Docker has accomplished in your network stack by examining your NAT tables. from here
In docker compose by default no ports will be created in case they collide with already opened ports
Related
Following this question, I edited my gateway container to use the host network mode:
services:
gateway:
...
network_mode: "host"
and then the docker compose up -d gives me this:
Error response from daemon: failed to add interface veth701c890 to
sandbox: error setting interface "veth701c890" IP to 172.26.0.11/16:
cannot program address 172.26.0.11/16 in sandbox interface because it
conflicts with existing route {Ifindex: 4 Dst: 172.26.0.0/16 Src:
172.26.0.1 Gw: Flags: [] Table: 254
I restarted the docker and even the server. No luck.
The docker-compose.yml looks like this (only the gateway container has published ports):
version: '3.4'
services:
gateway:
image: <ms-yarp>
environment:
- ASPNETCORE_URLS=https://+:443;http://+:80
ports:
- "80:80"
- "443:443"
volumes:
- ./tls/:/tls/
networks:
- mynet
restart: on-failure
orders:
image: <registry>/orders
environment:
- ASPNETCORE_URLS=http://+:80
networks:
- mynet
restart: on-failure
users:
image: <registry>/users
environment:
- ASPNETCORE_URLS=http://+:80
networks:
- mynet
restart: on-failure
smssender:
image: <registry>/smssender
environment:
- ASPNETCORE_URLS=http://+:80
networks:
- mynet
restart: on-failure
logger:
image: <registry>/logger
environment:
- ASPNETCORE_URLS=http://+:80
networks:
- mynet
restart: on-failure
notifications:
image: <registry>/notifications
environment:
- ASPNETCORE_URLS=http://+:80
networks:
- mynet
restart: on-failure
cacheserver:
image: <registry>/redis
networks:
- mynet
restart: on-failure
...
networks:
mynet:
You can't combine host networking with any other Docker networking option. At least some versions of Compose have given warnings if you combine network_mode: host with other networks: or ports: options.
The other thing host networking means in this particular setup is that the one container that's using it is "outside Docker" for purposes of connecting to other containers. It works exactly the same way a non-container process would. That means the other containers need to publish ports: to be reachable from the gateway, and in turn the gateway configuration needs to use localhost and the published port numbers to reach the other containers.
version: '3.8'
services:
gateway:
image: <ms-yarp>
network_mode: host
orders:
image: <registry>/orders
ports:
- '8001:80'
networks:
- mynet
{
"ReverseProxy": {
"Clusters": {
"cluster": {
"Destinations": {
"orders": {
"Address": "http://localhost:8001"
}
}
}
}
}
}
Something like this:
(doesn't work with Docker Desktop on windows WSL2, at least I couldn't even run the nginx example that is here in the docs)
version: '3.4'
services:
gateway:
image: <ms-yarp>
environment:
- ASPNETCORE_URLS=https://+:443;http://+:80
network_mode: host
volumes:
- ./tls/:/tls/
restart: on-failure
orders:
image: <registry>/orders
environment:
- ASPNETCORE_URLS=http://+:80
ports:
- 8080:80
networks:
- mynet
restart: on-failure
users:
image: <registry>/users
environment:
- ASPNETCORE_URLS=http://+:80
ports:
- 8081:80
networks:
- mynet
restart: on-failure
smssender:
image: <registry>/smssender
environment:
- ASPNETCORE_URLS=http://+:80
ports:
- 8082:80
networks:
- mynet
restart: on-failure
logger:
image: <registry>/logger
environment:
- ASPNETCORE_URLS=http://+:80
ports:
- 8082:80
networks:
- mynet
restart: on-failure
notifications:
image: <registry>/notifications
environment:
- ASPNETCORE_URLS=http://+:80
ports:
- 8083:80
networks:
- mynet
restart: on-failure
cacheserver:
image: <registry>/redis
restart: on-failure
networks:
- mynet
Also in your gateway service configuration you will need to change the
http://orders:80 to http://localhost:8080
http://users:80 to http://localhost:8081
and so on
Also restrict ports on the docker host of 8080 to 8083 to be accessible only from localhost and not from the internet.
You could even put all the containers (except the gateway) to a different docker host that is accessible only from the docker host where the gateway container is running and change the config in gateway from http://orders:80 to http://otherdockerhost:80 and so on.
But for this docker compose will not be viable you will need to "manually" create the containers with docker run commands (or have 2 separate compose project one for the gateway and one for the rest of the services) so this is where more serious container orchestration tools are required like kubernetes (you could try docker swarm or nomad or any other container orchestrator, but these are not so popular so if you are new to both kubernetes and docker swarm or all the other you are better off with starting with kubernetes, you will reap the benefits in the long run for both this project and your personal carrier too)
---
version: "3.6"
services:
postgres:
image: postgres:alpine
restart: on-failure
environment:
- POSTGRES_USER=${APP_POSTGRES_USER:-postgres}
- POSTGRES_PASSWORD=${APP_POSTGRES_PASS:-postgres}
- POSTGRES_DB=${APP_POSTGRES_DB:-my_proj}
ports:
- "5566:5432"
server:
container_name: my_proj_app
hostname: my_proj_app
build:
context: .
depends_on:
- postgres
network_mode: host
environment:
- PORT=8080
- HOST=my_proj_app
ports:
- "8080:8080"
Here is my docker-compose.yml
I can't ping google.com from my_proj_app container.
Have anybody ideas what I'm doing wrong?
The error is explained here https://docs.docker.com/network/host/ : you used the host network mode which cannot access outside network, unlike the NAT.
Host mode networking can be useful to optimize performance, and in situations where a container needs to handle a large range of ports, as it does not require network address translation (NAT), and no “userland-proxy” is created for each port.
Try commenting network_mode: host.
I have a web app running outside of a container (localhost:8090).
How can I access it from within a container in a docker-compose network?
I tried to follow this answer that help for docker.
version: '3.6'
services:
postgres:
image: postgres
restart: always
volumes:
- db_data:/var/lib/postgresql/data
networks:
- host
graphql-engine:
image: hasura/graphql-engine:v1.0.0-beta.6
ports:
- "8080:8080"
depends_on:
- "postgres"
restart: always
environment:
HASURA_GRAPHQL_AUTH_HOOK: "http://localhost:8090/verify"
volumes:
db_data:
Add network_mode: "host" to your graphql-engine: and remove port mapping:
graphql-engine:
image: hasura/graphql-engine:v1.0.0-beta.6
depends_on:
- "postgres"
restart: always
network_mode: "host"
environment:
HASURA_GRAPHQL_AUTH_HOOK: "http://localhost:8090/verify"
graphql-engine would listen on host port 8080 and would be able to connect to localhost:8090
To make sure it worked, verify /etc/hosts file from the docker host is inside graphql-engine contianer .
Docs
Looking to automatically add the nginx container ip address inside my phpfpm container /etc/hosts file.
Inside my yml file, I have a service called phpfpm, and I know you can use extra_hosts attribute to assign values into the /etc/hosts file, however I don't know how to dynamically call place the nginx container IP.
nginx:
build: ./nginx
ports:
- "80:80"
- "443:443"
volumes:
- ../public/:/var/www/html/public/
container_name: nginx
networks:
- backend
phpfpm:
build: ./php-fpm
volumes:
- ../public/:/var/www/html/public/
container_name: phpfpm
extra_hosts:
- "test.local:nginx" <insert nginx ip to test.local>
networks:
- backend
Any thoughts on how to do this?
Containers within a compose file will run on same network and you can just their names. phpfpm and nginx in your case. Also if you need more names for the same service you need to use aliases
nginx:
build: ./nginx
ports:
- "80:80"
- "443:443"
volumes:
- ../public/:/var/www/html/public/
container_name: nginx
networks:
backend:
aliases:
- test.local
phpfpm:
build: ./php-fpm
volumes:
- ../public/:/var/www/html/public/
container_name: phpfpm
networks:
- backend
Why do you need Nginx Ip address? You can call nginx from phpfpm container by hostname nginx
Today I switched from "Docker Toolbox" to "Docker for Mac", because Docker now has finally write-access to my User directory (which doesn't worked with "Docker Toolbox") - Yay!
But this change also includes that all containers now running under my localhost and not under Docker's IP as before (e.g. 192.168.99.100).
Since my localhost listens to various ports by default (80, 443, ...) and I don't want to always add new created ports, that doesn't conflict with the standard one's, to my local dev domains (e.g. example.dev:8443), I wonder how to run my containers as before.
I read about network configs and tried a lot of things (creating a new host network, exposing ports with an IP in front of it, ...), but didn't got it working.
What kind of config do I need to run my app container with the IP 192.168.99.100? Thats my docker-compose.yml so far.
version: '2'
services:
app:
build:
context: .
dockerfile: Dockerfile
depends_on:
- mysql
- redis
- memcached
ports:
- 80:80
- 443:443
- 22:22
- 3000:3000
- 3001:3001
volumes:
- ./app/:/app/
- /tmp/debug/:/tmp/debug/
- ./:/docker/
volumes_from:
- storage
# cap and privileged needed for slowlog
cap_add:
- SYS_PTRACE
privileged: true
env_file:
- etc/environment.yml
- etc/environment.development.yml
mysql:
build:
context: docker/mysql/
dockerfile: MariaDB-10
ports:
- 3306:3306
volumes_from:
- storage
volumes:
- ./data/mysql:/var/lib/mysql
- /tmp/debug/:/tmp/debug/
env_file:
- etc/environment.yml
- etc/environment.development.yml
redis:
build: docker/redis/
volumes_from:
- storage
env_file:
- etc/environment.yml
- etc/environment.development.yml
memcached:
build: docker/memcached/
volumes_from:
- storage
env_file:
- etc/environment.yml
- etc/environment.development.yml
storage:
build: docker/storage/
volumes:
- /storage
You need to declare "networks:" for each of your services:
e.g.
version: '2'
services:
app:
image: xxxx:xxx
ports:
- "80:80"
networks:
- my-network
mysql:
image: xxxx:xxx
networks:
- my-network
networks:
my-network:
driver: bridge
Then from side your app configuration, you can use "mysql" as the hostname of database server.
You can define a network in your compose file, then add any services to the network.
https://docs.docker.com/compose/networking/
But I would suggest you just use different ports now that you are running natively. I.e. 8080:80