How can I make my spring boot application running inside docker containers connect to postgres database that is running in remote server (non-docker environment). Here is my docker-compose.yml file:
version: "3.3"
services:
app1:
image: repo/app1:latest
ports:
- 8000:8000
restart: always
network_mode: "host"
extra_hosts:
- 'postgresdb:192.168.2.50'
app2:
image: repo/app2:latest
ports:
- 8001:8001
restart: always
network_mode: "host"
extra_hosts:
- 'postgresdb:192.168.2.50'
IP of remote PostgreSQL database machine is: 192.168.2.50(hostname: postgresdb)
I am using network_mode: "host" option and works without any problem but I believe this would defeat the purpose of using docker network. What other options are available to make this work without using network_mode? IP address and necessary ports on both, the docker machine and remote database server, are all whitelisted and have access through the firewalls.
Such implementation, obviously will not work.
Since your database deployed remotely, the only working solution will be provided with environment variables.
version: "3.3"
services:
app1:
image: repo/app1:latest
ports:
- 8000:8000
restart: always
network_mode: "host"
environment:
- DBHOST: "192.168.2.50"
All you need in your application is to use this variable.
Python example:
dbhost = os.getenv("DBHOST")
Related
My docker-compose.yml contains this:
version: '3.2'
services:
mysql:
image: mysql:latest
container_name: mysql
restart: always
network_mode: "host"
hostname: localhost
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
volumes:
- $HOME/data/datasql:/var/lib/mysql
ports:
- 3306:3306
user-management-service:
build: user-management-service/
container_name: user-management-service
restart: always
depends_on:
- mysql
- rabbitmq
- eureka
network_mode: "host"
hostname: localhost
ports:
- 8089:8089
When I try to do docker-compose up, I get the following error:
"host" network_mode is incompatible with port_bindings
Can anyone help me with the solution?
network_mode: host is almost never necessary. For straightforward servers, like the MySQL server you show or what looks like a normal HTTP application, it's enough to use normal (bridged) Docker networking and ports:, like you show.
If you do set up host networking, it completely disables Docker's networking stack. You can't call to other containers using their host name, and you can't remap a container's port using ports: (or choose to not publish it at all).
You should delete the network_mode: lines you show in your docker-compose.yml file. The container_name: and hostname: lines are also unnecessary, and you can delete those too (specific exception: RabbitMQ needs a fixed hostname:).
I feel like the two places I see host networking are endorsed are either to call back to the host machine (see From inside of a Docker container, how do I connect to the localhost of the machine?), or because the application code has hard-coded localhost as the host name of the database or other components (in which case Docker and a non-Docker development setup fundamentally act differently, and you should configure these locations using environment variable or another mechanism).
Quick solution:
Downgrade the docker-compose version and you'll be fine. The issue is with the latest docker-compose version and network_mode: "host"
I faced the same issue on v1.29.2 and while everything worked smooth on v1.27.4.
I had the same problem with network_mode: 'host'.
When downgrading docker-compose from 1.29.2 to 1.25.4, it worked fine. Maybe some bug added in new versions?
Get rid of the param ports in your services containing network_mode its like doing mapping twice.
mysql:
image: mysql:latest
container_name: mysql
restart: always
network_mode: "host"
hostname: localhost
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
volumes:
- $HOME/data/datasql:/var/lib/mysql
....
....
To access the host's http://localhost inside your docker, you need to replace:
network_mode: host
with:
ports:
- 80:80
You can do the same with any other port.
If you want to connect to a local database then, when connecting to that database, don't use "localhost" or "127.0.0.1". Instead use "host.docker.internal" and that will allow traffic between your container to the database.
I have 3 containers with my bot, server and db. after docker-compose up, server and db are working. telegram bot does get-request and takes this error:
Get "http://localhost:8080/user/": dial tcp 127.0.0.1:8080: connect: connection refused
docker-compose.yml
version: "3"
services:
db:
image: postgres
container_name: todo_postgres
restart: always
ports:
- "5432:5432"
environment:
# TODO: Change it to environment variables
POSTGRES_USER: user
POSTGRES_DB: somedb
POSTGRES_PASSWORD: pass
server:
depends_on:
- db
build: .
restart: always
ports:
- 8080:8080
environment:
DB_NAME: somedb
DB_USERNAME: user
DB_PASSWORD: pass
bot:
depends_on:
- server
build:
./src/telegram_bot
environment:
BOT_TOKEN: TOKEN
restart: always
links:
- server
When using compose, try using the containers hostname.. in the case your bot should try to connect to
server:8080
Compose will handle the name resolution to the IP you need
What you try is to access localhost within your container (service) bot.
Maybe this answer will help you to solve the problem. It sound similar to your problem.
But I want to provide you another solution to your problem:
In case it's not needed to access the containers form outside (from your host), one appraoch would be making use of the expose functionality and a docker network.
See docs.docker.com: network.
The expose functionality allows to access your other containers within your network
See docs.docker.com: expose
Expose ports without publishing them to the host machine - they’ll only be accessible to linked services. Only the internal port can be specified.
Example
What is this example doing?
A couple of steps that are not mandatory
Set a static ip within your docker container
These Steps are not needed and can be omitted. However, I like to do this, since you have now a better control over the network. You can access the containers by their hostname (which is the container name or service name) as well.
The steps that are needed are the following:
This exposes port 8080, but do not publish it.
expose:
- 8080
The network which allows static ip configuration
networks:
vpcbr:
driver: bridge
ipam:
config:
- subnet: 10.5.0.0/16
A complete file could look similar to this:
version: "3.8"
services:
first-service:
image: <your-image>
networks:
vpcbr:
ipv4_address: 10.5.0.2
expose:
- 8080
second-service:
image: <your-image>
networks:
vpcbr:
ipv4_address: 10.5.0.3
depends_on:
- first-service
networks:
vpcbr:
driver: bridge
ipam:
config:
- subnet: 10.5.0.0/16
Your bot container is up before your server & db containers.
When you use depends_on it's not accually waiting them to finish setup themeselves.
You should try some tricky algorithem for waiting the other container finish setup.
I remmember that when I used Nginx proxy I used something called wait-for-it.sh
So I have this docker compose file
version: "2.1"
services:
nginx:
image: pottava/proxy
ports:
- 8080:80
environment:
- PROXY_URL=http://transmission-container:5080/
- BASIC_AUTH_USER=admin
- BASIC_AUTH_PASS=admin
- ACCESS_LOG=true
transmission:
image: linuxserver/transmission
container_name: transmission-container
ports:
- 5080:9091
restart: unless-stopped
I'm new to docker compose and trying it out for the first time. I need to be able to access the transmission service via http://localhost:8080 but nginx is returning a 502.
How should I change my compose file so that http://localhost:8080 will connect to the transmission service?
How can I make the transmission service not accessible via http://localhost:5080 and only accessible via http://localhost:8080 using docker compose?
I have tested the code below, it is working
version: "2.1"
services:
nginx:
image: pottava/proxy
ports:
- 8080:80
environment:
- PROXY_URL=http://transmission-container:9091/
- BASIC_AUTH_USER=admin
- BASIC_AUTH_PASS=admin
- ACCESS_LOG=true
transmission:
image: linuxserver/transmission
container_name: transmission-container
expose:
- "9091"
restart: unless-stopped
You no need to expose port 5080 to the host, the Nginx container can access directly the container port. The proxy URL needs to point to port 9091. Now you can't directly access the transmission service but need to go though the proxy server.
You should be able to access the other container using the service name and container port:
- PROXY_URL=http://transmission:9091/
If you do not want to access the transmission service from locahost, do not declare the host port:
ports:
- 9091
I am trying to access a docker container from another container using localhost address.
The compose file is pretty simple. Both containers ports are exposed.
There are no problems when building.
In my host machine I can successfully execute curl http://localhost:8124/ and get a response.
But inside the django_container when trying the same command I get Connection refused error.
I tried adding them in the same network, still result didn't change.
Well if I try to execute with the internal ip of that container like curl 'http://172.27.0.2:8123/' I get the response.
Is this the default behavior? How can I reach clickhouse_container using localhost?
version: '3'
services:
django:
container_name: django_container
build: ./django
ports:
- "8007:8000"
links:
- clickhouse:clickhouse
volumes:
- ./django:/usr/src/run
command: bash /usr/src/run/run.sh
clickhouse:
container_name: clickhouse_container
build: ./clickhouse
ports:
- "9001:9000"
- "8124:8123"
- "9010:9009"
So with this line here - "8124:8123" you're mapping the port of clickhouse container to localhost 8124. Which allows you to access clickhouse from localhost at port 8124.
If you want to hit clickhouse container from within the dockerhost network you have to use the hostname for the container. This is what I like to do:
version: '3'
services:
django:
hostname: djano
container_name: django
build: ./django
ports:
- "8007:8000"
links:
- clickhouse:clickhouse
volumes:
- ./django:/usr/src/run
command: bash /usr/src/run/run.sh
clickhouse:
hostname: clickhouse
container_name: clickhouse
build: ./clickhouse
ports:
- "9001:9000"
- "8124:8123"
- "9010:9009"
If you make the changes like I have made above you should be able to access clickhouse from within the django container like this curl http://clickhouse:8123.
As in #Billy Ferguson's answer, you can visit using localhost in host machine just because: you define a port mapping to route localhost:8124 to clickhouse:8123.
But when from other container(django), you can't. But if you insist, there is a ugly workaround: share host's network namespace with network_mode, but with this the django container will just share all network of host.
services:
django:
hostname: djano
container_name: django
build: ./django
ports:
- "8007:8000"
links:
- clickhouse:clickhouse
volumes:
- ./django:/usr/src/run
command: bash /usr/src/run/run.sh
network_mode: "host"
It depends of config.xml settings. If in config.xml <listen_host> 0.0.0.0</listen_host> you can use clickhouse-client -h your_ip --port 9001
I'm trying to map a port from my container, to a port on the host following the docs but it doesn't appear to be working.
After I run docker-compose -f development.yml up --force-recreate I get no errors. But if I try to reach the frontend service using localhost:8081 the network is unreachable.
I used docker inspect to view the IP and tried to ping that and still nothing.
Here is the docker-compose file I am using. And I doing anything wrong?
development.yml
version: '3'
services:
frontend:
image: nginx:latest
ports:
- "8081:80"
volumes:
- ./frontend/public:/var/www/html
api:
image: richarvey/nginx-php-fpm:latest
ports:
- "8080:80"
restart: always
volumes:
- ./api:/var/www/html
environment:
APPLICATION_ENV: development
ERRORS: 1
REMOVE_FILES: 0
links:
- db
- mq
db:
image: mariadb
restart: always
volumes:
- ./data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: dEvE10pMeNtMoDeBr0
mq:
image: rabbitmq:latest
restart: always
environment:
RABBITMQ_DEFAULT_USER: developer
RABBITMQ_DEFAULT_PASS: dEvE10pMeNtMoDeBr0
You are using docker toolbox. Docker toolbox uses docker machine. In Windows with docker toolbox, you are running under a virtualbox with its own IP, so localhost is not where your containers live. You will need to go 192.168.99.100:8081 to find your frontend.
As per the documentation on docker machine(https://docs.docker.com/machine/get-started/#run-containers-and-experiment-with-machine-commands):
$ docker-machine ip default
192.168.99.100