I have a simple web app with a docker-compose.yml configuration like this:
version: "3"
services:
web:
build: .
ports:
- "8000:8000"
db:
image: postgres
ports:
- "5432:5432"
I can access the database service within the web app container with postgres://db:5432 because both containers share networking.
I'd like to access the database service from my host machine using postgres://db:5432. How do I map the db:5432 host from the container to db:5432 on my local host machine? I've tried adding 127.0.0.1:5432 db:5432 to my /etc/hosts file which does not seem to work.
The /etc/hosts file is simply a way to statically resolve names when no DNS server is present or resolved. It can't map port addresses.
Ref: https://serverfault.com/questions/54357/can-i-specify-a-port-in-an-entry-in-my-etc-hosts-on-os-x
It will work if you do,
127.0.0.1 db
Remove the port and it will work.
Also, you can directly access Postgres with localhost: 5432 from host machine.
Related
I'm making a sample ETL app for learning purposes. I want to have 2 containers: a MySQL Container, and a Python container running with Redis to serve the data.
I want to open ports on these containers, but I don't want to open them to the internet for obvious security reasons.
If I open ports on these containers, will they remain only open to the Host machine, or does the Host machine also have to open these ports?
If you don't have any Compose ports: or docker run -p options, the containers will be able to communicate with each other but will not be reachable from the host or off-host. A sample Compose setup:
version: '3.8'
services:
app:
build: .
ports: ['8080:8080'] # the application itself is visible
environment:
PGHOST: postgres # using normal Docker inter-container communication
REDIS_HOST: redis
postgres:
image: postgres:13
# no ports:
redis:
image: redis:6
# no ports:
You can also set ports: with an optional bind address. If that address is 127.0.0.1, then the published port will be reachable from non-container processes on the host system. Other hosts will not be able to connect to it, and containers will not be able to use the host gateway address or host.docker.internal to connect to it.
services:
postgres:
# from non-container processes on the host,
# PGHOST=localhost PGPORT=54321 reaches this container
#
# from other containers in this Compose file,
# PGHOST=postgres PGPORT=5432 still works
ports: ['127.0.0.1:54321:5432']
I build a website using Strapi and Gatsby, everythings works well when I try to connect to a remote database, but I'm trying to create a db inside a container and so far no luck.
Essentially, what I did is create the following docker-compose:
version: '3'
services:
backend:
container_name: myapp_backend
build: ./backend/
ports:
- '3002:3002'
volumes:
- ./backend:/usr/src/myapp/backend
- /usr/src/myapp/backend/node_modules
environment:
- APP_NAME=myapp_backend
- DATABASE_CLIENT=mysql
- DATABASE_HOST=db
- DATABASE_PORT=3307
- DATABASE_NAME=myapp_db
- DATABASE_USERNAME=johnny
- DATABASE_PASSWORD=stecchino
- DATABASE_SSL=false
- DATABASE_AUTHENTICATION_DATABASE=myapp_db
- HOST=localhost
depends_on:
- db
restart: always
db:
container_name: myapp_mysql
image: mysql:5.7
volumes:
- ./db.sql:/docker-entrypoint-initdb.d/db.sql
restart: always
ports:
- 3307:3307
environment:
MYSQL_ROOT_PASSWORD: 5!JF6!FgAkvt
MYSQL_DATABASE: myapp_db
MYSQL_USER: johnny
MYSQL_PASSWORD: stecchino
command: mysqld --character-set-server=utf8 --collation-server=utf8_general_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: 'myapp_phpmyadmin'
links:
- db
environment:
PMA_HOST: db
PMA_PORT: 3307
ports:
- '8081:80'
volumes:
- /sessions
depends_on:
- db
frontend:
container_name: myapp_frontend
build: ./frontend/
ports:
- '3001:3001'
depends_on:
- backend
volumes:
- ./frontend:/usr/src/myapp/frontend
the backend service contains the Strapi application, the db service contains the mysql instance which runs on the port 3307 'cause 3306 is already in use.
Then I have also installed phpmyadmin, and last but not least the Gastby site. When I run using docker-compose up --build, and try to access to phpmyadmin using:
http://localhost:8081/index.php
with the following credentials:
user: johnny
pwd: stecchino
I get:
MySQL mysqli::real_connect():(HY000/2002): Connection refused
now, what I did for fix that situation is pass the port 3306 instead of 3307 to backend and phpmyadmin service. And magically, everything works. But why? I have mapped container and host to 3307...
There are 2 things happening here.
Mysql is running on port 3306.
This is because you never told the mysql container to run on port 3307. The default configuration is running on 3306.
phpadmin can connect to mysql at port 3306.
Of course it can. This is because when you define multiple services within the same docker-compose file, they start on the same network. This means that they can see and connect to each other's internal ports without the need for external port binding like 3306:3306
I would suggest to keep port bindings only for services that you want access outside the docker environment (like the UI), and for internal components just expose the port like this
expose:
- 3306
Both answers are useful, I am particularly fond of Manish's answer
I wanted to add some additional wording:
There are the internal docker networks which nothing from the outside can gain access to. From inside any given service (or container), you can reach every other service (or container) via:
<service-name>:<port>/path/of/resources
<container-name>:<port>/path/of/resources
In order to access resources inside the docker network from outside of docker, whether that is from your host environment, or farther upstream on the internet, the docker daemon needs to bind to host ports, and then forward information received on those ports to a docker service (and ultimately a docker container).
In your docker-compose.yml when you do the 3307:3307 you are telling the docker daemon to listen on port 3307, and forward to your db service internally on it's port 3307.
However, from what we can all see, mysql is still internally (that is, inside the container) listening for traffic on port 3306. Any containers or services on the same docker networks as your db service (mysql running container(s)) would be able to access mysql via something like:
<driver>:mysql://db:3306/<dbname>
If you wanted all host traffic and docker network traffic to access mysql on port 3307, you would also need to configure mysql to listen on port 3307 instead of 3306. That tidbit of information does not appear to be in your question at the time of writing.
I hope the additional information helps! It's a topic I chat often about when talking docker with folks.
Because 3306 is the exposed port by the official Dockerfile.
What you can do is to map the port that is running MySQL to another port on your host: 3307:3306 for instance (always host:container)
I have a docker-compose file which is globally like this.
version '2'
services:
app:
image: myimage
ports:
- "80:80"
networks:
mynet:
ipv4_adress: 192.168.22.22
db:
image: postgres:9.5
ports:
- "6432:5432"
networks:
mynet:
ipv4_adress: 192.168.22.23
...
networks:
mynet:
driver: bridge
ipam:
driver: default
config:
- subnet: 192.168.22.0/24
I want to put my postgresql and application in subnetworks to avoid the ports to be exposed outside my computer/server.
From within the app container, I can't connect to 192.168.22.23, I installed net-tools to use ifconfig/netstat, and it doesn't seem the containers are able to communicate.
I assume I have this problem because I'm using subnetworks with static ipv4 adresses.
I can access both static IPs from the host (connect to postgres and access the application)
Do you have any piece of advice, the goal is to access the ports of another container to communicate with him, without removing the use of static ips (on app at least). Here, to connect to postgresql from the app container.
The docker run -p option and Docker Compose ports: option take a bind address as an optional parameter. You can use this to make a service accessible from the same host, but not from other hosts:
services:
db:
ports:
- '127.0.0.1:6432:5432'
(The other good use of this setting is if you have a gateway machine with both a public and private network interface, and you want a service to only be accessible from the private network.)
Once you have this, you can dispense with all of the manual networks: setup. Non-Docker services on the same host can reach the service via the special host name localhost and the published port number. Docker services can use inter-container networking; within the same docker-compose.yml file you can use the service name as a host name, and the internal port number.
host$ PGHOST=localhost PGPORT=6432 psql
services:
app:
environment:
- PGHOST=db
- PGPORT=5432
You should remove all of the manual networks: setup, and in general try not to think about the Docker-internal IP addresses at all. If your Docker is Docker for Mac or Docker Toolbox, you cannot reach the internal IP addresses at all. In a multi-host environment they will be similarly unreachable from hosts other than where the container itself is running.
Unable to connect to containers running on separate docker hosts
I've got 2 docker Tomcat containers running on 2 different Ubuntu vm's. System-A has a webservice running and System-B has a db. I haven't been able to figure out how to connect the application running on system-A to the db running on system-B. When I run the database on system-A, the application(which is also running on system-A) can connect to the database. I'm using docker-compose to setup the network(which works fine when both containers are running on the same VM). I've execd into etc/hosts file in the application container on system-A and I think whats missing is the ip address of System-B.
services:
db:
image: mydb
hostname: mydbName
ports:
- "8012: 8012"
networks:
data:
aliases:
- mydbName
api:
image: myApi
hostname: myApiName
ports:
- "8810: 8810"
networks:
data:
networks:
data:
You would configure this exactly the same way you would as if Docker wasn't involved: configure the Tomcat instance with the DNS name or IP address of the other server. You would need to make sure the service is published outside of Docker space using a ports: directive.
On server-a.example.com you could run this docker-compose.yml file:
version: '3'
services:
api:
image: myApi
ports:
- "8810:8810"
env:
DATABASE_URL: "http://server-b.example.com:8012"
And on server-b.example.com:
version: '3'
services:
db:
image: mydb
ports:
- "8012:8012"
In principle it would be possible to set up an overlay network connecting the two hosts, but this is a significantly more complicated setup.
(You definitely don't want to use docker exec to modify /etc/hosts in a container: you'll have to repeat this step every time you delete and recreate the container, and manually maintaining hosts files is tedious and error-prone, particularly if you're moving containers between hosts. Consul could work as a service-discovery system that provides a DNS service.)
I try to start Concourse CI with custom docker-compose
version: '2'
services:
concourse-web:
image: concourse/concourse
container_name: concourse-web
command: web
network_mode: host
volumes: ["./keys/web:/concourse-keys"]
environment:
CONCOURSE_BASIC_AUTH_USERNAME: concourse
CONCOURSE_BASIC_AUTH_PASSWORD: changeme
CONCOURSE_EXTERNAL_URL: http://my.internal.ip:8092
CONCOURSE_BIND_PORT: 8092
CONCOURSE_POSTGRES_DATA_SOURCE: |-
postgres://odoo:odoo#localhost:5432/concourse?sslmode=disable
concourse-worker:
image: concourse/concourse
container_name: concourse-worker
network_mode: host
privileged: true
command: worker
volumes: ["./keys/worker:/concourse-keys"]
environment:
CONCOURSE_BIND_PORT: 8092
And worker can't connect to web part.
Can you please help me with this.
P.S. Database postgtresql started on 5432 port on host machine, and with connection all right.
Worker errors:
{"timestamp":"1487953300.400844336","source":"tsa","message":"tsa.connection.channel.forward-worker.register.failed-to-fetch-containers","log_level":2,"data":{"error":"invalid character '\u003c' looking for beginning of value","remote":"127.0.0.1:57960","session":"4.1.1.582"}}
You need to set CONCOURSE_TSA_HOST: concourse-web on the worker as environment variable so that it knows which host to connect to. Right now it is trying to connect to the web part on localhost, but that is incorrect.
Another issue with your configuration is that you're trying to connect to Postgres through localhost: CONCOURSE_POSTGRES_DATA_SOURCE: |-
postgres://odoo:odoo#localhost:5432/concourse?sslmode=disable,
but your Postgres instance is running on the host machine. The host machine is not available on localhost inside a docker container as a docker container has it's own private network. It should instead be:
CONCOURSE_POSTGRES_DATA_SOURCE: |-
postgres://odoo:odoo#my.internal.ip:5432/concourse?sslmode=disable
|-
postgres://odoo:odoo#localhost:5432/concourse?sslmode=disable
should have that entire prefix removed. Replace with
CONCOURSE_POSTGRES_DATA_SOURCE: postgres://odoo:odoo#localhost:5432/concourse?sslmode=disable