How do you launch Postgres from Docker, using docker-compose?
My docker-compose.yml looks like:
version: "3.6"
services:
db:
container_name: db
image: postgres:14-alpine
environment:
- POSTGRES_USER=test
- POSTGRES_PASSWORD=test
- POSTGRES_DB=test
ports:
- "5432:5432"
command: -c fsync=off -c synchronous_commit=off -c full_page_writes=off --max-connections=200 --shared-buffers=4GB --work-mem=20MB
tmpfs:
- /var/lib/postgresql
web:
container_name: web
build:
context: ..
dockerfile: test_tools/Dockerfile
shm_size: '2gb'
volumes:
- /dev/shm:/dev/shm
depends_on:
- db
This is a simple test environment to mimic a web server and a database server.
Yet when I build this, it fails with:
Creating db ... error
ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint db (bdaebf844ee8ddd593b6bc75733d8aa6196112b62f7909be060017a9a33b3c34): Error starting userland proxy: listen tcp4 0.0.0.0:5432: bind: address already in use
Why is my Postgres container trying to allocate a port on the host?
I do have Postgres running on port 5432 of the host, but why would this be interfering? These are just test containers that only need to talk to each other, and should not be accessible to the host, much less allocate host ports.
I've confirmed with docker ps -a that there are no other containers that might also be consuming port 5432.
ports:
- 5432
will start your Postgres, but on a random (free) host port.
Try to map postgres to different port on host for example
ports:
5432:15432
will make your db works on port 15432 on your host.
Related
I have a problem about network in docker. In the docker-compose.yml includes 2 instance below
webserver (frontend + backend)
database
But i tried to bridge network and default but not working at all.The backend cannot connect to database show error "connection refuse". then i tried to docker exec -t .. into webserver and then ping to database it show "timeout".
I cannot connect database with ip address (i got a database ip address from docker exec and then hostname -i) but i connected success using "localhost"
this my docker-compose.yml
version: '3.8'
services:
postgres_server:
container_name: postgres14-4_container
image: postgres:14.4
command: postgres -c 'max_connections=200'
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- '5222:5432'
volumes:
- db:/var/lib/postgresql14/data
networks:
- web_network
webserver:
container_name: frontend_backend_container
image: webserver
ports:
- '9090:80'
- '8081:8081'
env_file:
- backend_env
depends_on:
- postgres_server
restart: always
networks:
- web_network
volumes:
db:
driver: local
networks:
web_network:
driver: bridge
To configure remote connections to postgres, you have to adjust pg_hba.conf. For example add:
# Remote access
host all all 0.0.0.0/0 trust
where is your backend_env file?
I guess you have there the host + port to connect to the db.
You don't need to define anything special (like the bridge).
The webserver container should be able to access the postgres_server via postgres_server:5432 (not localhost and not 5222).
I build a website using Strapi and Gatsby, everythings works well when I try to connect to a remote database, but I'm trying to create a db inside a container and so far no luck.
Essentially, what I did is create the following docker-compose:
version: '3'
services:
backend:
container_name: myapp_backend
build: ./backend/
ports:
- '3002:3002'
volumes:
- ./backend:/usr/src/myapp/backend
- /usr/src/myapp/backend/node_modules
environment:
- APP_NAME=myapp_backend
- DATABASE_CLIENT=mysql
- DATABASE_HOST=db
- DATABASE_PORT=3307
- DATABASE_NAME=myapp_db
- DATABASE_USERNAME=johnny
- DATABASE_PASSWORD=stecchino
- DATABASE_SSL=false
- DATABASE_AUTHENTICATION_DATABASE=myapp_db
- HOST=localhost
depends_on:
- db
restart: always
db:
container_name: myapp_mysql
image: mysql:5.7
volumes:
- ./db.sql:/docker-entrypoint-initdb.d/db.sql
restart: always
ports:
- 3307:3307
environment:
MYSQL_ROOT_PASSWORD: 5!JF6!FgAkvt
MYSQL_DATABASE: myapp_db
MYSQL_USER: johnny
MYSQL_PASSWORD: stecchino
command: mysqld --character-set-server=utf8 --collation-server=utf8_general_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: 'myapp_phpmyadmin'
links:
- db
environment:
PMA_HOST: db
PMA_PORT: 3307
ports:
- '8081:80'
volumes:
- /sessions
depends_on:
- db
frontend:
container_name: myapp_frontend
build: ./frontend/
ports:
- '3001:3001'
depends_on:
- backend
volumes:
- ./frontend:/usr/src/myapp/frontend
the backend service contains the Strapi application, the db service contains the mysql instance which runs on the port 3307 'cause 3306 is already in use.
Then I have also installed phpmyadmin, and last but not least the Gastby site. When I run using docker-compose up --build, and try to access to phpmyadmin using:
http://localhost:8081/index.php
with the following credentials:
user: johnny
pwd: stecchino
I get:
MySQL mysqli::real_connect():(HY000/2002): Connection refused
now, what I did for fix that situation is pass the port 3306 instead of 3307 to backend and phpmyadmin service. And magically, everything works. But why? I have mapped container and host to 3307...
There are 2 things happening here.
Mysql is running on port 3306.
This is because you never told the mysql container to run on port 3307. The default configuration is running on 3306.
phpadmin can connect to mysql at port 3306.
Of course it can. This is because when you define multiple services within the same docker-compose file, they start on the same network. This means that they can see and connect to each other's internal ports without the need for external port binding like 3306:3306
I would suggest to keep port bindings only for services that you want access outside the docker environment (like the UI), and for internal components just expose the port like this
expose:
- 3306
Both answers are useful, I am particularly fond of Manish's answer
I wanted to add some additional wording:
There are the internal docker networks which nothing from the outside can gain access to. From inside any given service (or container), you can reach every other service (or container) via:
<service-name>:<port>/path/of/resources
<container-name>:<port>/path/of/resources
In order to access resources inside the docker network from outside of docker, whether that is from your host environment, or farther upstream on the internet, the docker daemon needs to bind to host ports, and then forward information received on those ports to a docker service (and ultimately a docker container).
In your docker-compose.yml when you do the 3307:3307 you are telling the docker daemon to listen on port 3307, and forward to your db service internally on it's port 3307.
However, from what we can all see, mysql is still internally (that is, inside the container) listening for traffic on port 3306. Any containers or services on the same docker networks as your db service (mysql running container(s)) would be able to access mysql via something like:
<driver>:mysql://db:3306/<dbname>
If you wanted all host traffic and docker network traffic to access mysql on port 3307, you would also need to configure mysql to listen on port 3307 instead of 3306. That tidbit of information does not appear to be in your question at the time of writing.
I hope the additional information helps! It's a topic I chat often about when talking docker with folks.
Because 3306 is the exposed port by the official Dockerfile.
What you can do is to map the port that is running MySQL to another port on your host: 3307:3306 for instance (always host:container)
I am getting started with docker and docker-compose. I have the tutorials and I use docker-compose.yml file to run one of my sites in my local machine.
I can see my site running by going to http://localhost
My problem now is trying to run more than one site. If one of my sites is running and I try to run another site using docker-compose up -d I get the following error.
$ docker-compose up -d
Creating network "exampleCOM_default" with driver "bridge"
Creating exampleCOMphp-fpm ...
Creating exampleCOMmariadb ... error
ERROR: for exampleCOMmariadb Cannot start service db: driver failed programming external connectivity on endpoint exampleCOMmariadb (999572f33113c9fce034b4ed72aaCreating exampleCOMphp-fpm ... done
eady allocated
Creating exampleCOMnginx ... error
ERROR: for exampleCOMnginx Cannot start service nginx: driver failed programming external connectivity on endpoint exampleCOMnginx (9dc04f8b06825d7ff535afb1101933be7435c68f4350f845c756fc93e1a0322c): Bind for 0.0.0.0:443 failed: port is already allocated
ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint exampleCOMmariadb (999572f33113c9fce034b4ed72aa072708f6f477eb2af8ad614c0126ca457b64): Bind for 0.0.0.0:3306 failed: port is already allocated
ERROR: for nginx Cannot start service nginx: driver failed programming external connectivity on endpoint exampleCOMnginx (9dc04f8b06825d7ff535afb1101933be7435c68f4350f845c756fc93e1a0322c): Bind for 0.0.0.0:443 failed: port is already allocated
Encountered errors while bringing up the project.
This is my docker-compose file. I am using LEMP stack (PHP, NGINX, MARIADB)
version: '3'
services:
db:
container_name: ${SITE_NAME}_mariadb
build:
context: ./mariadb
volumes:
- ./mariadb/scripts:/docker-entrypoint-initdb.d
- ./.data/db:/var/lib/mysql
- ./logs/mariadb:/var/log/mysql
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
MYSQL_DATABASE: ${MYSQL_DATABASE}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
ports:
- '${MYSQL_PORT:-3306}:3306'
command:
'mysqld --innodb-flush-method=fsync'
networks:
- default
restart: always
nginx:
container_name: ${SITE_NAME}_nginx
build:
context: ./nginx
args:
- 'php-fpm'
- '9000'
volumes:
- ${APP_PATH}:/var/www/app
- ./logs/nginx/:/var/log/nginx
ports:
- "80:80"
- "443:443"
depends_on:
- php-fpm
networks:
- default
restart: always
php-fpm:
container_name: ${SITE_NAME}_php-fpm
build:
context: ./php7-fpm
args:
TIMEZONE: ${TIMEZONE}
volumes:
- ${APP_PATH}:/var/www/app
- ./php7-fpm/config/php.ini:/usr/local/etc/php/php.ini
environment:
DB_HOST: db
DB_PORT: 3306
DB_DATABASE: ${MYSQL_DATABASE}
DB_USERNAME: ${MYSQL_USER}
DB_PASSWORD: ${MYSQL_PASSWORD}
networks:
- default
restart: always
networks:
default:
driver: bridge
The host port you have mapped to is preventing you from starting another instance of the service even though the docker-compose creates a private network.
You can solve this problem by using random host ports assigned by docker-compose.
The ports entry in docker-compose is
ports
host_port:container_port
If you specify only the container port host port is randomly assigned. See here
You can provide the host_port values in ranges.
In below example, i've run the nginx containers and started multiple nginx containers that are automatically exposed to host ports based on the range values [30000-30005].
Command:
docker run -p 30000-30005:80 --name nginx1 -d nginx
Output:
9083d5fc97e0 nginx ... Up 2 seconds 0.0.0.0:30001->80/tcp nginx1
f2f9de1efd8c nginx ... Up 24 seconds 0.0.0.0:30000->80/tcp nginx
I am trying to access a docker container from another container using localhost address.
The compose file is pretty simple. Both containers ports are exposed.
There are no problems when building.
In my host machine I can successfully execute curl http://localhost:8124/ and get a response.
But inside the django_container when trying the same command I get Connection refused error.
I tried adding them in the same network, still result didn't change.
Well if I try to execute with the internal ip of that container like curl 'http://172.27.0.2:8123/' I get the response.
Is this the default behavior? How can I reach clickhouse_container using localhost?
version: '3'
services:
django:
container_name: django_container
build: ./django
ports:
- "8007:8000"
links:
- clickhouse:clickhouse
volumes:
- ./django:/usr/src/run
command: bash /usr/src/run/run.sh
clickhouse:
container_name: clickhouse_container
build: ./clickhouse
ports:
- "9001:9000"
- "8124:8123"
- "9010:9009"
So with this line here - "8124:8123" you're mapping the port of clickhouse container to localhost 8124. Which allows you to access clickhouse from localhost at port 8124.
If you want to hit clickhouse container from within the dockerhost network you have to use the hostname for the container. This is what I like to do:
version: '3'
services:
django:
hostname: djano
container_name: django
build: ./django
ports:
- "8007:8000"
links:
- clickhouse:clickhouse
volumes:
- ./django:/usr/src/run
command: bash /usr/src/run/run.sh
clickhouse:
hostname: clickhouse
container_name: clickhouse
build: ./clickhouse
ports:
- "9001:9000"
- "8124:8123"
- "9010:9009"
If you make the changes like I have made above you should be able to access clickhouse from within the django container like this curl http://clickhouse:8123.
As in #Billy Ferguson's answer, you can visit using localhost in host machine just because: you define a port mapping to route localhost:8124 to clickhouse:8123.
But when from other container(django), you can't. But if you insist, there is a ugly workaround: share host's network namespace with network_mode, but with this the django container will just share all network of host.
services:
django:
hostname: djano
container_name: django
build: ./django
ports:
- "8007:8000"
links:
- clickhouse:clickhouse
volumes:
- ./django:/usr/src/run
command: bash /usr/src/run/run.sh
network_mode: "host"
It depends of config.xml settings. If in config.xml <listen_host> 0.0.0.0</listen_host> you can use clickhouse-client -h your_ip --port 9001
I have a website hosted on shared hosting on production. The website connects to the database via localhost in the code. In my docker-compose I have a php:5.6-apache and mysql:5.6 instance.
Is there any way to tell docker-compose to have port 3306 on the web container port forwarded to 3306 on the db container, so that when the web container tries to connect to localhost on 3306 it gets sent to db on 3306 and also share port 80 on the web container to the outside world?
Current docker-compose.yml:
version: "3"
services:
web:
build: .
#image: php:5.6-apache
ports:
- "8080:80"
environment:
- "APP_LOG=php://stderr"
- "LOG_LEVEL=debug"
volumes:
- .:/var/www/html
network_mode: service:db # See https://stackoverflow.com/a/45458460/95195
# networks:
# - internal
working_dir: /var/www
db:
image: mysql:5.6
ports:
- "3306:3306"
environment:
- "MYSQL_XXXXX=*****"
volumes:
- ./provision/mysql/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
# networks:
# - internal
networks:
internal:
driver: bridge
Current error:
ERROR: for web Cannot create container for service web: conflicting options: port publishing and the container type network mode
Yes it is possible. You need to use the network_mode option. See the below example
version: '2'
services:
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: root
ports:
- "80:80"
- "3306:3306"
app:
image: ubuntu:16.04
command: bash -c "apt update && apt install -y telnet && sleep 10 && telnet localhost 3306"
network_mode: service:db
outputs
app_1 | Trying 127.0.0.1...
app_1 | Connected to localhost.
app_1 | Escape character is '^]'.
app_1 | Connection closed by foreign host.
network_mode: service:db instructs docker to not assign the app services it own private network. Instead let it join the network of db service. So any port mapping that you need to do, needs to happen on the db service itself.
The way I usually use it is different, I create a base service which runs a infinite loop and the db and app service both are launched on base service network. All ports mapping need to happen at the base service.