Program in docker can't connect to PostgreSQL in docker - docker

I run my Golang API and PostgreSQL with docker-compose.
My log with error connection refused:
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
api_1 | Unable to connect to database: dial tcp 0.0.0.0:5432: connect: connection refusedartpaper_api_1 exited with code 1
db_1 | 2021-12-26 15:18:35.152 UTC [1] LOG: starting PostgreSQL 14.1 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027, 64-bit
db_1 | 2021-12-26 15:18:35.152 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2021-12-26 15:18:35.152 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2021-12-26 15:18:35.216 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2021-12-26 15:18:35.329 UTC [22] LOG: database system was shut down at 2021-12-26 15:05:11 UTC
db_1 | 2021-12-26 15:18:35.515 UTC [1] LOG: database system is ready to accept connections
My config:
config := pgx.ConnConfig{
Host: "0.0.0.0",
Port: 5432,
Database: "artpaper",
User: "admin",
Password: "admin1",
}
I think mistake in docker-compose.yml or Dockerfile for API, because on correct ports docker ps:
0.0.0.0:5432->5432/tcp artpaper_db_1
Dockerfile for API:
FROM golang:1.17.5-alpine3.15
WORKDIR /artpaper
COPY ./ ./
RUN go mod download // download dependencies
RUN go build ./cmd/main/main.go // compile code to one binary file
EXPOSE 8080
CMD [ "./main" ] // run binary file
docker-compose.yml:
version: "3.3"
services:
db:
image: postgres:14.1-alpine3.15
volumes:
- ./data/db:/var/lib/postgresql/data
environment:
- POSTGRES_DB=artpaper
- POSTGRES_USER=admin
- POSTGRES_PASSWORD=admin1
ports:
- 5432:5432
api:
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080"
depends_on:
- db
Password and user in API config like in docker-compose.yml
Hostname from container with api is 0.0.0.0:5432

In your Golang application try using: db:5432, not 0.0.0.0:5432.
version: '3.8'
services:
db:
image: postgres:14.1-alpine3.15
volumes:
- ./data/db:/var/lib/postgresql/data
environment:
- POSTGRES_DB=artpaper
- POSTGRES_USER=admin
- POSTGRES_PASSWORD=admin1
ports:
- 5432:5432
api:
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080"
depends_on:
- db
debug:
image: postgres:14.1-alpine3.15
command: sleep 1d
Try connect to the database within:
docker-compose up -d
docker-compose exec debug ash -c 'psql -h db -U admin --dbname artpaper'

First: In config database host should be a db like in docker-compose.yml
Second: Postgres as a program does not have time to turn on inside the container. I added in api code delay before connection to database.

Related

API cannot connect to postgres database using docker compose

I'm trying to use docker compose to connect my ASP.Net Core web api to my postgres database.
Here is my docker-compose file
version: "3.9"
services:
db:
image: postgres:15.1-alpine
container_name: db
restart: always
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: "MyPassword123"
ports:
- "5432:5432"
volumes:
- db_data:/var/lib/postgresql/data
api:
image: ${DOCKER_REGISTRY-}api
container_name: api
environment:
ASPNETCORE_ENVIRONMENT: Development
ConnectionStrings_Default: Server=db;Port=5432;Database=DB;User Id=postgres;Password=MyPassword123;
ports:
- "7001:80"
depends_on:
- db
build:
context: .
dockerfile: src/Services/Api/Dockerfile
volumes:
db_data:
Here are the logs of my postgres DB
db | 2023-01-19 09:46:17.887 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db | 2023-01-19 09:46:17.887 UTC [1] LOG: listening on IPv6 address "::", port 5432
db | 2023-01-19 09:46:17.906 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db | 2023-01-19 09:46:17.921 UTC [50] LOG: database system was shut down at 2023-01-19 09:46:17 UTC
db | 2023-01-19 09:46:17.929 UTC [1] LOG: database system is ready to accept connections
But on the API side I get the messsage:
Unhandled exception. Npgsql.NpgsqlException (0x80004005): Failed to connect to [::1]:5432
The db is correctly created and I can access it from my machine using PGAdmin using localhost and port 5432.
Should I create a user defined bridge network? I'm pretty sure everything should work like this.
Any help would be greatly appreciated.
The [::1] in the error message from the API is the IPv6 address for localhost. So it looks like it's not picking up the server name correctly from your connection string.
Looking at the docs, it seems like the keyword for the server is 'Host' and not 'Server', so try
ConnectionStrings_Default: Host=db;Port=5432;Database=DB;User Id=postgres;Password=MyPassword123;
If that doesn't help, then please provide more information on how you take the connection string and establish the connection to the database in C#.

Docker takes a long time to start mysql

I have the follow docker compose file:
version: '3.9'
services:
db-production:
container_name: mysql-production
image: mysql:latest
restart: always
environment:
MYSQL_HOST: localhost
MYSQL_DATABASE: dota2learning-db
MYSQL_ROOT_PASSWORD: toor
ports:
- "3306:3306"
volumes:
- ./data/db-production:/home/db-production
db-testing:
container_name: mysql-testing
image: mysql:latest
restart: always
environment:
MYSQL_HOST: localhost
MYSQL_ROOT_PASSWORD: toor
ports:
- "3307:3306"
volumes:
- ./data/db-testing:/home/db-testing
volumes:
data:
I also have a sql script to dump my database. The problem that's docker take a long time to start mysql and the script don't work.
I tried add the follow command on docker compose file:
command: mysql --user=root --password=toor dota2learning-db < /home/db-production/dumb-db-production.sql
This command does not work because it tries to run before the mysql server is working.
I know because as soon as I created the container I got into it and tried to log into mysql and it wasn't available yet:
sudo docker exec -it mysql-production bash
on container:
mysql --user=root --password=toor
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
I also tried to start MySQL manually:
root#4c91b5407561:/# mysqld start
2022-06-20T14:56:18.448123Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.29) starting as process 97
2022-06-20T14:56:18.451281Z 0 [ERROR] [MY-010123] [Server] Fatal error: Please read "Security" section of the manual to find out how to run mysqld as root!
2022-06-20T14:56:18.451346Z 0 [ERROR] [MY-010119] [Server] Aborting
2022-06-20T14:56:18.451514Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.29) MySQL Community Server - GPL.
That's add the follow command on docker compose don't work:
command: mysqld start
NOTE:
But I know that if I wait 1 or 2 minutes mysql will be available to run the script though, I want to run this script automatically and not manually.
When I add the commands on docker compose the docker container keeps restarting forever, because it keeps trying to execute the commands and mysql is not available yet.

Docker containers in the same network cannot communicate with container names (M1 Mac)

I am trying to run express, Prisma ORM, and postgre applications in Docker.
I have two containers in the same network, but they cannot communicate with each other unless they use the actual IP address instead of the container name. Here is my docker-compose.yml file:
version: '3.8'
services:
web:
container_name: urefer-backend
networks:
- backend
build: .
volumes:
- .:/usr/app/
- /usr/app/node_modules
ports:
- "5000:5000"
depends_on:
- db
environment:
DATABASE_URL: postgresql://postgres:docker#urefer-db:5432/urefer?schema=public
stdin_open: true # docker run -i
tty: true # docker run -t
db:
networks:
- backend
image: postgres:latest
environment:
POSTGRES_USER: ${POSTGRES_USER:-postgres}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-docker}
POSTGRES_DB: "urefer"
container_name: urefer-db
restart: always
ports:
- 5432:5432
volumes:
- database-data:/var/lib/postgresql/data/
volumes:
database-data:
networks:
backend:
The urefer-backend container and the urefer-db container are in the same "backend" network. However, when I run docker-compose up --build, I always have a connection issue:
urefer-backend | Error: Error in migration engine: Can't reach database server at `urefer-db`:`5432`
urefer-backend |
urefer-backend | Please make sure your database server is running at `urefer-db`:`5432`.
When I replace "urefer-db" with the actual IP address of the db server, then it connects successfully. This I think means that there is something wrong with the DNS setup.
Could anyone please help me connect the two containers without using the actual IP address? the IP address for DB is always changing whenever I stop and restart the container, so it is really bothering to use.
#edit:
I got a suggestion from a comment to put all log and errors I got on the console.
urefer-db |
urefer-db | PostgreSQL Database directory appears to contain a database; Skipping initialization
urefer-db |
urefer-db | 2021-10-01 14:44:57.165 UTC [1] LOG: starting PostgreSQL 14.0 (Debian 14.0-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
urefer-db | 2021-10-01 14:44:57.165 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
urefer-db | 2021-10-01 14:44:57.165 UTC [1] LOG: listening on IPv6 address "::", port 5432
urefer-db | 2021-10-01 14:44:57.168 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
urefer-db | 2021-10-01 14:44:57.171 UTC [28] LOG: database system was shut down at 2021-10-01 14:44:49 UTC
urefer-db | 2021-10-01 14:44:57.177 UTC [1] LOG: database system is ready to accept connections
urefer-backend | Prisma schema loaded from prisma/schema.prisma
urefer-backend | Datasource "db": PostgreSQL database "urefer", schema "public" at "urefer-db:5432"
urefer-backend |
urefer-backend | Error: P1001: Can't reach database server at `urefer-db`:`5432`
urefer-backend |
urefer-backend | Please make sure your database server is running at `urefer-db`:`5432`.
urefer-backend | Prisma schema loaded from prisma/schema.prisma
urefer-backend |
When the backend service is up, it runs the command npx prisma migrate dev --name init, and this command creates this connection error.
You should use host name instead of container name. Containers uses hostname when communicate each other. So if you add hostname docker-compose for container it will work
version: '3.8'
services:
web:
container_name: urefer-backend
networks:
- backend
build: .
volumes:
- .:/usr/app/
- /usr/app/node_modules
ports:
- "5000:5000"
depends_on:
- db
environment:
DATABASE_URL: postgresql://postgres:docker#urefer-db:5432/urefer?schema=public
stdin_open: true # docker run -i
tty: true # docker run -t
db:
networks:
- backend
image: postgres:latest
environment:
POSTGRES_USER: ${POSTGRES_USER:-postgres}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-docker}
POSTGRES_DB: "urefer"
container_name: urefer-db
hostname: urefer-db
restart: always
ports:
- 5432:5432
volumes:
- database-data:/var/lib/postgresql/data/
volumes:
database-data:
networks:
backend:

Docker-compose Rails backend + postgresql could not translate host name "db" to address: Name or service not known

whenever I run the two containers (with docker-compose or individually, postgres first), the moment the backend container runs the db:create (via instruction RUN RAILS_ENV=production bin/rails db:create && RAILS_ENV=production bin/rails db:migrate in its Dockerfile or via CLI rails db:create when running the dangling image interactively) I have this message in the terminal:
could not translate host name "db" to address: Name or service not known
Couldn't create 'pia_production' database. Please check your configuration.
Here is my docker-compose yml:
version: '3'
services:
db:
build: ./db/
volumes:
- ./tmp/db:/var/lib/postgresql/data
restart: always
expose:
- "5432"
ports:
# exposed port:postgres port
- "5432:5432"
env_file:
- ./db/.env
# these environment variables should be defined in the .env file
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
POSTGRES_USER: ${POSTGRES_USER}
PG_DATA: ${PG_DATA}
networks:
- my-network
back:
build: ./pia-rails/
volumes:
- ./logs/:/home/adminpia/pia-back/log/
ports:
- 3000:3000
links:
- db:db
depends_on:
- db
networks:
- my-network
env_file:
- ./pia-rails/.env
networks:
my-network:
driver: bridge
I have verified the start order, I have tried wait-for-it (github.com/vishnubob/wait-for-it.git) and a script of my own:
#!/bin/bash
a=0
t=0
while ((a < 1))
do
t=$(( t + 1 ))
echo -ne "No process on port 5432, waiting $t s for postgresql...\r"
echo -ne "\b\b"
sleep 1s
b=$(netstat -tulpn | grep 5432)
if [ ! -z "$b" ]
then
a=1
echo "A process is listening on 5432, presuming postgresql"
exit 0
fi
done
which I know it's redundant info since running docker-compose up db and then the backend produces the same outcome.
I've looked for answers here but found none regarding Rails+posgresql specifically and none which wasn't resolved by waiting for the postgresql container to be listening on port 5432 (after configuring the port obviously).
db image: Postgres:12-alpine
backend image: ruby 2.7.0, rails 6.0.3.4, uname -a:
Linux 8e59808d6845 5.4.0-52-generic #57~18.04.1-Ubuntu SMP Thu Oct 15 14:04:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Docker version 19.03.13, build 4484c46d9d
In reply to #3DPrintScanner, I can see my db container :
Name Command State Ports
----------------------------------------------------------------
cnildocker_db_1 docker-entrypoint.sh postgres Exit 0
but it doesn't seem to be using the 5432 port even though docker run -p 5432:5432 --env-file ./.env -v PG_DATA -it postgres:12-alpine on its own seems to work properly:
PostgreSQL init process complete; ready for start up.
2020-11-02 13:12:04.054 UTC [1] LOG: starting PostgreSQL 13.0 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
2020-11-02 13:12:04.055 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2020-11-02 13:12:04.056 UTC [1] LOG: listening on IPv6 address "::", port 5432
2020-11-02 13:12:04.060 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-11-02 13:12:04.066 UTC [45] LOG: database system was shut down at 2020-11-02 13:12:03 UTC
2020-11-02 13:12:04.075 UTC [1] LOG: database system is ready to accept connections
Here is my database.yml from the backend image:
development:
adapter: postgresql
host: db
encoding: unicode
database: pia_development
pool: 5
username:
password:
template: template0
test:
adapter: postgresql
host: db
encoding: unicode
database: pia_test
pool: 5
username:
password:
template: template0
production:
adapter: postgresql
host: db
encoding: unicode
database: pia_production
pool: 5
username: pia
password: #{PIA_PWD}
template: template0

Docker-Compose freezing on database-creation

I wish to expand a web-application using docker (I'm beginner in Docker). My application demands PostgreSQL. Therefore I decided to use the docker-compose.
The docker-compose.yaml looks like:
version: '3.8'
services:
db:
image: postgres
command: "postgres -c listen_addresses='*'"
environment:
- POSTGRES_DB=CatsQMS
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=12345qwerty
application:
build: .
ports:
- "8080:8080"
depends_on:
- db
Also I have the file config.yaml which configures my web-application, it looks like this:
database_config:
host: db
user: postgres
password: 12345qwerty
port: 5432
database: CatsQMS
# Unimportant stuff
And when I lunch the docker-compose, using docker-compose up the build is freezing at this point:
Recreating cats_queue_management_system_db_1 ... done
Recreating cats_queue_management_system_application_1 ... done
Attaching to cats_queue_management_system_db_1, cats_queue_management_system_application_1
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2020-05-20 13:42:51.628 UTC [1] LOG: starting PostgreSQL 12.3 (Debian 12.3-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1 | 2020-05-20 13:42:51.628 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2020-05-20 13:42:51.628 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2020-05-20 13:42:51.635 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2020-05-20 13:42:51.660 UTC [24] LOG: database system was shut down at 2020-05-20 13:39:45 UTC
db_1 | 2020-05-20 13:42:51.673 UTC [1] LOG: database system is ready to accept connections
Perhaps it's an important thing, my Dockerfile:
FROM python:3.8
RUN mkdir /docker-entrypoint-initdb.d/
COPY ./application/sources/database/init.sql /docker-entrypoint-initdb.d/
RUN mkdir /app
WORKDIR /app
COPY . /app
RUN pip install -r requirements.txt
RUN python3 setup.py develop
ENTRYPOINT ["start_app", "-d"]
Where have I admitted a mistake?
Your application and PostgreSQL database are running and should work as expected. But Docker attached after running the containers to the db container.
You can avoid this by using the option -d or --detach on docker-compose up:
The docker-compose up command aggregates the output of each container (essentially running docker-compose logs -f). When the command exits, all containers are stopped. Running docker-compose up -d starts the containers in the background and leaves them running.
So your command looks like this:
docker-compose up -d
I don't think the issue is with the database container. It looks like it starts up normally and is listening for connections.
What is the ENTRYPOINT in your Dockerfile executing, "start_app"?
The -d argument looks like it may be starting the website as a daemon (background process) which doesn't give docker a process to hook into.
Maybe try
ENTRYPOINT ["start_app"]
Is the issue that your web app cant talk to the database? If so it might be that a) they don't share a network, and b) you're trying to connect on port 5432, but the postgres db container is not exposing or publishing this port.
I've added a few things to your docker-compose file that might make it work...
version: '3.8'
networks:
test:
services:
db:
image: postgres
command: "postgres -c listen_addresses='*'"
environment:
- POSTGRES_DB=CatsQMS
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=12345qwerty
ports:
- 5432:5432
networks:
- test
application:
build: .
ports:
- 8080:8080
networks:
- test
depends_on:
- db

Resources