docker-compose not starting postgres and gives error - docker

Question

I'm really new to Docker (also postgres) and still finding my feet. I get an error and can't seem to get one of my postgres services running, although when I start it, I'm able to access pgadmin and airflow via the browser. I think there is some sort of conflict happening but I'm not sure where. I have a docker-compose.yml file that starts a few containers, as well as the postgres one in question which has the servce name db:
version: '3.7'
services:
postgres:
image: postgres:9.6
environment:
- POSTGRES_USER=airflow
- POSTGRES_PASSWORD=airflow
- POSTGRES_DB=airflow
logging:
options:
max-size: 10m
max-file: "3"
db:
image: postgres:13.0-alpine
restart: always
environment:
POSTGRES_DB: postgres
POSTGRES_USER: admin_user
POSTGRES_PASSWORD: secret_password
# PGDATA: /var/lib/postgresql/data
volumes:
- ./db-data:/var/lib/postgresql/data
ports:
- "5433:5432"
pgadmin:
image: dpage/pgadmin4:4.27
restart: always
environment:
PGADMIN_DEFAULT_EMAIL: admin_user#test_email.com
PGADMIN_DEFAULT_PASSWORD: test_password
PGADMIN_LISTEN_PORT: 1111
ports:
- "1111:1111"
volumes:
- pgadmin-data:/var/lib/pgadmin
links:
- "db:pgsql-server"
webserver:
image: l/custom_airflow:1.5
container_name: l_custom_airflow
restart: always
depends_on:
- postgres
environment:
- LOAD_EX=n
- EXECUTOR=Local
logging:
options:
max-size: 10m
max-file: "3"
volumes:
- ./dags:/usr/local/airflow/dags
- ./db-data:/usr/local/airflow/db-data
- ./pgadmin-data:/usr/local/airflow/pgadmin-data
ports:
- "8080:8080"
command: webserver
healthcheck:
test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
interval: 30s
timeout: 30s
retries: 3
volumes:
db-data:
pgadmin-data:
The relevant part is this:
db:
image: postgres:13.0-alpine
restart: always
environment:
POSTGRES_DB: postgres
POSTGRES_USER: admin_user
POSTGRES_PASSWORD: secret_password
# PGDATA: /var/lib/postgresql/data
volumes:
- ./db-data:/var/lib/postgresql/data
ports:
- "5433:5432"
[I already have two version of postgres on my local machine, and I saw that they use ports 5432 and then 5433, so it looks like the latest one goes to 5433. Similarly, I have another service (airflow) that depends on an older version of postgres to run, so I assume since that one comes first it takes 5432, and then the new postgres service I want will likely be mapped to 5433 as default - please correct me if I'm wrong]
But when I run docker-compose up -d and check my containers with docker container ls -a I see that this particular container is continuously restarting. I ran docker logs --tail 50 --follow --timestamps pipeline_5_db_1 (the container name for the db service) and I see the following error:
2020-10-28T08:46:29.730973000Z chmod: /var/lib/postgresql/data: Operation not permitted 2020-10-28T08:46:30.468640800Z chmod: /var/lib/postgresql/data: Operation not permitted 2020-10-28T08:46:31.048144200Z chmod: /var/lib/postgresql/data: Operation not permitted 2020-10-28T08:46:31.803571400Z chmod: /var/lib/postgresql/data: Operation not permitted 2020-10-28T08:46:32.957604600Z chmod: /var/lib/postgresql/data: Operation not permitted 2020-10-28T08:46:34.885928500Z chmod: /var/lib/postgresql/data: Operation not permitted 2020-10-28T08:46:38.479922200Z chmod: /var/lib/postgresql/data: Operation not permitted 2020-10-28T08:46:45.384436400Z chmod: /var/lib/postgresql/data: Operation not permitted 2020-10-28T08:46:58.612202300Z chmod: /var/lib/postgresql/data: Operation not permitted
I googled the error and saw a couple of other SO posts but I can't see a clear explanation. This post and this post are a bit unclear to me (might be because I'm not so familiar), so I'm not sure how to use the responses to solve this issue.

Answer 1

You've got dbdata defined as a named volume at the bottom of the compose file but you're using ./dbdata within each service which is a bind mount. You might try using the named volume instead of the shared directory in your db and webserver services, like this:
volumes:
- db-data:/var/lib/postgresql/data
A bind mount should also work but can be troublesome if permissions on the mounted directory aren't quite right, which might be your problem.
The above also applies to pgadmin-data where the pgadmin service is using a named volume but webserver is using the bind mount (local directory). In fact, it's not clear why the webserver would need access to those data directories. Typically, a webserver would connect to the database via port 5432 (which doesn't even need to be mapped on the host). See for instance the bitnami/airflow docs on Docker Hub.