Can not share volume between services defined in docker-compose - docker

I'm running Docker for Mac Version 17.12.0-ce-mac55
I have a docker-compose file that I'm converting from docker-compose version 3 to version 2 to work better with Openshift.
---
version: '2'
services:
fpm:
build:
context: .
dockerfile: Dockerfile.openshift
args:
TIMEZONE: America/Chicago
APACHE_DOCUMENT_ROOT: /usr/local/apache2/htdocs
image: widget-fpm
restart: always
depends_on:
- es
- db
environment:
# taken from sample.env
- TIMEZONE=${TIMEZONE}
- APACHE_DOCUMENT_ROOT=/usr/local/apache2/htdocs
- GET_HOSTS_FROM=dns
- SYMFONY__DATABASE__HOST=db
- SYMFONY__DATABASE__PORT=5432
- SYMFONY__DATABASE__NAME=widget
- SYMFONY__DATABASE__USER=widget
- SYMFONY__DATABASE__PASSWORD=widget
- SYMFONY__DATABASE__SCHEMA=widget
- SYMFONY__DATABASE__DRIVER=pdo_pgsql
- SYMFONY_ENV=prod
- SYMFONY__ELASTICSEARCH__HOST=es:9200
- SYMFONY__SECRET=dsakfhakjhsdfjkhajhjds
- SYMFONY__LOCALE=en
- SYMFONY__RBAC__HOST=rbac
- SYMFONY__RBAC__PROTOCOL=http
- SYMFONY__RBAC__CONNECT__PATH=v1/connect
- SYMFONY__PROJECT_URL=http://localhost
- SYMFONY__APP__NAME=widget
- SYMFONY__CURRENT__API__VERSION=1
volumes:
# use docroot env to change this directory
- src:/usr/local/apache2/htdocs
- symfony-cache:/usr/local/apache2/htdocs/app/cache
- symfony-log:/usr/local/apache2/htdocs/app/logs
expose:
- "9000"
networks:
- client-network
- data-network
labels:
kompose.service.expose: "false"
webserver:
build: ./provisioning/webserver/apache
image: widget_web
restart: "no"
ports:
- "80"
- "443"
volumes_from:
- fpm:ro
depends_on:
- fpm
networks:
- client-network
labels:
com.singlehop.description: "Widget Service Web Server"
com.singlehop.development: "false"
kompose.service.expose: "true"
kompose.service.type: "nodeport"
db:
build: ./provisioning/database/postgres
image: widget_postgres
restart: always
volumes:
- data-volume:/var/lib/postgresql/data
environment:
POSTGRES_USER: widget
POSTGRES_PASSWORD: widget
expose:
- "5432"
networks:
- data-network
labels:
com.singlehop.description: "Widget Service Postgres Database Server"
com.singlehop.development: "false"
io.openshift.non-scalable: "true"
kompose.service.expose: "false"
kompose.volume.size: 100Mi
es:
image: elasticsearch:5.6
restart: always
environment:
#- cluster.name=docker-cluster
#- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
command: ["-Ecluster.name=docker-cluster", "-Ebootstrap.memory_lock=true"]
ulimits:
memlock:
soft: -1
hard: -1
labels:
com.singlehop.description: "Generic Elasticsearch5 DB"
com.singlehop.development: "false"
kompose.service.expose: "false"
kompose.volume.size: 100Mi
volumes:
- es-data:/usr/share/elasticsearch/data
expose:
- "9200-9300"
networks:
- data-network
migration:
# #todo can we use the exact same build/image I created above?
image: singlehop/widget-fpm
environment:
# taken from sample.env
- TIMEZONE=America/Chicago
- APACHE_DOCUMENT_ROOT=/usr/local/apache2/htdocs
- GET_HOSTS_FROM=dns
- SYMFONY__DATABASE__HOST=db
- SYMFONY__DATABASE__PORT=5432
- SYMFONY__DATABASE__NAME=widget
- SYMFONY__DATABASE__USER=widget
- SYMFONY__DATABASE__PASSWORD=widget
- SYMFONY__DATABASE__SCHEMA=widget
- SYMFONY__DATABASE__DRIVER=pdo_pgsql
- SYMFONY_ENV=prod
- SYMFONY__ELASTICSEARCH__HOST=es:9200
- SYMFONY__SECRET=dsakfhakjhsdfjkhajhjds
- SYMFONY__LOCALE=en
- SYMFONY__PROJECT_URL=http://localhost
- SYMFONY__APP__NAME=widget
- SYMFONY__CURRENT__API__VERSION=1
entrypoint: ["/usr/local/bin/php","app/console","--no-interaction"]
command: doctrine:migrations:migrate
volumes:
- src:/usr/local/apache2/htdocs
depends_on:
- db
networks:
- data-network
labels:
com.singlehop.description: "Widget Automated Symfony Migration"
com.singlehop.development: "false"
volumes:
src: {}
data-volume: {}
es-data: {}
symfony-cache: {}
symfony-log: {}
networks:
client-network:
data-network:
I'm using the fpm service to act like a data container and share PHP code to the webserver service. For some reason the named volume src is not being shared to the webserver service/container. I've tried both setting the volumes and using volumes_from.
I'm assuming this is possible and I feel like it would be bad practice to do another copy of the source code in the widget_web Dockerfile.

The depends_on in the fpm service is breaking the named volume src. When I removed the depends_on declaration it worked like I assumed it would work. I can't tell if this is a bug or working as designed.

Related

Adguard Home docker compose config and db missing

im trying to run adguard with docker compose. I created a lot more containers with docker compose but this one is not creating any files into the mapped folder.
I tried to rebuild the docker command of the official instruction but any time i recreate the container i end up at the setup page and all settings are deleted.
Any ideas?
This is my compose file:
version: "3"
volumes:
homematic_data:
external: true
networks:
homematic:
services:
samba:
image: dperson/samba
container_name: samba
restart: always
ports:
- "137:137/udp"
- "138:138/udp"
- "139:139/tcp"
- "445:445/tcp"
healthcheck:
disable: true
environment:
- TZ='Europe/Berlin'
- WORKGROUP=workgroup
- RECYCLE=false
- USER1=pi;PASSWORD;1000
- SHARE1=homematic_docker;/shares/homematic_docker;yes;no;yes;pi;pi
volumes:
- /home/pi:/shares/homematic_docker
networks:
- homematic
promtail:
image: grafana/promtail:latest
container_name: promtail
volumes:
- /var/log:/var/log
- ./promtail:/etc/promtail
restart: unless-stopped
command: -config.file=/etc/promtail/promtail-config.yml
networks:
- homematic
node-exporter:
image: quay.io/prometheus/node-exporter:latest
container_name: node_exporter
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
- /:/host:ro,rslave
command:
- '--path.rootfs=/host'
- '--path.procfs=/host/proc'
- '--path.sysfs=/host/sys'
- --collector.filesystem.ignored-mount-points
- "^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($$|/)"
ports:
- 9100:9100
networks:
- homematic
restart: always
###################### portainer
portainer:
image: portainer/portainer-ce:latest
container_name: portainer
restart: unless-stopped
security_opt:
- no-new-privileges:true
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./portainer:/data
ports:
- 9000:9000
adguard:
image: adguard/adguardhome
container_name: adguard
restart: unless-stopped
ports:
- 53:53/tcp
- 53:53/udp
- 67:67/udp
- 69:68/udp
- 80:80/tcp
- 443:443/tcp
- 443:443/udp
- 3000:3000/tcp
- 853:853/tcp
- 784:784/udp
- 853:853/udp
- 8853:8853/udp
- 5443:5443/tcp
- 5443:5443/udp
# environment:
# - TZ=Europe/Berlin
volumes:
- /home/pi/homematicDocker/adguard/work:/opt/adguardhome/work\
- /home/pi/homematicDocker/adguard/conf:/opt/adguardhome/conf\
# network_mode: host
raspberrymatic:
image: ghcr.io/jens-maus/raspberrymatic:3.67.10.20230117-27abde9
container_name: homematic
hostname: homematic-raspi
privileged: true
restart: unless-stopped
stop_grace_period: 30s
volumes:
- homematic_data:/usr/local:rw
- /lib/modules:/lib/modules:ro
- /run/udev/control:/run/udev/control
ports:
- "8080:80"
- "2001:2001"
- "2010:2010"
- "9292:9292"
- "8181:8181"
networks:
- homematic
Within the folder "/opt/adguardhome/work" I see a folder data with a database inside. After i finished the setup also the folder conf inside the container has a yaml file.
Unfortunately i copied the backslashes of the docker command into the volume mapping, thats was the problem why i didnt get any data. Thank you Mike!

The docker Container of caddy is in restarting state

This is docker-compose file that starts the containers all are working fine except the caddy.
version: '3'
services:
db:
image: postgres:latest
restart: always
expose:
- "5555"
volumes:
- pgdata:/var/lib/postgresql/data/
environment:
- POSTGRES_DB=chiefonboarding
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
networks:
- global
web:
image: chiefonboarding/chiefonboarding:latest
restart: always
expose:
- "9000"
environment:
- SECRET_KEY=somethingsupersecret
- BASE_URL=https://on.hr.gravesfoods.com
- DATABASE_URL=postgres://postgres:postgres#db:5432/chiefonboarding
- ALLOWED_HOSTS=on.hr.gravesfoods.com
- DEFAULT_FROM_EMAIL=hello#gravesfoods.com
depends_on:
- db
networks:
- global
caddy:
image: caddy:2.3.0-alpine
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- $PWD/Caddyfile:/etc/caddy/Caddyfile
- $PWD/site:/srv
- caddy_data:/data
- caddy_config:/config
networks:
- global
volumes:
pgdata:
caddy_data:
caddy_config:
networks:
global:
Also these are the logs it is generating:
[https://on.hr.gravesfoods.com:80] scheme and port violate convention "level":"info","ts":1656425557.6256478,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile" run: adapting config using caddyfile: server block 0, key 0 (https://on.hr.gravesfoods.com:80): determining listener address: [https://on.hr.gravesfoods.com:80] scheme and port violate convention.

Why am I unable to route to my API backend with Traefik

I had two container frontend (nginx :80) and backend (nodejs :3000).
I'm trying to redirect all path to my frontend : localhost/* to my frontend
Except one path to my backend API : localhost/v1/* to my backend
I secure my database container (mongodb) by allowing only communication with my backend
Here is my docker-compose.yml (I'm only using this)
version: '3'
services:
traefik:
image: traefik:v2.3
container_name: traefik
command:
- --api.insecure=true
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --entrypoints.web.address=:80
ports:
- "8080:8080"
- "443:443"
- "80:80"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
frontend:
image: registry.gitlab.com/test/frontend
container_name: frontend
build:
context: ../frontend/.
labels:
- traefik.enable=true
- traefik.http.routers.frontend.rule=PathPrefix(`/`)
- traefik.http.routers.frontend.entrypoints=web
networks:
- traefik-network
backend:
image: registry.gitlab.com/test/backend
container_name: backend
build:
context: ../backend/.
labels:
- traefik.enable=true
- traefik.http.routers.backend.rule=PathPrefix(`/v1`)
- traefik.http.routers.backend.service=backend
- traefik.http.routers.backend.entrypoints=web
- traefik.http.services.backend.loadbalancer.server.port=3000
command: yarn start
environment:
- MONGODB_URL=mongodb://mongodb:27017/backend
depends_on:
- mongodb
volumes:
- ../backend/.:/usr/src/backend
networks:
- traefik-network
- backend-network
mongodb:
image: mongo:4.2.1-bionic
container_name: mongodb
ports:
- 27017:27017
volumes:
- dbdata:/data/db
networks:
- backend-network
volumes:
dbdata:
networks:
backend-network:
traefik-network:
The problem is...
If the frontend (backend and traefik too) is turned on
the paths to localhost/* work (this is what I want),
but the paths to localhost/v1/* don't work (Problem here!).
If the frontend is turned off but traefik and backend is turned on
the paths to localhost/* don't work (of course, that's right),
but the paths to localhost/v1/* work (of course, this is what I want).
I've tried a lot of solutions but nothing seems to work the way I want it to.
What did I misunderstand?
Thanks for helping,
Have a nice day
Try to add the following labels to the backend service
- "traefik.http.routers.backend.rule=Host(`servicex.me`) && Path(`/v1`)"
and frontend
- traefik.http.routers.frontend.rule=Host(`servicex.me`)
you also need to add this line to your /etc/hosts
127.0.0.1 servicex.me
and make sure that you stop and start the services
Complete Example
version: '3'
services:
traefik:
image: traefik:v2.3
container_name: traefik
command:
- --api.insecure=true
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --entrypoints.web.address=:80
ports:
- "8080:8080"
- "443:443"
- "80:80"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
frontend:
image: registry.gitlab.com/test/frontend
container_name: frontend
build:
context: ../frontend/.
labels:
- traefik.enable=true
- traefik.http.routers.frontend.rule=Host(`servicex.me`)
- traefik.http.routers.frontend.entrypoints=web
- traefik.http.routers.frontend.service=frontend
- traefik.http.services.frontend.loadbalancer.server.port=80
networks:
- traefik-network
backend:
image: registry.gitlab.com/test/backend
container_name: backend
build:
context: ../backend/.
labels:
- traefik.enable=true
- "traefik.http.routers.backend.rule=Host(`servicex.me`) && Path(`/v1`)"
- traefik.http.routers.backend.service=backend
- traefik.http.routers.backend.entrypoints=web
- traefik.http.services.backend.loadbalancer.server.port=3000
command: yarn start
environment:
- MONGODB_URL=mongodb://mongodb:27017/backend
depends_on:
- mongodb
volumes:
- ../backend/.:/usr/src/backend
networks:
- traefik-network
- backend-network
mongodb:
image: mongo:4.2.1-bionic
container_name: mongodb
ports:
- 27017:27017
volumes:
- dbdata:/data/db
networks:
- backend-network
volumes:
dbdata:
networks:
backend-network:
traefik-network:
BTW, why do you need both traefik and nginx (Both are doing the same job), it would be better if you can replace one with another.
I added this label to my containers
traefik.docker.network=traefik-network
It works fine now

Nextcloud on Raspberry Pi via docker compose

I'm trying to run a Nextcloud instance on my Raspbery Pi 3B+ using a docker-compose file from this source: https://blog.ssdnodes.com/blog/installing-nextcloud-docker/
This works out of the box without any issues on a Ubuntu Server.
I've replaced the following images to be compatible with the arm infrastructure of the Pi:
jwilder/nginx-proxy:alpine with braingamer/nginx-proxy-arm or budry/jwilder-nginx-proxy-arm (I tried both)
jrcs/letsencrypt-nginx-proxy-companion with budry/jrcs-letsencrypt-nginx-proxy-companion-arm
mariadb with linuxserver/mariadb
nextcloud:latest with linuxserver/nextcloud
Unfortunately this doesn't work on the Pi, the Pi returns first a 502 Bad Gateway, then after some time the error ERR_TOO_MANY_REDIRECTS.
What am I doing wrong?
Thanks
My docker-compose.yml:
version: '3'
services:
proxy:
image: braingamer/nginx-proxy-arm
labels:
- "com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy=true"
container_name: nextcloud-proxy
networks:
- nextcloud_network
ports:
- 80:80
- 443:443
volumes:
- ./proxy/conf.d:/etc/nginx/conf.d:rw
- ./proxy/vhost.d:/etc/nginx/vhost.d:rw
- ./proxy/html:/usr/share/nginx/html:rw
- ./proxy/certs:/etc/nginx/certs:ro
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/tmp/docker.sock:ro
restart: unless-stopped
letsencrypt:
image: budry/jrcs-letsencrypt-nginx-proxy-companion-arm
container_name: nextcloud-letsencrypt
depends_on:
- proxy
networks:
- nextcloud_network
volumes:
- ./proxy/certs:/etc/nginx/certs:rw
- ./proxy/vhost.d:/etc/nginx/vhost.d:rw
- ./proxy/html:/usr/share/nginx/html:rw
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
restart: unless-stopped
db:
image: linuxserver/mariadb
container_name: nextcloud-mariadb
networks:
- nextcloud_network
volumes:
- db:/var/lib/mysql
- /etc/localtime:/etc/localtime:ro
environment:
- PUID=1000
- PGID=1000
- MYSQL_ROOT_PASSWORD=***PASSWORD***
- MYSQL_PASSWORD=***PASSWORD***
- MYSQL_DATABASE=nextcloud
- MYSQL_USER=nextcloud
ports:
- 3306:3306
restart: unless-stopped
app:
image: linuxserver/nextcloud
container_name: nextcloud-app
networks:
- nextcloud_network
depends_on:
- letsencrypt
- proxy
- db
volumes:
- nextcloud:/var/www/html
- ./app/config:/var/www/html/config
- ./app/custom_apps:/var/www/html/custom_apps
- ./app/data:/var/www/html/data
- ./app/themes:/var/www/html/themes
- /etc/localtime:/etc/localtime:ro
environment:
- PUID=1000
- PGID=1000
- TZ=Europe/London
- VIRTUAL_HOST=nextcloud.domain.tld
- LETSENCRYPT_HOST=nextcloud.domain.tld
- LETSENCRYPT_EMAIL=mail#nextcloud.domain.tld
volumes:
nextcloud:
db:
networks:
nextcloud_network:
The tutorial used a Nginx reverse proxy and Let’s Encrypt, for the latter you need a valid domain. If you look at your compose file for linuxserver/nextcloud under environment, it asks for a domain for VIRTUAL_HOST, LETSENCRYPT_HOST and LETSENCRYPT_EMAIL. It then tries to create a ssl certificate for the specified domain (nextcloud.domain.tld), which is not valid, so it doesn't work.
This was the case for me, so I just removed the proxy and ssl from my compose file and nextcloud works now :)
Here is my current working compose file:
version: '3'
services:
db:
image: tobi312/rpi-mariadb:10.5
container_name: nextcloud-mariadb
networks:
- nextcloud_network
volumes:
- db:/var/lib/mysql
- /etc/localtime:/etc/localtime:ro
environment:
- MYSQL_ROOT_PASSWORD=very_secure_password
- MYSQL_PASSWORD=very_secure_password
- MYSQL_DATABASE=nextcloud
- MYSQL_USER=nextcloud
restart: unless-stopped
app:
image: nextcloud:latest
container_name: nextcloud-app
networks:
- nextcloud_network
depends_on:
- db
volumes:
- nextcloud:/var/www/html
- ./app/config:/var/www/html/config
- ./app/custom_apps:/var/www/html/custom_apps
- ./app/data:/var/www/html/data
- ./app/themes:/var/www/html/themes
- /etc/localtime:/etc/localtime:ro
restart: unless-stopped
ports:
- 80:80
volumes:
nextcloud:
db:
networks:
nextcloud_network:
driver: bridge
Hope it helps.

Docker elastic stack cannot receive connection error

I have docker-compose file looking like below
version: '3'
services:
redis:
build: ./docker/redis
postgresql:
build: ./docker/postgresql
ports:
- "5433:5432"
env_file:
- .env
graphql:
build: .
command: npm run start
volumes:
- ./logs/:/usr/app/logs/
ports:
- "3000:3000"
env_file:
- .env
depends_on:
- "redis"
- "postgresql"
links:
- "redis"
- "postgresql"
elasticsearch:
build: ./docker/elasticsearch
container_name: elasticsearch
ports:
- "9200:9200"
depends_on:
- "graphql"
links:
- "kibana"
kibana:
build: ./docker/kibana
ports:
- "5601:5601"
depends_on:
- "graphql"
networks:
- elastic
environment:
- ELASTICSEARCH_URL=http://elasticsearch:9200
metricbeat:
build: ./docker/metricbeat
depends_on:
- "graphql"
- "elasticsearch"
- "kibana"
networks:
- elastic
environment:
- ELASTICSEARCH_URL=http://elasticsearch:9200
packetbeat:
build: ./docker/packetbeat
depends_on:
- "graphql"
- "elasticsearch"
- "kibana"
networks:
- elastic
environment:
- ELASTICSEARCH_URL=http://elasticsearch:9200
logstash:
build: ./docker/logstash
ports:
- "9600:9600"
volumes:
- ./logs:/usr/logs
depends_on:
- "graphql"
- "elasticsearch"
- "kibana"
networks:
- elastic
environment:
- ELASTICSEARCH_URL=http://elasticsearch:9200
networks:
elastic:
driver: bridge
When I run docker-compose build and docker-compose up, I get "unable to revive connection: http://elasticsearch:9200" from every container. I don't think any of the containers are able to talk to each other right now. However, it really feels like everything should work because I have exposed all the ports for elastic components, linked them with same networks and URL is also pointing to the correct alias. What am I doing wrong?
The dockerfile settings are all correct as each container runs correctly in isolation - just not able to talk to each other at all

Resources