I'm having a problem persisting data with docker-compose.
I want my service chatmysql to persist data I put inside a database, but everytime i run docker-compose down it all vanishes.
I checked directory /var/lib/docker/volumes to see if it stores data there when containers are running and the volume was completely empty.
I didn't have that issue when I was running containers with docker run command so I guess its fault of my docker-compose.yaml file. Can someone help me?
I'm running this on Ubuntu 20.04.
version: '3'
services:
chatmysql:
image: mysql/mysql-server
container_name: chatmysql
hostname: db
user: root
networks:
- chatnet
ports:
- 3307:3306
volumes:
- chatmysqlvolume:/lib/var/mysql
chatbackend:
depends_on:
- chatmysql
build:
context: backend/src
container_name: chatbackend
hostname: backend
networks:
- chatnet
ports:
- 8080:8080
environment:
- MYSQLUSERNAME=${MYSQLUSERNAME:-user}
- MYSQLPASSWORD=${MYSQLPASSWORD:?database password not set}
- MYSQLHOST=${MYSQLHOST:-db}
- MYSQLPORT=${MYSQLPORT:-3306}
- MYSQLDBNAME=${MYSQLDBNAME:-test}
restart: always
deploy:
restart_policy:
condition: on-failure
chatfrontend:
build: frontend
container_name: chatfrontend
hostname: front
networks:
- chatnet
ports:
- 3000:3000
volumes:
chatmysqlvolume:
networks:
chatnet:
driver: bridge
You need to change the mounted volume, try this :
version: '3.7'
services:
chatmysql:
image: mysql/mysql-server
container_name: chatmysql
hostname: db
user: root
networks:
- chatnet
ports:
- 3307:3306
volumes:
- chatmysqlvolume:/var/lib/mysql
chatbackend:
depends_on:
- chatmysql
build:
context: backend/src
container_name: chatbackend
hostname: backend
networks:
- chatnet
ports:
- 8080:8080
environment:
- MYSQLUSERNAME=${MYSQLUSERNAME:-user}
- MYSQLPASSWORD=${MYSQLPASSWORD:?database password not set}
- MYSQLHOST=${MYSQLHOST:-db}
- MYSQLPORT=${MYSQLPORT:-3306}
- MYSQLDBNAME=${MYSQLDBNAME:-test}
restart: always
deploy:
restart_policy:
condition: on-failure
chatfrontend:
build: frontend
container_name: chatfrontend
hostname: front
networks:
- chatnet
ports:
- 3000:3000
volumes:
chatmysqlvolume:
networks:
chatnet:
driver: bridge
Related
So I have two different applications one wordpress and other is api. And both running on docker containers and have their own configurations. This is their docker-compose settings:
version: "3.8"
services:
app:
container_name: ${APP_NAME}_app
build:
context: .
dockerfile: ./.docker/php/Dockerfile
expose:
- 9000
volumes:
- .:/usr/src/app
- ./public:/usr/src/app/public
depends_on:
- db
networks:
- app_network
nginx:
container_name: ${APP_NAME}_nginx
build:
context: .
dockerfile: ./.docker/nginx/Dockerfile
volumes:
- ./public:/usr/src/app/public
ports:
- "8081:8081"
expose:
- 8081
environment:
NGINX_FPM_HOST: app
NGINX_ROOT: /usr/src/app/public
depends_on:
- app
networks:
- app_network
db:
container_name: ${APP_NAME}_db
image: mysql:5.7
volumes:
- db_data:/var/lib/mysql
ports:
- "3307:3306"
environment:
MYSQL_ROOT_PASSWORD: ${DB_PASSWORD}
MYSQL_PASSWORD: ${DB_PASSWORD}
MYSQL_DATABASE: ${DB_DATABASE}
MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
networks:
- app_network
networks:
app_network:
driver: bridge
volumes:
db_data:
driver: local
And this is my wordpress configuration:
version: '3.8'
services:
mysql:
image: mysql:5.7
environment:
- MYSQL_ROOT_PASSWORD=somewordpress
- MYSQL_DATABASE=wordpress
- MYSQL_USER=wordpress
- MYSQL_PASSWORD=wordpress
expose:
- 3306
- 33060
healthcheck:
test: mysqladmin ping -h 127.0.0.1 -u $$MYSQL_USER --password=$$MYSQL_PASSWORD
interval: 1s
timeout: 3s
retries: 30
networks:
- app_network
wordpress:
build:
context: .
dockerfile: Dockerfile
depends_on:
mysql:
condition: service_healthy
volumes:
- .:/var/www/html/wp-content/plugins/name
ports:
- "80:80"
restart: always
environment:
- WORDPRESS_URL=http://localhost
- WORDPRESS_DB_HOST=mysql
- WORDPRESS_DB_USER=wordpress
- WORDPRESS_DB_PASSWORD=wordpress
- WORDPRESS_DB_NAME=wordpress
networks:
- app_network
networks:
app_network:
driver: bridge
And when I try to make request to api to this URL http://localhost:8081 well nothing happens. Locally works everything fine but on docker it doesn't.
Would appreciate some help how to make this work :)
If you have a docker container and you do http://localhost:8081, you wont go to your host pc, but to the container itself.
In docker-compose you need to replace localhost with the service name:
For example if you want to access 8081 of the nginx, you need to connect to http://nginx:8081
I want to make my nifi data volume and configuration persist means even if I delete container and docker compose up again I would like to keep what I built so far in my nifi. I try to mount volumes as follows in my docker compose file in volumes section nevertheless it doesn't work and my nifi processors are not saved. How can I do it correctly? Below my docker-compose.yaml file.
version: "3.7"
services:
nifi:
image: koroslak/nifi:latest
container_name: nifi
restart: always
environment:
- NIFI_HOME=/opt/nifi/nifi-current
- NIFI_LOG_DIR=/opt/nifi/nifi-current/logs
- NIFI_PID_DIR=/opt/nifi/nifi-current/run
- NIFI_BASE_DIR=/opt/nifi
- NIFI_WEB_HTTP_PORT=8080
ports:
- 9000:8080
depends_on:
- openldap
volumes:
- ./volume/nifi-current/state:/opt/nifi/nifi-current/state
- ./volume/database/database_repository:/opt/nifi/nifi-current/repositories/database_repository
- ./volume/flow_storage/flowfile_repository:/opt/nifi/nifi-current/repositories/flowfile_repository
- ./volume/nifi-current/content_repository:/opt/nifi/nifi-current/repositories/content_repository
- ./volume/nifi-current/provenance_repository:/opt/nifi/nifi-current/repositories/provenance_repository
- ./volume/log:/opt/nifi/nifi-current/logs
#- ./volume/conf:/opt/nifi/nifi-current/conf
postgres:
image: koroslak/postgres:latest
container_name: postgres
restart: always
environment:
- POSTGRES_PASSWORD=secret123
ports:
- 6000:5432
volumes:
- postgres:/var/lib/postgresql/data
pgadmin:
container_name: pgadmin
image: dpage/pgadmin4:4.18
restart: always
environment:
- PGADMIN_DEFAULT_EMAIL=admin
- PGADMIN_DEFAULT_PASSWORD=admin
ports:
- 8090:80
metabase:
container_name: metabase
image: metabase/metabase:v0.34.2
restart: always
environment:
MB_DB_TYPE: postgres
MB_DB_DBNAME: metabase
MB_DB_PORT: 5432
MB_DB_USER: metabase_admin
MB_DB_PASS: secret123
MB_DB_HOST: postgres
ports:
- 3000:3000
depends_on:
- postgres
openldap:
image: osixia/openldap:1.3.0
container_name: openldap
restart: always
ports:
- 38999:389
# Mocked source systems
jira-api:
image: danielgtaylor/apisprout:latest
container_name: jira-api
restart: always
ports:
- 8000:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/jira-api.json
pipedrive-api:
image: danielgtaylor/apisprout:latest
container_name: pipedrive-api
restart: always
ports:
- 8100:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/pipedrive-api.yaml
restcountries-api:
image: danielgtaylor/apisprout:latest
container_name: restcountries-api
restart: always
ports:
- 8200:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/restcountries-api.json
volumes:
postgres:
nifi:
openldap:
metabase:
pgadmin:
Using Registry you can achieve that all changes you are doing or your nifi are committed to git. I.e. if you change some processor configuration, it will be reflected in your git repo.
As for flow files, you may need to fix volumes mappings.
I have a container with RabbitMQ, it has to connect to the external IP of the application container, the problem is that this external IP is dynamic and changes every time. Accordingly, I have to manually change the configuration of my RabbitMQ every time. Maybe you can somehow set a static ip for the "App" container so that it does not change or somehow connect these two containers?
version: '3'
services:
#PHP Service
app:
build:
context: .
dockerfile: Dockerfile
image: digitalocean.com/php
container_name: app
restart: unless-stopped
tty: true
environment:
SERVICE_NAME: app
SERVICE_TAGS: dev
working_dir: /var/www
volumes:
- ./:/var/www
- ./docker/php/local.ini:/usr/local/etc/php/conf.d/local.ini
networks:
- postgres
#Nginx Service
webserver:
image: nginx:alpine
container_name: webserver
restart: unless-stopped
tty: true
ports:
- "80:80"
- "443:443"
volumes:
- ./:/var/www
- ./docker/nginx/conf.d/:/etc/nginx/conf.d/
networks:
- postgres
#Redis
redis:
image: 'redis:alpine'
ports:
- "6379:6379"
#PostgreSQL
postgres:
container_name: postgres_container
image: postgres
hostname: postgres
environment:
POSTGRES_DB: postgres
POSTGRES_USER: postgres
POSTGRES_PASSWORD: secret
PGDATA: /data/postgres
volumes:
- postgres:/data/postgres
ports:
- "5432:5432"
networks:
- postgres
restart: unless-stopped
pgadmin:
container_name: pgadmin_container
image: dpage/pgadmin4
environment:
PGADMIN_DEFAULT_EMAIL: ${PGADMIN_DEFAULT_EMAIL:-pgadmin4#pgadmin.org}
PGADMIN_DEFAULT_PASSWORD: ${PGADMIN_DEFAULT_PASSWORD:-admin}
volumes:
- pgadmin:/root/.pgadmin
ports:
- "${PGADMIN_PORT:-5050}:80"
networks:
- postgres
restart: unless-stopped
depends_on:
- postgres
#RabbitMQ
rabbit:
image: "rabbitmq:3-management"
hostname: "rabbit"
environment:
RABBITMQ_ERLANG_COOKIE: "SWQOKODSQALRPCLNMEQG"
RABBITMQ_DEFAULT_USER: "rabbitmq"
RABBITMQ_DEFAULT_PASS: "rabbitmq"
RABBITMQ_DEFAULT_VHOST: "/"
ports:
- "15672:15672"
- "5672:5672"
labels:
NAME: "rabbitmq"
networks:
- postgres
#ElasticSearch
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.4.3
ports:
- "9200:9200"
- "9300:9300"
networks:
- postgres
#Docker Networks
networks:
postgres:
driver: bridge
volumes:
postgres:
pgadmin:
If they are in the same networks You can simply use the service name instead of the IP, just use the app, You can ping the app service name inside the rabbitMQ container and it works.
as you are using docker-compose all your containers are already on the same network. You can access a container from another container using the name you have specified:
i.e for rabbitmq it is rabbit
for redis it will be redis
I have one docker-compose file that have nginx, mysql and phpmyadmin.
phpmyadmin is open in port 8080 in this address mydomain.com:8080 as well.
How can I convert this address to http://phpmyadmin.mydomain.com or http://pma.mydomain.com in my server?
I have used this docker-compose file:
version: '3'
services:
#PHP Service
app:
build:
context: .
dockerfile: ./Dockerfile/Dockerfile
container_name: app
restart: unless-stopped
tty: true
environment:
SERVICE_NAME: app
SERVICE_TAGS: dev
working_dir: /var/www
volumes:
- ./Beh:/var/www
- ./Config/php/local.ini:/usr/local/etc/php/conf.d/local.ini
networks:
- app-network
#Nginx Service
webserver:
image: nginx:alpine
container_name: webserver
restart: unless-stopped
tty: true
ports:
- "80:80"
# - "443:443"
volumes:
- ./Beh:/var/www
- ./Config/nginx/conf.d/:/etc/nginx/conf.d/
networks:
- app-network
database:
image: mariadb:latest
container_name: database
environment:
- "MYSQL_USERNAME=root"
- "MYSQL_ROOT_PASSWORD=secret"
ports:
- "3306:3306"
networks:
- app-network
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: phpmyadmin
environment:
- "MYSQL_USERNAME=root"
- "MYSQL_ROOT_PASSWORD=secret"
- "PMA_HOST=database"
links:
- database
ports:
- "8080:80"
networks:
- app-network
#Docker Networks
networks:
app-network:
driver: bridge
#Volumes
volumes:
dbdata:
driver: local
I don't know how to configure nginx file to achieve this goal.
You can use the repo -> https://github.com/jwilder/nginx-proxy
This is what I'm using to create new local domain name for each project.
You have to add "VIRTUAL_HOST" to the environment, expose the ports ("expose: 80") and also add the new domain in your /etc/hosts.
I'm using jwilder/nginx-proxy to host multiple (web)apps from a single server. This is working great except that all services can communicate with each other because they are all on the same network because that is required for the proxy to work.
Proxy docker-compose.yaml
version: "3"
services:
nginx-proxy:
image: jwilder/nginx-proxy:alpine
container_name: nginx-proxy
labels:
- "com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy"
ports:
- "80:80"
- "443:443"
volumes:
- ./data/certs:/etc/nginx/certs:ro
- ./data/nginx/vhost.d:/etc/nginx/vhost.d
- ./data/share/nginx/html:/usr/share/nginx/html
- /var/run/docker.sock:/tmp/docker.sock:ro
restart: always
letsencrypt-proxy:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: letsencrypt-proxy
depends_on:
- nginx-proxy
volumes:
- ./data/nginx/vhost.d:/etc/nginx/vhost.d
- ./data/share/nginx/html:/usr/share/nginx/html
- ./data/certs:/etc/nginx/certs:rw
- /var/run/docker.sock:/var/run/docker.sock:ro
restart: always
networks:
default:
external:
name: nginx-proxy
App 1 docker-compose.yaml
version: "3"
services:
app:
image: nginx:latest
depends_on:
- db
- cache
expose:
- 80
volumes:
- ./application:/var/www/html
restart: always
working_dir: /var/www/html
environment:
VIRTUAL_HOST: app1.example.com
LETSENCRYPT_HOST: app1.example.com
LETSENCRYPT_EMAIL: user#example.com
cache:
image: redis:alpine
restart: always
volumes:
- cachedata:/data
db:
image: mysql:5.7
restart: always
environment:
MYSQL_ROOT_PASSWORD: rootpasswd
MYSQL_DATABASE: database_name
MYSQL_USER: database_user
MYSQL_PASSWORD: database_passwd
volumes:
- dbdata:/var/lib/mysql
networks:
default:
external:
name: nginx-proxy
volumes:
dbdata:
driver: local
cachedata:
driver: local
App 2 docker-compose.yaml
version: "3"
services:
app:
image: nginx:latest
depends_on:
- db
- cache
expose:
- 80
volumes:
- ./application:/var/www/html
restart: always
working_dir: /var/www/html
environment:
VIRTUAL_HOST: app2.example.com
LETSENCRYPT_HOST: app2.example.com
LETSENCRYPT_EMAIL: user#example.com
cache:
image: redis:alpine
restart: always
volumes:
- cachedata:/data
db:
image: mysql:5.7
restart: always
environment:
MYSQL_ROOT_PASSWORD: rootpasswd
MYSQL_DATABASE: database_name
MYSQL_USER: database_user
MYSQL_PASSWORD: database_passwd
volumes:
- dbdata:/var/lib/mysql
networks:
default:
external:
name: nginx-proxy
volumes:
dbdata:
driver: local
cachedata:
driver: local
With this setup both applications will use de db and cache instance of App 1. The only way to solve that is to give those services unique names like app_1_db and app_2_db. But then App 1 is still able to connect to the app_2_db which I would like to prevent.
Is there a way to isolate all services within their docker-composer.yaml file and still use the nginx proxy?
Docker version 18.09.0, build 4d60db4
docker-compose version 1.21.2, build a133471
You can connect only the app(nginx) container from your apps to the nginx-proxy network. The only edit needed should be in the app's docker-compose:
version: '3'
services:
app:
networks:
- default
- nginx-proxy
networks:
nginx-proxy:
external: true
That way the app service will be connected to nginx-proxy and default networks at the same time. (If you omit networks key, service is always connected to the default network)
Resolving service names to containers ip's then works as expected as long as no container can see (across all networks it's connected to) two containers with same service name.
If you want even more isolation, you can create nginx-proxy network for every app.
So in your nginx-proxy docker-compose you will have:
version: "3"
services:
nginx-proxy:
networks:
- default
- nginx-proxy_app1
- nginx-proxy_app2
# letsencrypt-proxy service doesn't have to have networks key
networks:
nginx-proxy_app1:
external: true
nginx-proxy_app2:
external: true
and in your apps:
version: '3'
services:
app:
networks:
- default
- nginx-proxy_app1
networks:
nginx-proxy_app1:
external: true
and
version: '3'
services:
app:
networks:
- default
- nginx-proxy_app2
networks:
nginx-proxy_app2:
external: true
That way in every "proxy" network there is only one (if you are not using docker-compose scaling) app container and the nginx-proxy container.
More reading:
https://docs.docker.com/compose/networking/
https://docs.docker.com/network/overlay/#operations-for-standalone-containers-on-overlay-networks