When I use docker swarm with MySQL I met with problem (If docker container rerun mysql data will disappear)
I spend a lot of time to search how to fix this problem, use volume store data and use glusterFs across host share volume but experiencing database data inconsistency.
Is my method right ? or can someone tell we how to fix this problem?
At last , This is my example yaml file :
version: '3.8'
services:
www:
image: httpd:latest
ports:
- "8001:80"
volumes:
- /usr/papertest/src:/var/www/html/
db:
image: mariadb:latest
restart: always
volumes:
- /usr/test/src:/docker-entrypoint-initdb.d
- /etc/timezone:/etc/timezone:ro
- /usr/test/backup:/var/lib/mysql #/usr/test/backup is glusterfs mount place
environment:
MYSQL_ROOT_PASSWORD: MySQL_PASSWORD
MYSQL_DATABASE: MYSQL_DATABASE
#MYSQL_USER: MYSQL_USER
# MYSQL_PASSWORD: MYSQL_PASSWD
phpmyadmin:
image: phpmyadmin
#restart: always
ports:
- 4567:80
environment:
- PMA_ARBITRARY=1
Related
I am using docker-compose for my open-source web application. In the process of publishing my project on github I wanted to make my docker-compose.yaml file easier to understand and adapt. I'm still a beginner with Docker but the file works as intended. I just want to improve the readability and changeability of the volumes used by the containers. The values a/large/directory/or/disk:/var/lib/postgresql/data and /another/large/disk/:/something will most likely need to be adapted to the system the user is running my application on. Can I introduce variables for these? How can I make it more obvious that these values are to be changed by the person running my application?
My current docker-compose.yaml file
version: '3'
services:
postgres:
image: postgres:latest
restart: always
expose:
- 5432
ports:
- 5432:5432
environment:
POSTGRES_USER: 'postgres'
POSTGRES_PASSWORD: 'password'
POSTGRES_DB: 'sample'
volumes:
- /a/large/directory/or/disk:/var/lib/postgresql/data
networks:
- mynetwork
mysql:
image: mysql:5.7
restart: always
environment:
MYSQL_DATABASE: 'db'
MYSQL_USER: 'user'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
expose:
- 3306
ports:
- 3306:3306
volumes:
- ~/data/mysql:/var/lib/mysql
networks:
- mynetwork
depends_on:
- postgres
core:
restart: always
build: core/
environment:
SPRING_APPLICATION_JSON: '{
"database.postgres.url": "postgres:5432/sample",
"database.postgres.user": "postgres",
"database.postgres.password": "password",
"database.mysql.host": "mysql",
"database.mysql.user": "root",
"database.mysql.password": "password"
}'
volumes:
- ~/data/core:/var
- /another/large/disk/:/something
networks:
- mynetwork
depends_on:
- mysql
ports:
- 8080:8080
web:
restart: always
build: web/
networks:
- mynetwork
depends_on:
- core
ports:
- 3000:3000
networks:
mynetwork:
driver: bridge
volumes:
myvolume:
(I'd also appreciate any other suggestions for improvements to my file!)
Docker Compose supports variable interpolation. But then you need to document those values, and people might just assume docker compose up and have it work without extra setup.
Compose typically isn't used for production deployment, so you wouldn't use a real volume. That being said, you can simply use relative directories rather than home folder or absolute path (./data/app:/mount) to the file itself, or a Docker managed volume.
I am trying to setup a WordPress project on my machine using Docker. This is my docker-compose.yml file code:
version: "3"
services:
# Database
db:
image: mysql:5.7
volumes:
- db_data:/var/lib/mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
networks:
- wp
# Web Server
wordpress:
ports:
- "4000:80"
depends_on:
- db
image: wordpress:latest
restart: always
volumes:
- "./html/:/var/www/html/"
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: wordpress
WORDPRESS_DB_PASSWORD: wordpress
WORDPRESS_DB_NAME: wordpress
networks:
- wp
networks:
wp:
volumes:
db_data:
This works fine but what it does is, that the files that are mounted inside ./html folder has user and group permission assigned as www-data:www-data. I am working on Ubuntu desktop OS. So every time I try to update any code inside ./html folder, I get permission denied message.
Is there any way I can fix this issue?
I have tried this command to add my user to www-data group but that didn't work aswell.
sudo usermod -aG www-data aslam
Try to match the user's id on the host machine to match that of www-data inside the container or vice-versa. May be read this for more info and howto.
To be clear for everyone else, adding user: 0:0 to the docker-compose.yml file solved this issue for me too.
There isn't a problem with file permissions when you mount the files on windows... but I feel like mounting to Windows isn't a good idea because the response time to the application is very slow that way.
Here's the full docker-compose.yml
Running this in your WSL of choice will prevent the permissions issues.
version: '3.3'
services:
wptoolkit:
# add user declaration to avoid windows permission issues when editing files and folders through your favorite editor
user: 0:0
# Ensure DB is up
depends_on:
- db
# We'll use the most recent Wordpress install
image: wordpress:latest
# Map files from container back to linux at this directory location
volumes:
- ./:/var/www/html
ports:
# We'll access our site on http://localhost:8999
- "80:80"
restart: always
environment:
# We'll provide some SQL credentials -- these match what's
# in our db section
- WORDPRESS_DB_HOST=db
- WORDPRESS_DB_USER=wordpress
- WORDPRESS_DB_PASSWORD=wordpress
- WORDPRESS_DB_NAME=wordpress
# This is our MySQL database server
db:
# We use a mariadb image which supports both amd64 & arm64 architecture
image: mariadb:10.6.4-focal
# If you really want to use MySQL, uncomment the following line
#image: mysql:8.0.27
command: '--default-authentication-plugin=mysql_native_password'
volumes:
- db_data:/var/lib/mysql
restart: always
environment:
- MYSQL_ROOT_PASSWORD=wordpress
- MYSQL_DATABASE=wordpress
- MYSQL_USER=wordpress
- MYSQL_PASSWORD=wordpress
expose:
- 3306
- 33060
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: pma
links:
- db
environment:
PMA_HOST: db
PMA_PORT: 3306
PMA_ARBITRARY: 1
restart: always
ports:
- 8081:80
volumes:
db_data:
I'm trying to set up a docker-compose file for running Apache Guacamole.
The compose file has 3 services, 2 for guacamole itself and 1 database image. The problem is that the database has to be initialized before the guacamole container can use it, but the files to initialize the database are in the guacamole image. The solution I came up with is this:
version: "3"
services:
init:
image: guacamole/guacamole:latest
command: ["/bin/sh", "-c", "cp /opt/guacamole/postgresql/schema/*.sql /init/" ]
volumes:
- dbinit:/init
database:
image: postgres:latest
restart: unless-stopped
volumes:
- dbinit:/docker-entrypoint-initdb.d
- dbdata:/var/lib/postgresql/data
environment:
POSTGRES_USER: guac
POSTGRES_PASSWORD: guac
depends_on:
- init
guacd:
image: guacamole/guacd:latest
restart: unless-stopped
guacamole:
image: guacamole/guacamole:latest
restart: unless-stopped
ports:
- "8080:8080"
environment:
GUACD_HOSTNAME: guacd
POSTGRES_HOSTNAME: database
POSTGRES_DATABASE: guac
POSTGRES_USER: guac
POSTGRES_PASSWORD: guac
depends_on:
- database
- guacd
volumes:
dbinit:
dbdata:
So I have one container whose job is to copy the database initialization files into a volume and then I mount that volume in the database. The problem is that this creates a race condition and is ugly. Is there some elegant solution for this? Is it possible to mount the files from the guacamole image into the database container? I would rather avoid having an extra sql file with the docker-compose file.
Thanks in advance!
I try to learn about container, but i have a problem with my docker-compose.yml file, after i run the docker compose up, i always get the same error:
"ERROR: yaml.scanner.ScannerError: mapping values are not allowed
here"
even if i changed the mount path to docker volume, i got the same error, this is my yml file
version: "3"
services:
database:
image: mariadb
ports:
- "3260:3260"
volumes:
- /home/randy/Desktop/Latihan/wordpress-mariadb/mariadb:var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: root
wordpress:
image: wordpress
ports:
- "2000:80"
volumes:
- /home/randy/Desktop/Latihan/wordpress-mariadb/wordpress:/var/www/html
environment:
WORDPRESS_DB_PASSWORD: root
depends_on:
- database
links:
- database
It appears that your yaml is invalid. When I face these types of issues, what I will do is use a site called http://www.yamllint.com/ which will validate the syntax for you.
This yaml based on your example is valid:
Note: You can use 4 spaces (or 2 which I prefer), but never use tabs.
version: "3"
services:
database:
environment:
MYSQL_ROOT_PASSWORD: root
image: mariadb
ports:
- "3260:3260"
volumes:
- "/home/randy/Desktop/Latihan/wordpress-mariadb/mariadb:var/lib/mysql"
wordpress:
image: wordpress
ports:
- "2000:80"
volumes:
- /home/randy/Desktop/Latihan/wordpress-mariadb/wordpress:/var/www/html
environment:
WORDPRESS_DB_PASSWORD: root
depends_on:
- database
links:
- database
I can't seem to get MySQL data to persist if I run $ docker-compose down with the following .yml
version: '2'
services:
# other services
data:
container_name: flask_data
image: mysql:latest
volumes:
- /var/lib/mysql
command: "true"
mysql:
container_name: flask_mysql
restart: always
image: mysql:latest
environment:
MYSQL_ROOT_PASSWORD: 'test_pass' # TODO: Change this
MYSQL_USER: 'test'
MYSQL_PASS: 'pass'
volumes_from:
- data
ports:
- "3306:3306"
My understanding is that in my data container using volumes: - /var/lib/mysql maps it to my local machines directory where mysql stores data to the container and because of this mapping the data should persist even if the containers are destroyed. And the mysql container is just a client interface into the db and can see the local directory because of volumes_from: - data
Attempted this answer and it did not work. Docker-Compose Persistent Data Trouble
EDIT
Changed my .yml as shown below and created a the dir ./data but now when I run docker-compose up --build the mysql container wont start throws error saying
data:
container_name: flask_data
image: mysql:latest
volumes:
- ./data:/var/lib/mysql
command: "true"
mysql:
container_name: flask_mysql
restart: always
image: mysql:latest
environment:
MYSQL_ROOT_PASSWORD: 'test_pass' # TODO: Change this
MYSQL_USER: 'test'
MYSQL_PASS: 'pass'
volumes_from:
- data
ports:
- "3306:3306"
flask_mysql | mysqld: Can't create/write to file '/var/lib/mysql/is_writable' (Errcode: 13 - Permission denied)
flask_mysql | 2016-08-26T22:29:21.182144Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
flask_mysql | 2016-08-26T22:29:21.185392Z 0 [ERROR] --initialize specified but the data directory exists and is not writable. Aborting.
The data container is a superfluous workaround. Data-volumes would do the trick for you. Alter your docker-compose.yml to:
version: '2'
services:
mysql:
container_name: flask_mysql
restart: always
image: mysql:latest
environment:
MYSQL_ROOT_PASSWORD: 'test_pass' # TODO: Change this
MYSQL_USER: 'test'
MYSQL_PASS: 'pass'
volumes:
- my-datavolume:/var/lib/mysql
volumes:
my-datavolume:
Docker will create the volume for you in the /var/lib/docker/volumes folder. This volume persist as long as you are not typing docker-compose down -v
There are 3 ways:
First way
You need specify the directory to store mysql data on your host machine. You can then remove the data container. Your mysql data will be saved on you local filesystem.
Mysql container definition must look like this:
mysql:
container_name: flask_mysql
restart: always
image: mysql:latest
environment:
MYSQL_ROOT_PASSWORD: 'test_pass' # TODO: Change this
MYSQL_USER: 'test'
MYSQL_PASS: 'pass'
volumes:
- /opt/mysql_data:/var/lib/mysql
ports:
- "3306:3306"
Second way
Would be to commit the data container before typing docker-compose down:
docker commit my_data_container
docker-compose down
Third way
Also you can use docker-compose stop instead of docker-compose down (then you don't need to commit the container)
first, you need to delete all old mysql data using
docker-compose down -v
after that add two lines in your docker-compose.yml
volumes:
- mysql-data:/var/lib/mysql
and
volumes:
mysql-data:
your final docker-compose.yml will looks like
version: '3.1'
services:
php:
build:
context: .
dockerfile: Dockerfile
ports:
- 80:80
volumes:
- ./src:/var/www/html/
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: example
volumes:
- mysql-data:/var/lib/mysql
adminer:
image: adminer
restart: always
ports:
- 8080:8080
volumes:
mysql-data:
after that use this command
docker-compose up -d
now your data will persistent and will not be deleted even after using this command
docker-compose down
extra:- but if you want to delete all data then you will use
docker-compose down -v
You have to create a separate volume for mysql data.
So it will look like this:
volumes_from:
- data
volumes:
- ./mysql-data:/var/lib/mysql
And no, /var/lib/mysql is a path inside your mysql container and has nothing to do with a path on your host machine. Your host machine may even have no mysql at all. So the goal is to persist an internal folder from a mysql container.
Adding on to the answer from #Ohmen, you could also add an external flag to create the data volume outside of docker compose. This way docker compose would not attempt to create it. Also you wouldn't have to worry about losing the data inside the data-volume in the event of $ docker-compose down -v.
The below example is from the official page.
version: "3.8"
services:
db:
image: postgres
volumes:
- data:/var/lib/postgresql/data
volumes:
data:
external: true
Actually this is the path and you should mention a valid path for this to work. If your data directory is in current directory then instead of my-data you should mention ./my-data, otherwise it will give you that error in mysql and mariadb also.
volumes:
./my-data:/var/lib/mysql
Feasible bind mount solution:
mariadb:
image: mariadb:latest
restart: unless-stopped
environment:
- MARIADB_ROOT_PASSWORD=${MARIADB_ROOT_PASSWORD}
volumes:
- type: bind
source: /host/dir
target: /var/lib/mysql