Increasing docker volume size using docker-compose - docker

I'm working on deploying a Flask web app with Docker (using docker-compose). Users upload files which can be particularly big (>1GB). My question is -- do I have to be worried about the container running out of space? I've read that containers have a default max size of 10GB, and I will definitely exceed that quickly. If I create a volume in the "flask-app/uploads" directory where all the files are stored, does that solve my problem or is the volume just another container with the same size limitations? Is there any way I can just store everything that gets uploaded to "flask-app/uploads" to the host machine so nothing get written to the container?
Here is my docker-compose.yml file for reference:
web:
restart: always
build: ./web
expose:
- "8000"
links:
- postgres:postgres
volumes:
- /usr/src/flask-app/static
- /usr/src/flask-app/uploads (??)
env_file: .env
command: /usr/local/bin/gunicorn -w 2 -b :8000 app:app
nginx:
restart: always
build: ./nginx/
ports:
- "80:80"
volumes:
- /www/static
volumes_from:
- web
links:
- web:web
data:
restart: always
image: postgres:latest
volumes:
- /var/lib/postgresql
command: "true"
postgres:
restart: always
image: postgres:latest
volumes_from:
- data
ports:
- "5432:5432"

Yes, this is a common practice, you can use a host volume. Change the volume line to - ./uploads:/usr/src/flask-app/uploads.

Related

docker-compose force services to create seperate containers even when images are the same

I have 2 services which use the same image:, what can i do, to force docker-compose to generate 2 seperate containers?
Thanks!
EDIT:
Full docker-compose:
version: "3.5"
services:
database:
container_name: proj-database
env_file: ../orm/.env.${PROJ_ENV}
image: postgres
restart: always
ports:
- 5432:5432
networks:
- proj
api:
image: golang:1.17
container_name: proj-api
env_file: ../cryptoModuleAPI/.env.${PROJ_ENV}
restart: always
build: ../cryptoModuleAPI/
links:
- database:database
ports:
- 8080:8080
volumes:
- ../cryptoModuleAPI:/proj/api
- ../orm:/proj/orm
networks:
- proj
admin:
image: golang:1.17
container_name: proj-admin
env_file: ../admin/.env.${PROJ_ENV}
restart: always
build: ../admin/
links:
- database:database
ports:
- 8081:8081
volumes:
- ../admin:/proj/admin
- ../orm:/proj/orm
networks:
- proj
networks:
proj:
external:
name: proj
I just run with docker-compose up
You misunderstand how the build and image directives work when used together.
Paraphrasing the docs,
https://docs.docker.com/compose/compose-file/compose-file-v3/#build
If you specify image as well as build, then Compose names the built image with the value of the image directive.
Compose is going to build two images, both named the same thing. Only one will survive. I'm surprised your app spins up at all!
Provide a different name for the image directive of each service, or leave it out entirely.

Docker Volumes: Persisting Data

I have a docker-compose file below where I have a mariadb with volume. Each time i perform a docker-compose pull and docker-compose up -d i lose the data after updating the image
mariadb:
image: bitnami/mariadb:latest
container_name: mariadb-db
ports:
- 3303:3306
environment:
- "MARIADB_ROOT_PASSWORD=password"
- "MARIADB_EXTRA_FLAGS=--max-connect-errors=1000 --max_connections=300"
- "MARIADB_DATABASE=mydb"
restart: on-failure
volumes:
- mariadb-data:/var/lib/mysql
volumes:
mariadb-data:
How do i make the data persistent?
According to the readme, the database inside the container is at /bitnami/mariadb. This isn't standard, rather a feature of bitnami/mariadb image. It would be /var/lib/mysql if you had used an official one.
Thus, either change the path inside the container:
image: bitnami/mariadb:latest
volumes:
- mariadb-data:/bitnami/mariadb
or use the default path with an official image:
image: mariadb:latest
volumes:
- mariadb-data:/var/lib/mysql

Update in-docker-compose container with new env_file

I have the following docker-compose configuration:
version: '2'
services:
nginx:
image: 'nginx:latest'
expose:
- '80'
- '8080'
container_name: nginx
ports:
- '80:80'
- '8080:8080'
volumes:
- '/home/ubuntu/nginx.conf:/etc/nginx/nginx.conf'
networks:
- default
restart: always
inmates:
image: 'xxx/inmates:mysql'
container_name: 'inmates'
expose:
- '3000'
env_file: './inmates.env'
volumes:
- inmates_documents_images:/data
- inmates_logs:/logs.log
networks:
- default
restart: always
we19:
image: 'xxx/we19:dev'
container_name: 'we19'
expose:
- '3000'
env_file: './we19.env'
volumes:
- we19_logs:/logs.log
networks:
- default
restart: always
desktop:
image: 'xxx/desktop:dev'
container_name: 'desktop'
expose:
- '3000'
env_file: './desktop.env'
volumes:
- desktop_logs:/logs.log
networks:
- default
restart: always
volumes:
inmates_documents_images:
inmates_logs:
desktop_logs:
we19_logs:
Assume I did docker-compose up -d --buiild.
Now the 4 containers (services) are runnig.
Now, I want to update ./desktop.env file with new content. Is there any possible way to reset only desktop container with the new env file? Or docker-compose restart is neccessary?
Basically I'm trying to restart only desktop container with the new env file but keep all 3 others container up running without restarting them.
Extract from docker-compose up --help
[...]
If there are existing containers for a service, and the service's configuration or image was changed after the container's creation, docker-compose up picks up the changes by stopping and recreating the containers (preserving mounted volumes). To prevent Compose from picking up changes, use the --no-recreate flag.
[...]
Usage: up [options] [--scale SERVICE=NUM...] [SERVICE...]
[...]
The following command should do the trick in your case.
docker-compose up -d desktop
If not, see the documentation for other options you can use to meet your exact requirement (e.g. --force-recreate, --renew-anon-volumes, ...)

Docker compose up does not restart on reboot

I have successfully created docker containers and they work when loaded using:
sudo docker-compose up -d
The yml is as follows:
services:
nginx:
build: ./nginx
restart: always
ports:
- "80:80"
volumes:
- ./static:/static
links:
- node:node
node:
build: ./node
restart: always
ports:
- "8080:8080"
volumes:
- ./node:/usr/src/app
- /usr/src/app/node_modules
Am I supposed to create a service for this. Reading the documentation I thought that the containers would reload in restart was set to always.
FYI: the yml is inside a projects directory on the home of the base user: ubuntu.
I tried checking for solutions in stack but could not find anything appropriate. Thanks.

How to run Docker container in it's own network

Today I switched from "Docker Toolbox" to "Docker for Mac", because Docker now has finally write-access to my User directory (which doesn't worked with "Docker Toolbox") - Yay!
But this change also includes that all containers now running under my localhost and not under Docker's IP as before (e.g. 192.168.99.100).
Since my localhost listens to various ports by default (80, 443, ...) and I don't want to always add new created ports, that doesn't conflict with the standard one's, to my local dev domains (e.g. example.dev:8443), I wonder how to run my containers as before.
I read about network configs and tried a lot of things (creating a new host network, exposing ports with an IP in front of it, ...), but didn't got it working.
What kind of config do I need to run my app container with the IP 192.168.99.100? Thats my docker-compose.yml so far.
version: '2'
services:
app:
build:
context: .
dockerfile: Dockerfile
depends_on:
- mysql
- redis
- memcached
ports:
- 80:80
- 443:443
- 22:22
- 3000:3000
- 3001:3001
volumes:
- ./app/:/app/
- /tmp/debug/:/tmp/debug/
- ./:/docker/
volumes_from:
- storage
# cap and privileged needed for slowlog
cap_add:
- SYS_PTRACE
privileged: true
env_file:
- etc/environment.yml
- etc/environment.development.yml
mysql:
build:
context: docker/mysql/
dockerfile: MariaDB-10
ports:
- 3306:3306
volumes_from:
- storage
volumes:
- ./data/mysql:/var/lib/mysql
- /tmp/debug/:/tmp/debug/
env_file:
- etc/environment.yml
- etc/environment.development.yml
redis:
build: docker/redis/
volumes_from:
- storage
env_file:
- etc/environment.yml
- etc/environment.development.yml
memcached:
build: docker/memcached/
volumes_from:
- storage
env_file:
- etc/environment.yml
- etc/environment.development.yml
storage:
build: docker/storage/
volumes:
- /storage
You need to declare "networks:" for each of your services:
e.g.
version: '2'
services:
app:
image: xxxx:xxx
ports:
- "80:80"
networks:
- my-network
mysql:
image: xxxx:xxx
networks:
- my-network
networks:
my-network:
driver: bridge
Then from side your app configuration, you can use "mysql" as the hostname of database server.
You can define a network in your compose file, then add any services to the network.
https://docs.docker.com/compose/networking/
But I would suggest you just use different ports now that you are running natively. I.e. 8080:80

Resources