I have a web application running in container.
The application allows users to upload files. These files are stored in Docker volumes web_data1 and web_data2.
Due to changes in the application, I need to change the mountpoint of these volumes i.e.
the data that were in /srv/app/web_data1_mountpoint, now need to be moved to /srv/app/web_data1_changed_mountpoint.
What is the proper way to do this?
docker-compose.yml
version: "3"
volumes:
web_data1:
web_data2:
services:
web:
build:
context: .
dockerfile: .docker/Dockerfile
image: web-image
ports:
- 80:80
- 443:443
volumes:
- web_data1:/srv/app/web_data1_mountpoint
- web_data2:/srv/app/web_data2_mountpoint
That depends a bit of the image you are using. Just changing the volume would work in your docker-compose.yml like this:
volumes:
- web_data1:/srv/app/web_data1_changed_mountpoint
- web_data2:/srv/app/web_data2_changed_mountpoint
But I dont know, what your image does with the directory. Maybe something inside the image depends on the directory.
Related
I am using docker to run prometheus, grafana and node exporter. I am trying to use named volumes and I am having some issues with that. My docker-compose code is:
version: "3.7"
volumes:
grafana_ini:
prometheus_data:
grafana_data:
dashboards_data:
services:
grafana:
build: ./grafana
volumes:
- grafana_ini:/etc/grafana/grafana.ini
- grafana_data:/etc/grafana/provisioning/datasources/datasource.yml
- dashboards_data:/etc/grafana/provisioning/dashboards
- ./dashboards/linux_dashboard.json:/etc/grafana/provisioning/dashboards/linux_dashboard.json
ports:
- 3000:3000
links:
- prometheus
prometheus:
build: ./prometheus
volumes:
- prometheus_data:/etc/prometheus/prometheus.yml
ports:
- 9090:9090
node-exporter:
image: prom/node-exporter:latest
container_name: node_exporter
restart: unless-stopped
expose:
- 9100
and my dockerfile for grafana is:
FROM grafana/grafana:latest
COPY ./Ini/grafana.ini /etc/grafana/grafana.ini
COPY datasource.yml /etc/grafana/provisioning/datasources/datasource.yml
COPY ./dashboards/dashboard.yml /etc/grafana/provisioning/dashboards
COPY ./dashboards/server/linux_dashboard.json /etc/grafana/provisioning/dashboards
COPY ./dashboards/server/windows_dashboard.json /etc/grafana/provisioning/dashboards
EXPOSE 3000:3000
and I am getting this error while building it
ERROR: for 2022_grafana_1 Cannot create container for service grafana: source /var/lib/docker/overlay2/4ac5b487fd7fd52491b250c4afaa433801420cd907ac4a70ddb4589fdb99368b/merged/etc/grafana/grafana.ini is not directory
ERROR: for grafana Cannot create container for service grafana: source /var/lib/docker/overlay2/4ac5b487fd7fd52491b250c4afaa433801420cd907ac4a70ddb4589fdb99368b/merged/etc/grafana/grafana.ini is not directory
Can anybody please help me.
It looks like there are some problems with the volume configuration in your Grafana container:
First, I think this was simply a typo in your question:
- grafana_ini:/etc/grafana/grafana.inianticipated location in container
I suspect that you were actually intending this:
- grafana_ini:/etc/grafana/grafana.ini
Which doesn't make any sense: grafana.ini is a file, but a volume is
a directory. Docker won't allow you to mount a directory on top of a
file, hence the error:
ERROR: .../etc/grafana/grafana.ini is not directory
You have the same problem with the grafana_data volume, which you're
attempting to mount on top of datasource.yml:
- grafana_data:/etc/grafana/provisioning/datasources/datasource.yml
I think you may be approaching this configuration in the wrong way;
you may want to read through these documents:
https://grafana.com/docs/grafana/latest/installation/docker/
https://grafana.com/docs/grafana/latest/administration/configure-docker/
https://grafana.com/docs/grafana/latest/administration/provisioning/
It is possible to configure Grafana (and Prometheus!) using only bind
mounts and environment variables (this includes installing plugin,
data sources, and dashboards), so you don't need to build your own
custom images.
Unrelated to this particular problem, there are some other things in
your docker-compose.yml that are worth changing. You should no
longer be using the links directive...
links:
- prometheus
...because Docker maintains DNS for you automatically; your containers
can refer to each other by name with no additional configuration.
Docker doesn't use the latest code after running git checkout <non_master_branch>, while I can see it in the vscode.
I am using the following docker-compose file:
version: '2'
volumes:
pgdata:
backend_app:
services:
nginx:
container_name: nginx-angular-dev
image: nginx-angular-dev
build:
context: ./frontend
dockerfile: /.docker/nginx.dockerfile
ports:
- "80:80"
- "443:443"
depends_on:
- web
web:
container_name: django-app-dev
image: django-app-dev
build:
context: ./backend
dockerfile: /django.dockerfile
command: ["./wait-for-postgres.sh", "db", "./django-entrypoint.sh"]
volumes:
- backend_app:/code
ports:
- "8000:8000"
depends_on:
- db
env_file: .env
environment:
FRONTEND_BASE_URL: http://192.168.99.100/
BACKEND_BASE_URL: http://192.168.99.100/api/
MODE_ENV: DOCKER_DEV
db:
container_name: django-db
image: postgres:10
env_file: .env
volumes:
- pgdata:/var/lib/postgresql/data
I have tried docker-compose build --no-cache, followed by docker-compose up --force-recreate but it didn't solve the problem.
What is the root of my problem?
Your volumes: are causing problems. Docker volumes aren't intended to hold code, and you should delete the volume declarations that mention backend_app:.
Your docker-compose.yml file says in part:
volumes:
backend_app:
services:
web:
volumes:
- backend_app:/code
backend_app is a named volume: it keeps data that must be persisted across container runs. If the volume doesn't exist yet the first time then data will be copied into it from the image, but after that, Docker considers it to contain critical user data that must not be updated.
If you keep code or libraries in a Docker volume, Docker will never update it, even if the underlying image changes. This is a common problem in JavaScript applications that mount an anonymous volume on their node_modules directory.
As a temporary workaround, if you docker-compose down -v, it will delete all of the volumes, including the one with your code in it, and the next time you start it will get recreated from the image.
The best solution is to simply not use a volume here at all. Delete the lines above from your docker-compose.yml file. Develop and test your application in a non-Docker environment, and when you're ready to do integration testing, run docker-compose up --build. Your code will live in the image, and an ordinary docker build will produce a new image with new code.
I have 2 containers in a compose files,that i want to serve app static files through nginx.
I have read this: https://stackoverflow.com/a/43560093/7522096 and want to use host dir to share between app container and nginx container, for some reason I dont want to use named volume.
===
Using a host directory Alternately you can use a directory on the host
and mount that into the containers. This has the advantage of you
being able to work directly on the files using your tools outside of
Docker (such as your GUI text editor and other tools).
It's the same, except you don't define a volume in Docker, instead
mounting the external directory.
version: '3'
services:
nginx:
volumes:
- ./assets:/var/lib/assets
asset:
volumes:
- ./assets:/var/lib/assets
===
My docker-compose file:
version: "3.7"
services:
app:
container_name: app
restart: always
ports:
- 8888:8888
env_file:
- ./env/app.env
image: registry.gitlab.com/app/development
volumes:
- ./public/app/:/usr/app/static/
- app-log:/root/.pm2
nginx:
container_name: nginx
image: 'nginx:1.16-alpine'
restart: always
ports:
- 80:80
- 443:443
volumes:
- /home/devops/config/:/etc/nginx/conf.d/:ro
- /home/devops/ssl:/etc/nginx/ssl:ro
- ./public/app/:/etc/nginx/public/app/
depends_on:
- app
volumes:
# app-public:
app-log:
Yet when i do this in my compose, the dir always come up empty on nginx, and the static files in my app container got disappear too.
Please help, I tried a lot of ways but can not figure it out.
Thanks.
During the initialization of the container docker will bind the ./public/app directory on the host with the /usr/app/static/ directory in the container.
If the ./public/app does not exist it will be created. The bind is from the host to the container, meaning that the content of ./public/app folder is
reflected (copied) into the container and not viceversa. That's why after the initialization the static app directory is empty.
If my understanding is correct your goal is to share the application files between the app container and nginx.
Taken into consideration the above the only solution is to create the files in the volume after the volume is created. Here is an example for the relevant parts:
version: "3"
services:
app:
image: ubuntu
volumes:
- ./public/app/:/usr/app/static_copy/
entrypoint: /bin/bash
command: >
-c "mkdir /usr/app/static;
touch /usr/app/static/shared_file;
mv /usr/app/static/* /usr/app/static_copy;
rm -r /usr/app/static;
ln -sfT /usr/app/static_copy/ /usr/app/static;
exec sleep infinity"
nginx:
image: 'nginx:1.16-alpine'
volumes:
- ./public/app/:/etc/nginx/public/app/
depends_on:
- app
This will move the static files to the static_copy directory and link back those files to /usr/app/static. Those files will be shared with the host (public/app director)
and nginx container (/etc/nginx/public/app/). Adapt it to fit your needs.
In alternative you can of course use named volumes.
Hope it helps
I have docker-compose file, where I have specified volumes:
services:
portal:
env_file:
- ${ENV_FILE}
build:
context: .
dockerfile: ./Dockerfile
container_name: sensus-portal
image: sensus-portal
hostname: ${DOMAIN}
working_dir: /var/www/html
volumes:
- ../localhost-certificates:/var/certificates
- ./portal:/var/www/html
- ../wp-content:/var/www/html/wp-content
ports:
- "80:80"
- "443:443"
Problem is that when last volume added, it replaces previous stuffs which was in var/www/html/wp-content ... I want to assign volume /var/www/html and then into this add (merge) volume /var/www/html/wp-content.
Is there any chance how to do this?
No, there’s no way to merge the contents of two volumes or host directories. This works the same way as normal Linux mount(8): the directory or volume you’re mounting into the container hides whatever was there already, and there’s no way to get at the hidden content.
The Dockerfile COPY directive is a little more flexible this way and you might consider building this content into your image.
I have a few questions about Docker volumes. I have installed Docker and docker-compose on a fresh host running debian stretch. I managed to get a docker-compose file running for a simple nginx/php-fpm project, both containers mounted on the directory containing the source code. I wanted to try to create a single volume that would be shared across my containers but I have a few issue, and my understanding of the official documentation is not helping.
So this is an idea of what I'm trying to achieve :
Question 1 : Trying to create a volume from a dockerfile on a directory mounted from host
docker-compose.yml :
version: '3'
services:
php:
build:
context: .
dockerfile: php.dockerfile
volumes:
- ./host-project-directory:/project
php.dockerfile :
FROM php:7-fpm
VOLUME project
from my understanding, when running docker-compose we should have a volume created on host containing all files from /project from container. And /project from container should contain all files from ./host-project-directory from host.
If I ls the content of /project on container I can see the files from host, but using docker volume list, there are no volumes created on host, why ?
Question 2 : How to populate and use this volume from another container ?
version: '3'
services:
php:
build:
context: .
dockerfile: php.dockerfile
volumes:
- named-volume:/project
web:
image: nginx
links:
- php
volumes:
- named-volume:/project
volumes:
named-volume:
This should create a volume called 'named-volume' and bind it to /project directories on both containers php and web.
Now, how to populate this volume with content from ./host-project-directory ?
I've tried adding a dockerfile like
ADD ./host-project-directory /project
But nothing changed and the volume remained empty.
I'm sorry if this is due to my lack of experience using Docker but I can't figure out how to make this simple thing work.
Thank you for your time !
For the first question, I try a simple docker file like this:
FROM php:7-fpm
COPY ./project /project
And a docker-compose like this:
version: '3'
services:
php:
build: .
volumes:
- named-volume:/project
web:
image: nginx
links:
- php
volumes:
- named-volume:/project
volumes:
named-volume:
Since you create the volume on docker-compose you don't need to create that in the Dockerfile.
Running docker volume list, I'm able to see the volume created with a local driver. Making ls inside the folder I'm also able to see the file. It's important to note, that the file present in you local directory it's not the same that the file inside the container. So if you edit the files in the host this will not change the files in container. That's because you have your volume created in another path, probably at: /var/lib/docker/volumes/...
This happens because you map the volume to the path, but you not specifies where you want the volume. To do that just make your docker-compose like this:
version: '3'
services:
php:
build: .
volumes:
- ./project:/project
web:
image: nginx
links:
- php
volumes:
- ./project:/project
Making this I'm still able to see the volume with the volume list command but without a name.
So I don't know why you are not able to see the volume in the list.
For question 2:
Doing the example above I have the files inside the container that exists in my local "project" folder.
Please check that the path to the local folder is correct.
A bind mount is not the same thing as a volume. You're defining a named volume here, but wanting the functionality of a bind mount.
Try this
version: '3'
services:
php:
build:
context: .
dockerfile: php.dockerfile
volumes:
- ./host-project-directory:/project
web:
image: nginx
links:
- php
volumes:
- ./host-project-directory:/project