I'm new using docker and I want to mantain my media folder and sqlite file on the server when I update the container, but I don't know how to make it.
My dockerfile have:
WORKDIR /app
ADD . /app
RUN mkdir media
RUN mkdir media/media
RUN mkdir media/projetos
RUN mkdir media/uploads
RUN chmod g+w /app
RUN chmod g+w /app/db.sqlite3
RUN chmod g+w media/
RUN chmod g+w media/media
RUN chmod g+w media/projetos
RUN chmod g+w media/uploads
And my docker-compose.yml is:
version: '2.1'
services:
arq_web:
image: creativerender.azurecr.io/arq_web:latest
build: .
ports:
- 8000:8000
I don't know if I have to use the volume tag on my docker compose, I tried something like:
volumes:
- ./media:/app/media
But without success.
When you use volumes in the docker-compose it will override everything that you copied at build time.
volumes:
- ./media:/app/media
So anything that is copied in Dockerfile, the volume will override it.
ADD . /app
To want to keep just media and sqllite file from the host then you can try
volumes:
- ./media:/app/media/media
- ./db.sqlite3:/app/db.sqlite3
I think your media folder is /app/media/media as you creating in dockerfile.
RUN mkdir media/media
First, I suggest you upgrade to version 3 syntax of Docker Compose than I suggest you to use named volumes (https://success.docker.com/article/different-types-of-volumes) to handle volumes in your services because Docker will handle how the volume will be created.
This is the link to docker-compose file reference of using volumes (including named volumes) https://docs.docker.com/compose/compose-file/#volumes
Related
This is my Dockerfile.
FROM python:3.8.12-slim-bullseye as prod-env
RUN apt-get update && apt-get install unzip vim -y
COPY requirements.txt /app
RUN pip install -r requirements.txt
USER nobody:nogroup
This is how docker-compose.yml looks like.
api_server:
build:
context: .
target: prod-env
image: company/server
volumes:
- ./shared/model_server/models:/models
- ./static/images:/images
ports:
- 8200:8200
command: gunicorn -b 0.0.0.0:8200 --threads "8" --log-level info --reload "server:gunicorn_app(command='start', project='app_server')"
I want to add permissions read, write and execute permissions on shared directories.
And also need to run couple of other coommands as root.
So I have to execute this command with root every time after image is built.
docker exec -it -u root api_server_1 bash -c "python copy_stuffs.py; chmod -R a+rwx models; chmod -R a+rwx /images"
Now, I want docker-compose to execute these lines.
But as you can see, user in docker-compose has to be nobody as specified by Dockerfile. So how can I execute root commands in docker-compose file?
Option that I've been thinking:
Install sudo command from Dockerfile and use sudo
Is there any better way ?
In docker-compose.yml create another service using same image and volumes.
Override user with user: root:root, command: your_command_to_run_as_root, for this new service and add dependency to run this new service before starting regular working container.
api_server:
build:
context: .
target: prod-env
image: company/server
volumes:
- ./shared/model_server/models:/models
- ./static/images:/images
ports:
- 8200:8200
command: gunicorn -b 0.0.0.0:8200 --threads "8" --log-level info --reload "server:gunicorn_app(command='start', project='app_server')"
# This make sure that startup order is correct and api_server_decorator service is starting first
depends_on:
- api_server_decorator
api_server_decorator:
build:
context: .
target: prod-env
image: company/server
volumes:
- ./shared/model_server/models:/models
- ./static/images:/images
# No ports needed - it is only decorator
# Overriding USER with root:root
user: "root:root"
# Overriding command
command: python copy_stuffs.py; chmod -R a+rwx models; chmod -R a+rwx /images
There are other possibilities like changing Dockerfile by removing USER restriction and then you can use entrypoint script doing as root what you want as privileged user and running su - nobody or better exec gosu to retain PID=1 and proper signal handling.
In my eyes the approach giving a container root rights is quite hacky and dangerous.
If you want to e.g. remove the files written by container you need root rights on host as well.
If you want to allow a container to access files on host filesystem just run the container with appropriate user.
api_server:
user: my_docker_user:my_docker_group
then give on host the rights to that group
sudo chown -R my_docker_user:my_docker_group models
You should build all of the content you need into the image itself, especially if you have this use case of occasionally needing to run a process to update it (you are not trying to use an isolation tool like Docker to simulate a local development environment). In your Dockerfile, COPY these directories into the image
COPY shared/model_server/models /models
COPY static/images /images
Do not make these directories writeable, and do not make the individual files in the directories executable. The directories will generally be mode 0755 and the files mode 0644, owned by root, and that's fine.
In the Compose setup, do not mount host content over these directories either. You should just have:
services:
api_server:
build: . # use the same image in all environments
image: company/server
ports:
- 8200:8200
# no volumes:, do not override the image's command:
Now when you want to update the files, you can rebuild the image (without interrupting the running application, without docker exec, and without an alternate user)
docker-compose build api_server
and then do a relatively quick restart, running a new container on the updated image
docker-compose up -d
I want to create an uploads volume and set its owner to the node user. But upon running the container I find that the volume's owner is root. This is my Docker file:
FROM node:12.21
RUN apt-get update && apt-get -y install curl vim bash nano
WORKDIR /home/node/app
COPY package.json .
COPY yarn.lock .
RUN mkdir ./uploads
RUN chown -R node:node .
USER node
RUN yarn install
COPY --chown=node:node . .
VOLUME /home/node/app/uploads
I use docker-compose build then docker-compose up to build and run; my docker-compose.yml also contains a volume instruction:
services:
...
server:
...
volumes:
- ./uploads:/home/node/app/uploads
My tests show that this instruction in docker-compose.yml is what's causing the problem -- without it the uploads directory owner is correctly set to node -- but I don't understand why. Is the instruction in docker-compose.yml redundant in this case? How about if I wanted to map the volume to a local directory (for which I believe this instruction would be necessary)?
Bind volume will retain the origin ownership on the host. You can either change mode the directory on the host to 77x, or you can try this way.
Update: Base on your feedback, you can add the chmod in your docker startup script.
I have an app in Laravel with .env.local file (a and I made the next docker-compose file:
api:
container_name: nadal_api
build:
context: .
dockerfile: Dockerfile
volumes:
- .:/var/www/html/app
ports:
- ${APP_PORT}:80
links:
- db
- redis
And my Dockerfile:
FROM composer:latest AS composer
WORKDIR /var/www/html/app/
FROM php:7.2-fpm-stretch
RUN apt-get update && apt-get install -y \
supervisor \
nginx \
zip
ADD docker/nginx.conf /etc/nginx/nginx.conf ADD
docker/virtualhost.conf /etc/nginx/conf.d/default.conf ADD
docker/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
ARG enviroment
COPY --from=composer /usr/bin/composer /usr/bin/composer
COPY .env.local .env RUN chmod -R g+w /var/www/html/app/bootstrap
RUN composer install RUN php artisan key:generate
EXPOSE 80
CMD ["/usr/bin/supervisord"]
I want to clone the repository and when doing a docker-compose build that does the following in the dockerfile:
rename .env.local to .env
give permissions to the storage folder. I have an error in this line
RUN chmod -R g+w /var/www/html/app/bootstrap
chmod: cannot access '/var/www/html/app/bootstrap': No such file or
directory
docker-compose.yaml: ${APP_PORT} take values from .env.local (I tried with env_file but it does not work
In your Dockerfile there is no COPY action to copy all your current project code into created image. Therefore bootstrap folder is not exist in your image. So chmod tells you exactly that.
Volumes (this line - .:/var/www/html/app) will sync your current directory with container later when it will be created depending on image structure. So if you want to give permissions to bootstrap folder then copy project code into image before giving permissions to it.
Add this line before permission operations to make folders accessible.
COPY . /var/www/html/app
I'm using an Apache / MySql Docker-compose set up which is all good. However the issue comes when, as this is for local development, the web container points to a local folder, for which I need Apache to have permissions to.
Using
RUN mkdir /www \
&& chown -R apache:apache /www
VOLUME ["/www"]
is fine if I run the Apache dockerfile by itself or if I run it in docker-compose without specifying a volume. But this means that I can't point that volume at a local directory, in this scenario "www" exists inside the container but doesn't map to the host machine. If I specify a volume inside the docker-compose file then it maps as expected but doesn't allow me to CHOWN the folder / files (even if I exec into the container)
Below is a proof of concept, I'm running on Windows 10 / Docker Desktop Community Version 2.0.0.0-win81 (29211)
EDIT (commented exposing the port, built the dockerfile from docker-compose and changed the port to 80 from 81)
EDIT (I've updated the following files, see bottom, I'm leaving these for posterity)
docker-compose.yml
version: '3.2'
services:
web:
restart: always
build:
context: .
ports:
- 80:80
volumes:
- ./:/www
Dockerfile
FROM centos:centos6 as stage1
RUN yum -y update && yum clean all \
&& yum --setopt=tsflags=nodocs install -y yum-utils \
httpd \
php
FROM stage1 as stage2
RUN mkdir /www \
&& chown -R apache:apache /www
#VOLUME ["/www"]
#EXPOSE 80
ENTRYPOINT ["/usr/sbin/httpd", "-D", "FOREGROUND"]
UPDATED Proof of concept files
Docker-compose.yml
version: '3.2'
services:
web:
build:
context: .
ports:
- 80:80
volumes:
- ./:/www
Dockerfile
FROM centos:centos6
RUN yum -y update && yum clean all \
&& yum --setopt=tsflags=nodocs install -y yum-utils \
httpd \
php
COPY ./entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]
entrypoint.sh
#!/bin/bash
set -e #exit straight away if there's an issue
chown -R apache:apache /www
# Apache
/usr/sbin/httpd -D FOREGROUND
Docker for Windows uses a CIFS/Samba network file share to bind-mount host files into the Linux VM running docker. That is always done as root:root so all bind-mount files/dirs will always show that when seen from inside container. This is a known limitation of the way docker shares these files between the OS's.
Workarounds:
In many cases, this isn't an issue. The host files are shared into the container world-readable, so local app development while running in the container is fine. For cache files, user uploads, etc. just be sure they are written into a container path that isn't to the host-bind mount, so they stay in Linux where you can control the perms.
If needed, for development only, run the app in the container as root if it needs write permissions to host OS files. You can override this at runtime: e.g. docker run -u root or user:root in docker-compose.yml
For working with database files, don't bind-mount them, but use named volumes to keep the files in the Linux VM. You can always use docker cp to copy files in and out of volumes for a quick backup.
You're using
RUN mkdir /www \
&& chown -R apache:apache /www
Prior to docker-compose mapping the local . directory to www.
You need to create a file entrypoint.sh or similar. Give it a shbang. And inside that you should run chown -R apache:apache /www. You do not need the mkdir as that's created by docker compose volume config ./:/www.
After that command in your entrypoint.sh file you should add in what you currently have for your entrypoint /usr/sbin/httpd -D FOREGROUND.
Then finally you of course need to set your new entrypoint to use the entrypoint.sh file ENTRYPOINT ["/entrypoint.sh"]
I cannot create a directory with the mkdir command in a container with dockerfile.
My Dockerfile file is simply ;
FROM php:fpm
WORKDIR /var/www/html
VOLUME ./code:/var/www/html
RUN mkdir -p /var/www/html/foo
In this way I created a simple php: fpm container.
and I wrote to create a directory called foo.
docker build -t phpx .
I have built with the above code.
In my docker-compose file as follows.
version: '3'
services:
web:
container_name: phpx
build : .
ports:
- "80:80"
volumes:
- ./code:/var/www/html
later; run the following code and I entered the container kernel.
docker exec -it phpx /bin/bash
but there is no a directory called foo in / var / www / html.
I wonder where I'm doing wrong.
Can you help me?
The reason is that you are mounting a volume from your host to /var/www/html.
Step by step:
RUN mkdir -p /var/www/html/foo creates the foo directory inside the filesystem of your container.
docker-compose.yml ./code:/var/www/html "hides" the content of /var/www/html in the container filesystem behind the contents of ./code on the host filesystem.
So actually, when you exec into your container you see the contents of the ./code directory on the host when you look at /var/www/html.
Fix: Either you remove the volume from your docker-compose.yml or you create the foo-directory on the host before starting the container.
Additional Remark: In your Dockerfile you declare a volume as VOLUME ./code:/var/www/html. This does not work and you should probably remove it. In a Dockerfile you cannot specify a path on your host.
Quoting from docker:
The host directory is declared at container run-time: The host directory (the mountpoint) is, by its nature, host-dependent. This is to preserve image portability. since a given host directory can’t be guaranteed to be available on all hosts. For this reason, you can’t mount a host directory from within the Dockerfile. The VOLUME instruction does not support specifying a host-dir parameter. You must specify the mountpoint when you create or run the container.
I am able to create a directory inside the 'workdir' for docker as follows:
Dockerfile content
COPY src/ /app
COPY logging.conf /app
COPY start.sh /app/
COPY Pipfile /app/
COPY Pipfile.lock /app/
COPY .env /app/
RUN mkdir -p /app/logs
COPY logs/some_log.log /app/logs/
WORKDIR /app
I have not mentioned the volume parameter in my 'docker-compose.yaml' file
So here is what I suggest: Remove the volume parameter from the 'Dockerfile' as correctly pointed by the Fabian Braun.
FROM php:fpm
RUN mkdir -p /var/www/html/foo
WORKDIR /var/www/html
And remove the volume parameter from the docker-compose file. It will work. Additionally, I would like to know how you tested of there is a directory named 'foo'.
Docker-compose file content
version: '3'
services:
web:
build:
context: .
dockerfile: Dockerfile # The name of your docker file
container_name: phpx
ports:
- "80:80"
You can use the SHELL instruction of Dockerfile.
ENV HOME /usr/local
SHELL ["/bin/sh", "-c"]
RUN mkdir $HOME/logs