Docker compose mapping local directory to dockerfile volume - docker

I'm using an Apache / MySql Docker-compose set up which is all good. However the issue comes when, as this is for local development, the web container points to a local folder, for which I need Apache to have permissions to.
Using
RUN mkdir /www \
&& chown -R apache:apache /www
VOLUME ["/www"]
is fine if I run the Apache dockerfile by itself or if I run it in docker-compose without specifying a volume. But this means that I can't point that volume at a local directory, in this scenario "www" exists inside the container but doesn't map to the host machine. If I specify a volume inside the docker-compose file then it maps as expected but doesn't allow me to CHOWN the folder / files (even if I exec into the container)
Below is a proof of concept, I'm running on Windows 10 / Docker Desktop Community Version 2.0.0.0-win81 (29211)
EDIT (commented exposing the port, built the dockerfile from docker-compose and changed the port to 80 from 81)
EDIT (I've updated the following files, see bottom, I'm leaving these for posterity)
docker-compose.yml
version: '3.2'
services:
web:
restart: always
build:
context: .
ports:
- 80:80
volumes:
- ./:/www
Dockerfile
FROM centos:centos6 as stage1
RUN yum -y update && yum clean all \
&& yum --setopt=tsflags=nodocs install -y yum-utils \
httpd \
php
FROM stage1 as stage2
RUN mkdir /www \
&& chown -R apache:apache /www
#VOLUME ["/www"]
#EXPOSE 80
ENTRYPOINT ["/usr/sbin/httpd", "-D", "FOREGROUND"]
UPDATED Proof of concept files
Docker-compose.yml
version: '3.2'
services:
web:
build:
context: .
ports:
- 80:80
volumes:
- ./:/www
Dockerfile
FROM centos:centos6
RUN yum -y update && yum clean all \
&& yum --setopt=tsflags=nodocs install -y yum-utils \
httpd \
php
COPY ./entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]
entrypoint.sh
#!/bin/bash
set -e #exit straight away if there's an issue
chown -R apache:apache /www
# Apache
/usr/sbin/httpd -D FOREGROUND

Docker for Windows uses a CIFS/Samba network file share to bind-mount host files into the Linux VM running docker. That is always done as root:root so all bind-mount files/dirs will always show that when seen from inside container. This is a known limitation of the way docker shares these files between the OS's.
Workarounds:
In many cases, this isn't an issue. The host files are shared into the container world-readable, so local app development while running in the container is fine. For cache files, user uploads, etc. just be sure they are written into a container path that isn't to the host-bind mount, so they stay in Linux where you can control the perms.
If needed, for development only, run the app in the container as root if it needs write permissions to host OS files. You can override this at runtime: e.g. docker run -u root or user:root in docker-compose.yml
For working with database files, don't bind-mount them, but use named volumes to keep the files in the Linux VM. You can always use docker cp to copy files in and out of volumes for a quick backup.

You're using
RUN mkdir /www \
&& chown -R apache:apache /www
Prior to docker-compose mapping the local . directory to www.
You need to create a file entrypoint.sh or similar. Give it a shbang. And inside that you should run chown -R apache:apache /www. You do not need the mkdir as that's created by docker compose volume config ./:/www.
After that command in your entrypoint.sh file you should add in what you currently have for your entrypoint /usr/sbin/httpd -D FOREGROUND.
Then finally you of course need to set your new entrypoint to use the entrypoint.sh file ENTRYPOINT ["/entrypoint.sh"]

Related

Exposing Docker Volumes to Nginx

I'm trying to connect a Json file which resides in a docker volume of the following container to my main docker container which is running a django project.
Since I am using Caprover my Docker Compose options are very limited.
So Docker Composer is not really an option. I want to instead just expose the json file over the web with a link.
Something like domain.com/folder/jsonfile.json
Can somebody tell me if this is possible inside this dockerfile?
The image I am using is crucial to the container so can I just add an nginx image or do I need any other changes to make this work?
Or is nginx not even necessary?
FROM ubuntu:devel
ENV TZ=Etc/UTC
ARG APP_HOME=/app
WORKDIR ${APP_HOME}
ENV DEBIAN_FRONTEND=noninteractive
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime
RUN echo $TZ > /etc/timezone
RUN apt-get update && apt-get upgrade -y
RUN apt-get install gnumeric -y
RUN mkdir -p /etc/importer/data
RUN mkdir /voldata
COPY config.toml /etc/importer/
COPY datasets/* /etc/importer/data/
VOLUME /voldata
COPY importer /usr/bin/
RUN chmod +x /usr/bin/importer
COPY . ${APP_HOME}
CMD sleep 999d
Using the same volume in 2 containers
docker-compose:
volumes:
shared_vol:
services:
service1:
volumes:
- 'shared_vol:/path/to/file'
service2:
volumes:
- 'shared_vol:/path/to/file'
the mechanism above replaces the volumes_from since v3, but this works for v2 as well:
volumes:
shared_vol:
services:
service1:
volumes:
- 'shared_vol:/path/to/file'
service2:
volumes_from:
- service1
If you want to avoid unintentional altering add :ro for readonly to the target service:
service1:
volumes:
- 'shared_vol:/path/to/file'
service2:
volumes:
- 'shared_vol:/path/to/file:ro'
http-server
Surely you can provide the file via http (or other protocol). There are two oppertunities:
Including a http-service to your container (quite easy depending on what is already given in the container) e.g. using nodejs you can use this https://www.npmjs.com/package/http-server very easy. Size doesn't matter? So just install:
RUN apt-get install -y nodejs npm
RUN npm install -g http-server
EXPOSE 8080
CMD ["http-server", "--cors", "-p8080", "/path/to/your/json"]
docker-compose (Runs per default on 8080, so open this):
existing_service:
ports:
- '8080:8080'
Run a stand alone http-server (nginx, apache httpd,..) in another container, but then you depend again on using the same volume for two services, so for local solutions quite an overkill.
Base image
If you don't have good reasons i'll would never use something like :devel, :rolling or :latest as base image. Stick to a LTS version instead like ubuntu:22.04
Testing for http-server
Dockerfile
FROM ubuntu:20.04
ENV TZ=Etc/UTC
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt-get update
RUN apt-get install -y nodejs npm
RUN npm install -g http-server#13.1.0 # Issue with JSON-File in V14: https://github.com/http-party/http-server/issues/634
COPY ./test.json ./usr/wwwhttp/test.json
EXPOSE 8080
CMD ["http-server", "--cors", "-p8080", "/usr/wwwhttp/"]
# docker build -t test/httpserver:latest .
# docker run -p 8080:8080 test/httpserver:latest
Disclaimer:
I am not that familiar with node-docker-images, this is just to give a quick working solution and go on from there. I'm not using nodeJS in production, but I'm sure it can be optimized from being fat to.. well.. being rather fat. But for quick prototyping size doesn't matter.
If you want to just have two containers access the same file, just use a volume with --mount.

How to run only specific command as root and other commands with default user in docker-compose

This is my Dockerfile.
FROM python:3.8.12-slim-bullseye as prod-env
RUN apt-get update && apt-get install unzip vim -y
COPY requirements.txt /app
RUN pip install -r requirements.txt
USER nobody:nogroup
This is how docker-compose.yml looks like.
api_server:
build:
context: .
target: prod-env
image: company/server
volumes:
- ./shared/model_server/models:/models
- ./static/images:/images
ports:
- 8200:8200
command: gunicorn -b 0.0.0.0:8200 --threads "8" --log-level info --reload "server:gunicorn_app(command='start', project='app_server')"
I want to add permissions read, write and execute permissions on shared directories.
And also need to run couple of other coommands as root.
So I have to execute this command with root every time after image is built.
docker exec -it -u root api_server_1 bash -c "python copy_stuffs.py; chmod -R a+rwx models; chmod -R a+rwx /images"
Now, I want docker-compose to execute these lines.
But as you can see, user in docker-compose has to be nobody as specified by Dockerfile. So how can I execute root commands in docker-compose file?
Option that I've been thinking:
Install sudo command from Dockerfile and use sudo
Is there any better way ?
In docker-compose.yml create another service using same image and volumes.
Override user with user: root:root, command: your_command_to_run_as_root, for this new service and add dependency to run this new service before starting regular working container.
api_server:
build:
context: .
target: prod-env
image: company/server
volumes:
- ./shared/model_server/models:/models
- ./static/images:/images
ports:
- 8200:8200
command: gunicorn -b 0.0.0.0:8200 --threads "8" --log-level info --reload "server:gunicorn_app(command='start', project='app_server')"
# This make sure that startup order is correct and api_server_decorator service is starting first
depends_on:
- api_server_decorator
api_server_decorator:
build:
context: .
target: prod-env
image: company/server
volumes:
- ./shared/model_server/models:/models
- ./static/images:/images
# No ports needed - it is only decorator
# Overriding USER with root:root
user: "root:root"
# Overriding command
command: python copy_stuffs.py; chmod -R a+rwx models; chmod -R a+rwx /images
There are other possibilities like changing Dockerfile by removing USER restriction and then you can use entrypoint script doing as root what you want as privileged user and running su - nobody or better exec gosu to retain PID=1 and proper signal handling.
In my eyes the approach giving a container root rights is quite hacky and dangerous.
If you want to e.g. remove the files written by container you need root rights on host as well.
If you want to allow a container to access files on host filesystem just run the container with appropriate user.
api_server:
user: my_docker_user:my_docker_group
then give on host the rights to that group
sudo chown -R my_docker_user:my_docker_group models
You should build all of the content you need into the image itself, especially if you have this use case of occasionally needing to run a process to update it (you are not trying to use an isolation tool like Docker to simulate a local development environment). In your Dockerfile, COPY these directories into the image
COPY shared/model_server/models /models
COPY static/images /images
Do not make these directories writeable, and do not make the individual files in the directories executable. The directories will generally be mode 0755 and the files mode 0644, owned by root, and that's fine.
In the Compose setup, do not mount host content over these directories either. You should just have:
services:
api_server:
build: . # use the same image in all environments
image: company/server
ports:
- 8200:8200
# no volumes:, do not override the image's command:
Now when you want to update the files, you can rebuild the image (without interrupting the running application, without docker exec, and without an alternate user)
docker-compose build api_server
and then do a relatively quick restart, running a new container on the updated image
docker-compose up -d

docker-compose doesn't see contents of a particular directory

I've got a docker-compose.yml file that mounts the current directory as a volume to /app in a container. The .yml looks something like this:
version: "3"
services:
app:
build:
context: .
dockerfile: docker/Dockerfile-commandbox
volumes:
- .:/app
ports:
- "8080:8080"
environment:
- TZ=${TIMEZONE-America/Los_Angeles}
Pretty basic.
Now, when I ssh into that container and navigate to the /app directory, it can see the ./wwwroot folder, but its contents are empty, despite not being empty on my host machine. Performing a directory listing on the wwwroot folder in the container displays no results; whereas the folder has lots of content.
What would be causing the container to not be able to see the contents of the wwwroot folder?
The Dockerfile-commandbox file is pretty straightforward as well.
FROM ortussolutions/commandbox:4.8.0
RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y \
net-tools \
tzdata \
vim \
&& rm -rf /var/lib/apt/lists/*
COPY scheduled/scheduled-tasks.cfm /app/
WORKDIR /app
RUN box server start cfengine=lucee#5.3.7.47 port=8080 serverHomeDirectory=/root/serverHome host=0.0.0.0 openbrowser=false saveSettings=false heapSize=4096 minHeapSize=4096 \
&& box config set server.defaults.app.cfengine=lucee#5.3.7.47 server.defaults.web.AJP.enable=true \
&& curl -sS http://localhost:8080/scheduled-tasks.cfm \
&& box server stop
UPDATE
To be clear, there are other files in the directory structure and each one is visible, with its contents, from the container.
This must have been a glitch in Docker. On 12/18 release 3.0.2 was available. Installing the update, and bringing up the containers made the issue go away. To confirm I rolled back to 3.0.1, and witnessed the issue again. I also did a full purge of my containers by running docker system prune -a and rebuilding everything from scratch.
As of Docker Desktop Community edition for Mac 3.0.2, this is no longer an issue.

docker-compose up gives error: bash: sails: command not found

I have a docker-compose.yml file with the following content:
version: '2'
services:
MongoDB:
image: mongo
Parrot-API:
build: ./Parrot-API
image: sails-js:dev
volumes:
- "/user/Code/node/Parrot-API:/host"
command: bash -c "cd /host && sails lift"
links:
- MongoDB:MongoDB
ports:
- "3050:1337"
The file basically runs two containers: mongodb and web app (in directory ./Parrot-API) built in sails.js. However, when I run docker-compose up in the terminal, I got this error: Parrot-API_1 | bash: sails: command not found
node_Parrot-API_1 exited with code 127. Note that sails.js is a node.js web framework, and sails lift starts the app at port 1337.
I have done some google search and have found some similar questions, but not helpful in my case.
btw, I have the following Dockerfile in the Parrot-API folder:
FROM sails-js:dev
VOLUME /host
WORKDIR /host
RUN rm -rf node_modules && \
echo "hello world!" && \
pwd && \
ls -lrah
EXPOSE 1337
CMD npm install -g sails && npm install && sails lift
The file structure is following:
|- docker-compose.yml
|- Parrot-API/Dockerfile
|- Parrot-API/app.js, etc..
It is clear to me that the Parrot-API docker container exits immediately due to the reason that sails lift command is not executed, but how to make the container work? Thanks!
You showed a docker-compose.yml that builds a sails-js:dev image, and you showed a Dockerfile that is based on the sails-js:dev image. This appears to be recursive.
Your Dockerfile itself ends with a CMD in lieu of an ENTRYPOINT that does the npm install of sails. Since you did this as a CMD instead of a RUN, sails is not installed in your image, the install is launched on a container run, but only if you don't run the container with any arguments of your own, like you are doing in the docker-compose.yml with a custom command.
The fix is to update the Dockerfile with a proper base image and change the CMD to a RUN. I'm also seeing a few other mistakes like creating a volume and then modifying the contents, where volumes ignore other changes after they have been created. The FROM node is just a guess based on your npm commands, feel free to adjust:
FROM node
RUN mkdir -p /host && cd /host && npm install -g sails && npm install
EXPOSE 1337
WORKDIR /host
VOLUME /host
CMD sails lift

How can I write Dockerfile for Yesod? "RUN yesod init -n myApp -d postgresql" didn't work as expected

I tried to make a simple application with Yesod and PostgreSQL using Docker Compose but RUN yesod init -n myApp -d postgresql didn't seem to work as expected.
I defined Dockerfile and docker-compose.yml as below:
Dockerfile:
FROM shuny/ghc-7.8.4:latest
MAINTAINER shuny
# Create default config
RUN cabal update
# Add stackage remote repo
RUN sed -i 's/^remote-repo: [a-zA-Z0-9_\/:.]*$/remote-repo: stackage:http:\/\/www.stackage.org\/lts/g' /root/.cabal/config
# Update packages
RUN cabal update
# Generate locale otherwise happy (because of tf-random) will fail
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
RUN echo $LANG
# Install build tools for yesod
RUN cabal install alex happy yesod-bin
# Install library for yesod-postgres
RUN apt-get update && apt-get install -y libpq-dev
RUN mkdir /code
WORKDIR /code
RUN yesod init -n myApp -d postgresql
WORKDIR /code/myApp
RUN cabal sandbox init && cabal install --only-dependencies --max-backjumps=-1 --reorder-goals
RUN cabal configure && cabal build && cabal install
ADD . /code
WORKDIR /code
# ADD settings.yml /code/myApp/config/
docker-compose.yml:
database:
image: postgres
ports:
- "5432"
web:
build: .
tty: true
command: yesod devel
volumes:
- .:/code/
ports:
- "3000:3000"
links:
- database
and docker-compose build returned as below:
Step 0 : FROM shuny/ghc-7.8.4:latest
...
Step 17 : WORKDIR /code
---> Running in bf99d0aca48c
---> 37c3c94338d7
Removing intermediate container bf99d0aca48c
Successfully built 37c3c94338d7
but when I check like this:
$docker-compose run web /bin/bash
root#0fe5fb1a3b20:/code# ls
root#0fe5fb1a3b20:/code#
it showed nothing while this commands seem to work as expected:
docker run -ti 37c3c94338d7
root#31e94428de37:/code# ls
docker-compose.yml Dockerfile myApp settings.yml
root#31e94428de37:/code# ls myApp/
app config Handler Model.hs Settings.hs test
Application.hs dist Import myApp.cabal static
cabal.sandbox.config Foundation.hs Import.hs Settings templates
How can I fix it?
I really appliciate any feedback, thank you.
You are doing strange things with volumes and the ADD instruction.
First you build your application inside the image:
RUN yesod init -n myApp -d postgresql
WORKDIR /code/myApp
RUN cabal sandbox init && cabal install --only-dependencies --max-backjumps=-1 --reorder-goals
RUN cabal configure && cabal build && cabal install
Then you add the content of the folder that contains the Dockerfile in the /code folder of the image. I guess this step is useless.
ADD . /code
Then, if you run a container without -volume option, everything works fine
docker run -ti 37c3c94338d7
But in your docker-compose.yml file, you specified a volume option that overides the /code folder in the container with the folder that contains the docker-compose.yml file on the host machine. Therefore, you don't have the content generated during the build of your image anymore.
There are two possibilities:
Don't use the volume instruction in the docker-compose.yml file
Put the content of the /code/myApp/ folder of the image inside the ./myApp folder of the host.
It depends on why you want to use the volume option in docker-compose.yml.
I don't really know what is your goal. But if what you are trying to do is to access to the files built inside the container from the host machine, maybe this should do what you are looking for:
Remove the build steps from your Dockerfile
Run a shell inside a "web" container: docker-compose run web bash
Launch the build commands
So you will have built your application while the volume was mounted and will see the files on the host machine.
Exit the shell
Launch Docker Compose normally
If you just want to be able to backup the content of the /code/myApp/ folder, maybe you should omit the path on the host machine from the volume section of docker-compose.yml.
volumes:
- /code/
And follow this section of the documentation
I hope it helps

Resources