I am trying to change some configuration files required for application setup but do not want to change the original config file in the source code.
The path to the original config file is /usr/src/app/env_configs/local_db_setup.rb
The way I try to achieve this is in my Dockerfile
cp <path of new config on host>/local_db_setup.rb /usr/src/app/env_configs/
However, I perceive that due to my volume mounted in docker-compose.yml, the Copy is not taking place or overridden.
volumes:
-.:/usr/src/app
How can I go about this?
If you bind-mount into a non-empty directory on the container, the directory’s existing contents are obscured by the bind mount. So any existing contents at /usr/src/app inside the container are shadowed.
If you want to copy the new config file during image build time as mentioned in your question, you can copy it to a different directory in the image(/tmp/config/) and move it to correct location (/usr/src/app/env_configs/) using an entrypoint script which does the move first and then starts the actual entrypoint.
Instead you can also directly mount the config file from host machine if that is okay.
Related
I am checking the docker documentation on how to use named volumes to share data between containers.
In Populate a volume using a container it is specified that:
If you start a container which creates a new volume, as above, and the container has files or directories in the directory to be mounted (such as /app/ above), the directory’s contents are copied into the volume. The container then mounts and uses the volume, and other containers which use the volume also have access to the pre-populated content.
So I did a simple example where:
I start a container which creates the volume and mounts it to a directory with existing files
I start a second container on which I mount the volume and indeed I can see the first container's files.
So far so good.
However I wanted to see if it is possible to have pre-populated content from more than one containers.
What I did was
Create two simple images which have their respective configuration files in the same directory
FROM alpine:latest
WORKDIR /opt/test
RUN mkdir -p "/opt/test/conf" && \
echo "container from image 1" > /opt/test/conf/config_1.cfg
FROM alpine:latest
WORKDIR /opt/test
RUN mkdir -p "/opt/test/conf" && \
echo "container from image 2" > /opt/test/conf/config_2.cfg
Create a docker compose which defines a named volume which is mounted on both services
services:
test_container_1:
image:
test_image_1
volumes:
- test_volume:/opt/test/conf
tty: true
test_container_2:
image:
test_image_2
volumes:
- test_volume:/opt/test/conf
tty: true
volumes:
test_volume:
Started the services.
> docker-compose -p example up
Creating network "example_default" with the default driver
Creating volume "example_test_volume" with default driver
Creating example_test_container_2_1 ... done
Creating example_test_container_1_1 ... done
Attaching to example_test_container_1_1, example_test_container_2_1
According to the logs container_2 was created first and it pre-populated the volume. However, the volume was then mounted to container_1 and the only file available on the mount was apparently /opt/test/conf/config_2.cfg effectively removing config_1.
So my question is, if it is possible to have a volume populated with data from 2 or more containers.
The reason I want to explore this, is so that I can have additional app configuration loaded from different containers, to support a multi tenant scenario, without having to rework the app to read the tenant configuration from different folders.
Thank you in advance
Once there is any content in a named volume at all, Docker will never automatically copy content into it. It will not merge content from two different images, update the volume if one of the images changes, or anything else.
I'd advise you to ignore the paragraph you quote in the Docker documentation. Assume any volume you mount into the container is initially empty. This matches the behavior you'll get with Docker bind-mounts (host directories), Kubernetes persistent volumes, and basically any other kind of storage besides Docker named volumes proper. Don't mount a volume over the content in your image.
If you can, restructure your application to avoid sharing files at all. One common use of named volumes I see is trying to republish static assets to a reverse proxy, for example; rather than trying to use a named volume (which will never update itself) you can COPY the static assets into a dedicated Web server image. This avoids the various complexities around trying to use a volume here.
If you really don't have a choice in the matter, then you can approach this with dedicated code in both of the containers. The basic setup here is:
Have a data directory somewhere outside your application directory, and mount the volume there.
Include the original files in the image somewhere different.
In an entrypoint wrapper script, copy the original files into the data directory (the mounted volume).
Let's say for the sake of argument that you've installed the application into /opt/test, and the data directory will be /etc/test. The entrypoint wrapper script can be as little as
#!/bin/sh
# Copy config files from the application tree into the config tree
# (overwriting anything that's already there)
cp /opt/test/* "$TEST_CONFIG_DIR"
# Run the main container command
exec "$#"
In the Dockerfile, you need to make sure that directory exists (and if you'll use a non-root user, that user needs permission to write to it).
FROM alpine
WORKDIR /opt/test
COPY ./ ./
ENV TEST_CONFIG_DIR=/etc/test
RUN mkdir "$TEST_CONFIG_DIR"
ENTRYPOINT ["./entrypoint.sh"]
CMD ["./my_app"]
Finally, in the Compose setup, mount the volume on that data directory (you can't use the environment variable, but consider the filesystem path part of the image's API):
version: '3.8'
volumes:
test_config:
services:
one:
build: ./one
volumes:
- test_config:/etc/test
two:
build: ./two
volumes:
- test_config:/etc/test
You would be able to run, for example,
docker-compose run one ls /etc/test
docker-compose run two ls /etc/test
to see both sets of files appear there.
The entrypoint script is code you control. There's nothing especially magical about it beyond the final exec "$#" line to run the main container command. If you want to ignore files that already exist, for example, or if you have a way to merge in changes, then you can implement something more clever than a simple cp command.
I am trying to develop a project locally using Docker Compose and to prevent re-building my image on every update, I've added a bind-mount that maps my src directory to my WORDIR in Docker. All changes made on my local machine are then reflected in my Docker container...EXCEPT for one file. For some reason, there's a single file in my project, that when I change its contents, the change is not reflected in the Docker container even though other files adjacent to this file DO detect file changes. Which leads me to believe that the directory is mapped correctly but it's some other issue with the file itself?
docker-compose.yaml
graphql:
build:
context: .
dockerfile: ./app/graphql/src/Dockerfile
target: development
volumes:
- ./app/graphql/src:/workspace
- /workspace/node_modules/
Dockerfile
# ------------> Base Image
FROM node:14 AS base
WORKDIR /workspace
COPY ./app/graphql/src .
# ------------> Development Image
FROM base AS development
CMD ["npm", "run", "dev"]
I haven't figured out how to show directory structure but the files that I am modifying are located in:
/app/graphql/src/api/graphql
Where file a.ts detects changes and is reflected in the Docker container but b.ts does not. I read about how Docker depends on the inode of the file to match if bind mounting specific files. I'm mounting a directory, but for a sanity check, I ran:
ls -i
in both the host and container and confirmed that the inodes matched.
I have two M1 Mac computers and I confirmed that this is a problem between both machines.
Any additional thoughts to debug this problem? My only other thought is that I hit a max number of files that can be tracked, but that's why I removed the node_modules. Any assistance would be really helpful!
EDIT: I created a new file, c.ts and duplicated the contents of b.ts (the file that wasn't changing between host and container)...and c.ts detects changes properly! Is there a way to inspect why a certain file isn't broadcasting changes? This is so strange.
You should remove COPY ./app/graphql/src . directive from your Dockerfile because this folder will mounted to container as volume.
I'm trying to deploy a Nextcloud container, where the config is copied from the local directory to the container. I'm not getting any error when building or running the container, and I can see the steps are successfully executed per the terminal. Regardless, the copied file simply is not in the container. What's going on here?
Dockerfile:
FROM nextcloud:latest
# Copy local config
COPY ./config.php /var/www/html/config
All the evidence:
Thanks!
The file is copied but is being deleted later.
This is a very typical scenario, and in this cases, the best you can do is to see what happens in the parent image nextcloud:latest once the container starts.
In nextcloud's Dockerfile you can see
ENTRYPOINT ["/entrypoint.sh"]
if we open entrypoint.sh in the line 100 you can see clearly that the content of /var/www/html/config is modified
You can maybe do any of these options
Copy the file to a different temporary location, and create your own entrypoint (you can copy-paste from the original one to hit the ground running, or you can try to figure out a fancier solution)
Or also you can copy the file after creating and running the container
docker cp config.php copytest:/var/www/html/config
Currently I'm trying to mount two folders (./app + ./test/public) and one file (./test/test.py) into a shared folder in the container (type: bind), so I always have the current code in the container without restarting. The problem is that the content in /test is also mounted to /app in the host system. Can this be avoided?
Here is my example file:
volumes:
- "./app:/app"
- "./test/public:/app/test/public"
- "./test/test.py:/app/test.py"
I've searched the ninternet for about an hour now and read the docker-compose documentation, but i coudn't figure out how to solve this problem..
Hope you can help :)
edit: after docker-compose up the ./app on the host machine contains
./app/test/public and ./app/test.py too; So i simply want to mount and merge these folders without changing the host files.
I don't think you can avoid this behavior: Docker needs to create filesystem entries to which it can attach the bind mounts for your test/... mounts. If you're bind-mounting a file, a file must exist at the target location first; similarly for a directory.
That means that before performing your bind mounts, Docker first creates a new (empty) file or directory to provide the target for the mount. This is what you see inside your app directory.
Your options are either (a) just live with it, or (b) restructure your project so that you don't need to bind mount things into an existing bind mount.
A few lines from docker-compose.yml
volumes:
- ./yii-application:/app/yii-application
- /app/yii-application/common/config/
First line adds to the volume an entire application.
The second one makes some sort of exclude of config folder, so I do not need my config from host machine.
A few lines from Dockerfile
COPY ./config-${APP_ENV}/common /app/yii-application/common/config
Instead of COPY I tried
RUN cp -a /app/config-${APP_ENV}/common/. /app/yii-application/common/config
It does not work either.
I think there is an issue in the order of the commands that are being executed:
When you are building and image with Dockerfile, you are coping the code inside dir /app/yii-application/common/config.
Then, you are mounting a volume: volumes: /app/yii-application/common/config/
and overwriting the existing dir with an empty dir that serves as a volume.
You need to work around that issue.
I moved the config files out of volume directory.
So, now the volume looks like this
volumes:
- ./yii-application:/app/yii-application
Config files in
./