I have succesfully created a volume (/code in the container) which holds django app (/web locally). When I spin up the dev environment, I want to have a folder (holding dummy data) available within the container, but I want to COPY the data, not have it as a VOLUME (so that if it's deleted in the container, it's not deleted from my local file system).
Folder structure:
...
docker-compose.yml
docker-compose.dev.yml
webms/
data/
folder/
subdir1/
file1
subdir2/
web/
Dockerfile
...
docker-compose.yml
...
services:
web:
volumes:
- ./webms/web:/code
build:
context: ./webms
...
docker-compose.dev.yml
...
services:
web:
volumes:
- ./webms/data/folder:/code/folder
...
Dockerfile
...
WORKDIR /code
ADD ./web/ .
COPY ./data/folder/ ./folder/
...
I can add data/folder as a volume succesfully, but then changes in the container reflect on my file system.
If remove the volumes directive from docker-compose.dev.yml, volumes in docker-compose.yml overwrites /code in the container and /folder is not longer available.
If I remove all the volumes directives, all the files are there, but changes to code in /web isn't live anymore (have to restart the container each time).
How do I copy data/folder into the container and have it available every time I restart the container / have a copy of the local file system copied into the volume? It may not be possible.
I have looked at similar answers on SO and they don't address this case.
Related
I have a simple .dockerignore file with the following:
.git/
.idea/
venv/
My docker-compose.yml file mounts the volume:
version: "3"
services:
frontend:
build: .
command: ["gunicorn", "--bind", "0.0.0.0:8000", "project.app:create_app()"]
env_file:
- .env
volumes:
- .:/frontend
ports:
- "8000:8000"
Perhaps I don't understand the full syntax or intent of the .dockerignore file, but after running docker-compose up --build, .git/, .idea/ and venv/ end up in my container.
I've read up and saw this but it doesn't seem feasible that one cannot mount and prevent files and directories from landing in the container.
How do I prevent these directories from becoming available in the container?
The .dockerignore file will modify the context that is sent to the docker daemon to build the image. So the created image will not have these files inside unless you recreate them.
The volume mount is completely separate and a host volume mount, aka bind mount, will map the contents of the host directory directly into the container as is. This is a Linux OS level activity that doesn't follow .dockerignore.
You will need to exclude those files from the directory that you mount into the container, or not mount the volume into the container and rely on the image to take advantage of .dockerignore.
I have a docker-compose.yml file that mounts a couple of volumes. Here is a snippet:
version: '3'
services:
gitlab-runner:
build: '.'
volumes:
- gitlab-config-volume:/etc/gitlab-runner
volumes:
gitlab-config-volume:
external: false
However, my Dockerfile has a COPY action into /etc/gitlab-runner/certs
FROM gitlab/gitlab-runner:latest
COPY files/ca.crt /etc/gitlab-runner/certs/ca.crt
The problem is, that this COPY happens before the mount. Is there a way I can work around this issue?
The easiest approach is to not mount a volume over content configured in your Dockerfile.
services:
gitlab-runner:
build: .
# no volumes:
If the volume actually is configuration, and it includes environment-specific settings like TLS CA certificates, it might not make sense to include this in your image at all. It will usually be easier to inject these files from the host system than to try to copy them into a named Docker volume.
services:
gitlab-runner:
# (could still `build: .`, but don't copy config into the image)
image: gitlab/gitlab-runner:latest
volumes:
# uses a host directory and not a named volume
- ./gitlab-config:/etc/gitlab-runner
cp files/ca.crt gitlab-config/certs/ca.crt
docker-compose up -d
Finally, if you really want the file to be included in your image, but to be able to mount some other content into it, you need to run code in your container to copy the file into the volume. This happens at container startup, so it will happen after the volume mount. This is the most complex option, though.
The shell script is straightforward:
#!/bin/sh
# Copy the CA certificate into the configuration directory if required.
if [ -f /etc/gitlab-runner/ca.crt ]; then
cp /opt/config/ca.crt /etc/gitlab-runner/certs
fi
# Run the main container command.
exec "$#"
In your Dockerfile, you need to make this script be the ENTRYPOINT (with JSON-array syntax); you need to repeat the CMD from the base image; and you need to copy the default file into the filesystem, somewhere other than the volume mount point. Doing this correctly involves knowing some details of the base image, and finding its Dockerfile is all but essential.
FROM gitlab/gitlab-runner:latest
COPY files/ca.crt /opt/config
COPY entrypoint.sh /opt/config
# These last two lines are derived from the base image and are not generic
ENTRYPOINT ["/dumb-init", "/opt/config/entrypoint.sh", "/entrypoint"]
CMD ["run", "--user=gitlab-runner", "--working-directory=/home/gitlab-runner"]
If I attach myself to the container and check the files inside /app I can see my host content inside valve_controller, modify it, etc.
I can't see the files during the build process (RUN ls /app/ trows nothing). I need to verify the code and then compile it.
Are volumes mounted after the build generation?
Which option do I have that doesn't involve COPY?
version: '3.7'
services:
valve_controller:
container_name: "valve_controller"
build:
context: .
dockerfile: ./valve_controller/Dockerfile
working_dir: /app
tty: true
volumes:
- ./valve_controller:/app
Dockerfile
VOLUME /app
RUN ls /app/
Volumes are mounted only when the container is run, not during the build process. This is intentional, since the image generation should not depend on anything outside your build context (the directory where your Dockerfile is). If you need any files during image build, you should COPY them in.
Volumes are mounted the moment you run the container. Therefore you can't refer to the files during the build process.
Adding a command tag in the docker-compose with a list of commands that need to run separated with && or ; would do the trick.
It's also possible to create and initial image with the volume and import that one.
When i use docker-compose with volumes to sync my files from host to container i cant see any new files created in the dockerfile
my docker-compose.yml:
version: "3"
services:
web:
image: nginx:latest
volumes:
- ./:/code
links:
- php
php:
build: .
volumes:
- ./:/code
My dockerfile looks like this:
FROM php:7-fpm
WORKDIR /code
RUN touch testfile
Of course thats a simplified example, but why do I not see the "testfile" on my host System? if I use docker-compose exec php touch testfile everything works as expected, i see the testfile on my host.
From my understanding I do need to see it outside of my container for it to be shared with the other containers with the same volume (in this example nginx)
When you run RUN touch testfile within your Dockerfile it creates the testfile within the image itself.
Now when you start your container and volume mount your ./ directory to /code, it will mount over your existing /code folder in the image which is why you see it empty. If you didn't add the volume mount in your compose file, it would have the testfile in there.
Note: I don't fully understand your use case but if you wanted your image to create that file within the volume mount, you would need to add it to your entrypoint.
I have a simple .dockerignore file with the following:
.git/
.idea/
venv/
My docker-compose.yml file mounts the volume:
version: "3"
services:
frontend:
build: .
command: ["gunicorn", "--bind", "0.0.0.0:8000", "project.app:create_app()"]
env_file:
- .env
volumes:
- .:/frontend
ports:
- "8000:8000"
Perhaps I don't understand the full syntax or intent of the .dockerignore file, but after running docker-compose up --build, .git/, .idea/ and venv/ end up in my container.
I've read up and saw this but it doesn't seem feasible that one cannot mount and prevent files and directories from landing in the container.
How do I prevent these directories from becoming available in the container?
The .dockerignore file will modify the context that is sent to the docker daemon to build the image. So the created image will not have these files inside unless you recreate them.
The volume mount is completely separate and a host volume mount, aka bind mount, will map the contents of the host directory directly into the container as is. This is a Linux OS level activity that doesn't follow .dockerignore.
You will need to exclude those files from the directory that you mount into the container, or not mount the volume into the container and rely on the image to take advantage of .dockerignore.