I am building my first docker image. I am a beginner
It is a simple python http server. This is my DockerFile
FROM python:3.8.0-slim
WORKDIR /src
COPY src/ .
CMD [ "python", "-m", "http.server", "--cgi", "8000"]
I have a config folder in /src with some config files.
I named the image "my-server"
I create a container with
docker run -d \
--name "my-server" \
-p 8000:8000 \
-v /dockerdata/appdata/my-server/config/:/src/config \
--restart unless-stopped \
my-server
the issue is /dockerdata/appdata/my-server/config/ is empty on my host.
I see this done on all docker images on dockerhub I use and the mounted volumes are not empty for these images.
How do they do it?
Their startup sequence explicitly copies source files into the volume, or otherwise creates them. A Docker mount always replaces the content in the image with the content of whatever's being mounted; there is no way to mount the container content to the host.
(The one exception to this "always" is, if you're using native Docker, and you're mounting a named volume, and the named volume is empty, then content from the image is copied into the volume first; but the content is never ever updated, it only works for named volumes and not other kinds of mounts, and it doesn't work on other environments like Kubernetes. I would not rely on this approach.)
If the configuration is a single file, this isn't a huge imposition. You probably already need to distribute artifacts like a docker-compose.yml file separately from the image, so distributing a default configuration isn't much more. If defaults are compiled into your application and an empty configuration is valid, this also simplifies things. Another helpful approach could be to have a search path for configuration files, and read both a "user" and "system" configuration.
If you do need to copy files out to a host directory or other mount point, I would generally do this with an entrypoint wrapper script. You will need to keep a copy of the configuration in the image somewhere that's not the actual config directory so that you can copy it when it doesn't exist. The script can be fairly straightforward:
#!/bin/sh
# Copy the default configuration if it doesn't exist
if [ ! -f config/config.yml ]; then
cp default-config/config.yml config
fi
# Run the main container command
exec "$#"
You may need to do some shuffling in your Dockerfile; the important thing is to make this script be the ENTRYPOINT but leave the CMD unchanged.
# Save the "normal" config away; the entrypoint script will create
# the "real" config if one isn't mounted
RUN mv config default-config \
&& mkdir config
# Launch the server via the entrypoint wrapper
ENTRYPOINT ["./entrypoint.sh"] # must be JSON array syntax
CMD ["python", "-m", "http.server", "--cgi", "8000"] # unchanged
This is expected, bind mount to a container directory will result to the content in the directory to be obscured. If you mount to a named volume, the directory’s contents are copied into the volume.
docker run -d \
--name "my-server" \
-p 8000:8000 \
-v myvol:/src/config \
--restart unless-stopped \
my-server
Now if you run docker run -it -v myvol:/config --rm busybox ls /config you will see the copied content.
Related
I have a volume which contains data that needs to stay persisted. When creating the volume for the first time, and mounting it to my node container, all container contents are copied to the volume, and everything behaves as expected. The issue is that when I change a few files in my node container, I remove the old image and container, and rebuild them from scratch. When running the updated container, the container's files don't get copied into the volume. This means that the volume still contains the old files, and therefore when the volume is mounted in the container, no updated functionality is present, and I have to remove and recreate the volume from scratch, which I can't do since the volume's data needs to be persisted.
Here is my dockerfile
FROM mcr.microsoft.com/dotnet/sdk:5.0
COPY CommandLineTool App/CommandLineTool/
COPY NeedBackupNodeServer App/NeedBackupNodeServer/
WORKDIR /App/NeedBackupNodeServer
RUN curl -sL https://deb.nodesource.com/setup_14.x | bash - \
&& apt update \
&& apt install -y nodejs
EXPOSE 5001
ENTRYPOINT ["node", "--trace-warnings", "index.js"]
Here are my commands and expected output
docker volume create nodeServer-and-commandLineTool-volume
docker build -t need-backup-image -f Dockerfile .
docker run -p 5001:5001 --name need-backup-container -v nodeServer-and-commandLineTool-volume:/App need-backup-image -a
when running
docker exec need-backup-container cat index.js
the file is present and contains the latest updates, since the volume was just created.
Now when I update some files, I need to rebuild the image and the container, so I run
docker rm need-backup-container
docker rmi need-backup-image
docker build -t need-backup-image -f Dockerfile .
docker run -p 5001:5001 --name need-backup-container -v nodeServer-and-commandLineTool-volume:/App need-backup-image -a
Now I thought that when running
docker exec need-backup-container cat index.js
I'd see the updated file changes, but nope, I only see the old files that were first created when the volume was mounted for the first time.
So my question is, is there anyway to achieve overwriting the volume's files when creating a container?
If your application needs persistent data, it should be stored in a different directory from the application code. This can be in a dedicated /data directory or in a subdirectory of your application; the important thing is that, when you mount a volume to hold the persistent data, it does not hide your application code.
In a Node application, for example, you could refer to a ./data for your data files:
import { open } from 'fs/promises';
import { join } from 'path';
const dataDir = process.env.DATA_DIR || 'data';
const fh = await open(join(dataDir, 'file.txt'), 'rw');
Then in your Dockerfile you'd need to create that directory. If you set up a non-root user, that directory, but not your code, should be owned by the user.
FROM node:lts
# Create the non-root user
RUN adduser --system --no-create-home nonroot
# Install the Node application normally
WORKDIR /app
COPY package*.json .
RUN npm ci
COPY index.js .
# Create the data directory
RUN mkdir data && chown nonroot data
# Specify how to run the container
USER nonroot
CMD ["node", "index.js"]
Then when you launch the container, mount the volume only on the data directory, not over the entire /app tree.
docker run \
-p 5001:5001 \
--name need-backup-container \
-v nodeServer-and-commandLineTool-volume:/app/data \
need-backup-image
# ^^^^^^^^^
Note that the Dockerfile as shown here would also let you use a host directory instead of a Docker named volume, and specify the host uid when you run the container. You do not need to make any changes to the image to do this.
docker run \
-p 5002:5001 \
--name same-image-with-bind-mount \
-u $(id -u) \
-v "$PWD/app-data:/app/data" \
need-backup-image
I try to create a docker container with bind9 on it and I want to add my db.personal-domain.com file but when I run docker build and then docker run -tdp 53:53 -v config:/etc/bind <image id> the container doesn't have my db.personal-domain.com file. How to fix that ? Thanks !
tree structure
-DNS
--Dockerfile
--config
---db.personal-domain.com
Dockerfile
FROM ubuntu:20.04
RUN apt-get update
RUN apt-get install -y bind9
RUN apt-get install -y bind9utils
WORKDIR /etc/bind
VOLUME ["/etc/bind"]
COPY config/db.personal-domain.com /etc/bind/db.personal-domain.com
EXPOSE 53/tcp
CMD ["/usr/sbin/named", "-g", "-c", "/etc/bind/named.conf", "-u", "bind"]
There is a syntax issue in your docker run -v option. If you use docker run -v name:/container/path (even if name matches a local file or directory) it mounts a named volume over your configuration directory. You want the host content, and you need -v /absolute/host/path:/container/path syntax (the host path must start with /). So (on Linux/MacOS):
docker run -d -p 53:53 \
-v "$PWD:config:/etc/bind" \
my/bind-image
In your image you're trying to COPY in the config file. This could work also; but it's being hidden by the volume mount, and also rendered ineffective by the VOLUME statement. (The most obvious effect of VOLUME is to prevent subsequent changes to the named directory; it's not required to mount a volume later.)
If you delete the VOLUME line from the Dockerfile, it should also work to run the container without the -v option at all. (But if you'll have different DNS configuration on different setups, this probably isn't what you want.)
-v config:/etc/bind
That syntax creates a named volume, called config. It looks like you wanted a host volume pointing to a path in your current directory, and for that you need to include a fully qualified path with the leading slash, e.g. using $(pwd) to generate the path:
-v "$(pwd)/config:/etc/bind"
How can I use a dockerfile to copy files from my host to the image at create time, and make these file available to the host machine when the container is created? I'm lost between 1) VOLUME=[..] in the Dockerfile 2) --volume in the run command 3) docker volume create
I tried creating a dockerfile to copy the files to the image. though if I run the container with the -v option, the target on the container seems to get cleared?? example:
FROM node:latest
WORKDIR /usr/src/myapp
COPY package*.json ./
RUN npm install
COPY index.js /usr/src/myapp/index.js
EXPOSE 8080
CMD [ "node", "/usr/src/myapp/index.js" ]
Running it without volume works great
docker run -p 8080:8080 --name Test test:latest
With volume however, it seems the target dir on the container is cleared (Error: Cannot find module '/usr/src/myapp/index.js')
docker run -p 8080:8080 -v /mynasvolume1/test:/usr/src/myapp --name Test test:latest
I know my alternative is to create a named volume on the host and include that as VOLUME in the Dockerfile. That is not my preference though as it doesn't allow me to select the volume location on my host machine (Synology NAS doesn't give me solutions to change the default location from /volume1/#docker/volumes to a location I can share or mount)
Yes it seems cleared, because mounting /mynasvolume1/test to /usr/src/myapp will show only files from /mynasvolume1/test and override the files you COPYed in you Dockerfile. It's unclear what you're trying to achieve, but try mounting to a specific directory like: /mynasvolume1/test:/usr/src/myapp/test
I created a Flask Application. This application receives a XML from a url and saves it:
response = requests.get(base_url)
with open('currencies.xml', 'wb') as file:
file.write(response.content)
When I run the application without Docker, the file currencies.xml is correctly created inside my app folder.
However, this behaviour does not occur when I use docker.
In docker I run the following commands:
docker build -t my-api-docker:latest .
docker run -p 5000:5000 my-api-docker ~/Desktop/myApiDocker # This is where I want the file to be saved: inside the main Flask folder
When I run the second command, I get:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"/Users/name/Desktop/myApiDocker\": stat /Users/name/Desktop/myApiDocker: no such file or directory": unknown.
ERRO[0001] error waiting for container: context canceled
But If I run:
docker build -t my-api-docker:latest .
docker run -p 5000:5000 my-api-docker # Without specifying the PATH
I can access the website (but it is pretty useless without the file currencies.xml
Dockerfile
FROM python:3.7
RUN pip install --upgrade pip
COPY ./requirements.txt /app/requirements.txt
WORKDIR /app
RUN pip install -r requirements.txt
COPY . /app
EXPOSE 5000
CMD [ "flask", "run", "--host=0.0.0.0" ]
When you
docker run -p 5000:5000 my-api-docker ~/Desktop/myApiDocker
Docker interprets everything after the image name (my-api-docker) as the command to run. It runs /Users/name/Desktop/myApiDocker as a command, instead of what you have as the CMD in the Dockerfile, and when that path doesn't exist in the container, you get the error you see.
It's a little unlikely you'll be able to pass this path to your flask run command as a command-line argument. A typical way of dealing with this is by using an environment variable instead. In your code,
download_dir = os.environ.get('DOWNLOAD_DIR', '.')
currencies_xml = os.path.join(download_dir, 'currencies.xml')
with open(currencies_xml, 'wb') as file:
...
Then when you start your container, you can pass that as an environment variable with the docker run -e option. Note that this names a path inside the container; there's no particular need for this to match the path on the host.
docker run \
-p 5000:5000 \
-e DOWNLOAD_DIR=/data \
-v $HOME/Desktop/myApiDocker:/data \
my-api-docker
It's also fairly common to put an ENV statement in your Dockerfile or otherwise pick a fixed path for this, and just specify that your image's interface is that it will download the file into whatever is mounted on /data.
When you docker run the image, the process' context is the container's file system not your host's file system. So my-api-docker ~/Desktop/myApiDocker (attempts to) place the file in the container's (!) ~/Desktop.
Instead you need to mount one of your host's directories into the container's file system and store the file in the mounted directory.
Something like:
docker run ... \
--volume=[HOST-PATH]:[CONTAINER-PATH] \
... \
my-api-docker [CONTAINER-PATH]/thefile
The container then writes the file to [CONTAINER-PATH]/thefile but this is mapped to the host's [HOST-PATH]/thefile.
NB The values for [HOST-PATH] and [CONTAINER-PATH] must be absolute not relative paths.
You may prove this behavior to yourself using e.g. either python:3.7 or busybox:
# List my host's root
ls -l /
# List the container's root
docker run --rm busybox ls -l /
# Mount the host's /tmp into the container's /tmp
ls -l /tmp
docker run --rm --volume=/tmp:/tmp busybox ls -l /tmp
HTH!
I'm trying to dockerize some services for development on my machine and wondering how docker run --volume=.. works. For example, if I do something like
docker run --volume=/path/to/data:/data [...]
will /path/to/data be (re)created locally only if it doesn't exist? Is the initial data copied from the container's image?
Links to relevant documentation would be appreciated.
The --volume option is described in the docker run reference docs, which forwards you on to the dedicated Managed data in containers docs, which then forwards you on to the Bind mounts docs.
There, it says:
If you use -v or --volume to bind-mount a file or directory that does not yet exist on the Docker host, -v will create the endpoint for you. It is always created as a directory.
Example:
docker run -it -v ${PWD}/scripts:/code/scripts myimage bash
Yes, the directory on the host FS will be created only if it does not already exist.
The same time, Docker will not copy anything from the image into bind-mounted volume, so the mount path will appear as empty directory inside the container. Whatever was in the image will be hidden.
If you need original data to be copied over, you need to implement this functionality yourself. Fortunately, it is pretty easy thing to do.
Among the last steps in Dockerfile, move or copy the original directory elsewhere. E.g. RUN mv /data /original-data
Add a custom script to the image, which would serve as entrypoint, and will copy the data needed into the mounted volume (see code example below). Dockerfile directive: ADD entrypoint.sh /entrypoint.sh
Add ENTRYPOINT directive, to instruct Docker to invoke your script as a part of container initialization: ENTRYPOINT ['/entrypoint.sh']
The script entrypoint.sh could look like following (simplified example):
#!/bin/bash
set -e
SOURCE_DIR=/original-data
TARGET_DIR=/data
if [ $(find $TARGET_DIR -maxdepth 0 -type d -empty) 2>/dev/null) ]; then
cp -r --preserve-all $SOURCE_DIR/* $TARGET_DIR/
fi
# continue Docker container initialization, execute CMD
exec $#
If there's already some entrypoint script in your image, you can just add appropriate logic to it.