I have been trying to read up on Docker volumes for a while now, but something still seems to be missing.
Take this Dockerfile: (taken from docs)
FROM ubuntu
RUN mkdir /myvol
RUN echo "hello world" > /myvol/greeting
VOLUME /myvol
If I build and run this simply as docker run <img>, then a new volume is created (its name can be found using docker inspect <containerid>). The contents of /var/lib/docker/volumes/<volumeid> is just a file greeting containing hello world, as expected.
So far so good. Now, say I wanted, as a user, to place this volume to a custom location. I would run docker run -v /host/path/:/myvol. This does not work anymore. /host/path is created (if it does not exist), but is empty.
So, the question is, how do I put some files in a Volume in Dockerfile, whilst allowing the user to choose where they put this volume? This seems to me to be one of the reasons to use Volume directive in the first place.
So, the question is, how do I put some files in a Volume in
Dockerfile, whilst allowing the user to choose where they put this
volume?
You don't. A volume is created at runtime, while your Dockerfile is consulted at build time.
If you want to be able to pre-populate a volume at an arbitrary location when your container runs, allow the user to provide the path to the volume in an environment variable and then have an ENTRYPOINT script that copies the files into the volume when the container starts.
An alternative is to require the use of a specific path, and then taking advantage of the fact that if the user mounts a new, empty volume on that path, Docker will copy the contents of that directory into the volume. For example, if I have:
FROM alpine
RUN mkdir /data; echo "This is a test" > /data/datafile
VOLUME /data
And I build an image testimage from that, and then run:
docker run -it testimage sh
I see that my volume in /data contains the file(s) from the underlying filesystem:
/ # ls /data
datafile
This won't work if I mount a host path on that location, for example if I do:
host# mkdir /tmp/empty
host# docker run -it -v /tmp/empty:/data testimage sh
Then the resulting path will be empty:
/ # ls /data
/ #
This is another situation in which an ENTRYPOINT script could be used to populate the target directory.
Related
I would like to have a docker volume to persist data. The persisted data can be accessed by different containers based on different images.
It is not a host volume. It is a volume listed in volumes panel of Docker Desktop.
For example, the name of the volume is theVolume which is mounted at /workspace. The directory I need to inspect is /workspace/project.
I need to check whether a specific directory is available inside the volume. If it is not, create the directory, else leave it as is.
Is it possible to do this from within a Dockerfile or compose file?
It's possible to do this in an entrypoint wrapper script. This runs as the main container process, so it's invoked after the volume is mounted in the container. The script isn't aware of what specific thing might be mounted on /workspace, so this will work whether you've mounted a named volume, a host directory, or nothing at all. It does need to make sure to actually start the main container command when it's done.
#!/bin/sh
# entrypoint.sh
# Create the project directory if it doesn't exist
if [ ! -d /workspace/project ]; then
mkdir /workspace/project
fi
# Run the main container command
exec "$#"
Make sure this file is executable on your host system (run chmod +x entrypoint.sh before checking it in). Make sure it's included in your Docker image, and then make this script be the image's ENTRYPOINT.
COPY entrypoint.sh ./ # if a previous `COPY ./ ./` doesn't already get it
ENTRYPOINT ["./entrypoint.sh"] # must use JSON-array syntax
CMD the main container command # same as you have now
(If you're using ENTRYPOINT for the main container command, you may need to change it to CMD for this to work; if you've split the interpreter into its own ENTRYPOINT line, combine the whole container command into a single CMD.)
A Dockerfile RUN command happens before the volume is mounted (or maybe even exists at all) and so it can't modify the volume contents. A Compose file doesn't have any way to run commands, beyond replacing the image's entrypoint and command.
I'm a new leaner of docker.I came a cross a problem while I'm trying to make my own docker image.
Here's the thing.I create a new DockerFile to build my own mysql image in which I declared MYSQL_ROOT_PASSWORD and put some init scripts in the container.
Here is my Docker
FROM mysql:5.7
MAINTAINER CarbonFace<553127022#qq.com>
ENV TZ Asia/Shanghai
ENV MYSQL_ROOT_PASSWORD Carbon#mysqlRoot7
ENV INIT_DATA_DIR /initData/sql
ENV INIT_SQL_FILE_0 privileges.sql
ENV INIT_SQL_FILE_1 carbon_user_sql.sql
ENV INIT_SQL_FILE_2 carbonface_sql.sql
COPY ./my.cnf /etc/mysql/donf.d/
RUN mkdir -p $INIT_DATA_DIR
COPY ./sqlscript/$INIT_SQL_FILE_0 $INIT_DATA_DIR/
COPY ./sqlscript/$INIT_SQL_FILE_1 $INIT_DATA_DIR/
COPY ./sqlscript/$INIT_SQL_FILE_2 $INIT_DATA_DIR/
COPY ./sqlscript/$INIT_SQL_FILE_0 /docker-entrypoint-initdb.d/
COPY ./sqlscript/$INIT_SQL_FILE_1 /docker-entrypoint-initdb.d/
COPY ./sqlscript/$INIT_SQL_FILE_2 /docker-entrypoint-initdb.d/
CMD ["mysqld"]
I'm trying to build a docker image which contains my own config file and when mounted it would be showed in the local directory and can be modified.
I'm really confused that when I start my container with this image like the official description and also here is my commands:
docker run -dp 3306:3306 \
-v /usr/local/mysql/data:/var/lib/mysql \
-v/usr/local/mysql/conf:/etc/mysql/conf.d \
--name mysql mysql:<my builded tag>
You know I'm trying to mounted the
/usr/local/mysql/conf to the /etc/mysql/conf.d in the container which is been told as the custom config file mounted location.
And I supposed that my custom config file my.cnf which has been copied into the image during docker build and would be show in my local direcroty /usr/local/mysql/conf
And since I already copied my custom config file into image which you can see in my DockerFile.
But it turns out that the directory is empty and the /etc/mysql/conf.d is also overwrite by local directory.
Before I run my container, both /usr/local/mysql/conf and /usr/local/mysql/data is empty at all.
OK fine, I've been told that the volume mounted directory would overwrite the file inside the container.
But how could the empty data directory shows the data files inside the container but the empty conf directory overwrite the conf.d directory in the container.
It make no sense.
I was very confused and I would be very appreciate it if someone can explain why it happens.
My OS is MacOS Big Sur and I used the latest docker.
A host-directory bind mount, -v /host/path:/container/path, always hides the contents of the image and replaces it with the host directory. If the host directory is empty, the container directory will be the same empty directory at container startup time.
The Docker Hub mysql container has an involved entrypoint script that checks to see if the data directory is empty, and if so, initializes the database; abstracted out
#!/bin/sh
# (actually in hundreds of lines of shell code, with more options)
if [ ! -d /var/lib/mysql/data/mysql ]; then
mysql_install_db
# (...and start a temporary database server and run the
# /docker-entrypoint-initdb.d scripts)
fi
# then run the main container command
exec "$#"
Simply the presence of a volume doesn't cause files to be copied (with one exception at one specific point in the lifecycle for named volumes), so if you need to copy content from a container to the host you either need to do it manually with docker cp or have a way in the container code to do it.
I want to prepare a base docker image that executes a command and produces a report. I want this to be persisted within the image for the next docker file. Is this possible?
What you can do is to set a WORKDIR within your dockerfile e.g.
WORKDIR /data
and use a volume with the run command for the built image.
docker run -v /users/home/your_user/use_case_folder:/data -it your_image_tag /bin/bash
When you run your reports/predictions you have to write them to /data, then your files will be placed on your local system. You now can use the next
Dockerfile and use the same volume mount, which is defined as WORKDIR within the new Dockerfile. Sharing the results within the image itself for another image is not possible as far as I know. You will always have to use an outside mounted File System or Database or sth. similar.
Maybe here you can find some infos too:
https://docs.docker.com/storage/volumes/#restore-container-from-backup
I want to make a docker image that keeps my application configuration, so when something changes I can only change the config container and don't have to build a new image for my application.
Here is my Dockerfile:
FROM scratch
RUN mkdir -p /config
ADD config.properties /config
VOLUME /config
ENTRYPOINT /bin/true
But it can't even create the directory. Is there a best practice for such things?
Keep in mind that the scratch image is literally completely empty. You cannot create the directory, because there's no /usr/bin/mkdir executable in that image.
To create the directory anyway, you can exploit the fact that the ADD statement in a Dockerfile also implicitly creates directories, so the following Dockerfile should be enough:
FROM scratch
ADD config.properties /config/config.properties
VOLUME /config
Regarding the ENTRYPOINT; there's also no /bin/true in your image. This means that the container will not start (i.e. exit immediately with exec: "/bin/true": stat /bin/true: no such file or directory). However, as you intend to use this image for a data-only container, that's probably OK. Simply use docker create instead of docker run to create the container without starting it:
docker build -t config_image .
docker create --name config config_image
What I am trying to do is set up a docker container for ghost where I can easily modify the theme and other content. So I am making /opt/ghost/content a volume and mounting that on the host.
It looks like I will have to manually copy the theme into the host directory because when I mount it, it is an empty directory. So my content directory is totally empty. I am pretty sure I am doing something wrong.
I have tried a few different variations including using ADD with default themes folder, putting VOLUME at the end of the Dockerfile. I keep ending up with an empty content directory.
Does anyone have a Dockerfile doing something similar that is already working that I can look at?
Or maybe I can use the docker cp command somehow to populate the volume?
I may be missing something obvious or have made a silly mistake in my attempts to achieve this. But the basic thing is I want to be able to upload a new set of files into the ghost themes directory using a host-mounted volume and also have the casper theme in there by default.
This is what I have in my Dockerfile right now:
FROM ubuntu:12.04
MAINTAINER Jason Livesay "ithkuil#gmail.com"
RUN apt-get install -y python-software-properties
RUN add-apt-repository ppa:chris-lea/node.js
RUN echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list
RUN apt-get -qq update
RUN apt-get install -y sudo curl unzip nodejs=0.10.20-1chl1~precise1
RUN curl -L https://en.ghost.org/zip/ghost-0.3.2.zip > /tmp/ghost.zip
RUN useradd ghost
RUN mkdir -p /opt/ghost
WORKDIR /opt/ghost
RUN unzip /tmp/ghost.zip
RUN npm install --production
# Volumes
RUN mkdir /data
ADD run /usr/local/bin/run
ADD config.js /opt/ghost/config.js
ADD content /opt/ghost/content/
RUN chown -R ghost:ghost /opt/ghost
ENV NODE_ENV production
ENV GHOST_URL http://my-ghost-blog.com
EXPOSE 2368
CMD ["/usr/local/bin/run"]
VOLUME ["/data", "/opt/ghost/content"]
As far as I know, empty host-mounted (bound) volumes still will not receive contents of directories set up during the build, BUT data containers referenced with --volumes-from WILL.
So now I think the answer is, rather than writing code to work around non-initialized host-mounted volumes, forget host-mounted volumes and instead use data containers.
Data containers use the same image as the one you are trying to persist data for (so they have the same directories etc.).
docker run -d --name myapp_data mystuff/myapp echo Data container for myapp
Note that it will run and then exit, so your data containers for volumes won't stay running. If you want to keep them running you can use something like sleep infinity instead of echo, although this will obviously take more resources and isn't necessary or useful unless you have some specific reason -- like assuming that all of your relevant containers are still running.
You then use --volumes-from to use the directories from the data container:
docker run -d --name myapp --volumes-from myapp_data
https://docs.docker.com/userguide/dockervolumes/
You need to place the VOLUME directive before actually adding content to it.
My answer is completely wrong! Look here it seems there is actually a bug. If the VOLUME command happens after the directory already exists in the container, then changes are not persisted.
The Dockerfile should always end with a CMD or an ENTRYPOINT.
UPDATE
My solution would be to ADD files in the container home directory, then use a shell script as an entry point in which I'll copy the file in the shared volume and do all the other tasks.
I've been looking into the same thing. The problem I encountered was that I was using a relative local mount path, something like:
docker run -i -t -v ../data:/opt/data image
Switching to an absolute local path fixed this up for me:
docker run -i -t -v /path/to/my/data:/opt/data image
Can you confirm whether you were doing a relative path, and whether this helps?
Docker V1.8.1 preserves data in a volume if you mount it with the run command. From the docker docs:
Volumes are initialized when a container is created. If the container’s
base image contains data at the specified mount point, that existing
data is copied into the new volume upon volume initialization.
Example: An image defines the
/var/www/html
as a volume and populates it with the data of a web application. Your docker hosts provides a mount directory
/my/host/dir
You start the image by
docker run -v /my/host/dir:/var/www/html image
then you will get all the data from /var/www/html in the hosts /my/host/dir
This data will persist even if you delete the container or the image.