I have a bash script recipe that creates large assets. I'd like to create them once and use them in different docker containers. I'm assuming the best way to handle this is to create a docker volume containing these assets but how do I do that? I prefer not copy files into the volume directly from the host as that's not really version controlled. Can this be achieved using dockerfiles?
You can use docker-compose.yml to set up a volume sharing between containers. And you can declare mount points in respective Dockerfiles using VOLUME instruction. More in detail explanation is in Volumes Documentation
I think this answers my question. In summary:
Create a Dockerfile that's along these lines:
FROM ubuntu
RUN mkdir /dataset
RUN ***populate /dataset***
VOLUME /dataset
Build the docker image
Build a container from that image
Mount /dataset in any container you need using --volumes-from option
Related
Docker run command has option to mount host directory into container
-v=[]: Create a bind mount with: [host-dir]:[container-dir]:[rw|ro].
If "host-dir" is missing, then docker creates a new volume.
And Dockerfile has VOLUME instruction
VOLUME ["/data"] - The VOLUME instruction will add one or more new volumes
to any container created from the image.
From what I see, there is no way to specify host-dir or rw/ro status when using Dockerfile.
Is there any other use of VOLUME in docker file other than wanting to share it with some other container?
Dockerfiles are meant to be portable and shared. The host-dir volume is something 100% host dependent and will break on any other machine, which is a little bit off the Docker idea.
Because of this, it is only possible to use portable instructions within a Dockerfile. If you need a host-dir volume, you need to specify it at run-time.
A common usage of VOLUME from Dockerfile is to store configuration or website sources so that it can be updated later by another container.
I would like to distribute some larger static files/assets as a Docker image so that it is easy for user to pull those optional files down the same way they would be pulling the app itself. But I cannot really find a good way to expose files from one Docker image to the other? Is there a way to mount a Docker image itself (or a directory in it) as a volume to other Docker container?
I know that there are volume plugins I could use, but I could not find any where I could to this or something similar?
Is possible create any directory of an image to a docker volume, but not full image. At least not in a pretty or simple way.
If you want to create a directory from your image as a docker volume you can create a named volume:
docker volume create your_volume
docker run -d \
-it \
--name=yourcontainer \
-v your_volume:/dir_with_data_you_need \
your_docker_image
From this point, you'll have accessible your_volume with data from image your_docker_image
Reason why you cannot mount the whole image in a volume is because docker doesn't let specify / as source of named volume. You'll get Cannot create container for service your-srv: invalid volume spec "/": invalid volume specification: '/' even if you try with docker-compose.
Don't know any direct way.
You can use a folder in your host as a bridge to share things, this is a indirect way to acheive this.
docker run -d -v /any_of_your_host_folder:/your_assets_folder_in_your_image_container your_image
docker run -d -v /any_of_your_host_folder:/your_folder_of_your_new_container your_container_want_to_use_assets
For your_image, you need add CMD in dockerfile to copy the assets to your_assets_folder_in_your_image_container(the one you use as volume as CMD executes after volume)
This may waste time, but just at the first time the assets container starts. And after the container starts, the files in assets container in fact copy to the host folder, and has none business with assets image any more. So you can just delete the image of the assets image. Then no space waste.
You aim just want other people easy to use the assets, so why not afford script to them, automatically fetch the image -> start the container(CMD auto copy files to volume) -> delete the image/container -> the assets already on host, so people just use this host assets as a volume to do next things.
Of course, if container can directly use other image's resource, it is better than this solution. Anyway, this can be a solution although not perfect.
You can add the docker sock as a volume which will allow you to start one of your docker images from within your docker container.
To do this, add the two following volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
- "/usr/bin/docker:/usr/bin/docker"
If you need to share files between the containers map the volume /tmp:/tmp when starting both containers.
I want to share a file storage between two containers. From the documentation, I've seen that you can create and use volumes like this:
docker volume create --name DataVolume1
docker run -ti --rm -v DataVolume1:/datavolume1 ubuntu
However, I want containers to be able to access an initial set of shared data. Does docker support publishing of volumes? If not, does this mean I should write the initial data manually, after creating the volume, or is there another solution for publishing the data along with the images?
With a named volume (not with a host volume, aka bind mount) docker will initialize an empty named volume to the contents of the image at the location you mount it. So if you have files in your image at /datavolume1, and DataVolume1 is empty, docker will copy those files into the named volume.
I am trying to capture the state of a docker container as an image, in a way that includes files I have added to a volume within the container. So, if I run the original container in this way:
$ docker run -ti -v /cookbook ubuntu:14.04 /bin/bash
root#b78f3599d936:/# cd cookbook
root#b78f3599d936:/cookbook# touch foo.txt
Now, if I either export, or commit the container as a new docker image, and then run a container from the new image, then the file, foo.txt is never included in the /cookbook directory.
My question is whether there is a way to create an image from a container in a way that allows the image to include file content within its volumes.
whether there is a way to create an image from a container in a way that allows the image to include file content within its volumes?
No, because volume is designed to manage data inside and between your Docker containers, it's used to persist and share data. What's in image is usually your program(artifacts, executables, libs. e.g) with its whole environment, building/updating data to image does not make much sense.
And in docs of volumes, they told us:
Changes to a data volume will not be included when you update an image.
Also in docs of docker commit:
The commit operation will not include any data contained in volumes mounted inside the container.
Well, by putting the changes in a volume, you're excluding them from the actual container. The documentation for docker export includes this:
The docker export command does not export the contents of volumes associated with the container. If a volume is mounted on top of an existing directory in the container, docker export will export the contents of the underlying directory, not the contents of the volume.
Refer to Backup, restore, or migrate data volumes in the user guide for examples on exporting data in a volume.
This points to this documentation. Please follow the steps there to export the information stored in the volume.
You're probably looking for something like this:
docker run --rm --volumes-from <containerId> -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /cookbook
This would create a file backup.tar with the contents of the container's /cookbook directory and store it in the current directory of the host. You could then use this tar file to import it in another container.
Essentially, there are three ways to do persistence in Docker:
You can keep files in a volume, which is a filesystem managed by Docker. This is what happens in your example: because the /cookbook directory is part of a volume, your file does not get commited/exported with the image. It does however get stored in the volume, so if you remount the same volume in a different container, you will find your file there. You can list your volumes using docker volume ls. As you can see, you should probably give your volumes names if you plan to reuse them. You can mount an existing volume, or create a new one, if the name does not exist, with
docker run -v name:/directory ubuntu
You can keep files as part of the image. If you commit the container, all changes to its file hierarchy are stored in the new image except those made to mounted volumes. So if you just get rid of the -v flag, your file shows up in the commit.
You can bind mount a directory from the host machine to the container, by using the -v /hostdir:/targetdir syntax. The container then simply has access to a directory of the host machine.
Docker commit allows you to create an image from a container and its data (mounted volumes will be ignored)
Docker run command has option to mount host directory into container
-v=[]: Create a bind mount with: [host-dir]:[container-dir]:[rw|ro].
If "host-dir" is missing, then docker creates a new volume.
And Dockerfile has VOLUME instruction
VOLUME ["/data"] - The VOLUME instruction will add one or more new volumes
to any container created from the image.
From what I see, there is no way to specify host-dir or rw/ro status when using Dockerfile.
Is there any other use of VOLUME in docker file other than wanting to share it with some other container?
Dockerfiles are meant to be portable and shared. The host-dir volume is something 100% host dependent and will break on any other machine, which is a little bit off the Docker idea.
Because of this, it is only possible to use portable instructions within a Dockerfile. If you need a host-dir volume, you need to specify it at run-time.
A common usage of VOLUME from Dockerfile is to store configuration or website sources so that it can be updated later by another container.