I have two questions about docker volume.
First : When I run my docker with a volume which it creates with docker volume create or with docker-compose, new files on the default directory are not appended to the volume. I give an example :
docker volume create testdock-logs
docker run -d -p 80:80 --name testdock --restart=on-failure -v testdock-logs:/var/www/logs testdock:latest -s
After that when I create a new file or directory on /var/www/logs and build my image testdocker, I don't find my new file on the volume.
What am I doing wrong?
Docker doesn't replace files in a volume with those in a modified image. Doing so would defeat the purpose of a volume, this is the one persistent location you have to store data that isn't reverted back to the image state when the container is recreated.
If you need the files from your image instead of the files in your volume, then reconsider whether you actually need a volume.
Related
How would I go about starting a container with a volume that's already created? I have a volume for example named 'data' and 'docker volume ls' will show this. Now I need to start a container by attaching this existing volume.
Mounting a Data Volume
To mount a data volume to a container add the --mount flag to the docker run command. It adds the volume to the specified container, where it stores the data produced inside the virtual environment.
To run a container and mount a data volume to it, follow the basic syntax:
docker run --mount source=[volume_name],destination=[path_in_container]
Replace [path_in_container] with the path where you want to place the data volume in the container. Everything stored in that directory automatically gets saved on the data volume on the host as well.
For example, to launch an Ubuntu container and mount the data volume to it, run:
docker run -it --name=example1 --mount source=data,destination=/data ubuntu
I'm confused with what is different between creating docker volume create my-vol and VOLUME ["/var/www"].
My understanding is:
1) docker volume create my-vol creates a persistent volume on our machine and each container could be linked to my-vol.
2) VOLUME ["/var/www"] creates a volume inside its container.
And when I create another container, I could link my-vol as follows:
when running a container
$ docker run -d --name devtest --mount source=myvol2,target=/app nginx:latest
At that time, if I added VOLUME ["/var/www"] in my Dockerfile, all data of this docker file will be stored in both myvol2 and /var/www?
The Dockerfile VOLUME command says two things:
If the operator doesn't explicitly mount a volume on the specific container directory, create an anonymous one there anyways.
No Dockerfile step will ever be able to make further changes to that directory tree.
As an operator, you can mount a volume (either a named volume or a host directory) into a container with the docker run -v option. You can mount it over any directory in the container, regardless of whether or not there was a VOLUME declared for it in the Dockerfile.
(Since you can use docker run -v regardless of whether or not you declare a VOLUME, and it has confusing side effects, I would generally avoid declaring VOLUME in Dockerfiles.)
Just like in ordinary Linux, only one thing can be (usefully) mounted on any given directory. With the setup you describe, data will be stored in the myvol2 you create and mount, and it will be visible in /var/www in the container, but the data will only actually be stored in one place. If you deleted and recreated the container without the volume mount the data would not be there any more.
There are two types of persistent storage used in Docker,the first one is Docker Volumes and the second one is bind mounts. The differebce between them is that volumes are internal to Docker and stored in the Docker store (which is usually all under /var/lib/docker) and bind mounts use a physical location on your machine to store persistent data.
If you want to use a Docker Volume for nginx:
docker volume create nginx-vol
docker run -d --name devtest -v nginx-vol:/usr/share/nginx/html nginx
If you want to use a bind mount:
docker run -d --name devtest -v [path]:/usr/share/nginx/html nginx
[path] is the location in which you want to store the container's data.
Actually, I run my containers like this, for example :
docker run -v /nexus-data:/nexus-data sonatype/nexus3
^
After reading the documentation, I discover volumes that are completely managed by docker. For some reasons, I want to change the way to run my containers, to do something like this :
docker run -v nexus-data:/nexus-data sonatype/nexus3
^
I want to transfer my existing bind-mount to volumes.
But I don't want to lose the data into /nexus-data folder, is there a possibility to transfer this folder, to the new volume, whitout restart everything ? Because I've also Jenkins and Sonar containers for example, I just want to change the way to have persistent data. The is a proper way to do this ?
You can try out following steps so that you will not loose your current nexus-data.
#>docker run -v nexus-data:/nexus-data sonatype/nexus3
#>docker copy /nexus-data/. <container-name-or-id>:/nexus-data/
#>docker stop <container-name-or-id>
#>docker start <container-name-or-id>
docker copy will copy data from your host-machine's /nexus-data folder to container's FS /nexus-data folder which is your mounted volume.
Let me know if you face any issue while performing following steps.
Here's another way to do this, that I just used successfully with a Heimdall container. It's outlined in the documentation for the sonatype/nexus3 image:
Stop the running container (e.g. named nexus3)
Create a docker volume called nexus-data, creating it with the following command: docker volume create nexus-data)
By default, Docker will store the volume's content at /var/lib/docker/volumes/nexus-data/_data/
Simply copy the directory where you previously had been using a bind mount to the aforementioned volume directory (you'll need super user privileges to do this, or for the user to be part of the docker group): cp -R /path/to/nexus-data/* /var/lib/docker/volumes/nexus-data/_data/
Restart the nexus3 container with $ docker run -v nexus-data:/nexus-data sonatype/nexus3 --name=nexus3
Your container will be back up and running, with the files persisted in the directory /path/to/nexus-data/ now mirrored in the docker volume. Check if functionality is the same, of course, and if so, you can delete the /path/to/nexus-data/ directory
Q.E.D.
I am trying to capture the state of a docker container as an image, in a way that includes files I have added to a volume within the container. So, if I run the original container in this way:
$ docker run -ti -v /cookbook ubuntu:14.04 /bin/bash
root#b78f3599d936:/# cd cookbook
root#b78f3599d936:/cookbook# touch foo.txt
Now, if I either export, or commit the container as a new docker image, and then run a container from the new image, then the file, foo.txt is never included in the /cookbook directory.
My question is whether there is a way to create an image from a container in a way that allows the image to include file content within its volumes.
whether there is a way to create an image from a container in a way that allows the image to include file content within its volumes?
No, because volume is designed to manage data inside and between your Docker containers, it's used to persist and share data. What's in image is usually your program(artifacts, executables, libs. e.g) with its whole environment, building/updating data to image does not make much sense.
And in docs of volumes, they told us:
Changes to a data volume will not be included when you update an image.
Also in docs of docker commit:
The commit operation will not include any data contained in volumes mounted inside the container.
Well, by putting the changes in a volume, you're excluding them from the actual container. The documentation for docker export includes this:
The docker export command does not export the contents of volumes associated with the container. If a volume is mounted on top of an existing directory in the container, docker export will export the contents of the underlying directory, not the contents of the volume.
Refer to Backup, restore, or migrate data volumes in the user guide for examples on exporting data in a volume.
This points to this documentation. Please follow the steps there to export the information stored in the volume.
You're probably looking for something like this:
docker run --rm --volumes-from <containerId> -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /cookbook
This would create a file backup.tar with the contents of the container's /cookbook directory and store it in the current directory of the host. You could then use this tar file to import it in another container.
Essentially, there are three ways to do persistence in Docker:
You can keep files in a volume, which is a filesystem managed by Docker. This is what happens in your example: because the /cookbook directory is part of a volume, your file does not get commited/exported with the image. It does however get stored in the volume, so if you remount the same volume in a different container, you will find your file there. You can list your volumes using docker volume ls. As you can see, you should probably give your volumes names if you plan to reuse them. You can mount an existing volume, or create a new one, if the name does not exist, with
docker run -v name:/directory ubuntu
You can keep files as part of the image. If you commit the container, all changes to its file hierarchy are stored in the new image except those made to mounted volumes. So if you just get rid of the -v flag, your file shows up in the commit.
You can bind mount a directory from the host machine to the container, by using the -v /hostdir:/targetdir syntax. The container then simply has access to a directory of the host machine.
Docker commit allows you to create an image from a container and its data (mounted volumes will be ignored)
I'm working on a Docker-based setup for a simple web-app running in Nginx+php-fpm. The common suggestion I've seen for storing the actual PHP code is to store it on the host and then mount it read-only in both the Nginx and PHP containers.
However, I want my setup to be self-contained so I can easily use it on Amazon ECS with Auto Scaling. In other words, I want to bundle the code somehow, rather than pulling it from the host.
So it seems what I want is a read-only volume that can be shared between two Docker containers and won't persist after those containers are destroyed. Is this possible? Or is there a better approach?
Docker images can contain volumes that are pre-populated with data. To achieve this, in the Dockerfile, first populate a the directory (for example using COPY or RUN) and then declare it as a volume. This allows you to build an image that contains your application code inside a volume:
FROM php:7-fpm
COPY ./app /var/www/html
VOLUME /var/www/html
Creating a new container from this image will create a new volume, initialize it with the data from the image's /var/www/html directory and mount it inside your new container at the same location.
Compare the documentation for more information:
The docker run command initializes the newly created volume with any data that exists at the specified location within the base image. For example, consider the following Dockerfile snippet:
FROM ubuntu
RUN mkdir /myvol
RUN echo "hello world" > /myvol/greeting
VOLUME /myvol
This Dockerfile results in an image that causes docker run, to create a new mount point at /myvol and copy the greeting file into the newly created volume.
This allows you to simply start your application image with docker run:
docker run -d --name app my_application_image
Then you can run your Nginx container and configure it to use the same volumes as your application container using the --volumes-from flag:
docker run -d --name web --link app:app --volumes-from app my_nginx_image
After this, you will have a Docker volume containing your application code that is mounted in both containers at /var/www/html.