Assume that i have an application with this simple Dockerfile:
//...
RUN configure.sh --logmyfiles /var/lib/myapp
ENTRYPOINT ["starter.sh"]
CMD ["run"]
EXPOSE 8080
VOLUME ["/var/lib/myapp"]
And I run a container from that:
sudo docker run -d --name myapp -p 8080:8080 myapp:latest
So it works properly and stores some logs in /var/lib/myapp of docker container.
My question
I need these log files to automatically saved in host too, So how can i mount the /var/lib/myapp from the container to the /var/lib/myapp in host server (without removing current container) ?
Edit
I also see Docker - Mount Directory From Container to Host, but it doesn't solve my problem i need a way to backup my files from docker to host.
First, a little information about Docker volumes. Volume mounts occur only at container creation time. That means you cannot change volume mounts after you've started the container. Also, volume mounts are one-way only: From the host to the container, and not vice-versa. When you specify a host directory mounted as a volume in your container (for example something like: docker run -d --name="foo" -v "/path/on/host:/path/on/container" ubuntu), it is a "regular ole" linux mount --bind, which means that the host directory will temporarily "override" the container directory. Nothing is actually deleted or overwritten on the destination directory, but because of the nature of containers, that effectively means it will be overridden for the lifetime of the container.
So, you're left with two options (maybe three). You could mount a host directory into your container and then copy those files in your startup script (or if you bring cron into your container, you could use a cron to periodically copy those files to that host directory volume mount).
You could also use docker cp to move files from your container to your host. Now that is kinda hacky and definitely not something you should use in your infrastructure automation. But it does work very well for that exact purpose. One-off or debugging is a great situation for that.
You could also possibly set up a network transfer, but that's pretty involved for what you're doing. However, if you want to do this regularly for your log files (or whatever), you could look into using something like rsyslog to move those files off your container.
So how can i mount the /var/lib/myapp from the container to the /var/lib/myapp in host server
That is the opposite: you can mount an host folder to your container on docker run.
(without removing current container)
I don't think so.
Right now, you can check docker inspect <containername> and see if you see your log in the /var/lib/docker/volumes/... associated to the volume from your container.
Or you can redirect the result of docker logs <containername> to an host file.
For more example, see this gist.
The alternative would be to mount a host directory as the log folder and then access the log files directly on the host.
me#host~$ docker run -d -p 80:80 -v <sites-enabled-dir>:/etc/nginx/sites-enabled -v <certs-dir>:/etc/nginx/certs -v <log-dir>:/var/log/nginx dockerfile/nginx
me#host~$ ls <log-dir>
(again, that apply to a container that you start, not an existing running one)
Related
I am new to docker volumes, and my use case is the next:
I have two different containers running in the same host, and both need to read/write files from it. Is of my understanding that I should use docker volumes, but before I try that, I want to make sure that i can delete files of the host filesystem, from inside the containers (e.g. using a golang app)
Maybe, you should use docker volumes. It can share the directory between the host and containers. For example, you want to read/write the file in /mnt, you can mount the /mnt to container.
docker run -it -v /mnt:/mnt ubuntu:latest touch /mnt/hello.log
now, /mnt/hello.log was created. And you can edit the file /mnt/hello.log in you host filesystem.
Then,
docker run -it -v /mnt:/mnt ubuntu:latest rm /mnt/hello.log
After the command above, the file /mnt/hello.log will be deleted from inside the container.
Actually, you can delete the file in golang, like this:
os.Remove("/mnt/hello.log")
I'm running a container by sending to docker daemon so it can run a sibling container and in that container I try to run another container and mount a volume to access some data, however in the sibling container, the volume is either empty or the file is converted to a folder...
Running the first container:
$ docker run -v /var/run/docker.sock:/var/run/docker.sock -it example /bin/bash
root#3aa35965846a:/home/node/example# ls some_volume/
test.txt
root#3aa35965846a:/home/node/example# cat some_volume/test.txt
hello
// Running the second container
root#3aa35965846a:/home/node/example# docker run -v /home/node/example/some_volume/:/some_volume/ -it node:10 /bin/bash
root#6a84739fbb92:/# ls /some_volume/
* test.txt
root#6a84739fbb92:/# cat /some_volume/test.txt/
cat: /some_volume/test.txt/: Is a directory
The first time I run the second container the volume is empty, if I try to mount a file directly it is converted to a folder, and after that if I try to mount the folder like the example above, there is only the file I tried to mount earlier and it is a folder.
How is this possible ? If i try to mount a volume outside the first container I don't have any problem, how can I fix this ?
The first path in the docker run -v option is always on the host system. For example, if you
docker run -v /etc:/x busybox cat /x/shadow
it will dump out the host's encrypted password file, regardless of whether you ran this command directly from the host or from a container.
There isn't a way to share an arbitrary directory from one container to another. If the launching container knows something about its own directory structure (in particular that some directory was mounted from a specific host path or named volume) then it can replicate that to the other container, but that's not a generic answer. The other behaviors you're seeing are just a consequence of those directories not existing on the host system.
In general I would advise not using Docker for short-lived processes that principally interact with the outside world through the filesystem. Take whatever program you'd run in the other container, install it in your image's Dockerfile, and run it directly without going through Docker.
If you really can't avoid this workflow, the only thing I've found to work reliably is to docker create the container, docker cp files in, docker start it, and docker wait for it to finish. When it's done, docker cp the result out before docker rm it. That's a kind of painstaking workflow but it gets around the problem of the two containers not sharing any filesystem space.
I have a dockerized application that uses the filesystem to store lots of state. The application code is contained in the docker image
I am considering a update strategy which involves sharing the volume between two containers, but making sure that at most one container at a time can write to that filesystem.
The workflow would be:
start container A with /data mounted rw
start container B with /data mounted ro, and a newer version of the application
stop serving requests to container A
for container A, make the /data mount read-only
for container B, make the /data mount read-write
start serving requests to container B
You can re-mount your volume from inside the container, in the rw mode, like that:
mount -o remount,rw /mnt/data
The catch is that mount syscall is not allowed inside the Docker containers by default so that you would have to run it in a privileged mode:
docker run --privileged ...
or enable the SYS_ADMIN capability
SYS_ADMIN Perform a range of system administration operations.
docker run --cap-add=SYS_ADMIN --security-opt apparmor:unconfined
(note that I have had to also add --security-opt apparmor:unconfined, to make this work on Ubuntu).
Also, remounting the rw volume back to ro might be tricky, as some process(es) might have already opened some files inside it for writing , in which case the remount will fail with is busy error message.
But my guess is that you can just restart the container instead (as it would be the one running an old version of the app).
Not exactly what the OP requested, but I've had a similar question where i needed to get data OUT of the running container, but had mounted RW.
Other ways to extract the data would have taken too long.
My approach ? Stash the container as an image and start a new container from that Image with a mount as RW :D
Initial container start:
docker run -p 80:8080 --mount type=bind,source="C:\data-folder-local\",target=/data-folder-container-ro,readonly -d imageName:imageTag
Making an image from the container. You can stop this container before/after if you want.
docker commit -a "mud" -m "Damn, mount should be rw, stashing a snapshot to reuse." CONTAINER_ID_HERE snapshotImageName:snapshotImageTag
where CONTAINER_ID_HERE i got from the output of docker ps (https://docs.docker.com/engine/reference/commandline/ps/)
Start a new container from the image made, but this time mount with write rights!
docker run -p 80:8080 --mount type=bind,source="C:\data-folder-local\",target=/data-folder-container-rw -d snapshotImageName:snapshotImageTag
write out files to the mount folder (on local system) from within your container :D
Hope that helps somebody.
Let's say that I start a docker container with a bind-mounted local folder:
docker run --rm -v /ux1/dmtest:/data -it ubuntu
Then, locally - not inside the container, I bind-mount a directory from another fs into /ux1/dmtest:
mkdir /ux1/dmtest/bm
mount --bind /ux0/bm /ux1/dmtest/bm
Now, from the container, I see /data/bm/ and I can write content to it, but this content will not be visible on the host on /ux0/bm.
Where is this content stored?
And is there any way to mount additional storage into a running docker container (this workaround clearly doesn't work)?
Mounts done after the fact won't be seen by the container due to mount namespaces that Docker uses. The files will be in the /ux1/dmtest directory that was in place before your second bind mount.
If you do want to use a bind mount, put it in place, and then start the docker daemon, and then your container will see it.
I'm learning docker and reading their chapter "Manage data in containers". In the "Mount a host directory as a data volume". They mentioned the following paragraph:
In addition to creating a volume using the -v flag you can also mount a directory from your Docker engine’s host into a container.
$ docker run -d -P --name web -v /src/webapp:/opt/webapp training/webapp python app.py
This command mounts the host directory, /src/webapp, into the container at /opt/webapp. If the path /opt/webapp already exists inside the container’s image, the /src/webapp mount overlays but does not remove the pre-existing content. Once the mount is removed, the content is accessible again. This is consistent with the expected behavior of the mount command.
Experiment 1
Then when I tried to run this command and try to inspect the container, I found that that actually container doesn't even run. Then I use docker logs web and find this error:
can't open file 'app.py': [Errno 2] No such file or directory
I assume that the /src/webapp mount overlays on the /opt/webapp, which there is no content.
Question 1
How can I remove this mount and check if the content is still there as the quote said?
Experiment 2
When I tried to run
$ docker run -d -P --name web2 -v newvolume:/opt/webapp training/webapp python app.py
I found that the container ran correctly. Then I use docker exec -it web2 /bin/bash and find that all of the existing content are still inside the /opt/webapp. I can also add more files inside here. So in this case, it looks like that the volume is not overlay but combined. If I use docker inspect web and check Mounts, then I'll see that the volume is created under /var/lib/docker/volumes/newvolume/_data
Question 2
If I give a name instead of a host-dir absolute path, then the volume will not overlay the container-dir /opt/webapp but connect the two dir together?
An alternative solution is to commit the container (or export it) using docker cli and re-create it without doing the mapping.
Question 1 How can I remove this mount and check if the content is still there as the quote said?
You would create a new container without the volume mount. E.g.
$ docker run -d -P --name web training/webapp python app.py
(Theoretically it's possible to perform some privileged operations to remove the mount on a running container, but inside the container you will not normally have this permission, and it's a good practice to get into the habit of treating containers as ephemeral.)
Question 2 If I give a name instead of a host-dir absolute path, then the volume will not overlay the container-dir /opt/webapp but connect the two dir together?
Almost. What's happening with named volumes is that docker provides an initialization step when the volume is empty and the container is created with that volume mount. The initialization step copies the contents of the image at that directory into the volume, including all files and directories recursively, ownership, and permissions. This is very useful to running containers as a non-root user with a volume directory that the user inside the container needs to be able to write into. After that initialization has happened, future containers with the same named volume will skip the initialization, even if the image content has changed, e.g. if you add new content into the image.