I have 2 docker containers which I created from 2 Dockerfiles.
docker run container1 # It updates a txt (update.txt) file every minutes and store it in the same container
docker run container2 --link container1 # A web server which in intended to read the updated file in container1
Now I want to access the file update.txt in container2 but I can't do that. I don't want to just copy the file since it will become static but I want to read the dynamically updated file to read the latest updates. Can anyone suggest a way out?
Use named volume to store update.txt in that volume on host.
Mount this volume in both containers.
All changes that container 1 writes then will be accessible in container 2.
Firstly, create a docker volume by using the below command
$ docker volume create --name sharedVolume
sharedVolume
And then start the first container by mounting the above-created volume and write data in that location where volume will be mounted.
$ docker run -it -v sharedVolume:/dataToWrite ubuntu
root#1021d9260d7b:/# echo "DATA Written" >> /dataToWrite/Example.txt
root#1021d9260d7b:/# cat /dataToWrite/Example.txt
DATA Written
Now, start the second container and mount the same volume you created above and check whether the same file present in the second container or not
$ docker run -it -v sharedVolume:/dataToWrite alpine
/ # cat /dataToWrite/Example.txt
DATA Written
As you can see above, the first container is ubuntu and the second container is alpine. Contents which are written in the first container is present in second container.
Related
I'm running a container by sending to docker daemon so it can run a sibling container and in that container I try to run another container and mount a volume to access some data, however in the sibling container, the volume is either empty or the file is converted to a folder...
Running the first container:
$ docker run -v /var/run/docker.sock:/var/run/docker.sock -it example /bin/bash
root#3aa35965846a:/home/node/example# ls some_volume/
test.txt
root#3aa35965846a:/home/node/example# cat some_volume/test.txt
hello
// Running the second container
root#3aa35965846a:/home/node/example# docker run -v /home/node/example/some_volume/:/some_volume/ -it node:10 /bin/bash
root#6a84739fbb92:/# ls /some_volume/
* test.txt
root#6a84739fbb92:/# cat /some_volume/test.txt/
cat: /some_volume/test.txt/: Is a directory
The first time I run the second container the volume is empty, if I try to mount a file directly it is converted to a folder, and after that if I try to mount the folder like the example above, there is only the file I tried to mount earlier and it is a folder.
How is this possible ? If i try to mount a volume outside the first container I don't have any problem, how can I fix this ?
The first path in the docker run -v option is always on the host system. For example, if you
docker run -v /etc:/x busybox cat /x/shadow
it will dump out the host's encrypted password file, regardless of whether you ran this command directly from the host or from a container.
There isn't a way to share an arbitrary directory from one container to another. If the launching container knows something about its own directory structure (in particular that some directory was mounted from a specific host path or named volume) then it can replicate that to the other container, but that's not a generic answer. The other behaviors you're seeing are just a consequence of those directories not existing on the host system.
In general I would advise not using Docker for short-lived processes that principally interact with the outside world through the filesystem. Take whatever program you'd run in the other container, install it in your image's Dockerfile, and run it directly without going through Docker.
If you really can't avoid this workflow, the only thing I've found to work reliably is to docker create the container, docker cp files in, docker start it, and docker wait for it to finish. When it's done, docker cp the result out before docker rm it. That's a kind of painstaking workflow but it gets around the problem of the two containers not sharing any filesystem space.
I'm writing a cli which will generate a markdown file when finished, and I build a docker image for that cli.
I want to mount the markdown file generated by the container to host machine.
docker -v will create a folder not a file automatically when the path not exist on host.
For example.
~/result.md not exist at first.
docker run -it --rm -v ~/result.md:/usr/src/work_dir/result.md cli:latest generate_markdown
After running, ~/result.md folder is created but not file, and the cli throw an exception because of write to a directory not a file.
To avoid this, I have to create a file at first, and run the docker cli subsequently. It works fine.
Is it possible to avoid create the file at the beginning ?
Try -
$docker volume create myvol
$docker run -it --rm -v myvol:/usr/src/work_dir/ cli:latest generate_markdown
Alternately, you can just
$docker run -it --rm -v myvol:/usr/src/work_dir/ cli:latest generate_markdown
Want an explanation ?
You are using a bind mount; in your case
docker run -it --rm -v ~/result.md:/usr/src/work_dir/result.md cli:latest generate_markdown
The solution to your problem might just be a volume mount.
For more info refer - https://docs.docker.com/storage/volumes/
First create a docker volume by-
$docker volume create myvol
. You can give any name instead of myvol.
This docker volume will be created, you can check if the volume was created successfully by-
$docker volume ls
This will give a list of your all your volumes, your newly created volume should be listed.
ak#ubuntu:~$ docker volume create myvol
myvol
ak#ubuntu:~$ docker volume ls
DRIVER VOLUME NAME
local myvol
Docker Volumes are stored in a separate area on the host file system and are completely managed by docker as opposed to bind mounts.Docker volumes store state outside of containers, so your data survives when you replace the container to update your app.
Docker volumes also get automatically created in case you specify a name instead of a directory path. In the following example a volume by the name myvol2 will be automatically created -
$docker run -it -v myvol2:/home/myfiles imagename:tag
Docker volumes are usually created in /var/lib/docker/volumes
in linux and in C:\ProgramData\docker\volumes in Windows.
Now here's the useful part. Any data/file/directories that already exist in the specified container directory are automatically copied or 'mounted' onto the docker volume. Therefore if the '''/usr/src/work_dir/''' directory mentioned in the above example contains any files (like a markup file in your case), they are copied onto the volume automatically.
Hope this helps.
The volume mount will assume a directory name passed in, not a file. You can either mount the volume ~ or have a directory created and mount that volume.
mkdir ~/markdown
docker run -it --rm -v ~/markdown/:/usr/src/work_dir/ cli:latest generate_markdown
You should avoid mounting files all together accordijg to best practices. That being said it is not possible to change the x fault behaviour that you describe.
The solution to you problem (to not create the file each time before running the container) is to mount the directories, omitting the file names.
docker run -it --rm -v ~/:/usr/src/work_dir/ cli:latest generate_markdown
I did the following and lost all the changed data in my Docker container.
docker build -t <name:tag> .
docker run *-p 8080:80* --name <container_name> <name:tag>
docker exec (import and process some files, launch a server to host them)
Then I wanted to run it on a different port. docker stop & docker run does not work. Instead I did
docker stop
docker rm <container_name>
docker run (same parameters as before)
After the restart I saw the changes that happened in the container at 1-3 had disappeared, and had to re-run the import.
How do I do this correctly next time?
what you have to do is build the image from the container you just stopped after making changes. Because your old command still using the old image which doesn't have new changes(you have made changes in container which you just stopped not in image )
docker commit --help
Usage: docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]
Create a new image from a container's changes
docker commit -a me new_nginx myrepo/nginx:latest
then you can start container with the new image you just built
but if you dont want create image with the changes you made(like you dont want to put config containing password in the image) you can use volume mount
docker run -d -P --name web -v /src/webapp:/webapp training/webapp python app.py
This command mounts the host directory, /src/webapp, into the container at /webapp. If the path /webapp already exists inside the container’s image, the /src/webapp mount overlays but does not remove the pre-existing content. Once the mount is removed, the content is accessible again. This is consistent with the expected behavior of the mount command.
Manage data in containers
Every time you do a docker run it will spin up a fresh container based on your image. And once a container is started, there are very few things that docker allows you to change with the docker update. So instead, you should preserve your data in an external volume that needs to persist between instances of a container. E.g.
docker run -p 8080:80 -v app-data:/data --name <container_name> <name:tag>
The volume name (app-data) and mount point in the container (/data) can be changed for your own requirements. Then when you destroy and restart a new container, you can mount the same volume in the new container.
I'm learning docker and reading their chapter "Manage data in containers". In the "Mount a host directory as a data volume". They mentioned the following paragraph:
In addition to creating a volume using the -v flag you can also mount a directory from your Docker engine’s host into a container.
$ docker run -d -P --name web -v /src/webapp:/opt/webapp training/webapp python app.py
This command mounts the host directory, /src/webapp, into the container at /opt/webapp. If the path /opt/webapp already exists inside the container’s image, the /src/webapp mount overlays but does not remove the pre-existing content. Once the mount is removed, the content is accessible again. This is consistent with the expected behavior of the mount command.
Experiment 1
Then when I tried to run this command and try to inspect the container, I found that that actually container doesn't even run. Then I use docker logs web and find this error:
can't open file 'app.py': [Errno 2] No such file or directory
I assume that the /src/webapp mount overlays on the /opt/webapp, which there is no content.
Question 1
How can I remove this mount and check if the content is still there as the quote said?
Experiment 2
When I tried to run
$ docker run -d -P --name web2 -v newvolume:/opt/webapp training/webapp python app.py
I found that the container ran correctly. Then I use docker exec -it web2 /bin/bash and find that all of the existing content are still inside the /opt/webapp. I can also add more files inside here. So in this case, it looks like that the volume is not overlay but combined. If I use docker inspect web and check Mounts, then I'll see that the volume is created under /var/lib/docker/volumes/newvolume/_data
Question 2
If I give a name instead of a host-dir absolute path, then the volume will not overlay the container-dir /opt/webapp but connect the two dir together?
An alternative solution is to commit the container (or export it) using docker cli and re-create it without doing the mapping.
Question 1 How can I remove this mount and check if the content is still there as the quote said?
You would create a new container without the volume mount. E.g.
$ docker run -d -P --name web training/webapp python app.py
(Theoretically it's possible to perform some privileged operations to remove the mount on a running container, but inside the container you will not normally have this permission, and it's a good practice to get into the habit of treating containers as ephemeral.)
Question 2 If I give a name instead of a host-dir absolute path, then the volume will not overlay the container-dir /opt/webapp but connect the two dir together?
Almost. What's happening with named volumes is that docker provides an initialization step when the volume is empty and the container is created with that volume mount. The initialization step copies the contents of the image at that directory into the volume, including all files and directories recursively, ownership, and permissions. This is very useful to running containers as a non-root user with a volume directory that the user inside the container needs to be able to write into. After that initialization has happened, future containers with the same named volume will skip the initialization, even if the image content has changed, e.g. if you add new content into the image.
I'm trying to go deeper in my understanding of Docker's volume, and I'm having an hard time to figure out the differences / use-case of:
The docker volume create command
The docker run -v /path:/host_path
The VOLUME entry in the Dockerfile file
I particularly don't understand what happens if you combine the VOLUME entry with the -v flag.
A volume is a persistent data stored in /var/lib/docker/volumes/...
You can either declare it in a Dockerfile, which means each time a container is started from the image, the volume is created (empty), even if you don't have any -v option.
You can declare it on runtime docker run -v [host-dir:]container-dir.
combining the two (VOLUME + docker run -v) means that you can mount the content of a host folder into your volume persisted by the container in /var/lib/docker/volumes/...
docker volume create creates a volume without having to define a Dockerfile and build an image and run a container. It is used to quickly allow other containers to mount said volume.
If you had persisted some content in a volume, but since then deleted the container (which by default does not deleted its associated volume, unless you are using docker rm -v), you can re-attach said volume to a new container (declaring the same volume).
See "Docker - How to access a volume not attached to a container?".
With docker volume create, this is easy to reattached a named volume to a container.
docker volume create --name aname
docker run -v aname:/apath --name acontainer
...
# modify data in /apath
...
docker rm acontainer
# let's mount aname volume again
docker run -v aname:/apath --name acontainer
ls /apath
# you find your data back!
VOLUME instruction becomes interesting when you combine it with volumes-from runtime parameter.
Given the following Dockerfile:
FROM busybox
VOLUME /myvolume
Build an image with:
docker build -t my-busybox .
And spin up a container with:
docker run --rm -it --name my-busybox-1 my-busybox
The first thing to notice is you will have a folder in this image named myvolume. But it is not particularly interesting since when we exit the container the volume will be removed as well.
Create an empty file in this folder, so run the following in the container:
cd myvolume
touch hello.txt
Now spin up a new container, but share the same volume with my-busybox-1:
docker run --rm -it --volumes-from my-busybox-1 --name my-busybox-2 my-busybox
You will see that my-busybox-2 contains the file hello.txt in myvolume folder.
Once you exit both containers, the volume will be removed as well.
#radium226
Specifying VOLUME in Dockerfile makes sure the folder is to be treated as a volume(i.e., outside container) at runtime, as opposed to be a regular directory inside the container. Note the performance and accessibility implications.
If having forgot to specify "-v" in "docker run" command line, the above is still true. It's just the volume name becomes anonymous. But there are still ways to access or recover data from such anonymous volumes.
Using MYSQL from docker hub:
Running the below command as an example:
$ docker run --name some-mysql -v /my/own/datadir:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag
The -v /my/own/datadir:/var/lib/mysql part of the command mounts the /my/own/datadir directory from the underlying host system as /var/lib/mysql inside the container, where MySQL by default will write its data files.
Therefore, a directory that persists when the container is killed is mounted is available that also provided higher performance for some operations like databases actions.