I think that my question is simple but I want to make sure I am taking the right approach. On my host computer, I have a path, e.g. /my/docs, which contains HTML files which get updated automatically.
I have a Docker container with a small web server these html files. I would like to create a named coker volume to called my-docs to point to /my/docs, so that I can start the container as docker run -v my-docs:/public ...
Is this the right approach, and if so, what is the docker create volume command?
The default "local" driver for named volumes places the data inside of the docker directories. The correct way to do what you want is to use the built in host volume, instead of a named volume:
docker run -v /my/docs:/public ...
Alternatively, you can first copy the contents from /my/docs into the named volume and then use that named volume:
docker run --rm \
-v /my/docs:/source -v my-docs:/target \
busybox cp -av /source/. /target/
docker run --rm -it -v my-docs:/public busybox /bin/sh
It's also possible that someone has created a driver for this, or for you to create one yourself. See the volume driver plugin docs for more details.
Related
I have created a docker container image, and working inside the container (setting up filesystem, creating code files, installing dependencies etc).
On my Ubuntu machine, I keep a huge file under a directory so called /dataset/my_data/.
When I work in the container, it is not straightforward to access the above directory. Is there any possibility to access it from inside the container image interactively? If yes, how?
You can mount the directory as a volume:
docker ... --mount type=bind,source=/dataset/my_data,target=/some/directory
For example, to run a container called my-container with a bound volume:
docker run -it --mount \
type=bind,source=/dataset/my_data,target=/target/directory \
my-container
For all available possibilities, see Docker documentation.
I'm writing a cli which will generate a markdown file when finished, and I build a docker image for that cli.
I want to mount the markdown file generated by the container to host machine.
docker -v will create a folder not a file automatically when the path not exist on host.
For example.
~/result.md not exist at first.
docker run -it --rm -v ~/result.md:/usr/src/work_dir/result.md cli:latest generate_markdown
After running, ~/result.md folder is created but not file, and the cli throw an exception because of write to a directory not a file.
To avoid this, I have to create a file at first, and run the docker cli subsequently. It works fine.
Is it possible to avoid create the file at the beginning ?
Try -
$docker volume create myvol
$docker run -it --rm -v myvol:/usr/src/work_dir/ cli:latest generate_markdown
Alternately, you can just
$docker run -it --rm -v myvol:/usr/src/work_dir/ cli:latest generate_markdown
Want an explanation ?
You are using a bind mount; in your case
docker run -it --rm -v ~/result.md:/usr/src/work_dir/result.md cli:latest generate_markdown
The solution to your problem might just be a volume mount.
For more info refer - https://docs.docker.com/storage/volumes/
First create a docker volume by-
$docker volume create myvol
. You can give any name instead of myvol.
This docker volume will be created, you can check if the volume was created successfully by-
$docker volume ls
This will give a list of your all your volumes, your newly created volume should be listed.
ak#ubuntu:~$ docker volume create myvol
myvol
ak#ubuntu:~$ docker volume ls
DRIVER VOLUME NAME
local myvol
Docker Volumes are stored in a separate area on the host file system and are completely managed by docker as opposed to bind mounts.Docker volumes store state outside of containers, so your data survives when you replace the container to update your app.
Docker volumes also get automatically created in case you specify a name instead of a directory path. In the following example a volume by the name myvol2 will be automatically created -
$docker run -it -v myvol2:/home/myfiles imagename:tag
Docker volumes are usually created in /var/lib/docker/volumes
in linux and in C:\ProgramData\docker\volumes in Windows.
Now here's the useful part. Any data/file/directories that already exist in the specified container directory are automatically copied or 'mounted' onto the docker volume. Therefore if the '''/usr/src/work_dir/''' directory mentioned in the above example contains any files (like a markup file in your case), they are copied onto the volume automatically.
Hope this helps.
The volume mount will assume a directory name passed in, not a file. You can either mount the volume ~ or have a directory created and mount that volume.
mkdir ~/markdown
docker run -it --rm -v ~/markdown/:/usr/src/work_dir/ cli:latest generate_markdown
You should avoid mounting files all together accordijg to best practices. That being said it is not possible to change the x fault behaviour that you describe.
The solution to you problem (to not create the file each time before running the container) is to mount the directories, omitting the file names.
docker run -it --rm -v ~/:/usr/src/work_dir/ cli:latest generate_markdown
I'm trying to go deeper in my understanding of Docker's volume, and I'm having an hard time to figure out the differences / use-case of:
The docker volume create command
The docker run -v /path:/host_path
The VOLUME entry in the Dockerfile file
I particularly don't understand what happens if you combine the VOLUME entry with the -v flag.
A volume is a persistent data stored in /var/lib/docker/volumes/...
You can either declare it in a Dockerfile, which means each time a container is started from the image, the volume is created (empty), even if you don't have any -v option.
You can declare it on runtime docker run -v [host-dir:]container-dir.
combining the two (VOLUME + docker run -v) means that you can mount the content of a host folder into your volume persisted by the container in /var/lib/docker/volumes/...
docker volume create creates a volume without having to define a Dockerfile and build an image and run a container. It is used to quickly allow other containers to mount said volume.
If you had persisted some content in a volume, but since then deleted the container (which by default does not deleted its associated volume, unless you are using docker rm -v), you can re-attach said volume to a new container (declaring the same volume).
See "Docker - How to access a volume not attached to a container?".
With docker volume create, this is easy to reattached a named volume to a container.
docker volume create --name aname
docker run -v aname:/apath --name acontainer
...
# modify data in /apath
...
docker rm acontainer
# let's mount aname volume again
docker run -v aname:/apath --name acontainer
ls /apath
# you find your data back!
VOLUME instruction becomes interesting when you combine it with volumes-from runtime parameter.
Given the following Dockerfile:
FROM busybox
VOLUME /myvolume
Build an image with:
docker build -t my-busybox .
And spin up a container with:
docker run --rm -it --name my-busybox-1 my-busybox
The first thing to notice is you will have a folder in this image named myvolume. But it is not particularly interesting since when we exit the container the volume will be removed as well.
Create an empty file in this folder, so run the following in the container:
cd myvolume
touch hello.txt
Now spin up a new container, but share the same volume with my-busybox-1:
docker run --rm -it --volumes-from my-busybox-1 --name my-busybox-2 my-busybox
You will see that my-busybox-2 contains the file hello.txt in myvolume folder.
Once you exit both containers, the volume will be removed as well.
#radium226
Specifying VOLUME in Dockerfile makes sure the folder is to be treated as a volume(i.e., outside container) at runtime, as opposed to be a regular directory inside the container. Note the performance and accessibility implications.
If having forgot to specify "-v" in "docker run" command line, the above is still true. It's just the volume name becomes anonymous. But there are still ways to access or recover data from such anonymous volumes.
Using MYSQL from docker hub:
Running the below command as an example:
$ docker run --name some-mysql -v /my/own/datadir:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag
The -v /my/own/datadir:/var/lib/mysql part of the command mounts the /my/own/datadir directory from the underlying host system as /var/lib/mysql inside the container, where MySQL by default will write its data files.
Therefore, a directory that persists when the container is killed is mounted is available that also provided higher performance for some operations like databases actions.
I was just going through this tutorial on Youtube, trying to understand the use of the -v command.
Why is the author using the -v command? He uses the command, like so:
docker run -v /var/lib/mysql --name=my_datastore -d busybox echo "my datastore"
Now I understand the above command to an extent:
--name=my_datastore gives the container a specific name.
-d busybox starts a container in detached mode, based on the busybox image.
After the command is executed the below line is echoed to the console.
my datastore
Now, the part I don't understand is the following:
-v /var/lib/mysql
Why is the -v command used here, and why the path /var/lib/mysql specified?
I am having difficulty understanding why the above line is used and in what context. Can anybody explain?
The -v (or --volume) argument to docker run is for creating storage space inside a container that is separate from the rest of the container filesystem. There are two forms of the command.
When given a single argument, like -v /var/lib/mysql, this allocates space from Docker and mounts it at the given location. This is primarily a way of allocating storage from Docker that is distinct from your service container. For example, you may want to run a newer version of a database application, which involves tearing down your existing MySQL container and starting a new one. You want your data to survive this process, so you store it in a volume that can be accessed by your database container.
When given two arguments (host_path:container_path), like -v /data/mysql:/var/lib/mysql, this mounts the specified directory on the host inside the container at the specified path (and, to be accurate, this can also be used to expose host files inside the container; for example -v /etc/localtime:/etc/localtime would make /etc/localtime on the host available as /etc/localtime inside the container). This is a way of either feeding information into your container, or providing a way for your container to make files accessible to the host.
If a container has volumes available, either through the use of the -v command line argument or via the VOLUME directive in a Dockerfile, these volumes can be accessed from another container using the --volumes-from option:
docker run --volumes-from my_datastore ...
This will make any volumes defined in the source container available in the container you're starting with --volumes-from.
This is discussed in more detail in the Docker Volumes documentation.
I have a Docker container which is running some code and creating some HTML reports. I want these reports to be published into a specific directory on the host machine, i.e. at /usr/share/nginx/reports
The way I have gone about doing this is to mount this host directory as a data volume, i.e. docker run -v /usr/share/nginx/reports --name my-container com.containers/my-container
However, when I ssh into the host machine, and check the contents of the directory /usr/share/nginx/reports, I don't see any of the report data there.
Am I doing something wrong?
The host machine is an Ubuntu server, and the Docker container is also Ubuntu, no boot2docker weirdness going on here.
From "Managing data in containers", mounting a host folder to a container would be:
docker run -v /Users/<path>:/<container path>
(see "Use volume")
Using only -v /usr/share/nginx/reports would declare the internal container path /usr/share/nginx/reports as a volume, but would have nothing to do with the host folder.
This is one of the type of mounts available:
The answer to this question is problematic because it varies depending on your operating system and your full requirements. The answer by VonC makes some assumptions that should be addressed and is therefore only correct in some contexts. Other answers on this topic generally ignore the fact that some people are running linux, others windows, and still others are on OSX or other weird OS's.
As VonC mentioned in his answer, in a lot of cases it is possible to bind-mount a host directory straight into the container, using a -v host-path:container-path argument to the docker command (you can also use --volume for added readability or --mount for rocket-science).
One of the biggest problems (in 2020) is the use of the Windows Subsystem for Linux (WSL), where bind-mounting a host volume is fraught with error and may or may not work as expected depending on whether the path mounted is in the linux filesystem or the windows filesystem. VonC's answer was written before WSL became a big problem, but it still makes assumptions about the local filesystem being real rather than mounted into a virtual-machine of some kind.
I have found that a lot of engineers prefer to bypass this unnecessary confusion through the use of docker volumes. A docker volume can be created with the command:
docker volume create <name>
Listed with
docker volume ls
and removed with
docker volume rm <name>
You can mount this by specifying the name of the volume on the left-hand-side of the --volume argument. If your volume was called, for example, 'logs', you could use something like --volume logs:/usr/share/nginx/reports to bind it to the log dir you're interested in. You can view the contents of the directory with something like this:
docker run -it --rm --volume logs:/logs alpine ls -AlF /logs/
This should list the files in that directory. If you have a file called 'nginx.log' for example, you could view it like this:
docker run -it --rm --volume logs:/logs alpine less /logs/nginx.log
And the contents would be paged to your terminal.
You can bind this volume to multiple containers simultaneously if needed. This is useful if, for example, you're writing to your logs with one container, and paging them to a console with another.
If you want to copy the example log file from above into a tmp directory on your local filesystem you can achieve that with:
docker run -it --rm --volume logs:/logs --volume /tmp:/local_tmp alpine cp /logs/nginx.log /local_tmp/
I am using Docker toolbox on windows. I am Working on a Spring Boot Application using Docker. My application writes logs to
users/path/service.log
So when i started my application from host terminal the Log file was successfully updated.
But the same when i did on docker no file was created and neither updated.
So i changed my log file location to match with the Container's Directories
var/log/service.log
I started my container again and my file was updated again.
You can choose any location as long as it matches with the container Directory. Just bash into the container and see what suits you.
Next step is to copy log files from container to host.
So in order to copy those logs to your host. You can use one of two ways i know of-
1- use Volumes in docker
2- use following Docker command to copy file from docker container to host-:
docker cp <containerId>:/file/path/within/container /host/path/target
First, you need to create a directory where you want to share the data
mkdir -p /abc/def/
Now, you need to create a docker volume using the below command. As we see here, we are specifying device as '/abc/def/'
docker volume create --driver local \
--opt type=none \
--opt device=/abc/def/ \
--opt o=bind \
spark-volume
Now, start your container with below command..
docker run -d \
--mount type=volume,dst=/abc/def/,volume-driver=local,volume-opt=type=none,volume-opt=o=bind,volume-opt=device=/opt/spark/ \
--network host \
img:tag
Now, docker container will use /abc/def/ in local Filesystem as its storage and you will have all contents of /abc/def/ in docker container available in Local Filesystem
In your application, if you set a working directory for your php code (report path), the path must be the one on the container. Then docker will copie automaticly copy to your host directory. It wasn't docker mis-configuration, but my application that was writing to the wrong place. Weird at first, but did work in my case.