I have a question regarding the whole data volume process in Docker. Basically here are two Dockerfiles and their respective run commands:
Dockerfile 1 -
# Transmission over Debian
#
# Version 2.92
FROM debian:testing
RUN apt-get update \
&& apt-get -y install nano \
&& apt-get -y install transmission-daemon transmission-common transmission-cli \
&& mkdir -p /transmission/config /transmission/watch /transmission/download
ENTRYPOINT ["transmission-daemon", "--foreground"]
CMD ["--config-dir", "/transmission/config", "--watch-dir", "/transmission/watch", "--download-dir", "/transmission/download", "--allowed", "*", "--no-blocklist", "--no-auth", "--no-dht", "--no-lpd", "--encryption-preferred"]
Command 1 -
docker run --name transmission -d -p 9091:9091 -v C:\path\to\config:/transmission/config -v C:\path\to\watch:/transmission/watch -v C:\path\to\download:/transmission/download transmission
Dockerfile 2 -
# Nginx over Debian
#
# Version 1.10.3
FROM debian:testing
RUN apt-get update \
&& apt-get -y install nano \
&& apt-get -y install nginx
EXPOSE 80 443
CMD ["nginx", "-g", "daemon off;"]
Command 2 -
docker run --name nginx -d -p 80:80 -v C:\path\to\config:/etc/nginx -v C:\path\to\html:/var/www/html nginx
So, the weird thing is that the first dockerfile and command works as intended. Where the docker daemon mounts a directory from the container to the host. So, I am able to edit the configuration files as I please and they will be persisted to the container on a restart.
However, as for the second dockerfile and command it doesn't seem to be working. I know if you go to the Docker Volume documentation it says that volume mounts are only intended to go one-way, from host-to-container, but how come the Transmission container works as intended, while the Nginx container doesn't?
P:S - I'm running Microsoft Windows 10 Pro Build 14393 as my host and Version 17.03.0-ce-win1 (10300) Channel: beta as my Docker version.
Edit - Just to clarify. I'm trying to get the files from inside the Nginx container to the host. The first container (Transmission) works in that regard, by using a data volume. However, for the second container (Nginx), it doesn't want to copy the files in the mounted directory from inside the container to the host. Everything else is working though, it does successfully start.
The host volume will not copy data like a named volume will. However, you can create a named volume that performs a bind mount, which will then have the data initialization properties of any other named volume. The only prerequisite of a bind mount over a host volume is that the directory must exist in advance, docker will not create it for you like it does with a host volume. Here are three different examples of how to create a bind mount volume:
# create the volume in advance
$ docker volume create --driver local \
--opt type=none \
--opt device=/home/user/test \
--opt o=bind \
test_vol
# create on the fly with --mount
$ docker run -it --rm \
--mount type=volume,dst=/container/path,volume-driver=local,volume-opt=type=none,volume-opt=o=bind,volume-opt=device=/home/user/test \
foo
# inside a docker-compose file
...
volumes:
bind-test:
driver: local
driver_opts:
type: none
o: bind
device: /home/user/test
...
So in your example with a docker run command, you can use the mount syntax:
docker run --name nginx -d -p 80:80 \
--mount type=volume,dst=/etc/nginx,volume-driver=local,volume-opt=type=none,volume-opt=o=bind,volume-opt=device=/c/path/to/config \
--mount type=volume,dst=/var/www/html,volume-driver=local,volume-opt=type=none,volume-opt=o=bind,volume-opt=device=/c/path/to/html \
nginx
The only part that may need adjusting is the windows path names inside the Linux VM that docker runs in HyperV.
Host volumes don't copy data from the container > host. Host volumes mount over the top of what's in the container/image, so they effectively replace what's in the container with what's on the host.
A standard or "named" volume will copy the existing data from the container image into a new volume. These volumes are created by launching a container with the VOLUME command in it's Dockerfile or by the docker command
docker run -v myvolume:/var/whatever myimage
By default this is data stored in a "local" volume, "local" being on the Docker host. In your case that is on the VM running Docker rather than your Windows host so might not be easily accessible to you.
You could be mistaking transmission auto generating files in a blank directory for a copy?
If you really need the keep the VM Host > container mappings then you might have to copy the data manually:
docker create --name nginxcopy nginx
docker cp nginxcopy:/etc/nginx C:\path\to\config
docker cp nginxcopy:/var/www/html C:\path\to\html
docker rm nginxcopy
And then you can map the populated host directories into the container and they will have the default data the image came with.
Related
I am facing an issue where after runnig the container and using bind mount to mount the directory on host to container I am not able to see new files created in host machine inside container.Below is my project structure.
The python code creates a file inside the container which should be available inside the host machine too however this does happen when I start the container with below command. However updates to python code and html is available inside the container.
sudo docker container run -p 5000:5000 --name flaskapp --volume feedback1:/app/feedback/ --volume /home/deepak/PycharmProjects/NewDockerProject/sampleapp:/app flask_image
However after starting the container using below command, everything seems to work fine. I can see all the files from container to host and vice versa(new created , edited).I git this command from docker in the month of lunches book.
sudo docker container run --mount type=bind,source=/home/deepak/PycharmProjects/NewDockerProject/sampleapp,target=/app -p 5000:5000 --name flaskapp
Below is the content of my dockerfile
FROM python:3.8-alpine
WORKDIR /app
COPY ./requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python","main.py"]
Could someone please help me in figuring out the difference between the two commands ? I am using ubuntu. Thank you
In my case i got working volumes using following docker run args (but i am running without --mount type=bind):
docker run -it ... -v mysql_data:/var/lib/mysql -v storage:/usr/shared/app_storage
where:
mysql_data is a volume name
/var/lib/mysql path inside container machine
you could list volumes as:
docker volume ls
and inspect them to see where it points on your system (usually /var/lib/docker/volumes/{volume_nanme}/_data):
docker volume inspect mysql_data
to create volume use following command:
docker volume create {volume_name}
I have a docker image which I run it as below in my local mac:
1) creating filesystem -->
mkdir data &
mkdir license
2)Docker Run : --->
docker run \
--rm \
-u id -u:id -g \
-p 11225:11225 \
-p 9090:9090 \
-v pwd/data:/data \
-v pwd/license:/license \
Now, I would like to run this in Kubernetes. While I expose the ports as service , How can I create the folders (data and license) and then expose that as bind volume in kubernetes cluster ? sample yml or kubectl command helps
You can find explanation of kubernetes Volumes in https://kubernetes.io/docs/concepts/storage/volumes/ which is what you really need to look into and decide which volume type is best for your particular case/environment.
https://kubernetes.io/docs/concepts/storage/persistent-volumes/ is a good reading further to expand your understanding of persistent storage management in kubernetes with PVC/PV concepts.
Does anyone know how to mount nfs share inside docker container with centos base image? I've tried this command:
mount server:/dir /mount/point
and got the next error:
mount.nfs: rpc.statd is not running but is required for remote locking.
mount.nfs: Either use '-o nolock' to keep locks local, or start statd.
mount.nfs: an incorrect mount option was specified
when I try to use it with -o nolock option, the error is:
mount.nfs: Operation not permitted
Starting from docker 17.06, you can mount NFS shares to the container directly when you run it, without the need of extra capabilities
export NFS_VOL_NAME=mynfs
export NFS_LOCAL_MNT=/mnt/mynfs
export NFS_SERVER=my.nfs.server.com
export NFS_SHARE=/my/server/path
export NFS_OPTS=vers=4,soft
docker run --mount \
"src=$NFS_VOL_NAME,dst=$NFS_LOCAL_MNT,volume-opt=device=:$NFS_SHARE,\"volume-opt=o=addr=$NFS_SERVER,$NFS_OPTS\",type=volume,volume-driver=local,volume-opt=type=nfs" \
busybox ls $NFS_LOCAL_MNT
Alternatively, you can create the volume before the container:
docker volume create \
--driver local \
--opt type=nfs \
--opt o=addr=$NFS_SERVER,$NFS_OPTS \
--opt device=:$NFS_SHARE \
$NFS_VOL_NAME
docker run --rm -v $NFS_VOL_NAME:$NFS_LOCAL_MNT busybox ls $NFS_LOCAL_MNT
Got the hint from
https://github.com/moby/moby/issues/28809
official docs from docker: https://docs.docker.com/storage/volumes/#create-a-service-which-creates-an-nfs-volume
For using mount, you'll need the CAP_SYS_ADMIN capability, which is dropped by Docker when creating the container.
There are several solutions for this:
Start the container with the --cap-add sys_admin flag. This causes Docker to retain the CAP_SYS_ADMIN capability, which should allow you to mount a NFS share from within the container. This might be a security issue; do not do this in untrusted containers. [A previous version of this answer suggested using the --privileged=true to retain all capabilities, thanks to #earcam for the suggestion to use --cap-add instead].
Mount the NFS share on the host and pass it into the container as a host volume:
you#host > mount server:/dir /path/to/mount/point
you#host > docker run -v /path/to/mount/point:/path/to/mount/point
Use a Docker volume plugin (like the Netshare plugin) to directly mount the NFS share as a container volume:
you#host > docker run \
--volume-driver=nfs \
-v server/dir:/path/to/mount/point \
centos
For the second option listed in the accepted answer, I'm not sure if you have actually tried to use the "docker run -v" command to pass a NFS share on the host to docker container as a volume.
I have recently tried to do so, below is the info for the nfs share on host:
nfs-server:/path_to_mount on /path_dest type nfs
and then:
docker run -it -v /path_dest:/path_in_docker docker_name bash
But the docker daemon always reports below error:
docker: Error response from daemon: stat /path_dest: permission denied.
After many searches, I found that the error actually comes from docker daemon, which is running as "root". When docker runs a container with volume to mount, it will request docker daemon to mount it. The problem is, NFS server will handle "root" differently. By default, NFS server will map the "root" to "nobody", causing the error message: reference
I mount the nfs on docker container, thanks for #helmbert .
Run a docker container with the --privileged=true flag.
$ docker run -it --privileged=true centos:7 bash
[root#f7915ae635aa /]# yum install -y nfs-utils
Install the nfs tool package and mount nfs on CentOS.
[root#f7915ae635aa /]# yum install -y nfs-utils
[root#f7915ae635aa /]# mount -t nfs example.tw:/target/ /srv -o nolock
Show mount of the nfs server.
[root#f7915ae635aa /]# showmount example.tw
Hosts on example.tw:
10.10.10.1
10.10.10.2
By adding --cap-add sys_admin flag to client container wasn't enough for me. I was getting error:
mount.nfs: mount(2): Permission denied
mount.nfs: access denied by server while mounting 1.2.3.4:/exports
After hours of research I've found that it looks like full privilege --privileged is needed to mount correctly inside docker container ..
Also don't forget to install necessary nfs client packages inside your docker container. On debian based containers:
apt-get install -y nfs-common
I am currently using Boot2Docker on Windows. Is it possible to mount root to host?
Say that I'm using an Ubuntu image and I would like to mount / to the host. How can I do so?
I've been looking around and trying:
docker run -v /c/Users/ubuntu:/ --name ubuntu -dt ubuntu
But I ended up with an error:
docker: Error response from daemon: Invalid bind mount spec "/c/Users/ubuntu:/": volumeslash: Invalid specification: destination can't be '/' in '/c/Users/Leon/ubuntu:/'.
If I understand correctly, you are trying to mount root inside a container as a volume? If that is the case, rather create a new directory inside and expose that one.
For example, dockerfile:
RUN mkdir /something
VOLUME /something
As the Docker documentation says, the container directory must always be an absolute path such as /src/docs. The host-dir can either be an absolute path or a name value.
For more information read this: https://docs.docker.com/engine/userguide/containers/dockervolumes/#mount-a-host-directory-as-a-data-volume and part "Mount a host directory as a data volume" should give you better understanding.
It's the problem with how you are specifying the path. See the example of mounting a local volume to be used by a container for MongoDB:
docker run --name *container-name* -v **/Users/SKausha3/mongo/imageservicedb/data**:/*data* -v **/Users/SKausha3/mongo/imageservicedb/backup**:/*backup*
c:/Users/SKausha3/mongo/imageservicedb/data is my local folder, but you have to remove 'c:' from the path.
Since you cant mount "/" one option is to add a "WORKDIR" to your Dockerfile, that way all subsequent commands will be relative to that dir and you wont have to modify anything!
FROM python:latest
WORKDIR /myapp
COPY appfile.py appfile.py
In your docker image, the "appfile.py" file will be in the /myapp/appfily.py location.
You cannot specify the '/' root directory of container but you can mount all the folders in to docker volumes present in root directory.....
create volumes by running these command one by one or you can create bash script
docker volume create var
docker volume create usr
docker volume create tmp
docker volume create sys
docker volume create srv
docker volume create sbin
docker volume create run
docker volume create root
docker volume create proc
docker volume create opt
docker volume create mnt
docker volume create media
docker volume create libx32
docker volume create lib64
docker volume create lib32
docker volume create lib
docker volume create home
docker volume create etc
docker volume create dev
docker volume create boot
docker volume create bin
Then run this command
docker run -it -d \
--name=ubuntu-container \
--mount source=var,destination=/var \
--mount source=usr,destination=/usr \
--mount source=tmp,destination=/tmp \
--mount source=sys,destination=/sys \
--mount source=srv,destination=/srv \
--mount source=sbin,destination=/sbin \
--mount source=run,destination=/run \
--mount source=root,destination=/root \
--mount source=opt,destination=/opt \
--mount source=mnt,destination=/mnt \
--mount source=media,destination=/media \
--mount source=libx32,destination=/libx32 \
--mount source=lib64,destination=/lib64 \
--mount source=lib32,destination=/lib32 \
--mount source=lib,destination=/lib \
--mount source=home,destination=/home \
--mount source=etc,destination=/etc \
--mount source=boot,destination=/boot \
--mount source=bin,destination=/bin \
ubuntu:latest
I have a Docker container that I've created simply by installing Docker on Ubuntu and doing:
sudo docker run -i -t ubuntu /bin/bash
I immediately started installing Java and some other tools, spent some time with it, and stopped the container by
exit
Then I wanted to add a volume and realised that this is not as straightforward as I thought it would be. If I use sudo docker -v /somedir run ... then I end up with a fresh new container, so I'd have to install Java and do what I've already done before just to arrive at a container with a mounted volume.
All the documentation about mounting a folder from the host seems to imply that mounting a volume is something that can be done when creating a container. So the only option I have to avoid reconfiguring a new container from scratch is to commit the existing container to a repository and use that as the basis of a new one whilst mounting the volume.
Is this indeed the only way to add a volume to an existing container?
You can commit your existing container (that is create a new image from container’s changes) and then run it with your new mounts.
Example:
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5a8f89adeead ubuntu:14.04 "/bin/bash" About a minute ago Exited (0) About a minute ago agitated_newton
$ docker commit 5a8f89adeead newimagename
$ docker run -ti -v "$PWD/somedir":/somedir newimagename /bin/bash
If it's all OK, stop your old container, and use this new one.
You can also commit a container using its name, for example:
docker commit agitated_newton newimagename
That's it :)
We don't have any way to add volume in running container, but to achieve this objective you may use the below commands:
Copy files/folders between a container and the local filesystem:
docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH
docker cp [OPTIONS] SRC_PATH CONTAINER:DEST_PATH
For reference see:
https://docs.docker.com/engine/reference/commandline/cp/
I've successfully mount /home/<user-name> folder of my host to the /mnt folder of the existing (not running) container. You can do it in the following way:
Open configuration file corresponding to the stopped container, which can be found at /var/lib/docker/containers/99d...1fb/config.v2.json (may be config.json for older versions of docker).
Find MountPoints section, which was empty in my case: "MountPoints":{}. Next replace the contents with something like this (you can copy proper contents from another container with proper settings):
"MountPoints":{"/mnt":{"Source":"/home/<user-name>","Destination":"/mnt","RW":true,"Name":"","Driver":"","Type":"bind","Propagation":"rprivate","Spec":{"Type":"bind","Source":"/home/<user-name>","Target":"/mnt"},"SkipMountpointCreation":false}}
or the same (formatted):
"MountPoints": {
"/mnt": {
"Source": "/home/<user-name>",
"Destination": "/mnt",
"RW": true,
"Name": "",
"Driver": "",
"Type": "bind",
"Propagation": "rprivate",
"Spec": {
"Type": "bind",
"Source": "/home/<user-name>",
"Target": "/mnt"
},
"SkipMountpointCreation": false
}
}
Restart the docker service: service docker restart
This works for me with Ubuntu 18.04.1 and Docker 18.09.0
Jérôme Petazzoni has a pretty interesting blog post on how to Attach a volume to a container while it is running. This isn't something that's built into Docker out of the box, but possible to accomplish.
As he also points out
This will not work on filesystems which are not based on block devices.
It will only work if /proc/mounts correctly lists the block device node (which, as we saw above, is not necessarily true).
Also, I only tested this on my local environment; I didn’t even try on a cloud instance or anything like that
YMMV
Unfortunately the switch option to mount a volume is only found in the run command.
docker run --help
-v, --volume list Bind mount a volume (default [])
There is a way you can work around this though so you won't have to reinstall the applications you've already set up on your container.
Export your container
docker container export -o ./myimage.docker mycontainer
Import as an image
docker import ./myimage.docker myimage
Then docker run -i -t -v /somedir --name mycontainer myimage /bin/bash
A note for using Docker Windows containers after I had to look for this problem for a long time!
Condiditions:
Windows 10
Docker Desktop (latest version)
using Docker Windows Container for image microsoft/mssql-server-windows-developer
Problem:
I wanted to mount a host dictionary into my windows container.
Solution as partially discripted here:
create docker container
docker run -d -p 1433:1433 -e sa_password=<STRONG_PASSWORD> -e ACCEPT_EULA=Y microsoft/mssql-server-windows-developer
go to command shell in container
docker exec -it <CONTAINERID> cmd.exe
create DIR
mkdir DirForMount
stop container
docker container stop <CONTAINERID>
commit container
docker commit <CONTAINERID> <NEWIMAGENAME>
delete old container
docker container rm <CONTAINERID>
create new container with new image and volume mounting
docker run -d -p 1433:1433 -e sa_password=<STRONG_PASSWORD> -e ACCEPT_EULA=Y -v C:\DirToMount:C:\DirForMount <NEWIMAGENAME>
After this i solved this problem on docker windows containers.
My answer will be little different. You can stop your container, add the volume and restart it. How to do it, follow the steps.
docker volume create ubuntu-volume
docker stop <container-name>
sudo docker run -i -t --mount source=ubuntu-volume,target=<target-path-in-container> ubuntu /bin/bash
You can stop and remove the container, append the existing volume in a startup script, and restart from the image. If the already existing existing partitions do keep the data, you shouldn't experience any loss of information. This should also work the same way with Dockerfile and Docker composer.
eg (solr image).
(initial script)
#!/bin/sh
docker pull solr:8.5
docker stop my_solr
docker rm solr:8.5
docker create \
--name my_solr \
-v "/XXXX/docker/solr/solrdata":/var/solr \
-p 8983:8983 \
--restart unless-stopped \
--user 1000:1000 \
-e SOLR_HEAP=1g \
--log-opt max-size=10m \
--log-opt max-file=3 \
solr:8.5
docker cp /home/XXXX/docker/solr/XXXXXXXX.jar my_solr:/opt/solr/contrib/dataimporthandler-extras/lib
docker start my_solr
file with the second volume
#!/bin/sh
docker pull solr:8.5
docker stop my_solr
docker rm solr:8.5
docker create \
--name my_solr \
-v "/XXXX/docker/solr/solrdata":/var/solr \
-v "/XXXX/backups/solr_snapshot_folder":/var/solr_snapshots \
-p 8983:8983 \
--restart unless-stopped \
--user 1000:1000 \
-e SOLR_HEAP=1g \
--log-opt max-size=10m \
--log-opt max-file=3 \
solr:8.5
docker cp /home/XXXX/docker/solr/XXXXXXXX.jar my_solr:/opt/solr/contrib/dataimporthandler-extras/lib
docker start my_solr
Use symlink to the already mounted drive:
ln -s Source_path targer_path_which_is_already_mounted_on_the_running_docker
The best way is to copy all the files and folders inside a directory on your local file system by: docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH
SRC_PATH is on container
DEST_PATH is on localhost
Then do docker-compose down attach a volume to the same DEST_PATH and run Docker containers by using docker-compose up -d
Add volume by following in docker-compose.yml
volumes:
- DEST_PATH:SRC_PATH