Unable to mount cifs filesystem in Docker container - docker

I'm on Docker 17.06.0-ce and I'm attempting to mount a CIFS share in a container and only having some luck. If I use --privileged, it works, but that's not desirable for me. I've tried using --cap-add as well as suggested in this answer (even trying with --cap-add ALL with no success.
The same mount command works fine on the host system as well.
Here's a simple docker file I've tried playing with
FROM alpine:latest
RUN apk add --no-cache cifs-utils
Run with many different permutations, all with the same result below:
Works:
docker run --rm -it --privileged cifs-test /bin/sh
Doesn't Work:
docker run --rm -it --cap-add SYS_ADMIN --cap-add DAC_READ_SEARCH cifs-test /bin/sh
Doesn't Work:
docker run --rm -it --cap-add SYS_ADMIN --cap-add DAC_READ_SEARCH --cap-add NET_ADMIN cifs-test /bin/sh
Doesn't Work:
docker run --rm -it --cap-add ALL cifs-test /bin/sh
And the command:
mkdir /test && mount.cifs //myserver/testpath /test -o user=auser,password=somepass,domain=mydomain
And the result from each run command above except the first:
mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
Has something changed in Docker that requires --privileged all the time for these types of mounts now? Or is there something else I'm missing?

I started using docker-volume-netshare so far with good success. There are some minor problems, like volumes created with docker volume create not being persistent, but nevertheless it looks like this volume driver is quite usable. One advantage is that special caps/privileged mode are not necessary. Here are some hints on how to use it.
Install (Ubuntu/Debian)
$ curl -L -o /tmp/docker-volume-netshare_0.34_amd64.deb https://github.com/ContainX/docker-volume-netshare/releases/download/v0.34/docker-volume-netshare_0.34_amd64.deb
$ sudo dpkg -i /tmp/docker-volume-netshare_0.34_amd64.deb
$ rm /tmp/docker-volume-netshare_0.34_amd64.deb
Configure
$ sudo vi /etc/default/docker-volume-netshare
enter as single setting
DKV_NETSHARE_OPTS="cifs --netrc=/root/"
then
$ sudo vi /root/.netrc
enter the following settings per host:
machine <host>
username <user>
password <password>
domain <domain>
Note that <host> must be a host name or an IP address followed by a colon (e.g. 10.20.30.4:)
Enable the volume driver as a systemd service
Note: if your OS does not support systemd, another method to install it as a service is necessary.
$ sudo systemctl enable docker-volume-netshare
Use a volume in docker run and docker service create
$ sudo docker run -it --rm --mount type=volume,volume-driver=cifs,source=<myvol>,destination=<absolute-path-in-container>,volume-opt=share=<ip>:/<share> ubuntu:zesty bash
$ sudo docker service create --name <name> --mount type=volume,volume-driver=cifs,source=<myvol>,destination=<absolute-path-in-container>,volume-opt=share=<host>/<share> <image>
Obviously it is not necessary to use the identical volume in multiple containers, because the volumes only map to a cifs share which in turn is shared among containers mounting it. As mentioned above, don't use docker volume create with this volume driver, as volumes are lost as soon as docker-volume-netshare is stopped and/or restarted (and hence on reboot).
Get help
$ docker-volume-netshare --help
$ docker-volume-netshare cifs --help
Logs
Hint: for debugging use DKV_NETSHARE_OPTS="cifs --netrc=/root/ --verbose" in /etc/default/docker-volume-netshare or stop the service and start docker-volume-netshare cifs --netrc=/root/ --verbose in a shell)
$ dmesg | tail
$ tail -50 /var/log/docker-volume-netshare.log
Resources
github
project

Related

why can i not run a X11 application?

So, as the title states, I'm a docker newbie.
I downloaded and installed the archlinux/base container which seems to work great so far. I've setup a few things, and installed some packages (including xeyes) and I now would like to launch xeyes. For that I found out the CONTAINER ID by running docker ps and then used that ID in my exec command which looks now like:
$ docker exec -it -e DISPLAY=$DISPLAY 4cae1ff56eb1 xeyes
Error: Can't open display: :0
Why does it still not work though? Also, how can I stop my running instance without losing its configured state? Previously I have exited the container and all my configuration and software installations were gone when I restarted it. That was not desired. How do I handle this correctly?
Concerning the X Display you need to share the xserver socket (note: docker can't bind mount a volume during an exec) and set the $DISPLAY (example Dockerfile):
FROM archlinux/base
RUN pacman -Syyu --noconfirm xorg-xeyes
ENTRYPOINT ["xeyes"]
Build the docker image: docker build --rm --network host -t so:57733715 .
Run the docker container: docker run --rm -it -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix$DISPLAY so:57733715
Note: in case of No protocol specified errors you could disable host checking with xhost + but there is a warning to that (man xhost for additional information).

There is any "Podman Compose"?

I recently found out about Podman (https://podman.io). Having a way to use Linux fork processes instead of a Daemon and not having to run using root just got my attention.
But I'm very used to orchestrate the containers running on my machine (in production we use kubernetes) using docker-compose. And I truly like it.
So I'm trying to replace docker-compose. I will try to keep docker-compose and using podman as an alias to docker as Podman uses the same syntax as docker:
alias docker=podman
Will it work? Can you suggest any other tool? I really intend to keep my docker-compose.yml file, if possible.
Yes, that is doable now, check podman-compose, this is one way of doing it, another way is to convert the docker-compose yaml file to a kubernetes deployment using Kompose. there is a blog post from Jérôme Petazzoni #jpetazzo: from docker-compose to kubernetes deployment
Update 6 May 2022 : Podman now supports Docker Compose v2.2 and higher (see Podman 4.1.0 release notes)
Old answer:
Running docker-compose with Podman as a normal user (rootless)
Requirement: Podman version >= 3.2.1 (released in June 2021)
Install the executable docker-compose
curl -sL -o ~/docker-compose https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)
chmod 755 ~/docker-compose
Alternatively you could also run docker-compose in a container image (see below).
Run
systemctl --user start podman.socket
Set the environment variable DOCKER_HOST
export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock
Run
~/docker-compose up -d
Running docker-compose with Podman as root
Requirement: Podman version >= 3.0 (released in February 2021)
Follow the same procedure but remove the flag --user
systemctl start podman.socket
Running docker-compose in a container image
Use the container image docker.io/docker/compose to run
docker-compose
podman \
run \
--rm \
--detach \
--env DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock \
--security-opt label=disable \
--volume $XDG_RUNTIME_DIR/podman/podman.sock:$XDG_RUNTIME_DIR/podman/podman.sock \
--volume $(pwd):$(pwd) \
--workdir $(pwd) \
docker.io/docker/compose \
--verbose \
up -d
(the flag --verbose is optional)
The same command with short command-line options on a single line:
podman run --rm -d -e DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock --security-opt label=disable -v $XDG_RUNTIME_DIR/podman/podman.sock:$XDG_RUNTIME_DIR/podman/podman.sock -v $(pwd):$(pwd) -w $(pwd) docker.io/docker/compose --verbose up -d
Regarding SELINUX: Runnng Podman with SELINUX is preferable from a security point-of-view, but I didn't get it to work on a Fedora 34 computer so I disabled SELINUX by adding the command-line option
--security-opt label=disable
Troubleshooting tips
Test the Docker REST API
A minimal check to see that the Docker REST API is working:
$ curl -H "Content-Type: application/json" \
--unix-socket $XDG_RUNTIME_DIR/podman/podman.sock \
http://localhost/_ping
OK$
Avoid short container image names
If any of your docker-compose.yaml or Dockerfile files contain a short container image name, for instance
$ grep image: docker-compose.yaml
image: mysql:8.0.19
$
$ grep FROM Dockerfile
FROM python:3.9
$
edit the files to use the whole container image name instead
$ grep image: docker-compose.yaml
image: docker.io/library/mysql:8.0.19
$
$ grep FROM Dockerfile
FROM docker.io/library/python:3.9
$
Most often short names have been used to reference DockerHub Official Images
(a catalogue) so a good guess would be to prepend the container image name with docker.io/library/
There are currently many different container image registries, not just DockerHub (docker.io). Writing the whole container image name is thus a good practice. Podman might complain otherwise depending on how Podman is configured.
Rootless users can't bind to ports below 1024
If for instance
$ grep -A1 ports: docker-compose.yml
ports:
- 80:80
$
edit docker-compose.yaml so that the host port number >= 1024, for instance 8080
$ grep -A1 ports: docker-compose.yml
ports:
- 8080:80
$
An alternative solution is to adjust net.ipv4.ip_unprivileged_port_start with sysctl (see Shortcomings of Rootless Podman)
In case Systemd is missing
Most Linux distributions use Systemd where you would preferably start the Podman service (providing the REST API) by "starting" the Podman socket
systemctl --user start podman.socket
or
systemctl start podman.socket
but in case Systemd is missing you could also start the Podman service directly
podman system service --time 0 unix:/some/path/podman.sock
Systemd gives the extra benefit that the Podman service is started on demand with Systemd socket activation and stops after some time of inactivity.
Caveat: Swarm functionality is missing
A difference to Docker is that the functionality relating to Swarm is not supported when using docker-compose with Podman.
References:
https://www.redhat.com/sysadmin/podman-docker-compose
https://github.com/containers/podman/discussions/10644#discussioncomment-857897
Ensure Podman is installed on your machine.
You can install Podman Compose in a terminal with the following command:
pip3 install https://github.com/containers/podman-compose/archive/devel.tar.gz
cd into the directory your docker-compose file is located in
Run podman-compose up
See the following link for a decent introduction.

Mounting nfs shares inside docker container

Does anyone know how to mount nfs share inside docker container with centos base image? I've tried this command:
mount server:/dir /mount/point
and got the next error:
mount.nfs: rpc.statd is not running but is required for remote locking.
mount.nfs: Either use '-o nolock' to keep locks local, or start statd.
mount.nfs: an incorrect mount option was specified
when I try to use it with -o nolock option, the error is:
mount.nfs: Operation not permitted
Starting from docker 17.06, you can mount NFS shares to the container directly when you run it, without the need of extra capabilities
export NFS_VOL_NAME=mynfs
export NFS_LOCAL_MNT=/mnt/mynfs
export NFS_SERVER=my.nfs.server.com
export NFS_SHARE=/my/server/path
export NFS_OPTS=vers=4,soft
docker run --mount \
"src=$NFS_VOL_NAME,dst=$NFS_LOCAL_MNT,volume-opt=device=:$NFS_SHARE,\"volume-opt=o=addr=$NFS_SERVER,$NFS_OPTS\",type=volume,volume-driver=local,volume-opt=type=nfs" \
busybox ls $NFS_LOCAL_MNT
Alternatively, you can create the volume before the container:
docker volume create \
--driver local \
--opt type=nfs \
--opt o=addr=$NFS_SERVER,$NFS_OPTS \
--opt device=:$NFS_SHARE \
$NFS_VOL_NAME
docker run --rm -v $NFS_VOL_NAME:$NFS_LOCAL_MNT busybox ls $NFS_LOCAL_MNT
Got the hint from
https://github.com/moby/moby/issues/28809
official docs from docker: https://docs.docker.com/storage/volumes/#create-a-service-which-creates-an-nfs-volume
For using mount, you'll need the CAP_SYS_ADMIN capability, which is dropped by Docker when creating the container.
There are several solutions for this:
Start the container with the --cap-add sys_admin flag. This causes Docker to retain the CAP_SYS_ADMIN capability, which should allow you to mount a NFS share from within the container. This might be a security issue; do not do this in untrusted containers. [A previous version of this answer suggested using the --privileged=true to retain all capabilities, thanks to #earcam for the suggestion to use --cap-add instead].
Mount the NFS share on the host and pass it into the container as a host volume:
you#host > mount server:/dir /path/to/mount/point
you#host > docker run -v /path/to/mount/point:/path/to/mount/point
Use a Docker volume plugin (like the Netshare plugin) to directly mount the NFS share as a container volume:
you#host > docker run \
--volume-driver=nfs \
-v server/dir:/path/to/mount/point \
centos
For the second option listed in the accepted answer, I'm not sure if you have actually tried to use the "docker run -v" command to pass a NFS share on the host to docker container as a volume.
I have recently tried to do so, below is the info for the nfs share on host:
nfs-server:/path_to_mount on /path_dest type nfs
and then:
docker run -it -v /path_dest:/path_in_docker docker_name bash
But the docker daemon always reports below error:
docker: Error response from daemon: stat /path_dest: permission denied.
After many searches, I found that the error actually comes from docker daemon, which is running as "root". When docker runs a container with volume to mount, it will request docker daemon to mount it. The problem is, NFS server will handle "root" differently. By default, NFS server will map the "root" to "nobody", causing the error message: reference
I mount the nfs on docker container, thanks for #helmbert .
Run a docker container with the --privileged=true flag.
$ docker run -it --privileged=true centos:7 bash
[root#f7915ae635aa /]# yum install -y nfs-utils
Install the nfs tool package and mount nfs on CentOS.
[root#f7915ae635aa /]# yum install -y nfs-utils
[root#f7915ae635aa /]# mount -t nfs example.tw:/target/ /srv -o nolock
Show mount of the nfs server.
[root#f7915ae635aa /]# showmount example.tw
Hosts on example.tw:
10.10.10.1
10.10.10.2
By adding --cap-add sys_admin flag to client container wasn't enough for me. I was getting error:
mount.nfs: mount(2): Permission denied
mount.nfs: access denied by server while mounting 1.2.3.4:/exports
After hours of research I've found that it looks like full privilege --privileged is needed to mount correctly inside docker container ..
Also don't forget to install necessary nfs client packages inside your docker container. On debian based containers:
apt-get install -y nfs-common

Docker logging to container

I'm fresh user of Docker. The fist problem with which I'm faced is logging into container.
I'm found solutions to execute container bash commands by
docker exec -it ID bash
But, this is solution only for install/ remove packages. What to use if I want to edit nginx config in docker container ?
One of solutions can be loggin to container via ssh connection, but maybe Docker have something own for this ?, I mean easilly access without install OpenSSH ?
as you said,
docker exec -it container_id bash
and then use your favorite editor to edit any nginx config file. vi or nano is usually installed, but you may need to install emacs or vim, if this is your favorite editor
if you have just a few characters to modify,
docker exec container_id sed ...
might do the job. If you want to SSH into your container, you will need to install SSH and deal with the SSH keys, I am not sure this is what you need.
You're going about it the wrong way. You should rarely need to log into a container to edit files.
Instead, mount the nginx.conf with -v from the host. That way you can edit the file with your normal editor. Once you've got the config working the way you want it, you can then build a new image with it baked in.
In general, you have to get into the mindset of containers being ephemeral. You don't patch them; you throw them away and replace them with a fixed version.
How: Docker logging to container
Yes, you can. You can login the running container.
Exist docker exec or docker attach is not good enough. Looking to start a shell inside a Docker container? The solution is: jpetazzo/nsenter with two commands: nsenter and docker-enter.
If you are in Linux environment, then run below command:
docker run --rm -v /usr/local/bin:/target jpetazzo/nsenter
docker ps
# replace <container_name_or_ID> with real container name or ID.
PID=$(docker inspect --format {{.State.Pid}} <container_name_or_ID>)
nsenter --target $PID --mount --uts --ipc --net --pid
Then you are in that running container, you can run any linux commands now.
I prefer the other command docker-enter. Without login the container, you can directly run linux commands in container with docker-enter command. Second, I can't memory multiple options of nsenter command and no need to find out the container's PID.
docker-enter 0e8c248982c5 ls /opt
If you are mac or windows user, run docket with toolbox:
docker-machine ssh default
docker run --rm -v /usr/local/bin:/target jpetazzo/nsenter
PID=$(docker inspect --format {{.State.Pid}} 0e8c248982c5)
sudo nsenter --target $PID --mount --uts --ipc --net --pid
If you are mac or windows user, run docket with boot2docker:
boot2docker ssh
docker run --rm -v /usr/local/bin:/target jpetazzo/nsenter
PID=$(docker inspect --format {{.State.Pid}} 0e8c248982c5)
sudo nsenter --target $PID --mount --uts --ipc --net --pid
Note: The command docker run --rm -v /usr/local/bin:/target jpetazzo/nsenter only need run one time.
How: edit nginx config
For your second question, you can think about ONBUILD in Docker.
ONBUILD COPY nginx.conf /etc/nginx/nginx.conf
With this solution, you can:
edit nginx.conf in local, you can use any exist editor .
needn't build your image every time after you change nginx configuration.
every time, after you change nginx.conf file in local, you need stop, remove and re-run the containe, new nginx.conf file will be deployed into contrainer when docker run command.
You can refer the detail on how to use ONBUILD here: docker build

How can I add a volume to an existing Docker container?

I have a Docker container that I've created simply by installing Docker on Ubuntu and doing:
sudo docker run -i -t ubuntu /bin/bash
I immediately started installing Java and some other tools, spent some time with it, and stopped the container by
exit
Then I wanted to add a volume and realised that this is not as straightforward as I thought it would be. If I use sudo docker -v /somedir run ... then I end up with a fresh new container, so I'd have to install Java and do what I've already done before just to arrive at a container with a mounted volume.
All the documentation about mounting a folder from the host seems to imply that mounting a volume is something that can be done when creating a container. So the only option I have to avoid reconfiguring a new container from scratch is to commit the existing container to a repository and use that as the basis of a new one whilst mounting the volume.
Is this indeed the only way to add a volume to an existing container?
You can commit your existing container (that is create a new image from container’s changes) and then run it with your new mounts.
Example:
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5a8f89adeead ubuntu:14.04 "/bin/bash" About a minute ago Exited (0) About a minute ago agitated_newton
$ docker commit 5a8f89adeead newimagename
$ docker run -ti -v "$PWD/somedir":/somedir newimagename /bin/bash
If it's all OK, stop your old container, and use this new one.
You can also commit a container using its name, for example:
docker commit agitated_newton newimagename
That's it :)
We don't have any way to add volume in running container, but to achieve this objective you may use the below commands:
Copy files/folders between a container and the local filesystem:
docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH
docker cp [OPTIONS] SRC_PATH CONTAINER:DEST_PATH
For reference see:
https://docs.docker.com/engine/reference/commandline/cp/
I've successfully mount /home/<user-name> folder of my host to the /mnt folder of the existing (not running) container. You can do it in the following way:
Open configuration file corresponding to the stopped container, which can be found at /var/lib/docker/containers/99d...1fb/config.v2.json (may be config.json for older versions of docker).
Find MountPoints section, which was empty in my case: "MountPoints":{}. Next replace the contents with something like this (you can copy proper contents from another container with proper settings):
"MountPoints":{"/mnt":{"Source":"/home/<user-name>","Destination":"/mnt","RW":true,"Name":"","Driver":"","Type":"bind","Propagation":"rprivate","Spec":{"Type":"bind","Source":"/home/<user-name>","Target":"/mnt"},"SkipMountpointCreation":false}}
or the same (formatted):
"MountPoints": {
"/mnt": {
"Source": "/home/<user-name>",
"Destination": "/mnt",
"RW": true,
"Name": "",
"Driver": "",
"Type": "bind",
"Propagation": "rprivate",
"Spec": {
"Type": "bind",
"Source": "/home/<user-name>",
"Target": "/mnt"
},
"SkipMountpointCreation": false
}
}
Restart the docker service: service docker restart
This works for me with Ubuntu 18.04.1 and Docker 18.09.0
Jérôme Petazzoni has a pretty interesting blog post on how to Attach a volume to a container while it is running. This isn't something that's built into Docker out of the box, but possible to accomplish.
As he also points out
This will not work on filesystems which are not based on block devices.
It will only work if /proc/mounts correctly lists the block device node (which, as we saw above, is not necessarily true).
Also, I only tested this on my local environment; I didn’t even try on a cloud instance or anything like that
YMMV
Unfortunately the switch option to mount a volume is only found in the run command.
docker run --help
-v, --volume list Bind mount a volume (default [])
There is a way you can work around this though so you won't have to reinstall the applications you've already set up on your container.
Export your container
docker container export -o ./myimage.docker mycontainer
Import as an image
docker import ./myimage.docker myimage
Then docker run -i -t -v /somedir --name mycontainer myimage /bin/bash
A note for using Docker Windows containers after I had to look for this problem for a long time!
Condiditions:
Windows 10
Docker Desktop (latest version)
using Docker Windows Container for image microsoft/mssql-server-windows-developer
Problem:
I wanted to mount a host dictionary into my windows container.
Solution as partially discripted here:
create docker container
docker run -d -p 1433:1433 -e sa_password=<STRONG_PASSWORD> -e ACCEPT_EULA=Y microsoft/mssql-server-windows-developer
go to command shell in container
docker exec -it <CONTAINERID> cmd.exe
create DIR
mkdir DirForMount
stop container
docker container stop <CONTAINERID>
commit container
docker commit <CONTAINERID> <NEWIMAGENAME>
delete old container
docker container rm <CONTAINERID>
create new container with new image and volume mounting
docker run -d -p 1433:1433 -e sa_password=<STRONG_PASSWORD> -e ACCEPT_EULA=Y -v C:\DirToMount:C:\DirForMount <NEWIMAGENAME>
After this i solved this problem on docker windows containers.
My answer will be little different. You can stop your container, add the volume and restart it. How to do it, follow the steps.
docker volume create ubuntu-volume
docker stop <container-name>
sudo docker run -i -t --mount source=ubuntu-volume,target=<target-path-in-container> ubuntu /bin/bash
You can stop and remove the container, append the existing volume in a startup script, and restart from the image. If the already existing existing partitions do keep the data, you shouldn't experience any loss of information. This should also work the same way with Dockerfile and Docker composer.
eg (solr image).
(initial script)
#!/bin/sh
docker pull solr:8.5
docker stop my_solr
docker rm solr:8.5
docker create \
--name my_solr \
-v "/XXXX/docker/solr/solrdata":/var/solr \
-p 8983:8983 \
--restart unless-stopped \
--user 1000:1000 \
-e SOLR_HEAP=1g \
--log-opt max-size=10m \
--log-opt max-file=3 \
solr:8.5
docker cp /home/XXXX/docker/solr/XXXXXXXX.jar my_solr:/opt/solr/contrib/dataimporthandler-extras/lib
docker start my_solr
file with the second volume
#!/bin/sh
docker pull solr:8.5
docker stop my_solr
docker rm solr:8.5
docker create \
--name my_solr \
-v "/XXXX/docker/solr/solrdata":/var/solr \
-v "/XXXX/backups/solr_snapshot_folder":/var/solr_snapshots \
-p 8983:8983 \
--restart unless-stopped \
--user 1000:1000 \
-e SOLR_HEAP=1g \
--log-opt max-size=10m \
--log-opt max-file=3 \
solr:8.5
docker cp /home/XXXX/docker/solr/XXXXXXXX.jar my_solr:/opt/solr/contrib/dataimporthandler-extras/lib
docker start my_solr
Use symlink to the already mounted drive:
ln -s Source_path targer_path_which_is_already_mounted_on_the_running_docker
The best way is to copy all the files and folders inside a directory on your local file system by: docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH
SRC_PATH is on container
DEST_PATH is on localhost
Then do docker-compose down attach a volume to the same DEST_PATH and run Docker containers by using docker-compose up -d
Add volume by following in docker-compose.yml
volumes:
- DEST_PATH:SRC_PATH

Resources