Set volume to docker container from wsl from windows 10 - docker

In my windows i installed docker and ubuntu 18.4 wsl.So after expose daemon on localhost, i can use windows daemon on ubuntu wsl.
I created a directory's on drive d and i made a file into data directory like below:
PS D:\wsl\data> dir
Directory: D:\wsl\data
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 8/31/2019 2:27 PM 0 New Text Document.txt
PS D:\wsl\data>
after that, i run a postgres container like below on wsl teminal:
$ docker run --name=pg-docker3 -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=s123 -v /mnt/d/wsl/data:/var/lib/postgresql/data mdillon/postgis:11-alpine
After creating container i open container by bash :
$ docker exec -it pg-docker3 bash
This is data postgres directory:
bash-4.4# ls /var/lib/postgresql/data
PG_VERSION pg_commit_ts pg_logical pg_serial pg_subtrans pg_xact postmaster.pid
base pg_dynshmem pg_multixact pg_snapshots pg_tblspc postgresql.auto.conf
data pg_hba.conf pg_notify pg_stat pg_twophase postgresql.conf
global pg_ident.conf pg_replslot pg_stat_tmp pg_wal postmaster.opts
bash-4.4#
as you can see data directory construct in location but when i listed thing's inside data directory i can see my text file that created in host machine !!!
I created a directory inside data directory calld kk.
bash-4.4# ls /var/lib/postgresql/data
PG_VERSION pg_commit_ts pg_logical pg_serial pg_subtrans pg_xact postmaster.pid
base pg_dynshmem pg_multixact pg_snapshots pg_tblspc postgresql.auto.conf
data pg_hba.conf pg_notify pg_stat pg_twophase postgresql.conf
global pg_ident.conf pg_replslot pg_stat_tmp pg_wal postmaster.opts
bash-4.4# ls /var/lib/postgresql/data/data/
kk
bash-4.4#
But not create this directory inside host as you can see:
PS D:\wsl\data> dir
Directory: D:\wsl\data
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 8/31/2019 2:27 PM 0 New Text Document.txt
Where is the /var/lib/postgresql/data/data/ ? and why it not appeared in /mnt/d/wsl/data ?

You must move your mounting path from /mnt to /. In this answer there is the following explanation : Can you run Docker natively on the new Windows 10 (Ubuntu) bash userspace?
Volumes
Take care when adding volumes. The path C:\dir will be visible as /mnt/c/dir on WSL and as /c/dir/ by docker engine. You can overcome it permanently:
sudo bash -c "echo -e '[automount] \nroot = /'>/etc/wsl.conf"
You must exit and reload WSL after making the change to wsl.conf so that WSL reads in your changes on launch.
Regards

Related

Aerospike Connect to kafka: docker: Error response from daemon: invalid mode: /etc/aerospike-kafka-outbound/aerospike-kafka-outbound.yml

I am trying Aerospike Connect to Kafka.
OS: Windows
I referred https://enterprise.aerospike.com/docs/connect/streaming-from-asdb/installing/from-asdb-to-kafka-installing.html.
I referred: '# With Aerospike 5.0 and later' for deploying on docker
I have these running in docker:
My Windows path for the file: C:/aerospike-kafka-outbound/aerospike-kafka-outbound.yml
C:\Users\Nupur>docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1114b8a97406 aerospike/aerospike-kafka-outbound:4.0.0 "/bin/sh -c '/opt/${…" 37 seconds ago Up 29 seconds compassionate_hofstadter
93343be1d481 aerospike/aerospike-tools "wrapper aql -h 172.…" 4 hours ago Up 4 hours dreamy_mendeleev
5f00ee532c4e aerospike/amc "/opt/amc/amc ' -con…" 8 days ago Up 4 hours 0.0.0.0:8081->8081/tcp aerospike-consoles
a81ff04dbd74 aerospike/aerospike-server-enterprise "/usr/bin/tini -r SI…" 8 days ago Up 4 hours 0.0.0.0:3000-3002->3000-3002/tcp aerospike
Error:
C:\Users\Nupur>docker run -p 8080:8080 -v /C:/aerospike-kafka-outbound/aerospike-kafka-outbound.yml:/etc/aerospike-kafka-outbound/aerospike-kafka-outbound.yml aerospike/aerospike-kafka-outbound:4.0.0
docker: Error response from daemon: invalid mode: /etc/aerospike-kafka-outbound/aerospike-kafka-outbound.yml.
See 'docker run --help'.
As for the front slash used in Windows path, on using backslash:
C:\Users\Nupur>docker run -p 8080:8080 -v C:\\aerospike-kafka-outbound\\aerospike-kafka-outbound.yml:/etc/aerospike-kafka-outbound/aerospike-kafka-outbound.yml aerospike/aerospike-kafka-outbound:4.0.0
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:76: mounting "/run/desktop/mnt/host/c/aerospike-kafka-outbound/aerospike-kafka-outbound.yml" to rootfs at "/etc/aerospike-kafka-outbound/aerospike-kafka-outbound.yml" caused: mount through procfd: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type.
This is my docker directory:
C:\Users\Nupur>docker exec -it 1114b8a97406 bash
root#1114b8a97406:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
root#1114b8a97406:/# cd /etc
root#1114b8a97406:/etc# ls
adduser.conf ca-certificates.conf environment host.conf ld.so.cache machine-id pam.conf rc1.d resolv.conf shells timezone
aerospike-kafka-outbound cron.daily fstab hostname ld.so.conf mke2fs.conf pam.d rc2.d rmt skel update-motd.d
alternatives debconf.conf gai.conf hosts ld.so.conf.d motd passwd rc3.d securetty ssl xattr.conf
apt debian_version group init.d libaudit.conf mtab passwd- rc4.d security subgid
bash.bashrc default group- issue localtime nsswitch.conf profile rc5.d selinux subuid
bindresvport.blacklist deluser.conf gshadow issue.net login.defs opt profile.d rc6.d shadow systemd
ca-certificates dpkg gshadow- kernel logrotate.d os-release rc0.d rcS.d shadow- terminfo
root#1114b8a97406:/etc# cd aerospike-kafka-outbound
root#1114b8a97406:/etc/aerospike-kafka-outbound# ls
aerospike-kafka-outbound.yml
root#1114b8a97406:/etc/aerospike-kafka-outbound#

Why is File Mounted on Container via docker-compose not Accessible?

In my docker-compose file, I try to mount a file from the host into the docker container.
The docker-compose file I have something like this:
version "2"
services:
myservice:
image: images/previmage:1.0.0
volumes:
- /opt/files/aaa.conf:/aaa.conf
After the service is started, I look at the contents at the root of the container using docker from the host:
sudo docker container exec myservice_1 ls /
The result of that ls command for the aaa.conf entry shows that it looks like it is there, but permissions are not what I expect:
drwxr-xr-x. 2 root root 6 Apr 11 2018 opt
-?????????? ? ? ? ? ? aaa.conf
ls: cannot access /aaa.conf: Permission denied
Similarly, if I try other commands like 'cat aaa.conf', I get Permission denied.
I understand that permissions for the file need to be set on the host side.
On the host I made permissions both 755 and then 777, but I still get Permission denied.
Is this the expected behavior?
Edit [running on AWS/EC2]
sudo docker container exec myservice_1 cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
sudo docker container exec myservice_1 id -u
33016
I had same problem, It's for SELinux (Check this post)
Disable SELinux for a specific container
You can disable SELinux for a specific container by adding --security-opt label:disable to your docker run command:
docker container run --security-opt label:disable myservice_1
Adding SELinux Rule (Recommended)
According to this post, You can also use this command to enable access to the files
chcon -Rt svirt_sandbox_file_t /path/to/volume
Completely disable SELinux!
Not recommended, but also works:
su -c "setenforce 0"

Permission issues in nexus3 docker container

When I start nexus3 in a docker container I get the following error messages.
$ docker run --rm sonatype/nexus3:3.8.0
Warning: Cannot open log file: ../sonatype-work/nexus3/log/jvm.log
Warning: Forcing option -XX:LogFile=/tmp/jvm.log
Java HotSpot(TM) 64-Bit Server VM warning: Cannot open file ../sonatype-work/nexus3/log/jvm.log due to Permission denied
Unable to update instance pid: Unable to create directory /nexus-data/instances
/nexus-data/log/karaf.log (Permission denied)
Unable to update instance pid: Unable to create directory /nexus-data/instances
It indicates that there is a file permission issue.
I am using Red Hat Enterprise Linux 7.5 as host machine and the most recent docker version.
On another machine (ubuntu) it works fine.
The issue occurs in the persistent volume (/nexus-data). However, I do not mount a specific volume and let docker use a anonymous one.
If I compare the volumes on both machines I can see the following permissions:
For Red Hat, where it is not working is belongs to root.
$ docker run --rm sonatype/nexus3:3.8.0 ls -l /nexus-data
total 0
drwxr-xr-x. 2 root root 6 Mar 1 00:07 etc
drwxr-xr-x. 2 root root 6 Mar 1 00:07 log
drwxr-xr-x. 2 root root 6 Mar 1 00:07 tmp
On ubuntu, where it is working it belongs to nexus. Nexus is also the default user in the container.
$ docker run --rm sonatype/nexus3:3.8.0 ls -l /nexus-data
total 12
drwxr-xr-x 2 nexus nexus 4096 Mar 1 00:07 etc
drwxr-xr-x 2 nexus nexus 4096 Mar 1 00:07 log
drwxr-xr-x 2 nexus nexus 4096 Mar 1 00:07 tmp
Changing the user with the options -u is not an option.
I could solve it by deleting all local docker images: docker image prune -a
Afterwards it downloaded the image again and it worked.
This is strange because I also compared the fingerprints of the images and they were identical.
An example of docker-compose for Nexus :
version: "3"
services:
#Nexus
nexus:
image: sonatype/nexus3:3.39.0
expose:
- "8081"
- "8082"
- "8083"
ports:
# UI
- "8081:8081"
# repositories http
- "8082:8082"
- "8083:8083"
# repositories https
#- "8182:8182"
#- "8183:8183"
environment:
- VIRTUAL_PORT=8081
volumes:
- "./nexus/data/nexus-data:/nexus-data"
Setup the volume :
mkdir -p ./nexus/data/nexus-data
sudo chown -R 200 nexus/ # 200 because it's the UID of the nexus user inside the container
Start Nexus
sudo docker-compose up -d
hf
You should attribute correct right to the folder where the persistent volume is located.
chmod u+wxr -R <folder of /nexus-data volumes>
Be carefull, if you execute previous command, it would give write, read and execution right to all users. If you want to give more restricted right, you should modify the command.

Mounting Folders in Docker: Windows Host, Windows Image

How to use the docker mount option to share a local folder in Docker Container? Currently, I am using this command but I am not being successful.
docker run --mount source='c:\temp',target='c:\temp' -i newname:latest
I get this error -
C:\Program Files\Docker\docker.exe: Error response from daemon: invalid mount config for type "volume": invalid volume name.
My environment:
Host: Windows Server, version 1709
Docker Container: Windows Server Core, v1709
You need to use bind mount. Example below maps your host directory c:\users\public\ to the one which is inside container c:\users\public and then outputs content of that directory.
PS C:\Users\gsuvalia> docker run --rm --mount type=bind,source=c:\users\public\,destination=c:\users\public\ microsoft/nanoserver powershell get-childitem c:\users\public
Directory: C:\users\public
Mode LastWriteTime Length Name
---- ------------- ------ ----
d-r--- 12/1/2017 10:16 PM Documents
d-r--- 7/16/2016 6:47 AM Downloads
d-r--- 7/16/2016 6:47 AM Music
d-r--- 12/1/2017 10:16 PM Pictures
d----- 8/22/2017 10:26 PM Roaming
d-r--- 7/16/2016 6:47 AM Videos

Riak container does not start when its data volume is mounted

The following command works perfectly and the riak service starts as expected:
docker run --name=riak -d -p 8087:8087 -p 8098:8098 -v $(pwd)/schemas:/etc/riak/schema basho/riak-ts
The local schemas directory is mounted successfully and the sql file in it is read by riak. However if I try to mount the riak's data or log directories, the riak service does not start and timeouts after 15 seconds:
docker run --name=riak -d -p 8087:8087 -p 8098:8098 -v $(pwd)/logs:/var/log/riak -v $(pwd)/schemas:/etc/riak/schema basho/riak-ts
Output of docker logs riak:
+ /usr/sbin/riak start
riak failed to start within 15 seconds,
see the output of 'riak console' for more information.
If you want to
wait longer, set the environment variable
WAIT_FOR_ERLANG to the
number of seconds to wait.
Why does riak not start when it's logs or data directories are mounted to local directories?
This issue is with the directory owner of mounted log folder. The folder $GROUP and $USER are expected to be riak as follow:
root#20e489124b9a:/var/log# ls -l
drwxr-xr-x 2 riak riak 4096 Jul 19 10:00 riak
but with volumes you are getting:
root#3546d261a465:/var/log# ls -l
drwxr-xr-x 2 root root 4096 Jul 19 09:58 riak
One way to solve this is to have the directory ownership as riak user and group on host before starting the container. I looked the UID/GID (/etc/passwd) in docker image and they were:
riak:x:102:105:Riak user,,,:/var/lib/riak:/bin/bash
now change the ownership on host directories before starting the container as:
sudo chown 102:105 logs/
sudo chown 102:105 data/
This should solve it. At least for now. Details here.

Resources