how do I debug this docker permission denied error? - docker

I'm trying to follow the Bazel docker tutorial. I had IT install docker on one of our redhat 7.9 machines and followed the steps of the tutorial and ran the following command:
docker run -e USER="11021" -u="11021" -v `pwd`/workspace:/src/workspace -v `pwd`/build_output:/tmp/build_output -w /src/workspace gcr.io/bazel-public/bazel:latest --output_user_root=`pwd`/build_output build //absl/...
FATAL: mkdir('/<path>/abseil/build_output'): (error: 13): Permission denied
When I look at the file system I see:
drwxrwxrwx 2 11021 users 4096 Feb 3 17:35 build_output
drwxrwxrwx 2 11021 users 4096 Feb 3 17:35 workspace
How do I debug what the problem is?

Related

docker-compose.yml not found - error on build

I created a new VM with ubuntu 22.04 and asked to install docker
When I create a docker-compose file and having to run the build, the following errors occur:
pilati#ubuntu-web-containers:/var/www/mysql$ ls -la
total 12
drwxr-xr-x 2 root root 4096 out 1 16:42 .
drwxr-xr-x 4 root root 4096 out 1 16:40 ..
-rwxrwxrwx 1 root root 473 out 1 16:42 docker-compose.yml
pilati#ubuntu-web-containers:/var/www/mysql$ sudo docker-compose build
ERROR:
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?
Supported filenames: docker-compose.yml, docker-compose.yaml, compose.yml, compose.yaml
pilati#ubuntu-web-containers:/var/www/mysql$ sudo docker-compose -f /var/www/mysql/docker-compose.yml build
ERROR: .FileNotFoundError: [Errno 2] No such file or directory: '/var/www/mysql/docker-compose.yml'
pilati#ubuntu-web-containers:/var/www/mysql$
I reinstalled the VM from scratch and nothing works.
Any way to solve this problem?
Put your compose to your home folder, it should work from there. That is because you installed docker with snap, install it as is from the official site.

testing ssh key on host works fine but not in container with same keys

I have some ssh keys I use with github. Per this page, I tested them locally:
ssh -T git#github.com
Hi doug-companyname! You've successfully authenticated, but GitHub does not provide shell access.
I'm running a docker-container and under volumes in docker-compose I have:
- ~/.ssh:/home/rstudio/.ssh
This is to give the container the same keys as local.
When I run the container if I look in ~/.ssh I do see the keys:
rstudio#9b5b5114115b:~$ ls -l ~/.ssh
total 12
-rw------- 1 1001 1001 452 Nov 7 19:35 id_ed25519
-rw-r--r-- 1 1001 1001 131 Nov 7 19:35 id_ed25519.pub
-rw-r--r-- 1 1001 1001 2212 Nov 15 16:50 known_hosts
These are indeed the same as the hosts, my local.
However, when I try to do the same test I get:
rstudio#9b5b5114115b:~$ ssh -T git#github.com
bash: ssh: command not found
Why are my keys not working in my container but they work fine on local?
I suspect that your image does not have an SSH client installed by default. Add the below line in your Dockerfile.
RUN apk add --no-cache openssh-client

Docker - Can mount an NFS share into a container but not a sub-directory of it

I have an NFS share with the following properties:
Mounted on my host on /nfs/external_disk
Owner user is test_user with UID 1234
Group is test_group with GID 2222
Permissions is 750
I have a small Dockerfile with the following content
ARG tag=lts
from jenkins/jenkins:${tag}
user root
# Create a new user and new group that matches what is on the host.
ARG username=test_user
ARG groupname=test_group
ARG uid=1234
ARG gid=2222
RUN groupadd -g ${gid} ${groupname} && \
mkdir -p /users && \
useradd -l -m -u ${uid} -g ${groupname} -s /bin/bash -d /users/${username} ${username}
user ${username}
After building the image (named custom_jenkins), and when I run the following command, the container is started properly and I see the original Jenkins homer stuff now copied to the share.
docker run -td --rm -v /nfs/external_disk:/var/jenkins_home custom_jenkins
However if I want to mount a sub-directory of the NFS share, say ${NFS_SHARE}/jenkins_home, then I get an error:
docker run -td --rm -v /nfs/external_disk/jenkins_home:/var/jenkins_home custom_jenkins
docker: Error response from daemon: error while creating mount source path '/nfs/external_disk/jenkins_home': mkdir /nfs/external_disk/jenkins_home: permission denied.
Now even if I attempt to create the sub-directory myself before starting the container, I still get the same error. Even when I set the permissions of the sub-directory to be 777.
Note that I am running as test_user which has the same UID/GID as in the container and it actually owns the NFS share.
I have a feeling that when docker attempts to create a sub-directory, it attempts to create it as some different user (e.g. the "docker" user) which causes it to fail while creating the folder since it has no access inside the share.
Can anyone help? thanks in advance.
I tried to reproduce. Works just fine for me. Perhaps I am missing some constraint. Hope this helps anyway. Note at step 6 the owner and the group for the file that I created from the container. This might answer one of your questions.
Step 1: I created a NFS share somewhere in my LAN
Step 2: I mounted the share on the machine that's running the docker engine
sudo mount 192.168.0.xxx:/i-data/b4024d5b/nfs/NFS /mnt/nsa320/
neo#neo-desktop:nsa320$ mount | grep NFS
192.168.0.xxx:/i-data/b4024d5b/nfs/NFS on /mnt/nsa320 type nfs (rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.xxx,mountvers=3,mountport=3775,mountproto=udp,local_lock=none,addr=192.168.0.xxx)
Step 3: I created some sample files and a sub-directory:
neo#neo-desktop:nsa320$ ls -la /mnt/nsa320/
total 12
drwxrwxrwx 3 root root 4096 Jul 21 22:54 .
drwxr-xr-x 3 root root 4096 Jul 21 22:41 ..
-rw-r--r-- 1 neo neo 0 Jul 21 22:45 dummyFile
-rw-r--r-- 1 root root 0 Jul 21 22:53 fileCreatedFromContainer << THIS WAS CREATED FROM A CONTAINER THAT WAS NOT LAUNCHED WITH THE --user OPTION
drwxr-xr-x 2 neo neo 4096 Jul 21 22:54 subfolder
Step 4: Launched a dummy container and mounted the sub-directory (1000 is the UID of the user neo in the my OS):
docker run -d -v /mnt/nsa320/subfolder:/var/externalMount --user 1000 alpine tail -f /dev/null
Step 5: Connected in container to check the mount(I can read and write in the subfolder located on the NFS)
neo#neo-desktop:nsa320$ docker exec -ti ded1dc79773e sh
/ $ ls /var/externalMount/
fileInSubfolder
/ $ touch /var/externalMount/fileInSubfolderCreatedFromContainer
Step 6: Back on the host, to whom does the file that I created from the container belongs to:
neo#neo-desktop:nsa320$ ls -la /mnt/nsa320/subfolder/
total 8
drwxr-xr-x 2 neo neo 4096 Jul 21 23:23 .
drwxrwxrwx 3 root root 4096 Jul 21 22:54 ..
-rw-r--r-- 1 neo neo 0 Jul 21 22:54 fileInSubfolder
-rw-r--r-- 1 neo root 0 Jul 21 23:23 fileInSubfolderCreatedFromContainer
Maybe off-topic: whoami executed in the container returns just the UID:
$ whoami
whoami: unknown uid 1000

sh: ./bc2influx: not found when entering in a stopped container

I have an issue when running a docker container.
➜ bc_to_influx git:(master) ✗ docker run registry.gitlab.com/xxx/bc_to_influx:latest
standard_init_linux.go:207: exec user process caused "no such file or directory"
When I debug, I enter in the stopped container:
docker commit 0db73216baaf user/test_image
docker run -ti --entrypoint=sh user/test_image
on ls command, I can only my executable:
/bc2influx # ls -al
total 20552
drwxr-xr-x 1 root root 4096 Jun 6 10:32 .
drwxr-xr-x 1 root root 4096 Jun 6 11:53 ..
-rwxr-xr-x 1 root root 21034520 Jun 6 10:29 bc2influx
/bc2influx #
but when I try to execute, I get:
/bc2influx # ./bc2influx
sh: ./bc2influx: not found
I can vi, cat the execute, but not execute it
here is my Dockerfile
FROM alpine
WORKDIR /bc2influx/
COPY ./release/bc2influx /bc2influx/
RUN ls -al /bc2influx/
CMD [ "./bc2influx" ]
I previously build my executable with:
go build -o ./release/bc2influx -v -ldflags '-extldflags "-static"' ./...
Any idea what's going on ?
Looks like musl library issue try this build command go build -ldflags="-s -w".

Cannot docker build because of "Couldn't run auplink before unmount" error

When I run docker build I get this:
Sending build context to Docker daemon 10.24kB
WARN[11935] Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/21647778a50f097d4535246ec5206960dd909f4bb8b0e3d04fdd53a7402fc2de-init: exec: "auplink": executable file not found in $PATH
Step 1/2 : FROM debian:jessie
---> 86baf4e8cde9
Step 2/2 : RUN apt-get update
WARN[11935] Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/21647778a50f097d4535246ec5206960dd909f4bb8b0e3d04fdd53a7402fc2de: exec: "auplink": executable file not found in $PATH
---> Running in 1fef9bef5bf7
ERRO[11934] containerd: start container error="shim error: fork/exec /usr/bin/docker-runc: exec format error" id=1fef9bef5bf77141a97669d2aa785e74f9027a849919a937f714e93fbae3916d
ERRO[11935] Create container failed with error: shim error: fork/exec /usr/bin/docker-runc: exec format error
ERRO[11934] containerd: deleting container error="fork/exec /usr/bin/docker-runc: exec format error: \"\""
WARN[11935] Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/21647778a50f097d4535246ec5206960dd909f4bb8b0e3d04fdd53a7402fc2de: exec: "auplink": executable file not found in $PATH
shim error: fork/exec /usr/bin/docker-runc: exec format error
Here is the content of my Dockerfile:
FROM debian:jessie
RUN apt-get update
What is the issue? It makes no sens to me.
ll /usr/bin | grep docker
-rwxr-xr-x 1 root root 18471276 Aug 3 22:08 docker*
-rwxr-xr-x 1 root root 9938352 Aug 3 22:08 docker-containerd*
-rwxr-xr-x 1 root root 8941944 Aug 3 22:08 docker-containerd-ctr*
-rwxr-xr-x 1 root root 3824920 Aug 3 22:08 docker-containerd-shim*
-rwxr-xr-x 1 root root 40328816 Aug 3 22:08 dockerd*
-rwxr-xr-x 1 root root 0 Aug 3 22:08 docker-init*
-rwxr-xr-x 1 root root 0 Aug 3 22:08 docker-proxy*
-rwxr-xr-x 1 root root 0 Aug 3 22:08 docker-runc*
-rwxr-xr-x 1 root root 8962864 Aug 3 21:40 docker-volume-local-persist*
Installing from https://docs.docker.com/engine/installation/linux/docker-ce/binaries/
file $(which docker-runc):
/usr/bin/docker-runc: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, BuildID[sha1]=e3d80e183baf26a9d48c3f0435931d42aa1bf340, not stripped
lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu 17.04
Release: 17.04
Codename: zesty
docker --version
Docker version 17.06.0-ce, build 02c1d87
dockerd --version
Docker version 17.06.0-ce, build 02c1d87
docker-containerd --version
containerd version 0.2.3 commit: cfb82a876ecc11b5ca0977d1733adbe58599088a
docker-containerd-ctr --version
ctr version 0.2.3 commit: cfb82a876ecc11b5ca0977d1733adbe58599088a
docker-init --version
tini version 0.13.0 - git.949e6fa
docker-runc --version
runc version 1.0.0-rc3
commit: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
spec: 1.0.0-rc5
Is this running on a non-Intel/ARM 64-bit Ubuntu? For example a Raspberry Pi or ARM64 CPU? This error:
shim error: fork/exec /usr/bin/docker-runc: exec format error
Would imply that either (a) the binary install on your machine is corrupted in some way, or (b) you are attempting to run a binary for a different architecture on your system.
Can you post the output of uname -a and file /usr/bin/docker-runc? That might help narrow down the source of your problem.
It looks like something related to aufs FS, What OS Do you use ? and Did you recently updated your machine ?
Update:
For CONFIG_MEMCG_SWAP_ENABLED: missing, CONFIG_RT_GROUP_SCHED: missing and warning: /proc/config.gz does not exist, searching other paths for kernel config ...
These are missing kernel configuration and flags, Please make sure you installed linux-image-extra-$(uname -r) linux-image-extra-virtual, First make sure you stopped docker daemon sudo systemctl stop docker and install these packages (contains extra driver that support containers and aufs check docker docs https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu):
sudo apt-get install \
linux-image-extra-$(uname -r) \
linux-image-extra-virtual
And update your grub GRUB_CMDLINE_LINUX_DEFAULT add thesecgroup_enable=memory swapaccount=1 to your /etc/default/grubconfiguration file then update your grub sudo update-grub check https://github.com/moby/moby/issues/4250 && https://github.com/moby/moby/pull/4251
For aufs problems there is another solution for modern dockers as docker moved from aufs to overlay and overlay2 is to configure your machine and apply overlay but make sure you backed up your docker images and containers as this fix might make you lose them check Unable to start Docker Service in Ubuntu 16.04

Resources