I have an issue when running a docker container.
➜ bc_to_influx git:(master) ✗ docker run registry.gitlab.com/xxx/bc_to_influx:latest
standard_init_linux.go:207: exec user process caused "no such file or directory"
When I debug, I enter in the stopped container:
docker commit 0db73216baaf user/test_image
docker run -ti --entrypoint=sh user/test_image
on ls command, I can only my executable:
/bc2influx # ls -al
total 20552
drwxr-xr-x 1 root root 4096 Jun 6 10:32 .
drwxr-xr-x 1 root root 4096 Jun 6 11:53 ..
-rwxr-xr-x 1 root root 21034520 Jun 6 10:29 bc2influx
/bc2influx #
but when I try to execute, I get:
/bc2influx # ./bc2influx
sh: ./bc2influx: not found
I can vi, cat the execute, but not execute it
here is my Dockerfile
FROM alpine
WORKDIR /bc2influx/
COPY ./release/bc2influx /bc2influx/
RUN ls -al /bc2influx/
CMD [ "./bc2influx" ]
I previously build my executable with:
go build -o ./release/bc2influx -v -ldflags '-extldflags "-static"' ./...
Any idea what's going on ?
Looks like musl library issue try this build command go build -ldflags="-s -w".
Related
I don't understand why my entrypoint can't execute my command. My entrypoint look like this:
#!/bin/bash
...
exec "$#"
My script is existing I can run it when I go inside my container:
drwxrwxrwx 1 root root 512 mars 25 09:07 .
drwxrwxrwx 1 root root 512 mars 25 09:07 ..
-rwxrwxrwx 1 root root 128 mars 25 10:05 entrypoint.sh
-rwxrwxrwx 1 root root 481 mars 25 09:07 init-dev.sh
-rwxrwxrwx 1 root root 419 mars 25 10:02 migration.sh
root#0c0062fbf916:/app/scripts# pwd
/app/scripts
And when I run my container : docker run my_container "scripts/migration.sh"
I got this error:
scripts/entrypoint.sh: line 8: /app/scripts/migration.sh: No such file or directory
I have the same error if I just run ls -all
docker run my_container "ls -all"
exec: ls -all: not found
I'm switching linux to windows <-> windows to linux so I checked to change lf to crlf but there is no changes
Your first command doesn't work because your scripts are in /app/scripts (note the plural), but you're trying to run run script/migration.sh. Additionally, it's not clear what the current working directory is in your container: even if you wrote scripts/migration.sh, that would only work if either (a) your Dockerfile contains a WORKDIR /app, or if your docker run command line includes -w /app. You would be better off using a fully qualified path:
docker run mycontainer /app/scripts/migration.sh
Your second example (docker run my_container "ls -all") is over-quoted and would never work. You need to write docker run my_container ls -all, except that -all isn't actually an option that ls accepts, although it will work by virtue of being the combination of the -a and -l options.
I'm new to Docker so I might not have some of the terminology correct. Inside the container I'm getting a permission denied error on a directory shared with the host. They appear to have matching uid:gid and the permissions host side are 777. The container is not for running in the background.
I'm using the container to run a big series of untrusted programs one at a time each needing the same initial conditions. So I don't think it's feasible to copy stuff into the docker image at build time. I felt the optimal thing to do is copy the programs one at a time to a temp directory on the host and then share that directory with the fresh container for each run. I also need to collect the output from the container-run programs and keep them on the host so I can see how each program's output differs from the others.
I have looked at the following questions/answers:
Docker: Copying files from Docker container to host
How to fix docker: Got permission denied issue - successfully used to make docker run as someone other than root
How do I add a user when I'm using Alpine as a base image? and Setting up a new user - used to create the user and group
I am:
running docker as an ordinary user uid 1000, gid 1000, also belonging to the group docker
setting permissions on the shared directory host side to be 777 with uid:gid as 1000:1000 which is the same as the user
setting the uid and gid inside the container to match uid and gid from the host
using the Dockerfile to create a uid and gid each of 1000
I read here that If the first argument begins with a / or ~/, you’re creating a bindmount. Remove that, and you’re naming the volume. So I tried both. The bindmount version seems to have the correct uid:gid but is permission denied, the volume version comes out as root:root.
As a newbie it's hard to know what information to share so here's everything I think might be useful:
Docker command attempt 1
[osboxes#osboxes tmp]$ pwd
/var/tmp
osboxes#osboxes tmp]$ whoami
osboxes
[osboxes#osboxes tmp]$ grep osboxes /etc/passwd
osboxes:x:1000:1000:osboxes.org:/home/osboxes:/bin/bash
[osboxes#osboxes tmp]$ groups
osboxes wheel vboxsf docker
[osboxes#osboxes tmp]$ grep osboxes /etc/group
wheel:x:10:osboxes
osboxes:x:1000:osboxes
vboxsf:x:981:osboxes
docker:x:1001:osboxes
[osboxes#osboxes tmp]$ ls -al
total 2
drwxrwxrwt. 11 root root 4096 Dec 31 12:13 .
drwxr-xr-x. 21 root root 4096 Jul 5 05:00 ..
drwxr-xr-x. 2 abrt abrt 6 Jul 5 05:00 abrt
drwxrwxrwx. 2 osboxes osboxes 6 Dec 31 12:13 host
continues...
[osboxes#osboxes tmp]$ docker run --rm -v /var/tmp/host:/var/tmp/container:rw \
--user appuser:appgroup --workdir /var/tmp/container \
-it alpine_bash_jdk11 /bin/bash
bash-5.0$ pwd
/var/tmp/container
bash-5.0$ ls -al
ls: can't open '.': Permission denied
total 0
bash-5.0$ ls -al ..
total 0
drwxrwxrwt 1 root root 23 Dec 31 12:51 .
drwxr-xr-x 1 root root 17 Dec 16 10:31 ..
drwxrwxrwx 2 appuser appgroup 6 Dec 31 12:13 container
bash-5.0$ whoami
appuser
bash-5.0$ groups
appgroup
bash-5.0$ grep appuser /etc/passwd
appuser:x:1000:1000:Linux User,,,:/home/appuser:/sbin/nologin
bash-5.0$ grep appuser /etc/group
appgroup:x:1000:appuser
Docker command attempt 2
everything as before except
for removing the qualified path to the host's
/var/tmp/host directory
docker run --rm -v host:/var/tmp/container:rw \
--user appuser:appgroup --workdir /var/tmp/container \
-it alpine_bash_jdk11 /bin/bash
bash-5.0$ pwd
/var/tmp/container
bash-5.0$ ls -al
total 0
drwxr-xr-x 2 root root 6 Dec 31 12:13 .
drwxrwxrwt 1 root root 23 Dec 31 13:03 ..
bash-5.0$ ls -al ..
total 0
drwxrwxrwt 1 root root 23 Dec 31 13:03 .
drwxr-xr-x 1 root root 17 Dec 16 10:31 ..
drwxr-xr-x 2 root root 6 Dec 31 12:13 container
bash-5.0$ whoami
appuser
bash-5.0$ groups
appgroup
bash-5.0$ echo hello from contanier > container.msg.txt
bash: container.msg.txt: Permission denied
Docker build command
as user osboxes
docker build -t alpine_bash_jdk11 .
Dockerfile
FROM alpine:latest
RUN apk --no-cache update
RUN apk add --no-cache bash
RUN apk --no-cache add openjdk11 --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community
ENV JAVA_HOME="/usr/lib/jvm/default-jvm"
ENV PATH=$PATH:${JAVA_HOME}/bin
RUN addgroup -g 1000 -S appgroup && adduser -S appuser -G appgroup -u 1000
USER appuser
I haven't used docker compose because I'm still getting my head round basic docker.
Virtual Machine which is the Docker Host
CentOS 7.2003 from osboxes.org, organization's decision, not mine
Linux osboxes 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
I did a yum update, then yum installed all the stuff needed to install VirtualBox guest additions which is working ok
Docker version 1.13.1, build 0be3e21/1.13.1
Physical Host
Windows 10 64-bit
VirtualBox 6.1.4r136177
both these are the organization's decisions
tl;dr: had old version of docker due to wrong install command
The answer: install docker-ce instead of docker. Depending on your system that might be
sudo apt-get install -y docker-ce
or
sudo yum -y install docker-ce
instead of sudo apt-get install -y docker
or
sudo yum -y install docker
Solution: update docker
Having found this article I could see that I had the wrong version of docker. I justifiably thought the correct command was
sudo yum install -y docker
but it should have been docker-ce
I had to yum erase -y docker docker-common
Now I have Docker version 20.10.1, build 831ebea
I run R-Studio in a container on GitLab. R-Studio build a lot of csv and pdf files. When I run
docker run --rm -it registry.gitlab.com/user/paperboy /bin/bash
I can find in the folder /home/output/csv and /home/output/pdf the files. I will save all this files in a /output/csv and /output/pdf files on a host, in my case on GitLab. The question is how to save data outside the docker Container?
Here is my Dockerfile.
FROM rocker/r-base:latest
RUN apt-get update \
&& apt-get install -yq --no-install-recommends groff \
&& rm -rf /var/lib/apt/lists/*
# Create directories
RUN mkdir -p /home/output/ /home/output/csv/ /home/output/pdf/ /home/script/
WORKDIR /home/script
# Install R-packages
COPY /src/install_packages.R /home/script/install_packages.R
RUN Rscript /home/script/install_packages.R
# Copy data
COPY /src/pairs.csv /home/script/pairs.csv
COPY /src/master.R /home/script/master.R
COPY /src/paperboy.ms /home/script/paperboy.ms
# Run the script
RUN ["Rscript", "master.R"]
$ docker run -d
-v $(pwd)/output/:/home/output
-v $(pwd)/output/csv/:/home/output/csv
-v $(pwd)/output/pdf/:/home/output/pdf
$CONTAINER_IMAGE/$DOCKER_IMAGE
5d11eb7e3d93e8b98b6381f1970c25be426ff67abef5e378b715263f174849c9
This is a part from the .gitlab-ci.yml
run:
stage: run
script:
- git remote set-url origin https://$GIT_CI_USER:$GIT_CI_PASS#gitlab.com/$CI_PROJECT_PATH.git
- git config --global user.name ""
- git config --global user.email ""
- git checkout
- docker login registry.gitlab.com --username gitlab+deploy-token-aaaa --password bbbb
- docker pull $CONTAINER_IMAGE/$DOCKER_IMAGE
- docker image ls
- docker run -t -d
-v $(pwd)/output/:/home/output
-v $(pwd)/user/paperboy/output/csv/:/home/output/csv
-v $(pwd)/user/paperboy/output/pdf/:/home/output/pdf
$CONTAINER_IMAGE/$DOCKER_IMAGE
- rm -rf "%CACHE_PATH%/%CI_PIPELINE_ID%"
- pwd
- ls -la
- ls -laR output
- git status
only:
- master
The csv and pdf folder are empty.
$ ls -laR output
output:
total 32
drwxr-xr-x 4 root root 4096 Oct 8 11:37 .
drwxrwxrwx 5 root root 4096 Oct 8 11:37 ..
drwxr-xr-x 2 root root 4096 Oct 8 11:37 csv
drwxr-xr-x 2 root root 4096 Oct 8 11:37 pdf
output/csv:
total 16
drwxr-xr-x 2 root root 4096 Oct 8 11:37 .
drwxr-xr-x 4 root root 4096 Oct 8 11:37 ..
output/pdf:
total 16
drwxr-xr-x 2 root root 4096 Oct 8 11:37 .
drwxr-xr-x 4 root root 4096 Oct 8 11:37 ..
Running Docker version 17.06.0-ce, build 02c1d87, I have a dockerfile that looks like this:
FROM maven:3.5.2-jdk-8-alpine as builder
RUN chmod -R 777 /root/.m2 &&\
mkdir -p /root/.m2/repository/com/foo/bar &&\
echo "Text" > /root/.m2/repository/com/foo/bar/baz.txt &&\
ls -R -a -l /root/.m2/repository/com/foo
RUN ls -R -a -l /root/.m2/repository/com/foo
The first RUN command successfully creates a file, but the second command can't find it:
Step 1/46 : FROM maven:3.5.2-jdk-8-alpine as builder
---> 293423a981a7
Step 2/46 : RUN chmod -R 777 /root/.m2 && mkdir -p /root/.m2/repository/com/foo/bar && echo "Text" > /root/.m2/repository/com/foo/bar/baz.txt && ls -R -a -l /root/.m2/repository/com/foo
---> Running in a1c0fd142856
/root/.m2/repository/com/foo:
total 12
drwxr-xr-x 3 root root 4096 Nov 30 13:32 .
drwxr-xr-x 3 root root 4096 Nov 30 13:32 ..
drwxr-xr-x 2 root root 4096 Nov 30 13:32 bar
/root/.m2/repository/com/foo/bar:
total 12
drwxr-xr-x 2 root root 4096 Nov 30 13:32 .
drwxr-xr-x 3 root root 4096 Nov 30 13:32 ..
-rw-r--r-- 1 root root 5 Nov 30 13:32 baz.txt
---> b997ccbfd5b0
Step 3/46 : RUN ls -R -a -l /root/.m2/repository/com/foo
---> Running in 603671c87ecc
ls: /root/.m2/repository/com/foo: No such file or directory
The command '/bin/sh -c ls -R -a -l /root/.m2/repository/com/foo' returned a non-zero code: 1
What's going on? (NB. this is a toy example, but there is a real issue in that JARs installed into the Maven repository seem to disappear between layers.)
The upstream maven image defines this directory as a volume. Once an image does this, you cannot reliably make changes to that directory in the image.
From their Dockerfile:
ARG USER_HOME_DIR="/root"
...
VOLUME "$USER_HOME_DIR/.m2"
The Dockerfile documentation describes this behavior:
Changing the volume from within the Dockerfile: If any build steps change the data within the volume after it has been declared, those changes will be discarded.
Your options are to:
Use another directory for your build
Request that the upstream image removes this VOLUME definition
Build your own image without this definition (it's fairly easy to fork their repo and do your own build)
For more details, you can see an old blog post by me about this behavior and the problems it creates.
How can I control which user owns the files I copy in and out of a container?
The docker cp command says this about file ownership:
The cp command behaves like the Unix cp -a command in that directories are copied recursively with permissions preserved if possible. Ownership is set to the user and primary group at the destination. For example, files copied to a container are created with UID:GID of the root user. Files copied to the local machine are created with the UID:GID of the user which invoked the docker cp command. However, if you specify the -a option, docker cp sets the ownership to the user and primary group at the source.
It says that files copied to a container are created as the root user, but that's not what I see. I create two files owned by user id 1005 and 1006. Those owners are translated into the container's user namespace. The -a option seems to make no difference when I copy the file into a container.
$ sudo chown 1005:1005 test.txt
$ ls -l test.txt
-rw-r--r-- 1 1005 1005 29 Oct 6 12:43 test.txt
$ docker volume create sandbox1
sandbox1
$ docker run --name run1 -vsandbox1:/data alpine echo OK
OK
$ docker cp test.txt run1:/data/test1005.txt
$ docker cp -a test.txt run1:/data/test1005a.txt
$ sudo chown 1006:1006 test.txt
$ docker cp test.txt run1:/data/test1006.txt
$ docker cp -a test.txt run1:/data/test1006a.txt
$ docker run --rm -vsandbox1:/data alpine ls -l /data
total 16
-rw-r--r-- 1 1005 1005 29 Oct 6 19:43 test1005.txt
-rw-r--r-- 1 1005 1005 29 Oct 6 19:43 test1005a.txt
-rw-r--r-- 1 1006 1006 29 Oct 6 19:43 test1006.txt
-rw-r--r-- 1 1006 1006 29 Oct 6 19:43 test1006a.txt
When I copy files out of the container, they are always owned by me. Again, the -a option seems to do nothing.
$ docker run --rm -vsandbox1:/data alpine cp /data/test1006.txt /data/test1007.txt
$ docker run --rm -vsandbox1:/data alpine chown 1007:1007 /data/test1007.txt
$ docker cp run1:/data/test1006.txt .
$ docker cp run1:/data/test1007.txt .
$ docker cp -a run1:/data/test1006.txt test1006a.txt
$ docker cp -a run1:/data/test1007.txt test1007a.txt
$ ls -l test*.txt
-rw-r--r-- 1 don don 29 Oct 6 12:43 test1006a.txt
-rw-r--r-- 1 don don 29 Oct 6 12:43 test1006.txt
-rw-r--r-- 1 don don 29 Oct 6 12:47 test1007a.txt
-rw-r--r-- 1 don don 29 Oct 6 12:47 test1007.txt
-rw-r--r-- 1 1006 1006 29 Oct 6 12:43 test.txt
$
You can also change the ownership by logging in as root user into the container :
docker exec -it --user root <container-id> /bin/bash
chown -R <username>:<groupname> <folder/file>
In addition to #Don Kirkby's answer, let me provide a similar example in bash/shell script for the case that you want to copy something into a container while applying different ownership and permissions than those of the original file.
Let's create a new container from a small image that will keep running by itself:
docker run -d --name nginx nginx:alpine
Now wel'll create a new file which is owned by the current user and has default permissions:
touch foo.bar
ls -ahl foo.bar
>> -rw-rw-r-- 1 my-user my-group 0 Sep 21 16:45 foo.bar
Copying this file into the container will set ownership and group to the UID of my user and preserve the permissions:
docker cp foo.bar nginx:/foo.bar
docker exec nginx sh -c 'ls -ahl /foo.bar'
>> -rw-rw-r-- 1 4098 4098 0 Sep 21 14:45 /foo.bar
Using a little tar work-around, however, I can change the ownership and permissions that are applied inside of the container.
tar -cf - foo.bar --mode u=+r,g=-rwx,o=-rwx --owner root --group root | docker cp - nginx:/
docker exec nginx sh -c 'ls -ahl /foo.bar'
>> -r-------- 1 root root 0 Sep 21 14:45 /foo.bar
tar options explained:
c creates a new archive instead of unpacking one.
f - will write to stdout instead of a file.
foo.bar is the input file to be packed.
--mode specifies the permissions for the target. Similar to chown, they can be given in symbolic notation or as an octal number.
--owner sets the new owner of the file.
--group sets the new group of the file.
docker cp - reads the file that is to be copied into the container from stdin.
This approach is useful when a file needs to be copied into a created container before it starts, such that docker exec is not an option (which can only operate on running containers).
Just a one-liner (similar to #ramu's answer), using root to make the call:
docker exec -u 0 -it <container-id> chown node:node /home/node/myfile
In order to get complete control of file ownership, I used the tar stream feature of docker cp:
If - is specified for either the SRC_PATH or DEST_PATH, you can also stream a tar archive from STDIN or to STDOUT.
I launch the docker cp process, then stream a tar file to or from the process. As the tar entries go past, I can adjust the ownership and permissions however I like.
Here's a simple example in Python that copies all the files from /outputs in the sandbox1 container to the current directory, excludes the current directory so its permissions don't get changed, and forces all the files to have read/write permissions for the user.
from subprocess import Popen, PIPE, CalledProcessError
import tarfile
def main():
export_args = ['sudo', 'docker', 'cp', 'sandbox1:/outputs/.', '-']
exporter = Popen(export_args, stdout=PIPE)
tar_file = tarfile.open(fileobj=exporter.stdout, mode='r|')
tar_file.extractall('.', members=exclude_root(tar_file))
exporter.wait()
if exporter.returncode:
raise CalledProcessError(exporter.returncode, export_args)
def exclude_root(tarinfos):
print('\nOutputs:')
for tarinfo in tarinfos:
if tarinfo.name != '.':
assert tarinfo.name.startswith('./'), tarinfo.name
print(tarinfo.name[2:])
tarinfo.mode |= 0o600
yield tarinfo
main()