/ # which chasquid-util
/usr/local/bin/chasquid-util
/ # chasquid-util
/bin/sh: chasquid-util: not found
/ # /usr/local/bin/chasquid-util
/bin/sh: /usr/local/bin/chasquid-util: not found
/ # ls -al /usr/local/bin/
total 27432
drwxr-xr-x 1 root root 4096 Jul 26 16:18 .
drwxr-xr-x 1 root root 4096 Jul 26 16:18 ..
-rwxr-xr-x 1 root root 11721005 Jul 26 16:18 chasquid
-rwxr-xr-x 1 root root 5510494 Jul 26 16:18 chasquid-util
-rwxr-xr-x 1 root root 2910713 Jul 26 16:18 mda-lmtp
-rwxr-xr-x 1 root root 4767277 Jul 26 16:18 smtp-check
-rwxr-xr-x 1 root root 3164845 Jul 26 16:18 spf-check
/ #
Given your context, this typically means you are missing a shared library. With alpine, it's typically glibc since they ship with libmusl. You can check this with:
ldd chasquid-util
I've got several other reasons for this listed in my DC 2018 slidedeck:
Did you run the intended command? (e.g. docker run --rm my_image -it echo hello world will run the command -it)
Is docker trying to run a json string? (any json paring errors will show up as executing the json as a string)
Does the file exist... in the path and inside the container? (can't run stuff from the host inside a container without building it into the image or mounting a volume)
If it is a shell script, check the first line (e.g. #!/bin/bash)
Check for windows linefeeds on linux shell scripts (look for ^M or \r with different editors)
If it is a binary, there is likely a missing library (use ldd to check)
Related
I have a Docker image which contains an analysis pipeline. To run this pipeline, I need to provide input data and I want to keep the outputs. This pipeline must be able to be run by other users than myself, on their own laptops.
Briefly, my root (/) folder structure is as follows:
total 72
drwxr-xr-x 1 root root 4096 May 29 15:38 bin
drwxr-xr-x 2 root root 4096 Feb 1 17:09 boot
drwxr-xr-x 5 root root 360 Jun 1 15:31 dev
drwxr-xr-x 1 root root 4096 Jun 1 15:31 etc
drwxr-xr-x 2 root root 4096 Feb 1 17:09 home
drwxr-xr-x 1 root root 4096 May 29 15:49 lib
drwxr-xr-x 2 root root 4096 Feb 24 00:00 lib64
drwxr-xr-x 2 root root 4096 Feb 24 00:00 media
drwxr-xr-x 2 root root 4096 Feb 24 00:00 mnt
drwxr-xr-x 1 root root 4096 Mar 12 19:38 opt
drwxr-xr-x 1 root root 4096 Jun 1 15:24 pipeline
dr-xr-xr-x 615 root root 0 Jun 1 15:31 proc
drwx------ 1 root root 4096 Mar 12 19:38 root
drwxr-xr-x 3 root root 4096 Feb 24 00:00 run
drwxr-xr-x 1 root root 4096 May 29 15:38 sbin
drwxr-xr-x 2 root root 4096 Feb 24 00:00 srv
dr-xr-xr-x 13 root root 0 Apr 29 10:14 sys
drwxrwxrwt 1 root root 4096 Jun 1 15:25 tmp
drwxr-xr-x 1 root root 4096 Feb 24 00:00 usr
drwxr-xr-x 1 root root 4096 Feb 24 00:00 var
The pipeline scripts are in /pipeline and are packaged into the image with a "COPY. /pipeline" instruction in my Dockerfile.
For various reasons, this pipeline (which is a legacy pipeline) is set up so that the input data must be in a folder such /pipeline/project. To run my pipeline, I use:
docker run --rm --mount type=bind,source=$(pwd),target=/pipeline/project --user "$(id -u):$(id -g)" pipelineimage:v1
In other words, I mount a folder with the data to /pipeline/project. I found I needed to use the --user to insure the output files would have the correct permissions - i.e. I would have read/write/exec access on my host computer after the container exits.
The pipeline runs but I have one issue: one particular software used by the pipeline automatically tries to produce (and I can't change that) 1 folder in $HOME (so / - which I showed above) and 1 folder in my WORKDIR (which I have set up in my Dockerfile to be /pipeline). These attempts fails, and I'm guessing it's because I am not running the pipeline as root. But I need to use --user to make sure my outputs have the correct permissions - i.e. that I don't require sudo rights to read these outputs etc.
My question is: how am I meant to handle this? It seems that by using --user, I have the correct permissions set for the mounted folder (/pipeline/projects) where many output files are successfully made, no problems there. But how can I ensure the other 2 folders are correctly made outside of that mount?
I have tried the following but not success:
Doing "COPY -chown myhostuid:mygroupid" . pipeline/". This works but I have to hardcode my uid and gid so that won't work if another colleague tries to run the image.
Adding a new user with sudo rights and making it run the image: "RUN useradd -r newuser -g sudo" (I also tried using the "root" group but no success). This just gives me outputs which require sudo rights to read/write/exec. Which is not what I want.
Am I missing something? I don't understand why it's "easy" to handle permissions for a mounted folder but so much harder for the other folders in a container. Thanks.
If your software doesn't rely on relative paths (~/, ./), you can just set $HOME and WORKDIR to a directory that any user can write:
ENV HOME=/tmp
WORKDIR /tmp
If you can't do that, you can pass the uid/gid via the environment to an entrypoint script running as root, chown/chmod as necessary, then drop privileges to run the pipeline (runuser, su, sudo, setuidgid).
For example (untested):
entrypoint.sh
#!/bin/bash
[[ -v "RUN_UID" ]] || { echo "unset RUN_UID" >&2; exit 1; }
[[ -v "RUN_GID" ]] || { echo "unset RUN_GID" >&2; exit 1; }
# chown, chmod, set env, etc.
chown $RUN_UID:$RUN_GID "/path/that/requires/write/permissions"
export HOME=/tmp
# Run the pipeline as a non-root user.
sudo -E -u "#$RUN_UID" -g "#$RUN_GID" /path/to/pipeline
Dockerfile
...
COPY entrypoint.sh /usr/local/bin/entrypoint.sh
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
Finally, pass the user and group IDs via the environment when running:
docker run --rm --mount type=bind,source=$(pwd),target=/pipeline/project -e RUN_UID=$(id -u) -e RUN_GID=$(id -g) pipelineimage:v1
The relevant documentation
https://docs.docker.com/storage/bind-mounts/
says
For some development applications, the container needs to write into
the bind mount, so changes are propagated back to the Docker host. At
other times, the container only needs read access.
This example modifies the one above but mounts the directory as a
read-only bind mount, by adding ro to the (empty by default) list of
options, after the mount point within the container. Where multiple
options are present, separate them by commas.
I expect that to mean that there is no way to write to the folder that is mounted in that way from within the container. But if I minimally modify the example to give me a shell session and mount the root filesystem
~ $ docker run \
-it \
--name devtest2 \
--mount type=bind,source=/,target=/app,readonly \
ubuntu:latest
I see that I have write access as root from to the entirety of the host filesystem from within the container.
root#bde1f19c1de2:/# cd /app/home/
# Creates directory in the host /home folder
root#bde1f19c1de2:/app/home# mkdir patata
What does then mean that the mount is "readonly".
How do I make it actually read-only?
I observe this behavior with docker 17.05 as it comes with Ubuntu trusty:
$ docker --version
Docker version 17.05.0-ce, build 89658be
I don't know how you can use --mount option as it is only available for standalone container from Docker 17.06 and yours is 17.05.
Ref
Originally, the -v or --volume flag was used for standalone containers
and the --mount flag was used for swarm services. However, starting
with Docker 17.06, you can also use --mount with standalone
containers. In general, --mount is more explicit and verbose. The
biggest difference is that the -v syntax combines all the options
together in one field, while the --mount syntax separates them. Here
is a comparison of the syntax for each flag.
That said i tried it out on docker 17.09 and still saw the same result as your decribed. Only to realise that the "readonly" option is working but your linux permissions are as such which is allowing anyone to write on it!
Since you are mounting / and writing to home directory which has 0755 permission by default
0755 means public (anyone) read and execute. The execute is allowing you to execute the mkdir command
If you mount paths or folder which doesn't have public access then you will see the readonly option works irrective of it being a root user or not inside the container, which is you wont be allowed to write!
example I am mounting home directory which has a 0770 which is public doesn't have any access!
[root#jakku-admin-1 ~]# pwd
/root
[root#jakku-admin-1 ~]# ll
total 8
drwxr-xr-x. 2 root root 4096 Feb 7 21:19 archive
drwxrwx---. 2 root root 4096 Feb 7 20:38 home
[root#jakku-admin-1 ~]# docker run -it --name devtest --mount type=bind,source=`pwd`/home,target=/app,readonly ubuntu:latest
root#3ce55bba8904:/# ll
total 16
drwxr-xr-x. 22 root root 253 Feb 7 21:20 ./
drwxr-xr-x. 22 root root 253 Feb 7 21:20 ../
-rwxr-xr-x. 1 root root 0 Feb 7 21:20 .dockerenv*
drwxrwx---. 2 root root 4096 Feb 7 20:38 app/
drwxr-xr-x. 2 root root 4096 Jan 12 21:10 bin/
drwxr-xr-x. 2 root root 6 Apr 24 2018 boot/
drwxr-xr-x. 5 root root 360 Feb 7 21:20 dev/
drwxr-xr-x. 29 root root 4096 Feb 7 21:20 etc/
drwxr-xr-x. 2 root root 6 Apr 24 2018 home/
drwxr-xr-x. 8 root root 96 May 23 2017 lib/
drwxr-xr-x. 2 root root 34 Jan 12 21:10 lib64/
drwxr-xr-x. 2 root root 6 Jan 12 21:09 media/
drwxr-xr-x. 2 root root 6 Jan 12 21:09 mnt/
drwxr-xr-x. 2 root root 6 Jan 12 21:09 opt/
dr-xr-xr-x. 592 root root 0 Feb 7 21:20 proc/
drwx------. 2 root root 37 Jan 12 21:10 root/
drwxr-xr-x. 5 root root 58 Jan 16 01:20 run/
drwxr-xr-x. 2 root root 4096 Jan 16 01:20 sbin/
drwxr-xr-x. 2 root root 6 Jan 12 21:09 srv/
dr-xr-xr-x. 13 root root 0 Jan 29 22:42 sys/
drwxrwxrwt. 2 root root 6 Jan 12 21:10 tmp/
drwxr-xr-x. 10 root root 105 Jan 12 21:09 usr/
drwxr-xr-x. 11 root root 139 Jan 12 21:10 var/
root#3ce55bba8904:/# cd app/
root#3ce55bba8904:/app# ll
total 4
drwxrwx---. 2 root root 4096 Feb 7 20:38 ./
drwxr-xr-x. 22 root root 253 Feb 7 21:20 ../
root#3ce55bba8904:/app# mkdir test
mkdir: cannot create directory 'test': Read-only file system
root#3ce55bba8904:/app# touch test
touch: cannot touch 'test': Read-only file system
I am trying to learn docker. So i am trying to create an archlinux image. Presently i am not worried of size. But i am stuck up how to further go ahead to use this as my development for a project. My goal is to create and use different archlinux images for my different projects separately.
1) shift to root in terminal
2) mkdir archlinux
3) pacstrap -i -c -d ./archlinux base
4) echo 'en_US.UTF-8 UTF-8' > ./archlinux/etc/locale.gen
5) arch-chroot ./archlinux locale-gen
6) echo 'LANG=en_US.UTF-8' > ./archlinux/etc/locale.conf
Now the total size of the folder archlinux is 899 MB.
Now i am trying to import it as an docker image
cd archlinux
tar -c . | docker import - example_archlinux
tar: ./etc/pacman.d/gnupg/S.gpg-agent: socket ignored
tar: ./etc/pacman.d/gnupg/S.gpg-agent.extra: socket ignored
tar: ./etc/pacman.d/gnupg/S.gpg-agent.ssh: socket ignored
tar: ./etc/pacman.d/gnupg/S.scdaemon: socket ignored
tar: ./etc/pacman.d/gnupg/S.gpg-agent.browser: socket ignored
sha256:2b3ed6536389a1184f402ff5a9d20380a3f4aa2c49bdee31df9c7c10186eb889
Now I run the docker image
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
example_archlinux latest 2b3ed6536389 About a minute ago 881MB
Now i try to run the image:
# docker run -ti example_archlinux:latest /bin/bash
[root#3863ba31186b /]#
# docker run -ti example_archlinux:latest ls -al
total 52
drwxr-xr-x 1 root root 4096 Oct 16 08:32 .
drwxr-xr-x 1 root root 4096 Oct 16 08:32 ..
-rwxr-xr-x 1 root root 0 Oct 16 08:32 .dockerenv
lrwxrwxrwx 1 root root 7 Jan 5 2018 bin -> usr/bin
drwxr-xr-x 2 root root 4096 Oct 16 08:01 boot
drwxr-xr-x 5 root root 360 Oct 16 08:32 dev
drwxr-xr-x 1 root root 4096 Oct 16 08:32 etc
drwxr-xr-x 2 root root 4096 Jan 5 2018 home
lrwxrwxrwx 1 root root 7 Jan 5 2018 lib -> usr/lib
lrwxrwxrwx 1 root root 7 Jan 5 2018 lib64 -> usr/lib
drwxr-xr-x 2 root root 4096 Jan 5 2018 mnt
drwxr-xr-x 2 root root 4096 Jan 5 2018 opt
dr-xr-xr-x 275 root root 0 Oct 16 08:32 proc
drwxr-x--- 3 root root 4096 Oct 16 08:01 root
drwxr-xr-x 2 root root 4096 Oct 16 08:01 run
lrwxrwxrwx 1 root root 7 Jan 5 2018 sbin -> usr/bin
drwxr-xr-x 4 root root 4096 Oct 16 08:01 srv
dr-xr-xr-x 13 root root 0 Oct 16 08:32 sys
drwxrwxrwt 2 root root 4096 Oct 16 08:01 tmp
drwxr-xr-x 8 root root 4096 Oct 16 08:10 usr
drwxr-xr-x 12 root root 4096 Oct 16 08:01 var
Its great. Its working
Q1 : Will docker not ask for login and password of root, assuming i have set root passwd
I want to create my Django + ngingx + postgresql + redis + git. I will install and setup the required packages.
.
So i am testing whether run command will save the folders craeted
# docker run -ti example_archlinux:latest /bin/bash
[root#9f4e56ce38c5 /]# mkdir hare
[root#9f4e56ce38c5 /]# exit
# docker run -ti example_archlinux:latest ls /hare
ls: cannot access '/hare': No such file or directory
I have the main question:
Q2 Since i created a folder and if i exit its not there anymore.
Now what is the best way to use a docker image for my development.
I cant afford that my files are not there after i exit.
So is there any way that the container is permanently created and i can work in it for my development.
OR
Where to create my source code on host or docker. I want everything at one place.
Q1: I never tried setting the root password. But usually, when running the container, you'll be logged in as root except if you use the USER Dockerfile command, which is the more secure approach. More about it here
Q2: Everytime you remove your container, everything inside of it will be destroyed. So, you'll lose the files you've created, unless you bound a volume to your host. Volumes are the standard way to go. You can define a volume, for instance, on your docker run command:
docker run -ti -v /host/source/folder:/desired/guest/folder example_archlinux:latest ls -al
Now you can add/remove/change files both from container or host and it will be persisted. There wont be duplicated files. It's just that both have access to it.
more details here
I am trying to copy file from docker to host using the below command,
docker cp <container_name>:<file FQN> ./
But getting the below error,
Error response from daemon: not a directory
As verified, the file name and container name are valid.
Note: Using Docker in Mac
Thanks for all the answers. After a bit of struggle found out that the error message was not actually directly related to the docker cp command.
The scenario was, I ran the docker with the link to a local file. When the docker was running I deleted it. Then the file got created as a folder somehow (Probably, when I restarted the docker).
And whenever I am executing some command, the docker was giving me that error. Then once I created the file the error disappeared.
It seems your command is correct. You please try like the below from your local machine not from inside the container. sometimes unfortunately if we run this command with in the container we will get this kind of errors.
docker cp [container_name]:[docker dir abs path] [host dir path]
Hope it will help you.
Here is a full example on how to copy a file:
$ docker run -it ubuntu /bin/bash
root#9fc8a1af7f23:/#
root#9fc8a1af7f23:/# ll
total 72
drwxr-xr-x 34 root root 4096 Jul 13 21:51 ./
drwxr-xr-x 34 root root 4096 Jul 13 21:51 ../
-rwxr-xr-x 1 root root 0 Jul 13 21:51 .dockerenv*
drwxr-xr-x 2 root root 4096 Feb 14 23:29 bin/
drwxr-xr-x 2 root root 4096 Apr 12 2016 boot/
drwxr-xr-x 5 root root 360 Jul 13 21:51 dev/
drwxr-xr-x 45 root root 4096 Jul 13 21:51 etc/
drwxr-xr-x 2 root root 4096 Apr 12 2016 home/
drwxr-xr-x 8 root root 4096 Sep 13 2015 lib/
drwxr-xr-x 2 root root 4096 Feb 14 23:29 lib64/
drwxr-xr-x 2 root root 4096 Feb 14 23:28 media/
drwxr-xr-x 2 root root 4096 Feb 14 23:28 mnt/
drwxr-xr-x 2 root root 4096 Feb 14 23:28 opt/
dr-xr-xr-x 288 root root 0 Jul 13 21:51 proc/
drwx------ 2 root root 4096 Feb 14 23:29 root/
drwxr-xr-x 6 root root 4096 Feb 27 19:41 run/
drwxr-xr-x 2 root root 4096 Feb 27 19:41 sbin/
drwxr-xr-x 2 root root 4096 Feb 14 23:28 srv/
dr-xr-xr-x 13 root root 0 Jul 13 21:51 sys/
drwxrwxrwt 2 root root 4096 Feb 14 23:29 tmp/
drwxr-xr-x 11 root root 4096 Feb 27 19:41 usr/
drwxr-xr-x 13 root root 4096 Feb 27 19:41 var/
root#9fc8a1af7f23:/# cd tmp/
root#9fc8a1af7f23:/tmp# ls
root#9fc8a1af7f23:/tmp# echo "hello docker" > docker_test.txt
root#9fc8a1af7f23:/tmp# cat docker_test.txt
hello docker
root#9fc8a1af7f23:/tmp#
Then, in another terminal
dali#dali-X550JK:~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9fc8a1af7f23 ubuntu "/bin/bash" 2 minutes ago Up 2 minutes fervent_hodgkin
dali#dali-X550JK:~$ docker cp fervent_hodgkin:/tmp/docker_test.txt /tmp/
dali#dali-X550JK:~$ cat /tmp/docker_test.txt
hello docker
dali#dali-X550JK:~$
Please follow these instruction, make sure your don't have typo in the file paths, otherwise share a reproducible error.
This error also appears when trying to copy a file that is actually a volume in the container, but the file has been deleted on the host.
This is simply an error in the path you want to copy.
You may not believe it, but that it is.
I have a docker image https://github.com/carnellj/spmia-chapter1 which does not find its CMD ./run.sh executable although it is there in the file system.
I was able to run /bin/sh in the container, and I can ls -l:
D:\Dokumente\ws\spring-microservices\spmia-chapter1 (master)
λ docker run -i -t johncarnell/tmx-simple-service:chapter1 /bin/sh
/ # ls -l
total 56
drwxr-xr-x 2 root root 4096 Mar 3 11:20 bin
drwxr-xr-x 5 root root 360 Apr 22 07:10 dev
drwxr-xr-x 1 root root 4096 Apr 22 07:10 etc
drwxr-xr-x 2 root root 4096 Mar 3 11:20 home
drwxr-xr-x 1 root root 4096 Apr 22 06:01 lib
drwxr-xr-x 5 root root 4096 Mar 3 11:20 media
drwxr-xr-x 2 root root 4096 Mar 3 11:20 mnt
dr-xr-xr-x 123 root root 0 Apr 22 07:10 proc
drwx------ 1 root root 4096 Apr 22 07:10 root
drwxr-xr-x 2 root root 4096 Mar 3 11:20 run
-rwxr-xr-x 1 root root 245 Apr 22 06:50 run.sh
drwxr-xr-x 2 root root 4096 Mar 3 11:20 sbin
drwxr-xr-x 2 root root 4096 Mar 3 11:20 srv
dr-xr-xr-x 13 root root 0 Apr 22 07:10 sys
drwxrwxrwt 2 root root 4096 Mar 3 11:20 tmp
drwxr-xr-x 1 root root 4096 Mar 7 01:04 usr
drwxr-xr-x 1 root root 4096 Mar 7 01:04 var
/ # ./run.sh
/bin/sh: ./run.sh: not found
/ # ls run.sh
run.sh
/bin/sh does not find ./run.sh although it is there in the file system, as proven by ls run.sh. Also, cat shows the content of run.sh:
/ # cat run.sh
#!/bin/sh
echo "********************************************************"
echo "Starting simple-service "
echo "********************************************************"
java -jar /usr/local/simple-service/simple-service-0.0.1-SNAPSHOT.jar
When I run vi from sh and copy the content of run.sh into a new file myrun.sh and make myrun.sh executable, I can execute ./myrun.sh and the spring service starts.
What is going on here? Why would sh not see an executable which is there in the filesystem? Executables from PATH or executables which I add manually run fine.
I am running Docker on Windows 10.
OK the reason is, run.sh is created with Windows line endings in the docker image if you check out with automatic lf->crlf conversion. One possible solution is to tell git not to convert line endings.