When I tried to build mxnet inside a Docker image, I got the following message :
Step 14/16 : RUN ["/bin/bash", "-c", "cd /workspace/Project/mxnet"]
---> Running in c9337d66ee5e /workspace/Project/mxnet
Removing intermediate container c9337d66ee5e
---> b92ad26e0218 Step 15/16 : RUN ["/bin/bash", "-c", "make"]
---> Running in 653a66e430b2 make:
*** No targets specified and no makefile found.
Stop. The command '/bin/bash -c make' returned a non-zero code: 2
any suggestion?
The issue at
Step 15/16 : RUN ["/bin/bash", "-c", "make"]
[…] The command '/bin/bash -c make' returned a non-zero code: 2
probably comes from the fact you run beforehand this command:
Step 14/16 : RUN ["/bin/bash", "-c", "cd /workspace/Project/mxnet"]
which is a typical bug pattern in Docker.
To sum up, each Docker RUN command is run in a subshell, so that if you do
RUN ["/bin/bash", "-c", "cd /workspace/Project/mxnet"]
RUN other commands
The other commands won't be affected by the directory change.
To solve this, you can either:
replace both lines with
RUN cd /workspace/Project/mxnet && other commands
or (solution preferred) replace both lines with
WORKDIR /workspace/Project/mxnet
RUN other commands
Solution 2 is more idiomatic, and note that
WORKDIR /dir
RUN some command
can be semantically seen as the equivalent of the command
mkdir -p /dir && cd /dir && /bin/sh -c "some command"
Related
I am using as a base, the following image: https://github.com/Kaggle/docker-python/blob/main/Dockerfile.
Here is my Dockerfile:
FROM kaggle/python
RUN ["chmod", "+x", "/opt/conda/etc/profile.d/conda.sh"]
SHELL ["/bin/bash", "--login", "-c" ]
RUN ["/opt/conda/etc/profile.d/conda.sh"]
ENTRYPOINT [ "/usr/bin/env" ]
RUN ["exec '$#'"]
RUN ["bash"]
I am running the command: docker build -t kaggle/no-jupyter .
At line 4 I get the error:
> [3/7] RUN ["/opt/conda/etc/profile.d/conda.sh"]:
#6 0.263 standard_init_linux.go:228: exec user process caused: exec format error
Here is a link to the conda.sh file: https://pastebin.com/Epu4d7Nq
I read that this might be because I am building a Linux image on Windows10...Any ideas?
See RUN Syntax:
RUN
RUN has 2 forms:
RUN (shell form, the command is run in a shell, which by default is /bin/sh -c on Linux or cmd /S /C on Windows)
RUN ["executable", "param1", "param2"] (exec form)
What you choose is exec form which won't run in a shell, so your /opt/conda/etc/profile.d/conda.sh surely fails.
To fix it you need to change to:
RUN ["bash", "/opt/conda/etc/profile.d/conda.sh"]
Or just use shell form:
RUN /opt/conda/etc/profile.d/conda.sh
Minimal example:
run.sh:
echo "hello"
Dockerfile:
FROM ubuntu:16.04
COPY run.sh /
RUN ["bash", "/run.sh"]
Execution:
$ docker build -t abc:1 . --no-cache
Sending build context to Docker daemon 3.072kB
Step 1/3 : FROM ubuntu:16.04
---> 065cf14a189c
Step 2/3 : COPY run.sh /
---> 8741ec438afd
Step 3/3 : RUN ["bash", "/run.sh"]
---> Running in 5e6754c79bc1
hello
Removing intermediate container 5e6754c79bc1
---> 3edd77959de4
Successfully built 3edd77959de4
Successfully tagged abc:1
AND, if not use bash, it will show error likes next:
Dockerfile:
FROM ubuntu:16.04
COPY run.sh /
RUN ["/run.sh"]
Execution:
$ docker build -t abc:1 . --no-cache
Sending build context to Docker daemon 3.072kB
Step 1/3 : FROM ubuntu:16.04
---> 065cf14a189c
Step 2/3 : COPY run.sh /
---> 82412a703847
Step 3/3 : RUN ["/run.sh"]
---> Running in 2a2b00f966c7
standard_init_linux.go:211: exec user process caused "exec format error"
The command '/run.sh' returned a non-zero code: 1
I'm building a docker image based on Alpine.
FROM alpine
RUN apk update \
&& apk add lighttpd \
&& rm -rf /var/cache/apk/*
ENV COLOR red
COPY ./index.html /var/www/localhost/htdocs
RUN /bin/ash -c 'echo abcd'
#working
RUN /bin/ash -c "echo $COLOR; sed -i -e 's/red/\$COLOR/g' /var/www/localhost/htdocs/index.html; cat /var/www/localhost/htdocs/index.html;"
#not working
# RUN ["sh", "-c", "echo $COLOR; sed -i -e 's/red/\$COLOR/g' /var/www/localhost/htdocs/index.html; cat /var/www/localhost/htdocs/index.html;"]
CMD ["lighttpd","-D","-f","/etc/lighttpd/lighttpd.conf"]
When I run in shell form it's working fine, but when I run in exec form it's giving
/bin/sh: [sh,: not found
I tried using bin/sh, sh, bin/ash, ash. Same error for all of them.
Shell is responsible for expanding variables, but only variable in double quotes will be expanded.
Your error comes from wrong \ before $COLOR, in fact it did no meaning for you to get the value from shell, the correct way is next:
RUN ["sh", "-c", "echo $COLOR; sed -i -e \"s/red/$COLOR/g\" /var/www/localhost/htdocs/index.html; cat /var/www/localhost/htdocs/index.html;"]
A minimal example to show the effect, FYI:
Dockerfile:
FROM alpine
ENV COLOR rednew
RUN echo "red" > /tmp/index.html
RUN ["sh", "-c", "sed -i -e \"s/red/$COLOR/g\" /tmp/index.html; cat /tmp/index.html;"]
Result:
$ docker build -t abc:1 . --no-cache
Sending build context to Docker daemon 5.632kB
Step 1/4 : FROM alpine
---> 28f6e2705743
Step 2/4 : ENV COLOR rednew
---> Running in 05c43146fab0
Removing intermediate container 05c43146fab0
---> 28ea1434e626
Step 3/4 : RUN echo "red" > /tmp/index.html
---> Running in 2c8fbbc5fd10
Removing intermediate container 2c8fbbc5fd10
---> f884892ad8c4
Step 4/4 : RUN ["sh", "-c", "sed -i -e \"s/red/$COLOR/g\" /tmp/index.html; cat /tmp/index.html;"]
---> Running in 6930b3d03438
rednew
Removing intermediate container 6930b3d03438
---> b770475672cc
Successfully built b770475672cc
Successfully tagged abc:1
I've been using Docker for a few years and I did not know (until your question) that there are shell|exec forms for RUN ;-)
The issue is that your command includes environment variables ($COLOR) and there's no substituation|evaluation with the exec form.
See:
https://docs.docker.com/engine/reference/builder/#run
"Unlike the shell form, the exec form does not invoke a command shell. This means that normal shell processing does not happen"
When I try to execute a JavaScript file with a shebang such as #!/usr/bin/env node through the command argument of docker run ... it seems to "ignore" the shebang.
$ docker run --rm foobar/hello-world /hello-world.js
/hello-world.js: line 2: syntax error near unexpected token `'Hello, World!''
/hello-world.js: line 2: `console.log('Hello, World!');'
Dockerfile
FROM node:13.12-alpine
COPY hello-world.js /hello-world.js
RUN chmod +x /hello-world.js
RUN apk update && apk update && apk add bash
ENTRYPOINT ["/bin/bash"]
hello-world.js
#!/usr/bin/env node
console.log('Hello, World!');
When I use /hello-world.js as the entrypoint directly (ENTRYPOINT ["/hello-world.js"]) it works correctly.
Add -c to the entrypoint so bash will expect a command. Without -c it interprets its argument as the name of a bash script to execute.
ENTRYPOINT ["/bin/bash", "-c"]
I'd recommend just setting the default CMD to the program you're installing in your container, and generally preferring CMD to ENTRYPOINT if you only need one of them.
FROM node:13.12-alpine
COPY hello-world.js /hello-world.js
RUN chmod +x /hello-world.js
CMD ["/hello-world.js"]
When you provide a command at the docker run command line, it overrides the Dockerfile CMD (if any), and it's appended to the ENTRYPOINT. In your original example the ENTRYPOINT from the Dockerfile is combined with the docker run command and you're getting a combined command bash /hello-world.js.
If you do need an interactive shell to debug the container, you can launch that with
docker run --rm -it foobar/hello-world /bin/sh
I have a Dockerfile which has multiple stages. An intermediate stage runs tests. If the test fails, I would like the build to stop but it continues given below. You can see output where it continues beyond failed RUN command.
# Base image
FROM python:3.5 as base
# For running tests
FROM base
WORKDIR /root
RUN mkdir /root/src
ADD src/requirements.txt /root/src
RUN pip install -r /root/src/requirements.txt
ADD . /root
RUN ["/bin/bash", "-c", "cd test; ./run-tests.sh"]
# For publishing src files
FROM base
ADD src /root/src
ADD .pypirc /root/.pypirc
WORKDIR /root/src
CMD python setup.py sdist upload -r local
Output:
Step 6/13 : RUN pip install -r /root/src/requirements.txt
---> Using cache
---> 56fa7fc2f2e8
Step 7/13 : ADD . /root
---> 74c52977edcf
Step 8/13 : RUN ["/bin/bash", "-c", "cd test; ./run-tests.sh"]
---> Running in 68a184ab54af
...
----------------------------------------------------------------------
Ran 1 test in 10.122s
FAILED (failures=1)
The command '/bin/bash -c cd test; ./run-tests.sh' returned a non-zero code: 1
running sdist
running egg_info
...
running upload
Submitting dist/<artifact-name>-2.6.1.2.tar.gz to https://<subdomain>.jfrog.io/<context>/api/pypi/python-local
...
Your command, run-tests.sh, needs to exit with a non-zero exit code and docker will stop building. In this case, that has happened:
The command '/bin/bash -c cd test; ./run-tests.sh' returned a non-zero code: 1
Whatever you run to call docker build needs to handle that exit code and stop running at that point. Docker's behavior is to give you an exit code to indicate the failure:
$ cat df.fail
FROM busybox
RUN exit 1
RUN echo still running
$ docker build -f df.fail .
Sending build context to Docker daemon 23.04kB
Step 1/3 : FROM busybox
---> 59788edf1f3e
Step 2/3 : RUN exit 1
---> Running in 70d90fb88d6e
The command '/bin/sh -c exit 1' returned a non-zero code: 1
$ echo $?
1
From the above example, you can see that docker does stop as soon as the command returns a non-zero exit code, it does not run the echo still running line, and there's a non-zero return code from docker build itself that you can handle with whatever you use to run the build.
I am new to Docker and I am trying to create an image where I install Intel MPI runtime.
I set the WORKDIR and I have the .tgz file incompressed in that folder.
But I am unable to run the install.sh
When the docker build fails, it still creates an image. I can run a container off of that image and I can manually install using the same command. But this command fails during docker build and I cannot figure out why.
-Sachin
My Dockerfile:
[root#curly sachin_docker]# cat Dockerfile
FROM centos:7
RUN useradd --create-home -s /bin/bash dumuser
ADD l_mpi-rt_2018.2.199.tgz /home/dumuser
WORKDIR /home/dumuser/l_mpi-rt_2018.2.199
RUN "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"
USER dumuser
[root#curly sachin_docker]#
Output:
> [root#curly sachin_docker]# docker build -t mympi_test .
Sending build context to Docker daemon 47.53MB
Step 1/6 : FROM centos:7
---> 2d194b392dd1
Step 2/6 : RUN useradd --create-home -s /bin/bash dumuser
---> Using cache
---> a5b06d51c8c3
Step 3/6 : ADD l_mpi-rt_2018.2.199.tgz /home/dumuser
---> 1bfd6a2744f0
Step 4/6 : WORKDIR /home/dumuser/l_mpi-rt_2018.2.199
Removing intermediate container 3e599f1454c8
---> 44e5b5eed7b8
Step 5/6 : RUN "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"
---> Running in 2e8fc1017095
/bin/sh: ./install.sh --user-mode --silent ./silent.cfg --ignore-cpu: No such file or directory
The command '/bin/sh -c "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"' returned a non-zero code: 127 [root#curly sachin_docker]#
When I run the container off that image I can manually run the install with the same command:
[root#curly sachin_docker]# docker run --rm -ti 44e5b5eed7b8 bash
[root#6dfc30bbb006 l_mpi-rt_2018.2.199]# /bin/sh -c "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"
[root#6dfc30bbb006 l_mpi-rt_2018.2.199]# ls /opt/intel bin
compilers_and_libraries_2018 documentation_2018
parallel_studio_xe_2018 samples_2018 compilers_and_libraries
compilers_and_libraries_2018.2.199 impi
parallel_studio_xe_2018.2.046 uninstall
[root#6dfc30bbb006 l_mpi-rt_2018.2.199]#
It looks like the problem is due to the fact that you have included quotes in the following directive in your Dockerfile:
RUN "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"
Because you have included quotes, this entire string is getting evaluated as the command you want to run. Basically, what's happening is the following:
/bin/bash -c "\"./install.sh --user-mode --silent ./silent.cfg --ignore-cpu\""
Similarly, if you try to run this command with quotes in an interactive shell, you'll get a similar error:
$ "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"
bash: ./install.sh --user-mode --silent ./silent.cfg --ignore-cpu: No such file or directory
Remove the quotes and it will treat ./install.sh as the command and the rest of the string as the arguments to pass in.