I am new to Docker and I am trying to create an image where I install Intel MPI runtime.
I set the WORKDIR and I have the .tgz file incompressed in that folder.
But I am unable to run the install.sh
When the docker build fails, it still creates an image. I can run a container off of that image and I can manually install using the same command. But this command fails during docker build and I cannot figure out why.
-Sachin
My Dockerfile:
[root#curly sachin_docker]# cat Dockerfile
FROM centos:7
RUN useradd --create-home -s /bin/bash dumuser
ADD l_mpi-rt_2018.2.199.tgz /home/dumuser
WORKDIR /home/dumuser/l_mpi-rt_2018.2.199
RUN "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"
USER dumuser
[root#curly sachin_docker]#
Output:
> [root#curly sachin_docker]# docker build -t mympi_test .
Sending build context to Docker daemon 47.53MB
Step 1/6 : FROM centos:7
---> 2d194b392dd1
Step 2/6 : RUN useradd --create-home -s /bin/bash dumuser
---> Using cache
---> a5b06d51c8c3
Step 3/6 : ADD l_mpi-rt_2018.2.199.tgz /home/dumuser
---> 1bfd6a2744f0
Step 4/6 : WORKDIR /home/dumuser/l_mpi-rt_2018.2.199
Removing intermediate container 3e599f1454c8
---> 44e5b5eed7b8
Step 5/6 : RUN "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"
---> Running in 2e8fc1017095
/bin/sh: ./install.sh --user-mode --silent ./silent.cfg --ignore-cpu: No such file or directory
The command '/bin/sh -c "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"' returned a non-zero code: 127 [root#curly sachin_docker]#
When I run the container off that image I can manually run the install with the same command:
[root#curly sachin_docker]# docker run --rm -ti 44e5b5eed7b8 bash
[root#6dfc30bbb006 l_mpi-rt_2018.2.199]# /bin/sh -c "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"
[root#6dfc30bbb006 l_mpi-rt_2018.2.199]# ls /opt/intel bin
compilers_and_libraries_2018 documentation_2018
parallel_studio_xe_2018 samples_2018 compilers_and_libraries
compilers_and_libraries_2018.2.199 impi
parallel_studio_xe_2018.2.046 uninstall
[root#6dfc30bbb006 l_mpi-rt_2018.2.199]#
It looks like the problem is due to the fact that you have included quotes in the following directive in your Dockerfile:
RUN "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"
Because you have included quotes, this entire string is getting evaluated as the command you want to run. Basically, what's happening is the following:
/bin/bash -c "\"./install.sh --user-mode --silent ./silent.cfg --ignore-cpu\""
Similarly, if you try to run this command with quotes in an interactive shell, you'll get a similar error:
$ "./install.sh --user-mode --silent ./silent.cfg --ignore-cpu"
bash: ./install.sh --user-mode --silent ./silent.cfg --ignore-cpu: No such file or directory
Remove the quotes and it will treat ./install.sh as the command and the rest of the string as the arguments to pass in.
Related
I tried mkdir -p it didn't work.
I have the following Dockerfile:
FROM jenkins/jenkins:2.363-jdk11
ENV PLUGIN_DIR /var/jenkins_home/plugins
RUN echo $PLUGIN_DIR
RUN mkdir -p $PLUGIN_DIR
RUN ls $PLUGIN_DIR
# WORKDIR /var/jenkins_home/plugins # Can't use this, as it changes the permission to root
# which breaks the plugin installation step
# # COPY plugins.txt /usr/share/jenkins/plugins.txt
# # RUN jenkins-plugin-cli -f /usr/share/jenkins/plugins.txt --verbose
#
#
# # disable the setup wizard as we will set up jenkins as code
# ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false
#
# ENV CASC_JENKINS_CONFIG /configs/jcasc.yaml
The build fails!
docker build -t jenkins:test.1 .
Sending build context to Docker daemon 51.2kB
Step 1/5 : FROM jenkins/jenkins:2.363-jdk11
---> 90ff7cc5bfd1
Step 2/5 : ENV PLUGIN_DIR /var/jenkins_home/plugins
---> Using cache
---> 0a158958aab0
Step 3/5 : RUN echo $PLUGIN_DIR
---> Running in ce56ef9146fc
/var/jenkins_home/plugins
Step 4/5 : RUN mkdir -p $PLUGIN_DIR
---> Using cache
---> dbc4e12b9808
Step 5/5 : RUN ls $PLUGIN_DIR
---> Running in 9a0edb027862
I need this because Jenkins deprecated old plugin installation method. The new cli installs plugins to /usr/share/jenkins/ref/plugins instead.
Also:
+$ docker run -it --rm --entrypoint /bin/bash --name jenkins jenkins:test.1
jenkins#7ad71925f638:/$ ls /var/jenkins_home/
jenkins#7ad71925f638:/$
The official Jenkins image on dockerhub declare VOLUME /var/jenkins_home, and subsequent changes to that directory (even in derived images) are discarded.
To workaround, you can execute mkdir as ENTRYPOINT.
And to verify that its working you can add an sleep to enter into the container and verify. It work !.
FROM jenkins/jenkins:2.363-jdk11
ENV PLUGIN_DIR /var/jenkins_home/plugins
RUN echo $PLUGIN_DIR
USER root
RUN echo "#!/bin/sh \n mkdir -pv $PLUGIN_DIR && sleep inf" > ./mkdir.sh
RUN chmod a+x ./mkdir.sh
USER jenkins
ENTRYPOINT [ "/bin/sh", "-c", "./mkdir.sh"]
after
docker build . -t <image_name>
docker run -d <image_name> --name <container_name>
docker exec -it <container_name> bash
and you will see your directory
Sources:
https://forums.docker.com/t/simple-mkdir-p-not-working/42179
https://hub.docker.com/_/jenkins
I'm building a docker image based on Alpine.
FROM alpine
RUN apk update \
&& apk add lighttpd \
&& rm -rf /var/cache/apk/*
ENV COLOR red
COPY ./index.html /var/www/localhost/htdocs
RUN /bin/ash -c 'echo abcd'
#working
RUN /bin/ash -c "echo $COLOR; sed -i -e 's/red/\$COLOR/g' /var/www/localhost/htdocs/index.html; cat /var/www/localhost/htdocs/index.html;"
#not working
# RUN ["sh", "-c", "echo $COLOR; sed -i -e 's/red/\$COLOR/g' /var/www/localhost/htdocs/index.html; cat /var/www/localhost/htdocs/index.html;"]
CMD ["lighttpd","-D","-f","/etc/lighttpd/lighttpd.conf"]
When I run in shell form it's working fine, but when I run in exec form it's giving
/bin/sh: [sh,: not found
I tried using bin/sh, sh, bin/ash, ash. Same error for all of them.
Shell is responsible for expanding variables, but only variable in double quotes will be expanded.
Your error comes from wrong \ before $COLOR, in fact it did no meaning for you to get the value from shell, the correct way is next:
RUN ["sh", "-c", "echo $COLOR; sed -i -e \"s/red/$COLOR/g\" /var/www/localhost/htdocs/index.html; cat /var/www/localhost/htdocs/index.html;"]
A minimal example to show the effect, FYI:
Dockerfile:
FROM alpine
ENV COLOR rednew
RUN echo "red" > /tmp/index.html
RUN ["sh", "-c", "sed -i -e \"s/red/$COLOR/g\" /tmp/index.html; cat /tmp/index.html;"]
Result:
$ docker build -t abc:1 . --no-cache
Sending build context to Docker daemon 5.632kB
Step 1/4 : FROM alpine
---> 28f6e2705743
Step 2/4 : ENV COLOR rednew
---> Running in 05c43146fab0
Removing intermediate container 05c43146fab0
---> 28ea1434e626
Step 3/4 : RUN echo "red" > /tmp/index.html
---> Running in 2c8fbbc5fd10
Removing intermediate container 2c8fbbc5fd10
---> f884892ad8c4
Step 4/4 : RUN ["sh", "-c", "sed -i -e \"s/red/$COLOR/g\" /tmp/index.html; cat /tmp/index.html;"]
---> Running in 6930b3d03438
rednew
Removing intermediate container 6930b3d03438
---> b770475672cc
Successfully built b770475672cc
Successfully tagged abc:1
I've been using Docker for a few years and I did not know (until your question) that there are shell|exec forms for RUN ;-)
The issue is that your command includes environment variables ($COLOR) and there's no substituation|evaluation with the exec form.
See:
https://docs.docker.com/engine/reference/builder/#run
"Unlike the shell form, the exec form does not invoke a command shell. This means that normal shell processing does not happen"
The command /bin/sh -c docker login xxxx.live.dynatrace.com -u XXXX -p XXXXX returned a non-zero code: 127
I am also facing this error when i am trying to execute my default docker file.
The code is here as follows:
FROM node:10
WORKDIR /usr/src
RUN apt-get -y update
RUN docker login xxxx.live.dynatrace.com -u XXXX -p XXXXXX
COPY --from=sqx97905.live.dynatrace.com/linux/oneagent-codemodules:all / /
ENV LD_PRELOAD /opt/dynatrace/oneagent/agent/lib64/liboneagentproc.so
COPY . .
RUN npm install
EXPOSE 8080
CMD [ "node", "server.js" ]
i need to login so that i can copy the dynatrace agent so i had put the login command ahead of the COPY command. It works fine when i am logged into the terminal but i need to log into the dynatrace at run time then i am facing /bin/sh : 1 docker not found
When I tried to build mxnet inside a Docker image, I got the following message :
Step 14/16 : RUN ["/bin/bash", "-c", "cd /workspace/Project/mxnet"]
---> Running in c9337d66ee5e /workspace/Project/mxnet
Removing intermediate container c9337d66ee5e
---> b92ad26e0218 Step 15/16 : RUN ["/bin/bash", "-c", "make"]
---> Running in 653a66e430b2 make:
*** No targets specified and no makefile found.
Stop. The command '/bin/bash -c make' returned a non-zero code: 2
any suggestion?
The issue at
Step 15/16 : RUN ["/bin/bash", "-c", "make"]
[…] The command '/bin/bash -c make' returned a non-zero code: 2
probably comes from the fact you run beforehand this command:
Step 14/16 : RUN ["/bin/bash", "-c", "cd /workspace/Project/mxnet"]
which is a typical bug pattern in Docker.
To sum up, each Docker RUN command is run in a subshell, so that if you do
RUN ["/bin/bash", "-c", "cd /workspace/Project/mxnet"]
RUN other commands
The other commands won't be affected by the directory change.
To solve this, you can either:
replace both lines with
RUN cd /workspace/Project/mxnet && other commands
or (solution preferred) replace both lines with
WORKDIR /workspace/Project/mxnet
RUN other commands
Solution 2 is more idiomatic, and note that
WORKDIR /dir
RUN some command
can be semantically seen as the equivalent of the command
mkdir -p /dir && cd /dir && /bin/sh -c "some command"
I am trying to mount a storage bucker inside my container during docker build. I've read other threads, here, here and understood that this may be a privileged problem. It can be solved by adding the --privileged flag in the docker run process, but I would like to get the bucket mounted right off the building stage.
Attached to the container, checked that both fuse and gcsfuse are installed. GOOGLE_APPLICATION_CREDENTIALS is set, no problem with accessing Google APIs. Here's the error I am getting.
Opening GCS connection...
Opening bucket...
Mounting file system...
daemonize.Run: readFromProcess: sub-process: mountWithArgs: mountWithConn: Mount: mount: running fusermount: exit status 1
stderr:
fusermount: fuse device not found, try 'modprobe fuse' first
Dockerfile
FROM gcr.io/google-appengine/python
.
.
.
ENV GCSFUSE_REPO=gcsfuse-jessie
RUN apt-get update && apt-get install -y ca-certificates \
&& echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" > /etc/apt/sources.list.d/gcsfuse.list \
&& curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - \
&& apt-get update && apt-get install -y gcsfuse
# Config fuse
RUN chmod a+r /etc/fuse.conf
RUN perl -i -pe 's/#user_allow_other/user_allow_other/g' /etc/fuse.conf
# Alter permission
RUN chmod a+w mount-folder
RUN gcsfuse -o --implicit-dirs bucket mount-folder
For mounting a bucker using gcsfuse, you need to run docker with --privileged flag. Now the question is, "How to build a docker image without any error in such case"?
Scenario:
When in Dockerfile you have the line:
RUN chmod +x launcher.sh
USER ubuntu
RUN mkdir db
RUN gcsfuse mybucket /home/username/mypath
EXPOSE 8000
CMD ["python3.6","manage.py","runmodwsgi"]
You see
Step 8/12 : USER ubuntu
---> Running in 0d880396010b
Removing intermediate container 0d880396010b
---> 390ed28965e1
Step 9/12 : RUN mkdir mypath
---> Running in b1f657562f24
Removing intermediate container b1f657562f24
---> 002ca425ac4c
Step 10/12 : RUN gcsfuse mybucket /home/username/mypath
---> Running in cae87fd47c1d
Using mount point: /home/username/mypath
Opening GCS connection...
Opening bucket...
Mounting file system...
daemonize.Run: readFromProcess: sub-process: mountWithArgs: mountWithConn: Mount: mount: running fusermount: exit status 1
stderr:
fusermount: fuse device not found, try 'modprobe fuse' first
The command '/bin/sh -c gcsfuse database_simrut /home/ubuntu/db' returned a non-zero code: 1
Solution:
Remove the line
RUN gcsfuse mybucket /home/username/mypath
from the Dockerfile and create a launcher script instead which has following content:
#!/bin/bash
gcsfuse mybucket /home/username/mypath
python3.6 manage.py runmodwsgi
and your new docker should look like:
RUN chown $USER:$USER $HOME
RUN chmod +x launcher.sh
USER ubuntu
RUN mkdir db
EXPOSE 8000
CMD ["./launcher.sh"]
Output:
Step 8/11 : USER ubuntu
---> Running in 83c0e73295c2
Removing intermediate container 83c0e73295c2
---> fad81733c14d
Step 9/11 : RUN mkdir mypath
---> Running in 5d83416f035e
Removing intermediate container 5d83416f035e
---> 4575ba28dc44
Step 10/11 : EXPOSE 8000
---> Running in 081dc094c046
Removing intermediate container 081dc094c046
---> 546c8edd43c3
Step 11/11 : CMD ["./launcher.sh"]
---> Running in 6780900e1b2d
Removing intermediate container 6780900e1b2d
---> 9ef9fd5af68c
Successfully built 9ef9fd5af68c