gurus.
I encountered the following error while building nvidia-docker of CUDA v8.0 on ppc64le.
root#minsky:/data/nvidia-docker# make cuda OS=ubuntu-16.04
make -C /data/nvidia-docker/ubuntu-16.04/cuda
make[1]: Entering directory '/data/nvidia-docker/ubuntu-16.04/cuda'
docker build -f /data/nvidia-docker/ubuntu-16.04/cuda/8.0/runtime/Dockerfile.ppc64le -t cuda:8.0-runtime /data//nvidia-docker/ubuntu-16.04/cuda/8.0/runtime
Sending build context to Docker daemon 10.24 kB
...
Step 6 : RUN CUDNN_DOWNLOAD_SUM=51f698d468401cef2e3e2ef9bb557bd57cbeb4dca895d1d1ae8a751d090bbe39 && curl -fsSL http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-ppc64le-v5.1.tgz -O && echo "$CUDNN_DOWNLOAD_SUM cudnn-8.0-linux-ppc64le-v5.1.tgz" | sha256sum -c --strict - && tar -xzf cudnn-8.0-linux-ppc64le-v5.1.tgz -C /usr/local --wildcards 'cuda/targets/ppc64le-linux/lib/libcudnn.so.*' && ln -s cuda/targets/ppc64le-linux/lib /usr/local/cuda/lib64 && rm cudnn-8.0-linux-ppc64le-v5.1.tgz && ldconfig
---> Running in 19a5bb5c9c19
cudnn-8.0-linux-ppc64le-v5.1.tgz: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match
The command '/bin/sh -c CUDNN_DOWNLOAD_SUM=51f698d468401cef2e3e2ef9bb557bd57cbeb4dca895d1d1ae8a751d090bbe39 && curl -fsSL http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-ppc64le-v5.1.tgz -O && echo "$CUDNN_DOWNLOAD_SUM cudnn-8.0-linux-ppc64le-v5.1.tgz" | sha256sum -c --strict - && tar -xzf cudnn-8.0-linux-ppc64le-v5.1.tgz -C /usr/local --wildcards 'cuda/targets/ppc64le-linux/lib/libcudnn.so.*' && ln -s cuda/targets/ppc64le-linux/lib /usr/local/cuda/lib64 && rm cudnn-8.0-linux-ppc64le-v5.1.tgz && ldconfig' returned a non-zero code: 1
../../mk/cuda.mk:87: recipe for target '8.0-cudnn5-runtime' failed
make[1]: *** [8.0-cudnn5-runtime] Error 1
make[1]: Leaving directory '/data/fuzhiwen/nvidia-docker/ubuntu-16.04/cuda'
Makefile:37: recipe for target 'cuda' failed
make: *** [cuda] Error 2
Any idea on how to get around this gracefully ?
I solved this situation by editing the Dockerfile.ppc64le files as below.
root#minsky:/data/nvidia-docker# vi ./ubuntu-16.04/cuda/8.0/runtime/cudnn5/Dockerfile.ppc64le
...
RUN CUDNN_DOWNLOAD_SUM=663aac0328f821d90ae9c74ee43e90751706546c2ce769ea9c96f92864300af6 && \
#RUN CUDNN_DOWNLOAD_SUM=51f698d468401cef2e3e2ef9bb557bd57cbeb4dca895d1d1ae8a751d090bbe39 && \
root#minsky:/data/nvidia-docker# vi ./ubuntu-16.04/cuda/8.0/devel/cudnn5/Dockerfile.ppc64le
...
RUN CUDNN_DOWNLOAD_SUM=663aac0328f821d90ae9c74ee43e90751706546c2ce769ea9c96f92864300af6 && \
#RUN CUDNN_DOWNLOAD_SUM=51f698d468401cef2e3e2ef9bb557bd57cbeb4dca895d1d1ae8a751d090bbe39 && \
root#minsky:/data/nvidia-docker# make cuda OS=ubuntu-16.04
I got the checksum value of 663aac0328f821d90ae9c74ee43e90751706546c2ce769ea9c96f92864300af6 from the following test.
root#minsky:/data/nvidia-docker# sha256sum cudnn-8.0-linux-ppc64le-v5.1.tgz
663aac0328f821d90ae9c74ee43e90751706546c2ce769ea9c96f92864300af6 cudnn-8.0-linux-ppc64le-v5.1.tgz
This works fine. Still, I am not sure if this is the correct solution and why this error happens in the first place.
Related
I just make multarch images by using buildx
When running docker buildx build --platform linux/amd64,linux/arm64 -t maskertim/zookeeper-demo .
Error Message
The Error as following:
------
> [linux/arm64 3/5] RUN curl "https://archive.apache.org/dist/kafka/2.8.0/kafka_2.13-2.8.0.tgz" -o /tmp/kafka/kafka.tgz && mkdir /kafka && cd /kafka && tar -xzvf /tmp/kafka/kafka.tgz --strip 1:
#8 0.915 % Total % Received % Xferd Average Speed Time Time Time Current
#8 0.916 Dload Upload Total Spent Left Speed
100 68.0M 100 68.0M 0 0 936k 0 0:01:14 0:01:14 --:--:-- 3475k
#8 75.45 tar: Child returned status 1
#8 75.45 tar: Error is not recoverable: exiting now
------
dockerfile:9
--------------------
8 |
9 | >>> RUN curl "https://archive.apache.org/dist/kafka/${KAFKA_VERSION}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz" \
10 | >>> -o /tmp/kafka/kafka.tgz && \
11 | >>> mkdir /kafka && cd /kafka && \
12 | >>> tar -xzvf /tmp/kafka/kafka.tgz --strip 1
13 |
--------------------
error: failed to solve: process "/dev/.buildkit_qemu_emulator /bin/sh -c curl \"https://archive.apache.org/dist/kafka/${KAFKA_VERSION}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz\" \t-o /tmp/kafka/kafka.tgz && \tmkdir /kafka && cd /kafka && \ttar -xzvf /tmp/kafka/kafka.tgz --strip 1" did not complete successfully: exit code: 2
Dockerfile
And this is my dockerfile:
FROM openjdk:11.0.12-jre-buster
ENV KAFKA_VERSION 2.8.0
ENV SCALA_VERSION 2.13
RUN mkdir /tmp/kafka && \
apt-get update && \
apt-get install -y curl
RUN curl "https://archive.apache.org/dist/kafka/${KAFKA_VERSION}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz" \
-o /tmp/kafka/kafka.tgz && \
mkdir /kafka && cd /kafka && \
tar -xzvf /tmp/kafka/kafka.tgz --strip 1
COPY start-zookeeper.sh /usr/bin
RUN chmod +x /usr/bin/start-zookeeper.sh
CMD ["start-zookeeper.sh"]
What happens? I have no idea why to occur this error and no way to solve it.
Unzip it first:
gzip -d /tmp/kafka/kafka.tgz
And after that untar it with:
tar xpf /tmp/kafka/kafka.tar
I had the same problem and discovered https://stackoverflow.com/a/71143883
Running
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
allowed me to keep the decompression step in the tar command
I think the problem is related to tar or more specific one of its child processes is failing to untar or unzip because of a missing dependency, which seems not to be present in the qemu_emulator that buildx uses.
I ended up using the ADD command in the dockerfile instead of RUN tar, which has that functionality built in.
I try to compile docker images from alpine:3.14.
The error is as follows
make -f objs/Makefile
make: make: Operation not permitted
make: *** [Makefile:10: build] Error 127
Then I switched to version alpine:3.12, alpine:3.13 and found that both are OK!
The following is my key problematic code for compiling NGINX based on alpine:3.14 version
# Omit irrelevant code
RUN \
addgroup -S www && adduser www -D -S -s /bin/sh -G www \
&& wget -P /home/soft https://github.com/vozlt/nginx-module-vts/archive/v0.1.18.tar.gz \
&& wget -P /home/soft http://nginx.org/download/nginx-1.21.1.tar.gz \
&& wget -P /home/soft https://ftp.pcre.org/pub/pcre/pcre-8.44.tar.gz \
&& cd /home/soft && tar -zxf nginx-1.21.1.tar.gz && tar -zxf v0.1.18.tar.gz && tar -zxf pcre-8.44.tar.gz \
&& cd /home/soft/nginx-1.21.1 \
&& ./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --with-http_v2_module --with-http_gzip_static_module --with-stream --with-http_sub_module --with-pcre=/home/soft/pcre-8.44 --add-module=/home/soft/nginx-module-vts-0.1.18 \
&& make && make install \
# Omit irrelevant code
Error 127 seven could mean two things. Either the the binary for the command is not found in PATH variable (Please take a note here, sometimes it can also happen that the command is found but some library is missing) or the binary doesn't have the execution permissions. By the look of it, I suspect it's because it doesn't have the execution permissions.
I would suggest you to run the alpine:3.14 container and inspect the PATH variable, the make binary location and the permissions.
I am trying to build Jupyter notebook in docker following the guide here:
https://github.com/cordon-thiago/airflow-spark
and got an error with exit code: 8.
I ran:
$ docker build --rm --force-rm -t jupyter/pyspark-notebook:3.0.1 .
the building stops at the code:
RUN wget -q $(wget -qO- https://www.apache.org/dyn/closer.lua/spark/spark-${APACHE_SPARK_VERSION}/spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz\?as_json | \
python -c "import sys, json; content=json.load(sys.stdin); print(content['preferred']+content['path_info'])") && \
echo "${spark_checksum} *spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz" | sha512sum -c - && \
tar xzf "spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz" -C /usr/local --owner root --group root --no-same-owner && \
rm "spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz"
with error message like below:
=> ERROR [4/9] RUN wget -q $(wget -qO- https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz?as_json | python -c "import sys, json; content=json.load(sys.stdin); 2.3s
------
> [4/9] RUN wget -q $(wget -qO- https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz?as_json | python -c "import sys, json; content=json.load(sys.stdin); print(content[
'preferred']+content['path_info'])") && echo "F4A10BAEC5B8FF1841F10651CAC2C4AA39C162D3029CA180A9749149E6060805B5B5DDF9287B4AA321434810172F8CC0534943AC005531BB48B6622FBE228DDC *spark-3.0.1-bin-hadoop2.7.
tgz" | sha512sum -c - && tar xzf "spark-3.0.1-bin-hadoop2.7.tgz" -C /usr/local --owner root --group root --no-same-owner && rm "spark-3.0.1-bin-hadoop2.7.tgz":
------
executor failed running [/bin/bash -o pipefail -c wget -q $(wget -qO- https://www.apache.org/dyn/closer.lua/spark/spark-${APACHE_SPARK_VERSION}/spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz\
?as_json | python -c "import sys, json; content=json.load(sys.stdin); print(content['preferred']+content['path_info'])") && echo "${spark_checksum} *spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_
VERSION}.tgz" | sha512sum -c - && tar xzf "spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz" -C /usr/local --owner root --group root --no-same-owner && rm "spark-${APACHE_SPARK_VERSION}
-bin-hadoop${HADOOP_VERSION}.tgz"]: exit code: 8
Really appreciate if someone can enlighten me on this. Thanks!
The exit code 8 is likely from wget meaning an error response from the server. As an example, this path that the Dockerfile tries to wget from isn't valid anymore: https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz
From the issues on the repo, it appears that Apache version 3.0.1 is no longer valid so you should override the APACHE_SPARK version to 3.0.2 with a --build-arg:
docker build --rm --force-rm \
--build-arg spark_version=3.0.2 \
-t jupyter/pyspark-notebook:3.0.2 .
EDIT
See comment below for more, the command that worked was:
docker build --rm --force-rm \
--build-arg spark_version=3.1.1 \
--build-arg hadoop_version=2.7 \
-t jupyter/pyspark-notebook:3.1.1 .
And updated spark checksum to reflect the version for 3.1.1: https://downloads.apache.org/spark/spark-3.1.1/spark-3.1.1-bin-hadoop2.7.tgz.sha512
For this answer to be relevant in the future, it will likely need to update versions and checksum again for the latest spark/hadoop versions.
i am writing docker file to install java and sonar. everything works great till java installation. but when downloading of sonar starts using curl , it fails. more frustrating is , exact same command works outside docker file, same shell prompt and on the same network.
I am really confused how can this be happen. what silly mistake i am doing not getting.
please see the dockefile :
FROM centos:7
ENV JAVA_VERSION 8u201
ENV BUILD_VERSION b09
RUN yum -y install wget; wget --no-cookies --no-check-certificate --header "Cookie: oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/$JAVA_VERSION-$BUILD_VERSION/42970487e3af4f5aa5bca3f542482c60/jdk-$JAVA_VERSION-linux-x64.rpm" -O /tmp/jdk-8-linux-x64.rpm; yum -y install /tmp/jdk-8-linux-x64.rpm
# JDK stripping
RUN rm -f /usr/java/jdk1.8.0_201/src.zip /usr/java/jdk1.8.0_201/javafx-src.zip
RUN rm -rf /usr/java/jdk1.8.0_201/lib/missioncontrol/ /usr/java/jdk1.8.0_201/lib/visualvm/ /usr/java/jdk1.8.0_201/db/
RUN alternatives --install /usr/bin/java java /usr/java/latest/bin/java 1
RUN alternatives --install /usr/bin/javac javac /usr/java/latest/bin/javac 1
ENV JAVA_HOME /usr/java/latest
ENV PATH=$PATH:/usr/java/latest/bin/java
RUN echo "$PATH"
RUN rm -f /tmp/jdk-8-linux-x64.rpm; yum -y remove wget; yum -y clean all
ENV SONAR_VERSION=7.6 \
SONARQUBE_HOME=/opt/sonarqube \
SONARQUBE_JDBC_USERNAME=sonar \
SONARQUBE_JDBC_PASSWORD=sonar \
SONARQUBE_JDBC_URL=
# Http port
EXPOSE 9000
RUN groupadd -r devopsadmin && useradd -r -g devopsadmin devopsadmin
RUN set -x \
&& cd /opt \
&& curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip \
&& curl -o sonarqube.zip.asc -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip.asc \
&& /bin/bash /usr/bin/unzip /opt/sonarqube.zip \
&& mv sonarqube-$SONAR_VERSION sonarqube \
&& chown -R devopsadmin:devopsadmin sonarqube \
&& rm sonarqube.zip* \
&& rm -rf $SONARQUBE_HOME/bin/*
VOLUME "$SONARQUBE_HOME/data"
WORKDIR $SONARQUBE_HOME
COPY run.sh $SONARQUBE_HOME/bin/
USER devopsadmin
ENTRYPOINT ["./bin/run.sh"]
please suggest
EDIT 1 :
see the entire log :
Step 14/21 : EXPOSE 9000
---> Using cache
---> 6d8df8bfdaf5
Step 15/21 : RUN groupadd -r devopsadmin && useradd -r -g devopsadmin devopsadmin
---> Using cache
---> 4d1a28e9288c
Step 16/21 : RUN set -x && cd /opt && curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip && curl -o sonarqube.zip.asc -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip.asc && /bin/bash /usr/bin/unzip /opt/sonarqube.zip && mv sonarqube-$SONAR_VERSION sonarqube && chown -R devopsadmin:devopsadmin sonarqube && rm sonarqube.zip* && rm -rf $SONARQUBE_HOME/bin/*
---> Running in 2e0d5ad30767
+ cd /opt
+ curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-7.6.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:19 --:--:-- 0curl: (6) Could not resolve host: binaries.sonarsource.com; Unknown error
The command '/bin/sh -c set -x && cd /opt && curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip && curl -o sonarqube.zip.asc -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip.asc && /bin/bash /usr/bin/unzip /opt/sonarqube.zip && mv sonarqube-$SONAR_VERSION sonarqube && chown -R devopsadmin:devopsadmin sonarqube && rm sonarqube.zip* && rm -rf $SONARQUBE_HOME/bin/*' returned a non-zero code: 6
[devopsadmin#be-deb-prod-cicdtoold-vm newsonar]$ curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-7.6.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 252M 100 252M 0 0 26.5M 0 0:00:09 0:00:09 --:--:-- 32.1M
I am currently trying to setup a Gazebo Simulation with Docker.Since i want to use a custom Gazebo Version made by Alessio Rocchi I am building everything from source. But while trying to build everything, i get the following error code:
cc1plus: warning: /tmp/gazebo/deps/klampt/Simulation/include: No such file or directory
cc1plus: warning: /etc/klampt/Library/KrisLibrary: No such file or directory
In file included from /tmp/gazebo/deps/klampt/Simulation/ODECustomGeometry.cpp:1:0:
/tmp/gazebo/deps/klampt/Simulation/ODECustomGeometry.h:4:34: fatal error: geometry/AnyGeometry.h: No such file or directory
#include <geometry/AnyGeometry.h>
^
compilation terminated.
deps/klampt/CMakeFiles/gazebo_klampt.dir/build.make:54: recipe for target 'deps/klampt/CMakeFiles/gazebo_klampt.dir/Simulation/ODECustomGeometry.cpp.o' failed
make[2]: *** [deps/klampt/CMakeFiles/gazebo_klampt.dir/Simulation/ODECustomGeometry.cpp.o] Error 1
CMakeFiles/Makefile2:388: recipe for target 'deps/klampt/CMakeFiles/gazebo_klampt.dir/all' failed
make[1]: *** [deps/klampt/CMakeFiles/gazebo_klampt.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
I am guessing it is more a CMake fault than a Docker fault, but i still don't manage to find a way around it. Maybe somebody can point out my probably fairly obvious errors.
FROM stevekuznetsov/klampt:latest
RUN sh -c 'echo "deb http://packages.ros.org/ros/ubuntu vivid main" > /etc/apt/sources.list.d/ros-latest.list'
RUN apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-key 421C365BD9FF1F717815A3895523BAEEB01FA116
RUN apt-get update
RUN apt-get -y install ros-jade-desktop-full
RUN sh -c 'echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable vivid main" > /etc/apt/sources.list.d/gazebo-stable.list'
RUN apt-get -y install wget
RUN wget http://packages.osrfoundation.org/gazebo.key -O - | apt-key add -
RUN apt-get update
RUN wget https://bitbucket.org/osrf/release-tools/raw/default/jenkins-scripts/lib/dependencies_archive.sh -O /tmp/dependencies.sh
RUN apt-get -y install build-essential \
cmake \
mercurial
RUN hg clone https://bitbucket.org/ignitionrobotics/ign-math /tmp/ign-math
RUN cd /tmp/ign-math && \
hg up ign-math2 && \
mkdir build && \
cd build && \
cmake ../ && \
make -j4 && \
make install
RUN apt-get -y install build-essential \
cmake \
mercurial \
python \
libboost-system-dev \
libboost-filesystem-dev \
libboost-program-options-dev \
libboost-regex-dev \
libboost-iostreams-dev \
libtinyxml-dev \
libxml2-utils \
ruby-dev \
ruby
RUN hg clone https://bitbucket.org/osrf/sdformat /tmp/sdformat
RUN cd /tmp/sdformat && \
hg up sdf3 && \
mkdir build && \
cd build && \
cmake ../ && \
make -j4 && \
make install
RUN hg clone https://lpuck#bitbucket.org/arocchi/gazebo ./tmp/gazebo
RUN apt-get -y install protobuf-compiler
RUN cd /tmp/gazebo && \
hg pull && \
hg update blem_gz6_6.5.1 && \
mkdir build && \
cd build && \
cmake -DKRISLIBRARY_INCLUDE_DIR=/etc/Klampt/Library/KrisLibrary -DKRISLIBRARY_LIBRARY=etc/Klampt/Library/KrisLibrary/lib ../ && \
make -j4 && \
make install
My guess is, that this line (third last):
cmake -DKRISLIBRARY_INCLUDE_DIR=/etc/Klampt/Library/KrisLibrary -DKRISLIBRARY_LIBRARY=etc/Klampt/Library/KrisLibrary/lib ../ && \
is wrong and therefore responsible for the error. I guess i have some dependency which is set wrong. It would be highly appreciated if somebody could help me.
Edit:
The new error, and i updated the Dockerfile and the error Cmake line
Linking CXX shared library libgazebo_physics_ode.so
/usr/bin/ld: cannot find -lKrisLibrary
collect2: error: ld returned 1 exit status
make[2]: *** [gazebo/physics/ode/libgazebo_physics_ode.so.6.5.1] Error 1
gazebo/physics/ode/CMakeFiles/gazebo_physics_ode.dir/build.make:558: recipe for target 'gazebo/physics/ode/libgazebo_physics_ode.so.6.5.1' failed
CMakeFiles/Makefile2:4864: recipe for target 'gazebo/physics/ode/CMakeFiles/gazebo_physics_ode.dir/all' failed
make[1]: *** [gazebo/physics/ode/CMakeFiles/gazebo_physics_ode.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....