sha256sum error while building nvidia-docker on ppc64le - docker

gurus.
I encountered the following error while building nvidia-docker of CUDA v8.0 on ppc64le.
root#minsky:/data/nvidia-docker# make cuda OS=ubuntu-16.04
make -C /data/nvidia-docker/ubuntu-16.04/cuda
make[1]: Entering directory '/data/nvidia-docker/ubuntu-16.04/cuda'
docker build -f /data/nvidia-docker/ubuntu-16.04/cuda/8.0/runtime/Dockerfile.ppc64le -t cuda:8.0-runtime /data//nvidia-docker/ubuntu-16.04/cuda/8.0/runtime
Sending build context to Docker daemon 10.24 kB
...
Step 6 : RUN CUDNN_DOWNLOAD_SUM=51f698d468401cef2e3e2ef9bb557bd57cbeb4dca895d1d1ae8a751d090bbe39 && curl -fsSL http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-ppc64le-v5.1.tgz -O && echo "$CUDNN_DOWNLOAD_SUM cudnn-8.0-linux-ppc64le-v5.1.tgz" | sha256sum -c --strict - && tar -xzf cudnn-8.0-linux-ppc64le-v5.1.tgz -C /usr/local --wildcards 'cuda/targets/ppc64le-linux/lib/libcudnn.so.*' && ln -s cuda/targets/ppc64le-linux/lib /usr/local/cuda/lib64 && rm cudnn-8.0-linux-ppc64le-v5.1.tgz && ldconfig
---> Running in 19a5bb5c9c19
cudnn-8.0-linux-ppc64le-v5.1.tgz: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match
The command '/bin/sh -c CUDNN_DOWNLOAD_SUM=51f698d468401cef2e3e2ef9bb557bd57cbeb4dca895d1d1ae8a751d090bbe39 && curl -fsSL http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-ppc64le-v5.1.tgz -O && echo "$CUDNN_DOWNLOAD_SUM cudnn-8.0-linux-ppc64le-v5.1.tgz" | sha256sum -c --strict - && tar -xzf cudnn-8.0-linux-ppc64le-v5.1.tgz -C /usr/local --wildcards 'cuda/targets/ppc64le-linux/lib/libcudnn.so.*' && ln -s cuda/targets/ppc64le-linux/lib /usr/local/cuda/lib64 && rm cudnn-8.0-linux-ppc64le-v5.1.tgz && ldconfig' returned a non-zero code: 1
../../mk/cuda.mk:87: recipe for target '8.0-cudnn5-runtime' failed
make[1]: *** [8.0-cudnn5-runtime] Error 1
make[1]: Leaving directory '/data/fuzhiwen/nvidia-docker/ubuntu-16.04/cuda'
Makefile:37: recipe for target 'cuda' failed
make: *** [cuda] Error 2
Any idea on how to get around this gracefully ?

I solved this situation by editing the Dockerfile.ppc64le files as below.
root#minsky:/data/nvidia-docker# vi ./ubuntu-16.04/cuda/8.0/runtime/cudnn5/Dockerfile.ppc64le
...
RUN CUDNN_DOWNLOAD_SUM=663aac0328f821d90ae9c74ee43e90751706546c2ce769ea9c96f92864300af6 && \
#RUN CUDNN_DOWNLOAD_SUM=51f698d468401cef2e3e2ef9bb557bd57cbeb4dca895d1d1ae8a751d090bbe39 && \
root#minsky:/data/nvidia-docker# vi ./ubuntu-16.04/cuda/8.0/devel/cudnn5/Dockerfile.ppc64le
...
RUN CUDNN_DOWNLOAD_SUM=663aac0328f821d90ae9c74ee43e90751706546c2ce769ea9c96f92864300af6 && \
#RUN CUDNN_DOWNLOAD_SUM=51f698d468401cef2e3e2ef9bb557bd57cbeb4dca895d1d1ae8a751d090bbe39 && \
root#minsky:/data/nvidia-docker# make cuda OS=ubuntu-16.04
I got the checksum value of 663aac0328f821d90ae9c74ee43e90751706546c2ce769ea9c96f92864300af6 from the following test.
root#minsky:/data/nvidia-docker# sha256sum cudnn-8.0-linux-ppc64le-v5.1.tgz
663aac0328f821d90ae9c74ee43e90751706546c2ce769ea9c96f92864300af6 cudnn-8.0-linux-ppc64le-v5.1.tgz
This works fine. Still, I am not sure if this is the correct solution and why this error happens in the first place.

Related

Error: docker buildx on x86_64 for building multiarch--x86_64, arm64

I just make multarch images by using buildx
When running docker buildx build --platform linux/amd64,linux/arm64 -t maskertim/zookeeper-demo .
Error Message
The Error as following:
------
> [linux/arm64 3/5] RUN curl "https://archive.apache.org/dist/kafka/2.8.0/kafka_2.13-2.8.0.tgz" -o /tmp/kafka/kafka.tgz && mkdir /kafka && cd /kafka && tar -xzvf /tmp/kafka/kafka.tgz --strip 1:
#8 0.915 % Total % Received % Xferd Average Speed Time Time Time Current
#8 0.916 Dload Upload Total Spent Left Speed
100 68.0M 100 68.0M 0 0 936k 0 0:01:14 0:01:14 --:--:-- 3475k
#8 75.45 tar: Child returned status 1
#8 75.45 tar: Error is not recoverable: exiting now
------
dockerfile:9
--------------------
8 |
9 | >>> RUN curl "https://archive.apache.org/dist/kafka/${KAFKA_VERSION}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz" \
10 | >>> -o /tmp/kafka/kafka.tgz && \
11 | >>> mkdir /kafka && cd /kafka && \
12 | >>> tar -xzvf /tmp/kafka/kafka.tgz --strip 1
13 |
--------------------
error: failed to solve: process "/dev/.buildkit_qemu_emulator /bin/sh -c curl \"https://archive.apache.org/dist/kafka/${KAFKA_VERSION}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz\" \t-o /tmp/kafka/kafka.tgz && \tmkdir /kafka && cd /kafka && \ttar -xzvf /tmp/kafka/kafka.tgz --strip 1" did not complete successfully: exit code: 2
Dockerfile
And this is my dockerfile:
FROM openjdk:11.0.12-jre-buster
ENV KAFKA_VERSION 2.8.0
ENV SCALA_VERSION 2.13
RUN mkdir /tmp/kafka && \
apt-get update && \
apt-get install -y curl
RUN curl "https://archive.apache.org/dist/kafka/${KAFKA_VERSION}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz" \
-o /tmp/kafka/kafka.tgz && \
mkdir /kafka && cd /kafka && \
tar -xzvf /tmp/kafka/kafka.tgz --strip 1
COPY start-zookeeper.sh /usr/bin
RUN chmod +x /usr/bin/start-zookeeper.sh
CMD ["start-zookeeper.sh"]
What happens? I have no idea why to occur this error and no way to solve it.
Unzip it first:
gzip -d /tmp/kafka/kafka.tgz
And after that untar it with:
tar xpf /tmp/kafka/kafka.tar
I had the same problem and discovered https://stackoverflow.com/a/71143883
Running
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
allowed me to keep the decompression step in the tar command
I think the problem is related to tar or more specific one of its child processes is failing to untar or unzip because of a missing dependency, which seems not to be present in the qemu_emulator that buildx uses.
I ended up using the ADD command in the dockerfile instead of RUN tar, which has that functionality built in.

docker images alpine3.14 compile nginx 1.21.1 error,however docker images alpine3.12 is ok,why?

I try to compile docker images from alpine:3.14.
The error is as follows
make -f objs/Makefile
make: make: Operation not permitted
make: *** [Makefile:10: build] Error 127
Then I switched to version alpine:3.12, alpine:3.13 and found that both are OK!
The following is my key problematic code for compiling NGINX based on alpine:3.14 version
# Omit irrelevant code
RUN \
addgroup -S www && adduser www -D -S -s /bin/sh -G www \
&& wget -P /home/soft https://github.com/vozlt/nginx-module-vts/archive/v0.1.18.tar.gz \
&& wget -P /home/soft http://nginx.org/download/nginx-1.21.1.tar.gz \
&& wget -P /home/soft https://ftp.pcre.org/pub/pcre/pcre-8.44.tar.gz \
&& cd /home/soft && tar -zxf nginx-1.21.1.tar.gz && tar -zxf v0.1.18.tar.gz && tar -zxf pcre-8.44.tar.gz \
&& cd /home/soft/nginx-1.21.1 \
&& ./configure --user=www --group=www --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --with-http_v2_module --with-http_gzip_static_module --with-stream --with-http_sub_module --with-pcre=/home/soft/pcre-8.44 --add-module=/home/soft/nginx-module-vts-0.1.18 \
&& make && make install \
# Omit irrelevant code
Error 127 seven could mean two things. Either the the binary for the command is not found in PATH variable (Please take a note here, sometimes it can also happen that the command is found but some library is missing) or the binary doesn't have the execution permissions. By the look of it, I suspect it's because it doesn't have the execution permissions.
I would suggest you to run the alpine:3.14 container and inspect the PATH variable, the make binary location and the permissions.

Error when building docker image for jupyter spark notebook

I am trying to build Jupyter notebook in docker following the guide here:
https://github.com/cordon-thiago/airflow-spark
and got an error with exit code: 8.
I ran:
$ docker build --rm --force-rm -t jupyter/pyspark-notebook:3.0.1 .
the building stops at the code:
RUN wget -q $(wget -qO- https://www.apache.org/dyn/closer.lua/spark/spark-${APACHE_SPARK_VERSION}/spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz\?as_json | \
python -c "import sys, json; content=json.load(sys.stdin); print(content['preferred']+content['path_info'])") && \
echo "${spark_checksum} *spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz" | sha512sum -c - && \
tar xzf "spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz" -C /usr/local --owner root --group root --no-same-owner && \
rm "spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz"
with error message like below:
=> ERROR [4/9] RUN wget -q $(wget -qO- https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz?as_json | python -c "import sys, json; content=json.load(sys.stdin); 2.3s
------
> [4/9] RUN wget -q $(wget -qO- https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz?as_json | python -c "import sys, json; content=json.load(sys.stdin); print(content[
'preferred']+content['path_info'])") && echo "F4A10BAEC5B8FF1841F10651CAC2C4AA39C162D3029CA180A9749149E6060805B5B5DDF9287B4AA321434810172F8CC0534943AC005531BB48B6622FBE228DDC *spark-3.0.1-bin-hadoop2.7.
tgz" | sha512sum -c - && tar xzf "spark-3.0.1-bin-hadoop2.7.tgz" -C /usr/local --owner root --group root --no-same-owner && rm "spark-3.0.1-bin-hadoop2.7.tgz":
------
executor failed running [/bin/bash -o pipefail -c wget -q $(wget -qO- https://www.apache.org/dyn/closer.lua/spark/spark-${APACHE_SPARK_VERSION}/spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz\
?as_json | python -c "import sys, json; content=json.load(sys.stdin); print(content['preferred']+content['path_info'])") && echo "${spark_checksum} *spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_
VERSION}.tgz" | sha512sum -c - && tar xzf "spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz" -C /usr/local --owner root --group root --no-same-owner && rm "spark-${APACHE_SPARK_VERSION}
-bin-hadoop${HADOOP_VERSION}.tgz"]: exit code: 8
Really appreciate if someone can enlighten me on this. Thanks!
The exit code 8 is likely from wget meaning an error response from the server. As an example, this path that the Dockerfile tries to wget from isn't valid anymore: https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz
From the issues on the repo, it appears that Apache version 3.0.1 is no longer valid so you should override the APACHE_SPARK version to 3.0.2 with a --build-arg:
docker build --rm --force-rm \
--build-arg spark_version=3.0.2 \
-t jupyter/pyspark-notebook:3.0.2 .
EDIT
See comment below for more, the command that worked was:
docker build --rm --force-rm \
--build-arg spark_version=3.1.1 \
--build-arg hadoop_version=2.7 \
-t jupyter/pyspark-notebook:3.1.1 .
And updated spark checksum to reflect the version for 3.1.1: https://downloads.apache.org/spark/spark-3.1.1/spark-3.1.1-bin-hadoop2.7.tgz.sha512
For this answer to be relevant in the future, it will likely need to update versions and checksum again for the latest spark/hadoop versions.

curl command not working inside dockerfile but works outside in the same network

i am writing docker file to install java and sonar. everything works great till java installation. but when downloading of sonar starts using curl , it fails. more frustrating is , exact same command works outside docker file, same shell prompt and on the same network.
I am really confused how can this be happen. what silly mistake i am doing not getting.
please see the dockefile :
FROM centos:7
ENV JAVA_VERSION 8u201
ENV BUILD_VERSION b09
RUN yum -y install wget; wget --no-cookies --no-check-certificate --header "Cookie: oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/$JAVA_VERSION-$BUILD_VERSION/42970487e3af4f5aa5bca3f542482c60/jdk-$JAVA_VERSION-linux-x64.rpm" -O /tmp/jdk-8-linux-x64.rpm; yum -y install /tmp/jdk-8-linux-x64.rpm
# JDK stripping
RUN rm -f /usr/java/jdk1.8.0_201/src.zip /usr/java/jdk1.8.0_201/javafx-src.zip
RUN rm -rf /usr/java/jdk1.8.0_201/lib/missioncontrol/ /usr/java/jdk1.8.0_201/lib/visualvm/ /usr/java/jdk1.8.0_201/db/
RUN alternatives --install /usr/bin/java java /usr/java/latest/bin/java 1
RUN alternatives --install /usr/bin/javac javac /usr/java/latest/bin/javac 1
ENV JAVA_HOME /usr/java/latest
ENV PATH=$PATH:/usr/java/latest/bin/java
RUN echo "$PATH"
RUN rm -f /tmp/jdk-8-linux-x64.rpm; yum -y remove wget; yum -y clean all
ENV SONAR_VERSION=7.6 \
SONARQUBE_HOME=/opt/sonarqube \
SONARQUBE_JDBC_USERNAME=sonar \
SONARQUBE_JDBC_PASSWORD=sonar \
SONARQUBE_JDBC_URL=
# Http port
EXPOSE 9000
RUN groupadd -r devopsadmin && useradd -r -g devopsadmin devopsadmin
RUN set -x \
&& cd /opt \
&& curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip \
&& curl -o sonarqube.zip.asc -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip.asc \
&& /bin/bash /usr/bin/unzip /opt/sonarqube.zip \
&& mv sonarqube-$SONAR_VERSION sonarqube \
&& chown -R devopsadmin:devopsadmin sonarqube \
&& rm sonarqube.zip* \
&& rm -rf $SONARQUBE_HOME/bin/*
VOLUME "$SONARQUBE_HOME/data"
WORKDIR $SONARQUBE_HOME
COPY run.sh $SONARQUBE_HOME/bin/
USER devopsadmin
ENTRYPOINT ["./bin/run.sh"]
please suggest
EDIT 1 :
see the entire log :
Step 14/21 : EXPOSE 9000
---> Using cache
---> 6d8df8bfdaf5
Step 15/21 : RUN groupadd -r devopsadmin && useradd -r -g devopsadmin devopsadmin
---> Using cache
---> 4d1a28e9288c
Step 16/21 : RUN set -x && cd /opt && curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip && curl -o sonarqube.zip.asc -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip.asc && /bin/bash /usr/bin/unzip /opt/sonarqube.zip && mv sonarqube-$SONAR_VERSION sonarqube && chown -R devopsadmin:devopsadmin sonarqube && rm sonarqube.zip* && rm -rf $SONARQUBE_HOME/bin/*
---> Running in 2e0d5ad30767
+ cd /opt
+ curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-7.6.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:19 --:--:-- 0curl: (6) Could not resolve host: binaries.sonarsource.com; Unknown error
The command '/bin/sh -c set -x && cd /opt && curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip && curl -o sonarqube.zip.asc -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-$SONAR_VERSION.zip.asc && /bin/bash /usr/bin/unzip /opt/sonarqube.zip && mv sonarqube-$SONAR_VERSION sonarqube && chown -R devopsadmin:devopsadmin sonarqube && rm sonarqube.zip* && rm -rf $SONARQUBE_HOME/bin/*' returned a non-zero code: 6
[devopsadmin#be-deb-prod-cicdtoold-vm newsonar]$ curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/CommercialDistribution/sonarqube-enterprise/sonarqube-enterprise-7.6.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 252M 100 252M 0 0 26.5M 0 0:00:09 0:00:09 --:--:-- 32.1M

Build error using cmake in docker

I am currently trying to setup a Gazebo Simulation with Docker.Since i want to use a custom Gazebo Version made by Alessio Rocchi I am building everything from source. But while trying to build everything, i get the following error code:
cc1plus: warning: /tmp/gazebo/deps/klampt/Simulation/include: No such file or directory
cc1plus: warning: /etc/klampt/Library/KrisLibrary: No such file or directory
In file included from /tmp/gazebo/deps/klampt/Simulation/ODECustomGeometry.cpp:1:0:
/tmp/gazebo/deps/klampt/Simulation/ODECustomGeometry.h:4:34: fatal error: geometry/AnyGeometry.h: No such file or directory
#include <geometry/AnyGeometry.h>
^
compilation terminated.
deps/klampt/CMakeFiles/gazebo_klampt.dir/build.make:54: recipe for target 'deps/klampt/CMakeFiles/gazebo_klampt.dir/Simulation/ODECustomGeometry.cpp.o' failed
make[2]: *** [deps/klampt/CMakeFiles/gazebo_klampt.dir/Simulation/ODECustomGeometry.cpp.o] Error 1
CMakeFiles/Makefile2:388: recipe for target 'deps/klampt/CMakeFiles/gazebo_klampt.dir/all' failed
make[1]: *** [deps/klampt/CMakeFiles/gazebo_klampt.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
I am guessing it is more a CMake fault than a Docker fault, but i still don't manage to find a way around it. Maybe somebody can point out my probably fairly obvious errors.
FROM stevekuznetsov/klampt:latest
RUN sh -c 'echo "deb http://packages.ros.org/ros/ubuntu vivid main" > /etc/apt/sources.list.d/ros-latest.list'
RUN apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-key 421C365BD9FF1F717815A3895523BAEEB01FA116
RUN apt-get update
RUN apt-get -y install ros-jade-desktop-full
RUN sh -c 'echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable vivid main" > /etc/apt/sources.list.d/gazebo-stable.list'
RUN apt-get -y install wget
RUN wget http://packages.osrfoundation.org/gazebo.key -O - | apt-key add -
RUN apt-get update
RUN wget https://bitbucket.org/osrf/release-tools/raw/default/jenkins-scripts/lib/dependencies_archive.sh -O /tmp/dependencies.sh
RUN apt-get -y install build-essential \
cmake \
mercurial
RUN hg clone https://bitbucket.org/ignitionrobotics/ign-math /tmp/ign-math
RUN cd /tmp/ign-math && \
hg up ign-math2 && \
mkdir build && \
cd build && \
cmake ../ && \
make -j4 && \
make install
RUN apt-get -y install build-essential \
cmake \
mercurial \
python \
libboost-system-dev \
libboost-filesystem-dev \
libboost-program-options-dev \
libboost-regex-dev \
libboost-iostreams-dev \
libtinyxml-dev \
libxml2-utils \
ruby-dev \
ruby
RUN hg clone https://bitbucket.org/osrf/sdformat /tmp/sdformat
RUN cd /tmp/sdformat && \
hg up sdf3 && \
mkdir build && \
cd build && \
cmake ../ && \
make -j4 && \
make install
RUN hg clone https://lpuck#bitbucket.org/arocchi/gazebo ./tmp/gazebo
RUN apt-get -y install protobuf-compiler
RUN cd /tmp/gazebo && \
hg pull && \
hg update blem_gz6_6.5.1 && \
mkdir build && \
cd build && \
cmake -DKRISLIBRARY_INCLUDE_DIR=/etc/Klampt/Library/KrisLibrary -DKRISLIBRARY_LIBRARY=etc/Klampt/Library/KrisLibrary/lib ../ && \
make -j4 && \
make install
My guess is, that this line (third last):
cmake -DKRISLIBRARY_INCLUDE_DIR=/etc/Klampt/Library/KrisLibrary -DKRISLIBRARY_LIBRARY=etc/Klampt/Library/KrisLibrary/lib ../ && \
is wrong and therefore responsible for the error. I guess i have some dependency which is set wrong. It would be highly appreciated if somebody could help me.
Edit:
The new error, and i updated the Dockerfile and the error Cmake line
Linking CXX shared library libgazebo_physics_ode.so
/usr/bin/ld: cannot find -lKrisLibrary
collect2: error: ld returned 1 exit status
make[2]: *** [gazebo/physics/ode/libgazebo_physics_ode.so.6.5.1] Error 1
gazebo/physics/ode/CMakeFiles/gazebo_physics_ode.dir/build.make:558: recipe for target 'gazebo/physics/ode/libgazebo_physics_ode.so.6.5.1' failed
CMakeFiles/Makefile2:4864: recipe for target 'gazebo/physics/ode/CMakeFiles/gazebo_physics_ode.dir/all' failed
make[1]: *** [gazebo/physics/ode/CMakeFiles/gazebo_physics_ode.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Resources