Docker: Failed building wheel for scipy - docker

I'm trying for facerecogition from (https://github.com/jamct/facerec) on my raspberry pi 3. Therefore the by runs with the newest version of Raspbian Stretch.
I am running:
docker build -t facerec:latest .
But I get the following error:
ERROR: Failed building wheel for scipy
ERROR: Could not build wheels for scipy which use PEP 517 and cannot be installed directly
Here is a little bit more of the log:
creating build/temp.linux-armv7l-3.6/scipy/fft
creating build/temp.linux-armv7l-3.6/scipy/fft/_pocketfft
compile options: '-DPOCKETFFT_PTHREADS -I/root/.local/include/python3.6m -I/usr/local/include/python3.6m -I/tmp/pip-build-env-u7r4c8tr/overlay/lib/python3.6/site-packages/numpy/core/include -I/usr/local/include/python3.6m -c'
extra options: '-std=c++14 -fvisibility=hidden'
g++: scipy/fft/_pocketfft/pypocketfft.cxx
Running from scipy source directory.
/tmp/pip-build-env-u7r4c8tr/overlay/lib/python3.6/site-packages/numpy/distutils/system_info.py:572: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
self.calc_info()
/tmp/pip-build-env-u7r4c8tr/overlay/lib/python3.6/site-packages/numpy/distutils/system_info.py:664: UserWarning: Specified path /usr/include/python3.6m is invalid.
return self.get_paths(self.section, key)
scipy/fft/_pocketfft/pypocketfft.cxx:15:31: fatal error: pybind11/pybind11.h: No such file or directory
#include <pybind11/pybind11.h>
^
compilation terminated.
scipy/fft/_pocketfft/pypocketfft.cxx:15:31: fatal error: pybind11/pybind11.h: No such file or directory
#include <pybind11/pybind11.h>
^
compilation terminated.
error: Command "g++ -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPOCKETFFT_PTHREADS -I/root/.local/include/python3.6m -I/usr/local/include/python3.6m -I/tmp/pip-build-env-u7r4c8tr/overlay/lib/python3.6/site-packages/numpy/core/include -I/usr/local/include/python3.6m -c scipy/fft/_pocketfft/pypocketfft.cxx -o build/temp.linux-armv7l-3.6/scipy/fft/_pocketfft/pypocketfft.o -MMD -MF build/temp.linux-armv7l-3.6/scipy/fft/_pocketfft/pypocketfft.o.d -std=c++14 -fvisibility=hidden" failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for scipy
ERROR: Could not build wheels for scipy which use PEP 517 and cannot be installed directly
The command '/bin/sh -c cd /face_recognition && pip3 install -r requirements.txt && python3 setup.py install' returned a non-zero code: 1
This is the dockerfile:
#Dockerfile for face-recognition
#Based on https://github.com/denverdino/face_recognition_pi
FROM resin/raspberry-pi-python:3
COPY pip.conf /root/.pip/pip.conf
RUN apt-get -y update
RUN apt-get install -y --fix-missing \
build-essential \
cmake \
gfortran \
git \
wget \
curl \
graphicsmagick \
libgraphicsmagick1-dev \
libatlas-dev \
libavcodec-dev \
libavformat-dev \
libboost-all-dev \
libgtk2.0-dev \
libjpeg-dev \
liblapack-dev \
libswscale-dev \
pkg-config \
python3-dev \
zip \
&& apt-get clean && rm -rf /tmp/* /var/tmp/*
RUN python3 -m ensurepip --upgrade && pip3 install --upgrade picamera[array] dlib
RUN pip install --upgrade pip
RUN pip install --upgrade pip setuptools wheel
RUN git clone --single-branch https://github.com/ageitgey/face_recognition.git
RUN cd /face_recognition && \
pip3 install -r requirements.txt && \
python3 setup.py install
CMD cd /face_recognition/examples && \
python3 recognize_faces_in_pictures.py
Would be happy if somebody has an idea what to do now.
Thanks!

You have to install Pybind11 before building SciPy:
python3 -m pip install pybind11

Related

Can't install python-ldap in Docker

I'm getting the following error when trying to install python-ldap module in Docker image for aws:
In file included from Modules/LDAPObject.c:3:0:
Modules/common.h:15:10: fatal error: lber.h: No such file or directory
#include <lber.h>
^~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for python-ldap
Failed to build python-ldap
ERROR: Could not build wheels for python-ldap, which is required to install pyproject.toml-based projects
The command '/bin/sh -c pipenv lock -r > requirements.txt && pip install -r requirements.txt -t python' returned a non-zero code: 1
And my Dockerfile:
FROM public.ecr.aws/lambda/python:3.8
ARG TMP_BUILD=/tmp
ARG DIST=/opt/build-dist
RUN yum makecache fast; yum clean all && yum -y update && yum -y upgrade; yum clean all && \
yum install -y yum-plugin-ovl; yum clean all && yum -y groupinstall "Development Tools"; yum clean all
RUN yum -y install gcc gcc-c++ make autoconf aclocal automake libtool python-devel openldap-devel; yum clean all && \
pip install --upgrade pip && pip install pipenv
WORKDIR ${TMP_BUILD}/build
COPY Pipfile .
COPY Pipfile.lock .
RUN pipenv lock -r > requirements.txt && \
pip install -r requirements.txt -t python
# && \
# find ./python -depth -path '*dist-info*' -delete && \
# find ./python -depth -path '*test*' -delete && \
# find ./python -depth -path '*pycache*' -delete
WORKDIR /opt
RUN mkdir -p ${DIST}/python && \
cp -rf ${TMP_BUILD}/build/python ${DIST} && \
cp -rf ${TMP_BUILD}/build/requirements.txt ${DIST}/requirements.txt
WORKDIR /var/task
This build used to work until recently and as you can see i have the python-devel openldap-devel packages so what's the problem?
Was also having trouble installing this module on my regular machine which runs ManjaroLinux. I had to build from source and change the name of a binary file manually. Could this be a similar situation?
Here is the Pipfile if it helps
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
requests = "*"
slack-bolt = "*"
slack-sdk = "*"
aiohttp = "*"
python-ldap = "*"
[dev-packages]
black = "*"
boto3 = "*"
pytest = "*"
pytest-runner = "*"
pytest-mock = "*"
pandas = "*"
[requires]
python_version = "3.8"
[scripts]
lint = "pipenv run black . --check"
"lint:fix" = "pipenv run black ."
integrationtest = "pipenv run pytest . -m integration "
test = "pipenv run pytest . -m 'not integration' --ignore-glob='integration.py' --junitxml=./TEST-results-lambdas.xml"
[pipenv]
allow_prereleases = true
Below works - 2022
apt-get install build-essential python3-dev libmemcached-dev libldap2-dev libsasl2-dev libzbar-dev ldap-utils tox lcov valgrind
Sample:
FROM python:3.10-slim
RUN apt-get update && \
apt-get --yes install build-essential python3-dev libmemcached-dev libldap2-dev libsasl2-dev libzbar-dev ldap-utils tox lcov valgrind && \
apt-get clean
I follow the official doc python-ldap: Debian it stuck at sldap installation and prumpt input password.
If remove sldap it said:
fatal error: libmemcached/memcached.h: No such file or directory
After replace sldap with libmemcached-dev problem solved.
Python relies on some packages to be present, in order to have them installed just add
RUN apt-get -y install libldap2-dev libsasl2-dev
on your Dockerfile
(or yum install -y <package> as per your example)

Installation pyproj does not work returned 127

I want to install pyproj on Debian on Docker.
This is my script:
FROM debian:latest
RUN /bin/bash -c 'apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -yq apt-utils && \
DEBIAN_FRONTEND=noninteractive apt-get install -yq \
libpython3-dev \
python3-pip \
proj-bin \
curl \
libcurl4-gnutls-dev \
wget \
sqlite3 \
node-sqlite3 \
libsqlite3-dev \
libtiff5 \
libtiff5-dev \
make \
python-rtree && \
apt-get upgrade -yq && \
apt-get clean && \
apt-get update \
'
RUN wget https://download.osgeo.org/proj/proj-7.2.0.tar.gz
RUN tar -xvzf proj-7.2.0.tar.gz
RUN cd proj-7.2.0/ && ./configure && make && make install
RUN find / -name proj
RUN /bin/bash -c 'rm -rf /var/lib/apt/lists/* && \
pip3 install \
pyproj --no-binary pyproj \
'
This is the output:
Step 6/7 : RUN find / -name proj
---> Running in 6ae2ce5a5157
/usr/share/proj
/usr/local/share/proj
/usr/local/include/proj
/usr/local/bin/proj
/usr/bin/proj
/proj-7.2.0/src/.libs/proj
/proj-7.2.0/src/proj
/proj-7.2.0/include/proj
Removing intermediate container 6ae2ce5a5157
---> 9e652a740f23
Step 7/7 : RUN /bin/bash -c 'rm -rf /var/lib/apt/lists/* && pip3 install pyproj --no-binary pyproj '
---> Running in 2fa15199039b
Collecting pyproj
Downloading https://files.pythonhosted.org/packages/17/e5/3f5cdff3e955bcd768cdb0f4236f2d6e022aaa72f57caf7f4d5f552c88fc/pyproj-3.0.0.post1.tar.gz (663kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-r4rvjbmz/pyproj/setup.py", line 224, in <module>
ext_modules=get_extension_modules(),
File "/tmp/pip-install-r4rvjbmz/pyproj/setup.py", line 155, in get_extension_modules
proj_version = get_proj_version(proj_dir)
File "/tmp/pip-install-r4rvjbmz/pyproj/setup.py", line 22, in get_proj_version
proj_ver = subprocess.check_output(str(proj), stderr=subprocess.STDOUT).decode(
File "/usr/lib/python3.7/subprocess.py", line 395, in check_output
**kwargs).stdout
File "/usr/lib/python3.7/subprocess.py", line 487, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '/usr/local/bin/proj' returned non-zero exit status 127.
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-r4rvjbmz/pyproj/
Earlier I tried installing pyproj just via pip, it will install PROJ as a dependency, but with the wrong version. That's why I switched to the installation of the binaries via wget.
Also, I gave a find output to show that proj is actually in the right location.
Any suggestions?
https://pyproj4.github.io/pyproj/stable/installation.html
I would recommend either:
Upgrade to pip>=19 so PROJ 7.2 is in the wheel.
ENV PROJ_DIR=/path/to/proj/install and when you install PROJ: ./configure --prefix $PROJ_DIR

Dockerfile: Python.h: No such file or directory

I have a simple dockerfile that I am using to run containers on AWS, I'm hitting an issue though when installing s3fs, which is strange since I've used this snippet in previous dockerfiles without issue.
Is it something with the distribution?
Error:
multidict/_multidict.c:1:10: fatal error: Python.h: No such file or directory
Dockerfile:
FROM amazonlinux:latest
RUN yum -y install which unzip aws-cli \
&& yum install -y python3-pip python3 python3-setuptools \
&& yum install -y tar.x86_64 \
&& DEBIAN_FRONTEND=noninteractive yum install -y ksh
RUN pip3 install boto3 \
&& yum install -y gcc \
&& pip3 install s3fs
Here is the output log:
Installing collected packages: fsspec, docutils, botocore, typing-extensions, aioitertools, wrapt, attrs, chardet, multidict, async-timeout, idna, yarl, aiohttp, aiobotocore, s3fs
Found existing installation: botocore 1.19.11
Uninstalling botocore-1.19.11:
Successfully uninstalled botocore-1.19.11
Running setup.py install for wrapt: started
Running setup.py install for wrapt: finished with status 'done'
Running setup.py install for multidict: started
Running setup.py install for multidict: finished with status 'error'
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-l35mgnnl/multidict/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-7d6e5wlf-record/install-record.txt --single-version-externally-managed --compile:
**********************
* Accellerated build *
**********************
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/multidict
copying multidict/_abc.py -> build/lib.linux-x86_64-3.7/multidict
copying multidict/__init__.py -> build/lib.linux-x86_64-3.7/multidict
copying multidict/_multidict_base.py -> build/lib.linux-x86_64-3.7/multidict
copying multidict/_multidict_py.py -> build/lib.linux-x86_64-3.7/multidict
copying multidict/_compat.py -> build/lib.linux-x86_64-3.7/multidict
running egg_info
writing multidict.egg-info/PKG-INFO
writing dependency_links to multidict.egg-info/dependency_links.txt
writing top-level names to multidict.egg-info/top_level.txt
reading manifest file 'multidict.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files found matching 'multidict/_multidict.html'
warning: no previously-included files found matching 'multidict/*.so'
warning: no previously-included files found matching 'multidict/*.pyd'
warning: no previously-included files found matching 'multidict/*.pyd'
no previously-included directories found matching 'docs/_build'
writing manifest file 'multidict.egg-info/SOURCES.txt'
copying multidict/__init__.pyi -> build/lib.linux-x86_64-3.7/multidict
copying multidict/_multidict.c -> build/lib.linux-x86_64-3.7/multidict
copying multidict/py.typed -> build/lib.linux-x86_64-3.7/multidict
creating build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/defs.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/dict.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/istr.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/iter.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/pair_list.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/views.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
running build_ext
building 'multidict._multidict' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/multidict
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.7m -c multidict/_multidict.c -o build/temp.linux-x86_64-3.7/multidict/_multidict.o -O2 -std=c99 -Wall -Wsign-compare -Wconversion -fno-strict-aliasing -pedantic
multidict/_multidict.c:1:10: fatal error: Python.h: No such file or directory
#include "Python.h"
^~~~~~~~~~
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-l35mgnnl/multidict/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-7d6e5wlf-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-l35mgnnl/multidict/
The command '/bin/sh -c pip3 install boto3 && yum install -y gcc && pip3 install s3fs' returned a non-zero code: 1
Any help is much appreciated!
You should add python3-devel package that contains absent headers.
FROM amazonlinux:latest
RUN yum -y install which unzip aws-cli \
&& yum install -y python3-pip python3 python3-setuptools \
&& yum install -y python3-devel.x86_64 \
&& yum install -y tar.x86_64 \
&& DEBIAN_FRONTEND=noninteractive yum install -y ksh
RUN pip3 install boto3 \
&& yum install -y gcc \
&& pip3 install s3fs

Building Image from Dockerfile fails on ppc64 when using COPY --FROM

Currently i have image build process via jenkins that launches agents on ppc64le and x86 architecture.
Currently everything works perfectly on the x86 agent, but when executing on ppc64le it fails with the error described bellow:
Error that only happeons on ppc64le:
---> Running in 5458becfaa7b
/usr/bin/apt-get: 1: /usr/bin/apt-get: ELF: not found
/usr/bin/apt-get: 1: /usr/bin/apt-get: #8�#8: not found
/usr/bin/apt-get: 8: /usr/bin/apt-get: Syntax error: Unterminated quoted string
The command '/bin/sh -c apt-get -qq update && apt-get -qqy install python3 python3-dev python3-numpy python3-scipy python3-pip libkeyutils1 && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 2
script returned exit code 2
The sections where it fails:
FROM ubuntu:16.04
## Install random tests
COPY --from=appt /usr/ /usr/
COPY --from=appt /bin /bin
RUN apt-get -qq update \
&& apt-get -qqy install \
python3 \
python3-dev \
python3-numpy \
python3-scipy \
python3-pip \
libkeyutils1 \
&& rm -rf /var/lib/apt/lists/*
Your copy from COPY --from=appt is causing issues because it contains non-ppc64le executables. apt-get must be running something from either the /usr or /bin directories.

Build error using cmake in docker

I am currently trying to setup a Gazebo Simulation with Docker.Since i want to use a custom Gazebo Version made by Alessio Rocchi I am building everything from source. But while trying to build everything, i get the following error code:
cc1plus: warning: /tmp/gazebo/deps/klampt/Simulation/include: No such file or directory
cc1plus: warning: /etc/klampt/Library/KrisLibrary: No such file or directory
In file included from /tmp/gazebo/deps/klampt/Simulation/ODECustomGeometry.cpp:1:0:
/tmp/gazebo/deps/klampt/Simulation/ODECustomGeometry.h:4:34: fatal error: geometry/AnyGeometry.h: No such file or directory
#include <geometry/AnyGeometry.h>
^
compilation terminated.
deps/klampt/CMakeFiles/gazebo_klampt.dir/build.make:54: recipe for target 'deps/klampt/CMakeFiles/gazebo_klampt.dir/Simulation/ODECustomGeometry.cpp.o' failed
make[2]: *** [deps/klampt/CMakeFiles/gazebo_klampt.dir/Simulation/ODECustomGeometry.cpp.o] Error 1
CMakeFiles/Makefile2:388: recipe for target 'deps/klampt/CMakeFiles/gazebo_klampt.dir/all' failed
make[1]: *** [deps/klampt/CMakeFiles/gazebo_klampt.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
I am guessing it is more a CMake fault than a Docker fault, but i still don't manage to find a way around it. Maybe somebody can point out my probably fairly obvious errors.
FROM stevekuznetsov/klampt:latest
RUN sh -c 'echo "deb http://packages.ros.org/ros/ubuntu vivid main" > /etc/apt/sources.list.d/ros-latest.list'
RUN apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-key 421C365BD9FF1F717815A3895523BAEEB01FA116
RUN apt-get update
RUN apt-get -y install ros-jade-desktop-full
RUN sh -c 'echo "deb http://packages.osrfoundation.org/gazebo/ubuntu-stable vivid main" > /etc/apt/sources.list.d/gazebo-stable.list'
RUN apt-get -y install wget
RUN wget http://packages.osrfoundation.org/gazebo.key -O - | apt-key add -
RUN apt-get update
RUN wget https://bitbucket.org/osrf/release-tools/raw/default/jenkins-scripts/lib/dependencies_archive.sh -O /tmp/dependencies.sh
RUN apt-get -y install build-essential \
cmake \
mercurial
RUN hg clone https://bitbucket.org/ignitionrobotics/ign-math /tmp/ign-math
RUN cd /tmp/ign-math && \
hg up ign-math2 && \
mkdir build && \
cd build && \
cmake ../ && \
make -j4 && \
make install
RUN apt-get -y install build-essential \
cmake \
mercurial \
python \
libboost-system-dev \
libboost-filesystem-dev \
libboost-program-options-dev \
libboost-regex-dev \
libboost-iostreams-dev \
libtinyxml-dev \
libxml2-utils \
ruby-dev \
ruby
RUN hg clone https://bitbucket.org/osrf/sdformat /tmp/sdformat
RUN cd /tmp/sdformat && \
hg up sdf3 && \
mkdir build && \
cd build && \
cmake ../ && \
make -j4 && \
make install
RUN hg clone https://lpuck#bitbucket.org/arocchi/gazebo ./tmp/gazebo
RUN apt-get -y install protobuf-compiler
RUN cd /tmp/gazebo && \
hg pull && \
hg update blem_gz6_6.5.1 && \
mkdir build && \
cd build && \
cmake -DKRISLIBRARY_INCLUDE_DIR=/etc/Klampt/Library/KrisLibrary -DKRISLIBRARY_LIBRARY=etc/Klampt/Library/KrisLibrary/lib ../ && \
make -j4 && \
make install
My guess is, that this line (third last):
cmake -DKRISLIBRARY_INCLUDE_DIR=/etc/Klampt/Library/KrisLibrary -DKRISLIBRARY_LIBRARY=etc/Klampt/Library/KrisLibrary/lib ../ && \
is wrong and therefore responsible for the error. I guess i have some dependency which is set wrong. It would be highly appreciated if somebody could help me.
Edit:
The new error, and i updated the Dockerfile and the error Cmake line
Linking CXX shared library libgazebo_physics_ode.so
/usr/bin/ld: cannot find -lKrisLibrary
collect2: error: ld returned 1 exit status
make[2]: *** [gazebo/physics/ode/libgazebo_physics_ode.so.6.5.1] Error 1
gazebo/physics/ode/CMakeFiles/gazebo_physics_ode.dir/build.make:558: recipe for target 'gazebo/physics/ode/libgazebo_physics_ode.so.6.5.1' failed
CMakeFiles/Makefile2:4864: recipe for target 'gazebo/physics/ode/CMakeFiles/gazebo_physics_ode.dir/all' failed
make[1]: *** [gazebo/physics/ode/CMakeFiles/gazebo_physics_ode.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Resources