Docker build fails with dependency issues even though I am satisfying them - docker

I'm looking for ideas as to why this is failing.
I've done the same steps on my host and they work well, the packages I'm calling out provide the needed dependencies.
Build Command:
docker build -t testbuild .
DockerFile:
FROM registry.redhat.io/rhel7:latest
RUN yum install -y yum-utils
RUN yum-config-manager --enable \
EPEL_7_EPEL_7 \
Community_mysql-connectors-community \
Community_mysql-tools-community \
Community_mysql57-community \
rhel-7-server-extras-rpms/x86_64 \
rhel-7-server-optional-rpms/7Server/x86_64 \
rhel-7-server-rh-common-rpms/7Server/x86_64 \
rhel-7-server-rpms/7Server/x86_64 \
rhel-server-rhscl-7-rpms/7Server/x86_64
RUN true \
&& yum install -y \
cairo \
yum-utils\
collectd \
openldap-devel \
rrdtool \
gcc \
rrdtool-devel \
pyrrd \
rrdtool-python \
python-ldap \
wget \
pycairo-devel \
pycairo \
python-devel \
collectd-nginx \
findutils \
rrdtool \
logrotate \
memcached \
nginx \
nodejs \
npm \
redis \
pkgconfig \
sqlite \
expect \
git \
python3\
python3-devel\
libffi-devel \
postgresql-devel \
postgresql-devel \
mysql-community-client \
mysql-community-libs \
mysql-community-common \
mysql-community-libs-compat
RUN yum clean all
RUN pip3 install \
virtualenv\
# && /usr/bin/easy_install virtualenv \
&& /usr/local/bin/virtualenv /opt/graphite \
&& . /opt/graphite/bin/activate \
&& pip3 install \
PyMySQL \
django==1.11.24 \
django-statsd-mozilla \
fadvise \
gunicorn \
msgpack-python \
redis \
rrdtool \
python-ldap \
mysqlclient \
psycopg2
The error I am getting is ERROR: Command errored out with exit status 1:
command: /opt/graphite/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] =
'"'"'/tmp/pip-install-67zlal6a/rrdtool/setup.py'"'"';
file='"'"'/tmp/pip-install-67zlal6a/rrdtool/setup.py'"'"';f=getattr(tokenize,
'"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))'
egg_info --egg-base pip-egg-info
cwd: /tmp/pip-install-67zlal6a/rrdtool/
Complete output (5 lines):
/tmp/tmp_python_rrdtoolfr1k715h/test_rrdtool.c:2:17: fatal error: rrd.h: No such file or directory
#include
^
compilation terminated.
Error: Unable to compile the binary module. Do you have the rrdtool header and libraries installed?
---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for
full command output.
But I've ensured the files exist with the yum-installed packages.
I've commented out this package and then I get a mysql_config error which I also don't get on my current test host. Which leads me to believe something is going wrong earlier in the build.
Any Ideas?

Don't have access to RHEL repository, so I tried with centos 7 and was having same issues.
But installing these additional packages helped to successfully build image:
RUN yum install -y epel-release
RUN yum install -y python36-pip
RUN yum install -y mysql mysql-devel
RUN yum install -y python36-devel
Here is final DockerFile:
FROM centos:7
RUN yum install -y yum-utils
RUN yum-config-manager --enable \
EPEL_7_EPEL_7 \
Community_mysql-connectors-community \
Community_mysql-tools-community \
Community_mysql57-community \
rhel-7-server-extras-rpms/x86_64 \
rhel-7-server-optional-rpms/7Server/x86_64 \
rhel-7-server-rh-common-rpms/7Server/x86_64 \
rhel-7-server-rpms/7Server/x86_64 \
rhel-server-rhscl-7-rpms/7Server/x86_64
RUN true \
&& yum install -y \
cairo \
yum-utils\
collectd \
openldap-devel \
rrdtool \
gcc \
rrdtool-devel \
pyrrd \
rrdtool-python \
python-ldap \
wget \
pycairo-devel \
pycairo \
python-devel \
collectd-nginx \
findutils \
rrdtool \
logrotate \
memcached \
nginx \
nodejs \
npm \
redis \
pkgconfig \
sqlite \
expect \
git \
python3\
python3-devel\
libffi-devel \
postgresql-devel \
postgresql-devel \
mysql-community-client \
mysql-community-libs \
mysql-community-common \
mysql-community-libs-compat
RUN yum install -y epel-release
RUN yum install -y python36-pip
RUN yum install -y mysql mysql-devel
RUN yum install -y python36-devel
RUN yum clean all
RUN pip3 install \
virtualenv\
&& /usr/local/bin/virtualenv /opt/graphite \
&& . /opt/graphite/bin/activate \
&& pip3 install \
PyMySQL \
django==1.11.24 \
django-statsd-mozilla \
fadvise \
gunicorn \
msgpack-python \
redis \
rrdtool \
python-ldap \
mysqlclient \
psycopg2

Related

GCC compilation within docker with "-j"

I got strange errors while compiling my project within the docker container when I am using -j option of make.
The errors look like this:
/root/projects/obj/linux_debug/src/myos_make/libmyos.a: error adding symbols: Cannot allocate memory
/usr/protobuf-3.9.2/lib/libprotobuf.a: error adding symbols: Bad address
Compiling the same project inside the container without -j and outside the container with -j passes ok.
The Dockerfile :
#
# This docker must be built from projects/src folder as context
#
FROM ubuntu:18.04 as base
# copy the dockerfile to make it possible to keep track of the cntent of the image
COPY ./docker_build/docker_files/Dockerfile.ubuntu18.04-builder $HOME/.
# Disable Prompt During Packages Installation
ARG DEBIAN_FRONTEND=noninteractive
# Update Ubuntu Software repository
RUN apt update -y && apt upgrade -y
# Install base utils
RUN apt install -y \
nano \
wget \
sudo \
curl \
&& apt clean -y
# Install additional packages needed for build.
RUN apt install -y \
nasm \
pkg-config \
bc \
python \
python3 \
python3-pip \
sshpass \
libapr1 \
libapr1-dev \
&& apt clean -y
RUN apt install -y \
rpm \
libaio-dev \
libnuma-dev \
numactl \
valgrind \
openssl \
libssl-dev \
ldap-utils \
libldap2-dev \
libncurses5-dev \
libncursesw5-dev \
uuid-dev \
ncurses-base \
expat \
libfuse-dev \
cmake \
build-essential \
autotools-dev \
autoconf \
automake \
doxygen \
linux-headers-4.15.0-173-generic \
g++-multilib \
lib32z1-dev \
libasan4 \
software-properties-common \
libtool \
unzip \
xsltproc \
&& apt clean -y
RUN ln -s /usr/bin/doxygen /bin/doxygen
# install GCC compiler
RUN add-apt-repository ppa:ubuntu-toolchain-r/test -y \
&& apt update -y\
&& apt install gcc-10 -y \
&& apt install g++-10 -y \
&& apt clean -y && \
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 60 --slave /usr/bin/g++ g++ /usr/bin/g++-10 && \
update-alternatives --config gcc
RUN curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output get-pip.py && \
python get-pip.py
RUN ln -sf /usr/local/bin/pip2 /usr/bin/pip
RUN pip install \
pyyaml \
jinja2 \
gcovr
FROM base as env-prepare
COPY external/protobuf /protobuf
RUN cd protobuf && ./build_proto.sh
FROM base as builder
COPY --from=env-prepare /usr/protobuf-3.9.2/ /usr/protobuf-3.9.2/
COPY --from=env-prepare /usr/protobuf-c-1.3.2/ /usr/protobuf-c-1.3.2/
COPY docker_build/docker-image-release /etc/docker-image-release
ARG UID=1000
USER ${UID}
# This entrypoint is to make everything run inside bash, so the devtools script will alway run before the command
ENTRYPOINT [ "/bin/bash", "-c" ]

docker buildkit opencv compilation

I have a Dockerfile which compiles opencv for quite a while. Recently I decided to try out buildkit (for cross compilation). However, with buildkit the opencv compilation fails. With the following error:
#9 130.0 [ 87%] Linking CXX shared library ../../lib/libopencv_imgcodecs.so
#9 130.5 [ 87%] Built target opencv_imgcodecs
#9 130.5 [ 87%] Generating src/moc_window_QT.cpp
#9 130.5 [ 87%] Generating qrc_window_QT.cpp
#9 130.5 standard input:0: Note: No relevant classes found. No output generated.
#9 130.5 RCC: Error in '/opencv/modules/highgui/src/window_QT.qrc': Cannot find file 'files_Qt/Milky/48/28.png'
#9 130.5 make[2]: *** [modules/highgui/CMakeFiles/opencv_highgui.dir/build.make:78: modules/highgui/qrc_window_QT.cpp] Error 1
#9 130.5 make[1]: *** [CMakeFiles/Makefile2:1684: modules/highgui/CMakeFiles/opencv_highgui.dir/all] Error 2
#9 130.5 make: *** [Makefile:163: all] Error 2
I tried stopping the build after the git clone to see if the file files_Qt/Milky/48/28.png exists inside the container and it does. Moreover, if I start the compilation process from the interactive shell in the container, the compilation process succeeds.
So it seems the compilation only fails if it is part of the docker build.
Does anybody know more details about how buildkit works and if this could have an impact on the compilation process?
Here is a minimal example of the build command and Dockerfile:
DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -t "buildkit_error:latest" -f Dockerfile .
FROM ubuntu:20.04
WORKDIR /
# Setup apt repositories
RUN \
apt update && \
apt install -y --no-install-recommends \
wget \
software-properties-common && \
add-apt-repository ppa:deadsnakes/ppa && \
apt install -y --fix-broken && \
apt update && \
apt upgrade -y && \
apt dist-upgrade -y && \
apt remove -y python3 && \
apt clean && \
apt autoremove -y && \
apt autoclean -y
# Install apt packages
# ORDER ALPHABETICALLY!
RUN \
apt install -y --no-install-recommends \
build-essential \
clang-10 \
cmake \
cpio \
git \
libdrm-dev \
libcairo2-dev \
libomp-10-dev \
llvm-10-dev \
openssh-client \
pkg-config \
python3.6-dev \
python3.6-distutils\
python3.6-minimal \
python3-pip \
qt5-default
# Install pip dependencies
# ORDER ALPHABETICALLY!
RUN python3.6 -m pip install --no-cache-dir \
numpy==1.16.1 \
setuptools==59.6.0 \
wheel==0.37.1
RUN \
ln -sf /usr/bin/python3.6 /usr/bin/python3 && \
ln -sf /usr/bin/python3 /usr/bin/python
ENV opencv=3.4.3
RUN git clone --branch $opencv --depth 1 https://github.com/opencv/opencv.git
RUN ls -al /opencv/modules/highgui/src/files_Qt/Milky/48
# build opencv from source
RUN \
mkdir opencv/build && cd opencv/build && cmake -DBUILD_TIFF=ON \
-DCMAKE_C_COMPILER=clang-10 \
-DCMAKE_CXX_COMPILER=clang++-10 \
-DENABLE_FAST_MATH=ON \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_opencv_java=OFF \
-DWITH_CUDA=OFF \
-DENABLE_CXX11=ON \
-DWITH_OPENGL=ON \
-DWITH_OPENCL=ON \
-DWITH_IPP=ON \
-DWITH_TBB=ON \
-DWITH_EIGEN=ON \
-DWITH_V4L=OFF \
-DWITH_GTK=ON \
-DWITH_GTK_2_X=ON \
-DBUILD_PERF_TESTS=OFF \
-DBUILD_opencv_apps=OFF \
-DBUILD_JAVA=OFF \
-DBUILD_PROTOBUF=OFF \
-DBUILD_PACKAGE=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_opencv=OFF \
-DBUILD_opencv_dnn=OFF \
-DBUILD_opencv_java_bindings_generator=OFF \
-DBUILD_opencv_shape=OFF \
-DBUILD_opencv_stitching=OFF \
-DBUILD_opencv_superres=OFF \
-DBUILD_opencv_ts=OFF \
-DBUILD_opencv_video=OFF \
-DBUILD_opencv_videoio=OFF \
-DBUILD_opencv_videostab=OFF \
-DBUILD_opencv_world=OFF \
-DBUILD_opencv_ml=OFF \
-DBUILD_opencv_photo=OFF \
-DWITH_1394=OFF \
-DWITH_FFMPEG=OFF \
-DWITH_GSTREAMER=OFF \
-DWITH_IMGCODEC_HDR=OFF \
-DWITH_IMGCODEC_PXM=OFF \
-DWITH_IMGCODEC_SUNRASTER=OFF \
-DWITH_JASPER=OFF \
-DWITH_OPENCVAMDBLAS=OFF \
-DWITH_OPENCVAMDFFT=OFF \
-DWITH_OPENEXR=OFF \
-DWITH_OPENNI=OFF \
-DWITH_OPENNI2=OFF \
-DWITH_OPENVX=OFF \
-DWITH_VTK=OFF \
-DWITH_V4L=OFF \
-DWITH_QUIRC=OFF \
-DWITH_QT=ON \
-DBUILD_opencv_calib3d=ON \
-DBUILD_opencv_core=ON \
-DBUILD_opencv_python3=ON \
-DBUILD_opencv_flann=ON \
-DBUILD_opencv_python_binding_generator=ON \
-DBUILD_opencv_highgui=ON \
-DBUILD_opencv_imgcodecs=ON \
-DBUILD_opencv_features2d=ON \
-DBUILD_opencv_imgproc=ON \
-DWITH_OPENMP=ON \
-DWITH_PNG=ON \
-DWITH_HALIDE=OFF .. && \
make -j6 && make install
After a lot of investigation, I finally stumbled upon this ticket:
https://gitlab.archlinux.org/archlinux/archlinux-docker/-/issues/32
The comments gave me the idea that the kernel might be the problem. I upgraded the kernel and that seemed to have resolved the problem.
Hopefully this helps someone else in the future.

the command ... returned a non-zero code 100 docker

I'm trying to install OpenVino on my Raspberry using Docker.
I have this Dockerfile:
FROM raspbian/stretch
ARG INSTALL_DIR="/opt/intel/inference_engine_vpu_arm"
RUN apt-get -y update \
&& DEBIAN_FRONTEND=noninteractive && apt-get -y upgrade && apt-get autoremove && \
apt-get install -y \
apt-transport-https \
build-essential \
cmake \
cpio \
lsb-release \
pciutils \
python3.5 \
python3.5-dev \
python3-pip \
python3-setuptools \
ffmpeg \
libjpeg-dev \
libtiff5-dev \
libjasper-dev \
libpng12-dev \
libavcodec-dev \
libavformat-dev \
libswscale-dev \
libv4l-dev \
libxvidcore-dev \
libx264-dev \
libgtk2.0-dev \
libgtk-3-dev \
libatlas-base-dev \
gfortran \
libgstreamer1.0-0 \
libgstreamer-plugins-base1.0-0
RUN usermod -a -G users "$(whoami)"
COPY inference_engine_vpu_arm $INSTALL_DIR
RUN sed -i "s|<INSTALLDIR>|$INSTALL_DIR|" $INSTALL_DIR/bin/setupvars.sh && \
echo "source $INSTALL_DIR/bin/setupvars.sh" >> $HOME/.bashrc
RUN ["/bin/bash", "-c", "source $INSTALL_DIR/bin/setupvars.sh && /bin/bash $INSTALL_DIR/install_dependencies/install_NCS_udev_rules.sh"]
RUN pip3 install numpy
RUN apt autoremove -y && \
rm -rf /var/lib/apt/lists/*
CMD ["/bin/bash"]
But I have this error when I try to build:
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c apt-get -y update..... returned a non-zero code: 100
Do you have any idea?
Thanks
After a some google search it seems the error happens because the apt daemon is not able to connect to the configured repositories. This is likely since the base image was not updated for a while as i can see on docker hub.
If you not familiar with the available repositories you can generate them easily with online tools such as: https://debgen.simplylinux.ch/index.php?generate
You can put them into the docker image with a simple COPY command like
COPY sources.list /etc/apt/sources.list
where the first argument refers to a local file, the second to the docker image

running curl on dockerfile but getting error because of missing dependency

Trying to run curl from within my container but getting this error, i already copied into the container after installing curl so not sure what I am missing
FROM debian:stretch
RUN apt-get update && \
apt-get install -y --no-install-recommends \
build-essential \
cmake \
curl \
make \
wget \
unzip \
bash \
jq \
libssl1.0-dev \
libasl-dev \
libsasl2-dev \
pkg-config \
libsystemd-dev \
zlib1g-dev
COPY /lib/x86_64-linux-gnu/libcom_err.so* /lib/x64_64-linux-gnu/`
docker run -it --env LD_LIBRARY_PATH="/usr/lib/x64_64-linux-gnu/" myimage .
#curl
curl: error while loading shared libraries: libcom_err.so.2: cannot open shared object file: No such file or directory
Instead of trying to copy it, just add it to the apt install phase, and let apt figure out how to get it and resolve dependencies for it:
RUN apt-get update && \
apt-get install -y --no-install-recommends \
build-essential \
cmake \
curl \
make \
wget \
unzip \
bash \
jq \
libcomerr2 \
libssl1.0-dev \
libasl-dev \
libsasl2-dev \
pkg-config \
libsystemd-dev \
zlib1g-dev

How to install ffmpeg in a docker container

I'm using aeneas python module which uses ffmpeg. I install ffmpeg in the dockerfile as follows:
RUN apt-get update && apt-get install -y ffmpeg
Now when I run the program, it fails with: aeneas.ffprobewrapper.FFPROBEPathError: Unable to call the 'ffprobe' ffprobe executable : [Errno 2] No such file or directory: 'ffprobe' and aeneas.audiofile.AudioFileProbeError: Unable to call ffprobe executable. So my question is, how can I successfully use ffmpeg in a docker container? I'm running Ubuntu 16.04.
Install fresh ffmpeg version from sources in docker container on Debian 9/10/Ubuntu.
You can replace 4.2.2 version to any other available on https://ffmpeg.org/releases/
# Compile and install fresh ffmpeg from sources:
# See: https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu
RUN apt-get update -qq && apt-get -y install \
autoconf \
automake \
build-essential \
cmake \
git-core \
libass-dev \
libfreetype6-dev \
libsdl2-dev \
libtool \
libva-dev \
libvdpau-dev \
libvorbis-dev \
libxcb1-dev \
libxcb-shm0-dev \
libxcb-xfixes0-dev \
pkg-config \
texinfo \
wget \
zlib1g-dev \
nasm \
yasm \
libx265-dev \
libnuma-dev \
libvpx-dev \
libmp3lame-dev \
libopus-dev \
libx264-dev \
libfdk-aac-dev
RUN mkdir -p ~/ffmpeg_sources ~/bin && cd ~/ffmpeg_sources && \
wget -O ffmpeg-4.2.2.tar.bz2 https://ffmpeg.org/releases/ffmpeg-4.2.2.tar.bz2 && \
tar xjvf ffmpeg-4.2.2.tar.bz2 && \
cd ffmpeg-4.2.2 && \
PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure \
--prefix="$HOME/ffmpeg_build" \
--pkg-config-flags="--static" \
--extra-cflags="-I$HOME/ffmpeg_build/include" \
--extra-ldflags="-L$HOME/ffmpeg_build/lib" \
--extra-libs="-lpthread -lm" \
--bindir="$HOME/bin" \
--enable-libfdk-aac \
--enable-gpl \
--enable-libass \
--enable-libfreetype \
--enable-libmp3lame \
--enable-libopus \
--enable-libvorbis \
--enable-libvpx \
--enable-libx264 \
--enable-libx265 \
--enable-nonfree && \
PATH="$HOME/bin:$PATH" make -j8 && \
make install -j8 && \
hash -r
RUN mv ~/bin/ffmpeg /usr/local/bin && mv ~/bin/ffprobe /usr/local/bin && mv ~/bin/ffplay /usr/local/bin
It could be faulty library, then this should help:
apt-get install ffmpeg libavcodec-extra-53
If not it can be path problem:
$ffmpeg = FFMpeg\FFMpeg::create(array(
'ffmpeg.binaries' => '/usr/bin/ffmpeg',
'ffprobe.binaries' => '/usr/bin/ffprobe'));
The similar issue resolved here: https://github.com/PHP-FFMpeg/PHP-FFMpeg/issues/172
in the last message on the bottom of the website.
You can do it if you have linux. Or anywhere you can install X (haven't tested it). Basic idea is that linux is using X11, so you can fart out X data to your host machine's X server.
Dockerfile
FROM ubuntu:latest
RUN apt-get update -qq && apt-get install ffmpeg -y
ENTRYPOINT ["ffplay"]
Build that with
docker build -t ffplay:latest .
play.sh
docker run \
--rm \
-u `id -u` \
-e DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix \
-v "$HOME/Desktop:/media" \
ffplay:latest \
-i /media/test.mp4

Resources