install opencv in existing dockerfile in gitlab - opencv

enter image description herei added below code to my docker file and the job run successfully but , in modules java in not available.
first FROM python:3.9
LABEL mantainer="Baher Elnaggar eng.baher77#gmail.com"
WORKDIR /opt/build
ENV OPENCV_VERSION="4.5.1"
RUN apt-get -qq update
&& apt-get -qq install -y --no-install-recommends
build-essential
cmake
git
wget
unzip
yasm
pkg-config
libswscale-dev
libtbb2
libtbb-dev
libjpeg-dev
libpng-dev
libtiff-dev
libopenjp2-7-dev
libavformat-dev
libpq-dev
&& pip install numpy
&& wget -q https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip -O opencv.zip \
&& unzip -qq opencv.zip -d /opt \
&& rm -rf opencv.zip \
&& cmake \
-D BUILD_TIFF=ON \
-D BUILD_opencv_java=OFF \
-D WITH_CUDA=OFF \
-D WITH_OPENGL=ON \
-D WITH_OPENCL=ON \
-D WITH_IPP=ON \
-D WITH_TBB=ON \
-D WITH_EIGEN=ON \
-D WITH_V4L=ON \
-D BUILD_TESTS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=$(python3.9 -c "import sys; print(sys.prefix)") \
-D PYTHON_EXECUTABLE=$(which python3.9) \
-D PYTHON_INCLUDE_DIR=$(python3.9 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-D PYTHON_PACKAGES_PATH=$(python3.9 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
/opt/opencv-${OPENCV_VERSION} \
&& make -j$(nproc) \
&& make install \
&& rm -rf /opt/build/* \
&& rm -rf /opt/opencv-${OPENCV_VERSION} \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get -qq autoremove \
&& apt-get -qq clean
Then i added Java_home
RUN apt-get update &&
apt-get install -y openjdk-8-jdk &&
apt-get install -y ant &&
apt-get clean &&
rm -rf /var/lib/apt/lists/ &&
rm -rf /var/cache/oracle-jdk8-installer;
enter code here
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ RUN export JAVA_HOME
**But still job run successfully but my requirements is not satisfied . Can any one tell me to install opencv with java_home 8
My requirement is to add opencv layer to my docker image . i need to install OPENCV , i am added this code in my existing dockerfile in gitlab.**
but Old build getting deployed when you run the pipeline

Related

Install libjpeg.so.8 Debian 11 inside Docker

I tried to integrate an application - QCPump - inside an existing Docker, with an other application - QAtrack+. The goal is to use QCPump inside QAtrack+.
The application code seems to be integrated but when I launch it, I have an error :
ImportError: libjpeg.so.8: cannot open shared object file: No such file or directory
The error is raised by the wxPython package.
Okay, so I have to install it. Unfortunately, my Docker linux is Debian 11, and Debian seems to grab this package several years ago. So, after some reseach, I found that this package is "replaced" - for Debian - by libjpeg-dev. So, I did it. And same result ...
I found the code of the librairy (wxPython) and a docker part has done for Debian 10 : https://github.com/wxWidgets/Phoenix/blob/master/docker/build/debian-10/Dockerfile
I took this part and integrated it in my DockerFile :
RUN apt-get install -y \
freeglut3 \
freeglut3-dev \
libgl1-mesa-dev \
libglu1-mesa-dev \
libgstreamer-plugins-base1.0-dev \
libgtk-3-dev \
libjpeg-dev \
libnotify-dev \
libsdl2-dev \
libsm-dev \
libtiff-dev \
libwebkit2gtk-4.0-dev \
libxtst-dev; \
apt-get clean;
But same ...
In some forum, people mentionned the LD have to be update. I tried this way but I am not pretty sure :
RUN export LD_LIBRARY_PATH=/usr/local/lib
And to be honest, I am not sure this is the problem and so the solution here...
Any idea about this problem ?
Following, my complete DockerFile if you need it ;)
FROM python:3.6
RUN echo 'deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main' > /etc/apt/sources.list.d/pgdg.list
RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
RUN apt-get update && apt-get install -y \
cron postgresql-client-10 cifs-utils dos2unix \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get install tzdata
ENV TZ 'Europe/Paris'
RUN dpkg-reconfigure -f noninteractive tzdata
RUN touch /root/.is_inside_docker
RUN pip install virtualenv
RUN date "+%H:%M:%S %d/%m/%y"
RUN apt-get -q update && \
apt-get install -yq chromium && \
rm -rf /var/lib/apt/lists/*
RUN apt-get update -y && apt-get install -y libsdl2-ttf-2.0-0 && \
apt-get update -y && apt-get install -y libjpeg-dev libaio1 libaio-dev && \
wget -q -O /tmp/libpng12.deb http://mirrors.kernel.org/ubuntu/pool/main/libp/libpng/libpng12- 0_1.2.54-1ubuntu1_amd64.deb \
&& dpkg -i /tmp/libpng12.deb \
&& rm /tmp/libpng12.deb \
&& apt-get install -y \
freeglut3 \
freeglut3-dev \
libgl1-mesa-dev \
libglu1-mesa-dev \
libgstreamer-plugins-base1.0-dev \
libgtk-3-dev \
libjpeg-dev \
libnotify-dev \
libsdl2-dev \
libsm-dev \
libtiff-dev \
libwebkit2gtk-4.0-dev \
libxtst-dev; \
apt-get clean;
RUN export LD_LIBRARY_PATH=/usr/local/lib
WORKDIR /usr/src/qatrackplus

Docker Buildx feature - does not cache

I'm trying to build a multi-arch docker image using the buildx experimental feature. The problem I face is that whenever I do a "make" or a "make install" docker does not cache that layer.
I'm using different cmake commands depending on the arch.
What am I missing? Is what I am trying to achieve posible or should I just make multiple dockerfiles?
FROM ubuntu:18.04 as builder
#RUN add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main"
RUN set -ex && \
apt update && apt upgrade -y && \
apt install -y build-essential cmake pkg-config && \
apt install -y libjpeg-dev libtiff5-dev libpng-dev && \
apt install -y libavcodec-dev libavformat-dev libswscale-dev libv4l-dev && \
apt install -y libxvidcore-dev libx264-dev && \
apt install -y libfontconfig1-dev libcairo2-dev && \
apt install -y libgdk-pixbuf2.0-dev libpango1.0-dev && \
apt install -y libgtk2.0-dev libgtk-3-dev && \
apt install -y libatlas-base-dev gfortran && \
apt install -y libhdf5-dev libhdf5-serial-dev && \
apt install -y python3.7 python3.7-dev && \
apt install -y wget unzip git
RUN wget https://bootstrap.pypa.io/get-pip.py
RUN python get-pip.py && python3 get-pip.py
RUN rm -rf ~/.cache/pip
RUN python3 -m pip install --upgrade setuptools
RUN python3.7 get-pip.py
RUN python3.7 -m pip install --upgrade setuptools
RUN python3.7 --version
RUN python3 --version
#RUN git clone https://github.com/jasperproject/jasper-client.git jasper && \
# chmod +x jasper/jasper.py && \
# python3.7 -m pip install -r jasper/client/requirements.txt
#
#RUN apt install -y ash
RUN ARCH=`uname -m` && echo $ARCH
RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
if [ "$ARCH" == "x86_64" ]; then \
echo "x86_64" && \
apt-get install -y software-properties-common && \
add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main" && \
apt update && \
apt install -y libjasper-dev ; \
fi'
RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
if [[ "$ARCH" == arm* ]]; then \
echo "probably arm $ARCH" && \
git clone https://github.com/jasperproject/jasper-client.git jasper && \
chmod +x jasper/jasper.py && \
python3.7 -m pip install -r jasper/client/requirements.txt ; \
fi'
ARG NR_PROCS=1
ENV BUILD_FOLDER=/tmp_build/
RUN mkdir ${BUILD_FOLDER}
WORKDIR ${BUILD_FOLDER}
ADD https://github.com/opencv/opencv/archive/4.1.1.zip opencv.zip
ADD https://github.com/opencv/opencv_contrib/archive/4.1.1.zip opencv_contrib.zip
RUN set -ex && \
unzip -qq opencv.zip && \
unzip -qq opencv_contrib.zip && \
mv opencv-4.1.1 opencv && \
mv opencv_contrib-4.1.1 opencv_contrib
RUN set -ex && mkdir opencv/build && \
echo "unzipped! ";
WORKDIR ${BUILD_FOLDER}opencv/build
RUN python3.7 -m pip install numpy
RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
if [ "$ARCH" == "x86_64" ]; then \
echo "x86_64" && \
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D BUILD_opencv_python3=yes \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D OPENCV_ENABLE_NONFREE=ON \
-D CMAKE_SHARED_LINKER_FLAGS=-latomic \
-D OPENCV_EXTRA_MODULES_PATH=${BUILD_FOLDER}opencv_contrib/modules \
-D INSTALL_C_EXAMPLES=OFF \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D BUILD_EXAMPLES=OFF \
-D PYTHON3_EXECUTABLE=$(which python3.7) \
-D PYTHON_INCLUDE_DIR=$(python3.7 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-D PYTHON_INCLUDE_DIR2=$(python3.7 -c "from os.path import dirname; from distutils.sysconfig import get_config_h_filename; print(dirname(get_config_h_filename()))") \
-D PYTHON_LIBRARY=$(python3.7 -c "from distutils.sysconfig import get_config_var;from os.path import dirname,join ; print(join(dirname(get_config_var('LIBPC')),get_config_var('LDLIBRARY')))") \
-D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3.7 -c "import numpy; print(numpy.get_include())") \
-D PYTHON3_PACKAGES_PATH=$(python3.7 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
.. ; \
fi'
RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
if [[ "$ARCH" == arm* ]]; then \
echo "probably arm $ARCH" && \
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D BUILD_opencv_python3=yes \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D OPENCV_ENABLE_NONFREE=ON \
-D ENABLE_NEON=ON \
-D ENABLE_VFPV3=ON \
-D CMAKE_SHARED_LINKER_FLAGS=-latomic \
-D OPENCV_EXTRA_MODULES_PATH=${BUILD_FOLDER}opencv_contrib/modules \
-D INSTALL_C_EXAMPLES=OFF \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D BUILD_EXAMPLES=OFF \
-D PYTHON3_EXECUTABLE=$(which python3.7) \
-D PYTHON_INCLUDE_DIR=$(python3.7 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-D PYTHON_INCLUDE_DIR2=$(python3.7 -c "from os.path import dirname; from distutils.sysconfig import get_config_h_filename; print(dirname(get_config_h_filename()))") \
-D PYTHON_LIBRARY=$(python3.7 -c "from distutils.sysconfig import get_config_var;from os.path import dirname,join ; print(join(dirname(get_config_var('LIBPC')),get_config_var('LDLIBRARY')))") \
-D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3.7 -c "import numpy; print(numpy.get_include())") \
-D PYTHON3_PACKAGES_PATH=$(python3.7 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
.. ; \
fi'
RUN set -xe && make -j${NR_PROCS} && make install && ldconfig
RUN python3.7 -c "import cv2; print(cv2.__version__)"
RUN apt clean
RUN rm -rf /var/lib/apt/lists/*
Remove as builder . It is used in multi-stage dockerfile.
There seems to be a bug, most probably related to moby/buildkit.
For more info an updates please check https://github.com/docker/buildx/issues/206

Sudo not available in docker image despite being installed in base image

I can't access Sudo from my container, despite it looking like the base image has it installed.
nistmni#ca5af2f4aace:~$ sudo echo x
bash: sudo: command not found
nistmni#ca5af2f4aace:~$ /bin/sudo
bash: /bin/sudo: No such file or directory
My Dockerfile is simple:
FROM nistmni/minc-toolkit
RUN mkdir ~/execute
COPY . ~/execute/
CMD /bin/bash
The Dockerfile for nistmni/minc-toolkit is:
FROM ubuntu:xenial
# install basic system packages
RUN apt-get -y update && \
apt-get -y dist-upgrade && \
apt-get install -y --no-install-recommends \
sudo \
build-essential g++ gfortran bc \
bison flex \
libx11-dev x11proto-core-dev \
libxi6 libxi-dev \
libxmu6 libxmu-dev libxmu-headers \
libgl1-mesa-dev libglu1-mesa-dev \
libjpeg-dev \
libssl-dev ccache libapt-inst2.0 git lsb-release \
curl ca-certificates unzip && \
apt-get autoclean && \
rm -rf /var/lib/apt/lists/*
# add user to build all tools
RUN useradd -ms /bin/bash nistmni && \
echo "nistmni ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/nistmni && \
chmod 0440 /etc/sudoers.d/nistmni
ENV PATH=/usr/lib/ccache:$PATH
# add new cmake
RUN mkdir src && \
cd src && \
curl -L --output cmake-3.14.5.tar.gz https://github.com/Kitware/CMake/releases/download/v3.14.5/cmake-3.14.5.tar.gz && \
tar zxf cmake-3.14.5.tar.gz && \
cd cmake-3.14.5 && \
./configure --prefix=/usr --no-qt-gui && \
make && \
make install && \
cd ../../ && \
rm -rf src
USER nistmni
ENV HOME /home/nistmni
WORKDIR /home/nistmni
Is this:
Some subtlety of base images I don't understand
Sudo is getting removed somewhere I'm not seeing
Not actually the Dockerfile being used to create the image
Thanks.

the command ... returned a non-zero code 100 docker

I'm trying to install OpenVino on my Raspberry using Docker.
I have this Dockerfile:
FROM raspbian/stretch
ARG INSTALL_DIR="/opt/intel/inference_engine_vpu_arm"
RUN apt-get -y update \
&& DEBIAN_FRONTEND=noninteractive && apt-get -y upgrade && apt-get autoremove && \
apt-get install -y \
apt-transport-https \
build-essential \
cmake \
cpio \
lsb-release \
pciutils \
python3.5 \
python3.5-dev \
python3-pip \
python3-setuptools \
ffmpeg \
libjpeg-dev \
libtiff5-dev \
libjasper-dev \
libpng12-dev \
libavcodec-dev \
libavformat-dev \
libswscale-dev \
libv4l-dev \
libxvidcore-dev \
libx264-dev \
libgtk2.0-dev \
libgtk-3-dev \
libatlas-base-dev \
gfortran \
libgstreamer1.0-0 \
libgstreamer-plugins-base1.0-0
RUN usermod -a -G users "$(whoami)"
COPY inference_engine_vpu_arm $INSTALL_DIR
RUN sed -i "s|<INSTALLDIR>|$INSTALL_DIR|" $INSTALL_DIR/bin/setupvars.sh && \
echo "source $INSTALL_DIR/bin/setupvars.sh" >> $HOME/.bashrc
RUN ["/bin/bash", "-c", "source $INSTALL_DIR/bin/setupvars.sh && /bin/bash $INSTALL_DIR/install_dependencies/install_NCS_udev_rules.sh"]
RUN pip3 install numpy
RUN apt autoremove -y && \
rm -rf /var/lib/apt/lists/*
CMD ["/bin/bash"]
But I have this error when I try to build:
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c apt-get -y update..... returned a non-zero code: 100
Do you have any idea?
Thanks
After a some google search it seems the error happens because the apt daemon is not able to connect to the configured repositories. This is likely since the base image was not updated for a while as i can see on docker hub.
If you not familiar with the available repositories you can generate them easily with online tools such as: https://debgen.simplylinux.ch/index.php?generate
You can put them into the docker image with a simple COPY command like
COPY sources.list /etc/apt/sources.list
where the first argument refers to a local file, the second to the docker image

How do i clean up / minimize the storage usage of my container from building opencv in dockerfile

Given my docker file
FROM nvidia/cuda:8.0-cudnn6-devel-ubuntu14.04
RUN apt-get update
# disable interactive functions
ENV DEBIAN_FRONTEND noninteractive
#################Install MiniConda and other dependencies##########
ENV CONDA_DIR /opt/conda
ENV PATH $CONDA_DIR/bin:$PATH
ENV OPENBLAS_NUM_THREADS $(nproc)
RUN mkdir -p $CONDA_DIR && \
echo export PATH=$CONDA_DIR/bin:'$PATH' > /etc/profile.d/conda.sh && \
apt-get update -y && \
apt-get install -y \
wget \
git \
g++ \
graphviz \
software-properties-common \
python-software-properties \
python3-dev \
libhdf5-dev \
libopenblas-dev \
liblapack-dev \
libblas-dev \
libtbb2 \
libtbb-dev && \
apt-get -qq install -y \
build-essential \
unzip \
cmake \
pkg-config \
libjpeg8-dev \
libtiff4-dev \
libjasper-dev \
libpng12-dev \
libgtk2.0-dev \
libavcodec-dev \
libavformat-dev \
libswscale-dev \
libv4l-dev \
libatlas-base-dev \
gfortran && \
rm -rf /var/lib/apt/lists/* && \
wget --quiet https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
/bin/bash /Miniconda3-latest-Linux-x86_64.sh -f -b -p $CONDA_DIR && \
rm Miniconda3-latest-Linux-x86_64.sh
#########################MPI###########################
RUN cd /tmp && \
wget "https://www.open-mpi.org/software/ompi/v2.1/downloads/openmpi-2.1.1.tar.gz" && \
tar xzf openmpi-2.1.1.tar.gz && \
cd openmpi-2.1.1 && \
./configure --with-cuda && make -j"$(nproc)" install # && ldconfig
#######################NCCL###########################
ENV CPATH /usr/local/cuda/include:/usr/local/include:$CPATH
RUN cd /usr/local && git clone https://github.com/NVIDIA/nccl.git && cd nccl && \
sed -i '/NVCC_GENCODE ?=/a \ -gencode=arch=compute_30,code=sm_30 \\' Makefile && \
make CUDA_HOME=/usr/local/cuda -j"$(nproc)" && \
make install && ldconfig
######### Compile for devices with cuda compute compatibility 3 (e.g. GRID K520 on aws) second line above
# UNCOMMENT line below to compile for GPUs with cuda compute compatibility 3.0
# sed -i '/NVCC_GENCODE ?=/a \ -gencode=arch=compute_30,code=sm_30 \\' Makefile && \
##########
###################Setup User##########################
ENV NB_USER chainer
ENV NB_UID 1000
RUN useradd -m -s /bin/bash -N -u $NB_UID $NB_USER -g sudo -p $(perl -e'print crypt("chainer", "aa")') && \
mkdir -p $CONDA_DIR && \
chown chainer $CONDA_DIR -R && \
mkdir -p /src && \
chown -R chainer /src
# mkdir -p /opencv && \
# chown -R chainer /opencv && \
# mkdir -p /opencv_contrib && \
# chown -R chainer /opencv_contrib
####################Python 3#########################
ARG python_version=3.5.2
RUN conda install -y python=${python_version} && \
pip install -U pip && \
conda install Pillow scikit-learn notebook pandas matplotlib mkl nose pyyaml six h5py && \
pip install numpy && \
pip install chainer && \
pip install chainercv && \
# pip install opencv-python && \
pip install pyyaml && \
conda clean -yt
# pip install -U pip && \
#
# conda install Pillow scikit-learn notebook pandas matplotlib mkl nose pyyaml six h5py && \
#
#
# pip install mpi4py && \
# pip install cython && \
#
# pip install chainer && \
# pip install chainercv && \
# pip install chainermn && \
ENV PYTHONPATH $CONDA_DIR/lib/python3.5/site-packages/:$PYTHONPATH
ENV PYTHONPATH /src/:$PYTHONPATH
WORKDIR /src
##OPENCV##
RUN wget https://github.com/Itseez/opencv/archive/3.3.0.zip -O opencv3.zip && \
unzip -q opencv3.zip && mv /src/opencv-3.3.0 /opencv && \
wget https://github.com/Itseez/opencv_contrib/archive/3.3.0.zip -O opencv_contrib3.zip && \
unzip -q opencv_contrib3.zip && mv /src/opencv_contrib-3.3.0 /opencv_contrib
RUN rm -r -f /opencv/build && mkdir /opencv/build && \
cd /opencv/build && \
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D BUILD_PYTHON_SUPPORT=ON \
-D CMAKE_INSTALL_PREFIX=$CONDA_DIR \
-D PYTHON3_LIBRARY=$CONDA_DIR/lib/python3.5 \
-D PYTHON3_INCLUDE_DIRS=$CONDA_DIR/include/python3.5m \
-D PYTHON3_EXECUTABLE=$CONDA_DIR/bin/python3 \
-D PYTHON3_PACKAGES_PATH=$CONDA_DIR/lib/python3.5/site-packages \
-D WITH_TBB=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D OPENCV_EXTRA_MODULES_PATH=/opencv_contrib/modules \
-D BUILD_EXAMPLES=OFF \
-D BUILD_NEW_PYTHON_SUPPORT=ON \
-D WITH_IPP=OFF \
-D WITH_V4L=ON .. && \
make -j"$(nproc)" && \
make install && \
ldconfig
######################################################
USER chainer
It ends up at whooping 15GB in size.
What can I do to reduce the size in terms of all the selfbuild stuff happening? can some be deleted again?

Resources