Docker Buildx feature - does not cache - docker

Question

I'm trying to build a multi-arch docker image using the buildx experimental feature. The problem I face is that whenever I do a "make" or a "make install" docker does not cache that layer.
I'm using different cmake commands depending on the arch.
What am I missing? Is what I am trying to achieve posible or should I just make multiple dockerfiles?
FROM ubuntu:18.04 as builder
#RUN add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main"
RUN set -ex && \
apt update && apt upgrade -y && \
apt install -y build-essential cmake pkg-config && \
apt install -y libjpeg-dev libtiff5-dev libpng-dev && \
apt install -y libavcodec-dev libavformat-dev libswscale-dev libv4l-dev && \
apt install -y libxvidcore-dev libx264-dev && \
apt install -y libfontconfig1-dev libcairo2-dev && \
apt install -y libgdk-pixbuf2.0-dev libpango1.0-dev && \
apt install -y libgtk2.0-dev libgtk-3-dev && \
apt install -y libatlas-base-dev gfortran && \
apt install -y libhdf5-dev libhdf5-serial-dev && \
apt install -y python3.7 python3.7-dev && \
apt install -y wget unzip git
RUN wget https://bootstrap.pypa.io/get-pip.py
RUN python get-pip.py && python3 get-pip.py
RUN rm -rf ~/.cache/pip
RUN python3 -m pip install --upgrade setuptools
RUN python3.7 get-pip.py
RUN python3.7 -m pip install --upgrade setuptools
RUN python3.7 --version
RUN python3 --version
#RUN git clone https://github.com/jasperproject/jasper-client.git jasper && \
# chmod +x jasper/jasper.py && \
# python3.7 -m pip install -r jasper/client/requirements.txt
#
#RUN apt install -y ash
RUN ARCH=`uname -m` && echo $ARCH
RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
if [ "$ARCH" == "x86_64" ]; then \
echo "x86_64" && \
apt-get install -y software-properties-common && \
add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main" && \
apt update && \
apt install -y libjasper-dev ; \
fi'
RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
if [[ "$ARCH" == arm* ]]; then \
echo "probably arm $ARCH" && \
git clone https://github.com/jasperproject/jasper-client.git jasper && \
chmod +x jasper/jasper.py && \
python3.7 -m pip install -r jasper/client/requirements.txt ; \
fi'
ARG NR_PROCS=1
ENV BUILD_FOLDER=/tmp_build/
RUN mkdir ${BUILD_FOLDER}
WORKDIR ${BUILD_FOLDER}
ADD https://github.com/opencv/opencv/archive/4.1.1.zip opencv.zip
ADD https://github.com/opencv/opencv_contrib/archive/4.1.1.zip opencv_contrib.zip
RUN set -ex && \
unzip -qq opencv.zip && \
unzip -qq opencv_contrib.zip && \
mv opencv-4.1.1 opencv && \
mv opencv_contrib-4.1.1 opencv_contrib
RUN set -ex && mkdir opencv/build && \
echo "unzipped! ";
WORKDIR ${BUILD_FOLDER}opencv/build
RUN python3.7 -m pip install numpy
RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
if [ "$ARCH" == "x86_64" ]; then \
echo "x86_64" && \
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D BUILD_opencv_python3=yes \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D OPENCV_ENABLE_NONFREE=ON \
-D CMAKE_SHARED_LINKER_FLAGS=-latomic \
-D OPENCV_EXTRA_MODULES_PATH=${BUILD_FOLDER}opencv_contrib/modules \
-D INSTALL_C_EXAMPLES=OFF \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D BUILD_EXAMPLES=OFF \
-D PYTHON3_EXECUTABLE=$(which python3.7) \
-D PYTHON_INCLUDE_DIR=$(python3.7 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-D PYTHON_INCLUDE_DIR2=$(python3.7 -c "from os.path import dirname; from distutils.sysconfig import get_config_h_filename; print(dirname(get_config_h_filename()))") \
-D PYTHON_LIBRARY=$(python3.7 -c "from distutils.sysconfig import get_config_var;from os.path import dirname,join ; print(join(dirname(get_config_var('LIBPC')),get_config_var('LDLIBRARY')))") \
-D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3.7 -c "import numpy; print(numpy.get_include())") \
-D PYTHON3_PACKAGES_PATH=$(python3.7 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
.. ; \
fi'
RUN /bin/bash -c 'set -xe && ARCH=`uname -m` && \
if [[ "$ARCH" == arm* ]]; then \
echo "probably arm $ARCH" && \
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D BUILD_opencv_python3=yes \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D OPENCV_ENABLE_NONFREE=ON \
-D ENABLE_NEON=ON \
-D ENABLE_VFPV3=ON \
-D CMAKE_SHARED_LINKER_FLAGS=-latomic \
-D OPENCV_EXTRA_MODULES_PATH=${BUILD_FOLDER}opencv_contrib/modules \
-D INSTALL_C_EXAMPLES=OFF \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D BUILD_EXAMPLES=OFF \
-D PYTHON3_EXECUTABLE=$(which python3.7) \
-D PYTHON_INCLUDE_DIR=$(python3.7 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-D PYTHON_INCLUDE_DIR2=$(python3.7 -c "from os.path import dirname; from distutils.sysconfig import get_config_h_filename; print(dirname(get_config_h_filename()))") \
-D PYTHON_LIBRARY=$(python3.7 -c "from distutils.sysconfig import get_config_var;from os.path import dirname,join ; print(join(dirname(get_config_var('LIBPC')),get_config_var('LDLIBRARY')))") \
-D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3.7 -c "import numpy; print(numpy.get_include())") \
-D PYTHON3_PACKAGES_PATH=$(python3.7 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
.. ; \
fi'
RUN set -xe && make -j${NR_PROCS} && make install && ldconfig
RUN python3.7 -c "import cv2; print(cv2.__version__)"
RUN apt clean
RUN rm -rf /var/lib/apt/lists/*

Answer 1

Remove as builder . It is used in multi-stage dockerfile.

Answer 2

There seems to be a bug, most probably related to moby/buildkit.
For more info an updates please check https://github.com/docker/buildx/issues/206