Dockerfile build RUN cmd fails but works from interactive - docker

I'm running into an issue where building a Dockerfile fails at a 'make' command but if I run -it the container and make again it succeeds. Specifically the make command seems to be not seeing a file:
make[2]: *** No rule to make target /usr/local/nvidia/lib64/libcuda.so, needed by bin/gpu-example-alpha_comp. Stop.
But this is based on nvidia/cuda:8.0-cudnn5-devel and libcuda.so is there, i.e. if I run -it the container:
root#cf5637b85467:/opt/opencv/opencv/build# ll /usr/local/nvidia/lib64/libcuda*.*
lrwxrwxrwx 1 999 999 17 Nov 21 20:42 /usr/local/nvidia/lib64/libcuda.so -> libcuda.so.370.28
lrwxrwxrwx 1 999 999 17 Nov 21 20:42 /usr/local/nvidia/lib64/libcuda.so.1 -> libcuda.so.370.28
-rw-r--r-- 2 root root 8219624 Sep 2 02:25 /usr/local/nvidia/lib64/libcuda.so.370.28
and make completes without issue if I manually run it (I tried running from both /bin/bash and /bin/sh).
The only thing I can see is the user & group 999 of the .so link, but I don't know if that's relevant. In the worst case I could commit after building, but this seems a bad approach going forward. Any suggestions? Full Dockerfile below:
FROM nvidia/cuda:8.0-cudnn5-devel
RUN apt-get update &&\
apt-get install -y --no-install-recommends \
build-essential \
cmake \
curl \
gfortran \
git \
libatlas-base-dev \
libavcodec-dev \
libavformat-dev \
libfreetype6-dev \
libgtk2.0-dev \
libjasper-dev \
libjpeg8-dev \
libpng12-dev \
libswscale-dev \
libtiff4-dev \
libv4l-dev \
pkg-config \
python2.7-dev \
&& \
apt-get clean && \
apt-get autoremove && \
rm -rf /var/lib/apt/lists/*
# Get pip and pip installs
RUN curl -O https://bootstrap.pypa.io/get-pip.py && \
python get-pip.py && \
rm get-pip.py
RUN pip --no-cache-dir --timeout=60 install \
numpy
# Install opencv and opencv_contrib
WORKDIR /opt/opencv
RUN git clone --depth 1 https://github.com/opencv/opencv.git && \
git clone --depth 1 https://github.com/opencv/opencv_contrib.git
WORKDIR /opt/opencv/opencv/build
RUN cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_C_EXAMPLES=ON \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D OPENCV_EXTRA_MODULES_PATH=/opt/opencv/opencv_contrib/modules \
-D CUDA_CUDA_LIBRARY=/usr/local/nvidia/lib64/libcuda.so \
-D BUILD_EXAMPLES=ON ..
RUN make -j8
RUN make install

Related

Shiny app (leafviz) in docker container not loading in browser

I am building a docker container to use the Leafcutter and Leafviz analysis tools. The final visualization commanded from an R file uses shiny app, it works fine on my machine but when ran in the container I can't connect to it.
I run the container with the following line of code:
docker run -dit --name leaf --rm -p 1234:1234 leafcutter
All is in order when I check with docker ps:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
783f550df965 leafcutter "bash" 2 minutes ago Up 2 minutes 0.0.0.0:1234->1234/tcp, :::1234->1234/tcp leaf
Executing the pipeline in the container calls several snakefiles, and the last rule of the last one calls the R file:
Rscript /path/to/run_leafviz.R -i {input}
That script calls several functions before calling leafviz() which basically just runs the shiny app, I've modified the options like so:
shiny::runApp(launch.browser=FALSE, appDir = system.file("application", package = "leafviz"), host = "0.0.0.0", port = 1234)
All the shiny files and packages are installed with the R leafviz package. When I execute my container all goes fine, the output shows "Listening on http://0.0.0.0:1234", but when I try to connect I get the "connexion failed" page.
My Dockerfile is fairly large but here are the most relevant parts:
FROM ubuntu:22.04
ENV DEBIAN_FRONTEND noninteractive
#Install Ubuntu packages
RUN apt-get update && apt-get install \
-y cmake -y curl \
-y default-jre \
-y gdebi-core \
-y less -y libarchive13 -y libbz2-dev -y libcairo2-dev -y libcurl4-openssl-dev -y libgsl-dev \
-y liblzma-dev -y libncurses5-dev -y libncursesw5-dev -y libssl-dev -y libxml2-dev -y libxt-dev \
-y pandoc -y pandoc-citeproc -y python-pip \
-y tabix \
-y unzip \
-y wget \
-y zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*
#Install conda, snakemake, samtools, regtools...
#[...]
#Install R
LABEL org.label-schema.license="GPL-2.0" \
org.label-schema.vcs-url="https://github.com/rocker-org/r-apt" \
org.label-schema.vendor="Rocker Project" \
maintainer="Dirk Eddelbuettel <edd#debian.org>"
RUN useradd docker \
&& mkdir /home/docker \
&& chown docker:docker /home/docker \
&& addgroup docker staff
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
littler r-base r-base-dev r-recommended r-cran-docopt
RUN ln -s /usr/lib/R/site-library/littler/examples/install.r /usr/local/bin/install.r \
&& ln -s /usr/lib/R/site-library/littler/examples/install2.r /usr/local/bin/install2.r \
&& ln -s /usr/lib/R/site-library/littler/examples/installGithub.r /usr/local/bin/installGithub.r \
&& ln -s /usr/lib/R/site-library/littler/examples/testInstalled.r /usr/local/bin/testInstalled.r \
&& rm -rf /tmp/downloaded_packages/ /tmp/*.rds \
&& rm -rf /var/lib/apt/lists/*
#Install shiny server (I have tried with and without this RUN command)
RUN wget --no-verbose https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/VERSION -O "version.txt" && \
VERSION=$(cat version.txt) && \
wget --no-verbose "https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/shiny-server-$VERSION-amd64.deb" -O ss-latest.deb && \
gdebi -n ss-latest.deb && \
rm -f version.txt ss-latest.deb
EXPOSE 1234
#Install R packages
COPY install_packages.R /tmp/install_packages.R
RUN Rscript /tmp/install_packages.R
CMD ["bash"]
I've been learning docker quite chaotically so I don't know if some elements are clashing, but I have exposed the container port and published it to the host and no error message seems to indicate there is a problem coming from somewhere else. I would really appreciate some help.

install opencv in existing dockerfile in gitlab

enter image description herei added below code to my docker file and the job run successfully but , in modules java in not available.
first FROM python:3.9
LABEL mantainer="Baher Elnaggar eng.baher77#gmail.com"
WORKDIR /opt/build
ENV OPENCV_VERSION="4.5.1"
RUN apt-get -qq update
&& apt-get -qq install -y --no-install-recommends
build-essential
cmake
git
wget
unzip
yasm
pkg-config
libswscale-dev
libtbb2
libtbb-dev
libjpeg-dev
libpng-dev
libtiff-dev
libopenjp2-7-dev
libavformat-dev
libpq-dev
&& pip install numpy
&& wget -q https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip -O opencv.zip \
&& unzip -qq opencv.zip -d /opt \
&& rm -rf opencv.zip \
&& cmake \
-D BUILD_TIFF=ON \
-D BUILD_opencv_java=OFF \
-D WITH_CUDA=OFF \
-D WITH_OPENGL=ON \
-D WITH_OPENCL=ON \
-D WITH_IPP=ON \
-D WITH_TBB=ON \
-D WITH_EIGEN=ON \
-D WITH_V4L=ON \
-D BUILD_TESTS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=$(python3.9 -c "import sys; print(sys.prefix)") \
-D PYTHON_EXECUTABLE=$(which python3.9) \
-D PYTHON_INCLUDE_DIR=$(python3.9 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-D PYTHON_PACKAGES_PATH=$(python3.9 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
/opt/opencv-${OPENCV_VERSION} \
&& make -j$(nproc) \
&& make install \
&& rm -rf /opt/build/* \
&& rm -rf /opt/opencv-${OPENCV_VERSION} \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get -qq autoremove \
&& apt-get -qq clean
Then i added Java_home
RUN apt-get update &&
apt-get install -y openjdk-8-jdk &&
apt-get install -y ant &&
apt-get clean &&
rm -rf /var/lib/apt/lists/ &&
rm -rf /var/cache/oracle-jdk8-installer;
enter code here
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/ RUN export JAVA_HOME
**But still job run successfully but my requirements is not satisfied . Can any one tell me to install opencv with java_home 8
My requirement is to add opencv layer to my docker image . i need to install OPENCV , i am added this code in my existing dockerfile in gitlab.**
but Old build getting deployed when you run the pipeline

Sudo not available in docker image despite being installed in base image

I can't access Sudo from my container, despite it looking like the base image has it installed.
nistmni#ca5af2f4aace:~$ sudo echo x
bash: sudo: command not found
nistmni#ca5af2f4aace:~$ /bin/sudo
bash: /bin/sudo: No such file or directory
My Dockerfile is simple:
FROM nistmni/minc-toolkit
RUN mkdir ~/execute
COPY . ~/execute/
CMD /bin/bash
The Dockerfile for nistmni/minc-toolkit is:
FROM ubuntu:xenial
# install basic system packages
RUN apt-get -y update && \
apt-get -y dist-upgrade && \
apt-get install -y --no-install-recommends \
sudo \
build-essential g++ gfortran bc \
bison flex \
libx11-dev x11proto-core-dev \
libxi6 libxi-dev \
libxmu6 libxmu-dev libxmu-headers \
libgl1-mesa-dev libglu1-mesa-dev \
libjpeg-dev \
libssl-dev ccache libapt-inst2.0 git lsb-release \
curl ca-certificates unzip && \
apt-get autoclean && \
rm -rf /var/lib/apt/lists/*
# add user to build all tools
RUN useradd -ms /bin/bash nistmni && \
echo "nistmni ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/nistmni && \
chmod 0440 /etc/sudoers.d/nistmni
ENV PATH=/usr/lib/ccache:$PATH
# add new cmake
RUN mkdir src && \
cd src && \
curl -L --output cmake-3.14.5.tar.gz https://github.com/Kitware/CMake/releases/download/v3.14.5/cmake-3.14.5.tar.gz && \
tar zxf cmake-3.14.5.tar.gz && \
cd cmake-3.14.5 && \
./configure --prefix=/usr --no-qt-gui && \
make && \
make install && \
cd ../../ && \
rm -rf src
USER nistmni
ENV HOME /home/nistmni
WORKDIR /home/nistmni
Is this:
Some subtlety of base images I don't understand
Sudo is getting removed somewhere I'm not seeing
Not actually the Dockerfile being used to create the image
Thanks.

the command ... returned a non-zero code 100 docker

I'm trying to install OpenVino on my Raspberry using Docker.
I have this Dockerfile:
FROM raspbian/stretch
ARG INSTALL_DIR="/opt/intel/inference_engine_vpu_arm"
RUN apt-get -y update \
&& DEBIAN_FRONTEND=noninteractive && apt-get -y upgrade && apt-get autoremove && \
apt-get install -y \
apt-transport-https \
build-essential \
cmake \
cpio \
lsb-release \
pciutils \
python3.5 \
python3.5-dev \
python3-pip \
python3-setuptools \
ffmpeg \
libjpeg-dev \
libtiff5-dev \
libjasper-dev \
libpng12-dev \
libavcodec-dev \
libavformat-dev \
libswscale-dev \
libv4l-dev \
libxvidcore-dev \
libx264-dev \
libgtk2.0-dev \
libgtk-3-dev \
libatlas-base-dev \
gfortran \
libgstreamer1.0-0 \
libgstreamer-plugins-base1.0-0
RUN usermod -a -G users "$(whoami)"
COPY inference_engine_vpu_arm $INSTALL_DIR
RUN sed -i "s|<INSTALLDIR>|$INSTALL_DIR|" $INSTALL_DIR/bin/setupvars.sh && \
echo "source $INSTALL_DIR/bin/setupvars.sh" >> $HOME/.bashrc
RUN ["/bin/bash", "-c", "source $INSTALL_DIR/bin/setupvars.sh && /bin/bash $INSTALL_DIR/install_dependencies/install_NCS_udev_rules.sh"]
RUN pip3 install numpy
RUN apt autoremove -y && \
rm -rf /var/lib/apt/lists/*
CMD ["/bin/bash"]
But I have this error when I try to build:
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c apt-get -y update..... returned a non-zero code: 100
Do you have any idea?
Thanks
After a some google search it seems the error happens because the apt daemon is not able to connect to the configured repositories. This is likely since the base image was not updated for a while as i can see on docker hub.
If you not familiar with the available repositories you can generate them easily with online tools such as: https://debgen.simplylinux.ch/index.php?generate
You can put them into the docker image with a simple COPY command like
COPY sources.list /etc/apt/sources.list
where the first argument refers to a local file, the second to the docker image

PHP 7.1 Docker build will not compile with pgsql

I am using the polinux/httpd:centos repo to run Apache and PHP 7.1. The build seems to go ok. There are a few warnings related to keys, but none related to php or pgsql. The build completes successfully, but when I ssh into the container the module is not listed (php -m) and there's no extension config file in php.d.
I verified it is listed in the Dockerfile multiple times.
I can install php71-php-pgsql manually after starting the container, but then I can't restart Apache without restarting the container.
I've tried moving yum install php71-php-pgsql to the end of the Dockerfile as a separate RUN command (in addition to the original), but it reports it has already been installed, yet when I ssh into the container its not listed in the modules and no config, as mentioned above.
When I rebuild a container I stop and remove it, then run build with the no-cache option.
I'm stumped...
The Dockerfile is quite long, but I can post if that would be helpful.
Thanks.
UPDATE: Dockerfile per request...
FROM polinux/httpd:centos
ENV \
NVM_DIR="/usr/local/nvm" \
NODE_VERSION="9.2.0" \
GIT_VERSION="2.15.0" \
PHP_VERSION="71"
ADD mariadb.repo /etc/yum.repos.d/mariadb.repo
RUN \
rpm --rebuilddb && yum clean all && rm -rf /var/cache/yum && \
yum update -y && \
yum install -y \
wget \
patch \
bzip2 \
unzip \
make \
openssh-clients \
git \
MariaDB-client && \
rpm -Uvh http://rpms.remirepo.net/enterprise/remi-release-7.rpm && \
yum install -y \
php${PHP_VERSION}-php \
php${PHP_VERSION}-php-bcmath \
php${PHP_VERSION}-php-cli \
php${PHP_VERSION}-php-common \
php${PHP_VERSION}-php-devel \
php${PHP_VERSION}-php-fpm \
php${PHP_VERSION}-php-gd \
php${PHP_VERSION}-php-gmp \
php${PHP_VERSION}-php-intl \
php${PHP_VERSION}-php-json \
php${PHP_VERSION}-php-mbstring \
php${PHP_VERSION}-php-mcrypt \
php${PHP_VERSION}-php-mysqlnd \
php${PHP_VERSION}-php-pgsql \
php${PHP_VERSION}-php-opcache \
php${PHP_VERSION}-php-pdo \
php${PHP_VERSION}-php-pear \
php${PHP_VERSION}-php-process \
php${PHP_VERSION}-php-pspell \
php${PHP_VERSION}-php-xml \
php${PHP_VERSION}-php-pecl-imagick \
php${PHP_VERSION}-php-pecl-mysql \
php${PHP_VERSION}-php-pecl-uploadprogress \
php${PHP_VERSION}-php-pecl-uuid \
php${PHP_VERSION}-php-pecl-memcache \
php${PHP_VERSION}-php-pecl-memcached \
php${PHP_VERSION}-php-pecl-redis \
php${PHP_VERSION}-php-pecl-zip && \
ln -sfF /opt/remi/php${PHP_VERSION}/enable /etc/profile.d/php${PHP_VERSION}-paths.sh && \
ln -sfF /opt/remi/php${PHP_VERSION}/root/usr/bin/{pear,pecl,phar,php,php-cgi,php-config,phpize} /usr/local/bin/. && \
mv -f /etc/opt/remi/php${PHP_VERSION}/php.ini /etc/php.ini && ln -s /etc/php.ini /etc/opt/remi/php${PHP_VERSION}/php.ini && \
rm -rf /etc/php.d && mv /etc/opt/remi/php${PHP_VERSION}/php.d /etc/. && ln -s /etc/php.d /etc/opt/remi/php${PHP_VERSION}/php.d && \
yum install -y \
ImageMagick \
GraphicsMagick \
gcc \
gcc-c++ \
libffi-devel \
libpng-devel \
zlib-devel && \
yum install -y ruby ruby-devel && \
echo 'gem: --no-document' > /etc/gemrc && \
gem update --system && \
gem install bundler && \
export PROFILE=/etc/profile.d/nvm.sh && touch $PROFILE && \
curl -sSL https://raw.githubusercontent.com/creationix/nvm/v0.31.2/install.sh | bash && \
source $NVM_DIR/nvm.sh && \
nvm install $NODE_VERSION && \
nvm alias default $NODE_VERSION && \
nvm use default && \
npm install -g \
gulp \
grunt-cli \
bower \
browser-sync && \
echo -e "StrictHostKeyChecking no" >> /etc/ssh/ssh_config && \
curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer && \
chown apache /usr/local/bin/composer && composer --version && \
yum clean all && rm -rf /tmp/yum* && \
sed -i 's|SetHandler application/x-httpd-php|SetHandler "proxy:fcgi://127.0.0.1:9000"|g' /etc/httpd/conf.d/php${PHP_VERSION}-php.conf
ADD container-files /
ENV \
NODE_PATH=$NVM_DIR/versions/node/v$NODE_VERSION/lib/node_modules \
PATH=$NVM_DIR/versions/node/v$NODE_VERSION/bin:$PATH
RUN \
mkdir -p /data/tmp/php && \
chmod -R 777 /data/tmp
# Weird issue: For some reason pgsql is not installed above. May be OoO...
Manually installing worked, so adding here at the end.
RUN \
yum install php71-php-pgsql -y
Chalk this up to inexperience...
This turned out to be a problem with how I was referencing images, first when building and then running the instance. I'm still not sure I fully understand, but I think I was running the base container instead of the modified build.
For others that may run into similar problems, the 2 commands that helped me get this working are:
docker build --rm -t local/httpd-php71 .
... and then ...
docker run \
-d \
--name httpd-php71 \
--restart unless-stopped \
--net dockersubnet \
--volume /www:/var/www \
local/httpd-php71
Where 'local/httpd-php71' is my local/custom build. Before I was not using any tag reference in the build command and then I was referencing 'polinux/httpd:centos', the base, in the run command.
Thanks.

Resources