Dockerfile not correctly installing dependencies - docker

I am having an issue with installing dependencies with a Dockerfile. The image builds successfully, but when I attempt to test some of the dependencies installed in the Dockerfile below, they do not seem to be found. I am new to creating Dockerfiles so I am not sure if I am missing something. This may having something to do with environment variables? For example, I install sl as a test to confirm that something is not right. When a access a shell of my running image, running 'sl' returns "Command not found". Please let me know what I am doing wrong.
Here is my Dockerfile that I am using to build an image:
#FROM specifies the parent image from which we will construct our own custom image. This tomcat image contains a limited dist of Ubuntu, Tomcat9, and JRE11 ("temurin" is just a limited version of Java)
FROM tomcat:9-jre11-temurin
#add a maintainer label to the image
LABEL maintainer="Kerrick Cavanaugh - kerrickcavanaugh#ufl.edu"
#SciPy, Pandas, Numpy, other deps
RUN apt-get update && \
yes | apt-get -qq -y install \
git \
build-essential \
software-properties-common \
# python-dev \
python3 \
python3-dev \
python-dev \
libssl-dev \
# libffi-dev \
libxml2-dev \
libxslt1-dev \
apt-utils \
zlib1g-dev \
pip \
python3-pip \
sl \
gfortran \
libopenblas-dev \
liblapack-dev \
wget
# add /root/.local/bin to python path (causing warnings)
RUN python3 -c "import sys; sys.path.append('/root/.local/bin');"
RUN pip3 install --upgrade pip && pip3 install -vvv \
wheel \
matplotlib \
ipython \
jupyter \
sympy \
cython \
et-xmlfile==1.0.1 \
imbalanced-learn==0.5.0 \
imblearn==0.0 \
jdcal==1.4.1 \
joblib==0.14.0 \
numpy==1.16.4 \
scipy==1.3.1 \
openpyxl==2.6.4 \
pandas==0.24.2 \
python-dateutil==2.8.0 \
pytest==4.6.3 \
pytest-cov==2.7.1 \
pytz==2019.3 \
scikit-learn==0.21.3 \
six==1.12.0 \
xlrd==1.2.0
#R
RUN apt update && echo 'Y' | apt upgrade && \
#removed sudo from next 4
echo 'Y' | apt install dirmngr gnupg apt-transport-https ca-certificates software-properties-common && \
echo 'Y' | apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9 && \
echo 'Y' | add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu focal-cran40/' && \
echo 'Y' | apt install r-base && \
R --version
#MATLAB???
#FSL
COPY ./aidp_docker/fslinstaller.py /usr/local/tomcat
#printf '36' | <-- for keyboard
#removed sudo
RUN apt-get update && apt-get -qq install python2
RUN { echo; echo '36'; } | python2 /usr/local/tomcat/fslinstaller.py -E -v -q > /dev/null
#AFNI
RUN cd && \
curl -O https://raw.githubusercontent.com/afni/afni/master/src/other_builds/OS_notes.linux_ubuntu_20_64_a_admin.txt && \
curl -O https://raw.githubusercontent.com/afni/afni/master/src/other_builds/OS_notes.linux_ubuntu_20_64_b_user.tcsh && \
curl -O https://raw.githubusercontent.com/afni/afni/master/src/other_builds/OS_notes.linux_ubuntu_20_64_c_nice.tcsh && \
#removed sudo
bash OS_notes.linux_ubuntu_20_64_a_admin.txt 2>&1 | tee o.ubuntu_20_a.txt && \
tcsh OS_notes.linux_ubuntu_20_64_b_user.tcsh 2>&1 | tee o.ubuntu_20_b.txt
#dcm2niix
RUN curl -fLO https://github.com/rordenlab/dcm2niix/releases/latest/download/dcm2niix_lnx.zip
#ANTsR, ANTsRCore, ITKR
RUN git clone https://github.com/stnava/ITKR.git && \
git clone https://github.com/ANTsX/ANTsRCore.git && \
git clone https://github.com/ANTsX/ANTsR.git && \
R -e 'install.packages(c("Rcpp", "RcppEigen", "magrittr"))' && \
R CMD INSTALL ITKR && \
R CMD INSTALL ANTsRCore && \
R CMD INSTALL ANTsR
#COPY the specified folder structure and associated scripts to /usr/local/tomcat
COPY ./aidp_docker/ /usr/local/tomcat
#COPY wAIDP.war to the image
COPY ./wAIDP.war /usr/local/tomcat/webapps
#build complete
RUN echo 'Build complete!'
#runs the Docker image
CMD ["catalina.sh", "run"]

Related

GCC compilation within docker with "-j"

I got strange errors while compiling my project within the docker container when I am using -j option of make.
The errors look like this:
/root/projects/obj/linux_debug/src/myos_make/libmyos.a: error adding symbols: Cannot allocate memory
/usr/protobuf-3.9.2/lib/libprotobuf.a: error adding symbols: Bad address
Compiling the same project inside the container without -j and outside the container with -j passes ok.
The Dockerfile :
#
# This docker must be built from projects/src folder as context
#
FROM ubuntu:18.04 as base
# copy the dockerfile to make it possible to keep track of the cntent of the image
COPY ./docker_build/docker_files/Dockerfile.ubuntu18.04-builder $HOME/.
# Disable Prompt During Packages Installation
ARG DEBIAN_FRONTEND=noninteractive
# Update Ubuntu Software repository
RUN apt update -y && apt upgrade -y
# Install base utils
RUN apt install -y \
nano \
wget \
sudo \
curl \
&& apt clean -y
# Install additional packages needed for build.
RUN apt install -y \
nasm \
pkg-config \
bc \
python \
python3 \
python3-pip \
sshpass \
libapr1 \
libapr1-dev \
&& apt clean -y
RUN apt install -y \
rpm \
libaio-dev \
libnuma-dev \
numactl \
valgrind \
openssl \
libssl-dev \
ldap-utils \
libldap2-dev \
libncurses5-dev \
libncursesw5-dev \
uuid-dev \
ncurses-base \
expat \
libfuse-dev \
cmake \
build-essential \
autotools-dev \
autoconf \
automake \
doxygen \
linux-headers-4.15.0-173-generic \
g++-multilib \
lib32z1-dev \
libasan4 \
software-properties-common \
libtool \
unzip \
xsltproc \
&& apt clean -y
RUN ln -s /usr/bin/doxygen /bin/doxygen
# install GCC compiler
RUN add-apt-repository ppa:ubuntu-toolchain-r/test -y \
&& apt update -y\
&& apt install gcc-10 -y \
&& apt install g++-10 -y \
&& apt clean -y && \
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 60 --slave /usr/bin/g++ g++ /usr/bin/g++-10 && \
update-alternatives --config gcc
RUN curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output get-pip.py && \
python get-pip.py
RUN ln -sf /usr/local/bin/pip2 /usr/bin/pip
RUN pip install \
pyyaml \
jinja2 \
gcovr
FROM base as env-prepare
COPY external/protobuf /protobuf
RUN cd protobuf && ./build_proto.sh
FROM base as builder
COPY --from=env-prepare /usr/protobuf-3.9.2/ /usr/protobuf-3.9.2/
COPY --from=env-prepare /usr/protobuf-c-1.3.2/ /usr/protobuf-c-1.3.2/
COPY docker_build/docker-image-release /etc/docker-image-release
ARG UID=1000
USER ${UID}
# This entrypoint is to make everything run inside bash, so the devtools script will alway run before the command
ENTRYPOINT [ "/bin/bash", "-c" ]

node installed in docker container but not npm

I have this DockerFile
FROM docker:17.12.0-ce as static-docker-source
FROM ubuntu:18.04
FROM python:3.8-slim-buster
ARG CLOUD_SDK_VERSION=335.0.0
ENV CLOUD_SDK_VERSION=$CLOUD_SDK_VERSION
ENV PATH "$PATH:/opt/google-cloud-sdk/bin/"
COPY --from=static-docker-source /usr/local/bin/docker /usr/local/bin/docker
RUN apt-get -qqy update && apt-get install -qqy \
curl \
gcc \
apt-transport-https \
lsb-release \
openssh-client \
git \
gnupg && \
pip install crcmod && \
export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" && \
echo "deb https://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" > /etc/apt/sources.list.d/google-cloud-sdk.list && \
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
apt-get update && \
apt-get install -y google-cloud-sdk=${CLOUD_SDK_VERSION}-0 \
google-cloud-sdk-app-engine-python=${CLOUD_SDK_VERSION}-0 \
google-cloud-sdk-app-engine-python-extras=${CLOUD_SDK_VERSION}-0 \
google-cloud-sdk-cbt=${CLOUD_SDK_VERSION}-0 \
kubectl && \
gcloud --version && \
docker --version && kubectl version --client
VOLUME ["/root/.config"]
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
RUN curl -sL https://sentry.io/get-cli/ | bash -
RUN apt-get install -y nodejs build-essential pkg-config libcairo2-dev libjpeg-dev libgif-dev
ADD requirements.txt /
ADD requirements-prod.txt /
ADD submodules /
RUN python --version
RUN pip3 install --upgrade pip
RUN pip3 install -r /requirements.txt
RUN pip3 install -r /requirements-prod.txt
when I run node --version it returns v10.24.0 (I think it should be 8.x.y ....).
And when I run npm --version it returns bash: npm: command not found
Shouldn't node install npm too ?
Thanks
So I fixed it by changing my DockerFile.
I deleted FROM python:3.8-slim-buster as I read that having multiple 'FROM' on a dockerfile could cause conflict. And I installed python with apt like this:
RUN apt-get -qqy update && apt-get install -qqy \
curl \
gcc \
python3.8 \
python3-pip \
python3.8-dev \
However, it is not perfect to me because I now have python3.6.x under python 3, python 2.7.x under python and I must specified python3.8 when I need it...
My new DockerFile:
FROM docker:17.12.0-ce as static-docker-source
FROM ubuntu:18.04
ARG CLOUD_SDK_VERSION=335.0.0
ENV CLOUD_SDK_VERSION=$CLOUD_SDK_VERSION
ENV PATH "$PATH:/opt/google-cloud-sdk/bin/"
COPY --from=static-docker-source /usr/local/bin/docker
/usr/local/bin/docker
RUN apt-get -qqy update && apt-get install -qqy \
curl \
gcc \
python3.8 \
python3-pip \
python3.8-dev \
apt-transport-https \
lsb-release \
openssh-client \
git \
gnupg && \
pip3 install crcmod && \
export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" && \
echo "deb https://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" > /etc/apt/sources.list.d/google-cloud-sdk.list && \
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
apt-get update && \
apt-get install -y google-cloud-sdk=${CLOUD_SDK_VERSION}-0 \
google-cloud-sdk-app-engine-python=${CLOUD_SDK_VERSION}-0 \
google-cloud-sdk-app-engine-python-extras=${CLOUD_SDK_VERSION}-0 \
google-cloud-sdk-cbt=${CLOUD_SDK_VERSION}-0 \
kubectl && \
gcloud --version && \
docker --version && kubectl version --client
VOLUME ["/root/.config"]
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
RUN curl -sL https://sentry.io/get-cli/ | bash -
RUN apt-get install -y nodejs build-essential pkg-config libcairo2-dev
libjpeg-dev libgif-dev
RUN node --version
RUN npm --version
RUN python --version
ADD requirements.txt /
ADD requirements-prod.txt /
ADD submodules /
RUN python --version
RUN pip3 install --upgrade pip
RUN pip3 install -r /requirements.txt
RUN pip3 install -r /requirements-prod.txt

Sudo not available in docker image despite being installed in base image

I can't access Sudo from my container, despite it looking like the base image has it installed.
nistmni#ca5af2f4aace:~$ sudo echo x
bash: sudo: command not found
nistmni#ca5af2f4aace:~$ /bin/sudo
bash: /bin/sudo: No such file or directory
My Dockerfile is simple:
FROM nistmni/minc-toolkit
RUN mkdir ~/execute
COPY . ~/execute/
CMD /bin/bash
The Dockerfile for nistmni/minc-toolkit is:
FROM ubuntu:xenial
# install basic system packages
RUN apt-get -y update && \
apt-get -y dist-upgrade && \
apt-get install -y --no-install-recommends \
sudo \
build-essential g++ gfortran bc \
bison flex \
libx11-dev x11proto-core-dev \
libxi6 libxi-dev \
libxmu6 libxmu-dev libxmu-headers \
libgl1-mesa-dev libglu1-mesa-dev \
libjpeg-dev \
libssl-dev ccache libapt-inst2.0 git lsb-release \
curl ca-certificates unzip && \
apt-get autoclean && \
rm -rf /var/lib/apt/lists/*
# add user to build all tools
RUN useradd -ms /bin/bash nistmni && \
echo "nistmni ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/nistmni && \
chmod 0440 /etc/sudoers.d/nistmni
ENV PATH=/usr/lib/ccache:$PATH
# add new cmake
RUN mkdir src && \
cd src && \
curl -L --output cmake-3.14.5.tar.gz https://github.com/Kitware/CMake/releases/download/v3.14.5/cmake-3.14.5.tar.gz && \
tar zxf cmake-3.14.5.tar.gz && \
cd cmake-3.14.5 && \
./configure --prefix=/usr --no-qt-gui && \
make && \
make install && \
cd ../../ && \
rm -rf src
USER nistmni
ENV HOME /home/nistmni
WORKDIR /home/nistmni
Is this:
Some subtlety of base images I don't understand
Sudo is getting removed somewhere I'm not seeing
Not actually the Dockerfile being used to create the image
Thanks.

the command ... returned a non-zero code 100 docker

I'm trying to install OpenVino on my Raspberry using Docker.
I have this Dockerfile:
FROM raspbian/stretch
ARG INSTALL_DIR="/opt/intel/inference_engine_vpu_arm"
RUN apt-get -y update \
&& DEBIAN_FRONTEND=noninteractive && apt-get -y upgrade && apt-get autoremove && \
apt-get install -y \
apt-transport-https \
build-essential \
cmake \
cpio \
lsb-release \
pciutils \
python3.5 \
python3.5-dev \
python3-pip \
python3-setuptools \
ffmpeg \
libjpeg-dev \
libtiff5-dev \
libjasper-dev \
libpng12-dev \
libavcodec-dev \
libavformat-dev \
libswscale-dev \
libv4l-dev \
libxvidcore-dev \
libx264-dev \
libgtk2.0-dev \
libgtk-3-dev \
libatlas-base-dev \
gfortran \
libgstreamer1.0-0 \
libgstreamer-plugins-base1.0-0
RUN usermod -a -G users "$(whoami)"
COPY inference_engine_vpu_arm $INSTALL_DIR
RUN sed -i "s|<INSTALLDIR>|$INSTALL_DIR|" $INSTALL_DIR/bin/setupvars.sh && \
echo "source $INSTALL_DIR/bin/setupvars.sh" >> $HOME/.bashrc
RUN ["/bin/bash", "-c", "source $INSTALL_DIR/bin/setupvars.sh && /bin/bash $INSTALL_DIR/install_dependencies/install_NCS_udev_rules.sh"]
RUN pip3 install numpy
RUN apt autoremove -y && \
rm -rf /var/lib/apt/lists/*
CMD ["/bin/bash"]
But I have this error when I try to build:
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c apt-get -y update..... returned a non-zero code: 100
Do you have any idea?
Thanks
After a some google search it seems the error happens because the apt daemon is not able to connect to the configured repositories. This is likely since the base image was not updated for a while as i can see on docker hub.
If you not familiar with the available repositories you can generate them easily with online tools such as: https://debgen.simplylinux.ch/index.php?generate
You can put them into the docker image with a simple COPY command like
COPY sources.list /etc/apt/sources.list
where the first argument refers to a local file, the second to the docker image

PHP 7.1 Docker build will not compile with pgsql

I am using the polinux/httpd:centos repo to run Apache and PHP 7.1. The build seems to go ok. There are a few warnings related to keys, but none related to php or pgsql. The build completes successfully, but when I ssh into the container the module is not listed (php -m) and there's no extension config file in php.d.
I verified it is listed in the Dockerfile multiple times.
I can install php71-php-pgsql manually after starting the container, but then I can't restart Apache without restarting the container.
I've tried moving yum install php71-php-pgsql to the end of the Dockerfile as a separate RUN command (in addition to the original), but it reports it has already been installed, yet when I ssh into the container its not listed in the modules and no config, as mentioned above.
When I rebuild a container I stop and remove it, then run build with the no-cache option.
I'm stumped...
The Dockerfile is quite long, but I can post if that would be helpful.
Thanks.
UPDATE: Dockerfile per request...
FROM polinux/httpd:centos
ENV \
NVM_DIR="/usr/local/nvm" \
NODE_VERSION="9.2.0" \
GIT_VERSION="2.15.0" \
PHP_VERSION="71"
ADD mariadb.repo /etc/yum.repos.d/mariadb.repo
RUN \
rpm --rebuilddb && yum clean all && rm -rf /var/cache/yum && \
yum update -y && \
yum install -y \
wget \
patch \
bzip2 \
unzip \
make \
openssh-clients \
git \
MariaDB-client && \
rpm -Uvh http://rpms.remirepo.net/enterprise/remi-release-7.rpm && \
yum install -y \
php${PHP_VERSION}-php \
php${PHP_VERSION}-php-bcmath \
php${PHP_VERSION}-php-cli \
php${PHP_VERSION}-php-common \
php${PHP_VERSION}-php-devel \
php${PHP_VERSION}-php-fpm \
php${PHP_VERSION}-php-gd \
php${PHP_VERSION}-php-gmp \
php${PHP_VERSION}-php-intl \
php${PHP_VERSION}-php-json \
php${PHP_VERSION}-php-mbstring \
php${PHP_VERSION}-php-mcrypt \
php${PHP_VERSION}-php-mysqlnd \
php${PHP_VERSION}-php-pgsql \
php${PHP_VERSION}-php-opcache \
php${PHP_VERSION}-php-pdo \
php${PHP_VERSION}-php-pear \
php${PHP_VERSION}-php-process \
php${PHP_VERSION}-php-pspell \
php${PHP_VERSION}-php-xml \
php${PHP_VERSION}-php-pecl-imagick \
php${PHP_VERSION}-php-pecl-mysql \
php${PHP_VERSION}-php-pecl-uploadprogress \
php${PHP_VERSION}-php-pecl-uuid \
php${PHP_VERSION}-php-pecl-memcache \
php${PHP_VERSION}-php-pecl-memcached \
php${PHP_VERSION}-php-pecl-redis \
php${PHP_VERSION}-php-pecl-zip && \
ln -sfF /opt/remi/php${PHP_VERSION}/enable /etc/profile.d/php${PHP_VERSION}-paths.sh && \
ln -sfF /opt/remi/php${PHP_VERSION}/root/usr/bin/{pear,pecl,phar,php,php-cgi,php-config,phpize} /usr/local/bin/. && \
mv -f /etc/opt/remi/php${PHP_VERSION}/php.ini /etc/php.ini && ln -s /etc/php.ini /etc/opt/remi/php${PHP_VERSION}/php.ini && \
rm -rf /etc/php.d && mv /etc/opt/remi/php${PHP_VERSION}/php.d /etc/. && ln -s /etc/php.d /etc/opt/remi/php${PHP_VERSION}/php.d && \
yum install -y \
ImageMagick \
GraphicsMagick \
gcc \
gcc-c++ \
libffi-devel \
libpng-devel \
zlib-devel && \
yum install -y ruby ruby-devel && \
echo 'gem: --no-document' > /etc/gemrc && \
gem update --system && \
gem install bundler && \
export PROFILE=/etc/profile.d/nvm.sh && touch $PROFILE && \
curl -sSL https://raw.githubusercontent.com/creationix/nvm/v0.31.2/install.sh | bash && \
source $NVM_DIR/nvm.sh && \
nvm install $NODE_VERSION && \
nvm alias default $NODE_VERSION && \
nvm use default && \
npm install -g \
gulp \
grunt-cli \
bower \
browser-sync && \
echo -e "StrictHostKeyChecking no" >> /etc/ssh/ssh_config && \
curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer && \
chown apache /usr/local/bin/composer && composer --version && \
yum clean all && rm -rf /tmp/yum* && \
sed -i 's|SetHandler application/x-httpd-php|SetHandler "proxy:fcgi://127.0.0.1:9000"|g' /etc/httpd/conf.d/php${PHP_VERSION}-php.conf
ADD container-files /
ENV \
NODE_PATH=$NVM_DIR/versions/node/v$NODE_VERSION/lib/node_modules \
PATH=$NVM_DIR/versions/node/v$NODE_VERSION/bin:$PATH
RUN \
mkdir -p /data/tmp/php && \
chmod -R 777 /data/tmp
# Weird issue: For some reason pgsql is not installed above. May be OoO...
Manually installing worked, so adding here at the end.
RUN \
yum install php71-php-pgsql -y
Chalk this up to inexperience...
This turned out to be a problem with how I was referencing images, first when building and then running the instance. I'm still not sure I fully understand, but I think I was running the base container instead of the modified build.
For others that may run into similar problems, the 2 commands that helped me get this working are:
docker build --rm -t local/httpd-php71 .
... and then ...
docker run \
-d \
--name httpd-php71 \
--restart unless-stopped \
--net dockersubnet \
--volume /www:/var/www \
local/httpd-php71
Where 'local/httpd-php71' is my local/custom build. Before I was not using any tag reference in the build command and then I was referencing 'polinux/httpd:centos', the base, in the run command.
Thanks.

Resources