Error trying to install Python inside a Docker container - docker

I am relatively new to docker. I have an application which I want to containerize.
Below is is my docker file:
FROM ubuntu:16.04
## ENV Variables
ENV PYTHON_VERSION="3.6.5"
# Update and Install packages
RUN apt-get update -y \
&& apt-get install -y \
curl \
wget \
tar
# Install Python 3.6.5
RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tar.xz \
&& tar -xvf Python-${PYTHON_VERSION}.tar.xz \
&& cd Python-${PYTHON_VERSION} \
&& ./configure \
&& make altinstall \
&& cd / \
&& rm -rf Python-${PYTHON_VERSION}
# Install Google Cloud SDK
# Downloading gcloud package
RUN curl https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz > /tmp/google-cloud-sdk.tar.gz
# Installing the package
RUN mkdir -p /usr/local/gcloud \
&& tar -C /usr/local/gcloud -xvf /tmp/google-cloud-sdk.tar.gz \
&& /usr/local/gcloud/google-cloud-sdk/install.sh
# Adding the package path to local
ENV PATH $PATH:/usr/local/gcloud/google-cloud-sdk/bin
I am trying to install python3.6.5 version but I am receiving the following error.
020-01-09 17:26:13 (107 KB/s) - 'Python-3.6.5.tar.xz' saved [17049912/17049912]
tar (child): xz: Cannot exec: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
The command '/bin/sh -c wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tar.xz && tar -xvf Python-${PYTHON_VERSION}.tar.xz && cd Python-${PYTHON_VERSION} && ./configure && make altinstall && cd / && rm -rf Python-${PYTHON_VERSION}' returned a non-zero code: 2

Decompressing an .xz file requires the xz binary which under ubuntu is provided by the package xz-utils So You have to instal xz-utils on your image prior to decompressing an .xz file.
You can add this to your previous apt-get install run:
# Update and Install packages
RUN apt-get update -y \
&& apt-get install -y \
curl \
wget \
tar \
xz-utils
This should fix the following call to tar in the next RUN expression

Instead of trying to install Python, just start with a base image that has Python preinstalled, e.g. python:3.6-buster. This image is based on Debian Buster, which was released in 2019. Since Ubuntu is based on Debian, everything will be pretty similar, and since it's from 2019 (as opposed to Ubuntu 16.04, which is from 2016) you'll get more up-to-date software.
See https://pythonspeed.com/articles/base-image-python-docker-images/ for longer discussion.

Related

Create a Docker container for Machine Learning

I'm not a Docker expert and I'm trying to create a container for machine learning projects. This is mostly for academic purpose, as I'm studying machine learning.
I wrote a dockerfile (and a devcontainer.json file to open the container in vscode) that runs fine until I add the lines to build tensorflow. I found three problems but I don't know what I'm missing:
I got a warning about the fact that Bazel installation isn't a release version
When the building phase arrives at ./configure I can't configure through prompt question
I got an error at the very building phase with
ERROR: The project you're trying to build requires Bazel 5.3.0 (specified in /docklearning/tensorflow/.bazelversion), but it wasn't found in /usr/bin.
This is the Dockerfile:
FROM nvidia/cuda:12.0.0-runtime-ubuntu22.04 as base
ARG USER_UID=1000
#switch to non-interactive frontend
ENV DEBIAN_FRONTEND=noninteractive
WORKDIR /docklearning
ADD . /docklearning
# Install packages
RUN apt-get update -q && apt-get install -q -y --no-install-recommends \
apt-transport-https curl gnupg apt-utils wget gcc g++ npm unzip build-essential ca-certificates curl git gh \
make nano iproute2 nano openssh-client openssl procps \
software-properties-common bzip2 subversion neofetch \
fontconfig && \
curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor >bazel-archive-keyring.gpg && \
mv bazel-archive-keyring.gpg /usr/share/keyrings && \
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/bazel-archive-keyring.gpg] https://storage.googleapis.com/bazel-apt stable jdk1.8" \
| tee /etc/apt/sources.list.d/bazel.list && \
apt update && apt install -q -y bazel && \
apt-get full-upgrade -q -y && \
cd ~ && \
wget https://github.com/ryanoasis/nerd-fonts/releases/download/v2.1.0/Meslo.zip && \
mkdir -p .local/share/fonts && \
unzip Meslo.zip -d .local/share/fonts && \
cd .local/share/fonts && rm *Windows* && \
cd ~ && \
rm Meslo.zip && \
fc-cache -fv && \
apt-get install -y zsh zsh-doc chroma
# Install anaconda
RUN wget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh -O Anaconda.sh && \
/bin/bash Anaconda.sh -b -p /opt/conda && \
rm Anaconda.sh
# Install LSD for ls substitute and clean up
RUN wget https://github.com/Peltoche/lsd/releases/download/0.23.1/lsd_0.23.1_amd64.deb -P /tmp && \
dpkg -i /tmp/lsd_0.23.1_amd64.deb && \
rm /tmp/lsd_0.23.1_amd64.deb && \
apt-get autoremove -y && \
apt-get autoclean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*/apt/lists/* && \
useradd -r -m -s /bin/bash -u ${USER_UID} docklearning
# Add conda to PATH
ENV PATH=/opt/conda/bin:$PATH
ENV HOME=/home/docklearning
# Make zsh default shell
RUN chsh -s /usr/bin/zsh docklearning
# link conda to /etc/profile.d
RUN ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh
# Update Conda Base env
RUN conda update -n base --all -y && \
conda install -c conda-forge bandit opt_einsum keras-preprocessing && \
conda clean -a -q -y
# Download Tensorflow from github and build from source
RUN git clone https://github.com/tensorflow/tensorflow.git && \
cd tensorflow && \
./configure && \
bazel build --config=opt --config=cuda --cxxopt="-mavx2" //tensorflow/tools/pip_package:build_pip_package && \
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg && \
pip install /tmp/tensorflow_pkg/tensorflow-*.whl && \
cd .. && \
rm -rf tensorflow
# Create shell history file
RUN mkdir ${HOME}/zsh_history && \
chown docklearning ${HOME}/zsh_history && \
mkdir ${HOME}/.ssh
# Switch to internal user
USER docklearning
WORKDIR ${HOME}
# Copy user configuration files
COPY --chown=docklearning ./config/.aliases.sh ./
COPY --chown=docklearning ./config/.bashrc ./
COPY --chown=docklearning ./config/.nanorc ./
# Configure Zsh for internal user
ENV ZSH=${HOME}/.oh-my-zsh
ENV ZSH_CUSTOM=${ZSH}/custom
ENV ZSH_PLUGINS=${ZSH_CUSTOM}/plugins
ENV ZSH_THEMES=${ZSH_CUSTOM}/themes
RUN wget -qO- https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh | zsh || true
RUN git clone --single-branch --branch 'master' --depth 1 https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_PLUGINS}/zsh-syntax-highlighting \
&& git clone --single-branch --branch 'master' --depth 1 https://github.com/zsh-users/zsh-autosuggestions ${ZSH_PLUGINS}/zsh-autosuggestions \
&& git clone --single-branch --depth 1 https://github.com/romkatv/powerlevel10k.git ${ZSH_THEMES}/powerlevel10k
COPY --chown=docklearning ./config/.p10k.zsh ./
COPY --chown=docklearning ./config/.zshrc ./
CMD [ "/bin/zsh" ]
I wouldn't say this question about docker or TensorFlow. It is about package installation and OS configuration.
In this case, you need to fix the bazel version first by specifying it explicitly:
apt install -q -y bazel-5.3.0
But that package is quite wired and doesn't create a symlink, so you need to create it:
ln -s /bin/bazel-5.3.0 /bin/bazel
You can validate if bazel installed by running it:
`bazel --version
bazel 5.3.0`

404 error for some packages when building docker image

when building docker image for gitlab runner base image getting error as :
ERRO[2021-12-29T09:46:32Z] Application execution failed PID=6622 error="executing the script on the remote host: executing script on container with IP \"3.x.x.x\": connecting to server: connecting to server \"3.x.x.x:x\" as user \"root\": dial tcp 3.x.x.x:x: connect: connection refused"
ERROR: Job failed (system failure): prepare environment: exit status 2. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information
Dockerfile:
FROM registry.gitlab.com/tmaczukin-test-projects/fargate-driver-debian:latest
RUN apt-get install -y wget && \
apt-get install -y python3-pip && \
wget https://releases.hashicorp.com/terraform/0.12.24/terraform_0.12.24_linux_amd64.zip && \
unzip terraform_0.12.24_linux_amd64.zip
mv terraform /usr/local/bin && \
chmod -R 777 /usr/local/bin
I'm assuming the error mentioned in the title is from the apt-get install commands. You should be running an apt-get update first to get an updated package list. Otherwise apt will be looking for packages from a stale state (whenever the base image was created). You can also merge the install commands and include a cleanup of temporary files in the same step to reduce layer size.
FROM registry.gitlab.com/tmaczukin-test-projects/fargate-driver-debian:latest
RUN apt-get update && \
apt-get install -y \
python3-pip \
wget && \
wget https://releases.hashicorp.com/terraform/0.12.24/terraform_0.12.24_linux_amd64.zip && \
unzip terraform_0.12.24_linux_amd64.zip
mv terraform /usr/local/bin && \
chmod -R 777 /usr/local/bin && \
rm terraform_0.12.24_linux_amd64.zip && \
rm -rf /var/lib/apt/lists/*

Problems installing csvtk with Docker using debian:stretch-slim

I am a newbie to Docker and I am trying to install csvtk via Docker using debian:stretch-slim.
This below is my Dockerfile
FROM debian:stretch-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
jq \
perl \
python3 \
wget \
&& rm -rf /var/lib/apt/lists/*
RUN wget -qO- https://github.com/shenwei356/csvtk/releases/download/v0.23.0/csvtk_linux_amd64.tar.gz | tar -xz \
&& cp csvtk /usr/local/bin/
It fails at the csvtk step with the below error message:
Step 3/3 : RUN wget -qO- https://github.com/shenwei356/csvtk/releases/download/v0.23.0/csvtk_linux_amd64.tar.gz | tar -xz && cp csvtk /usr/local/bin/
---> Running in 0f3a0e75a5de
gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
The command '/bin/sh -c wget -qO- https://github.com/shenwei356/csvtk/releases/download/v0.23.0/csvtk_linux_amd64.tar.gz | tar -xz && cp csvtk /usr/local/bin/' returned a non-zero code: 2
I would appreciate any help/suggestions.
Thanks in advance.
wget was exiting with error code meaning 5 SSL verification failed on wget. From this answer, you just needed to install ca-certificates before wget.
This Dockerfile should build successfully:
FROM debian:stretch-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
jq \
perl \
python3 \
wget \
# added this package to help with ssl certs in Docker
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN wget -qO- https://github.com/shenwei356/csvtk/releases/download/v0.23.0/csvtk_linux_amd64.tar.gz | tar -xz \
&& cp csvtk /usr/local/bin/
As a general tip when debugging issues like these, it's likely easiest to remove the offending RUN line from your Dockerfile and then try building and running the container in a shell and manually executing the commands you want. Like this:
docker build -t test:v1 .
docker run --rm -it test:v1 /bin/bash
# run commands manually and check the full error output
While combining different RUN instructions with && is best practice to reduce the number of image layers, it's difficult to debug when building.

How to install aws-cli in docker container based on maven:3.6.3-openjdk-14 image?

I would like to install aws-cli for below images but I received below error. I tried with apk, apt but none of then did not work. Can you please help how should I update my dockerfile?
I do not want to change my base image, I need to use maven:3.6.3-openjdk-14.
sh: apt-get: command not found
FROM maven:3.6.3-openjdk-14
RUN apt-get update \
&& apt-get install -y vim jq unzip curl \
&& apt-get upgrade -y \
#install aws 2
RUN curl --silent --show-error --fail "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \
unzip awscliv2.zip && \
./aws/install && \
rm -rf awscliv2.zip
Docker image maven:3.6.3-openjdk-14 is based on Oracle Linux which uses rpm to manage packages, so apt-get is not available.
docker run -i -t maven:3.6.3-openjdk-14 -- cat /etc/os-release

docker - tomcat8 on centos 7

I am trying to deploy a war to tomcat 8 on centos 7. I am getting the following error while deploying a war into the tomcat 8.
Caused by: org.postgresql.util.PSQLException: ERROR: could not load library "/usr/pgsql-9.3/lib/plperl.so": /usr/pgsql-9.3/lib/plperl.so: undefined symbol: errcontext
Where: PL/pgSQL function db.c_scr(text,text,text,text,text,text,text,text,date,date,character,character,boolean,boolean,text,character,text,text,character,text,character varying,text,text,boolean,text,text,boolean,character,text,text,text) line 1285 at FOR over EXECUTE statement
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2412) ~[postgresql-42.0.0.jre6.jar:42.0.0.jre6]
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2125) ~[postgresql-42.0.0.jre6.jar:42.0.0.jre6]
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:297) ~[postgresql-42.0.0.jre6.jar:42.0.0.jre6]
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:428) ~[postgresql-42.0.0.jre6.jar:42.0.0.jre6]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354) ~[postgresql-42.0.0.jre6.jar:42.0.0.jre6]
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:301) ~[postgresql-42.0.0.jre6.jar:42.0.0.jre6]
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:287) ~[postgresql-42.0.0.jre6.jar:42.0.0.jre6]
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:264) ~[postgresql-42.0.0.jre6.jar:42.0.0.jre6]
at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:231) ~[postgresql-42.0.0.jre6.jar:42.0.0.jre6]
at org.apache.tomcat.dbcp.dbcp2.DelegatingStatement.executeQuery(DelegatingStatement.java:254) ~[tomcat-dbcp.jar:8.5.47]
at org.apache.tomcat.dbcp.dbcp2.DelegatingStatement.executeQuery(DelegatingStatement.java:254) ~[tomcat-dbcp.jar:8.5.47]
at org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:452) ~[spring-jdbc-4.0.6.RELEASE.jar:4.0.6.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:402) ~[spring-jdbc-4.0.6.RELEASE.jar:4.0.6.RELEASE]
FROM centos:7
RUN ./initialize_env.sh
initialize_env.sh is as follows.
yum -y update && yum clean all
yum -y install unzip wget httpd httpd-devel gcc* make && yum clean all
# Install mod_jk
curl -SL https://archive.apache.org/dist/tomcat/tomcat-connectors/jk/tomcat-connectors-1.2.40-src.tar.gz -o tomcat-connectors-1.2.40-src.tar.gz \
&& mkdir -p src/tomcat-connectors \
&& tar xzf tomcat-connectors-1.2.40-src.tar.gz -C src/tomcat-connectors --strip-components=1 \
&& cd src/tomcat-connectors/native/ \
&& ./configure --with-apxs=/usr/bin/apxs \
&& make \
&& cp apache-2.0/mod_jk.so /usr/lib64/httpd/modules/ \
&& ./libtool --finish /usr/lib64/httpd/modules/ \
&& cd / \
&& rm -rf src/ \
&& rm -f tomcat-connectors-1.2.40-src.tar.gz
echo "Downloading tomcat"
cd /apps/
wget http://apache.mirrors.pair.com/tomcat/tomcat-8/v8.5.47/bin/apache-tomcat-8.5.47.zip
unzip apache-tomcat-8.5.47.zip
mv apache-tomcat-8.5.47 tomcat
chmod -R 755 tomcat
chmod -R +x tomcat/bin
yum -y install postgresql-plperl
yum -y install java
wget https://download.postgresql.org/pub/repos/yum/9.3/redhat/rhel-7-x86_64/pgdg-centos93-9.3-3.noarch.rpm
rpm -ivh pgdg-centos93-9.3-3.noarch.rpm
yum -y install postgresql93 postgresql93-server postgresql93-libs postgresql93-contrib postgresql93-devel postgresql93-plperl
After this installation, i see this output.
sh-4.2# ls -l /usr/pgsql-9.3/lib/plperl.so
-rwxr-xr-x 1 root root 87304 Nov 7 2018 /usr/pgsql-9.3/lib/plperl.so
I need to run Apache Web server with Tomcat. I have a postgres jdbc driver (version 9) under $tomcat_home/lib directory. I even tried a higher version of jdbc driver, but it is still throwing the same error.
does anybody know why?
Is there anything else that I need to do?

Resources