How do i dockerize a sagemath server? - docker

i've been using sagemath for a few weeks now, and i've been experiencing some downtime. i saw that i am able to make my own sagemath server, so what i'm trying to do is that i want to dockerize it but am unable to do so, i've followed the documentation on the github but im unable to run docker build . to run the Dockerfile. Is it possible to dockerize a sagemath server?
FROM node:alpine
RUN ln -s /usr/bin/nodejs /usr/bin/node
RUN npm install -g requirejs
RUN apk update && apk add git
RUN git clone https://github.com/sagemath/sage.git
#temporarily run nocache flag to speed up the runtime
#install python?
RUN apk add --no-cache python3 py3-pip
#install gcc
RUN apk add --no-cache gcc musl-dev
#install bash to process the next few commands
RUN apk add --no-cache bash
RUN /bin/bash -c " \
pushd sage; \
./bootstrap/; \
./configure --enable-download-from-upstream-url/; \
make; \
popd; \
sage/sage -pip install lockfile; \
sage/sage -pip install sockjs-tornado; \
sage/sage -pip install sqlalchemy; \
sage/sage -pip install paramiko; \
"
RUN git clone https://github.com/sagemath/sagecell.git
RUN /bin/bash -c " \
pushd sagecell; \
"
RUN git submodule update --init --recursive \
RUN ../sage/sage -sh -c \
make; \
../sage/sage web_server.py [-p 8888] \
The command prompt just exploded me with a bunch of error messages, this is just a small amount of it
#13 173.2 ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-t4f6ko9s/cffi_04531dcfd0fd4757914fa4410902e57d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-t4f6ko9s/cffi_04531dcfd0fd4757914fa4410902e57d/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install
--record /tmp/pip-record-ldckhbb6/install-record.txt --single-version-externally-managed --compile --install-headers /usr/include/python3.8/cffi Check the logs for full command output.
------
executor failed running [/bin/sh -c /bin/bash -c " pushd sage; ./bootstrap/; ./configure --enable-download-from-upstream-url/; make; popd; sage/sage -pip install lockfile; sage/sage -pip install sockjs-tornado; sage/sage -pip install sqlalchemy; sage/sage -pip install paramiko; "]: exit code: 1

Related

Can't compile Rust project in conda environment in Docker

I'm trying to build the following sightglass benchmarking suite/ Dockerfile:
FROM ubuntu:22.04
RUN echo 'APT::Install-Suggests "0";' >> /etc/apt/apt.conf.d/00-docker
RUN echo 'APT::Install-Recommends "0";' >> /etc/apt/apt.conf.d/00-docker
RUN DEBIAN_FRONTEND=noninteractive \
apt-get update \
&& apt-get install -y python3 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /usr/src
ADD rust-benchmark rust-benchmark
WORKDIR /usr/src/rust-benchmark
RUN apt update --yes
RUN apt install clang lldb lld wget curl git xz-utils bzip2 --yes
RUN apt-get install --reinstall ca-certificates --yes
RUN apt-get install libgl1-mesa-glx libegl1-mesa libxrandr2 libxrandr2 libxss1 libxcursor1 libxcomposite1 libasound2 libxi6 libxtst6 -y
RUN mkdir /usr/local/share/ca-certificates/cacert.org
RUN wget -P /usr/local/share/ca-certificates/cacert.org http://www.cacert.org/certs/root.crt http://www.cacert.org/certs/class3.crt
RUN update-ca-certificates
RUN git config --global http.sslCAinfo /etc/ssl/certs/ca-certificates.crt
RUN wget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh --no-check-certificate
RUN cd / && find . -name cargo
RUN chmod +x Anaconda3-2022.10-Linux-x86_64.sh
RUN yes yes | ./Anaconda3-2022.10-Linux-x86_64.sh
RUN rm Anaconda3-2022.10-Linux-x86_64.sh
RUN echo "export PATH=./yes/bin:$PATH" >> ~/.bashrc
ENV CONDA ./yes/bin/
ENV PATH="${CONDA}:${PATH}"
RUN ln -s ./yes/bin/conda /usr/local/bin/conda
RUN eval $(conda shell.bash hook)
RUN conda init bash
RUN conda update --all
RUN cd / && find . -name cargo
RUN conda create -c conda-forge -n rustenv rust
RUN activate rustenv
SHELL ["./yes/bin/conda", "run", "-n", "rustenv", "/bin/bash", "-c"]
RUN rustc --version
ENV GIT_SSL_NO_VERIFY=1
RUN git clone https://github.com/emscripten-core/emsdk.git
RUN cd emsdk && git pull
RUN chmod +x ./emsdk/emsdk
RUN ./emsdk/emsdk install latest
RUN ./emsdk/emsdk activate latest
RUN chmod +x ./emsdk/emsdk_env.sh
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
RUN cd emsdk && source ./emsdk_env.sh
RUN ./emsdk/emsdk_env.sh
ENV EMSDK ./emsdk
ENV EMSCRIPTEN=${EMSDK}/emscripten/sdk
ENV EM_DATA ${EMSDK}/.data
ENV EM_CONFIG ${EMSDK}/.emscripten
ENV EM_CACHE ${EM_DATA}/cache
ENV EM_PORTS ${EM_DATA}/ports
ENV PATH="${EMSDK}:${EMSDK}/emscripten/sdk:${EMSDK}/llvm/clang/bin:${EMSDK}/node/current/bin:${EMSDK}/binaryen/bin:${PATH}"
RUN curl https://sh.rustup.rs -ksSf | sh -s -- -y
RUN chmod +x $HOME/.cargo/env
RUN $HOME/.cargo/env
ENV RUST ~/.cargo/bin
ENV PATH="${RUST}:${PATH}"
RUN rustup default nightly
RUN rustup target add wasm32-wasi --toolchain nightly
RUN ./yes/envs/rustenv/bin/cargo build --release --target wasm32-wasi
RUN cp target/wasm32-wasi/release/bls-381-wasm-benchmark.wasm /benchmark.wasm
The build process always aborts on the compile step with the following error:
error[E0463]: can't find crate for `core`
|
= note: the `wasm32-wasi` target may not be installed
= help: consider downloading the target with `rustup target add wasm32-wasi`
error[E0463]: can't find crate for `compiler_builtins`
My full setup can be found here: https://github.com/achimcc/arkworks-wasmtime-benchmarks/tree/main/benchmarks/bls12-381
It seems you are compiling something whose target is wasm32-wasi.
Rust can compile source codes for different "targets", but only few of them was enabled by default.
To install the wasm32-wasi target, you can run this command:
rustup target add wasm32-wasi
Any other questions about compling or environments, feel easy to comment here.

docker suddenly stopped working, executor failed running [/bin/sh -c echo

Hello this is my docker file:
FROM golang:alpine3.15 as builder
RUN apk add ca-certificates git make gcc musl-dev libc6-compat curl chromium bash curl
RUN echo "http://dl-cdn.alpinelinux.org/alpine/edge/main" > /etc/apk/repositories \
&& echo "http://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories \
&& echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories \
&& echo "http://dl-cdn.alpinelinux.org/alpine/v3.12/main" >> /etc/apk/repositories \
&& apk upgrade -U -a \
&& apk add \
libstdc++ \
chromium \
harfbuzz \
nss \
freetype \
ttf-freefont \
font-noto-emoji \
wqy-zenhei
RUN mkdir /build
ADD go.* /build/
WORKDIR /build
RUN go mod download -x
ADD main.go /build/
RUN CGO_ENABLED=0 GOOS=linux go build -a -o /api
FROM alpine:3.15
COPY --from=builder /api .
EXPOSE 8080
ENTRYPOINT [ "/api" ]
STOPSIGNAL SIGKILL
I have been using same image on save ubuntu version, since past 4 months. But now when I run docker-compose up --build I always get this error:
failed to solve: executor failed running [/bin/sh -c echo "http://dl-cdn.alpinelinux.org/alpine/edge/main" > /etc/apk/repositories && echo "http://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories && echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories && echo "http://dl-cdn.alpinelinux.org/alpine/v3.12/main" >> /etc/apk/repositories && apk upgrade -U -a && apk add libstdc++ chromium harfbuzz nss freetype ttf-freefont font-noto-emoji wqy-zenhei]: exit code: 1
What could be the issue? Thanks
Found these errors too:
#0 3.144 ERROR: ca-certificates-bundle-20220614-r2: trying to overwrite etc/ssl1.1/cert.pem owned by libcrypto1.1-1.1.1q-r0.
#0 3.144 ERROR: ca-certificates-bundle-20220614-r2: trying to overwrite etc/ssl1.1/certs owned by libcrypto1.1-1.1.1q-r0.

Docker image build failed with error "Operation not permitted"

I am getting docker image build error "Operation not permitted". chmod 777 R /PredictionIO-0.14.0/lib/spark/ did not solve it.
I am trying to create PredictionIO docker image through git CICD to run in K8t
pio build --verbose
[INFO] DOCKER> [91mls: cannot access '/PredictionIO-0.14.0/lib/spark/pio-data-elasticsearch-assembly-0.14.0.jar': Operation not permitted ls: cannot access '/PredictionIO-0.14.0/lib/spark/pio-data-hbase-assembly-0.14.0.jar': Operation not permitted
[INFO] DOCKER> [91mls: cannot access '/PredictionIO-0.14.0/lib/spark/pio-data-hdfs-assembly-0.14.0.jar': Operation not permitted
ls: cannot access '/PredictionIO-0.14.0/lib/spark/pio-data-jdbc-assembly-0.14.0.jar': Operation not permitted
ls: cannot access '/PredictionIO-0.14.0/lib/spark/pio-data-localfs-assembly-0.14.0.jar': Operation not permitted
ls: cannot access '/PredictionIO-0.14.0/lib/spark/pio-data-s3-assembly-0.14.0.jar': Operation not permitted
[INFO] DOCKER> [91mError: Could not find or load main class org.apache.predictionio.tools.console.Console
[INFO] DOCKER> Removing intermediate container 8c164b439d1b
[ERROR] DOCKER> Unable to build image [myimage/bellmedia-pio-0.14.0:latest] : "The command '/bin/sh -c pio build --verbose' returned a non-zero code: 1" ["The command '/bin/sh -c pio build --verbose' returned a non-zero code: 1" ]
Dockerfile
FROM openjdk:8
MAINTAINER Jhon
ARG HTTP_PROXY=http://myproxyServer:8083/
ARG HTTPS_PROXY=http://myproxyServer:8083/
ENV DEBIAN_FRONTEND noninteractive
RUN export http_proxy=${HTTP_PROXY} \
&& export https_proxy=${HTTPS_PROXY} \
&& apt-get update && \
apt-get install -y net-tools curl patch gawk g++ gcc make libc6-dev vim telnet bash patch libreadline6-dev zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 autoconf libgdbm-dev libncurses5-dev automake libtool bison pkg-config libffi-dev libffi-dev krb5-user libpam-krb5 libpam-ccreds python3-pip tini libc6 libpam-modules libnss3 cron
ENV PIO_VERSION 0.14.0
ENV SPARK_VERSION 2.4.4
ENV ELK_VERSION 5.6.9
ENV SCALA_VERSION 2.11.12
ENV HBASE_VERSION 1.0.0
ENV HADOOP_VERSION 2.7
ENV PIO_GIT_URL http://archive.apache.org/dist/predictionio/
ENV PIO_HOME /PredictionIO-${PIO_VERSION}
ENV PATH=${PIO_HOME}/bin:/PredictionIO-0.14.0/vendors/spark-2.4.4-bin-hadoop2.7/bin:$PATH
ENV JAVA_HOME /usr/local/openjdk-8
EXPOSE 9200 9300
RUN export http_proxy=${HTTP_PROXY} \
&& export https_proxy=${HTTPS_PROXY} \
&& curl -O https://archive.apache.org/dist/predictionio/${PIO_VERSION}/apache-predictionio-${PIO_VERSION}.tar.gz
RUN mkdir predictionio
RUN tar -zxf apache-predictionio-${PIO_VERSION}.tar.gz -C /predictionio \
&& rm apache-predictionio-${PIO_VERSION}.tar.gz
COPY sbt-launch.jar /root/.sbt/launchers/1.2.3/
WORKDIR /predictionio
RUN export http_proxy=${HTTP_PROXY} \
&& export https_proxy=${HTTPS_PROXY} \
&& ./make-distribution.sh -Dscala.version=${SCALA_VERSION} -Dspark.version=${SPARK_VERSION} -Delasticsearch.version=${ELK_VERSION}
WORKDIR /
RUN tar -zxvf predictionio/PredictionIO-${PIO_VERSION}.tar.gz \
&& rm predictionio/PredictionIO-${PIO_VERSION}.tar.gz
COPY pio-env.sh ${PIO_HOME}/conf/
COPY log4j.properties ${PIO_HOME}/conf/
RUN mkdir ${PIO_HOME}/vendors
WORKDIR ${PIO_HOME}/vendors
RUN export http_proxy=${HTTP_PROXY} \
&& export https_proxy=${HTTPS_PROXY} \
&& curl -O https://archive.apache.org/dist/spark/spark-${SPARK_VERSION}/spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz
RUN tar -zxvf spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz \
&& rm spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}.tgz
COPY spark-defaults.conf spark-${SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}/conf/
WORKDIR ${PIO_HOME}
RUN export http_proxy=${HTTP_PROXY} \
&& export https_proxy=${HTTPS_PROXY} \
&& mkdir URM \
&& cd URM \
&& git clone https://github.com/actionml/universal-recommender.git
COPY build.sbt /PredictionIO-0.14.0/URM/universal-recommender/
COPY engine.json /PredictionIO-0.14.0/URM/universal-recommender/
RUN rm -rf target
RUN ln -s /PredictionIO-0.14.0/vendors/spark-2.4.4-bin-hadoop2.7/ /opt/spark
WORKDIR ${PIO_HOME}/URM/universal-recommender/
RUN mkdir lib
COPY sbt-launch.jar /root/.sbt/launchers/0.13.13/
RUN export SBT_OPTS="-Dhttp.proxyHost=myproxy.ca -Dhttp.proxyPort=8083 -Dhttps.proxyHost=myproxy.ca -Dhttps.proxyPort=8083"
RUN chmod 777 -R /PredictionIO-0.14.0/lib/spark/
########################### Command below is failing ##################
RUN export http_proxy=${HTTP_PROXY} \
&& export https_proxy=${HTTPS_PROXY} \
&& pio build --verbose
CMD ["sh", "-c", "tail -f /dev/null"]
Any suggestion is highly applicated.
Thanks
Jhon

Installation pyproj does not work returned 127

I want to install pyproj on Debian on Docker.
This is my script:
FROM debian:latest
RUN /bin/bash -c 'apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -yq apt-utils && \
DEBIAN_FRONTEND=noninteractive apt-get install -yq \
libpython3-dev \
python3-pip \
proj-bin \
curl \
libcurl4-gnutls-dev \
wget \
sqlite3 \
node-sqlite3 \
libsqlite3-dev \
libtiff5 \
libtiff5-dev \
make \
python-rtree && \
apt-get upgrade -yq && \
apt-get clean && \
apt-get update \
'
RUN wget https://download.osgeo.org/proj/proj-7.2.0.tar.gz
RUN tar -xvzf proj-7.2.0.tar.gz
RUN cd proj-7.2.0/ && ./configure && make && make install
RUN find / -name proj
RUN /bin/bash -c 'rm -rf /var/lib/apt/lists/* && \
pip3 install \
pyproj --no-binary pyproj \
'
This is the output:
Step 6/7 : RUN find / -name proj
---> Running in 6ae2ce5a5157
/usr/share/proj
/usr/local/share/proj
/usr/local/include/proj
/usr/local/bin/proj
/usr/bin/proj
/proj-7.2.0/src/.libs/proj
/proj-7.2.0/src/proj
/proj-7.2.0/include/proj
Removing intermediate container 6ae2ce5a5157
---> 9e652a740f23
Step 7/7 : RUN /bin/bash -c 'rm -rf /var/lib/apt/lists/* && pip3 install pyproj --no-binary pyproj '
---> Running in 2fa15199039b
Collecting pyproj
Downloading https://files.pythonhosted.org/packages/17/e5/3f5cdff3e955bcd768cdb0f4236f2d6e022aaa72f57caf7f4d5f552c88fc/pyproj-3.0.0.post1.tar.gz (663kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-r4rvjbmz/pyproj/setup.py", line 224, in <module>
ext_modules=get_extension_modules(),
File "/tmp/pip-install-r4rvjbmz/pyproj/setup.py", line 155, in get_extension_modules
proj_version = get_proj_version(proj_dir)
File "/tmp/pip-install-r4rvjbmz/pyproj/setup.py", line 22, in get_proj_version
proj_ver = subprocess.check_output(str(proj), stderr=subprocess.STDOUT).decode(
File "/usr/lib/python3.7/subprocess.py", line 395, in check_output
**kwargs).stdout
File "/usr/lib/python3.7/subprocess.py", line 487, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '/usr/local/bin/proj' returned non-zero exit status 127.
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-r4rvjbmz/pyproj/
Earlier I tried installing pyproj just via pip, it will install PROJ as a dependency, but with the wrong version. That's why I switched to the installation of the binaries via wget.
Also, I gave a find output to show that proj is actually in the right location.
Any suggestions?
https://pyproj4.github.io/pyproj/stable/installation.html
I would recommend either:
Upgrade to pip>=19 so PROJ 7.2 is in the wheel.
ENV PROJ_DIR=/path/to/proj/install and when you install PROJ: ./configure --prefix $PROJ_DIR

cant' build docker file for my service because of pip-req file not found error

i have created a Dockerfile that creates virtual env, installs all libs and activates that virtual env for further use.
the dockerfile:
FROM debian:9.4-slim
##################################################
# Install Anaconda 3:
##################################################
RUN apt-get update && apt-get install -y \
--fix-missing \
build-essential \
cmake \
gfortran \
git \
wget \
curl \
graphicsmagick \
libgraphicsmagick1-dev \
libatlas-dev \
libavcodec-dev \
libavformat-dev \
libgtk2.0-dev \
libjpeg-dev \
liblapack-dev \
libswscale-dev \
pkg-config \
python3-dev \
python3-numpy \
software-properties-common \
zip \
&& apt-get clean && rm -rf /tmp/* /var/tmp/* \
&& curl -o miniconda_install.sh https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
&& bash miniconda_install.sh -b -p ~/anaconda3 \
&& echo 'export PATH="~/anaconda3/bin:$PATH"' >> ~/.bashrc \
&& ~/anaconda3/bin/conda update -n base conda \
&& rm miniconda_install.sh \
&& rm -rf /var/lib/apt/lists/* \
&& /bin/bash -c "source ~/.bashrc"
ENV PATH="~/anaconda3/bin:${PATH}"
##################################################
# Setup env for current project:
##################################################
EXPOSE 1487 1488
RUN /bin/bash -c "conda create -y -n voice"
ADD erjan_environment_var_for_voice.txt /tmp/setup/requirements.txt
RUN /bin/bash -c "source activate voice"
RUN /bin/bash -c "pip install -r /tmp/setup/requirements.txt"
WORKDIR /Service
ADD Service /Service
ENTRYPOINT ["/bin/bash", "-c", "source activate voice && ./run.sh"]
when running this command:
sudo docker build . -t jysan_bank:dev
the error:
ERROR: Command errored out with exit status 1:
command: /root/anaconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-1mbm1qnz/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-1mbm1qnz/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-req-build-1mbm1qnz/pip-egg-info
cwd: /tmp/pip-req-build-1mbm1qnz/
Complete output (5 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/root/anaconda3/lib/python3.7/tokenize.py", line 447, in open
buffer = _builtin_open(filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-req-build-1mbm1qnz/setup.py'
I got this dockerfile from a similar project, it runs ok. I changed the requirements.txt file to include libs i needed.
why does it not work and asking for some file tmp/pip-req?

Resources