Is there a more efficient way to dockerize Luarocks? - docker

I am trying to build a lean alpine docker container for unit testing Lua in Google Cloud Build.
It runs fine, but takes about 30 - 50 seconds to build. When I run busted and luacheck, it only takes a few seconds for each. Any thoughts on how I could optimize this build process?
I was using wget and then switched to git. I added curl and unzip since luarocks expects it and openssl for one of luacheck's dependencies. Are there different dependencies I could/should use?
FROM alpine
ENV LUA_VERSION 5.1
RUN apk update
RUN apk add lua${LUA_VERSION}
RUN apk add lua${LUA_VERSION}-dev
RUN apk add bash build-base curl git openssl unzip
RUN cd /tmp && \
git clone https://github.com/keplerproject/luarocks.git && \
cd luarocks && \
sh ./configure && \
make build install && \
cd && \
rm -rf /tmp/luarocks
RUN luarocks install busted
RUN luarocks install luacheck
RUN luarocks install luacov

You can try this
Dockerfile
FROM alpine:3.12
# Set environment
ENV LUA_VERSION=5.1.5 \
LUAROCKS_VERSION=3.4.0
# Install dependency packages
RUN set -xe && \
apk add --no-cache --virtual .build-deps \
curl \
gcc \
g++ \
libc-dev \
make \
readline-dev \
&& \
apk add --no-cache \
readline \
&& \
# Install Lua
wget http://www.lua.org/ftp/lua-${LUA_VERSION}.tar.gz && \
tar zxf lua-${LUA_VERSION}.tar.gz && rm -f lua-${LUA_VERSION}.tar.gz && \
cd lua-${LUA_VERSION} && \
make -j $(getconf _NPROCESSORS_ONLN) linux && make install && \
cd / && rm -rf lua-${LUA_VERSION} && \
# Install LuaRocks
wget https://luarocks.org/releases/luarocks-${LUAROCKS_VERSION}.tar.gz && \
tar zxf luarocks-${LUAROCKS_VERSION}.tar.gz && rm -f luarocks-${LUAROCKS_VERSION}.tar.gz && \
cd luarocks-${LUAROCKS_VERSION} && \
./configure && \
make -j $(getconf _NPROCESSORS_ONLN) build && make install && \
cd / && rm -rf luarocks-${LUAROCKS_VERSION} && \
# Remove all build deps
apk del .build-deps && \
# Test
lua -v && luarocks
COPY docker-entrypoint.sh /usr/local/bin
docker-entrypoint.sh
#!/bin/sh
set -e
buildDepsApk="
curl
libc-dev
gcc
wget
"
pm='unknown'
if [ -e /lib/apk/db/installed ]; then
pm='apk'
fi
if [ "$pm" = 'apk' ]; then
apk add --no-cache ${buildDepsApk}
fi
luarocks install $#
if [ "$pm" = 'apk' ]; then
apk del ${buildDepsApk}
fi

You don't have to build luarocks. You can just install the package using,
RUN apk add luarocks

Related

Create a Docker container for Machine Learning

I'm not a Docker expert and I'm trying to create a container for machine learning projects. This is mostly for academic purpose, as I'm studying machine learning.
I wrote a dockerfile (and a devcontainer.json file to open the container in vscode) that runs fine until I add the lines to build tensorflow. I found three problems but I don't know what I'm missing:
I got a warning about the fact that Bazel installation isn't a release version
When the building phase arrives at ./configure I can't configure through prompt question
I got an error at the very building phase with
ERROR: The project you're trying to build requires Bazel 5.3.0 (specified in /docklearning/tensorflow/.bazelversion), but it wasn't found in /usr/bin.
This is the Dockerfile:
FROM nvidia/cuda:12.0.0-runtime-ubuntu22.04 as base
ARG USER_UID=1000
#switch to non-interactive frontend
ENV DEBIAN_FRONTEND=noninteractive
WORKDIR /docklearning
ADD . /docklearning
# Install packages
RUN apt-get update -q && apt-get install -q -y --no-install-recommends \
apt-transport-https curl gnupg apt-utils wget gcc g++ npm unzip build-essential ca-certificates curl git gh \
make nano iproute2 nano openssh-client openssl procps \
software-properties-common bzip2 subversion neofetch \
fontconfig && \
curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor >bazel-archive-keyring.gpg && \
mv bazel-archive-keyring.gpg /usr/share/keyrings && \
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/bazel-archive-keyring.gpg] https://storage.googleapis.com/bazel-apt stable jdk1.8" \
| tee /etc/apt/sources.list.d/bazel.list && \
apt update && apt install -q -y bazel && \
apt-get full-upgrade -q -y && \
cd ~ && \
wget https://github.com/ryanoasis/nerd-fonts/releases/download/v2.1.0/Meslo.zip && \
mkdir -p .local/share/fonts && \
unzip Meslo.zip -d .local/share/fonts && \
cd .local/share/fonts && rm *Windows* && \
cd ~ && \
rm Meslo.zip && \
fc-cache -fv && \
apt-get install -y zsh zsh-doc chroma
# Install anaconda
RUN wget https://repo.anaconda.com/archive/Anaconda3-2022.10-Linux-x86_64.sh -O Anaconda.sh && \
/bin/bash Anaconda.sh -b -p /opt/conda && \
rm Anaconda.sh
# Install LSD for ls substitute and clean up
RUN wget https://github.com/Peltoche/lsd/releases/download/0.23.1/lsd_0.23.1_amd64.deb -P /tmp && \
dpkg -i /tmp/lsd_0.23.1_amd64.deb && \
rm /tmp/lsd_0.23.1_amd64.deb && \
apt-get autoremove -y && \
apt-get autoclean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*/apt/lists/* && \
useradd -r -m -s /bin/bash -u ${USER_UID} docklearning
# Add conda to PATH
ENV PATH=/opt/conda/bin:$PATH
ENV HOME=/home/docklearning
# Make zsh default shell
RUN chsh -s /usr/bin/zsh docklearning
# link conda to /etc/profile.d
RUN ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh
# Update Conda Base env
RUN conda update -n base --all -y && \
conda install -c conda-forge bandit opt_einsum keras-preprocessing && \
conda clean -a -q -y
# Download Tensorflow from github and build from source
RUN git clone https://github.com/tensorflow/tensorflow.git && \
cd tensorflow && \
./configure && \
bazel build --config=opt --config=cuda --cxxopt="-mavx2" //tensorflow/tools/pip_package:build_pip_package && \
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg && \
pip install /tmp/tensorflow_pkg/tensorflow-*.whl && \
cd .. && \
rm -rf tensorflow
# Create shell history file
RUN mkdir ${HOME}/zsh_history && \
chown docklearning ${HOME}/zsh_history && \
mkdir ${HOME}/.ssh
# Switch to internal user
USER docklearning
WORKDIR ${HOME}
# Copy user configuration files
COPY --chown=docklearning ./config/.aliases.sh ./
COPY --chown=docklearning ./config/.bashrc ./
COPY --chown=docklearning ./config/.nanorc ./
# Configure Zsh for internal user
ENV ZSH=${HOME}/.oh-my-zsh
ENV ZSH_CUSTOM=${ZSH}/custom
ENV ZSH_PLUGINS=${ZSH_CUSTOM}/plugins
ENV ZSH_THEMES=${ZSH_CUSTOM}/themes
RUN wget -qO- https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh | zsh || true
RUN git clone --single-branch --branch 'master' --depth 1 https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_PLUGINS}/zsh-syntax-highlighting \
&& git clone --single-branch --branch 'master' --depth 1 https://github.com/zsh-users/zsh-autosuggestions ${ZSH_PLUGINS}/zsh-autosuggestions \
&& git clone --single-branch --depth 1 https://github.com/romkatv/powerlevel10k.git ${ZSH_THEMES}/powerlevel10k
COPY --chown=docklearning ./config/.p10k.zsh ./
COPY --chown=docklearning ./config/.zshrc ./
CMD [ "/bin/zsh" ]
I wouldn't say this question about docker or TensorFlow. It is about package installation and OS configuration.
In this case, you need to fix the bazel version first by specifying it explicitly:
apt install -q -y bazel-5.3.0
But that package is quite wired and doesn't create a symlink, so you need to create it:
ln -s /bin/bazel-5.3.0 /bin/bazel
You can validate if bazel installed by running it:
`bazel --version
bazel 5.3.0`

Failed to get jdk (jdk-8u301-linux-x64.tar.gz) on alpine:3.8

I'm trying to build the image which was built in the past but failing this time as it is failing to download Oracle JDK, it downloads the tar with just a 4 KB file.
I have referred this Docker alpine + oracle java: cannot find java already and tried with the latest version of glibc but no luck.
I'm though able to install the lower version than jdk-8u201.
Any suggestion is appreciated! I have to go with alpine:3.8 only.
FROM alpine:3.8
RUN apk upgrade --update && \
apk add --update curl ca-certificates bash && \
for pkg in glibc-2.23-r1 glibc-bin-2.23-r1 glibc-i18n-2.23-r1; do curl -sSL https://github.com/andyshinn/alpine-pkg-glibc/releases/download/2.23-r1/${pkg}.apk -o /tmp/${pkg}.apk; done && \
apk add --allow-untrusted /tmp/*.apk && \
rm -v /tmp/*.apk && \
mkdir -p /opt/springboot/apps/ && \
( /usr/glibc-compat/bin/localedef --force --inputfile POSIX --charmap UTF-8 C.UTF-8 || true ) && \
echo "export LANG=C.UTF-8" > /etc/profile.d/locale.sh && \
/usr/glibc-compat/sbin/ldconfig /lib /usr/glibc-compat/lib && \
curl -jksSLH "Cookie: oraclelicense=accept-securebackup-cookie" -o /tmp/java.tar.gz \
https://download.oracle.com/otn-pub/java/jdk/8u201-b09/42970487e3af4f5aa5bca3f542482c60/jdk-8u201-linux-x64.tar.gz && \
gunzip /tmp/java.tar.gz

Docker image with v8 compiled reached almost 5GB. What am I doing wrong?

Sorry, this is my first time using docker so there may be chances that I'm using the wrong term. Basically, I need to use this image as the base of our project, however, when I try to build our project using docker-compose it takes quite a long time build which I suspect is because of the file size of the image. Is there anything I can do to reduce the file size down to 500MB? Here is what I have in the image's docker file.
FROM php:7.2-apache-buster
ENV V8_VERSION=7.4.288.21
RUN apt-get update -y --fix-missing && apt-get upgrade -y;
# Install v8js (see https://github.com/phpv8/v8js/blob/php7/README.Linux.md)
RUN apt-get install -y --no-install-recommends \
libtinfo5 libtinfo-dev \
build-essential \
curl \
git \
libglib2.0-dev \
libxml2 \
python \
patchelf \
&& cd /tmp \
\
&& git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git --progress --verbose \
&& export PATH="$PATH:/tmp/depot_tools" \
\
&& fetch v8 \
&& cd v8 \
&& git checkout $V8_VERSION \
&& gclient sync \
\
&& tools/dev/v8gen.py -vv x64.release -- is_component_build=true use_custom_libcxx=false
RUN export PATH="$PATH:/tmp/depot_tools" \
&& cd /tmp/v8 \
&& ninja -C out.gn/x64.release/ \
&& mkdir -p /opt/v8/lib && mkdir -p /opt/v8/include \
&& cp out.gn/x64.release/lib*.so out.gn/x64.release/*_blob.bin out.gn/x64.release/icudtl.dat /opt/v8/lib/ \
&& cp -R include/* /opt/v8/include/ \
&& apt-get install patchelf \
&& for A in /opt/v8/lib/*.so; do patchelf --set-rpath '$ORIGIN' $A;done
# Install php-v8js
RUN cd /tmp \
&& git clone https://github.com/phpv8/v8js.git \
&& cd v8js \
&& phpize \
&& ./configure --with-v8js=/opt/v8 LDFLAGS="-lstdc++" \
&& make \
&& make test \
&& make install
RUN docker-php-ext-enable v8js
As #david-maze pointed me to right direction which is to use multi-stage build I managed to reduce the file size from almost 5GB down to 470MB by tracing all the files I need to copy to another build. Here is what I got
FROM php:7.2-apache-buster
COPY --from=BASE_PHP /opt /opt
COPY --from=BASE_PHP /usr/local/etc/php/conf.d/docker-php-ext-v8js.ini /usr/local/etc/php/conf.d/
COPY --from=BASE_PHP /usr/local/lib/php/extensions/no-debug-non-zts-20170718 /usr/local/lib/php/extensions/no-debug-non-zts-20170718
Thank you so much

Dockerfile not able to find ionic or cordova after install

I have a Dockerfile for which I'm trying to install ionic and cordova. The installation seems to work without error but I'm not able use them inside the container. The thing is, they are installed globally so I not sure what I'm missing. Any assistance is appreciated.
FROM openjdk:8
MAINTAINER me
# Install Git and dependencies
RUN dpkg --add-architecture i386 \
&& apt-get update \
&& apt-get install -y file git curl zip libncurses5:i386 libstdc++6:i386 zlib1g:i386 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists /var/cache/apt
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
RUN apt-get install -y nodejs
RUN mkdir -p /tmp/yarn && \
mkdir -p /opt/yarn/dist && \
cd /tmp/yarn && \
wget -q https://yarnpkg.com/latest.tar.gz && \
tar zvxf latest.tar.gz && \
find /tmp/yarn -maxdepth 2 -mindepth 2 -exec mv {} /opt/yarn/dist/ \; && \
rm -rf /tmp/yarn
RUN ln -sf /opt/yarn/dist/bin/yarn /usr/local/bin/yarn && \
ln -sf /opt/yarn/dist/bin/yarn /usr/local/bin/yarnpkg && \
yarn --version
RUN yarn global add ionic
RUN yarn global add cordova
# Set up environment variables
ENV ANDROID_HOME="/home/user/android-sdk-linux" \
SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip" \
GRADLE_URL="https://services.gradle.org/distributions/gradle-4.5.1-all.zip"
# Create a non-root user
RUN useradd -m user
USER user
WORKDIR /home/user
# Download Android SDK
RUN mkdir "$ANDROID_HOME" .android \
&& cd "$ANDROID_HOME" \
&& curl -o sdk.zip $SDK_URL \
&& unzip sdk.zip \
&& rm sdk.zip \
&& yes | $ANDROID_HOME/tools/bin/sdkmanager --licenses
# Install Gradle
RUN wget $GRADLE_URL -O gradle.zip \
&& unzip gradle.zip \
&& mv gradle-4.5.1 gradle \
&& rm gradle.zip \
&& mkdir .gradle
ENV PATH="/home/user/gradle/bin:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools:${PATH}"

Laradock: Install image magik

I am trying to install image magik inside laradock
https://github.com/laradock/laradock
I have setup nginx , mysql, redis and elasticsearch.
I have tried installing the image magik inside php-fpm with
RUN apt-get -y update && \
apt-get install pkg-config libmagickwand-dev -y && \
pecl install imagick
RUN docker-php-ext-enable imagick
But still image are not generated.
I got it to working by adding following into the docker file of php-fpm for docker image of https://github.com/laradock/laradock
RUN apt-get update && apt-get install -y
git libmagick++-dev
--no-install-recommends &&
git clone https://github.com/mkoppanen/imagick.git &&
cd imagick && git checkout phpseven && phpize && ./configure &&
make && make install &&
docker-php-ext-enable imagick &&
cd ../ && rm -rf imagick
Also set PHP_FPM_INSTALL_EXIF=true in the .env next to docker-compose.yml
Remove the previous containers and images and create new build
In the .env file there is a variable for installing ImageMagick (In Laradock v7.x )
It's false by default but setting it to PHP_FPM_INSTALL_IMAGEMAGICK=true should do it.
Please try to rebuild the docker containers via the following command.
docker-compose build --no-cache
in .env file turn WORKSPACE_INSTALL_IMAGEMAGICK to true and
in workspace's Dockerfile add apt-get update &&\ on IMAGEMAGICK section like this:
###########################################################################
# ImageMagick:
###########################################################################
USER root
ARG INSTALL_IMAGEMAGICK=false
ARG IMAGEMAGICK_VERSION=latest
ENV IMAGEMAGICK_VERSION ${IMAGEMAGICK_VERSION}
RUN if [ ${INSTALL_IMAGEMAGICK} = true ]; then \
apt-get update && \
apt-get install -y libmagickwand-dev imagemagick && \
if [ $(php -r "echo PHP_MAJOR_VERSION;") = "8" ]; then \
apt-get install -y git && \
cd /tmp && \
if [ ${IMAGEMAGICK_VERSION} = "latest" ]; then \
git clone https://github.com/Imagick/imagick; \
else \
git clone --branch ${IMAGEMAGICK_VERSION} https://github.com/Imagick/imagick; \
fi && \
cd imagick && \
phpize && \
./configure && \
make && \
make install && \
rm -r /tmp/imagick; \
else \
pecl install imagick; \
fi && \
echo "extension=imagick.so" >> /etc/php/${LARADOCK_PHP_VERSION}/mods-available/imagick.ini && \
ln -s /etc/php/${LARADOCK_PHP_VERSION}/mods-available/imagick.ini /etc/php/${LARADOCK_PHP_VERSION}/cli/conf.d/20-imagick.ini && \
php -m | grep -q 'imagick' \
;fi

Resources