How can I build a similar docker image based on alpine that works on ubuntu? - docker

I am trying to rewrite a Dockerfile (https://github.com/orangefoil/rcssserver-docker/blob/master/Dockerfile) so that it uses alpine instead of ubuntu. Goal is to reduce the file size.
In the original image the robocup soccer server is built from scratch using g++, flex, bison, etc.
FROM ubuntu:18.04 AS build
ARG VERSION=16.0.0
WORKDIR /root
RUN apt update && \
apt -y install autoconf bison clang flex libboost-dev libboost-all-dev libc6-dev make wget
RUN wget https://github.com/rcsoccersim/rcssserver/archive/rcssserver-$VERSION.tar.gz && \
tar xfz rcssserver-$VERSION.tar.gz && \
cd rcssserver-rcssserver-$VERSION && \
./bootstrap && \
./configure && \
make && \
make install && \
ldconfig
I tried to do the same on alpine and had to exchange some packages:
FROM alpine:latest
ARG VERSION=16.0.0
WORKDIR /root
# Add basics first
RUN apk — no-cache update \
&& apk upgrade \
&& apk add autoconf bison clang-dev flex-dev boost-dev make wget automake libtool-dev g++ build-base
RUN wget https://github.com/rcsoccersim/rcssserver/archive/rcssserver-$VERSION.tar.gz
RUN tar xfz rcssserver-$VERSION.tar.gz
RUN cd rcssserver-rcssserver-$VERSION && \
./bootstrap && \
./configure && \
make && \
make install && \
ldconfig
Unfortunately, my version doesn't work yet. It fails with
/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lrcssclangparser
From what I found so far, this can happen, if dev packages are not installed (see ld cannot find an existing library), but I changed to dev packages where I could find them and still no luck.
So, my current assumption is that ubuntu has some package installed, that I need to add in my alpine image. I would exclude a code problem, since the ubuntu version works.
Any ideas, what could be missing? I would also be happy to understand how to compare the packages myself, but the package namings are not the same in ubuntu and alpine, so I find it pretty hard to figure this out.

You should break this up using a multi-stage build. In the image you're building now, the final image contains the C toolchain and all of the development libraries and headers that those -dev packages install; you don't need any of those to actually run the built application. The basic idea is to build the application exactly as you have it now, but then COPY only the built application into a new image with fewer dependencies.
This would look something like this (untested):
FROM ubuntu:18.04 AS build
# ... exactly what's in the original question ...
FROM ubuntu:18.04
# Install the shared libraries you need to run the application,
# but not -dev headers or the full C toolchain. You may need to
# run `ldd` on the built binary to see what exactly it needs.
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install --assume-yes --no-install-recommends \
libboost-atomic1.65.1 \
libboost-chrono1.65.1 \
# ... more libboost-* libraries as required ...
# Get the built application out of the original image.
# Autoconf's default is to install into /usr/local, and in a
# typical Docker base image nothing else will be installed there.
COPY --from=build /usr/local /usr/local
RUN ldconfig
# Describe how to run a container.
EXPOSE 12345
CMD ["/usr/local/bin/rcssserver"]
Compared to the size of the C toolchain, header files, and build-time libraries, the difference between an Alpine and Ubuntu image is pretty small, and Alpine has well-documented library compatibility issues with its minimal libc implementation.

Related

What is the correct way to write PATH variable in Docker ENV instruction?

I have tried to build a docker image and found that the PATH variable I set has some issues. A Minimal non-working example is:
FROM ubuntu:latest
SHELL ["/bin/bash", "-cu"]
ARG CTAGS_DIR=/root/tools/ctags
# Install common dev tools
RUN apt-get update --allow-unauthenticated \
&& apt-get install --allow-unauthenticated -y git curl autoconf pkg-config zsh
# Compile ctags
RUN cd /tmp \
&& git clone https://github.com/universal-ctags/ctags.git \
&& cd ctags \
&& ./autogen.sh \
&& ./configure --prefix=${CTAGS_DIR} \
&& make -j$(nproc) \
&& make install \
&& rm -rf /tmp/ctags
ENV PATH=$HOME/tools/ctags/bin:$PATH
RUN echo "PATH is $PATH"
RUN which ctags
In the above Dockerfile, the line ENV PATH=$HOME/tools/ctags/bin:$PATH does not work as expected. It seems that $HOME is not correctly expanded. The following two instructions also do not work:
ENV PATH=~/tools/ctags/bin:$PATH
ENV PATH="~/tools/ctags/bin:$PATH"
Only settings the absolute path works:
# the following setting works.
ENV PATH="/root/tools/ctags/bin:$PATH"
I have looked up the docker references but can not find document about this.
In general, when you're building a Docker image, it's okay to install things into the normal "system" directories. Whatever you're building will be isolated inside the image, and it can't conflict with other tools.
The easiest answer to your immediate question is to arrange things so you don't need to set $PATH.
In the example you give, you can safely use Autoconf's default installation directory of /usr/local. That will almost certainly be empty when you start your image build and only things you install will be there.
RUN ... \
&& ./configure \
&& make \
&& make install
(The Python corollary is to not create a virtual environment for your application; just use the system pip to install things into the default Python library directories.)
Don't expect there to be a home directory. If you have to install in some non-default place, /app is common, and /opt/whatever is consistent with non-Docker Linux practice. Avoid $HOME or ~, they aren't generally well-defined in Docker (unless you go out of your way to make them be).

Up to date (lts-13.0) minimal base docker image for haskell/stack?

I'm would like to deploy my haskell application on docker and the base image fco/stack-build that I've found takes 9GB ! Do you know a base image more minimal than that ??
stack-build is as large as it is, because it contains the required system dependencies of all packages on Stackage.
I am using the following base image for building and deploying:
FROM ubuntu:18.04
RUN apt-get update
# Build dependencies
RUN apt-get install --assume-yes curl
RUN curl -sSL https://get.haskellstack.org/ | sh
RUN apt-get install --assume-yes libtinfo-dev
# Without this haddock crashes for modules containing
# non-ASCII characters.
ENV LANG C.UTF-8
It's not really minimal if you just want to use the image during runtime as you wouldn't need stack in that case.
Firstly, that image might only be necessary to build the excutable, once you have it built you could use a multistage docker build or just copy the executable directly to a more slim image.
The dockerfile is available here: https://github.com/commercialhaskell/stack/blob/master/etc/dockerfiles/stack-build/lts-13.0/Dockerfile
You could remove these commands (which probably add up to the bulk of the size):
# Use Stackage's debian-bootstrap.sh script to install system libraries and
# tools required to build any Stackage package.
#
RUN apt-get update && \
apt-get install -y wget && \
wget -qO- https://raw.githubusercontent.com/fpco/stackage/$BOOTSTRAP_COMMIT/debian-bootstrap.sh | bash && \
rm -rf /var/lib/apt/lists/*

How to create the smallest possible Docker image after installing apt dependencies

I've created a Docker image using debian as the parent image. In my Dockerfile I've installed some dependencies using apt and pip.
Now, I want to get rid off everything that is not completely necessary to run my app, which of course, needs the dependencies installed.
For now I have the following lines in my Dockerfile after installing the dependencies.
RUN rm -rf /var/lib/apt/lists/* \
&& rm -Rf /usr/share/doc && rm -Rf /usr/share/man \
&& apt-get clean
I've also installed the dependencies using the --no-install-recommends option.
Anything else I can do to reduce the footprint of my Docker image?
PS: just in case, this is how I installed the dependencies:
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
sudo systemd \
build-essential libffi-dev libssl-dev \
python-pip python-dev python-setuptools python-wheel
To reduce the size of the image, you need to combine your RUN commands into one. When you create files in one layer and delete them in another, the files still exist on the drive and are shipped over the network. Their existence is just hidden when the layers of the filesystem are assembled for your container.
The Dockerfile best practices explain this in more detail: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#run
I'd also recommend building with docker build --rm=false --no-cache . (temporarily) and then reviewing the output of docker diff on each of the created images to see what files are created in each layer.

docker-compose update from S3 bucket

Our Dockerfile invokes a python script which copies a binary from S3 to /usr/bin. This works fine the first time. But from then on "docker-compose build" does nothing because everything is cached. This is a problem if the binary has changed.
Short of building with --no-cache, what is the best way to make sure "docker-compose build" will always pick up the new binary if there is one. We don't mind if it unnecessarily downloads the binary even if unchanged, so long as it does work then the binary has changed.
Seems like we want a Dockerfile step that always executes?
FROM ubuntu:trusty
RUN apt-get update
RUN apt-get -y install software-properties-common
RUN apt-get -y install --reinstall ca-certificates
RUN add-apt-repository ppa:fkrull/deadsnakes
RUN apt-get update && apt-get install -y \
curl \
wget \
vim \
git \
python3.5 \
python3-pip \
python3-setuptools \
libpcap0.8-dev
RUN ln -sf /usr/bin/python3.5 /usr/bin/python3
ADD . /app
WORKDIR /app
# Install Python Requirements
RUN pip3 install -r etc/python/requirements.txt
# Download/Install processor and associated libs
RUN python3 setup_processor.py
RUN mkdir -p /logs
ENTRYPOINT ["/app/entrypoint.sh"]
Where setup_processor.py downloads directly from S3 to /usr/bin.
So as of now there is no direct feature like this. But there is a workaround to your solution.
Add Build argument before your download step
ARG BUILD_ON=now
# Download/Install processor and associated libs
RUN python3 setup_processor.py
While building the image use below
docker build --build-arg BUILD_ON=$(date) ....
This will always make sure that you get a change in the ARG step and all steps cache after that will be invalidated
A feature has already been requested and being worked out on below thread
https://github.com/moby/moby/issues/1996

Docker + older version of Elixir/Phoenix

I have been requested to move an Elixir/Phoenix app to Docker, with which I have no prior experience. The app uses non-latest versions of Elixir and Phoenix so I have had to diverge from the code online which generally focuses on latest versions. That led me to write this Dockerfile
# FROM bitwalker/alpine-elixir:latest
FROM bitwalker/alpine-elixir:1.3.4
MAINTAINER Paul Schoenfelder <paulschoenfelder#gmail.com>
# Important! Update this no-op ENV variable when this Dockerfile
# is updated with the current date. It will force refresh of all
# of the base images and things like `apt-get update` won't be using
# old cached versions when the Dockerfile is built.
ENV REFRESHED_AT=2017-07-26 \
# Set this so that CTRL+G works properly
TERM=xterm
# Install NPM
RUN \
mkdir -p /opt/app && \
chmod -R 777 /opt/app && \
apk update && \
apk --no-cache --update add \
git make g++ wget curl inotify-tools \
nodejs nodejs-current-npm && \
npm install npm -g --no-progress && \
update-ca-certificates --fresh && \
rm -rf /var/cache/apk/*
# Add local node module binaries to PATH
ENV PATH=./node_modules/.bin:$PATH \
HOME=/opt/app
# Install Hex+Rebar
RUN mix local.hex --force && \
mix local.rebar --force
WORKDIR /opt/app
CMD ["/bin/sh"]
<then it goes on to add some elixir depedencies>
On running
sudo docker build -t phoenix .
I'm ending up with this error and wondering how to get around it? Noting 'current' in the title I'm wondering whether using an older version of nodejs, and if so, how to do that? Beyond that I am open to any and all suggestions
ERROR: unsatisfiable constraints:
nodejs-current-npm (missing):
required by: world[nodejs-current-npm]
musl-1.1.14-r14:
breaks: musl-dev-1.1.14-r15[musl=1.1.14-r15]
That looks like bitwalker/alpine-elixir issue 5:
when using tagged images, you may sometimes need to explicitly upgrade packages, as the installed packages are at the versions found when building the image.
Generally it's as simple as adding apk --update upgrade before any commands which install packages.
Indeed, when you compare the old elixir 1.4.4-based Dockerfile, and the latest one, you will see an upgrade first in the latter:
apk --no-cache --update upgrade && \
apk add --no-cache --update --virtual .elixir-build \
...
Try and add that to your Dockerfile.

Resources