We are having a docker image whose size is around 858 MB. We want to reduce it's size.
Out of 858 MB
APP Size: 352 MB (EAR FILE)
WILDFLY (18.0.1): 212 MB
adoptopenjdk:11: 123 MB
USR DIR : 116 MB
Is there any way to reduce size?
First Dockerfile --wildfly-11.8:latest
FROM adoptopenjdk/openjdk11:jre-11.0.6_10-alpine
ENV WILDFLY_VERSION 18.0.1.Final
ENV WILDFLY_SHA1 ef0372589a0f08c53e7291721a7e3f7d9
ENV MODULES_FILENAME modules.tar
ENV MODULES_SHA1 2dcfee4045b7d026d7d6290cebc772482
ENV JBOSS_HOME /opt/wildfly
RUN useradd -ms /bin/bash jboss
RUN cd $HOME \
&& curl -O -k https://download.jboss.org/wildfly/$WILDFLY_VERSION/wildfly-$WILDFLY_VERSION.tar.gz \
&& sha1sum wildfly-$WILDFLY_VERSION.tar.gz | grep $WILDFLY_SHA1 \
&& tar xf wildfly-$WILDFLY_VERSION.tar.gz \
&& mv $HOME/wildfly-$WILDFLY_VERSION $JBOSS_HOME \
&& rm wildfly-$WILDFLY_VERSION.tar.gz \
&& chown -R jboss:0 ${JBOSS_HOME} \
&& chmod -R g+rw ${JBOSS_HOME} \
&& curl -p ftp://ftp.co.il//wildfly/1801/$MODULES_FILENAME --user "app:qax" --ftp-create-dirs -O \
&& sha1sum $MODULES_FILENAME | grep $MODULES_SHA1 \
&& tar xf ${MODULES_FILENAME} \
&& rm ${MODULES_FILENAME} \
&& cp -r ./* ${JBOSS_HOME} \
&& rm -rf ./*
COPY configuration /opt/wildfly/standalone/configuration
COPY standalone.conf /opt/wildfly/bin
USER jboss
EXPOSE 8080
CMD cd /opt/wildfly/bin && ./standalone.sh -b="0.0.0.0" -c=$STANDALONE_CONFIG
Second Dockerfile
FROM wildfly-11.8:latest
ENV DEPLOYMENT_LOCATION /opt/wildfly/standalone/deployments
ARG ear_file_path
COPY $ear_file_path $DEPLOYMENT_LOCATION
There are many ways to reduce the size of a Docker image.
1. Use an alpine image or minimum sized one as the base image.
For an official Docker image in the Docker hub, there will be different tags for a single repository. Example: ubuntu:stable, ubuntu:latest, ubuntu:alpine, where alpine would consume the least size.
2. Use Multi Staged Dockerfile.
When writing a Dockerfile, split the requirement into sections.
Like:
(a) If you need to copy the files to the work directory first, do them in the first stage.
(b) Next if you require to install packages, do that in the next stage.
(c) And, finally you need to expose a port and run some commands, do that in the last and final stage.
Example dockerfile:
# Use an official Ubuntu runtime as a base image.
FROM ubuntu:latest AS base
#To create working directory.
WORKDIR /application
# To add contents to the working directory.
ADD . /application
# Use the base image to create a packaged image.
FROM base AS package
#To create add to the existing working directory.
WORKDIR /application
# To install necessary packages.
RUN apt update \
&& apt install nginx -y
&& apt install docker -y
# Use the packaged image to create a final image.
FROM package AS final
#To create add to the existing working directory.
WORKDIR /application
# To Run Commands after the container is run.
CMD ["echo", "Hello world"]
This way, you can reduce the size of the Docker image thats created.
Refer: https://docs.docker.com/develop/develop-images/multistage-build/
Thanks.
Related
I have the following file arrangement for a docker image (salmon):
salmon
├── docker
│ └── Dockerfile
└── src
├── align_utils.py
├── job_utils.py
├── run_salmon.py
└── s3_utils.py
My entrypoint script in this case is run_salmon.py, which also makes use of the other .py scripts in src/. When I try to build the docker image via docker build -t salmon:pipeline . within docker/, I get the error:
COPY failed: stat /var/lib/docker/tmp/docker-builder013511307/src/run_salmon.py: no such file or directory
How do I figure out where the entrypoint script is located relative to the working dir in the dockerfile?
Dockerfile:
# Use Python base image from DockerHub
FROM python:2.7
# INSTALL CMAKE
RUN apt-get update && apt-get install -y sudo \
&& sudo apt-get update \
&& sudo apt-get install -y \
cmake \
wget
#INSTALL BOOST
RUN wget https://dl.bintray.com/boostorg/release/1.66.0/source/boost_1_66_0.tar.gz \
&& mv boost_1_66_0.tar.gz /usr/local/bin/ \
&& cd /usr/local/bin/ \
&& tar -xzf boost_1_66_0.tar.gz \
&& cd ./boost_1_66_0/ \
&& ./bootstrap.sh \
&& ./b2 install
#INSTALL SALMON
RUN wget https://github.com/COMBINE-lab/salmon/releases/download/v0.14.1/salmon-0.14.1_linux_x86_64.tar.gz \
&& mv salmon-0.14.1_linux_x86_64.tar.gz /usr/local/bin/ \
&& cd /usr/local/bin/ \
&& tar -xzf salmon-0.14.1_linux_x86_64.tar.gz \
&& cd salmon-latest_linux_x86_64/
ENV PATH=$PATH:/usr/local/bin/salmon-latest_linux_x86_64/bin/
# Copy files to root directory of a Docker
WORKDIR /
COPY src/run_salmon.py /
COPY src/s3_utils.py /
COPY src/job_utils.py /
COPY src/align_utils.py /
ENTRYPOINT ["python", "/run_salmon.py"]
When you run docker build -t salmon:pipeline . from inside the docker directory, you are specifying the current directory as a context for the build.
When the build run COPY src/run_salmon.py / it tries to find the path relative to the root of your context (i.e., salmon/docker/src/run_salmon.py), where the files don't exist.
It's better that you specify your root context as the salmon directory, specifying the full path of the Dockerfile with the -f flag. Run this from inside salmon directory:
docker build -t salmon:pipeline -f docker/Dockerfile .
I'm writting a Dockerfile in order to create an image for a web server (a shiny server more precisely). It works well, but it depends on a huge database folder (db/) that it is not distributed with the package, so I want to do all this preprocessing while creating the image, by running the corresponding script in the Dockerfile.
I expected this to be simple, but I'm struggling figuring out where my files are being located within the image.
This repo has the following structure:
Dockerfile
preprocessing_files
configuration_files
app/
application_files
db/
processed_files
So that app/db/ does not exist, but is created and filled with files when preprocessing_files are run.
The Dockerfile is the following:
# Install R version 3.6
FROM r-base:3.6.0
# Install Ubuntu packages
RUN apt-get update && apt-get install -y \
sudo \
gdebi-core \
pandoc \
pandoc-citeproc \
libcurl4-gnutls-dev \
libcairo2-dev/unstable \
libxml2-dev \
libxt-dev \
libssl-dev
# Download and install ShinyServer (latest version)
RUN wget --no-verbose https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/VERSION -O "version.txt" && \
VERSION=$(cat version.txt) && \
wget --no-verbose "https://s3.amazonaws.com/rstudio-shiny-server-os-build/ubuntu-12.04/x86_64/shiny-server-$VERSION-amd64.deb" -O ss-latest.deb && \
gdebi -n ss-latest.deb && \
rm -f version.txt ss-latest.deb
# Install R packages that are required
RUN R -e "install.packages(c('shiny', 'flexdashboard','rmarkdown','tidyverse','plotly','DT','drc','gridExtra','fitdistrplus'), repos='http://cran.rstudio.com/')"
# Copy configuration files into the Docker image
COPY shiny-server.conf /etc/shiny-server/shiny-server.conf
COPY /app /srv/shiny-server/
COPY /app/db /srv/shiny-server/app/
# Make the ShinyApp available at port 80
EXPOSE 80
CMD ["/usr/bin/shiny-server"]
This above file works well if preprocessing_files are run in advance, so app/application_files can successfully read app/db/processed_files. How could this script be run in the Dockerfile? To me the intuitive solution would be simply to write:
RUN bash -c "preprocessing.sh"
Before the ADD instruction, but then preprocessing_files are not found. If the above instruction is written below ADD and also WORKDIR app/, the same error happens. I cannot understand why.
You cannot execute code on the host machine from Dockerfile. RUN command executes inside the container being built. You can:
Copy preprocessing_files inside docker container and run preprocessing.sh inside the container (this would increase size of the container)
Create a makefile/build.sh script which launches preprocessing.sh before executing docker build
In the Alpine linux package site https://pkgs.alpinelinux.org/packages
NSCA packages are yet to get added. Is there an alternative to setup NSCA in Alpine Linux for passive-check?
If there is no package for it, you can always build it yourself.
FROM alpine AS builder
ARG NSCA_VERSION=2.9.2
RUN apk update && apk add build-base build-base gcc wget git
RUN wget http://prdownloads.sourceforge.net/nagios/nsca-$NSCA_VERSION.tar.gz
RUN tar xzf nsca-$NSCA_VERSION.tar.gz
RUN cd nsca-$NSCA_VERSION&& ./configure && make all
RUN ls -lah nsca-$NSCA_VERSION/src
RUN mkdir -p /dist/bin && cp nsca-$NSCA_VERSION/src/nsca /dist/bin
RUN mkdir -p /dist/etc && cp nsca-$NSCA_VERSION/sample-config/nsca.cfg /dist/etc
FROM alpine
COPY --from=builder /dist/bin/nsca /bin/
COPY --from=builder /dist/etc/nsca.cfg /etc/
Since this is using multiple stages, your resulting image will not contain development files and will still be small.
Probably I'm missing something obvious, but could someone please explain the following:
When I pull and run an image, e.g docker pull dgraziotin/lamp && docker run -t -i -p 80:80 -p 3306:3306 --name osxlamp dgraziotin/lamp - it works just fine
Now I want to play with Dockerfile and build it manually on my computer (I can do this, right?)
So I download the source files from Github https://github.com/dgraziotin/osx-docker-lamp, cd to unpacked folder and run docker build -t test .
The building process starts but I see lot of weird errors like "Package php5-mysql is not available". I tried different images with the same result. How to properly build local images?
UPD:
Dockerfile
FROM phusion/baseimage:latest
MAINTAINER Daniel Graziotin <daniel#ineed.coffee>
ENV REFRESHED_AT 2016-03-29
# based on tutumcloud/tutum-docker-lamp
# MAINTAINER Fernando Mayo <fernando#tutum.co>, Feng Honglin <hfeng#tutum.co>
ENV DOCKER_USER_ID 501
ENV DOCKER_USER_GID 20
ENV BOOT2DOCKER_ID 1000
ENV BOOT2DOCKER_GID 50
# Tweaks to give Apache/PHP write permissions to the app
RUN usermod -u ${BOOT2DOCKER_ID} www-data && \
usermod -G staff www-data && \
useradd -r mysql && \
usermod -G staff mysql
RUN groupmod -g $(($BOOT2DOCKER_GID + 10000)) $(getent group $BOOT2DOCKER_GID | cut -d: -f1)
RUN groupmod -g ${BOOT2DOCKER_GID} staff
# Install packages
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && \
apt-get -y install supervisor wget git apache2 libapache2-mod-php5 mysql-server php5-mysql pwgen php-apc php5-mcrypt zip unzip && \
echo "ServerName localhost" >> /etc/apache2/apache2.conf
# needed for phpMyAdmin
run php5enmod mcrypt
# Add image configuration and scripts
ADD start-apache2.sh /start-apache2.sh
ADD start-mysqld.sh /start-mysqld.sh
ADD run.sh /run.sh
RUN chmod 755 /*.sh
ADD supervisord-apache2.conf /etc/supervisor/conf.d/supervisord-apache2.conf
ADD supervisord-mysqld.conf /etc/supervisor/conf.d/supervisord-mysqld.conf
# Remove pre-installed database
RUN rm -rf /var/lib/mysql
# Add MySQL utils
ADD create_mysql_users.sh /create_mysql_users.sh
RUN chmod 755 /*.sh
# Add phpmyadmin
RUN wget -O /tmp/phpmyadmin.tar.gz https://files.phpmyadmin.net/phpMyAdmin/4.6.0/phpMyAdmin-4.6.0-all-languages.tar.gz
RUN tar xfvz /tmp/phpmyadmin.tar.gz -C /var/www
RUN ln -s /var/www/phpMyAdmin-4.6.0-all-languages /var/www/phpmyadmin
RUN mv /var/www/phpmyadmin/config.sample.inc.php /var/www/phpmyadmin/config.inc.php
ENV MYSQL_PASS:-$(pwgen -s 12 1)
# config to enable .htaccess
ADD apache_default /etc/apache2/sites-available/000-default.conf
RUN a2enmod rewrite
# Configure /app folder with sample app
RUN mkdir -p /app && rm -fr /var/www/html && ln -s /app /var/www/html
ADD app/ /app
#Environment variables to configure php
ENV PHP_UPLOAD_MAX_FILESIZE 10M
ENV PHP_POST_MAX_SIZE 10M
# Add volumes for the app and MySql
VOLUME ["/etc/mysql", "/var/lib/mysql", "/app" ]
EXPOSE 80 3306
CMD ["/run.sh"]
SOLVED As I understood many of custom images contain outdated/invalid code and must be avoided as much as possible. We should rely on official well known and supported images.
Unrelated to the exact problem, but your Dockerfile could use some rework based on Best Practices for writing Dockerfiles.
I'd like to point out the ADD vs COPY best practice and the deprecated MAINTAINER Instruction (you should use LABEL maintainer="Daniel Graziotin ").
Also on the part where you add phpmyadmin it's useless to use RUN instead of ADD if you don't extract and delete the archive in the same layer (using multiline arguments). This can also be found under the ADD vs COPY best practices.
Other than that I can say this is a pretty solid Dockerfile! Sad it won't work because of the application...
I'm slowly making my way through the Riot Taking Control of your Docker Image tutorial http://engineering.riotgames.com/news/taking-control-your-docker-image. This tutorial is a little old, so there are some definite changes to how the end file looks. After hitting several walls I decided to work in the opposite order of the tutorial. I successfully folded the official jenkinsci image into my personal Dockerfile, starting with FROM: openjdk:8-dk. But when I try to fold in the openjdk:8-dk file into my personal image I receive the following error
E: Version '8u102-b14.1-1~bpo8+1' for 'openjdk-8-jdk' was not found
ERROR: Service 'jenkinsmaster' failed to build: The command '/bin/sh
-c set -x && apt-get update && apt-get install -y openjdk-8-jdk="$JAVA_DEBIAN_VERSION"
ca-certificates-java="$CA_CERTIFICATES_JAVA_VERSION" && rm -rf
/var/lib/apt/lists/* && [ "$JAVA_HOME" = "$(docker-java-home)" ]'
returned a non-zero code: 100 Cosettes-MacBook-Pro:docker-test
Cosette$
I'm receiving this error even when I gave up and directly copied and pasted the openjdk:8-jdk Dockerfile into my own. My end goal is to bring my personal Dockerfile down to the point that it starts FROM debian-jessie. Any help would be appreciated.
My Dockerfile:
FROM buildpack-deps:jessie-scm
# A few problems with compiling Java from source:
# 1. Oracle. Licensing prevents us from redistributing the official JDK.
# 2. Compiling OpenJDK also requires the JDK to be installed, and it gets
# really hairy.
RUN apt-get update && apt-get install -y --no-install-recommends \
bzip2 \
unzip \
xz-utils \
&& rm -rf /var/lib/apt/lists/*
RUN echo 'deb http://deb.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list
# Default to UTF-8 file.encoding
ENV LANG C.UTF-8
# add a simple script that can auto-detect the appropriate JAVA_HOME value
# based on whether the JDK or only the JRE is installed
RUN { \
echo '#!/bin/sh'; \
echo 'set -e'; \
echo; \
echo 'dirname "$(dirname "$(readlink -f "$(which javac || which java)")")"'; \
} > /usr/local/bin/docker-java-home \
&& chmod +x /usr/local/bin/docker-java-home
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV JAVA_VERSION 8u102
ENV JAVA_DEBIAN_VERSION 8u102-b14.1-1~bpo8+1
# see https://bugs.debian.org/775775
# and https://github.com/docker-library/java/issues/19#issuecomment-70546872
ENV CA_CERTIFICATES_JAVA_VERSION 20140324
RUN set -x \
&& apt-get update \
&& apt-get install -y \
openjdk-8-jdk="$JAVA_DEBIAN_VERSION" \
ca-certificates-java="$CA_CERTIFICATES_JAVA_VERSION" \
&& rm -rf /var/lib/apt/lists/* \
&& [ "$JAVA_HOME" = "$(docker-java-home)" ]
# see CA_CERTIFICATES_JAVA_VERSION notes above
RUN /var/lib/dpkg/info/ca-certificates-java.postinst configure
# Jenkins Specifics
# install Tini
ENV TINI_VERSION 0.9.0
ENV TINI_SHA fa23d1e20732501c3bb8eeeca423c89ac80ed452
# Use tini as subreaper in Docker container to adopt zombie processes
RUN curl -fsSL https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini-static -o /bin/tini && chmod +x /bin/tini \
&& echo "$TINI_SHA /bin/tini" | sha1sum -c -
# Set Jenkins Environmental Variables
ENV JENKINS_HOME /var/jenkins_home
ENV JENKINS_SLAVE_AGENT_PORT 50000
# jenkins version being bundled in this docker image
ARG JENKINS_VERSION
ENV JENKINS_VERSION ${JENKINS_VERSION:-2.19.1}
# jenkins.war checksum, download will be validated using it
ARG JENKINS_SHA=dc28b91e553c1cd42cc30bd75d0f651671e6de0b
ENV JENKINS_UC https://updates.jenkins.io
ENV COPY_REFERENCE_FILE_LOG $JENKINS_HOME/copy_reference_file.log
ENV JAVA_OPTS="-Xmx8192m"
ENV JENKINS_OPTS="--handlerCountMax=300 --logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war"
# Can be used to customize where jenkins.war get downloaded from
ARG JENKINS_URL=http://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
# Jenkins is run with user `jenkins`, uid = 1000. If you bind mount a volume from the host or a data
# container, ensure you use the same uid.
RUN groupadd -g ${gid} ${group} \
&& useradd -d "$JENKINS_HOME" -u ${uid} -g ${gid} -m -s /bin/bash ${user}
# Jenkins home directory is a volume, so configuration and build history
# can be persisted and survive image upgrades
VOLUME /var/jenkins_home
# `/usr/share/jenkins/ref/` contains all reference configuration we want
# to set on a fresh new installation. Use it to bundle additional plugins
# or config file with your custom jenkins Docker image.
RUN mkdir -p /usr/share/jenkins/ref/init.groovy.d
# Install Jenkins. Could use ADD but this one does not check Last-Modified header neither does it
# allow to control checksum. see https://github.com/docker/docker/issues/8331
RUN curl -fsSL ${JENKINS_URL} -o /usr/share/jenkins/jenkins.war \
&& echo "${JENKINS_SHA} /usr/share/jenkins/jenkins.war" | sha1sum -c -
# Prep Jenkins Directories
USER root
RUN chown -R ${user} "$JENKINS_HOME" /usr/share/jenkins/ref
RUN mkdir /var/log/jenkins
RUN mkdir /var/cache/jenkins
RUN chown -R ${group}:${user} /var/log/jenkins
RUN chown -R ${group}:${user} /var/cache/jenkins
# Expose ports for web (8080) & node (50000) agents
EXPOSE 8080
EXPOSE 50000
# Copy in local config filesfiles
COPY init.groovy /usr/share/jenkins/ref/init.groovy.d/tcp-slave-agent-port.groovy
COPY jenkins-support /usr/local/bin/jenkins-support
COPY jenkins.sh /usr/local/bin/jenkins.sh
# NOTE : Just set pluginID to download latest version of plugin.
# NOTE : All plugins need to be listed as there is no transitive dependency resolution.
# from a derived Dockerfile, can use `RUN plugins.sh active.txt` to setup
# /usr/share/jenkins/ref/plugins from a support bundle
COPY plugins.sh /usr/local/bin/plugins.sh
RUN chmod +x /usr/local/bin/plugins.sh
RUN chmod +x /usr/local/bin/jenkins.sh
# Switch to the jenkins user
USER ${user}
# Tini as the entry point to manage zombie processes
ENTRYPOINT ["/bin/tini", "--", "/usr/local/bin/jenkins.sh"]
Try a JAVA_DEBIAN_VERSION of 8u111-b14-2~bpo8+1
Here's what happens: when you build the docker file, docker tries to execute all the lines in the dockerfile. One of those is this apt command: apt-get install -y openjdk-8-jdk="$JAVA_DEBIAN_VERSION". This comand says "Install OpenJDK version $JAVA_DEBIAN_VERSION, exactly. Nothing else.". This version is no longer available in Debian repositories, so it can't be apt-get installed! I believe this happens with all packages in official mirrors: if a new version of the package is released, the older version is no longer around to be installed.
If you want to access older Debian packages, you can use something like http://snapshot.debian.org/. The older OpenJDK package has known security vulnerabilities. I recommend using the latest version.
You can use the latest version by leaving out the explicit version in the apt-get command. On the other hand, this will make your image less reproducible: building the image today may get you u111, building it tomorrow may get you u112.
As for why the instructions worked in the other Dockerfile, I think the reason is that at the time the other Dockerfile was built, the package was available. So docker could apt-get install it. Docker then built the image containing the (older) OpenJDK. That image is a binary, so you can install it, or use it in FROM without any issues. But you can't reproduce the image: if you were to try and build the same image yourself, you would run into the same errors.
This also brings up an issue about security updates: since docker images are effectively static binaries (built once, bundle in all dependencies), they don't get security updates once built. You need to keep track of any security updates affecting your docker images and rebuild any affected docker images.