building jenkins docker image from official Dockerfile - docker

I am trying to build a jenkins docker image from official jenkins git repo:
https://github.com/jenkinsci/docker.
But when I try to run the container of the image using docker run -it -dP jenkins, it exits immediately and when i check the docker logs, I get the following error:
: invalid option
I read that the error could be because the pid of tini is not 1. I looked at the documents and saw that if we do the following, it should solve the issue.
Passing the -s argument to Tini (tini -s -- ...)
Setting the environment variable TINI_SUBREAPER (e.g. export TINI_SUBREAPER=).
But it did not solve anything.
The following is the exact copy of the Dockerfile being built with docker build -t jenkins .:
FROM openjdk:8-jdk
RUN apt-get update && apt-get install -y git curl && rm -rf /var/lib/apt/lists/*
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
ARG http_port=8080
ARG agent_port=50000
ENV JENKINS_HOME /var/jenkins_home
ENV JENKINS_SLAVE_AGENT_PORT ${agent_port}
ENV TINI_SUBREAPER=
# Jenkins is run with user `jenkins`, uid = 1000
# If you bind mount a volume from the host or a data container,
# ensure you use the same uid
RUN groupadd -g ${gid} ${group} \
&& useradd -d "$JENKINS_HOME" -u ${uid} -g ${gid} -m -s /bin/bash ${user}
# Jenkins home directory is a volume, so configuration and build history
# can be persisted and survive image upgrades
VOLUME /var/jenkins_home
# `/usr/share/jenkins/ref/` contains all reference configuration we want
# to set on a fresh new installation. Use it to bundle additional plugins
# or config file with your custom jenkins Docker image.
RUN mkdir -p /usr/share/jenkins/ref/init.groovy.d
ENV TINI_VERSION 0.14.0
ENV TINI_SHA 6c41ec7d33e857d4779f14d9c74924cab0c7973485d2972419a3b7c7620ff5fd
# Use tini as subreaper in Docker container to adopt zombie processes
RUN curl -fsSL https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini-static-amd64 -o /bin/tini && chmod +x /bin/tini \
&& echo "$TINI_SHA /bin/tini" | sha256sum -c -
COPY init.groovy /usr/share/jenkins/ref/init.groovy.d/tcp-slave-agent-port.groovy
# jenkins version being bundled in this docker image
ARG JENKINS_VERSION
ENV JENKINS_VERSION ${JENKINS_VERSION:-2.60.1}
# jenkins.war checksum, download will be validated using it
ARG JENKINS_SHA=34fde424dde0e050738f5ad1e316d54f741c237bd380bd663a07f96147bb1390
# Can be used to customize where jenkins.war get downloaded from
ARG JENKINS_URL=https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war
# could use ADD but this one does not check Last-Modified header neither does it allow to control checksum
# see https://github.com/docker/docker/issues/8331
RUN curl -fsSL ${JENKINS_URL} -k -o /usr/share/jenkins/jenkins.war \
&& echo "${JENKINS_SHA} /usr/share/jenkins/jenkins.war" | sha256sum -c -
ENV JENKINS_UC https://updates.jenkins.io
RUN chown -R ${user} "$JENKINS_HOME" /usr/share/jenkins/ref
# for main web interface:
EXPOSE ${http_port}
# will be used by attached slave agents:
EXPOSE ${agent_port}
ENV COPY_REFERENCE_FILE_LOG $JENKINS_HOME/copy_reference_file.log
USER ${user}
COPY jenkins-support /usr/local/bin/jenkins-support
COPY jenkins.sh /usr/local/bin/jenkins.sh
ENTRYPOINT ["/bin/tini", "--", "/usr/local/bin/jenkins.sh"]
# from a derived Dockerfile, can use `RUN plugins.sh active.txt` to setup /usr/share/jenkins/ref/plugins from a support bundle
COPY plugins.sh /usr/local/bin/plugins.sh
COPY install-plugins.sh /usr/local/bin/install-plugins.sh

The problem was with the docker version. My Docker version was old. Not sure which command was not supported, but the new docker built the dockerfile.

Related

Creating an Elastic search image. Issue with non-root user and image size

Solution - Used Multi-stage build file to reduce the size significantly. solution pasted below
Edit: Already played with default elastic search image, But the purpose of this exercise is to learn working of Dockerfile. That is how i noticed my image being much larger (2.5G) than official image (742 M)
New to dockers/containers landscape ( was using vagrants till now ) .
To get better understanding of Dockerfile working, decided to create an ES image ( similar to ones i created in the past for a vagrant box ) .
Can someone help in reviewing the docker file and answer following issues being encountered.
Running ES as root is not allowed and running it from /home/newuser gives following error.
What am i missing here ? How should i create a new user/group to resolve this issue.
newuser#9f5820d430eb:~$ elasticsearch
Exception in thread "main" java.lang.RuntimeException: starting java failed with [1]
output:
[0.000s][error][logging] Error opening log file 'logs/gc.log': Permission denied
[0.001s][error][logging] Initialization of output 'file=logs/gc.log' using options 'filecount=32,filesize=64m' failed.
error:
Invalid -Xlog option '-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m', see error log for details.
Error: Could not create the Java Virtual Machine.
The dockerfile is installing JDK and ES but the images size is over 2GB. Can this be reduced . Found something called multi-stage images. not sure how to fit that concept in my dockerfile.
In vagrant provisions i update paths to /etc/environment.
Should this be done for containers ? I am not sure if it adds any value.
Docerkfile -
# Base image stage 1
FROM ubuntu
#MAINTAINER demo#gmail.com
LABEL maintainer="demo#foo.com"
############################################
### Install openjava
############################################
#RUN apt-get update
ARG JAVA_HOME=/opt/java
ARG JDK_PACKAGE=openjdk-14.0.2_linux-x64_bin.tar.gz
# setup paths
ENV JAVA_HOME $JAVA_HOME
# Setup JAVA_HOME, this is useful for docker commandline
ENV PATH $PATH:$JAVA_HOME/bin
## write to environment file for all future sessions
# sudo /bin/sh -c 'echo JAVA_HOME="/opt/java/" >> /etc/environment'
# sudo /bin/sh -c '. /etc/environment ; echo PATH="$JAVA_HOME/bin:$PATH" >> /etc/environment'
## download open java
# ADD https://download.java.net/java/GA/jdk14.0.2/205943a0976c4ed48cb16f1043c5c647/12/GPL/$JDK_PACKAGE /
# ADD $JDK_PACKAGE /
COPY $JDK_PACKAGE /
RUN mkdir -p $JAVA_HOME/ && \
tar -zxf /$JDK_PACKAGE --strip-components 1 -C $JAVA_HOME && \
rm -f /$JDK_PACKAGE
############################################
### Install elastic search
############################################
ARG ES_HOME=/opt/elasticsearch
ARG ES_PACKAGE=elasticsearch-7.10.1-linux-x86_64.tar.gz
# setup paths
ENV ES_HOME $ES_HOME
# Setup ES_HOME, this is useful for docker commandline
ENV PATH $PATH:$ES_HOME/bin
##write to environment file for all future sessions
#sudo /bin/sh -c 'echo ES_HOME="/opt/elasticsearch/" >> /etc/environment'
#sudo /bin/sh -c '. /etc/environment ; echo PATH="$ES_HOME/bin:$PATH" >> /etc/environment'
## download es
# ADD https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.1-linux-x86_64.tar.gz /
# ADD $JDK_PACKAGE /
COPY $ES_PACKAGE /
RUN mkdir -p $ES_HOME/ && \
tar -zxf /$ES_PACKAGE --strip-components 1 -C $ES_HOME && \
rm -f /$ES_PACKAGE
# Mount elasticsearch.yml config
ADD config/elasticsearch.yml /elasticsearch/config/elasticsearch.yml
#ADD config/elasticsearch.yml /
############################################
### Others
############################################
# Expose ports
EXPOSE 9200
EXPOSE 9300
## give permission to entire elasticsearch setup directory
RUN chmod 755 -R $ES_HOME
RUN chmod 755 -R $JAVA_HOME
RUN chmod 755 -R /var/log
# add non root user
RUN useradd newuser --create-home --shell /bin/bash
RUN echo 'newuser:newpassword' | chpasswd
RUN adduser newuser sudo
USER newuser
WORKDIR /home/newuser
# Define default command.
#CMD ["elasticsearch"]
Solution -
Multi stage build file with non-root user
ARG JAVA_HOME=/opt/java
ARG JDK_PACKAGE=openjdk-14.0.2_linux-x64_bin.tar.gz
ARG ES_HOME=/opt/elasticsearch
ARG ES_PACKAGE=elasticsearch-7.10.1-linux-x86_64.tar.gz
#MAINTAINER demo#gmail.com
#LABEL maintainer="demo#foo.com"
############################################
### Install openjava
############################################
# Base image stage 1
FROM ubuntu as jdk
ARG JAVA_HOME
ARG JDK_PACKAGE
WORKDIR /opt/
## download open java
# ADD https://download.java.net/java/GA/jdk14.0.2/205943a0976c4ed48cb16f1043c5c647/12/GPL/$JDK_PACKAGE /
# ADD $JDK_PACKAGE /
COPY $JDK_PACKAGE .
RUN mkdir -p $JAVA_HOME/ && \
tar -zxf $JDK_PACKAGE --strip-components 1 -C $JAVA_HOME && \
rm -f $JDK_PACKAGE
############################################
### Install elastic search
############################################
# Base image stage 2
From ubuntu as es
#ARG JAVA_HOME
ARG ES_HOME
ARG ES_PACKAGE
WORKDIR /opt/
## download es
# ADD https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.1-linux-x86_64.tar.gz /
# ADD $JDK_PACKAGE /
COPY $ES_PACKAGE .
RUN mkdir -p $ES_HOME/ && \
tar -zxf $ES_PACKAGE --strip-components 1 -C $ES_HOME && \
rm -f $ES_PACKAGE
# Mount elasticsearch.yml config
ADD config/elasticsearch.yml /opt/elasticsearch/config/elasticsearch.yml
############################################
### final
############################################
From ubuntu as finalbuild
ARG JAVA_HOME
ARG ES_HOME
ARG ES_PACKAGE
WORKDIR /opt/
# get artifacts from previous stages
COPY --from=jdk $JAVA_HOME $JAVA_HOME
COPY --from=es $ES_HOME $ES_HOME
# Setup JAVA_HOME, this is useful for docker commandline
ENV JAVA_HOME $JAVA_HOME
ENV ES_HOME $ES_HOME
# setup paths
ENV PATH $PATH:$JAVA_HOME/bin
ENV PATH $PATH:$ES_HOME/bin
# Expose ports
EXPOSE 9200
EXPOSE 9300
# Define mountable directories.
#VOLUME ["/data"]
## give permission to entire elasticsearch setup directory
RUN useradd newuser --create-home --shell /bin/bash && \
echo 'newuser:newpassword' | chpasswd && \
chown -R newuser $ES_HOME $JAVA_HOME && \
chown -R newuser:newuser /home/newuser && \
chmod 755 /home/newuser
#chown -R newuser:newuser /home/newuser
#chown -R newuser /home/newuser && \
USER newuser
WORKDIR /home/newuser
#RUN chown -R newuser /home/newuser
#RUN apt-get update && \
# apt-get install -yq curl
# Define default command.
CMD ["elasticsearch"]
The concept of docker is that you have tons of out-of-the-box images ready for you!
Why do you want to build your own Dockerfile for a common tech like Elasticsearch?
Why not simply:
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.10.1
and you have the image ready locally for run?
You can read more about running Elasticseach with docker here.
BTW, this image size is ~774MB
EDIT:
If it's for learning purpose, I can recommend dive which can analyze baked images (like the elasticsearch:7.10.1 and shows you each step of the image build (in other words, the dockerfile that built that image) and the base image it start FROM.

Gradle installation through docker file

I wrote a docker file with gradle installations inside it. It shows Gradle version with gradle -v command but while I am running through jenkins job with gradle -v command in execute shell while building a job it shows as gradle:not found
Please check the image mentioned
This is gradle installation in docker file
#Install gradle
RUN cd /usr/lib \
&& wget https://downloads.gradle.org/distributions/gradle-3.4.1-bin.zip -o gradle-bin.zip \
&& unzip "gradle-3.4.1-bin.zip" \
&& ln -s "/usr/gradle-3.4.1/bin/gradle" /usr/bin/gradle \
&& rm "gradle-bin.zip"
#Env set up
ENV GRADLE_HOME=usr/lib/gradle-3.4.1
#ENV PATH=$PATH:$GRADLE_HOME/bin:$PATH
ENV PATH=$PATH:$GRADLE_HOME/bin JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
Try this, work for me.
# Start with a base image containing Java runtime
FROM openjdk:8-jdk-alpine
# Add Maintainer Info
# Add a volume pointing to /tmp
VOLUME /tmp
# Make port 8080 available to the world outside this container
EXPOSE 8080
RUN mkdir /app
WORKDIR /app
COPY . /app
RUN ./gradlew build
ENTRYPOINT ["java","-jar","./build/libs/app-0.1.0.jar"]

How to access docker daemon from container with other user than root

I'm trying to run a Jenkins container that builds docker images. I've started last week with docker and I'm a bit confused with the use of volumes from host and how users are handled.
I've been searching on internet and I've found a git issue were someone posted a solution to have access to the docker daemon from the container. Basically, the idea is to mound inside the Jenkins container the volumes that contain the docker bin folder and the docker.sock from the host like this:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /usr/local/bin/docker:/usr/local/bin/docker
I've done that and it works but only if I'm root. When I started to learn docker, I followed the example in a blog where, instead of directly using a jenkins image, the author copied the Dockerfiles from the jenkins image itself and its dependencies to explain the process. As part of the process, a jenkins user is created and it is the one in used when starting the container. My problem now is that I cannot make the jenkins user have access to the docker.sock mounted as it belongs to root and the group docker in the host. I tried adding the user docker in the Dockerfile but I still get a permission denied error from a Jenkins job when accessing the docker.sock. If I inspect the mounted /var/run/docker.sock inside the container I can see that docker.sock belongs to group user instead of docker so I don't know exactly what's going on when the directory is mounted. I haven't worked much with Linux so my guess is that the user docker doesn't exist when the directory is mounted and that it then uses a default user but I may probably be completely wrong.
Another thing I still don't get is, if I create a container specifically to be used as a Jenkins container and nothing else is supposed to be run there, what's the purpose of creating a specific jenkins user? Is there any reason why I cannot use directly the user root?
This is the Dockerfile I use. Thanks.
FROM centos:7
# Yum workaround to stalled mirror
RUN sed -i -e 's/enabled=1/enabled=0/g' /etc/yum/pluginconf.d/fastestmirror.conf
RUN rm -f /var/lib/rpm/__*
RUN rpm --rebuilddb -v -v
RUN yum clean all
# see https://bugs.debian.org/775775
# and https://github.com/docker-library/java/issues/19#issuecomment-70546872
ENV CA_CERTIFICATES_JAVA_VERSION 20140324
RUN yum -v install -y \
wget \
zip \
which \
openssh-client \
unzip \
java-1.8.0-openjdk-devel \
git \
&& yum clean all
#RUN /var/lib/dpkg/info/ca-certificates-java.postinst configure
# Install Tini
ENV TINI_VERSION 0.9.0
ENV TINI_SHA fa23d1e20732501c3bb8eeeca423c89ac80ed452
# Use tini as subreaper in Docker container to adopt zombie processes
RUN curl -fsSL https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini-static -o /bin/tini && chmod +x /bin/tini \
&& echo "$TINI_SHA /bin/tini" | sha1sum -c -
# SET Jenkins Environment Variables
ENV JENKINS_HOME /var/jenkins_home
ENV JENKINS_SLAVE_AGENT_PORT 50000
ENV JENKINS_VERSION 2.22
ENV JENKINS_SHA 5b89b6967e7af8119c52c7e86223b47665417a22
ENV JENKINS_UC https://updates.jenkins-ci.org
ENV COPY_REFERENCE_FILE_LOG $JENKINS_HOME/copy_reference_file.log
# SET Java variables
ENV JAVA_HOME /usr/lib/jvm/java/jre
ENV PATH /usr/lib/jvm/java/bin:$PATH
# Jenkins is run with user `jenkins`, uid = 1000
# If you bind mount a volume from the host or a data container,
# ensure you use the same uid
RUN useradd -d "$JENKINS_HOME" -u 1000 -m -s /bin/bash jenkins
#Not working. Folder not yet mounted?
#RUN DOCKER_GID=$(stat -c '%g' /var/run/docker.sock) && \
#Using gid from host
RUN groupadd -for -g 50 docker && \
usermod -aG docker jenkins
# Jenkins home directory is a volume, so configuration and build history
# can be persisted and survive image upgrades
VOLUME /var/jenkins_home
# `/usr/share/jenkins/ref/` contains all reference configuration we want
# to set on a fresh new installation. Use it to bundle additional plugins
# or config file with your custom jenkins Docker image.
RUN mkdir -p /usr/share/jenkins/ref/init.groovy.d
# Install Jenkins
RUN curl -fL http://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war -o /usr/share/jenkins/jenkins.war \
&& echo "$JENKINS_SHA /usr/share/jenkins/jenkins.war" | sha1sum -c -
ENV JAVA_OPTS="-Xmx8192m"
ENV JENKINS_OPTS="--logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war"
# Prep Jenkins Directories
RUN chown -R jenkins "$JENKINS_HOME" /usr/share/jenkins/ref
RUN mkdir /var/log/jenkins
RUN mkdir /var/cache/jenkins
RUN chown -R jenkins:jenkins /var/log/jenkins
RUN chown -R jenkins:jenkins /var/cache/jenkins
# Expose Ports for web and slave agents
EXPOSE 8080
EXPOSE 50000
# Copy in local config files
COPY init.groovy /usr/share/jenkins/ref/init.groovy.d/tcp-slave-agent-port.groovy
COPY jenkins.sh /usr/local/bin/jenkins.sh
COPY plugins.sh /usr/local/bin/plugins.sh
RUN chmod +x /usr/local/bin/plugins.sh
RUN chmod +x /usr/local/bin/jenkins.sh
# Install default plugins
COPY plugins.txt /tmp/plugins.txt
RUN /usr/local/bin/plugins.sh /tmp/plugins.txt
# Add ssh key
RUN eval "$(ssh-agent -s)"
RUN mkdir /usr/share/jenkins/ref/.ssh && \
chmod 700 /usr/share/jenkins/ref/.ssh && \
ssh-keyscan github.com > /usr/share/jenkins/ref/.ssh/known_hosts
COPY id_rsa /usr/share/jenkins/ref/.ssh/id_rsa
COPY id_rsa /usr/share/jenkins/ref/.ssh/id_rsa.pub
COPY hudson.tasks.Maven.xml /usr/share/jenkins/ref/hudson.tasks.Maven.xml
RUN chown -R jenkins:jenkins /usr/share/jenkins/ref && \
chmod 600 /usr/share/jenkins/ref/.ssh/id_rsa && \
chmod 600 /usr/share/jenkins/ref/.ssh/id_rsa.pub && \
chmod 600 /usr/share/jenkins/ref/hudson.tasks.Maven.xml
COPY id_rsa /root/.ssh/id_rsa
COPY id_rsa /root/.ssh/id_rsa.pub
# ssh keys for root. To use root as the user
RUN chmod 600 /root/.ssh/id_rsa && \
chmod 600 /root/.ssh/id_rsa.pub && \
ssh-keyscan github.com > /root/.ssh/known_hosts
# Switch to the jenkins user
USER jenkins
# Tini as the entry point to manage zombie processes
ENTRYPOINT ["/bin/tini", "--", "/usr/local/bin/jenkins.sh"]
Apparently the issue was in the gid. For some reason I thought the docker gid of the group in the host was 50 but actually it was actually 100. When I changed it to be 100, the jenkins job started to work.
I still don't know why docker.sock shows it belongs to group user instead of docker in the container though. If I do cat /etc/group in the container I see
root:x:0:
...
users:x:100:
...
jenkins:x:1000:
docker:x:100:jenkins
and in the host
root:x:0:
lp:x:7:lp
nogroup:x:65534:
staff:x:50:docker
docker:x:100:docker
dockremap:x:101:dockremap

How to create a Jenkins job and/or user from a dockerfile?

I am trying to set up a customised Jenkins 2 server from a dockerfile.
I use the official image and I want to be able to add things that I need like custom jobs and an admin user.
This is my dockerfile so far:
FROM openjdk:8-jdk
RUN apt-get update && apt-get install -y git curl && rm -rf /var/lib/apt/lists/*
ENV JENKINS_HOME /var/jenkins_home
ENV JENKINS_SLAVE_AGENT_PORT 50000
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
# Jenkins is run with user `jenkins`, uid = 1000
# If you bind mount a volume from the host or a data container,
# ensure you use the same uid
RUN groupadd -g ${gid} ${group} \
&& useradd -d "$JENKINS_HOME" -u ${uid} -g ${gid} -m -s /bin/bash ${user}
# Jenkins home directory is a volume, so configuration and build history
# can be persisted and survive image upgrades
VOLUME /var/jenkins_home
# `/usr/share/jenkins/ref/` contains all reference configuration we want
# to set on a fresh new installation. Use it to bundle additional plugins
# or config file with your custom jenkins Docker image.
RUN mkdir -p /usr/share/jenkins/ref/init.groovy.d
ENV TINI_VERSION 0.9.0
ENV TINI_SHA fa23d1e20732501c3bb8eeeca423c89ac80ed452
# Use tini as subreaper in Docker container to adopt zombie processes
RUN curl -fsSL https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini-static -o /bin/tini && chmod +x /bin/tini \
&& echo "$TINI_SHA /bin/tini" | sha1sum -c -
COPY init.groovy /usr/share/jenkins/ref/init.groovy.d/tcp-slave-agent-port.groovy
# jenkins version being bundled in this docker image
ARG JENKINS_VERSION
ENV JENKINS_VERSION ${JENKINS_VERSION:-2.19.2}
# jenkins.war checksum, download will be validated using it
ARG JENKINS_SHA=32b8bd1a86d6d4a91889bd38fb665db4090db081
# Can be used to customize where jenkins.war get downloaded from
ARG JENKINS_URL=https://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war
# could use ADD but this one does not check Last-Modified header neither does it allow to control checksum
# see https://github.com/docker/docker/issues/8331
RUN curl -fsSL ${JENKINS_URL} -o /usr/share/jenkins/jenkins.war \
&& echo "${JENKINS_SHA} /usr/share/jenkins/jenkins.war" | sha1sum -c -
ENV JENKINS_UC https://updates.jenkins.io
RUN chown -R ${user} "$JENKINS_HOME" /usr/share/jenkins/ref
# for main web interface:
EXPOSE 8080
# will be used by attached slave agents:
EXPOSE 50000
ENV COPY_REFERENCE_FILE_LOG $JENKINS_HOME/copy_reference_file.log
USER ${user}
COPY jenkins-support /usr/local/bin/jenkins-support
COPY jenkins.sh /usr/local/bin/jenkins.sh
ENTRYPOINT ["/bin/tini", "--", "/usr/local/bin/jenkins.sh"]
# from a derived Dockerfile, can use `RUN plugins.sh active.txt` to setup /usr/share/jenkins/ref/plugins from a support bundle
COPY plugins.txt /usr/share/jenkins/plugins.txt
COPY plugins.sh /usr/local/bin/plugins.sh
COPY install-plugins.sh /usr/local/bin/install-plugins.sh
# Add the command line tools
COPY jenkins-cli.jar "$JENKINS_HOME"
# Create jobs
ARG job_name_1="my_super_job"
#ARG job_name_2="my_ultra_job"
# create the jobs folder recursively
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/workspace/
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/builds
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/builds/lastFailedBuild
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/builds/lastStableBuild
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/builds/lastSuccessfulBuild
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/builds/lastUnstableBuild
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/builds/lastUnsuccessfulBuild
RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_1}/builds/legacyIds
#RUN mkdir -p "$JENKINS_HOME"/jobs/${job_name_2}
## add the custom configs to the container
COPY ${job_name_1}_config.xml "$JENKINS_HOME"/jobs/${job_name_1}/config.xml
USER root
#RUN chmod 600 "$JENKINS_HOME"/jobs/${job_name_1}/config.xml
RUN java -jar /var/jenkins_home/jenkins-cli.jar -s http://localhost:8080 create-job my_super_job < /var/jenkins_home/jobs/my_super_job/config.xml
#COPY ${job_name_2}_config.xml "$JENKINS_HOME"/jobs/${job_name_2}/config.xml
# --Install plugins--
# Notice: Deprecated method which however works with a 'plugins.txt' file
#USER root
#RUN chmod 600 /usr/share/jenkins/plugins.txt
#RUN chmod 600 /usr/local/bin/install-plugins.sh
#RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins.txt
# Notice: Recommended method with open case on Github [https://github.com/jenkinsci/docker/issues/348]
# Notice: Select whichever plugins you want
#RUN /usr/local/bin/install-plugins.sh \
#dashboard-view:2.9.10 \
#pipeline-stage-view:2.2 \
#parameterized-trigger:2.32 \
#bitbucket:1.1.5 \
#git:3.0.0 \
#github:1.22.4
# --Install plugins--
I have tried to create a job on build time by first launching a container, creating the job manually, saving the config.xml file, and then copying it in the image from the Dockerfile. Moreover, I am trying to replicate the files/folder structure when a job is being created.
But it is not working. The job is not appearing in Jenkins.
I also tried to use the jenkins-cli.jar, but as I understood , there must be a live Jenkins server to connect to and execute anything which is not the case at build time.
Finally, I suppose creating an admin user in build time must be way more complicated that creating a job...
So, does anyone have any experience on this?

Can't build openjdk:8-jdk image directly

I'm slowly making my way through the Riot Taking Control of your Docker Image tutorial http://engineering.riotgames.com/news/taking-control-your-docker-image. This tutorial is a little old, so there are some definite changes to how the end file looks. After hitting several walls I decided to work in the opposite order of the tutorial. I successfully folded the official jenkinsci image into my personal Dockerfile, starting with FROM: openjdk:8-dk. But when I try to fold in the openjdk:8-dk file into my personal image I receive the following error
E: Version '8u102-b14.1-1~bpo8+1' for 'openjdk-8-jdk' was not found
ERROR: Service 'jenkinsmaster' failed to build: The command '/bin/sh
-c set -x && apt-get update && apt-get install -y openjdk-8-jdk="$JAVA_DEBIAN_VERSION"
ca-certificates-java="$CA_CERTIFICATES_JAVA_VERSION" && rm -rf
/var/lib/apt/lists/* && [ "$JAVA_HOME" = "$(docker-java-home)" ]'
returned a non-zero code: 100 Cosettes-MacBook-Pro:docker-test
Cosette$
I'm receiving this error even when I gave up and directly copied and pasted the openjdk:8-jdk Dockerfile into my own. My end goal is to bring my personal Dockerfile down to the point that it starts FROM debian-jessie. Any help would be appreciated.
My Dockerfile:
FROM buildpack-deps:jessie-scm
# A few problems with compiling Java from source:
# 1. Oracle. Licensing prevents us from redistributing the official JDK.
# 2. Compiling OpenJDK also requires the JDK to be installed, and it gets
# really hairy.
RUN apt-get update && apt-get install -y --no-install-recommends \
bzip2 \
unzip \
xz-utils \
&& rm -rf /var/lib/apt/lists/*
RUN echo 'deb http://deb.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list
# Default to UTF-8 file.encoding
ENV LANG C.UTF-8
# add a simple script that can auto-detect the appropriate JAVA_HOME value
# based on whether the JDK or only the JRE is installed
RUN { \
echo '#!/bin/sh'; \
echo 'set -e'; \
echo; \
echo 'dirname "$(dirname "$(readlink -f "$(which javac || which java)")")"'; \
} > /usr/local/bin/docker-java-home \
&& chmod +x /usr/local/bin/docker-java-home
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV JAVA_VERSION 8u102
ENV JAVA_DEBIAN_VERSION 8u102-b14.1-1~bpo8+1
# see https://bugs.debian.org/775775
# and https://github.com/docker-library/java/issues/19#issuecomment-70546872
ENV CA_CERTIFICATES_JAVA_VERSION 20140324
RUN set -x \
&& apt-get update \
&& apt-get install -y \
openjdk-8-jdk="$JAVA_DEBIAN_VERSION" \
ca-certificates-java="$CA_CERTIFICATES_JAVA_VERSION" \
&& rm -rf /var/lib/apt/lists/* \
&& [ "$JAVA_HOME" = "$(docker-java-home)" ]
# see CA_CERTIFICATES_JAVA_VERSION notes above
RUN /var/lib/dpkg/info/ca-certificates-java.postinst configure
# Jenkins Specifics
# install Tini
ENV TINI_VERSION 0.9.0
ENV TINI_SHA fa23d1e20732501c3bb8eeeca423c89ac80ed452
# Use tini as subreaper in Docker container to adopt zombie processes
RUN curl -fsSL https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini-static -o /bin/tini && chmod +x /bin/tini \
&& echo "$TINI_SHA /bin/tini" | sha1sum -c -
# Set Jenkins Environmental Variables
ENV JENKINS_HOME /var/jenkins_home
ENV JENKINS_SLAVE_AGENT_PORT 50000
# jenkins version being bundled in this docker image
ARG JENKINS_VERSION
ENV JENKINS_VERSION ${JENKINS_VERSION:-2.19.1}
# jenkins.war checksum, download will be validated using it
ARG JENKINS_SHA=dc28b91e553c1cd42cc30bd75d0f651671e6de0b
ENV JENKINS_UC https://updates.jenkins.io
ENV COPY_REFERENCE_FILE_LOG $JENKINS_HOME/copy_reference_file.log
ENV JAVA_OPTS="-Xmx8192m"
ENV JENKINS_OPTS="--handlerCountMax=300 --logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war"
# Can be used to customize where jenkins.war get downloaded from
ARG JENKINS_URL=http://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
# Jenkins is run with user `jenkins`, uid = 1000. If you bind mount a volume from the host or a data
# container, ensure you use the same uid.
RUN groupadd -g ${gid} ${group} \
&& useradd -d "$JENKINS_HOME" -u ${uid} -g ${gid} -m -s /bin/bash ${user}
# Jenkins home directory is a volume, so configuration and build history
# can be persisted and survive image upgrades
VOLUME /var/jenkins_home
# `/usr/share/jenkins/ref/` contains all reference configuration we want
# to set on a fresh new installation. Use it to bundle additional plugins
# or config file with your custom jenkins Docker image.
RUN mkdir -p /usr/share/jenkins/ref/init.groovy.d
# Install Jenkins. Could use ADD but this one does not check Last-Modified header neither does it
# allow to control checksum. see https://github.com/docker/docker/issues/8331
RUN curl -fsSL ${JENKINS_URL} -o /usr/share/jenkins/jenkins.war \
&& echo "${JENKINS_SHA} /usr/share/jenkins/jenkins.war" | sha1sum -c -
# Prep Jenkins Directories
USER root
RUN chown -R ${user} "$JENKINS_HOME" /usr/share/jenkins/ref
RUN mkdir /var/log/jenkins
RUN mkdir /var/cache/jenkins
RUN chown -R ${group}:${user} /var/log/jenkins
RUN chown -R ${group}:${user} /var/cache/jenkins
# Expose ports for web (8080) & node (50000) agents
EXPOSE 8080
EXPOSE 50000
# Copy in local config filesfiles
COPY init.groovy /usr/share/jenkins/ref/init.groovy.d/tcp-slave-agent-port.groovy
COPY jenkins-support /usr/local/bin/jenkins-support
COPY jenkins.sh /usr/local/bin/jenkins.sh
# NOTE : Just set pluginID to download latest version of plugin.
# NOTE : All plugins need to be listed as there is no transitive dependency resolution.
# from a derived Dockerfile, can use `RUN plugins.sh active.txt` to setup
# /usr/share/jenkins/ref/plugins from a support bundle
COPY plugins.sh /usr/local/bin/plugins.sh
RUN chmod +x /usr/local/bin/plugins.sh
RUN chmod +x /usr/local/bin/jenkins.sh
# Switch to the jenkins user
USER ${user}
# Tini as the entry point to manage zombie processes
ENTRYPOINT ["/bin/tini", "--", "/usr/local/bin/jenkins.sh"]
Try a JAVA_DEBIAN_VERSION of 8u111-b14-2~bpo8+1
Here's what happens: when you build the docker file, docker tries to execute all the lines in the dockerfile. One of those is this apt command: apt-get install -y openjdk-8-jdk="$JAVA_DEBIAN_VERSION". This comand says "Install OpenJDK version $JAVA_DEBIAN_VERSION, exactly. Nothing else.". This version is no longer available in Debian repositories, so it can't be apt-get installed! I believe this happens with all packages in official mirrors: if a new version of the package is released, the older version is no longer around to be installed.
If you want to access older Debian packages, you can use something like http://snapshot.debian.org/. The older OpenJDK package has known security vulnerabilities. I recommend using the latest version.
You can use the latest version by leaving out the explicit version in the apt-get command. On the other hand, this will make your image less reproducible: building the image today may get you u111, building it tomorrow may get you u112.
As for why the instructions worked in the other Dockerfile, I think the reason is that at the time the other Dockerfile was built, the package was available. So docker could apt-get install it. Docker then built the image containing the (older) OpenJDK. That image is a binary, so you can install it, or use it in FROM without any issues. But you can't reproduce the image: if you were to try and build the same image yourself, you would run into the same errors.
This also brings up an issue about security updates: since docker images are effectively static binaries (built once, bundle in all dependencies), they don't get security updates once built. You need to keep track of any security updates affecting your docker images and rebuild any affected docker images.

Resources