Dockerfile to create an image to be used as an container agent in jenkins - docker

I am trying to use a node agent container in Jenkins to run npm instructions on it. So that, I am creating a Dockerfile to get a valid image with ssh and nodejs. The executor runs fine, but when I use npm it says that it doesn't know the command.
The same problem happens when (after building the dockerfile) I do docker exec -it af5451297d85 bash and after that, inside the container, I try to do npm --v (for example).
# This Dockerfile is used to build an image containing an node jenkins agent
FROM node:9.0
MAINTAINER Estefania Castro <estefania.castro#luceit.es>
# Upgrade and Install packages
RUN apt-get update && apt-get -y upgrade && apt-get install -y git openssh-server
# Install NGINX to test.
RUN apt-get install nginx -y
# Prepare container for ssh
RUN mkdir /var/run/sshd && adduser --quiet jenkins && echo "jenkins:jenkins" | chpasswd
RUN npm install
ENV CI=true
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
I would like to run npm instructions like npm install, npm publish, ... to manage my project in a jenkinsfile. Could anyone help?
Thanks


I have already solved the problem (after two weeks haha).
FROM jenkins/ssh-slave
# Install selected extensions and other stuff
RUN apt-get update && apt-get -y --no-install-recommends install && apt-get clean
RUN apt-get install -y curl
# Install nodejs
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
RUN apt-get install -y nodejs && apt-get install -y nginx

Related

Container docker remains in exited status

Hi I'm building a docker image that starts Mysql, Elasticsearch and application, it completes the build of the image but when I run the container it doesn't start and remains in exited status.
DOCKERFILE:
FROM ubuntu:20.04
WORKDIR \\wsl.localhost\Ubuntu\home\marco\project\multiple-docker-container\backend
COPY . .
RUN apt update
RUN apt-get update
RUN apt-get install wget -y
RUN apt install apt-transport-https ca-certificates gnupg -y
RUN wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
RUN sh -c 'echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" > /etc/apt/sources.list.d/elastic-7.x.list'
RUN apt-get update -y
RUN apt-get install elasticsearch -y
RUN apt install mysql-server -y
RUN apt install npm -y
RUN npm install
RUN npm install nodemon
RUN apt install nodejs -y
EXPOSE 3306
EXPOSE 9200
EXPOSE 5000
COPY entrypoint.sh /bin/entrypoint.sh
RUN chmod +x /bin/entrypoint.sh
ENTRYPOINT ["/bin/entrypoint.sh"]
ENTRYPOINT.SH
#!/bin/sh
service elasticsearch start
service elasticsearch enable
service mysql start
npm start
I expect at least that it goes up, I would like to understand which is the problem since on WSL ubuntu gives me this problem while on the virtual machine it works

Docker: cannot install openssh-server

I build the Docker based on Ubuntu 16 and want to allow PuTTY access to the Ubuntu.
I have added the line to the docker file:
#Download base image ubuntu 16.04
FROM ubuntu:16.04
# Update Software repository
RUN apt-get update
# Install nginx, php-fpm and supervisord from ubuntu repository
RUN apt-get install -y nginx php7.0-fpm supervisor && \
rm -rf /var/lib/apt/lists/*
RUN apt-get autoclean -y supervisor
RUN apt-get install openssh-server -y supervisor
But when I build the image it gives me
Step 5/18 : RUN apt-get install openssh-server -y supervisor --->
Running in c9425deece29 Reading package lists... Building dependency
tree... Reading state information... E: Unable to locate package
openssh-server
How to fix it? My task is: to allow connection from a host (Windows) to the docker container via PuTTY.

Following Dockerfile should work.
#Download base image ubuntu 16.04
FROM ubuntu:16.04
# Update Software repository
RUN apt-get update && \
apt-get upgrade -y
RUN apt-get install openssh-server -y supervisor
# Install nginx, php-fpm and supervisord from ubuntu repository
RUN apt-get install -y nginx php7.0-fpm supervisor && \
rm -rf /var/lib/apt/lists/*
RUN apt-get autoclean -y supervisor
There is two thing it seems problematic to me.
After update I'm always using upgrade to update all packages on my system. It's not necessary but I find it's a good practice
You are removing /var/lib/apt/lists/ * then you are trying to install openssh-server. apt can't find anything on that path when it need.

Docker, sbt - cannot install sbt with centos docker image

I would like to have sbt in my docker image. I created a Dockerfile base on centos:centos8 image:
FROM centos:centos8
ENV SCALA_VERSION 2.13.1
ENV SBT_VERSION 0.13.18
RUN yum install -y epel-release
RUN yum update -y && yum install -y wget
RUN wget -O /usr/local/bin/sbt-launch.jar http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/$SBT_VERSION/sbt-launch.jar
WORKDIR /root
EXPOSE 8080
RUN sbt compile
CMD sbt run
And also I need to have sbt installed here, but when I ran this script I got an error:
Step 10/11 : RUN sbt compile
---> Running in 0aadcd774ba0
/bin/sh: sbt: command not found
I cannot understand why sbt could not been found. Is it a good way to achieve what I need or I should try other one? But I need to do it with centos
EDIT:
Finally it works after help from answer below. Working script looks like:
FROM centos:centos8
ENV SBT_VERSION 0.13.17
RUN yum install -y java-11-openjdk && \
yum install -y epel-release && \
yum update -y && yum install -y wget && \
wget http://dl.bintray.com/sbt/rpm/sbt-$SBT_VERSION.rpm && \
yum install -y sbt-$SBT_VERSION.rpm
WORKDIR /root
EXPOSE 8080
RUN sbt compile
CMD sbt run

You would need to install sbt inside your Dockerfile. Here is an example:
FROM centos:centos8
ENV SCALA_VERSION 2.13.1
ENV SBT_VERSION 0.13.17
RUN yum install -y epel-release
RUN yum update -y && yum install -y wget
# INSTALL JAVA
RUN yum install -y java-11-openjdk
# INSTALL SBT
RUN wget http://dl.bintray.com/sbt/rpm/sbt-${SBT_VERSION}.rpm
RUN yum install -y sbt-${SBT_VERSION}.rpm
RUN wget -O /usr/local/bin/sbt-launch.jar http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/$SBT_VERSION/sbt-launch.jar
WORKDIR /root
EXPOSE 8080
RUN sbt compile
CMD sbt run
Note: I did not see the version you had in your env variable (0.13.18) so I changed it to 0.13.17.

I ran into an issue where bintray.com was returning 403 randomly. I'm assuming it might be some kind of traffic throttle. Added the rpm file locally.
COPY sbt-0.13.18.rpm /
RUN yum install -y sbt-0.13.18.rpm

rebuild docker image from specific step

I have the below Dockerfile.
FROM ubuntu:14.04
MAINTAINER Samuel Alexander <samuel#alexander.com>
RUN apt-get -y install software-properties-common
RUN apt-get -y update
# Install Java.
RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections
RUN add-apt-repository -y ppa:webupd8team/java
RUN apt-get -y update
RUN apt-get install -y oracle-java8-installer
RUN rm -rf /var/lib/apt/lists/*
RUN rm -rf /var/cache/oracle-jdk8-installer
# Define working directory.
WORKDIR /work
# Define commonly used JAVA_HOME variable
ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
# JAVA PATH
ENV PATH /usr/lib/jvm/java-8-oracle/bin:$PATH
# Install maven
RUN apt-get -y update
RUN apt-get -y install maven
# Install Open SSH and git
RUN apt-get -y install openssh-server
RUN apt-get -y install git
# clone Spark
RUN git clone https://github.com/apache/spark.git
WORKDIR /work/spark
RUN mvn -DskipTests clean package
# clone and build zeppelin fork
RUN git clone https://github.com/apache/incubator-zeppelin.git
WORKDIR /work/incubator-zeppelin
RUN mvn clean package -Pspark-1.6 -Phadoop-2.6 -DskipTests
# Install Supervisord
RUN apt-get -y install supervisor
RUN mkdir -p var/log/supervisor
# Configure Supervisord
COPY conf/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
# bash
RUN sed -i s#/home/git:/bin/false#/home/git:/bin/bash# /etc/passwd
EXPOSE 8080 8082
CMD ["/usr/bin/supervisord"]
While building image it failed in step 23 i.e.
RUN mvn clean package -Pspark-1.6 -Phadoop-2.6 -DskipTests
Now when I rebuild it starts to build from step 23 as docker is using cache.
But if I want to rebuild the image from step 21 i.e.
RUN git clone https://github.com/apache/incubator-zeppelin.git
How can I do that?
Is deleting the cached image is the only option?
Is there any additional parameter to do that?

You can rebuild the entire thing without using the cache by doing a
docker build --no-cache -t user/image-name
To force a rerun starting at a specific line, you can pass an arg that is otherwise unused. Docker passes ARG values as environment variables to your RUN command, so changing an ARG is a change to the command which breaks the cache. It's not even necessary to define it yourself on the RUN line.
FROM ubuntu:14.04
MAINTAINER Samuel Alexander <samuel#alexander.com>
RUN apt-get -y install software-properties-common
RUN apt-get -y update
# Install Java.
RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections
RUN add-apt-repository -y ppa:webupd8team/java
RUN apt-get -y update
RUN apt-get install -y oracle-java8-installer
RUN rm -rf /var/lib/apt/lists/*
RUN rm -rf /var/cache/oracle-jdk8-installer
# Define working directory.
WORKDIR /work
# Define commonly used JAVA_HOME variable
ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
# JAVA PATH
ENV PATH /usr/lib/jvm/java-8-oracle/bin:$PATH
# Install maven
RUN apt-get -y update
RUN apt-get -y install maven
# Install Open SSH and git
RUN apt-get -y install openssh-server
RUN apt-get -y install git
# clone Spark
RUN git clone https://github.com/apache/spark.git
WORKDIR /work/spark
RUN mvn -DskipTests clean package
# clone and build zeppelin fork, changing INCUBATOR_VER will break the cache here
ARG INCUBATOR_VER=unknown
RUN git clone https://github.com/apache/incubator-zeppelin.git
WORKDIR /work/incubator-zeppelin
RUN mvn clean package -Pspark-1.6 -Phadoop-2.6 -DskipTests
# Install Supervisord
RUN apt-get -y install supervisor
RUN mkdir -p var/log/supervisor
# Configure Supervisord
COPY conf/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
# bash
RUN sed -i s#/home/git:/bin/false#/home/git:/bin/bash# /etc/passwd
EXPOSE 8080 8082
CMD ["/usr/bin/supervisord"]
And then just run it with a unique arg:
docker build --build-arg INCUBATOR_VER=20160613.2 -t user/image-name .
To change the argument with every build, you can pass a timestamp as the arg:
docker build --build-arg INCUBATOR_VER=$(date +%Y%m%d-%H%M%S) -t user/image-name .
or:
docker build --build-arg INCUBATOR_VER=$(date +%s) -t user/image-name .
As an aside, I'd recommend the following changes to keep your layers smaller, the more you can merge the cleanup and delete steps on a single RUN command after the download and install, the smaller your final image will be. Otherwise your layers will include all the intermediate steps between the download and cleanup:
FROM ubuntu:14.04
MAINTAINER Samuel Alexander <samuel#alexander.com>
RUN DEBIAN_FRONTEND=noninteractive \
apt-get -y install software-properties-common && \
apt-get -y update
# Install Java.
RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \
add-apt-repository -y ppa:webupd8team/java && \
apt-get -y update && \
DEBIAN_FRONTEND=noninteractive \
apt-get install -y oracle-java8-installer && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* && \
rm -rf /var/cache/oracle-jdk8-installer && \
# Define working directory.
WORKDIR /work
# Define commonly used JAVA_HOME variable
ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
# JAVA PATH
ENV PATH /usr/lib/jvm/java-8-oracle/bin:$PATH
# Install maven
RUN apt-get -y update && \
DEBIAN_FRONTEND=noninteractive \
apt-get -y install
maven \
openssh-server \
git \
supervisor && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# clone Spark
RUN git clone https://github.com/apache/spark.git
WORKDIR /work/spark
RUN mvn -DskipTests clean package
# clone and build zeppelin fork
ARG INCUBATOR_VER=unknown
RUN git clone https://github.com/apache/incubator-zeppelin.git
WORKDIR /work/incubator-zeppelin
RUN mvn clean package -Pspark-1.6 -Phadoop-2.6 -DskipTests
# Configure Supervisord
RUN mkdir -p var/log/supervisor
COPY conf/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
# bash
RUN sed -i s#/home/git:/bin/false#/home/git:/bin/bash# /etc/passwd
EXPOSE 8080 8082
CMD ["/usr/bin/supervisord"]

One workaround:
Locate the step you want to execute from.
Before that step put a simple dummy operation like "RUN pwd"
Then just build your Dockerfile. It will take everything up to that step from the cache and then execute the lines after the dummy command.

To complete Dmitry's answer, you can use uniq arg like date +%s to keep always same commanline
docker build --build-arg DUMMY=`date +%s` -t me/myapp:1.0.0
Dockerfile:
...
ARG DUMMY=unknown
RUN DUMMY=${DUMMY} git clone xxx
...

A simpler technique.
Dockerfile:Add this line where you want the caching to start being skipped.
COPY marker /dev/null
Then build using
date > marker && docker build .

Another option is to delete the cached intermediate image you want to rebuild.
Find the hash of the intermediate image you wish to rebuild in your build output:
Step 27/42 : RUN lolarun.sh
---> Using cache
---> 328dfe03e436
Then delete that image:
$ docker image rmi 328dfe03e436
Or if it gives you an error and you're okay with forcing it:
$ docker image rmi -f 328dfe03e436
Finally, rerun your build command, and it will need to restart from that point because it's not in the cache.

If place ARG INCUBATOR_VER=unknown at top, then cache will not be used in case of change of INCUBATOR_VER from command line (just tested the build).
For me worked:
# The rebuild starts from here
ARG INCUBATOR_VER=unknown
RUN INCUBATOR_VER=${INCUBATOR_VER} git clone https://github.com/apache/incubator-zeppelin.git

As there is no official way to do this, the most simple way is to temporarily change the specific RUN command to include something harmless like echo:
RUN echo && apt-get -qq update && \
apt-get install nano
After building remove it.

Supervisor is not starting up

I am following cloudera cdh4 installation guide.
My base file
FROM ubuntu:precise
RUN apt-get update -y
#RUN apt-get install -y curl
RUN apt-get install -y software-properties-common python-software-properties
RUN add-apt-repository ppa:webupd8team/java
RUN apt-get update -y
RUN echo debconf shared/accepted-oracle-license-v1-1 select true | \
debconf-set-selections
RUN apt-get install -y oracle-java7-installer
#Checking java version
RUN java -version
My hadoop installation file
java_ubuntu is the image build from my base file.
FROM java_ubuntu:latest
RUN apt-get update -y
RUN apt-get install -y curl
RUN curl http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb > cdh4-repository_1.0_all.deb
RUN dpkg -i cdh4-repository_1.0_all.deb
RUN curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | apt-key add -
RUN apt-get update -y
RUN apt-get install -y hadoop-0.20-conf-pseudo
#Check for /etc/hadoop/conf.pseudo.mrl to verfiy hadoop packages
RUN echo "dhis"
RUN dpkg -L hadoop-0.20-conf-pseudo
Supervisor part
hadoop_ubuntu is the image build from my hadoop installation docker file
FROM hadoop_ubuntu:latest
USER hdfs
RUN hdfs namenode -format
USER root
RUN apt-get install -y supervisor
RUN echo "[supervisord] nodameon=true [program=namenode] command=/etc/init.d/hadoop-hdfs-namenode -D" > /etc/supervisorconf.d
CMD ["/usr/bin/supervisord"]
Program is successfully build. But namenode is not starting up? How to use supervisor?

You have your config in /etc/supervisorconf.d and I don't believe that's the right location.
It should be /etc/supervisor/conf.d/supervisord.conf instead.
Also it's easier to maintain if you make a file locally and then use the COPY instruction to put it in the image.
Then as someone mentioned you can connect to the container after it's running (docker exec -it <container id> /bin/bash) and then run supervisorctl to see what's running and what might be wrong.

Perhaps you need line breaks in your supervisor.conf. Try hand crafting one and COPY it into your dockerfile for testing.
Docker and supervisord

Resources