MultiStage Docker Image - docker

I have created a Multistage dockerfile.
FROM dot-portal.de.pri.o2.com:8079/centos:centos7 as base
# Install a basic SSH server GIT, UNZIP, LSOF and JDK 8
ADD myrepo.repo /etc/yum.repos.d/myrepo.repo
RUN rm -rf /etc/yum.repos.d/CentOS*
COPY docker /opt/docker
RUN yum repolist refresh && yum clean all && yum repolist && yum install -y openssh-server git unzip lsof epel-release bzip2 fontconfig && yum clean all && rm -rf /var/cache/yum && cd /opt/docker && rpm -ivh *.rpm && rm -rf /opt/docker
ADD docker-compose-Linux-x86_64 /usr/local/bin/docker-compose
RUN ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
# update sshd settings, create jenkins user, set jenkins user pw, generate ssh keys
RUN sed -i 's|session required pam_loginuid.so|session optional pam_loginuid.so|g' /etc/pam.d/sshd \
&& mkdir -p /var/run/sshd \
&& useradd -u 1000 -m -s /bin/bash jenkins \
&& echo "jenkins:jenkins" | chpasswd \
&& usermod -aG wheel jenkins \
&& /usr/bin/ssh-keygen -A \
&& sed -i "s/#UsePrivilegeSeparation.*/UsePrivilegeSeparation no/g" /etc/ssh/sshd_config \
&& sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config \
&& echo export JAVA_HOME="/`alternatives --display java | grep best | cut -d "/" -f 2-6`" >> /etc/environment \
&& usermod -a -G docker jenkins \
&& sed 's#session\s*required\s*pam_loginuid.so#session optional pam_loginuid.so#g' -i /etc/pam.d/sshd
RUN git config --global http.sslVerify false && mkdir -p /opt/app/jenkins
COPY --chown=jenkins:jenkins jenkins_tools /opt/app/jenkins/jenkins_tools/
ADD node-v10.13.0-linux-x64.tar.gz /opt/app/jenkins/jenkins_tools
ENV CI_TOOLS_HOME /opt/app/jenkins/jenkins_tools
ENV ARCH Linux-x86_64
ENV CI_TOOLS_HOME /opt/app/jenkins/jenkins_tools
ENV JAVA_HOME /usr/java/jdk1.8.0_241-amd64/
COPY dot-portal.de.pri.o2.com.crt $CI_TOOLS_HOME/dot-portal.de.pri.o2.com.crt
RUN cd $CI_TOOLS_HOME && echo yes | keytool -importcert -file dot-portal.de.pri.o2.com.crt -alias "dot-portal.de.pri.o2.com" -keystore "$JAVA_HOME/jre/lib/security/cacerts" -storepass changeit
COPY --chown=jenkins:jenkins authorized_keys /home/jenkins/.ssh/
RUN mkdir /root/.ssh && mkdir -p /opt/app/home/jenkins && chown -R jenkins:jenkins /opt/app/home/jenkins
COPY dockerslave /etc/default/docker
RUN ln -s $CI_TOOLS_HOME/phantomjs-2.1.1-linux-x86_64/bin/phantomjs /usr/bin/ && chown -R jenkins:jenkins /opt/app/jenkins/jenkins_tools
FROM base
COPY --from=base /opt/app/jenkins/jenkins_tools /opt/app/jenkins/jenkins_tools
COPY --from=base /etc/default/docker /etc/default/docker
COPY --from=base /usr/bin/phantomjs /usr/bin/phantomjs
ENV CI_TOOLS_HOME /opt/app/jenkins/jenkins_tools
ENV ARCH Linux-x86_64
ENV CI_TOOLS_HOME /opt/app/jenkins/jenkins_tools
ENV JAVA_HOME /usr/java/jdk1.8.0_241-amd64/
ENV MAVEN_HOME /opt/app/jenkins/jenkins_tools/nonarch/maven-3.3.9
RUN echo "export PATH=$CI_TOOLS_HOME/node-v10.13.0-linux-x64/bin/:$CI_TOOLS_HOME/phantomjs-2.1.1-linux-x86_64/bin/:$JAVA_HOME/bin:$MAVEN_HOME/bin:$PATH" >> ~/.bashrc
# Standard SSH port
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D", "-e"]
My Question is, the package and user which I installed/created in the base Image wont be reflect in the Final Image. From Base only I'm creating final Image, but when I connect to the container, I'm unable to see the user "jenkins" created in base image and the packages also missing.

What are you trying to achieve with a multistage docker build here? If your second stage starts from the result of first stage in a build with two stages, you might as well have only one stage...
Usually multi stage builds are used because one uses the first stage (e.g. "builder") to build some artefact which requires a lot of development tools. In the second stage only that artefact will be copied into a small base image to get a final image which is very small as it does not contain the tooling needed for building the artefact.
Example from the docker docs:
FROM golang:1.7.3 as builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
In Your case I am pretty sure that your final image does contain the jenkins user, because your second stage is started from the base stage. However these copy commands which copy from the base image will reset the permissions to root in your second stage:
COPY --from=base /opt/app/jenkins/jenkins_tools /opt/app/jenkins/jenkins_tools
COPY --from=base /etc/default/docker /etc/default/docker
COPY --from=base /usr/bin/phantomjs /usr/bin/phantomjs
This copy commands are redundant as you start FROM base anyways. If you'd remove these lines the jenkins_tools directory will be correctly owned by the jenkins user.
Note that you can change the user which is used to execute commands during the docker build with USER <username>

Related

Run Python scripts on command line running Docker images

I built a docker image using Dockerfile with Python and some libraries inside (no my project code inside). In my local work dir, there are some scripts to be run on the docker. So, here what I did
$ cd /path/to/my_workdir
$ docker run -it --name test -v `pwd`:`pwd` -w `pwd` my/code:test python src/main.py --config=test --results-dir=/home/me/Results
The command python src/main.py --config=test --results-dir=/home/me/Results is what I want to run inside the Docker container.
However, it returns,
/home/docker/miniconda3/bin/python: /home/docker/miniconda3/bin/python: cannot execute binary file
How can I fix it and run my code?
Here is my Dockerfile
FROM nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04
MAINTAINER Me <me#me.com>
RUN apt update -yq && \
apt install -yq curl wget unzip git vim cmake sudo
RUN adduser --disabled-password --gecos '' docker && \
adduser docker sudo && \
echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER docker
WORKDIR /home/docker/
RUN chmod a+rwx /home/docker/ && \
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
bash Miniconda3-latest-Linux-x86_64.sh -b && rm Miniconda3-latest-Linux-x86_64.sh
ENV PATH /home/docker/miniconda3/bin:$PATH
Run pip install absl-py==0.5.0 atomicwrites==1.2.1 attrs==18.2.0 certifi==2018.8.24 chardet==3.0.4 cycler==0.10.0 docopt==0.6.2 enum34==1.1.6 future==0.16.0 idna==2.7 imageio==2.4.1 jsonpickle==1.2 kiwisolver==1.0.1 matplotlib==3.0.0 mock==2.0.0 more-itertools==4.3.0 mpyq==0.2.5 munch==2.3.2 numpy==1.15.2 pathlib2==2.3.2 pbr==4.3.0 Pillow==5.3.0 pluggy==0.7.1 portpicker==1.2.0 probscale==0.2.3 protobuf==3.6.1 py==1.6.0 pygame==1.9.4 pyparsing==2.2.2 pysc2==3.0.0 pytest==3.8.2 python-dateutil==2.7.3 PyYAML==3.13 requests==2.19.1 s2clientprotocol==4.10.1.75800.0 sacred==0.8.1 scipy==1.1.0 six==1.11.0 sk-video==1.1.10 snakeviz==1.0.0 tensorboard-logger==0.1.0 torch==0.4.1 torchvision==0.2.1 tornado==5.1.1 urllib3==1.23
USER docker
ENTRYPOINT ["/bin/bash"]
Try making the file executable before running it.
as John mentioned to do in the dockerfile
FROM python:latest
COPY src/main.py /usr/local/share/
RUN chmod +x /usr/local/share/src/main.py #<-**--- just add this also
# I have some doubts about the pathing
CMD ["/usr/local/share/src/main.py", "--config=test --results-dir=/home/me/Results"]
You can run a python script in docker by adding this to your docker file:
FROM python:latest
COPY src/main.py /usr/local/share/
CMD ["src/main.py", "--config=test --results-dir=/home/me/Results"]

How to build go docker image as non root user?

I am having difficulty building a docker image as a non root user from base-image go.13.
My main issue is that before go1.12 you could set the go env variableGOCACHE="off" however since go1.12 this is no longer an option.
Here is my current Dockerfile that at Step: RUN mkdir ./build && go mod download && go mod verify && CGO_ENABLED=0 GOOS=linux go build -o app -x -mod vendor -trimpath returns output
failed to initialize build cache at /nonexistent/.cache/go-build: mkdir /nonexistent: permission denied:
# Go version
FROM golang:1.13 AS build-env
RUN chmod -R o=,g=rwX $GOPATH/
RUN mkdir /service
ENV USER=trevorjo
ENV UID=10001
# create a sytstem group dev with no password, no home directory set, and no shell so prevents the user form
# being a login account and reduces the attack vector
RUN adduser \
--disabled-password \
--gecos "" \
--home "/nonexistent" \
--shell "/sbin/nologin" \
--no-create-home \
--uid "${UID}" \
${USER}
#RUN groupadd -r dev && \
#useradd -r -s /bin/false -g dev trevorjo sudo
WORKDIR /service
COPY . /service
# change ownership of all /service content to created user
RUN chown -R trevorjo /service
USER trevorjo
#RUN echo "trevorjo ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/user && \
#chmod 0440 /etc/sudoers.d/user
# GOCACHE disable as get a permission denied error due to running as non root user
RUN mkdir ./build && \
go mod download && \
go mod verify && \
CGO_ENABLED=0 GOOS=linux go build -o app -x -mod vendor -trimpath
FROM scratch AS run-env
WORKDIR /build
COPY --from=build-env /service/build/app /build/
ENTRYPOINT ["/build/app"]

python and oracle client on docker image

I want to create a docker image with oracle client and cx_oracle of python. I am using multi stage docker to build the image but I am missing an env variable due to which cx_oracle is not able to find an oracle client library.
FROM oraclelinux:7-slim
RUN curl -o /etc/yum.repos.d/public-yum-ol7.repo https://yum.oracle.com/public-yum-ol7.repo && \
yum-config-manager --enable ol7_oracle_instantclient && \
yum -y install oracle-instantclient18.3-basic oracle-instantclient18.3-devel oracle-instantclient18.3-sqlplus && \
rm -rf /var/cache/yum && \
echo /usr/lib/oracle/18.3/client64/lib > /etc/ld.so.conf.d/oracle-instantclient18.3.conf && \
ldconfig
ENV PATH=$PATH:/usr/lib/oracle/18.3/client64/bin
FROM python:slim
COPY ./requirement.txt ./requirement.txt
RUN pip install -r ./requirement.txt
COPY --from=0 /usr/lib/oracle/18.3/client64/lib /root/usr/lib/oracle/18.3/client64/lib
COPY --from=0 /usr/lib/oracle/18.3/client64/bin /root/usr/lib/oracle/18.3/client64/bin
ENV PATH=$PATH:/root/usr/lib/oracle/18.3/client64/bin:/root/usr/lib/oracle/18.3/client64/lib
ENV ORACLE_HOME=/root/usr/lib/oracle/18.3/client64/:$ORACLE_HOME
ENV LD_LIBRARY_PATH=/root/usr/lib/oracle/18.3/client64/:$LD_LIBRARY_PATH
RUN echo $PATH
RUN echo $ORACLE_HOME
RUN chmod 755 /root/usr/lib/oracle/18.3/client64/lib/*
RUN ls -l /root/usr/lib/oracle/18.3/client64/lib
CMD ["chmod","755","/root/usr/lib/oracle/18.3/client64/lib/*"]
CMD ["ls", "-l" ,"/root/usr/lib/oracle/18.3/client64/lib"]
CMD ["python","test.py"]
Below is the error
DPI-1047: 64-bit Oracle Client library cannot be loaded: "libclntsh.so: cannot open shared object file: No such file or directory". See https://oracle.github.io/odpi/doc/installation.html#linux
Oracle has Python Dockerfiles at https://github.com/oracle/docker-images/tree/main/OracleLinuxDevelopers
Also see https://blogs.oracle.com/opal/docker-for-oracle-database-applications-in-nodejs-and-python-part-1

Dockerfile entrypoint unable to switch user

I am unable to switch user to a non-root user from the entry point script. The User directive to change the user in Dockerfile works, but I am not able to change permissions using chmod. To overcome this issue I created entrypoint.sh script to change the folder permissions but when I try to switch user using su command, it apparently doesn't work, the container is still running as root.
The Dockerfile
FROM php:7.2-fpm
# Installing dependencies
RUN apt-get update && apt-get install -y \
build-essential \
mysql-client \
libpng-dev \
libjpeg62-turbo-dev \
libfreetype6-dev \
locales \
zip \
jpegoptim optipng pngquant gifsicle \
vim \
unzip \
git \
curl
# Installing composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
ENV USER_ID=1000
ENV GROUP_ID=1000
ENV USER_NAME=www
ENV GROUP_NAME=www
RUN groupadd -g $GROUP_ID $GROUP_NAME
RUN useradd -u $USER_ID -ms /bin/bash -g $GROUP_NAME $USER_NAME
RUN mkdir /app
WORKDIR /app
EXPOSE 9000
COPY ./entrypoint.sh /
RUN ["chmod", "+x", "/entrypoint.sh"]
ENTRYPOINT ["/entrypoint.sh"]
Entrypoint.sh file
#!/bin/bash
if [ -n "$USER_ID" -a -n "$GROUP_ID" ]; then
chown -R $USER_NAME:$GROUP_NAME .
su $USER_NAME
fi
php-fpm
exec "$#"
whatever I do I am not able to switch user from the entrypoint.sh script.
My case is to run the container as non-root user.
I think that your su command should be something like
su $USERNAME --command "/doit.sh"
b/c your entrpoiny script is switching user, doing nothing, and then switching back to root.
To solve this you need to change your dockerfile and add:
RUN echo "root ALL = NOPASSWD: /bin/su ALL" >> /etc/sudoers
Or use gosu what is better:
# install gosu
# seealso:
# https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
# https://github.com/tianon/gosu/blob/master/INSTALL.md
# https://github.com/tianon/gosu
RUN set -eux; \
apt-get update; \
apt-get install -y gosu; \
rm -rf /var/lib/apt/lists/*; \
# verify that the binary works
gosu nobody true
Then inside entrypoint.sh:
gosu root yourservice &
#ie: gosu root /usr/sbin/sshd -D &
exec gosu no-root-user yourservice2
# ie: exec gosu no-root-user tail -f /dev/null

How can i use volume in my dockerfile for copy the jmeter result in my local

How can I use volume in my dockerfile for copy the JMeter result in my local?
Need to display the result in local, how can I copy the result and paste in local with the help of VOLUME.
For example:- I am saving the JMeter HTML report in my container but after that container is automatically stopped. So someone suggests me to use the docker VOLUME command for RUN the HTML.
FROM alpine
ARG JMETER_VERSION="4.0"
ENV JMETER_HOME /opt/apache-jmeter-${JMETER_VERSION}
ENV JMETER_BIN ${JMETER_HOME}/bin
ENV JMETER_DOWNLOAD_URL https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-${JMETER_VERSION}.tgz
ENV JMETER_PLUGINS_DOWNLOAD_URL http://repo1.maven.org/maven2/kg/apc/jmeter-plugins-functions/2.0/jmeter-plugins-functions-2.0.jar
ENV JMETER_PLUGINS_FOLDER ${JMETER_HOME}/lib/ext/
# Change TimeZone TODO: TZ still is not set!
ARG TZ="Australia/Melbourne"
RUN apk update \
&& apk upgrade \
&& apk add ca-certificates \
&& update-ca-certificates \
&& apk add --update openjdk8-jre tzdata curl unzip bash \
&& rm -rf /var/cache/apk/* \
&& mkdir -p /tmp/dependencies \
&& curl -L --silent ${JMETER_DOWNLOAD_URL} > /tmp/dependencies/apache-jmeter-${JMETER_VERSION}.tgz \
&& mkdir -p /opt \
&& tar -xzf /tmp/dependencies/apache-jmeter-${JMETER_VERSION}.tgz -C /opt \
&& rm -rf /tmp/dependencies
RUN curl -L --silent ${JMETER_PLUGINS_DOWNLOAD_URL}/jmeter-plugins-dummy/0.2/jmeter-plugins-dummy-0.2.jar -o ${JMETER_PLUGINS_FOLDER}/jmeter-plugins-dummy-0.2.jar
RUN curl -L --silent ${JMETER_PLUGINS_DOWNLOAD_URL}/jmeter-plugins-cmn-jmeter/0.5/jmeter-plugins-cmn-jmeter-0.5.jar -o ${JMETER_PLUGINS_FOLDER}/jmeter-plugins-cmn-jmeter-0.5.jar
# TODO: plugins (later)
# && unzip -oq "/tmp/dependencies/JMeterPlugins-*.zip" -d $JMETER_HOME
# Set global PATH such that "jmeter" command is found
ENV PATH $PATH:$JMETER_BIN
ENV URL_PATH=${URL}
WORKDIR ${JMETER_HOME}
#RUN export DATETIME=$(date +%Y%m%d)
RUN mkdir -p /var/www/html/"$(date +%Y%m%d)"
VOLUME /var/www/html/
#Copy the *.jmx file jmeter bin file
COPY Get_Ping_Node_API.jmx ./bin
CMD ./bin/jmeter -n -t ./bin/Get_Ping_Node_API.jmx -l ./bin/result.jtl -e -o ./bin/result_html'
You can use docker volume while you run the docker image.
You can add --volume , -v flag to your docker run command.
docker run -v "HOST DIR":"CONTAINER DIR"

Resources