I am having difficulty building a docker image as a non root user from base-image go.13.
My main issue is that before go1.12 you could set the go env variableGOCACHE="off" however since go1.12 this is no longer an option.
Here is my current Dockerfile that at Step: RUN mkdir ./build && go mod download && go mod verify && CGO_ENABLED=0 GOOS=linux go build -o app -x -mod vendor -trimpath returns output
failed to initialize build cache at /nonexistent/.cache/go-build: mkdir /nonexistent: permission denied:
# Go version
FROM golang:1.13 AS build-env
RUN chmod -R o=,g=rwX $GOPATH/
RUN mkdir /service
ENV USER=trevorjo
ENV UID=10001
# create a sytstem group dev with no password, no home directory set, and no shell so prevents the user form
# being a login account and reduces the attack vector
RUN adduser \
--disabled-password \
--gecos "" \
--home "/nonexistent" \
--shell "/sbin/nologin" \
--no-create-home \
--uid "${UID}" \
${USER}
#RUN groupadd -r dev && \
#useradd -r -s /bin/false -g dev trevorjo sudo
WORKDIR /service
COPY . /service
# change ownership of all /service content to created user
RUN chown -R trevorjo /service
USER trevorjo
#RUN echo "trevorjo ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/user && \
#chmod 0440 /etc/sudoers.d/user
# GOCACHE disable as get a permission denied error due to running as non root user
RUN mkdir ./build && \
go mod download && \
go mod verify && \
CGO_ENABLED=0 GOOS=linux go build -o app -x -mod vendor -trimpath
FROM scratch AS run-env
WORKDIR /build
COPY --from=build-env /service/build/app /build/
ENTRYPOINT ["/build/app"]
Related
I'm trying create a Quarkus Docker Image using Jenkins.
I'm using the default JVM dockerFile provided by Quarkus
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.1
ARG JAVA_PACKAGE=java-11-openjdk-headless
ARG RUN_JAVA_VERSION=1.3.8
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en'
# Install java and the run-java script
# Also set up permissions for user `1001`
RUN microdnf install curl ca-certificates ${JAVA_PACKAGE} \
&& microdnf update \
&& microdnf clean all \
&& mkdir /deployments \
&& chown 1001 /deployments \
&& chmod "g+rwX" /deployments \
&& chown 1001:root /deployments \
&& curl https://repo1.maven.org/maven2/io/fabric8/run-java-sh/${RUN_JAVA_VERSION}/run-java-sh-${RUN_JAVA_VERSION}-sh.sh -o /deployments/run-java.sh \
&& chown 1001 /deployments/run-java.sh \
&& chmod 540 /deployments/run-java.sh \
&& echo "securerandom.source=file:/dev/urandom" >>
/etc/alternatives/jre/lib/security/java.security
# Configure the JAVA_OPTIONS, you can add -XshowSettings:vm to also display the heap size.
ENV JAVA_OPTIONS="\
-Dquarkus.datasource.jdbc.url=${DB_HOST} \
-Dquarkus.datasource.username=${DB_USER_NAME} \
-Dquarkus.datasource.password=${DB_PASSWORD} \
-Dslack.alert.remote.active=true \
-Dslack-api/mp-rest/url="some Path" \
-Dquarkus.mailer.recipients=${EMAIL_RECIPIENTS}"
COPY target/lib/* /deployments/lib/
COPY target/*-runner.jar /deployments/app.jar
EXPOSE 8080
USER 1001
ENTRYPOINT [ "/deployments/run-java.sh" ]
.dockerignore
#*
!target/*-runner
!target/*-runner.jar
!target/lib/*
!target/quarkus-app/*
and here is the stack I got
Step 7/11 : COPY target/*-runner.jar /deployments/app.jar
ERROR: Build step failed with exception
com.github.dockerjava.api.exception.DockerClientException: Could not build image: COPY failed: no source files were specified
prior to that step in jenkins i'm doing the command ./mvnw clean install and then the docker build starts.
When I look at the workspace in Jenkins, the target folder is there and it contains the lib and the jar file
What am I doing wrong?
EDIT
We finally figured it out. To fix the issue we had to move the dockerfile directly to the root of the project and it worked.
I have created a dockerfile that builds a scratch image from which it serves a static golang build. When I start a container from the image, I receive this error:
idea_service_prod | standard_init_linux.go:211: exec user process caused "no such file or directory"
idea_service_prod exited with code 1
The dockerfile:
FROM golang:1.14 as base
# install librdkafka
ENV LIBRDKAFKA_VERSION=1.4.2
RUN curl -Lk -o /root/librdkafka-${LIBRDKAFKA_VERSION}.tar.gz https://github.com/edenhill/librdkafka/archive/v${LIBRDKAFKA_VERSION}.tar.gz && \
tar -xzf /root/librdkafka-${LIBRDKAFKA_VERSION}.tar.gz -C /root && \
cd /root/librdkafka-${LIBRDKAFKA_VERSION} && \
./configure --prefix /usr && make && make install && make clean && ./configure --clean
# install inotify-tools
FROM base as dev
EXPOSE 80
RUN apt-get -y update \
&& apt-get install -y --no-install-recommends inotify-tools \
&& apt-get clean
FROM base as build
# install git
RUN apt-get -y update \
&& apt-get install -y --no-install-recommends git \
&& apt-get clean
# configure user
ENV USER=appuser
ENV UID=10001
RUN adduser \
--disabled-password \
--gecos "" \
--home "/nonexistent" \
--shell "/sbin/nologin" \
--no-create-home \
--uid "${UID}" \
"${USER}"
# download dependencies
WORKDIR $GOPATH/src/ideaservice
COPY . .
RUN go get -d -v
RUN go mod download
RUN go mod verify
# build binary
RUN GOOS=linux GOARCH=amd64 go build -tags static -ldflags="-w -s" -o /go/bin/main cmd/ideaservice/main.go
FROM scratch
EXPOSE 80
COPY --from=build /etc/passwd /etc/passwd
COPY --from=build /etc/group/ /etc/group
COPY --from=build /go/bin/main /go/bin/main
USER appuser:appuser
CMD ["/go/bin/main"]
What could be causing this? It seems that it should work - the binary is copied from the past stage in the build process, to a destination in the scratch image, where it is then executed from.
Dynamic linking issue. I would suggest you read article by Jerome. It is explains why does this append and how to avoid it, what is difference betwean base images for go dev, linux and libraries support provided by them.
I have created a Multistage dockerfile.
FROM dot-portal.de.pri.o2.com:8079/centos:centos7 as base
# Install a basic SSH server GIT, UNZIP, LSOF and JDK 8
ADD myrepo.repo /etc/yum.repos.d/myrepo.repo
RUN rm -rf /etc/yum.repos.d/CentOS*
COPY docker /opt/docker
RUN yum repolist refresh && yum clean all && yum repolist && yum install -y openssh-server git unzip lsof epel-release bzip2 fontconfig && yum clean all && rm -rf /var/cache/yum && cd /opt/docker && rpm -ivh *.rpm && rm -rf /opt/docker
ADD docker-compose-Linux-x86_64 /usr/local/bin/docker-compose
RUN ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
# update sshd settings, create jenkins user, set jenkins user pw, generate ssh keys
RUN sed -i 's|session required pam_loginuid.so|session optional pam_loginuid.so|g' /etc/pam.d/sshd \
&& mkdir -p /var/run/sshd \
&& useradd -u 1000 -m -s /bin/bash jenkins \
&& echo "jenkins:jenkins" | chpasswd \
&& usermod -aG wheel jenkins \
&& /usr/bin/ssh-keygen -A \
&& sed -i "s/#UsePrivilegeSeparation.*/UsePrivilegeSeparation no/g" /etc/ssh/sshd_config \
&& sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config \
&& echo export JAVA_HOME="/`alternatives --display java | grep best | cut -d "/" -f 2-6`" >> /etc/environment \
&& usermod -a -G docker jenkins \
&& sed 's#session\s*required\s*pam_loginuid.so#session optional pam_loginuid.so#g' -i /etc/pam.d/sshd
RUN git config --global http.sslVerify false && mkdir -p /opt/app/jenkins
COPY --chown=jenkins:jenkins jenkins_tools /opt/app/jenkins/jenkins_tools/
ADD node-v10.13.0-linux-x64.tar.gz /opt/app/jenkins/jenkins_tools
ENV CI_TOOLS_HOME /opt/app/jenkins/jenkins_tools
ENV ARCH Linux-x86_64
ENV CI_TOOLS_HOME /opt/app/jenkins/jenkins_tools
ENV JAVA_HOME /usr/java/jdk1.8.0_241-amd64/
COPY dot-portal.de.pri.o2.com.crt $CI_TOOLS_HOME/dot-portal.de.pri.o2.com.crt
RUN cd $CI_TOOLS_HOME && echo yes | keytool -importcert -file dot-portal.de.pri.o2.com.crt -alias "dot-portal.de.pri.o2.com" -keystore "$JAVA_HOME/jre/lib/security/cacerts" -storepass changeit
COPY --chown=jenkins:jenkins authorized_keys /home/jenkins/.ssh/
RUN mkdir /root/.ssh && mkdir -p /opt/app/home/jenkins && chown -R jenkins:jenkins /opt/app/home/jenkins
COPY dockerslave /etc/default/docker
RUN ln -s $CI_TOOLS_HOME/phantomjs-2.1.1-linux-x86_64/bin/phantomjs /usr/bin/ && chown -R jenkins:jenkins /opt/app/jenkins/jenkins_tools
FROM base
COPY --from=base /opt/app/jenkins/jenkins_tools /opt/app/jenkins/jenkins_tools
COPY --from=base /etc/default/docker /etc/default/docker
COPY --from=base /usr/bin/phantomjs /usr/bin/phantomjs
ENV CI_TOOLS_HOME /opt/app/jenkins/jenkins_tools
ENV ARCH Linux-x86_64
ENV CI_TOOLS_HOME /opt/app/jenkins/jenkins_tools
ENV JAVA_HOME /usr/java/jdk1.8.0_241-amd64/
ENV MAVEN_HOME /opt/app/jenkins/jenkins_tools/nonarch/maven-3.3.9
RUN echo "export PATH=$CI_TOOLS_HOME/node-v10.13.0-linux-x64/bin/:$CI_TOOLS_HOME/phantomjs-2.1.1-linux-x86_64/bin/:$JAVA_HOME/bin:$MAVEN_HOME/bin:$PATH" >> ~/.bashrc
# Standard SSH port
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D", "-e"]
My Question is, the package and user which I installed/created in the base Image wont be reflect in the Final Image. From Base only I'm creating final Image, but when I connect to the container, I'm unable to see the user "jenkins" created in base image and the packages also missing.
What are you trying to achieve with a multistage docker build here? If your second stage starts from the result of first stage in a build with two stages, you might as well have only one stage...
Usually multi stage builds are used because one uses the first stage (e.g. "builder") to build some artefact which requires a lot of development tools. In the second stage only that artefact will be copied into a small base image to get a final image which is very small as it does not contain the tooling needed for building the artefact.
Example from the docker docs:
FROM golang:1.7.3 as builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
In Your case I am pretty sure that your final image does contain the jenkins user, because your second stage is started from the base stage. However these copy commands which copy from the base image will reset the permissions to root in your second stage:
COPY --from=base /opt/app/jenkins/jenkins_tools /opt/app/jenkins/jenkins_tools
COPY --from=base /etc/default/docker /etc/default/docker
COPY --from=base /usr/bin/phantomjs /usr/bin/phantomjs
This copy commands are redundant as you start FROM base anyways. If you'd remove these lines the jenkins_tools directory will be correctly owned by the jenkins user.
Note that you can change the user which is used to execute commands during the docker build with USER <username>
I'm trying to run a Flask app with Celery (worker + beat) on Docker Alpine using docker-compose.
I want it to run with a non-root user celery in my Docker container.
The flask app is building ok and works, but my celery containers are failing with this error:
File "/usr/lib/python3.6/site-packages/celery/platforms.py", line 543, in maybe_drop_privileges
_setuid(uid, gid)
File "/usr/lib/python3.6/site-packages/celery/platforms.py", line 564, in _setuid
initgroups(uid, gid)
File "/usr/lib/python3.6/site-packages/celery/platforms.py", line 507, in initgroups
return os.initgroups(username, gid)
PermissionError: [Errno 1] Operation not permitted
My Dockerfile:
I tried to add RUN chown celery:celery /etc/group thinking that was the issue but it's still failing
FROM alpine:3.8
RUN apk update && \
apk add build-base python3 python3-dev libffi-dev libressl-dev && \
cd /usr/bin && \
ln -sf python3 python && \
ln -sf pip3 pip && \
pip install --upgrade pip
COPY requirements.txt .
RUN pip install -r requirements.txt
RUN addgroup celery
RUN adduser celery -G celery -s /bin/sh -D
RUN mkdir -p /var/log/celery/ && chown celery:celery /var/log/celery/
RUN mkdir -p /var/run/celery/ && chown celery:celery /var/run/celery/
RUN chown celery:celery /etc/group # added to try fixing the issue
USER celery
ENV FLASK_APP=flask_app
WORKDIR app/
COPY flask_app flask_app
My docker-compose:
(...)
celeryworker:
build: .
command: celery -A flask_app.tasks worker --loglevel=INFO --uid=celery --pidfile=/tmp/celeryworker-shhh.pid
celerybeat:
build: .
command: celery -A flask_app.tasks beat --loglevel=INFO --uid=celery --pidfile=/tmp/celerybeat-shhh.pid
You should do like that
RUN mkdir -p /var/log/celery/ /var/run/celery/
RUN useradd -G root celery && \
chgrp -Rf root /var/log/celery/ /var/run/celery/ && \
chmod -Rf g+w /var/log/celery/ /var/run/celery/c && \
chmod g+w /etc/passwd
...
RUN chmod a+x /start.sh
USER celery
ENTRYPOINT ["/start.sh"]
You should create user celery firsts. Then, add this user into group root. After that you need set write permission for this folder you need to put logs and /etc/passwd.
You also need to have one script to add your user into /etc/passwd
#!/bin/bash
#
if [ `id -u` -ge 10000 ]; then
echo "celery:x:`id -u`:`id -g`:,,,:/home/web:/bin/bash" >> /etc/passwd
fi
Both answers from #Shashank V and #Kine were really relevant and helpful but still had some issues afterward.
After doing some research, I finally made it works with the following configuration
Dockerfile
FROM alpine:3.11.0
RUN apk update && \
apk add build-base python3 python3-dev libffi-dev libressl-dev && \
ln -sf /usr/bin/python3 /usr/bin/python && \
ln -sf /usr/bin/pip3 usr/bin/pip && \
pip install --upgrade pip
RUN mkdir -p /var/log/celery/ /var/run/celery/
RUN addgroup app && \
adduser --disabled-password --gecos "" --ingroup app --no-create-home app && \
chown app:app /var/run/celery/ && \
chown app:app /var/log/celery/
USER app
ENV PATH="/home/app/.local/bin:${PATH}"
WORKDIR app/
COPY requirements.txt .
RUN pip install --user -r requirements.txt\
COPY flask_app flask_app
ENV FLASK_APP=flask_app
docker-compose
(...)
celeryworker:
build: .
command: >
celery -A shhh.tasks worker
--loglevel=INFO
--logfile=/var/log/celery/celeryworker-shhh.log
--pidfile=/var/run/celery/celeryworker-shhh.pid
celerybeat:
build: .
command: >
celery -A shhh.tasks beat
--loglevel=INFO
--logfile=/var/log/celery/celerybeat-shhh.log
--pidfile=/var/run/celery/celerybeat-shhh.pid
--schedule=/var/run/celery/celerybeat-schedule # specify schedule db in a loc where app has read/write access
You have to be root user if you want to use --uid or --gid argument. Try removing these arguments.
I am unable to switch user to a non-root user from the entry point script. The User directive to change the user in Dockerfile works, but I am not able to change permissions using chmod. To overcome this issue I created entrypoint.sh script to change the folder permissions but when I try to switch user using su command, it apparently doesn't work, the container is still running as root.
The Dockerfile
FROM php:7.2-fpm
# Installing dependencies
RUN apt-get update && apt-get install -y \
build-essential \
mysql-client \
libpng-dev \
libjpeg62-turbo-dev \
libfreetype6-dev \
locales \
zip \
jpegoptim optipng pngquant gifsicle \
vim \
unzip \
git \
curl
# Installing composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
ENV USER_ID=1000
ENV GROUP_ID=1000
ENV USER_NAME=www
ENV GROUP_NAME=www
RUN groupadd -g $GROUP_ID $GROUP_NAME
RUN useradd -u $USER_ID -ms /bin/bash -g $GROUP_NAME $USER_NAME
RUN mkdir /app
WORKDIR /app
EXPOSE 9000
COPY ./entrypoint.sh /
RUN ["chmod", "+x", "/entrypoint.sh"]
ENTRYPOINT ["/entrypoint.sh"]
Entrypoint.sh file
#!/bin/bash
if [ -n "$USER_ID" -a -n "$GROUP_ID" ]; then
chown -R $USER_NAME:$GROUP_NAME .
su $USER_NAME
fi
php-fpm
exec "$#"
whatever I do I am not able to switch user from the entrypoint.sh script.
My case is to run the container as non-root user.
I think that your su command should be something like
su $USERNAME --command "/doit.sh"
b/c your entrpoiny script is switching user, doing nothing, and then switching back to root.
To solve this you need to change your dockerfile and add:
RUN echo "root ALL = NOPASSWD: /bin/su ALL" >> /etc/sudoers
Or use gosu what is better:
# install gosu
# seealso:
# https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
# https://github.com/tianon/gosu/blob/master/INSTALL.md
# https://github.com/tianon/gosu
RUN set -eux; \
apt-get update; \
apt-get install -y gosu; \
rm -rf /var/lib/apt/lists/*; \
# verify that the binary works
gosu nobody true
Then inside entrypoint.sh:
gosu root yourservice &
#ie: gosu root /usr/sbin/sshd -D &
exec gosu no-root-user yourservice2
# ie: exec gosu no-root-user tail -f /dev/null