building inside Docker container - docker

In the simple case of running Gitea from inside a Docker container, the current Dockerfile assumes you have already compiled Gitea and have it available to create the image.
In my case, I do not have Gitea currently installed on my machine. I am on Mac OSX and my thought was that if I build on this machine, but the Gitea Dockerfile uses Alpine Linux as its base image, there would be some conflict in this case? Can someone clarify if that would be an issue?
The workaround I thought of was to build Gitea directly inside the container with Alpine Linux. The other possibility would be for me to compile on Mac OSX and then change the base image I suppose.
Do these two workarounds make sense and which one would be preferable? Does the binary that we use in the image also have to be compatible with the base image? I think so, but welcome your inputs as well.
Edit: I've tried compiling inside the container, but ran into a problem. From the DockerHub available Dockerfile is below:
FROM alpine:3.7
LABEL maintainer="The Gitea Authors"
EXPOSE 22 3000
RUN apk --no-cache add \
su-exec \
ca-certificates \
sqlite \
bash \
git \
linux-pam \
s6 \
curl \
openssh \
gettext \
tzdata \
make
RUN addgroup \
-S -g 1000 \
git && \
adduser \
-S -H -D \
-h /data/git \
-s /bin/bash \
-u 1000 \
-G git \
git && \
echo "git:$(dd if=/dev/urandom bs=24 count=1 status=none | base64)" |
chpasswd
# Dockerfile assumes Gitea is already up and running
# In this case, copy the source to container and Compile Gitea in Alpine
Environment
COPY gitea-master.zip /
RUN unzip gitea-master.zip
RUN cd gitea-master
RUN make generate all
You can see I've added instructions to copy gitea into the container, unzip, cd into the directory and the build, but I get the following output:
Step 1/17 : FROM alpine:3.7
---> 3fd9065eaf02
Step 2/17 : LABEL maintainer="The Gitea Authors"
---> Using cache
---> b6bb053b3e89
Step 3/17 : EXPOSE 22 3000
---> Using cache
---> ade88a29df64
Step 4/17 : RUN apk --no-cache add su-exec ca-certificates
sqlite bash git linux-pam s6 curl openssh
gettext tzdata make
---> Using cache
---> 460a5562c60f
Step 5/17 : RUN addgroup -S -g 1000 git && adduser -S -H -D
-h /data/git -s /bin/bash -u 1000 -G git git && echo
"git:$(dd if=/dev/urandom bs=24 count=1 status=none | base64)" | chpasswd
---> Using cache
---> 07ff1f0b2d3e
Step 6/17 : COPY gitea-master.zip /
---> Using cache
---> 3ad1e3177659
Step 7/17 : RUN unzip gitea-master.zip
---> Using cache
---> 61a2beb19e1e
Step 8/17 : RUN cd gitea-master
---> Using cache
---> 6985914927b9
Step 9/17 : RUN make generate all
---> Running in 3e603f88c302
make: *** No rule to make target 'generate'. Stop.
The command '/bin/sh -c make generate all' returned a non-zero code: 2
The Makefile clearly shows that those targets are available however, but this has got me a bit stumped. Any suggestions ?

In order to build Go binaries (I checked out gitea and it seems to be built with Go) for Linux on OSX you need to provide have env variable GOOS set to linux. You can try adding GOOS=linux in front of this line in gitea makefile: https://github.com/go-gitea/gitea/blob/master/Makefile#L221 , or simply run GOOS=linux make generate build . This should create a go binary which you can use in Docker image.
Edit in response to edit: possibly you unzipped the gitea-master into a deeper dir, check output of ls while building image to see if Makefile is available to you (it seems like there is no Makefile in your pwd while building image).

Related

Why does docker rebuild all layers every time I change build args

I have a docker file which has a lot of layers. At the top of the file I have some args like
FROM ubuntu:18.04
ARGS USER=test-user
ARGS UID=1000
#ARGS PW=test-user
# Then several Layers which does not use any ARGS. Example
LABEL version="1.0"
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
RUN mkdir ~/mapped-volume
RUN apt-get update && apt-get install -y wget bzip2 ca-certificates build-essential curl git-core htop pkg-config unzip unrar tree freetds-dev vim \
sudo nodejs npm net-tools flex perl automake bison libtool byacc
# And so on
# And finally towards the end
# Setup User
RUN useradd -m -d /home/${USER} --uid ${UID} -G sudo -s /bin/bash ${USER}
# && echo "${USER}:${PW}" | chpasswd
# Couple of more commands to change dir, entry point etc. Example
When I build this docker file with any arg value different from the last build and/or after small changes in the last two layers, the build builds everything again. It does not use cached layer. The command I use to build is something like this
docker build --build-arg USER=new-user --build-arg UID=$UID -t my-image:1.0 .
And every time I change the values the build goes all through again. With a truncated top like below
UID -t my-image:1.0 .
Sending build context to Docker daemon 44.54kB
Step 1/23 : FROM ubuntu:18.04
---> ccc6e87d482b
Step 2/23 : ARG USER=ml-user
---> Using cache
---> 6c0c5d5c5056
Step 3/23 : ARG UID=1000
---> Using cache
---> b25867c282c7
Step 4/23 : LABEL version="1.0"
---> Running in 1ffff70d56c1
Removing intermediate container 1ffff70d56c1
---> 0f1277def3ca
Step 5/23 : ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
---> Running in 49d08c41b233
Removing intermediate container 49d08c41b233
---> f5b345573c1f
Step 6/23 : RUN mkdir ~/mapped-volume
---> Running in e4f8a5956450
Removing intermediate container e4f8a5956450
---> 1b22731d9051
Step 7/23 : RUN apt-get update && apt-get install -y wget bzip2 ca-certificates build-essential curl git-core htop pkg-config unzip unrar tree freetds-dev vim sudo nodejs npm net-tools flex perl automake bison libtool byacc
---> Running in ffc297de6234
Get:1 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
So from step 7 it keeps doing all steps without using the cache of that layer which should have a bunch of packages
Why? How can I stop this? Previously when I did not have args, this layer and other layers used to be picked up from cache.
Move your args to just before you need them. Docker does not replace args in the RUN commands before running them. Instead, the args are passed as environment variables and expanded by the shell within the temporary container. Because of that, a change to an arg is a change to the environment, and a miss of the build cache for that step. Once one step misses the cache, all following steps must be rebuilt.
FROM ubuntu:18.04
# Then several Layers which does not use any ARGS. Example
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
RUN mkdir ~/mapped-volume
RUN apt-get update && apt-get install -y wget bzip2 ca-certificates build-essential curl git-core htop pkg-config unzip unrar tree freetds-dev vim \
sudo nodejs npm net-tools flex perl automake bison libtool byacc
# And so on
# And finally towards the end
# Setup User
ARGS USER=test-user
ARGS UID=1000
RUN useradd -m -d /home/${USER} --uid ${UID} -G sudo -s /bin/bash ${USER}
# && echo "${USER}:${PW}" | chpasswd
# Couple of more commands to change dir, entry point etc. Example
LABEL version="1.0"
Also, labels, environment variables that aren't needed at build time, exposed ports, and any other meta data is often best left to the end of the Dockerfile since they have minimal impact on build time and there's no need to miss the cache when they change.

Docker : Dockerfile can't execute COPY

I ve to build a docker images .
Inside my repositiory , i ve those files :
Dockerfile
docker-prompt
My Dockerfile is :
FROM fortio/fortio:1.3.1 as fortiobuild
FROM docker:stable-dind
RUN apk add --no-cache tcpdump apache2-utils lynx git tmux py2-pip apache2-utils vim build-base gettext-dev curl bash-completion bash util-linux jq openssh openssl tree python python-dev py-pip libffi-dev openssl-dev libgcc nfs-utils
ENV COMPOSE_VERSION=1.24.1
RUN pip install docker-compose==${COMPOSE_VERSION}
RUN mkdir /etc/bash_completion.d \
&& curl https://raw.githubusercontent.com/docker/cli/master/contrib/completion/bash/docker -o /etc/bash_completion.d/docker \
&& sed -i "s/ash/bash/" /etc/passwd
RUN rm /sbin/modprobe && echo '#!/bin/true' >/sbin/modprobe && chmod +x /sbin/modprobe
COPY ["docker-prompt", "sudo", "/usr/local/bin/"]
I ve run this cmd :
docker build -t "myImage"
but if fails throwing this :
Step 8/8: COPY ["docker-prompt", "sudo", "/usr/local/bin/"] COPY
failed: stat /var/lib/docker/tmp/docker-builder273771066/sudo: no such
file or directory
Since it's not clear what is the problem ,
Suggestions ?
COPY command work with source and destination only, if you want to own file to sudo then you need to use --chown. otherwise the copy command will consider the Sudo as the source path.
COPY
COPY has two forms:
COPY [--chown=<user>:<group>] <src>... <dest>
COPY [--chown=<user>:<group>] ["<src>",... "<dest>"] (this form is required for paths containing whitespace)
Note:
The --chown feature is only supported on Dockerfiles used to build
Linux containers, and will not work on Windows containers. Since user
and group ownership concepts do not translate between Linux and
Windows, the use of /etc/passwd and /etc/group for translating user
and group names to IDs restricts this feature to only be viable for
Linux OS-based containers.
I assume that you are looking for a way like
COPY --chown=root:root docker-prompt /usr/local/bin/

Why do I get "unzip: short read" when I try to build an image from Dockerfile?

From Spring Microservices in Action book: I am trying to use the Docker Maven Plugin to build a docker image for deploy a Java microservice as Docker container to the cloud.
Dockerfile:
FROM openjdk:8-jdk-alpine
RUN mkdir -p /usr/local/configserver
ADD jce_policy-8.zip /tmp/
RUN unzip /tmp/jce_policy-8.zip && \
rm /tmp/jce_policy-8.zip && \
yes | cp -v /tmp/UnlimitedJCEPolicyJDK8/*.jar /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/
ADD #project.build.finalName#.jar /usr/local/configserver/
ADD run.sh run.sh
RUN chmod +x run.sh
CMD ./run.sh
Output related to step 4 in Dockerfile:
...
---> Using cache
---> dd33d4c12d29
Step 4/8 : RUN unzip /tmp/jce_policy-8.zip && rm /tmp/jce_policy-8.zip && yes | cp -v /tmp/UnlimitedJCEPolicyJDK8/*.jar /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/
---> Running in 1071273ceee5
Archive: /tmp/jce_policy-8.zip
unzip: short read
Why do I get unzip: short read when I try to build the image?
Somehow, curl on alpine linux distro can't set cookie headers correctly while downloading jce zip file. It seems it downloads a zip file but in fact it is an html error page. If you view the file you can see that it is an html file. I've used wget instead of curl and it successfully downloaded file. Then unzip operation worked as expected.
FROM openjdk:8-jdk-alpine
RUN apk update && apk upgrade && apk add netcat-openbsd
RUN mkdir -p /usr/local/configserver
RUN cd /tmp/ && \
wget 'http://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip' --header "Cookie: oraclelicense=accept-securebackup-cookie" && \
unzip jce_policy-8.zip && \
rm jce_policy-8.zip && \
yes |cp -v /tmp/UnlimitedJCEPolicyJDK8/*.jar /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/
ADD #project.build.finalName#.jar /usr/local/configserver/
ADD run.sh run.sh
RUN chmod +x run.sh
CMD ./run.sh
It's possible your jce_policy-8.zip archive is being recognized as a compressed archive and expanded by the ADD instruction. If so, you can skip unzipping on the next line. Or, switch to the COPY instruction, which does no special processing of local archives.
In general, I recommend always using the COPY instruction to bring in files and directories from the build context. Only use ADD when you specifically want the extra unpacking behaviour.
I'm find solved link
FROM openjdk:8-jdk-alpine
RUN apk update && apk upgrade && apk add netcat-openbsd && apk add curl
RUN mkdir -p /usr/local/configserver
RUN cd /tmp/ && \
**curl -L -b "oraclelicense=a" http://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip -O** && \
unzip jce_policy-8.zip && \
rm jce_policy-8.zip && \
yes |cp -v /tmp/UnlimitedJCEPolicyJDK8/*.jar /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/
ADD #project.build.finalName#.jar /usr/local/configserver/
ADD run.sh run.sh
RUN chmod +x run.sh
CMD ./run.sh
Maybe it is related to the fact that the unzip command in alpine is provided busybox and not the standard unzip tool.
Busybox do have bugs related to this error:
https://bugs.busybox.net/show_bug.cgi?id=8821
Here is a related issue with more details:
https://github.com/wahern/luaossl/issues/103
As a workaround installing the standard unzip command should work.

Trying to set JAVA_HOME in a docker image, don't know why its not working

I'm a linux scripting novice so go easy please.
I am trying to build a docker image that includes jdk 1.8.0 and has a variable JAVA_HOME that points to the location of the jdk.
Here's my Dockerfile:
FROM centos
RUN yum install -y wget java-1.8.0-openjdk zip unzip
RUN echo "JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")" | tee -a /etc/profile
RUN source /etc/profile
RUN echo $JAVA_HOME
I'm echoing $JAVA_HOME just to check that it does get set but here's the output when I build the image:
Sending build context to Docker daemon 50.18kB
Step 1/5 : FROM centos
---> 49f7960eb7e4
Step 2/5 : RUN yum install -y wget java-1.8.0-openjdk zip unzip
---> Using cache
---> 98745c0d3d08
Step 3/5 : RUN echo "JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")" | tee -a /etc/profile
---> Using cache
---> 9e3244072966
Step 4/5 : RUN source /etc/profile
---> Using cache
---> 99809c08f2e3
Step 5/5 : RUN echo $JAVA_HOME
---> Running in c15274d8429f
Removing intermediate container c15274d8429f
---> a561301e0546
Successfully built a561301e0546
I'm sure there's a simple reason why this isn't working but I can't figure it out.
By the way, I'm doing this because I want to install maven as per these instructions: https://www.vultr.com/docs/how-to-install-apache-maven-3-5-on-centos-7 .
Any help much appreciated.
You can echo it if you run it part of the same RUN command:
FROM centos
RUN yum install -y wget java-1.8.0-openjdk zip unzip
RUN echo "JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")" | tee -a /etc/profile && source /etc/profile && echo $JAVA_HOME

Error on building Dockerfile to Image

I have the following Dockerfile. I'm trying to build it to an image, but somehow I receive the following error: ADD service /container/service
ADD failed: stat /mnt/sda1/var/lib/docker/tmp/docker-builder005872257/service: no such file or directory at Step 6/9. I don't know why... Can anyone help me?
FROM osixia/light-baseimage:1.1.1
ARG LDAP_OPENLDAP_GID
ARG LDAP_OPENLDAP_UID
RUN if [ -z "${LDAP_OPENLDAP_GID}" ]; then groupadd -r openldap; else groupadd -r -g ${LDAP_OPENLDAP_GID} openldap; fi && if [ -z "${LDAP_OPENLDAP_UID}" ]; then useradd -r -g openldap openldap; else useradd -r -g openldap -u ${LDAP_OPENLDAP_UID} openldap; fi
RUN echo "path-include /usr/share/doc/krb5*" >> /etc/dpkg/dpkg.cfg.d/docker && apt-get -y update && /container/tool/add-service-available :ssl-tools \
&& LC_ALL=C DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
ldap-utils \
libsasl2-modules \
libsasl2-modules-db \
libsasl2-modules-gssapi-mit \
libsasl2-modules-ldap \
libsasl2-modules-otp \
libsasl2-modules-sql \
openssl \
slapd \
krb5-kdc-ldap \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
ADD service /container/service
RUN /container/tool/install-service
ADD environment /container/environment/99-default
EXPOSE 389 636
EDIT
After adding some ls commands in the Dockerfile I've seen the following line in logs:
Step 6/11 : RUN ls /container/
---> Running in 623dca399324
environment
service
service-available
tool
Removing intermediate container 623dca399324
---> 5f7fcb8a1857
Step 7/11 : RUN ls
---> Running in 7f3bd8662113
bin
boot
container
dev
etc
home
lib
lib64
media
mnt
opt
proc
root
run
sbin
srv
sys
tmp
usr
var
Removing intermediate container 7f3bd8662113
---> 99c17cefc572
Step 8/11 : ADD service /container/service
ADD failed: stat /mnt/sda1/var/lib/docker/tmp/docker-builder200387466/service: no such file or directory
Any idea how can I resolve this?
The error means it can't find the directory which mean it probably doesn't exist or you are doing it the wrong way.
One of the things you can do is to make directory and add service to it. Below is a snippet explanation that could teach or help you:
RUN mkdir /container/
Then ADD service to the directory you created. Thus
ADD service /container/service
This can only serve as what could help to put you to track. However I will advice #mohan08p answer above because that works for me.
it successfully build on my local machine.Can you delete the respective files or directories and try once. Also, check the permissions. Did you configure .dockerignore which will not allow to ADD those files. Or else try running with -f or --file command like,
$ docker build . -f Dockerfile
Hope this helps.

Resources