Duplication in Dockerfiles - docker

Question

I have a Django Web-Application that uses celery in the background for periodic tasks.
Right now I have three docker images
one for the django application
one for celery workers
one for the celery scheduler
whose Dockerfiles all look like this:
FROM alpine:3.7
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY Pipfile Pipfile.lock ./
RUN apk update && \
apk add python3 postgresql-libs jpeg-dev git && \
apk add --virtual .build-deps gcc python3-dev musl-dev postgresql-dev zlib-dev && \
pip3 install --no-cache-dir pipenv && \
pipenv install --system && \
apk --purge del .build-deps
COPY . ./
# Run the image as a non-root user
RUN adduser -D noroot
USER noroot
EXPOSE $PORT
CMD <Different CMD for all three containers>
So they are all exactly the same except the last line.
Would it make sense here to create some kind of base image that contains everything except CMD. And all three images use that as base and add only their respective CMD?
Or won't that give me any advantages, because everything is cached anyway?
Is a seperation like you see above reasonable?
Two small bonus questions:
Sometimes the apk update .. layer is cached by docker. How does docker know that there are no updates here?
I often read that I should decrease layers as far a possible to reduce image size. But isn't that against the caching idea and will result in longer builds?

Answer 1

I will suggest to use one Dockerfile and just update your CMD during runtime. Litle bit modification will work for both local and Heroku as well.
As far Heroku is concern they provide environment variable to start container with the environment variable.
heroku set-up-your-local-environment-variables
FROM alpine:3.7
ENV PYTHONUNBUFFERED 1
ENV APPLICATION_TO_RUN=default_application
RUN mkdir /code
WORKDIR /code
COPY Pipfile Pipfile.lock ./
RUN apk update && \
apk add python3 postgresql-libs jpeg-dev git && \
apk add --virtual .build-deps gcc python3-dev musl-dev postgresql-dev zlib-dev && \
pip3 install --no-cache-dir pipenv && \
pipenv install --system && \
apk --purge del .build-deps
COPY . ./
# Run the image as a non-root user
RUN adduser -D noroot
USER noroot
EXPOSE $PORT
CMD $APPLICATION_TO_RUN
So When run you container pass your application name to run command.
docker run -it --name test -e APPLICATION_TO_RUN="celery beat" --rm test

Answer 2

I would recommend looking at docker-compose to simplify management of multiple containers.
Use a single Dockerfile like the one you posted above, then create a docker-compose.yml that might look something like this:
version: '3'
services:
# a django service serving an application on port 80
django:
build: .
command: python manage.py runserver
ports:
- 8000:80
# the celery worker
worker:
build: .
command: celery worker
# the celery scheduler
scheduler:
build: .
command: celery beat
Of course, modify the commands here to be whatever you are using for your currently separate Dockerfiles.
When you want to rebuild the image, docker-compose build will rebuild your container image from your Dockerfile for the first service, then reuse the built image for the other services (because they already exist in the cache). docker-compose up will spin up 3 instances of your container image, but overriding the run command each time.
If you want to get more sophisticated, there are plenty of resources out there for the very common combination of django and celery.