I want to create a dockerfile which contains 2 stages.
The first stage is to set up a MySQL server and the second stage is to start a backend service that accesses the server.
The problem is that the backend service stops when no MySQL server is available. Is there a way to make the stage dependent on the first stage being started?
what is a little strange is that when i create the dockerfile with the database at the top, the log of the backend is displayed. If the backend is on top, the log of the MySQL is displayed when starting.
Actual Dockerfile:
FROM mysql:latest AS BackendDatabase
RUN chown -R mysql:root /var/lib/mysql/
ARG MYSQL_DATABASE="DienstplanverwaltungDatabase"
ARG MYSQL_USER="user"
ARG MYSQL_PASSWORD="password"
ARG MYSQL_ROOT_PASSWORD="password"
ENV MYSQL_DATABASE=$MYSQL_DATABASE
ENV MYSQL_USER=$MYSQL_USER
ENV MYSQL_PASSWORD=$MYSQL_PASSWORD
ENV MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD
EXPOSE 3306
FROM openjdk:10-jre-slim AS Backend
LABEL description="Backend Dienstplanverwaltung"
LABEL maintainer="Martin"
COPY ./SpringDienstplanverwaltung/build/libs/dienstplanverwaltung-0.0.1-SNAPSHOT.jar /usr/local/app.jar
EXPOSE 8080
ENTRYPOINT java -jar /usr/local/app.jar
actually you need Docker-composer of two containers. One for Mysql one for java app.
Multistage is mostly for cases like #1 build something, for example java or Go. #2 create second image and copy results of build. The general idea is to keep the second stage clean. We do not need to build tools, only results in second stage.
please see example:
FROM
Learn more about the "FROM" Dockerfile command.
golang:1.16
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go ./
RUN CGO_ENABLED=0 go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=0 /go/src/github.com/alexellis/href-counter/app ./
CMD ["./app"]
Okay you seem to be a little confused with various things here. First of all, multi-stage builds are for building an application that needs some kind of build/compiling process, copying that build into another container with fewer dependencies and with just the executable, so in this context, trying to run a database in a multistage build makes no sense at all, due to the fact that building the container does not run it.
Now, you want to have a multi stage to build the java app and then copy that build into another container and then run it. Also, when you are running that container you need a mysql database, using docker-compose is a good tool for that, like this example:
version: '3.8'
services:
db:
image: mysql:8.0
cap_add:
- SYS_NICE
restart: always
environment:
- MYSQL_DATABASE=mydatabase
- MYSQL_ROOT_PASSWORD=mypassword
ports:
- '3306:3306'
volumes:
- db:/var/lib/mysql
# - ./db/init.sql:/docker-entrypoint-initdb.d/init.sql
api:
container_name: your-backend
build:
context: .
image: your-backend
depends_on:
- db
ports:
- 8080:8080
environment:
ENV_VAR_EXAMPLE: example
links:
- db
volumes:
db:
driver: local
Also, an example multi-stage Dockerfile for java applications:
# First stage: complete build environment
FROM maven:3.5.0-jdk-8-alpine AS builder
# add pom.xml and source code
ADD ./pom.xml pom.xml
ADD ./src src/
# package jar
RUN mvn clean package
# Second stage: minimal runtime environment
From openjdk:8-jre-alpine
# copy jar from the first stage
COPY --from=builder target/my-app-1.0-SNAPSHOT.jar my-app-1.0-SNAPSHOT.jar
EXPOSE 8080
CMD ["java", "-jar", "my-app-1.0-SNAPSHOT.jar"]
Related
My Setup:
I have 3 Services defined in my docker-compose.yml: frontend backend and postgresql. postgresql is pulled from docker-hub.
frontend and backend are built from their own Dockerfiles, most of the Code of these Dockerfiles is the same and only EXPOSE ENTRPOINT CMD and ARG-Values differ from each other. That is why I wanted to create a 'base-Dockerfile' that these two Services can "include".
Sadly I found out I can not simply "include" a Dockerfile into another Dockerfile, I have to create an Image.
So I tried to create a base image for frontend and backend in my docker-compose.yml:
services:
frontend_base:
image: frontend_base_image
build:
context: ./
dockerfile: base.dockerfile
args:
- WORKDIR=/app/frontend/
- TOOLSDIR=${PWD}/docker/tools
- LOCALDIR=${PWD}/app/frontend/client
backend_base:
image: backend_base_image
build:
context: ./
dockerfile: base.dockerfile
args:
- WORKDIR=/app/backend/
- TOOLSDIR=${PWD}/docker/tools
- LOCALDIR=${PWD}/app/backend/api
frontend:
depends_on:
- frontend_base
# Some more stuff for the service
backend:
depends_on:
- backend_base
# Some more stuff for the service
My 'base-Dockerfile':
FROM node:18
# Set in docker-compose.yml-file
ARG WORKDIR
ARG TOOLSDIR
ARG LOCALDIR
ENV WORKDIR=${WORKDIR}
# Install dumb-init for the init system
RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_x86_64
RUN chmod +x /usr/local/bin/dumb-init
WORKDIR ${WORKDIR}
RUN mkdir -p ${WORKDIR}
# Copy package.json to the current workdir (for npm install)
COPY ${LOCALDIR}/package*.json ${WORKDIR}
# Install all Packages (refereed from package.json)
RUN npm install
COPY ${TOOLSDIR}/start.sh /usr/local/bin/start.sh
COPY ${LOCALDIR}/ ${WORKDIR}
The Problem I am facing:
My frontend and backend Dockerfiles try to pull the 'base-image' from docker.io
=> ERROR [docker-backend internal] load metadata for docker.io/library/backend_base_image:latest 0.9s
=> ERROR [docker-frontend internal] load metadata for docker.io/library/frontend_base_image:latest 0.9s
=> CANCELED [frontend_base_image internal] load metadata for docker.io/library/node:18
My Research:
I do not know if my approach is possible, I did not find much Resources about this (integrated with docker-compose) online, only Resources about building the Images via Shell and then using them in a Dockerfile. I also tried this and ran into some other issues, where I could not provide correct arguments to the base-Dockerfile.
So I firstly wanted to find out if it is possible with docker-compose.
I am sorry if this is super obvious and my Question is dumb, I am relatively new to Docker.
We could use the feature of a multistage containerfile to define all three images in a single containerfile:
FROM node:18 AS base
# Set in docker-compose.yml-file
ARG WORKDIR
ARG TOOLSDIR
ARG LOCALDIR
ENV WORKDIR=${WORKDIR}
# Install dumb-init for the init system
RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_x86_64
RUN chmod +x /usr/local/bin/dumb-init
WORKDIR ${WORKDIR}
RUN mkdir -p ${WORKDIR}
# Copy package.json to the current workdir (for npm install)
COPY ${LOCALDIR}/package*.json ${WORKDIR}
# Install all Packages (refereed from package.json)
RUN npm install
COPY ${TOOLSDIR}/start.sh /usr/local/bin/start.sh
COPY ${LOCALDIR}/ ${WORKDIR}
FROM base AS frontend
...
FROM base AS backend
...
In our docker-compose.yml, we can then build a specific stage for the frontend- and backend-service:
...
frontend:
image: frontend
build:
context: ./
target: frontend
dockerfile: base.dockerfile
...
backend:
image: backend
build:
context: ./
target: backend
dockerfile: base.dockerfile
...
If you want a single base image with shared tools, you can do this almost exactly the way you describe; but the one caveat is that you can't describe the base image in the docker-compose.yml file. You need to run separately from Compose
docker build -t base-image -f base.dockerfile .
I would not try to install any application code in that base Dockerfile. Where you for example install an init wrapper that needs to be shared across all of your application images, that does make sense. I think it's fine to tie a Dockerfile to a specific source-tree and image layout, and don't typically recommend passing filesystem paths as ARGs.
# base.dockerfile
FROM node:18
RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_x86_64 \
&& chmod +x /usr/local/bin/dumb-init
COPY docker/tools/start.sh /usr/local/bin/
ENTRYPOINT ["dumb-init", "--"]
CMD ["start.sh"]
The per-image Dockerfiles will look pretty similar – and like every other Node Dockerfile – but there's no harm in repeating this, in much the same way that your components probably have similar-looking but self-contained package.json files.
# */Dockerfile
FROM base-image
WORKDIR /app # also creates it
COPY package*.json ./
RUN npm ci
COPY ./ ./
RUN npm build
EXPOSE 3000
# CMD ["npm", "run", "start"] # if the start.sh from the base is wrong
Of note, this gives you some flexibility to change things if the two image setups aren't identical; if you need an additional build step, or if you want to run a dev server, or package the frontend into a lighter-weight Nginx server.
In the Compose file you'd declare these normally with a build: block. Compose isn't aware of the base image and there's no way to tell it about it.
version: '3.8'
services:
frontend:
build: ./app/frontend/client
ports: ['3000:3000']
backend:
build: ./app/backend/api
ports: ['3001:3000']
One thing I've done here which at least reduces the number of variable references is to consistently use . as the current directory name. In the Compose file that's the directory containing the docker-compose.yml; on the left-hand side of COPY it's the build: context directory on the host; on the right-hand side of COPY it's the most recent WORKDIR. Using . where appropriate means you don't have to repeat the directory name, so you do have a little flexibility if you do need to rearrange your source tree or container filesystem.
I have 1 dockerfile, 1 stage of the build for the node server, serving some data, and the 2nd stage is a react app. I use a docker compose file to run the dockerfile.
I am able to access the react app via port 3000, but the 2nd stage server isn't running so I can't access the data.
Any idea how to solve it?
FROM node:12.6
WORKDIR /usr/src/app
COPY package.json .
COPY . .
EXPOSE 5500 // node server
CMD ["npm","run", "server"]
FROM node:12.6
WORKDIR /usr/src/app
COPY package.json .
RUN npm i
COPY . .
EXPOSE 3000 // react app
CMD ["npm","run", "dev"]
version: "3.9"
services:
testingapp:
container_name: testingApp
build: .
volumes:
- ./src:/app/src:delegated
ports:
- "3000:3000"
I have read various docs online.
You're trying to run the front- and back-ends in the same container. A container only runs one process, though; if you need two separate processes from the same code base then you can run two separate containers off the same image, overriding the command: on one of them.
So reduce the Dockerfile to copy the code base in, and declare one process or the other as the main container command:
FROM node:12.6
WORKDIR /usr/src/app
COPY package.json package-lock.json ./
RUN npm ci
COPY ./ ./
EXPOSE 3000
CMD ["npm", "run", "server"]
Now in your Compose file, declare two separate containers. For the second, override the command: with the alternate program to run. Both can build: the same image; the second build will come entirely from the Docker layer cache and be all but free. The code is built into the image and you don't need to replace it using volumes:.
version: '3.8'
services:
express:
build: .
ports: ['5500:3000']
react:
build: .
command: npm run dev
ports: ['3000:3000']
I have this files:
docker-compose.yml (shortened):
version: '3.7'
services:
php-fpm:
build:
context: .
dockerfile: docker/php/Dockerfile
target: dev
volumes:
- .:/app
frontend:
build:
context: .
dockerfile: docker/php/Dockerfile
target: frontend
volumes:
- .:/app
docker/php/Dockerfile (shortened):
FROM alpine:3.13 AS frontend
WORKDIR /app
COPY . .
RUN apk add npm
RUN npm install
RUN npx webpack -p --color --progress
FROM php:7.4-fpm AS dev
ENTRYPOINT ["docker-php-entrypoint"]
WORKDIR /app
COPY ./docker/php/www-dev.conf /usr/local/etc/php-fpm.d/www.conf
CMD ["php-fpm"]
I want to use all what building in frontend (as I understood at the stage build at this time volumes are not available) in php-fpm container, but I get something like this: file_get_contents(/app/static/frontend.version): failed to open stream.
How I can do this? I don't understand very well in Docker and the only solution I have is to move build script to php-fpm container.
You need to delete the volumes: in your docker-compose.yml file. They replace the entire contents of the image's /app directory with content from the host, which means everything that gets done in the Dockerfile gets completely ignored.
The Dockerfile you show uses a setup called a multi-stage build. The important thing you can do with this is build the first part of your image using Node, then COPY --from=frontend the static files into the second part. You do not need to declare a second container in docker-compose.yml to run the first stage, the build sequence runs this automatically. This at a minimum looks like
COPY --from=frontend /app/build ./static
You will also need to COPY the rest of your application code into the image.
If you move the Dockerfile up to the top of your project's source tree, then the docker-compose.yml file becomes as simple as
version: '3.8'
services:
php-fpm:
build: . # default Dockerfile, default target (last stage)
# do not overwrite application code with volumes:
# no separate frontend: container
But you've put a little bit more logic in the Dockerfile. I might write:
FROM node:lts AS frontend # use a prebuilt Node image
WORKDIR /app
COPY package*.json . # install dependencies first to save time on rebuild
RUN npm install
COPY . . # (or a more specific subdirectory?)
RUN npx webpack -p --color --progress
FROM php:7.4-fpm AS dev
WORKDIR /app
COPY . . # (or a more specific subdirectory?)
COPY --from=frontend /app/build ./static
COPY ./docker/php/www-dev.conf /usr/local/etc/php-fpm.d/www.conf
# don't need to repeat unmodified ENTRYPOINT/CMD from base image
I'm trying to containerize two services an socket service and a django application
My file structure is
\main file {docker-compose file}
\ django application {Dockerfile}
\ socket app {Dockerfile}
When I run docker build . it build the image
then when I run docker-compose build,
I notice that the socket app and django app are copied to the container instead of only the django application as specified by the Dockerfile.
I get the idea that the Dockerfile is executed in the main directory instead of the django directory?
Here is Dockerfile that is inside the django app application
# Pull base image
FROM python:3
# Set environment varibles
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Set work directory
WORKDIR /code
# Install dependencies
COPY requirements.txt /code/
RUN pip install -r requirements.txt
# Copy project
COPY . /code/
RUN ls
And here is the docker-compose file.
With the usage of the ls command I tried to figure out what happend and the output is that the applications in the main folder are copied instead of the django application.
version: '3'
services:
db:
image: postgres:10.1-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
web:
build: ./django_app
command: ls /code/
volumes:
- .:/code
ports:
- 8000:8000
depends_on:
- db
volumes:
postgres_data:
is this intended use or am I doing something wrong?
The volumes: directive in your docker-compose.yml file is hiding literally everything your Dockerfile does. You'll solve your immediate problem by changing the two directories to match: in the volumes: directive, bind-mount ./django_app:/code.
In a more production-oriented workflow, I'd recommend making your Docker image totally self-contained: make sure it has a CMD that runs your application, and do not use volumes: to inject your code. Delete command: and volumes: from the docker-compose.yml and let the image provide its own code and default command. (To do development, use a Python virtual environment for local code isolation, and make sure all of your tests and a basic hand-run workflow pass before using Docker for anything.)
I'm using docker and docker-compose to set up a build pipeline. I've got a front-end that's written in javascript and needs to be built before being used. The backend is written in go.
To make this component integrate with the rest of our docker-compose setup, I want to do the building in a docker image as well.
This is the flow I'm going for:
during build do:
build the frontend stuff and put it in /output (that is bound to the
output volume
build the backend server
when running do:
run the server, it has access to the build files in /output
I'm quite new to docker and docker-compose so I'm not sure if this is possible, or even the right thing to do.
For reference, here's my docker-compose.yml:
version: '2'
volumes:
output:
driver: local
services:
frontend:
build: .
volumes:
- output:/output
backend:
build: ./backend
depends_on:
- frontend
volumes:
- output:/output
and Dockerfile:
FROM node
# create working dir
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
ADD package.json /usr/src/app/package.json
# install packages
RUN npm install
COPY . /usr/src/app
# build frontend files and place results in /output
RUN npm build
RUN cp /usr/src/app/build/* /output
And backend/Dockerfile:
FROM go
# copy and build server
COPY . /usr/src/backend
WORKDIR /usr/src/backend
RUN go build
# run the server
ENTRYPOINT ["/usr/src/backend/main"]
Something is wrong here, but I do not know what. It seems as though the output of the build step are not persisted in the output volume. What can I do to fix this?
You cannot attach a volume during docker build.
The reason for this is that the goal of the docker build command is to build an image, and nothing else, it doesn't need to have volumes, as Dockerfile has ADD / COPY.
To produce your output, you should create a script which mostly does the npm install ; npm build ; cp /usr/src/app/build/* /output from your current dockerfile and use this script as the entrypoint / cmd in your dockerfile.
I'm not sure compose can run this, but in any case, I find it more clear wrapped in a shell script that first executes the frontend builder container, then executing the backend container with the output directory as a volume.