Docker: Shared volume when build - docker

I have this files:
docker-compose.yml (shortened):
version: '3.7'
services:
php-fpm:
build:
context: .
dockerfile: docker/php/Dockerfile
target: dev
volumes:
- .:/app
frontend:
build:
context: .
dockerfile: docker/php/Dockerfile
target: frontend
volumes:
- .:/app
docker/php/Dockerfile (shortened):
FROM alpine:3.13 AS frontend
WORKDIR /app
COPY . .
RUN apk add npm
RUN npm install
RUN npx webpack -p --color --progress
FROM php:7.4-fpm AS dev
ENTRYPOINT ["docker-php-entrypoint"]
WORKDIR /app
COPY ./docker/php/www-dev.conf /usr/local/etc/php-fpm.d/www.conf
CMD ["php-fpm"]
I want to use all what building in frontend (as I understood at the stage build at this time volumes are not available) in php-fpm container, but I get something like this: file_get_contents(/app/static/frontend.version): failed to open stream.
How I can do this? I don't understand very well in Docker and the only solution I have is to move build script to php-fpm container.

You need to delete the volumes: in your docker-compose.yml file. They replace the entire contents of the image's /app directory with content from the host, which means everything that gets done in the Dockerfile gets completely ignored.
The Dockerfile you show uses a setup called a multi-stage build. The important thing you can do with this is build the first part of your image using Node, then COPY --from=frontend the static files into the second part. You do not need to declare a second container in docker-compose.yml to run the first stage, the build sequence runs this automatically. This at a minimum looks like
COPY --from=frontend /app/build ./static
You will also need to COPY the rest of your application code into the image.
If you move the Dockerfile up to the top of your project's source tree, then the docker-compose.yml file becomes as simple as
version: '3.8'
services:
php-fpm:
build: . # default Dockerfile, default target (last stage)
# do not overwrite application code with volumes:
# no separate frontend: container
But you've put a little bit more logic in the Dockerfile. I might write:
FROM node:lts AS frontend # use a prebuilt Node image
WORKDIR /app
COPY package*.json . # install dependencies first to save time on rebuild
RUN npm install
COPY . . # (or a more specific subdirectory?)
RUN npx webpack -p --color --progress
FROM php:7.4-fpm AS dev
WORKDIR /app
COPY . . # (or a more specific subdirectory?)
COPY --from=frontend /app/build ./static
COPY ./docker/php/www-dev.conf /usr/local/etc/php-fpm.d/www.conf
# don't need to repeat unmodified ENTRYPOINT/CMD from base image

Related

Using base Images for Services in docker-compose with different args

My Setup:
I have 3 Services defined in my docker-compose.yml: frontend backend and postgresql. postgresql is pulled from docker-hub.
frontend and backend are built from their own Dockerfiles, most of the Code of these Dockerfiles is the same and only EXPOSE ENTRPOINT CMD and ARG-Values differ from each other. That is why I wanted to create a 'base-Dockerfile' that these two Services can "include".
Sadly I found out I can not simply "include" a Dockerfile into another Dockerfile, I have to create an Image.
So I tried to create a base image for frontend and backend in my docker-compose.yml:
services:
frontend_base:
image: frontend_base_image
build:
context: ./
dockerfile: base.dockerfile
args:
- WORKDIR=/app/frontend/
- TOOLSDIR=${PWD}/docker/tools
- LOCALDIR=${PWD}/app/frontend/client
backend_base:
image: backend_base_image
build:
context: ./
dockerfile: base.dockerfile
args:
- WORKDIR=/app/backend/
- TOOLSDIR=${PWD}/docker/tools
- LOCALDIR=${PWD}/app/backend/api
frontend:
depends_on:
- frontend_base
# Some more stuff for the service
backend:
depends_on:
- backend_base
# Some more stuff for the service
My 'base-Dockerfile':
FROM node:18
# Set in docker-compose.yml-file
ARG WORKDIR
ARG TOOLSDIR
ARG LOCALDIR
ENV WORKDIR=${WORKDIR}
# Install dumb-init for the init system
RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_x86_64
RUN chmod +x /usr/local/bin/dumb-init
WORKDIR ${WORKDIR}
RUN mkdir -p ${WORKDIR}
# Copy package.json to the current workdir (for npm install)
COPY ${LOCALDIR}/package*.json ${WORKDIR}
# Install all Packages (refereed from package.json)
RUN npm install
COPY ${TOOLSDIR}/start.sh /usr/local/bin/start.sh
COPY ${LOCALDIR}/ ${WORKDIR}
The Problem I am facing:
My frontend and backend Dockerfiles try to pull the 'base-image' from docker.io
=> ERROR [docker-backend internal] load metadata for docker.io/library/backend_base_image:latest 0.9s
=> ERROR [docker-frontend internal] load metadata for docker.io/library/frontend_base_image:latest 0.9s
=> CANCELED [frontend_base_image internal] load metadata for docker.io/library/node:18
My Research:
I do not know if my approach is possible, I did not find much Resources about this (integrated with docker-compose) online, only Resources about building the Images via Shell and then using them in a Dockerfile. I also tried this and ran into some other issues, where I could not provide correct arguments to the base-Dockerfile.
So I firstly wanted to find out if it is possible with docker-compose.
I am sorry if this is super obvious and my Question is dumb, I am relatively new to Docker.
We could use the feature of a multistage containerfile to define all three images in a single containerfile:
FROM node:18 AS base
# Set in docker-compose.yml-file
ARG WORKDIR
ARG TOOLSDIR
ARG LOCALDIR
ENV WORKDIR=${WORKDIR}
# Install dumb-init for the init system
RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_x86_64
RUN chmod +x /usr/local/bin/dumb-init
WORKDIR ${WORKDIR}
RUN mkdir -p ${WORKDIR}
# Copy package.json to the current workdir (for npm install)
COPY ${LOCALDIR}/package*.json ${WORKDIR}
# Install all Packages (refereed from package.json)
RUN npm install
COPY ${TOOLSDIR}/start.sh /usr/local/bin/start.sh
COPY ${LOCALDIR}/ ${WORKDIR}
FROM base AS frontend
...
FROM base AS backend
...
In our docker-compose.yml, we can then build a specific stage for the frontend- and backend-service:
...
frontend:
image: frontend
build:
context: ./
target: frontend
dockerfile: base.dockerfile
...
backend:
image: backend
build:
context: ./
target: backend
dockerfile: base.dockerfile
...
If you want a single base image with shared tools, you can do this almost exactly the way you describe; but the one caveat is that you can't describe the base image in the docker-compose.yml file. You need to run separately from Compose
docker build -t base-image -f base.dockerfile .
I would not try to install any application code in that base Dockerfile. Where you for example install an init wrapper that needs to be shared across all of your application images, that does make sense. I think it's fine to tie a Dockerfile to a specific source-tree and image layout, and don't typically recommend passing filesystem paths as ARGs.
# base.dockerfile
FROM node:18
RUN wget -O /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_x86_64 \
&& chmod +x /usr/local/bin/dumb-init
COPY docker/tools/start.sh /usr/local/bin/
ENTRYPOINT ["dumb-init", "--"]
CMD ["start.sh"]
The per-image Dockerfiles will look pretty similar – and like every other Node Dockerfile – but there's no harm in repeating this, in much the same way that your components probably have similar-looking but self-contained package.json files.
# */Dockerfile
FROM base-image
WORKDIR /app # also creates it
COPY package*.json ./
RUN npm ci
COPY ./ ./
RUN npm build
EXPOSE 3000
# CMD ["npm", "run", "start"] # if the start.sh from the base is wrong
Of note, this gives you some flexibility to change things if the two image setups aren't identical; if you need an additional build step, or if you want to run a dev server, or package the frontend into a lighter-weight Nginx server.
In the Compose file you'd declare these normally with a build: block. Compose isn't aware of the base image and there's no way to tell it about it.
version: '3.8'
services:
frontend:
build: ./app/frontend/client
ports: ['3000:3000']
backend:
build: ./app/backend/api
ports: ['3001:3000']
One thing I've done here which at least reduces the number of variable references is to consistently use . as the current directory name. In the Compose file that's the directory containing the docker-compose.yml; on the left-hand side of COPY it's the build: context directory on the host; on the right-hand side of COPY it's the most recent WORKDIR. Using . where appropriate means you don't have to repeat the directory name, so you do have a little flexibility if you do need to rearrange your source tree or container filesystem.

2 docker builds into a multi-build

I have 1 dockerfile, 1 stage of the build for the node server, serving some data, and the 2nd stage is a react app. I use a docker compose file to run the dockerfile.
I am able to access the react app via port 3000, but the 2nd stage server isn't running so I can't access the data.
Any idea how to solve it?
FROM node:12.6
WORKDIR /usr/src/app
COPY package.json .
COPY . .
EXPOSE 5500 // node server
CMD ["npm","run", "server"]
FROM node:12.6
WORKDIR /usr/src/app
COPY package.json .
RUN npm i
COPY . .
EXPOSE 3000 // react app
CMD ["npm","run", "dev"]
version: "3.9"
services:
testingapp:
container_name: testingApp
build: .
volumes:
- ./src:/app/src:delegated
ports:
- "3000:3000"
I have read various docs online.
You're trying to run the front- and back-ends in the same container. A container only runs one process, though; if you need two separate processes from the same code base then you can run two separate containers off the same image, overriding the command: on one of them.
So reduce the Dockerfile to copy the code base in, and declare one process or the other as the main container command:
FROM node:12.6
WORKDIR /usr/src/app
COPY package.json package-lock.json ./
RUN npm ci
COPY ./ ./
EXPOSE 3000
CMD ["npm", "run", "server"]
Now in your Compose file, declare two separate containers. For the second, override the command: with the alternate program to run. Both can build: the same image; the second build will come entirely from the Docker layer cache and be all but free. The code is built into the image and you don't need to replace it using volumes:.
version: '3.8'
services:
express:
build: .
ports: ['5500:3000']
react:
build: .
command: npm run dev
ports: ['3000:3000']

Docker compose is failing when copying npmrc

I am getting this error when trying to run my docker with docker compose
=> ERROR [6/9] COPY .npmrc .npmrc
0.0s => ERROR [7/9] ADD package.json /Users/gs/Documents/Dev/DockerDi 0.0s
version: "3"
services:
gs-service:
build:
context: .
dockerfile: ./gs/Dockerfile
ports:
- 8081:8081
ui-service:
build:
context: .
dockerfile: ./ui/Dockerfile
ports:
- 8083:8083
Here is the detail of the Dockerfile which is failing. This is a subdirectory. For example /user/gs/dockerexperiment/ui
docker-compose file is in parent directory. For example
/user/gs/dockerexperiment
Dockerfile is in subdirectory. For example
/user/gs/dockerexperiment/ui
I know something is going wrong with the class path but I am having a difficult time figuring out what I am missing
A Dockerfile operates on a build context; this is a copy of the directory tree you name as the build: { context: } in the docker-compose.yml file (or, if you're using docker build directly, its directory argument).
build:
context: .
In the Dockerfile, any COPY (or ADD) instructions are interpreted as relative to this directory tree. (If it looks like an absolute path, it's still relative to this directory; any .. path steps that would step outside this directory get ignored.)
COPY package*.json ./
# copies ./package.json, relative to the context directory
But, you clarify in a comment, this isn't actually where your files are. Relative to the docker-compose.yml file, you need ./ui/package.json; the Dockerfile is also in the same subdirectory.
The Dockerfile you show doesn't seem to need any content from outside its immediate tree, so the easiest way to address this is to set the ui directory as the build context:
version: '3.8'
services:
ui:
build: ./ui
# Same as
# build:
# context: ./ui
# dockerfile: Dockerfile # relative to the context directory
ports:
- 8085:8085
In the Dockerfile, you can remove the duplicate ADD package.json line, and you should remove the dangerous RUN npm set config line. You can also set a much shorter WORKDIR, something like /app would be typical.
If you don't want to change the docker-compose.yml file, the other possible change is to adjust the Dockerfile so that its COPY paths are relative to the (parent) build-context directory.
COPY ui/package*.json .
The advantage of doing this is that you can access files from the other application, if you need to, or you can store shared files in that parent directory.
FROM node:14
# Add the public TLS key for our internal repository
# (from the parent directory)
COPY repository.example.com.crt /usr/local/share/ca-certificates
RUN update-ca-certificates
WORKDIR /app
# Also get our local npm configuration (from the parent directory)
COPY .npmrc .
# Install the ./ui application
COPY ui/package*.json .
RUN npm ci
COPY ui/src ./src
EXPOSE 8085
CMD ["npm", "start"]

"No Go files in..." error when i use go with docker compose

I installed Go on Ubuntu 16.04. This is my GOPATH=/home/{username}/work.
I created a project into /home/{username}/work/src.
This is my project folder hierarchy.
project-name
services
configuration
api
main.go
Dockerfile
bff
api
main.go
Dockerfile
docker-compose.yml
favicon.ico
README.md
I can build and run with my dockerfile but I can't build and up with docker-compose.
I couldn't find any solution.
Configuration service dockerfile:
FROM golang:1.11.1-alpine3.8 as builder
RUN apk update && apk add git && go get gopkg.in/natefinch/lumberjack.v2
RUN mkdir -p /go/src/project-name/services/configuration
RUN CGO_ENABLED=0
RUN GOOS=linux
ADD . /go/src/project-name/services/configuration
ENV GOPATH /go
WORKDIR /go/src/project-name/services/configuration/api
RUN go get
RUN go build
FROM alpine
RUN apk update
RUN apk add curl
RUN mkdir -p /app
COPY --from=builder /go/src/project-name/services/configuration/api/ /app/
RUN chmod +x /app/api
WORKDIR /app
EXPOSE 5001
ENTRYPOINT ["/app/api"]
It works with dockerfile.
This is my docker-compose file:
version: '3.4'
services:
bff:
image: project-name/bff:${TAG:-latest}
build:
context: .
dockerfile: services/bff/Dockerfile
ports:
- "5000:5000"
container_name: bff
depends_on:
- configuration
configuration:
image: project-name/configuration:${TAG:-latest}
build:
context: .
dockerfile: services/configuration/Dockerfile
ports:
- "5001:5001"
container_name: configuration
It didn't work.
When the “run go get” command runs, it gives an error, the error is:
can't load package: package project-name/services/configuration/api: no Go files in /go/src/project-name/services/configuration/api
ERROR: Service 'configuration' failed to build: The command '/bin/sh -c go get' returned a non-zero code: 1
In your Dockerfile, you say
ADD . /go/src/project-name/services/configuration
which expects the build context directory on the host to contain the source files. But your docker-compose.yml file says
build:
context: .
dockerfile: services/configuration/Dockerfile
where the context directory is the root of your source control tree, not the specific Go source directory you're trying to build. If you change this to
build:
context: services/configuration
# Default value of "dockerfile: Dockerfile" will be right
it will likely work better.
In plain Docker commands, your current docker-compose.yml file says the equivalent of
cd $GOPATH/src/project-name
docker build -f services/configuration/Dockerfile .
But you're probably actually running
cd $GOPATH/src/project-name/services/configuration
docker build .
and what directory is the current directory matters.

Binary isn't persisted in dockerfile

Newbie to Docker and what appears to be a simple issue isn't working. I would like to build a binary inside of a dockerfile and execute the binary in docker-compose.
Dockerfile:
FROM golang:1.10
ENV DIR=/go/src/api
RUN go get github.com/Masterminds/glide
WORKDIR $DIR
ADD glide.yaml glide.yaml
ADD glide.lock glide.lock
RUN glide install
COPY . $DIR
RUN CGO_ENABLED=0 GOOS=linux go build -o main .
CMD main
And in my docker-compose.yml I have this:
version: '2'
services:
api:
build:
context: ./api
dockerfile: Dockerfile
ports:
- 8000:8000
However, there's no main executable when run from docker-compose. I've actually added a command: ls to the docker-compose file and it's not there. I've then added a ls to the Dockerfile image and main is there.
How do I prevent main from being lost in translation?

Resources