I have a Dockerfile for a Django and Vue.js app that I use along with Gitlab.
The problem that I'm about to describe only happens when deploying via Gitlab CI and the corresponding .gitlab-ci.yml file. When running the docker-compose up command in my local machine, this doesn happen.
So I run docker-compose up and all the instructions in the Dockerfile run apparently fine. But when I check the production server, the dist folder (where the bundle.js and bundle.css should be stored) doesn't exist.
The logs that are spit out while running the Dockerfile confirm that the npm install and npm run build commands are run, and it even confirms that the dist/bundle.js and dist/bundle.css files have been generated. But for some reason they seem to be deleted.
This is my Dockerfile:
FROM python:3.7-alpine
MAINTAINER My Name
ENV PYTHONUNBUFFERED 1
RUN mkdir /app
# make the 'app' folder the current working directory
WORKDIR /app
# copy project files and folders to the current working directory (i.e. 'app' folder)
COPY ./app .
COPY ./requirements.txt /requirements.txt
RUN apk add --update --no-cache postgresql-client
RUN apk add --update --no-cache --virtual .tmp-build-deps \
gcc libc-dev linux-headers postgresql-dev
RUN pip install -r /requirements.txt
RUN apk del .tmp-build-deps
# copy both 'package.json' and 'package-lock.json' (if available)
COPY app/frontend/package*.json ./frontend/
# Install npm
RUN apk add --update nodejs && apk add --update nodejs-npm
# install project dependencies
WORKDIR /app/frontend
RUN npm install
# build app for production with minification
RUN npm run build
RUN adduser -D user
USER user
CMD ["sh ../scripts/entrypoint.sh"]
This is the .gitlab-ci.yml file:
image: docker:latest
services:
- docker:dind
before_script:
- echo "Runnig before_script"
- sudo apt-get install -y python-pip
- sudo apt-get install -y nodejs
- pip install docker-compose
stages:
- test
- build
- deploy
test:
stage: test
script:
- echo "Testing the app"
- docker-compose run app sh -c "python /app/manage.py test && flake8"
build:
stage: build
only:
- develop
- production
- feature/gitlab_ci
script:
- echo "Building the app"
- docker-compose build
deploy:
stage: deploy
only:
- master
- develop
- feature/gitlab_ci
script:
- echo "Deploying the app"
- docker-compose up --build -d
This is the content of the docker-compose.yml file:
version: "3"
services:
app:
build:
context: .
ports:
- "8000:8000"
volumes:
- ./app:/app
command: >
sh -c "python /app/manage.py runserver 0.0.0.0:8000"
environment:
- DB_HOST=db
- DB_NAME=app
- DB_USER=postgres
- DB_PASS=postgres
depends_on:
- db
db:
image: postgres:10-alpine
environment:
- POSTGRES_DB=app
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
This is the content of the entrypoint.sh file:
#!/bin/bash
(cd .. && ./manage.py collectstatic --noinput)
# Migration files are commited to git. Makemigrations is not needed.
# ./manage.py makemigrations app_name
(cd .. && ./manage.py migrate)
I would like to know why the dist/ folder disappears and how to keep it.
When your docker-compose.yml file says
volumes:
- ./app:/app
that hides everything that your Dockerfile builds in the /app directory and replaces it with whatever's in your local system. If your host doesn't have a ./app/frontend/dist then your container won't have that path either, regardless of whatever the Dockerfile does.
I would generally recommend just deleting this volumes: block entirely. It introduces an awkward live-development path (where all of your tooling needs to know that the actual service runs in Docker) and simultaneously isn't what you'd run in development (you want the image to be self-contained and not to need to copy the application separately from the image).
In your compose file, you set a volume which is going to replace your local environment with the one in your container even after npm run build
volumes:
- ./app:/app
You can either build it in your local or remove volumes
We had a similar issue with a nestjs build. Lately we noticed, that we had excluded the src file in the .dockerignore.
Issue is not with docker file. It issue with your dependency. please check package.json file in root folder.
Related
I'm new in docker and I want to setting-up a docker-compose for my django app. in the backend of my app, I have golang packages too and run that in djang with subprocess library.
But, when I want to install a package using go install github.com/x/y#latest and then copy its binary to the project directory, it gives me the error: package github.com/x/y#latest: cannot use path#version syntax in GOPATH mode
I searched a lot in the internet but didn't find a solution to solve my problem. Could you please tell me where I'm wrong?
here is my Dockerfile:
FROM golang:1.18.1-bullseye as go-build
# Install go package
RUN go install github.com/hakluke/hakrawler#latest \
&& cp $GOPATH/bin/hakrawler /usr/local/bin/
# Install main image for backend
FROM python:3.8.11-bullseye
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Install Dist packages
RUN apt-get update \
&& apt-get -y install --no-install-recommends software-properties-common libpq5 python3-dev musl-dev git netcat-traditional golang \
&& rm -rf /var/lib/apt/lists/
# Set work directory
WORKDIR /usr/src/redteam_toolkit/
# Install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# Copy project, and then the go package
COPY . .
COPY --from=go-build /usr/local/bin/hakrawler /usr/src/redteam_toolkit/toolkit/scripts/webapp/
docker-compose.yml:
version: '3.3'
services:
webapp:
build: .
command: python manage.py runserver 0.0.0.0:4334
container_name: toolkit_webapp
volumes:
- .:/usr/src/redteam_toolkit/
ports:
- 4334:4334
env_file:
- ./.env
depends_on:
- db
db:
image: postgres:13.4-bullseye
container_name: database
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=password
- POSTGRES_DB=redteam_toolkit_db
volumes:
postgres_data:
the get.py file inside /usr/src/redteam_toolkit/toolkit/scripts/webapp/ directory, to just run the go package, and list files in this dir:
import os
import subprocess
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
print(f"Current path is: {BASE_DIR}")
def go(target_url):
run_go_package = subprocess.getoutput(
f"echo {target_url} | {BASE_DIR}/webapp/hakrawler -t 15 -u"
)
list_files = subprocess.getoutput(f"ls {BASE_DIR}/webapp/")
print(run_go_package)
print(list_files)
go("https://example.org")
and then I just run:
$ docker-compose up -d --build
$ docker-compose exec webapp python toolkit/scripts/webapp/get.py
The output is:
Current path is: /usr/src/redteam_toolkit/toolkit/scripts
/bin/sh: 1: /usr/src/redteam_toolkit/toolkit/scripts/webap/hakrawler: not found
__init__.py
__pycache__
scr.py
gather.py
This looks like a really good candidate for a multi-stage build:
FROM golang:1.18.0 as go-build
# Install packages
RUN go install github.com/x/y#latest \
&& cp $GOPATH/bin/pacakge /usr/local/bin/
FROM python:3.8.11-bullseye as release
...
COPY --from=go-build /usr/local/bin/package /usr/src/toolkit/toolkit/scripts/webapp/
...
Your compose file also needs to be updated, it is masking the entire /usr/src/redteam_toolkit folder with the volume mount. Delete that volume mount to see the content of the image.
GOPATH mode does not work with Golang modules, in your Dockerfile file, add:
RUN unset GOPATH
use RUN go get <package_repository>
I'm facing an issue, am trying to run my go fiber project inside docker with air but am getting this error
uni-blog | /bin/sh: 1: /app/tmpmain.exe: not found
am using
Windows 11
Docker desktop
golang latest
air 1.27.10
fiber latest
Here is my docker compose and dockerfile
# docker-compose up -d --build
version: "3.8"
services:
app:
container_name: uni-blog
image: app-dev
build:
context: .
target: development
volumes:
- ./:/app
ports:
- 3000:3000
FROM golang:1.17 as development
RUN apt update && apt upgrade -y && \
apt install -y git \
make openssh-client
RUN curl -fLo install.sh https://raw.githubusercontent.com/cosmtrek/air/master/install.sh \
&& chmod +x install.sh && sh install.sh && cp ./bin/air /bin/air
RUN air -v
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
EXPOSE 3000
CMD air
I also tried installing air following the readME instructions still it gives me this error
Please help
Thanks in advance
The volumes: mount you have replaces the /app directory in the image with content from the host. If the binary is built in the Dockerfile, that volumes: mount hides it; if you don't have a matching compatible binary on the host in the same place, you'll get an error like what you see.
I'd remove that volumes: block so you're actually running the binary that's built into the image. The docker-compose.yml file can be reduced to as little as:
version: '3.8'
services:
app:
build: .
ports:
- '3000:3000'
If you look the error, you ca notice there is a typo between tmp/main.exe:
/bin/sh: 1: /app/tmpmain.exe: not found
This is coming from .air.toml config file:
bin = "tmp\\main.exe"
Create .air.toml file in project root like so:
root = "."
tmp_dir = "tmp"
[build]
# Build binary.
cmd = "go build -o ./tmp/main.exe ."
# Read binary.
bin = "tmp/main.exe"
# Watch changes in those files
include_ext = [ "go", "yml"]
# Ignore changes in these files
exclude_dir = ["tmp"]
# Stop builds from triggering too fast
delay = 1000 # ms
[misc]
clean_on_exit = true
I'm trying to copy my ./dist after building my angular app.
here is my Dockerfile
# Create image based off of the official Node 10 image
FROM node:12-alpine
RUN apk update && apk add --no-cache make git
RUN mkdir -p /home/project/frontend
# Change directory so that our commands run inside this new directory
WORKDIR /home/project/frontend
# Copy dependency definitions
COPY package*.json ./
RUN npm cache verify
## installing packages
RUN npm install
COPY ./ ./
RUN npm run build --output-path=./dist
COPY /dist /var/www/front
but when I run docker-compose build dashboard I get this error
Service 'dashboard' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builderxxx/dist: no such file or directory
I don't know why is there something wrong?
if you need to check also docker-compose file
...
dashboard:
container_name: dashboard
build: ./frontend
image: dashboard
container_name: dashboard
restart: unless-stopped
networks:
- app-network
...
The Dockerfile COPY directive copies content from the build context (the host-system directory in the build: line) into the image. If you're just trying to move around content within the image, you can RUN cp or RUN mv to use the ordinary Linux shell commands instead.
RUN npm run build --output-path=./dist \
&& cp -a dist /var/www/front
I have a spring boot application I want to test via .gitlab-ci.yml.
It's set up already like this:
image: openjdk:12
# services:
# - docker:dind
stages:
- build
before_script:
# - apk add --update python py-pip python-dev && pip install docker-compose
# - docker version
# - docker-compose version
- chmod +x mvnw
build:
stage: build
script:
# - docker-compose up -d
- ./mvnw package
artifacts:
paths:
- target/rest-SNAPSHOT.jar
The commented out portions are from the answer to Run docker-compose build in .gitlab-ci.yml which I noticed has a fully distinct docker image.
Obviously I need java installed to run my spring boot application, so does that mean docker is just not an option?
I would like to configure a CI such as TravisCI to build my application from Docker. My application has two part: Javascript and Python.
I thought to use docker-compose to do this:
version: '3'
services:
node:
image: node:12.8.0-buster
volumes:
- .:/srv
python:
image: python:3.7.4-buster
volumes:
- .:/src
I would like to have a Makefile such as:
all: foo bar
foo:
docker-compose exec node /bin/bash -c ' \
cd /workdir; \
npm install; \
npm run build'
bar:
docker-compose exec python /bin/bash -c ' \
cd /workdir; \
pip install sphinx; \
make html'
Is this correct to use docker compose like this? And what should I change to make it work?
docker compose not only support container run, but also image build, see this.
So, for your scenario, you should add your package build in Dockerfile and exeucte it with docker-compose up -d --build which will first build out a docker image then start the service base on the new docker image.
A simple fake code is as next, note next is just to explain the main idea, not a fully workable example, you need to add your stuff base on your real situation.
Dockerfile.node:
FROM node:12.8.0-buster
# Add related to build
ADD . /srv
# Add all package install
RUN cd /workdir && npm install && npm run build
# Others
......
Dockerfile.python:
FROM python:3.7.4-buster
# Add related to build
ADD . /srv
# Add all package install
RUN cd /workdir && pip install sphinx && make html
# Others
......
docker-compose.yaml:
version: '3'
services:
node:
build:
context: .
dockerfile: Dockerfile.node
volumes:
- .:/srv
python:
build:
context: .
dockerfile: Dockerfile.python
volumes:
- .:/src