I have this Dockerfile
FROM node:15.11.0-alpine
#ENVIRONNEMENT
ENV GLIB_PACKAGE_BASE_URL https://github.com/sgerrand/alpine-pkg-glibc/releases/download
ENV GLIB_VERSION 2.25-r0
ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk
ENV GRADLE_HOME /usr/local/gradle
ENV GRADLE_VERSION 4.4
ENV ANDROID_HOME /usr/local/android-sdk-linux
ENV ANDRDOID_TOOLS_VERSION r25.2.5
ENV ANDROID_API_LEVELS android-26
ENV ANDROID_BUILD_TOOLS_VERSION 26.0.2
ENV IONIC_VERSION 5
ENV PATH ${GRADLE_HOME}/bin:${JAVA_HOME}/bin:${ANDROID_HOME}/tools:$ANDROID_HOME/platform-tools:$PATH
# INSTALL JAVA
RUN apk update ...
# INSTALL IONIC AND CORDOVA
RUN npm install -g cordova ionic#${IONIC_VERSION}
#INSTALL Graddle
RUN mkdir -p ${GRADLE_HOME} ...
# INSTALL ANDROID
RUN mkdir -p ${ANDROID_HOME} ...
# INSTALL GLIBC
RUN curl -L ...
# CONFIGURATION
RUN echo y | android update sdk --no-ui -a --filter platform-tools,${ANDROID_API_LEVELS},build-tools-${ANDROID_BUILD_TOOLS_VERSION}
# Make license agreement
RUN mkdir $ANDROID_HOME/licenses ...
#FILES DELETION
RUN rm -rf /tmp/* /var/cache/apk/*
WORKDIR /usr/app
RUN npm install
COPY ./ /usr/app
I am building it with CMD:
docker build -t <image-name> .
I have this docker-compose.yml file:
version: '3.6'
services:
app:
container_name: karma5_ionic
build:
context: .
dockerfile: Dockerfile
ports:
- '8100:8100'
- '35729:35729'
command: ionic serve --external
I run the following command:
sudo docker-compose up -d
The application is displaying fine in the browser in localhost:8100
Problem:
When I make changes, there is no hot reload.
localhost:35729 might be temporarily down or it may have moved permanently to a new web address.
Or localhost:35729 the connections was reset
The only way that i can see changes is if I
run docker-compose build
docker-compose up -d again
To whom it may concern:
I just needed to add
volumes:
- "./:/usr/app"
Whole docker-compose.yml file:
version: '3.6'
services:
app:
container_name: karma5_ionic
build:
context: .
dockerfile: Dockerfile
volumes:
- "./:/usr/app"
ports:
- '8100:8100'
- '35729:35729'
command: ionic serve --external
Related
I'm new in docker and I want to setting-up a docker-compose for my django app. in the backend of my app, I have golang packages too and run that in djang with subprocess library.
But, when I want to install a package using go install github.com/x/y#latest and then copy its binary to the project directory, it gives me the error: package github.com/x/y#latest: cannot use path#version syntax in GOPATH mode
I searched a lot in the internet but didn't find a solution to solve my problem. Could you please tell me where I'm wrong?
here is my Dockerfile:
FROM golang:1.18.1-bullseye as go-build
# Install go package
RUN go install github.com/hakluke/hakrawler#latest \
&& cp $GOPATH/bin/hakrawler /usr/local/bin/
# Install main image for backend
FROM python:3.8.11-bullseye
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Install Dist packages
RUN apt-get update \
&& apt-get -y install --no-install-recommends software-properties-common libpq5 python3-dev musl-dev git netcat-traditional golang \
&& rm -rf /var/lib/apt/lists/
# Set work directory
WORKDIR /usr/src/redteam_toolkit/
# Install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# Copy project, and then the go package
COPY . .
COPY --from=go-build /usr/local/bin/hakrawler /usr/src/redteam_toolkit/toolkit/scripts/webapp/
docker-compose.yml:
version: '3.3'
services:
webapp:
build: .
command: python manage.py runserver 0.0.0.0:4334
container_name: toolkit_webapp
volumes:
- .:/usr/src/redteam_toolkit/
ports:
- 4334:4334
env_file:
- ./.env
depends_on:
- db
db:
image: postgres:13.4-bullseye
container_name: database
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=password
- POSTGRES_DB=redteam_toolkit_db
volumes:
postgres_data:
the get.py file inside /usr/src/redteam_toolkit/toolkit/scripts/webapp/ directory, to just run the go package, and list files in this dir:
import os
import subprocess
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
print(f"Current path is: {BASE_DIR}")
def go(target_url):
run_go_package = subprocess.getoutput(
f"echo {target_url} | {BASE_DIR}/webapp/hakrawler -t 15 -u"
)
list_files = subprocess.getoutput(f"ls {BASE_DIR}/webapp/")
print(run_go_package)
print(list_files)
go("https://example.org")
and then I just run:
$ docker-compose up -d --build
$ docker-compose exec webapp python toolkit/scripts/webapp/get.py
The output is:
Current path is: /usr/src/redteam_toolkit/toolkit/scripts
/bin/sh: 1: /usr/src/redteam_toolkit/toolkit/scripts/webap/hakrawler: not found
__init__.py
__pycache__
scr.py
gather.py
This looks like a really good candidate for a multi-stage build:
FROM golang:1.18.0 as go-build
# Install packages
RUN go install github.com/x/y#latest \
&& cp $GOPATH/bin/pacakge /usr/local/bin/
FROM python:3.8.11-bullseye as release
...
COPY --from=go-build /usr/local/bin/package /usr/src/toolkit/toolkit/scripts/webapp/
...
Your compose file also needs to be updated, it is masking the entire /usr/src/redteam_toolkit folder with the volume mount. Delete that volume mount to see the content of the image.
GOPATH mode does not work with Golang modules, in your Dockerfile file, add:
RUN unset GOPATH
use RUN go get <package_repository>
I would like to configure a CI such as TravisCI to build my application from Docker. My application has two part: Javascript and Python.
I thought to use docker-compose to do this:
version: '3'
services:
node:
image: node:12.8.0-buster
volumes:
- .:/srv
python:
image: python:3.7.4-buster
volumes:
- .:/src
I would like to have a Makefile such as:
all: foo bar
foo:
docker-compose exec node /bin/bash -c ' \
cd /workdir; \
npm install; \
npm run build'
bar:
docker-compose exec python /bin/bash -c ' \
cd /workdir; \
pip install sphinx; \
make html'
Is this correct to use docker compose like this? And what should I change to make it work?
docker compose not only support container run, but also image build, see this.
So, for your scenario, you should add your package build in Dockerfile and exeucte it with docker-compose up -d --build which will first build out a docker image then start the service base on the new docker image.
A simple fake code is as next, note next is just to explain the main idea, not a fully workable example, you need to add your stuff base on your real situation.
Dockerfile.node:
FROM node:12.8.0-buster
# Add related to build
ADD . /srv
# Add all package install
RUN cd /workdir && npm install && npm run build
# Others
......
Dockerfile.python:
FROM python:3.7.4-buster
# Add related to build
ADD . /srv
# Add all package install
RUN cd /workdir && pip install sphinx && make html
# Others
......
docker-compose.yaml:
version: '3'
services:
node:
build:
context: .
dockerfile: Dockerfile.node
volumes:
- .:/srv
python:
build:
context: .
dockerfile: Dockerfile.python
volumes:
- .:/src
My Dockerfile:
FROM node:10 AS builder
RUN npm install multi-file-swagger -g
WORKDIR /usr/src/app
COPY swagger/* ./
ARG API_HOST
ENV APP_HOST=$API_HOST
RUN sed -i 's+replace_host+'"$API_HOST"'+g' index.yaml
RUN multi-file-swagger index.yaml > index.json
FROM golang:1.12
WORKDIR /go/src/app
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
VOLUME /go/src/app
EXPOSE 8080
COPY --from=builder /usr/src/app/ swagger/
CMD ["app"]
My docker-compose.yml file:
version: '3.4'
services:
myapp:
build:
context: .
args:
- API_HOST=api.my-real-domain.com
volumes:
- ./:/go/src/app
ports:
- "8080:8080"
If run docker build only:
docker build -t myapp . --build-arg API_HOST=api.my-real-domain.com
It can run the commands:
RUN sed -i 's+replace_host+'"$API_HOST"'+g' index.yaml
RUN multi-file-swagger index.yaml > index.json
And when lunch the container, I can find the index.json exists.
But if use docker-compose build and docker-compose up, then check the index.json in container, can't find it.
you overwrite all your files in app folder by using:
volumes:
- ./:/go/src/app
you need to remove the volume section from your compose
I have a Dockerfile that contains steps that create a directory and runs an angular build script outputing to that directory. This all seems to run correctly. However when the container runs, the built files and directory are not there.
If I run a shell in the image:
docker run -it pnb_web sh
# cd /code/static
# ls
assets favicon.ico index.html main.js main.js.map polyfills.js polyfills.js.map runtime.js runtime.js.map styles.js styles.js.map vendor.js vendor.js.map
If I exec a shell in the container:
docker exec -it ea23c7d30333 sh
# cd /code/static
sh: 1: cd: can't cd to /code/static
# cd /code
# ls
Dockerfile api docker-compose.yml frontend manage.py mysite.log pnb profiles requirements.txt settings.ini web_variables.env
david#lightning:~/Projects/pnb$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ea23c7d30333 pnb_web "python3 manage.py r…" 13 seconds ago Up 13 seconds 0.0.0.0:8000->8000/tcp pnb_web_1_267d3a69ec52
This is my dockerfile:
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash -
RUN apt install nodejs
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
RUN mkdir /code/static
WORKDIR /code/frontend
RUN npm install -g #angular/cli
RUN npm install
RUN ng build --outputPath=/code/static
and associated docker-compose:
version: '3'
services:
db:
image: postgres
web:
build:
context: .
dockerfile: Dockerfile
working_dir: /code
env_file:
- web_variables.env
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
In the second example, the static directory has never been created or built into. I thought that a container is an instance of an image. How can the container be missing files from the image?
You're confusing build-time and run-time, along playing with Volumes.
Remember that host mount has priority over FS provided by the running container, so even your built image has assets, they are going to be overwritten by .services.web.volumes because you're mounting the host filesystem that overwrites the build result.
If you try to avoid volumes mounting you'll notice that everything is working as expected.
I added RUN go get to install packages during "docker-compose". However, the following cannot find package error was occurred when I run go build. I found that the packages are saved in /go/pkg/linux_amd64/.
run docker-compose and go build
$ docker-compose up -d
$ docker exec -it explorer-cli /bin/bash
# pwd
/go
# ls
bin pkg src
# echo $GOPATH
/go
# ls /go/pkg/linux_amd64/github.com/
go-sql-driver
# go build -i -o /go/bin/explorer-cli src/main.go
src/main.go:6:2: cannot find package "github.com/go-sql-driver/mysql" in any of:
/usr/local/go/src/github.com/go-sql-driver/mysql (from $GOROOT)
/go/src/github.com/go-sql-driver/mysql (from $GOPATH)
(it worked if I run "go get" manually)
# go get github.com/go-sql-driver/mysql
# ls src/
github.com main.go
# go build -i -o /go/bin/explorer-cli src/main.go
docker-compose.yml
version: '3.4'
services:
mysql:
image: mysql:latest
container_name: database
volumes:
- ./docker/:/etc/mysql/conf.d
- ./docker/:/docker-entrypoint-initdb.d
environment:
- MYSQL_RANDOM_ROOT_PASSWORD=true
- MYSQL_DATABASE=explorer
- MYSQL_USER=admin
- MYSQL_PASSWORD=12dlql*41
app:
build: .
tty: true
image: explorer-cli:latest
container_name: explorer-cli
volumes:
- ./src:/go/src
external_links:
- database
Dockerfile
FROM golang:latest
RUN apt-get update
RUN apt-get upgrade -y
ENV GOBIN /go/bin
RUN go get github.com/go-sql-driver/mysql
main.go
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
)
func main() {
db, err := sql.Open("mysql", "XUSER:XXXX#(database:3306)/explorer")
if err != nil {
panic(err.Error())
}
defer db.Close()
}
Update 1
I noticed big differences between the following directories.
# ls /go/pkg/linux_amd64/github.com/go-sql-driver/
mysql.a
# ls /go/src/github.com/go-sql-driver/mysql/
AUTHORS connection_go18_test.go packets.go
CHANGELOG.md connection_test.go packets_test.go
CONTRIBUTING.md const.go result.go
LICENSE driver.go rows.go
README.md driver_go18_test.go statement.go
appengine.go driver_test.go statement_test.go
benchmark_go18_test.go dsn.go transaction.go
benchmark_test.go dsn_test.go utils.go
buffer.go errors.go utils_go17.go
collations.go errors_test.go utils_go18.go
connection.go fields.go utils_go18_test.go
connection_go18.go infile.go utils_test.go
Update 2
As #aerokite said, the "volumes" was overwriting the downloaded packages. I changed like the followings and it worked.
Dockerfile
version: '3.4'
FROM golang:latest
RUN apt-get update
RUN apt-get upgrade -y
ENV GOBIN /go/bin
RUN go get github.com/go-sql-driver/mysql
RUN mkdir /go/src/explorer-cli
docker-compose
services:
mysql:
image: mysql:latest
container_name: database
volumes:
- ./docker/:/etc/mysql/conf.d
- ./docker/:/docker-entrypoint-initdb.d
environment:
- MYSQL_RANDOM_ROOT_PASSWORD=true
- MYSQL_DATABASE=explorer
- MYSQL_USER=XUSER
- MYSQL_PASSWORD=XXXX
app:
build: .
tty: true
image: explorer-cli:latest
container_name: explorer-cli
volumes:
- ./src/explorer-cli:/go/src/explorer-cli
external_links:
- database
go build
go build -i -o /go/bin/explorer-cli src/explorer-cli/main.go
I have tried to recreate your problem.
FROM golang:latest
RUN apt-get update
RUN apt-get upgrade -y
ENV GOBIN /go/bin
RUN go get github.com/go-sql-driver/mysql
You have provided this Dockerfile. I have build it
$ docker build -t test .
Now I exec into this image to run your go build command.
$ docker run -it test bash
Then I have created main.go, you provided, in /go/src directory.
And finally, I have built successfully without any error
$ go build -i -o /go/bin/explorer-cli src/main.go
And I think I have found your problem. I have never used docker-compose. But you will understand.
Problem is here:
app:
build: .
tty: true
image: explorer-cli:latest
container_name: explorer-cli
volumes:
- ./src:/go/src <-- problem is here
external_links:
- database
You are mounting ./src into /go/src directory in your docker. This process is overwriting directory /go/src in your docker with your local ./src. And this is removing data you got from go get github.com/go-sql-driver/mysql
Do you understand?
But when you are running go get github.com/go-sql-driver/mysql, its now getting data again.
Solution (01):
Mount your local volume into somewhere else.
volumes:
- ./src:/tmp/src
And modify your Dockerfile to move this main.go to /go/src
Solution (02):
Copy main.go into your docker. Add this line in Dockerfile
COPY ./src/main.go /go/src