Dockerfile Build Doesn't Work But Works with Compose - docker

I have a Dockerfile that won't build, but will work with docker-compose, I need it to work though with docker build command.
###############
# CACHE IMAGE #
###############
ARG GO_IMAGE=golang:1.17.3-alpine3.14
ARG BASE_IMAGE=alpine:3.14.2
FROM ${GO_IMAGE} AS cache
# Add the keys
ARG GITHUB_ID
ENV GITHUB_ID=$GITHUB_ID
ARG GITHUB_TOKEN
ENV GITHUB_TOKEN=$GITHUB_TOKEN
# Install Git
RUN apk add git
# TODO: ENCRYPT THE GITHUB_ID AND GITHUB_TOKEN
# Make Git Configuration
RUN git config \
--global \
url."https://${GITHUB_ID}:${GITHUB_TOKEN}#github.com/".insteadOf \
"https://github.com/"
WORKDIR /bin
COPY go.mod go.sum bin/
RUN go mod download
##############
# BASE IMAGE #
##############
FROM cache AS dataeng_github_metrics
COPY . /bin
WORKDIR /bin
# Setup Git Terminal Prompt & Go Build
RUN go build .
###############
# FINAL IMAGE #
###############
FROM ${BASE_IMAGE}
COPY --from=dataeng_github_metrics /bin/dataeng_github_metrics bin/
ENTRYPOINT [ "bin/dataeng_github_metrics" ]
The directory looks like this:
rics   Docker-Publish-Workflows-To-GHCR ●  tree .  1 ↵  8955  20:37:41
.
├── Dockerfile
├── Makefile
├── README.md
├── dataeng_github_metrics
├── go.mod
├── go.sum
├── infra
│ ├── README.md
│ ├── k8s
│ │ ├── README.md
│ │ ├── configmaps
│ │ │ ├── README.md
│ │ │ └── teams-payload-configmap.yaml
│ │ ├── cronworkflow
│ │ │ ├── README.md
│ │ │ └── argo_cron_workflow.yaml
│ │ ├── deployment
│ │ │ ├── README.md
│ │ │ ├── git-hub-contributions-deployment.yaml
│ │ │ └── postgres-deployment.yaml
│ │ └── volumeclaims
│ │ ├── README.md
│ │ ├── git-hub-contributions-claim0-persistentvolumeclaim.yaml
│ │ ├── git-hub-contributions-claim1-persistentvolumeclaim.yaml
│ │ └── git-hub-contributions-claim2-persistentvolumeclaim.yaml
│ ├── payloads
│ │ ├── README.md
│ │ ├── dataeng_github_metrics.csv
│ │ └── teams.yaml
│ └── terraform
│ ├── README.md
│ └── manifests
│ └── README.md
├── local
│ ├── README.md
│ ├── dependencies
│ │ └── wait-for-postgres.sh
│ ├── docker-compose.yaml
│ └── images
│ ├── ER_Diagram.png
│ ├── print_execution.png
│ └── print_query.png
└── main.go
What's weird to me is it fails at the COPY step for go.mod and go.sum, and I haven't a clue why it's not copying over the files:
Command to build Dockerfile in working directory:
docker build - < Dockerfile
[+] Building 0.7s (12/17)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 928B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:3.14.2 0.4s
=> [internal] load metadata for docker.io/library/golang:1.17.3-alpine3.14 0.0s
=> [cache 1/7] FROM docker.io/library/golang:1.17.3-alpine3.14 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 2B 0.0s
=> [stage-2 1/2] FROM docker.io/library/alpine:3.14.2#sha256:e1c082e3d3c45cccac829840a2594 0.0s
=> CACHED [cache 2/7] RUN apk add git 0.0s
=> CACHED [cache 3/7] RUN git config --global url."https://:#github.com/".insteadO 0.0s
=> CACHED [cache 4/7] WORKDIR /bin 0.0s
=> [cache 5/7] RUN pwd 0.2s
=> ERROR [cache 6/7] COPY go.mod go.sum ./
Why is it not letting me copy a file into my WORKDIR with docker build, but when I use docker-compose it works just fine.

Try running the Docker build like so:
docker build .
The build with - doesn't work as expected because no context is given. See Docker build docs:
This will read a Dockerfile from STDIN without context. Due to the lack of a context, no contents of any local directory will be sent to the Docker daemon. Since there is no context, a Dockerfile ADD only works if it refers to a remote URL.

Related

Copy all files of sub and nested sub directories

This is my project file structure:
java-project/
├── docker.compose.yml
├── pom.xml
└── services/
├── a/
│ ├── Dockerfile
│ ├── pom.xml
│ ├── src/
│ │ ├── pom.xml
│ │ ├── xxx
│ │ └── xxx
│ └── target/
│ ├── pom.xml
│ └── xxxx
└── b/
├── Dockerfile
├── pom.xml
├── src/
│ ├── pom.xml
│ ├── xxx
│ └── xxx
└── target/
├── pom.xml
└── xxxx
I want to copy all of the contents of the services folder of the project (including all the subfolders inside the services). Basically, I want to replicate the current project structure with every file and folder in the docker image as well for the mvn build to execute successfully.
I am doing the following in the Dockerfile, but I don't see all of the contents:
COPY services/**/pom.xml ./services/
What am I doing wrong here? TIA
Let's look at your COPY instruction:
# <src> <dest>
COPY services/**/pom.xml ./services/
Under the hood, Docker reads the <src> using Go's filepath.Match method. This means that the instruction doesn't use the globstar (**) the way glob patterns do. However, your question suggests you want to copy everything inside services — not only pom.xml files.
You can copy everything inside your local services directory using:
COPY services ./services/
If you want to exclude certain subdirectories or files, you can specify this using a .dockerignore.

Docker mosquitto - Error unable to load auth plugin

I really need your help !
I'm encountering a problem with the loading of a plugin in a docker mosquitto.
I tried to load it on a local version of mosquitto and it worked well.
The error return in the docker console is:
dev_instance_mosquitto_1 exited with code 13
The errors return in the log file of mosquitto are:
1626352342: Loading plugin: /mosquitto/config/mosquitto_message_timestamp.so
1626352342: Error: Unable to load auth plugin "/mosquitto/config/mosquitto_message_timestamp.so".
1626352342: Load error: Error relocating /mosquitto/config/mosquitto_message_timestamp.so: __sprintf_chk: symbol not found
Here is a tree output of the project:
mosquitto/
├── Dockerfile
├── config
│ ├── acl
│ ├── ca_certificates
│ │ ├── README
│ │ ├── broker_CA.crt
│ │ ├── mqtt.test.perax.com.p12
│ │ ├── private_key.key
│ │ └── server_ca.crt
│ ├── certs
│ │ ├── CA_broker_mqtt.crt
│ │ ├── README
│ │ ├── serveur_broker.crt
│ │ └── serveur_broker.key
│ ├── conf.d
│ │ └── default.conf
│ ├── mosquitto.conf
│ ├── mosquitto_message_timestamp.so
│ └── pwfile
├── data
│ └── mosquitto.db
└── log
└── mosquitto.log
Here is the Dockerfile:
FROM eclipse-mosquitto
COPY config/ /mosquitto/config
COPY config/mosquitto_message_timestamp.so /usr/lib/mosquitto_message_timestamp.so
RUN install /usr/lib/mosquitto_message_timestamp.so /mosquitto/config/
here is the docker-compose.yml:
mosquitto:
restart: always
build: ./mosquitto/
image: "eclipse-mosquitto/latests"
ports:
- "1883:1883"
- "9001:9001"
volumes:
- ./mosquitto/config/:/mosquitto/config/
- ./mosquitto/data/:/mosquitto/data/
- ./mosquitto/log/mosquitto.log:/mosquitto/log/mosquitto.log
user: 1883:1883
environment:
- PUID=1883
- PGID=1883
Here is the mosquitto.conf:
persistence true
persistence_location /mosquitto/data
log_dest file /mosquitto/log/mosquitto.log
include_dir /mosquitto/config/conf.d
plugin /mosquitto/config/mosquitto_message_timestamp.so
I'm using mosquitto 2.0.10 on a ubuntu serveur with the version 18.04.5 LTS.
In thanking you for your help.
Your best bet here is probably to set up a multi step Docker build file that uses an Alpine based image to build the plugin then copy it into the eclipse-mosquitto image.

Jumping around of the dockerfile/docker-compose context

My current projects structure looks something like that:
/home/some/project
│ .credentials_dev
│ .credentials_prod
│ ca.cer
│ docker-compose.yml
│ Dockerfile
│ init.sql
│ nginx_dev.conf
│ nginx_prod.conf
│
├───src
└───users
Compose makes four containers, two of them are dev & prod version of application, who uses appropriate prod & dev files. As you can see, following structure root is little overloaded, so i'd like to move all the deployment staff into the separate directory to make the following:
/home/some/project
├───deployment
│ .credentials_dev
│ .credentials_prod
│ ca.cer
│ docker-compose.yml
│ Dockerfile
│ init.sql
│ nginx_dev.conf
│ nginx_prod.conf
│
├───src
└───users
The idea is to receieve in the result following structure on the docker host:
host
├───dev
│ ├───src
│ └───users
├───prod
│ ├───src
│ └───users
└───project
├───deployment
│ .credentials_dev
│ .credentials_prod
│ ca.cer
│ docker-compose.yml
│ Dockerfile
│ init.sql
│ nginx_dev.conf
│ nginx_prod.conf
│
├───src
└───users
and two containers app_dev and app_prod, which volumes are appropriate mounted into folders /host/dev and /host/prod.
I tried multiple solutions found here, but all of them in different variations returned the following errors:
ERROR: Service 'app_dev' failed to build: COPY failed: stat /var/lib/docker/tmp/docker-builder264200969/ca.cer: no such file or directory
ERROR: Service 'app_dev' failed to build: COPY failed: Forbidden path outside the build context: ../ca.cer ()
Error is always appears while docker-compose is trying to build an image, on that string:
COPY deployment/ca.cer /code/
Please tell me how to achieve the desired result.
The Deployment folder is outside of the build context. Docker will pass all the files inside the deployment file as the build context. However the deployment folder itself is outside of it.
Change your copy statement to be instead :
COPY ./ca.cer /code/
Since in the image you are already in that folder.

Docker isn't mounting the directory? "OCI runtime create failed: container_linux.go:346: no such file or directory: unknown"

On my Windows 10 Home computer with Docker Toolbox, Docker is having trouble mounting the drives. I've already run dos2unix on the entrypoint.sh file.
The full error is as such:
ERROR: for users Cannot start service users: OCI runtime create failed: container_linux.go:346: starting container process caused "exec: \"/usr/src/app/entrypoint.sh\": stat /usr/src/app/entrypoint.sh: no such file or directory": unknown
My docker-compose.yml:
version: '3.7'
services:
users:
build:
context: ./services/users
dockerfile: Dockerfile
entrypoint: ['/usr/src/app/entrypoint.sh']
volumes:
- './services/users:/usr/src/app'
ports:
- 5001:5000
environment:
- FLASK_ENV=development
- APP_SETTINGS=project.config.DevelopmentConfig
- DATABASE_URL=postgresql://postgres:postgres#users-db:5432/users_dev
- DATABASE_TEST_URL=postgresql://postgres:postgres#users-db:5432/users_test
depends_on:
- users-db
Curiously, when I comment out the "volumes" section, it works! But I want to be able to mount volumes in the future.
Directory structure can be seen as such:
D:\flask-react-auth
│ .gitignore
│ .gitlab-ci.yml
│ docker-compose.yml
│ README.md
│ release.sh
│
└───services
│
└───users
│ .coveragerc
│ .dockerignore
│ Dockerfile
│ Dockerfile.prod
│ entrypoint.sh
│ manage.py
│ requirements-dev.txt
│ requirements.txt
│ setup.cfg
│ tree.txt
│
└───project
│ config.py
│ __init__.py
│
├───api
│ │ ping.py
│ │ __init__.py
│ │
│ └───users
│ admin.py
│ crud.py
│ models.py
│ views.py
│ __init__.py
│
├───db
│ create.sql
│ Dockerfile
│
└───tests
conftest.py
pytest.ini
test_admin.py
test_config.py
test_ping.py
test_users.py
test_users_unit.py
__init__.py
I have added the D:\flask-react-auth\ to the 'Shared Folders' on virtualbox as well.
The answer seems obvious to me:
When you run the code as is
* it mounts the current working directory to '/usr/src/app'.
* The current working directory does not have a file 'entrypoint.sh'.
* It tries to run '/usr/src/app/entrypoint.sh' but it is not there so it fails.
When you comment out that volume mount
* I assume the image already has '/usr/src/app/entrypoint.sh' so it just works.
I think you probably should change the mounting code from
volumes:
- '.:/usr/src/app'
to
volumes:
- './services/users:/usr/src/app'

How to dockerzise beego application

I tried to Dockerize a Beego application, but the HTML rendering is not finding HTML files stored inside the view/templates directory.
FROM golang:1.13
WORKDIR /go/src/fileUpload
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
EXPOSE 8080
# Install server application
CMD ["go", "run", "./main/main.go"]
You could try to set the directory containing the templates inside the Docker image.
beego.BConfig.WebConfig.ViewsPath = "myviewpath"
https://beego.me/docs/mvc/view/view.md#template-directory
Edit: directory structure
It is difficult to answer the question, as the directory layout is not clear. However, I can give an example based on quickstart:
export GOPATH="$HOME/go/src"
bee new quickstart
In $GOPATH/src/quickstart/Dockerfile:
FROM golang:1.13
WORKDIR /go/src/quickstart
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
EXPOSE 8080
# Install server application
CMD ["go", "run", "main.go"]
Note that I do not have a directory (./main) in front of main.go. This is what the structure of the app looks like:
tim#sky:~/go/src/quickstart$ tree
.
├── conf
│ └── app.conf
├── controllers
│ └── default.go
├── Dockerfile
├── main.go
├── models
├── routers
│ └── router.go
├── static
│ ├── css
│ ├── img
│ └── js
│ └── reload.min.js
├── tests
│ └── default_test.go
└── views
└── index.tpl
If the views directory in you app is in a different place, you need to add the correct path to main.go as described in my initial answer.

Resources