golang docker build for linux - docker

I need to compile a Golang application for Linux and I can't cross-compile under Mac, because of another library. So I decided to compile within a Docker container. This is my first time to use Docker.
This is my current directory structure:
.
├── Dockerfile
├── Gopkg.lock
├── Gopkg.toml
├── Vagrantfile
├── bootstrap.sh
├── src
│   ├── cmd
│   │   ├── build.bat
│   │   ├── build.sh
│   │   ├── config.json
│   │   ├── readme.md
│   │   └── server.go
│   ├── consumers.go
│   ├── endpoints
│   │   ├── json.go
│   │   ├── rate.go
│   │   ├── test_payment.go
│   │   └── wallet.go
│   ├── middleware
│   │   └── acl.go
│   ├── models.go
│   ├── network
│   │   └── network.go
│   ├── qr
│   │   └── qr.go
│   ├── router
│   │   └── router.go
│   ├── service
│   │   └── walletService.go
│   ├── services.go
│   ├── setup.sql
│   ├── store
│   │   └── wallet.go
│   ├── stores.go
│   └── wallet
│   ├── coin.go
│   └── ethereum.go
Dockerfile:
FROM golang:latest
WORKDIR /src/cmd
RUN ls
RUN go get github.com/go-sql-driver/mysql
RUN go build -o main ./src/cmd/server.go
CMD ["./main"]
I try to build the Docker image with:
docker build -t outyet .
This is the error it returns:
Sending build context to Docker daemon 5.505MB
Step 1/6 : FROM golang:latest
---> d0e7a411e3da
Step 2/6 : WORKDIR /src/cmd
---> Using cache
---> 0c4c2b99e294
Step 3/6 : run ls
---> Using cache
---> 23d3e491a2e1
Step 4/6 : RUN go get github.com/go-sql-driver/mysql
---> Running in f34447e51f6c
Removing intermediate container f34447e51f6c
---> 5731ab22ee43
Step 5/6 : RUN go build -o main server.go
---> Running in ecc48fcf5488
stat server.go: no such file or directory
The command '/bin/sh -c go build -o main server.go' returned a non-zero code: 1
How i can build my Golang application with docker?

The error you're seeing is:
stat server.go: no such file or directory
The command '/bin/sh -c go build -o main server.go' returned a non-zero code: 1
... and in other words, it's telling you that the Go compiler can't find server.go which you're trying to build. Update your Dockerfile to copy your local files into the Docker image:
FROM golang:latest
COPY . /go/src/workdir
WORKDIR /go/src/workdir
RUN go build -o main ./src/cmd/server.go
CMD ["/go/src/workdir/main"]
In your directory I spotted Gopkg.toml which is a dependency manifest used by dep. dep will use a directory called vendor to include all dependencies for your Go project. Before you build the Docker image, you need to ensure all dependencies are present with dep ensure:
$ dep ensure
$ docker build -t outyet .

You need to add your sources in your build container. Add this line in your Dockerfile (after FROM golang:latest for example) :
ADD src/ /src/cmd
Then, you could access files inside the container (/src/cmd) : RUN ls should now return something.

Related

Building a Docker with Multiple Proto - Proto File not found

I'm running with an issue where I can't build Dockerfile that includes multiple proto files(server and text). The server proto is within the Dockerfile dir, but the text proto is within the Dockerfile parent. So I'm building the Dockerfile in the parent dir to COPY the text proto to the Docker build.
The Docker build complaining about proto/text.proto: File not found. even though I COPY the proto/text.proto to the exact location as server/proto/server.proto.
Here are all my files:
DockerFile
FROM --platform=linux/x86_64 golang:1.19.3-bullseye
# Install grpc
RUN go install google.golang.org/grpc/cmd/protoc-gen-go-grpc#v1.2 && \
go install google.golang.org/protobuf/cmd/protoc-gen-go#v1.28
WORKDIR /app
COPY server/. /app
COPY proto/text.proto /app/proto/text.proto
# Install protoc and zip system library
RUN apt-get update && apt-get install -y zip && \
mkdir /opt/protoc && cd /opt/protoc && wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.0/protoc-3.7.0-linux-x86_64.zip && \
unzip protoc-3.7.0-linux-x86_64.zip
# Copy the grpc proto file and generate the go module
RUN /opt/protoc/bin/protoc --go_out=/app/proto --proto_path=/app/proto --go_opt=paths=source_relative --go-grpc_out=/app/proto --go-grpc_opt=paths=source_relative /app/proto/text.proto /app/proto/server.proto
EXPOSE 5051
RUN go build -o /server
ENTRYPOINT ["/server"]
Dir Tree
1.text
├── admin
│   ├── Dockerfile
│   ├── app.js
│   ├── package.json
│   └── web
│   ├── html
│   │   └── index.html
│   └── resources
├── compose.yaml
├── db
│   ├── Dockerfile
│   ├── main.go
│   ├── proto
│   │   ├── db.pb.go
│   │   ├── db.proto
│   │   └── db_grpc.pb.go
│   └── text.db
├── go.mod
├── go.sum
├── proto
│   ├── text.pb.go
│   └── text.proto
└── server
├── Dockerfile
├── main.go
├── proto
│   ├── server.pb.go
│   ├── server.proto
│   └── server_grpc.pb.go
└── text
├── text.go
└── text_test.go
I'm able to run the following protoc in the root text dir:
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative proto/text.proto db/proto/db.proto server/proto/server.proto
And run the server locally, but I'm not able to build my Docker:
CMD
docker build -f server/Dockerfile -t server .
Error
=> ERROR [7/8] RUN /opt/protoc/bin/protoc --go_out=/app/proto --proto_path=/app/proto --go_opt=paths=source_relative --go-grpc_out=/app/proto --go-grpc_opt=paths=source_relative 0.4s
------
> [7/8] RUN /opt/protoc/bin/protoc --go_out=/app/proto --proto_path=/app/proto --go_opt=paths=source_relative --go-grpc_out=/app/proto --go-grpc_opt=paths=source_relative /app/proto/text.proto /app/proto/server.proto:
#11 0.427 proto/text.proto: File not found.
#11 0.429 server.proto: Import "proto/text.proto" was not found or had errors.
#11 0.431 server.proto:25:5: "text.Status" seems to be defined in "text.proto", which is not imported by "server.proto". To use it here, please add the necessary import.
------
executor failed running [/bin/sh -c /opt/protoc/bin/protoc --go_out=/app/pro
text/server/proto
syntax="proto3";
package server;
import "proto/text.proto";
option go_package = "github.com/amb1s1/text/server/proto/server";
message SendMessageRequest {
string token = 1;
string phone = 2;
string message = 3;
bool dry_run = 4;
};
message SendMessageResponse {
text.Status status = 1;
};
service Text {
// SendMessage sents SMS message.
rpc SendMessage(SendMessageRequest) returns (SendMessageResponse) {}
}
text/proto/
syntax="proto3";
package text;
option go_package = "github.com/amb1s1/text/proto";
enum Status {
UNKNOW = 0;
OK = 1;
TOKENS_EXISTS = 2;
TOKEN_NOT_FOUND = 3;
FAILED_NOT_SENT= 4;
DRY_RUN_OK = 5;
ZERO_BALANCE = 6;
WRONG_TOKEN = 7;
}
As per the comments; within your docker image you have the directory structure:
/app/proto/server.proto
/app/proto/text.proto
server.proto imports text.proto with import "proto/text.proto".
This means that protoc will be looking for a file called proto/text.proto within the import path. You specified --proto_path=/app/proto as an argument to protoc meaning that protoc will check for /app/proto/proto/text.proto which does not exist (hence the issue). To fix this remove the --proto_path=/app/proto (so protoc uses the working folder) or specify --proto_path=/app.

COPY failed while using docker

Im building a express app but when i use the command sudo docker build - < Dockerfile i get the error COPY failed: file not found in build context or excluded by .dockerignore: stat package.json: file does not exist.
This is how my proyect structure looks like:
.
├── build
│   ├── server.js
│   └── server.js.map
├── Dockerfile
├── esbuild.js
├── package.json
├── package-lock.json
├── Readme.md
└── src
├── index.ts
├── navigate.ts
├── pages.ts
├── responses
│   ├── Errors.ts
│   └── index.ts
└── server.ts
And this is my Dockerfile content
FROM node:14.0.0
WORKDIR /usr/src/app
RUN ls -all
COPY [ "package.json", \
"./"]
COPY src/ ./src
RUN npm install
RUN node esbuild.js
RUN npx nodemon build/server.js
EXPOSE 3001
CMD ["npm", "run", "serve", ]
At the moment of run the command, im located in the root of the project.

Docker COPY no such file or directory

Building docker image fails on copy task. No such file or directory. I am using the hello world example from spring
Building from openjdk:8-jdk-alpine
Run echo ${PWD} prints /
Run ls prints a set of normal directories (/usr /var etc) but no project files are present
Why is docker not using the WORKING directory?
FROM openjdk:8-jdk-alpine
VOLUME /tmp
ARG DEPENDENCY=target/dependency
COPY ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY ${DEPENDENCY}/META-INF /app/META-INF
COPY ${DEPENDENCY}/BOOT-INF/classes /app
ENTRYPOINT ["java","-cp","app:app/lib/*","hello.Application"]
Files to copy are prepared by gradle and i can confirm that they are present:
task unpack(type: Copy) {
dependsOn bootJar
from(zipTree(tasks.bootJar.outputs.files.singleFile))
into("build/dependency")
}
I am running
docker build .
docker gradle task
docker {
name "${project.group}/${bootJar.baseName}"
copySpec.from(tasks.unpack.outputs).into("dependency")
buildArgs(['DEPENDENCY': "dependency"])
}
Problem
docker build -t springio/gs-spring-boot-docker .
Sending build context to Docker daemon 16.78MB
Step 1/8 : FROM openjdk:8-jdk-alpine
---> a3562aa0b991
Step 2/8 : RUN addgroup -S spring && adduser -S spring -G spring
---> Using cache
---> 5b122a1db135
Step 3/8 : USER spring:spring
---> Using cache
---> 3bdba2028e42
Step 4/8 : ARG DEPENDENCY=target/dependency
---> Using cache
---> 616945cc41ed
Step 5/8 : COPY ${DEPENDENCY}/BOOT-INF/lib /app/lib
COPY failed: file not found in build context or excluded by .dockerignore: stat target/dependency/BOOT-INF/lib: file does not exist
Fix
mkdir -p target/dependency && (cd target/dependency; jar -xf ../*.jar)
The below folder layout will be created.
This is done to explain the layering options for your docker image. It is all explained in Example 3. Dockerfile
$pwd
/java-training/workspace/gs-spring-boot-docker/complete
$tree -L 3 target/dependency
target/dependency
├── BOOT-INF
│   ├── classes
│   │   ├── application.yml
│   │   └── hello
│   ├── classpath.idx
│   └── lib
│   ├── jackson-annotations-2.11.0.jar
│   ├── jackson-core-2.11.0.jar
│   ├── jackson-databind-2.11.0.jar
│   ├── jackson-datatype-jdk8-2.11.0.jar
│   ├── jackson-datatype-jsr310-2.11.0.jar
│   ├── jackson-module-parameter-names-2.11.0.jar
│   ├── jakarta.annotation-api-1.3.5.jar
│   ├── jakarta.el-3.0.3.jar
│   ├── jul-to-slf4j-1.7.30.jar
│   ├── log4j-api-2.13.2.jar
│   ├── log4j-to-slf4j-2.13.2.jar
│   ├── logback-classic-1.2.3.jar
│   ├── logback-core-1.2.3.jar
│   ├── slf4j-api-1.7.30.jar
│   ├── snakeyaml-1.26.jar
│   ├── spring-aop-5.2.6.RELEASE.jar
│   ├── spring-beans-5.2.6.RELEASE.jar
│   ├── spring-boot-2.3.0.RELEASE.jar
│   ├── spring-boot-autoconfigure-2.3.0.RELEASE.jar
│   ├── spring-boot-starter-2.3.0.RELEASE.jar
│   ├── spring-boot-starter-json-2.3.0.RELEASE.jar
│   ├── spring-boot-starter-logging-2.3.0.RELEASE.jar
│   ├── spring-boot-starter-tomcat-2.3.0.RELEASE.jar
│   ├── spring-boot-starter-web-2.3.0.RELEASE.jar
│   ├── spring-context-5.2.6.RELEASE.jar
│   ├── spring-core-5.2.6.RELEASE.jar
│   ├── spring-expression-5.2.6.RELEASE.jar
│   ├── spring-jcl-5.2.6.RELEASE.jar
│   ├── spring-web-5.2.6.RELEASE.jar
│   ├── spring-webmvc-5.2.6.RELEASE.jar
│   ├── tomcat-embed-core-9.0.35.jar
│   └── tomcat-embed-websocket-9.0.35.jar
├── META-INF
│   ├── MANIFEST.MF
│   └── maven
│   └── org.springframework
└── org
└── springframework
└── boot
Now you can build it!
 docker build -t springio/gs-spring-boot-docker .
Sending build context to Docker daemon 33.44MB
Step 1/8 : FROM openjdk:8-jdk-alpine
---> a3562aa0b991
Step 2/8 : RUN addgroup -S spring && adduser -S spring -G spring
---> Using cache
---> 5b122a1db135
Step 3/8 : USER spring:spring
---> Using cache
---> 3bdba2028e42
Step 4/8 : ARG DEPENDENCY=target/dependency
---> Using cache
---> 616945cc41ed
Step 5/8 : COPY ${DEPENDENCY}/BOOT-INF/lib /app/lib
---> Using cache
---> 1d1404d8b0e7
Step 6/8 : COPY ${DEPENDENCY}/META-INF /app/META-INF
---> Using cache
---> ad8c03fa8c4e
Step 7/8 : COPY ${DEPENDENCY}/BOOT-INF/classes /app
---> Using cache
---> 2aaadf41ccd2
Step 8/8 : ENTRYPOINT ["java","-cp","app:app/lib/*","hello.Application"]
---> Using cache
---> a21e7f0c88fc
Successfully built a21e7f0c88fc
Successfully tagged springio/gs-spring-boot-docker:latest
Best of luck!
You're getting a "no such file or directory" error, and it looks like that's the truth.
The Dockerfile sets:
ARG DEPENDENCY=target/dependency
And then attempts a COPY operation:
COPY ${DEPENDENCY}/BOOT-INF/lib /app/lib
If you resolve ${DEPENDENCY}, that COPY command look like:
COPY target/dependency/BOOT-INF/lib /app/lib
And there is no target directory in the repository. Maybe this is something you're supposed to create by following the tutorial? From that document:
This Dockerfile has a DEPENDENCY parameter pointing to a directory where we have unpacked the fat jar. If we get that right, it already contains a BOOT-INF/lib directory with the dependency jars in it, and a BOOT-INF/classes directory with the application classes in it. Notice that we are using the application’s own main class hello.Application (this is faster than using the indirection provided by the fat jar launcher).
Run this inside your project
mkdir target/dependency
(cd target/dependency; jar -xf ../*.jar)
I had the same problem. for solving this:
before execute your
docker build
command, just run this to commands in root of your project (where Dockerfile and target exist):
mkdir target/dependency
(cd target/dependency; jar -xf ../*.jar)
or,
2. just add those two command in your .gitlab-ci.yml file (if using gitlab CI/CD pipeline):
docker-build:
stage: package
tags:
- vasci2_shell_runner
script:
- mkdir target/dependency
- (cd target/dependency; jar -xf ../*.jar)
- docker build -t nexus.css.ir:30005/vas/harim/apc:lastest .

Dockerfile COPY instruction failing with explicit file reference

If my project structure is like this:
├── compose
│   ├── local
│   │   └── django
│   │   ├── Dockerfile
│   │   ├── celery
│   │   │   ├── beat
│   │   │   │   └── start.sh
│   │   │   └── worker
│   │   │   └── start.sh
│   │   └── start.sh
│   └── production
│   ├── caddy
│   │   ├── Caddyfile
│   │   └── Dockerfile.caddy
│   ├── django
│   │   ├── Dockerfile.django
I am calling from root, $  heroku container:push --recursive
However, build will fail citing COPY fail:
Sending build context to Docker daemon 3.072kB
Step 1/2 : FROM abiosoft/caddy:0.10.6
---> d8ff070e1cee
Step 2/2 : COPY ./compose/production/caddy/Caddyfile /etc/Caddyfile
COPY failed: stat /var/lib/docker/tmp/docker-builder022230374/compose/production/caddy/Caddyfile: no such file or directory
▸ Error: docker build exited with 1
The actual Dockerfile:
FROM abiosoft/caddy:0.10.6
COPY ./compose/production/caddy/Caddyfile /etc/Caddyfile
I have read this similar stackoverflow post: Dockerfile COPY instruction failing?
Per the top answer, I did believe I'm referencing the file explicitly at build.
I guess not. What am I missing?

Why docker ADD destroys folder structure?

I have the following folder structure:
> tree -L 3
├── pom.xml
├── src
│   ├── main
│   │   ├── java
│   │   └── resources
│   └── test
│   ├── groovy
│   └── resources
I tried to build docker image containing those folders together with files using the following Dockerfile:
FROM jamesdbloom/docker-java8-maven
USER root
RUN mkdir src
ADD ./src/* ./src/
ADD pom.xm
However, the structure in docker image is different. Particularly, I can no longer find main and test folders.
$ tree -L 3
.
├── pom.xml
├── src
│   ├── groovy
│   │   └── com
│   ├── java
│   │   └── com
│   └── resources
│   ├── ext_sample_input.json
│   ├── hist_sample_input.json
│   └── sample_input.json
Why is it so?
From official documentation:
Note: The directory itself is not copied, just its contents.
Change your ADD statement to:
ADD ./src ./src/

Resources