I'm building a multi-stage Dockerfile for my go project.
FROM golang:latest as builder
COPY ./go.mod /app/go.mod
COPY ./go.sum /app/go.sum
#exporting go1.11 module support variable
ENV GO111MODULE=on
WORKDIR /app/
#create vendor directory
RUN go mod download
COPY . /app/
RUN go mod vendor
#building source code
RUN go build -mod=vendor -o main -v ./src/
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY --from=builder /app/main /app/main
WORKDIR /app/
ARG port="80"
ENV PORT=$port
EXPOSE $PORT
CMD ["./main"]
When I'm running the image, it throws error:
standard_init_linux.go:207: exec user process caused "no such file or directory"
I've verified that the 'main' file exist in /app/main.
I also tried to give executable permission by adding
chmod +x /app/main
but still it doesn't work.
What can possibly be wrong?
The "latest" version of the golang image is debian based, which uses libc. Alpine uses musl. If you do not compile with CGO_ENABLED=0, networking libraries will link to libc and the no such file or directory error point to a missing library. You can check these shared library links with ldd /app/main. A few solutions I can think of:
compile your program with CGO_ENABLED=0
switch your build image to FROM golang:alpine
change your second stage to be FROM debian
Related
I had a docker file that was working fine. However to remote debug it , I read that I needed to install dlv on it and then I need to run dlv and pass the parameter of the app I am trying to debug. So after installing dlv on it and attempting to run it. I get the error
exec /dlv: no such file or directory
This is the docker file
FROM golang:1.18-alpine AS builder
# Build Delve for debugging
RUN go install github.com/go-delve/delve/cmd/dlv#latest
# Create and change to the app directory.
WORKDIR /app
ENV CGO_ENABLED=0
# Retrieve application dependencies.
COPY go.* ./
RUN go mod download
# Copy local code to the container image.
COPY . ./
# Build the binary.
RUN go build -gcflags="all=-N -l" -o fooapp
# Use the official Debian slim image for a lean production container.
FROM debian:buster-slim
EXPOSE 8000 40000
RUN set -x && apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
ca-certificates && \
rm -rf /var/lib/apt/lists/*
# Copy the binary to the production image from the builder stage.
#COPY --from=builder /app/fooapp /app/fooapp #commented this out
COPY --from=builder /go/bin/dlv /dlv
# Run dlv as pass fooapp as parameter
CMD ["/dlv", "--listen=:40000", "--headless=true", "--api-version=2", "--accept-multiclient", "exec", "/app/fooapp"]
The above results in exec /dlv: no such file or directory
I am not sure why this is happening. Being new to docker , I have tried different ways to debug it. I tried using dive to check and see if the image has dlv on it in the path /dlv and it does. I have also attached an image of it
You built dlv in alpine-based distro. dlv executable is linked against libc.musl:
# ldd dlv
linux-vdso.so.1 (0x00007ffcd251d000)
libc.musl-x86_64.so.1 => not found
But then you switched to glibc-based image debian:buster-slim. That image doesn't have the required libraries.
# find / -name libc.musl*
<nothing found>
That's why you can't execute dlv - the dynamic linker fails to find the proper lib.
You need to build in glibc-based docker. For example, replace the first line
FROM golang:bullseye AS builder
BTW. After you build you need to run the container in the priviledged mode
$ docker build . -t try-dlv
...
$ docker run --privileged --rm try-dlv
API server listening at: [::]:40000
2022-10-30T10:51:02Z warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted)
In non-priviledged container dlv is not allowed to spawn a child process.
$ docker run --rm try-dlv
API server listening at: [::]:40000
2022-10-30T10:55:46Z warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted)
could not launch process: fork/exec /app/fooapp: operation not permitted
Really Minimal Image
You use debian:buster-slim to minimize the image, it's size is 80 MB. But if you need a really small image, use busybox, it is only 4.86 MB overhead.
FROM golang:bullseye AS builder
# Build Delve for debugging
RUN go install github.com/go-delve/delve/cmd/dlv#latest
# Create and change to the app directory.
WORKDIR /app
ENV CGO_ENABLED=0
# Retrieve application dependencies.
COPY go.* ./
RUN go mod download
# Copy local code to the container image.
COPY . ./
# Build the binary.
RUN go build -o fooapp .
# Download certificates
RUN set -x && apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
ca-certificates
# Use the official Debian slim image for a lean production container.
FROM busybox:glibc
EXPOSE 8000 40000
# Copy the binary to the production image from the builder stage.
COPY --from=builder /app/fooapp /app/fooapp
# COPY --from=builder /app/ /app
COPY --from=builder /go/bin/dlv /dlv
COPY --from=builder /etc/ssl /etc/ssl
# Run dlv as pass fooapp as parameter
CMD ["/dlv", "--listen=:40000", "--headless=true", "--api-version=2", "--accept-multiclient", "exec", "/app/fooapp"]
# ENTRYPOINT ["/bin/sh"]
The image size is 25 MB, of which 18 MB are from dlv and 2 MB are from Hello World application.
While choosing the images care should be taken to have the same flavors of libc. golang:bullseye links against glibc. Hence, the minimal image must be glibc-based.
But if you want a bit more comfort, use alpine with gcompat package installed. It is a reasonably rich linux with lots of external packages for just extra 6 MB compared to busybox.
FROM golang:bullseye AS builder
# Build Delve for debugging
RUN go install github.com/go-delve/delve/cmd/dlv#latest
# Create and change to the app directory.
WORKDIR /app
ENV CGO_ENABLED=0
# Copy local code to the container image.
COPY . ./
# Retrieve application dependencies.
RUN go mod tidy
# Build the binary.
RUN go build -o fooapp .
# Use alpine lean production container.
# FROM busybox:glibc
FROM alpine:latest
# gcompat is the package to glibc-based apps
# ca-certificates contains trusted TLS CA certs
# bash is just for the comfort, I hate /bin/sh
RUN apk add gcompat ca-certificates bash
EXPOSE 8000 40000
# Copy the binary to the production image from the builder stage.
COPY --from=builder /app/fooapp /app/fooapp
# COPY --from=builder /app/ /app
COPY --from=builder /go/bin/dlv /dlv
# Run dlv as pass fooapp as parameter
CMD ["/dlv", "--listen=:40000", "--headless=true", "--api-version=2", "--accept-multiclient", "exec", "/app/fooapp"]
# ENTRYPOINT ["/bin/bash"]
TL;DR
Run apt-get install musl, then /dlv should work as expected.
Explanation
Follow these steps:
docker run -it <image-name> sh
apt-get install file
file /dlv
Then you can see the following output:
/dlv: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, Go BuildID=xV8RHgfpp-zlDlpElKQb/DOLzpvO_A6CJb7sj1Nxf/aCHlNjW4ruS1RXQUbuCC/JgrF83mgm55ntjRnBpHH, not stripped
The confusing no such file or directory (see this question for related discussions) is caused by the missing /lib/ld-musl-x86_64.so.1.
As a result, the solution is to install the musl library by following its documentation.
My answer is inspired by this answer.
The no such file or directory error indicates either your binary file does not exist, or your binary is dynamically linked to a library that does not exist.
As said in this answer, delve is linked against libc.musl. So for your delve build, you can disable CGO since that can result in dynamic links to libc/libmusl:
# Build Delve for debugging
RUN CGO_ENABLED=0 go install github.com/go-delve/delve/cmd/dlv#latest
...
This even allows you later to use a scratch build for your final target image and does not require you to install any additional packages like musl or use any glibc based image and require you to run in privileged mode.
I'm running a dotnet webapp in my local (windows machine) and it works just fine. When I deploy the same application to an AKS container and try running it, it fails with
System.TypeInitializationException: The type initializer for 'System.Runtime.Caching.MemoryMonitor' threw an exception.
---> System.DllNotFoundException: Unable to load shared library 'kernel32.dll' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libkernel32.dll: cannot open shared object file: No such file or directory
at Interop.Kernel32.GlobalMemoryStatusEx(MEMORYSTATUSEX& lpBuffer)
at System.Runtime.Caching.MemoryMonitor..cctor()
--- End of inner exception stack trace ---
Below is my Docker
FROM mcr.microsoft.com/dotnet/aspnet:5.0-bullseye-slim
RUN apt-get update && apt-get install -y \
curl
WORKDIR /app
EXPOSE 80
EXPOSE 443
COPY . .
ENTRYPOINT ["dotnet", "MyApp.dll"]
Let me know if anyone has solved or seen this before
It looks like you're copying in your binaries directly into the container. The tag you are using is 5.0-bullseye-slim is Linux-based. So if the binaries you're copying in were built for the Windows platform, which I'm guessing they are since it's looking for kernel32.dll, then that's not going to run in a Linux environment.
You'll either need to build your binaries so that they are targeting Linux, which can be done with dotnet build -r linux-x64 ..., or a better option would be to build your project as part of the Dockerfile.
Here's an example Dockerfile that demonstrates this:
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /source
# copy csproj and restore as distinct layers
COPY *.csproj .
RUN dotnet restore
# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app --no-restore
# final stage/image
FROM mcr.microsoft.com/dotnet/runtime:6.0
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "dotnetapp.dll"]
(Taken from
https://github.com/dotnet/dotnet-docker/blob/0fc0e2c6af6303cfd4676f1ac8c21090d82b0072/samples/dotnetapp/Dockerfile)
I am trying to containerize and as well as start my Go lang application using Docker-compose,
The image is built successfully according to the logs but my container does not for docker-compose up and it throws the following error to my console.
Cannot start service app: OCI runtime create failed: container_linux.go:346: starting container process caused "exec: \"go\": executable file not found in $PATH": unknown
Here is what my Docker file looks like.
ARG GO_VERSION=1.13
FROM golang:${GO_VERSION}-alpine AS builder
# We create an /app directory within our
# image that will hold our application source
# files
RUN mkdir /raedar
# Create the user and group files that will be used in the running container to
# run the process as an unprivileged user.
RUN mkdir /user && \
echo 'nobody:x:65534:65534:nobody:/:' > /user/passwd && \
echo 'nobody:x:65534:' > /user/group
# Install git.
# Git is required for fetching the dependencies.
# Allow Go to retrieve the dependencies for the buld
RUN apk update && apk add --no-cache ca-certificates git
RUN apk add --no-cache libc6-compat
# Force the go compiler to use modules
ENV GO111MODULE=on
ADD . /raedar/
WORKDIR /raedar/
RUN go get -d -v golang.org/x/net/html
COPY go.mod go.sum ./
COPY . .
# Compile the binary, we don't want to run the cgo
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o bin/main cmd/app/main.go
# Final stage: the running container.
FROM scratch AS final
WORKDIR /root/
# Import the user and group files from the first stage.
COPY --from=builder /user/group /user/passwd /etc/
# Import the Certificate-Authority certificates for enabling HTTPS.
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
# Import the compiled executable from the first stage.
COPY --from=builder /raedar/bin/main .
EXPOSE 8080
# Perform any further action as an unprivileged user.
USER nobody:nobody
# Run the compiled binary.
CMD ["./main"]
The error shows you're trying to run go, not ./main:
exec: \"go\": executable file not found in $PATH
A matching Dockerfile would have the line CMD ["go", "./main"] rather than CMD ["./main"].
So either you're unexpectedly building a different Dockerfile, or you're changing the command when you run a container with that image. In particular, if you're using docker-compose, make sure you're not setting command: go ./main or entrypoint: go, either of which could cause this behavior.
I'm beginner with Docker, and I'm trying to build an image in two stages, as explained here: https://docs.docker.com/develop/develop-images/multistage-build/
You can selectively copy artifacts from one stage to another
Looking at the examples given there, I had thought that one could build some files during a first stage, and then make them available for the next one:
FROM golang:1.7.3 AS builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
(Example taken from the above-linked page)
Isn't that what the COPY app.go . and the COPY --from=builder /go/src/github.com/alexellis/href-counter/app . are supposed to do?
I probably have a complete misunderstanding of what is going on, because when I try to do something similar (see below), it seems that the COPY command from the first stage is not able to see the files that have just been built (I can confirm that they have been actually built using a RUN ls step, but then I get a lstat <the file>: no such file or directory error).
And indeed, most other information I can gather regarding COPY (except the examples in the above link) rather suggest that COPY is actually meant to copy files from the directory where the docker build command was launched, not from within the build environment.
Here is my Dockerfile:
FROM haskell:8.6.5 as haskell
RUN git clone https://gitlab+deploy-token-75:sakyTxfe-PxPHDwqsoGm#gitlab.pasteur.fr/bli/bioinfo_utils.git
WORKDIR bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
RUN stack --resolver ghc-8.6.5 build && \
stack --resolver ghc-8.6.5 install --local-bin-path .
RUN pwd; echo "---"; ls
COPY remove-duplicates-from-sorted-fastq .
FROM python:3.7-buster
RUN python3.7 -m pip install snakemake
RUN mkdir -p /opt/bin
COPY --from=haskell /bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell/remove-duplicates-from-sorted-fastq /opt/bin/remove-duplicates-from-sorted-fastq
CMD ["/bin/bash"]
And here is how the build ends when I run docker build . from the directory containing the Dockerfile:
Step 5/11 : RUN pwd; echo "---"; ls
---> Running in 28ff49fe9150
/bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
---
LICENSE
Setup.hs
install.sh
remove-duplicates-from-sorted-fastq
remove-duplicates-from-sorted-fastq.cabal
src
stack.yaml
---> f551efc6bba2
Removing intermediate container 28ff49fe9150
Step 6/11 : COPY remove-duplicates-from-sorted-fastq .
lstat remove-duplicates-from-sorted-fastq: no such file or directory
How am I supposed to proceed to have the built file available for the next stage?
Well, apparently, I was mislead by the COPY step used in the first stage in the doc example. In my case, this is actually useless, and I can just COPY --from=haskell in my second stage, without any COPY in the first stage.
The following Dockerfile builds without issues:
FROM haskell:8.6.5 as haskell
RUN git clone https://gitlab+deploy-token-75:sakyTxfe-PxPHDwqsoGm#gitlab.pasteur.fr/bli/bioinfo_utils.git
WORKDIR bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell
RUN stack --resolver ghc-8.6.5 build && \
stack --resolver ghc-8.6.5 install --local-bin-path .
FROM python:3.7-buster
RUN python3.7 -m pip install snakemake
RUN mkdir -p /opt/bin
COPY --from=haskell /bioinfo_utils/remove-duplicates-from-sorted-fastq/Haskell/remove-duplicates-from-sorted-fastq /opt/bin/remove-duplicates-from-sorted-fastq
CMD ["/bin/bash"]
So, I am trying to dockerize a golang application with different directories containing supplementary code for my main file.
I am using gorilla/mux. The directory structure looks like this.
$GOPATH/src/github.com/user/server
|--- Dockerfile
|--- main.go
|--- routes/
handlers.go
|--- public/
index.gohtml
It works on my host machine with no problem. The problem is that when I try to deploy the docker image it does not run and exits shortly after creation. I have tried changing the WORKDIR command in my dockerfile to /go/src and dump all my files there, but still no luck. I have also tried the official documentation on docker hub. Doesn't work either.
My Dockerfile.
FROM golang:latest
WORKDIR /go/src/github.com/user/server
COPY . .
RUN go get -d github.com/gorilla/mux
EXPOSE 8000
CMD ["go","run","main.go"]
My golang main.go
package main
import (
"github.com/gorilla/mux"
"github.com/user/server/routes"
"log"
"net/http"
"time"
)
func main(){
//...
}
I get this error message when I check the logs of my docker image.
Error Message
main.go:5:2: cannot find package "github.com/user/server/routes" in any of:
/usr/local/go/src/github.com/user/server/routes (from $GOROOT)
/go/src/github.com/user/server/routes (from $GOPATH)
Try the following Docker file:
# GO Repo base repo
FROM golang:1.12.0-alpine3.9 as builder
RUN apk add git
# Add Maintainer Info
LABEL maintainer="<>"
RUN mkdir /app
ADD . /app
WORKDIR /app
COPY go.mod go.sum ./
# Download all the dependencies
RUN go mod download
COPY . .
# Build the Go app
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
# GO Repo base repo
FROM alpine:latest
RUN apk --no-cache add ca-certificates curl
RUN mkdir /app
WORKDIR /app/
# Copy the Pre-built binary file from the previous stage
COPY --from=builder /app/main .
# Expose port 8000
EXPOSE 8000
# Run Executable
CMD ["./main"]
Here, we are creating an intermediate docker builder container, copying the code into it, build the code inside the builder container and then copy the binary image to the actual docker.
This will help in both having all the dependencies in the final container and also, the size of the final image will be very small