I am having an issue with accessing a static file for my local webserver when I am building it with docker. I am using the github.com/xeipuuv/gojsonschema tool kit for validating incoming json request with a local json schema file via
schemaLoader := gojsonschema.NewReferenceLoader("file://C:/Users/user/Workspace/jsonschema.json")
But when I am trying to access the file with docker it says "no such file or directory". The Dockerfile I am using is:
FROM golang:1.17-alpine
WORKDIR /app
COPY go.mod .
COPY go.sum .
COPY jsonschema.json .
RUN go mod download
COPY *.go ./
RUN go build -o /main
EXPOSE 8080
CMD ["/main"]
Thank You very much in advance.
Best regards
I tried changing the directory to, i.e.
schemaLoader := gojsonschema.NewReferenceLoader("file://app/jsonschema.json")
but it didn't help.
Your fix is nearly correct but you are missing a single / in the file:// path.
Here is an explanation of the difference between file:/, file://, and file:///.
You want this:
schemaLoader := gojsonschema.NewReferenceLoader("file:///app/jsonschema.json")
which means: use the file uri (file://) to load file with absolute path (/app/jsonschema.json).
Related
I have a simple static website, which I can host it from docker by writing a dockerfile.txt with the following commands
FROM nginx
RUN mkdir /usr/share/nginx/html/blog
COPY . /usr/share/nginx/html/blog
This works pretty well for me.
Now I'm trying to dockerize a static that was build using docker, what should I exactly write in the docker file
FROM klakegg/hugo
COPY ?????????????????????
Does hugo have a dir where I can place the website files in it? or does Hugo works completely diff?
Thanks in advance!
Your files need to be placed in /src
The klakegg/hugo container only acts as the "compiler". In order to host the generated files you also need nginx.
This can the achieved with multistage-builds
FROM klakegg/hugo AS build
COPY . /src
FROM nginx
COPY --from=build /src/public /usr/share/nginx/html
I have a local folder called images that contains bunch of folders and files within. When I run the container I get the following error:
the command I execute: docker run -t -i file-uploader -token=abcdefgh
panic: failed Walk: Failed to walk directory: *fs.PathError lstat ./images/: no such file or directory
goroutine 1 [running]:
main.main()
/src/main.go:57 +0x357
Here is the Dockerfile I created:
FROM golang:1.16
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY ./images/ images/
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT ["/bin/app"]
FROM scratch
COPY --from=0 /bin/app /bin/app
ENTRYPOINT ["/bin/app"]
And, here is the code in the program:
var (
token = flag.String("token", "", "user's token for application")
rootpath = flag.String("rootpath", "./images/","folder path to be uploaded")
)
func main() {
flag.Parse()
if *token == "" {
log.Fatal(Red + "please provide a client token => -token={$token}")
}
tokenSource := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: *token})
oauthClient := oauth2.NewClient(context.TODO(), tokenSource)
client := putio.NewClient(oauthClient)
paths := make(chan string)
var wg = new(sync.WaitGroup)
for i := 0; i < 20; i++ {
wg.Add(1)
go worker(paths, wg, client)
}
if err := filepath.Walk(*rootpath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return fmt.Errorf("Failed to walk directory: %T %w", err, err)
}
if !info.IsDir() {
paths <- path
}
return nil
}); err != nil {
panic(fmt.Errorf("failed Walk: %w", err))
}
close(paths)
wg.Wait()
}
}
If flag is not provided, its default value is the folder itself which is ./images/. When I run this normally like: go run main.go -token="abcde", it works properly. I did some changes on Dockerfile. Some of the changes I made and tried again and again.:
replacing COPY . . with COPY ./images/ /images. It should automatically creates a folder inside /src like /src/images and get the local folder from host and put into it. It didn't work.
I also did try COPY . ., believing that it will copy everything from host into docker container. It didn't work either.
I did put 2 COPY command together. Didn't work.
COPY . ./ didn't work either.
My structure of the project is as follows:
/file-uploader-cli
/images
Dockerfile
file-uploader-cli (binary)
go.mod with go.sum
main.go
How can I put /images folder into container and run it properly?
Extra question: /images folder is approx. 500 MB or something. Is it a good practice to put that folder into a container?
I guess it is possible to copy a folder like docker cp {$folder_name} ${container_id}:/{$path}, but it must be a running container or something? I did try this using image_id replacing the container_id but I got an error like No such container:path: 12312312312:/.
EDIT:
the problem is the scratch image which was done in order to reduce the size. However, when I delete the scratch thing, the size of the image became 1.1 GB. Any easier or convenient way to utilize the images folder without having too much size?
You don't need images when building your app, you need it when executing your app. Instead of adding images to the first image, add it to the final image:
FROM golang:1.16 AS builder
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
FROM scratch
WORKDIR /
copy images .
COPY --from=builder /bin/app /bin/app
ENTRYPOINT ["/bin/app"]
Or, if you want to provide images dynamically, you could mount it at runtime.
I also recommend removing docker containers automatically unless you actually want them sticking around. Otherwise you end up with lots and lots of Exited containers.
docker run --rm -it -v /my/path/to/images:/images:ro file-uploader -token=abcdefgh
I would also recommend you put your token in an environment variable so its not saved in bash history and docker runtime information.
So, you're saying don't dockerize the app?
I don't usually containerize Go programs unless containers are a good fit for deployment and operations (eg kubernetes). Most languages like C/C++, Java, Erlang, Python, Javascript, all require significant runtime components provided by the filesystem - compiled ones are usually dynamically linked with shared libraries from the operating system, VM based languages like Java or Erlang require the VM to be installed and configured (and it will likely also have runtime dependencies), and interpreted languages like Python, Ruby, or Javascript require the entire interpreter runtime, as well as any shared libraries the language's libraries are linked to.
Go is an exception to this though. Ignoring CGo (which I recommend avoiding whenever possible), Go binaries are statically linked and have minimal userspace runtime requirements. This is why a Go binary is one of the few things that can acutally work in a container built FROM scratch. The one exception to this, is that the Go program will need CA Certificates from the operating system to validate the certificates on HTTPS servers and other protocols secured with TLS.
I recommend that you don't dockerize the app unless docker is helping you distribute or operate it.
The problem is that the second build stage does not include the images directory. I don't think you can use the scratch base for this purpose, so we will have to change that. If you are concerned about image size, you should look into into alpine linux. I have converted your Dockerfile to use alpine.
FROM golang:1.16-alpine
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
ENTRYPOINT ["/bin/app"]
FROM alpine:3.15.0
WORKDIR /opt/app
COPY --from=0 /bin/app app
COPY images .
ENTRYPOINT ["/opt/app/app"]
Note that I changed the app path in the second build stage to /opt/app. I did this because it would be odd to have an image folder under /bin. And /opt is a common place to store user applications.
As for whether or not you should containerize your code, that is up to you. One of Go's advantages is static compilation and easy cross-compiling. So you could distribute your binary (and images) as is.
I have a problem when I try to run my app in a Docker container. It is running fine with a simple go run main.go, but whenever I build an image and I run the docker container, I got the error of panic: html/template: pattern matches no files: *.html, so I guess GOPATH is not properly set in the docker container (tho I use this same docker file from other projects and I don't have any problems). I am a little lost here, since this method I been using already for a while without problems.
I am using gin as a framework for develop.
The docker file is:
FROM golang:alpine as builder
RUN apk update && apk add git && apk add ca-certificates
# For email certificate
RUN apk add -U --no-cache ca-certificates
COPY . $GOPATH/src/github.com/kiketordera/advanced-performance/
WORKDIR $GOPATH/src/github.com/kiketordera/advanced-performance/
RUN go get -d -v $GOPATH/src/github.com/kiketordera/advanced-performance
# For Cloud Server
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags="-w -s" -o /go/bin/advanced-performance $GOPATH/src/github.com/kiketordera/advanced-performance
FROM scratch
COPY --from=builder /go/bin/advanced-performance /advanced-performance
COPY --from=builder /go/src/github.com/kiketordera/advanced-performance/media/ /go/src/github.com/kiketordera/advanced-performance/media/
# For email certificate
VOLUME /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt
COPY --from=alpine /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
EXPOSE 8050/tcp
ENV GOPATH /go
ENTRYPOINT ["/advanced-performance"]
Main function is:
package main
import (
"fmt"
"net/http"
"github.com/gin-gonic/gin"
i18n "github.com/suisrc/gin-i18n"
"golang.org/x/text/language"
)
func main() {
// We create the instance for Gin
r := gin.Default()
/* Internationalization for showing the right language to match the browser's default settings
*/
bundle := i18n.NewBundle(
language.English,
"text/en.toml",
"text/es.toml",
)
// Tell Gin to use our middleware. This means that in every single request (GET, POST...), the call to i18n will be executed
r.Use(i18n.Serve(bundle))
// Path to the static files. /static is rendered in the HTML and /media is the link to the path to the images, svg, css.. the static files
r.StaticFS("/static", http.Dir("media"))
// Path to the HTML templates. * is a wildcard
r.LoadHTMLGlob("*.html")
// Redirects when users introduces a wrong URL
r.NoRoute(redirect)
// This get executed when the users gets into our website in the home domain ("/")
r.GET("/", renderHome)
r.POST("/", getForm)
// Listen and serve on 0.0.0.0:8080 (for windows "localhost:8080")
r.Run()
}
The full project can be found in https://github.com/kiketordera/advanced-performance, is a simple website rendering with i18n and a POST form-handler
GOPATH is not relevant; it's used to "resolve import statements" and plays no role when running an executable (unless your code references it specifically!). The WORKDIR is the issue here.
FROM "clears any state created by previous instructions". This includes the WORKDIR. For example if you use the docker file:
FROM alpine:3.12
WORKDIR /test
copy 1.txt .
FROM alpine:3.12
copy 2.txt .
The final resulting image will have file 2.txt in the root folder (and no /test folder).
In your dockerfile you are copying the media folder to /go/src/github.com/kiketordera/advanced-performance/media/ on the assumption that the WORKDIR will be set; but that is not the case (it defaults to /). Simplest fix is to change COPY --from=builder /go/src/github.com/kiketordera/advanced-performance/media/ /go/src/github.com/kiketordera/advanced-performance/media/ to COPY --from=builder /go/src/github.com/kiketordera/advanced-performance/media/ /media/.
You are also accessing files from the root folder so need to copy these in with COPY --from=builder /go/src/github.com/kiketordera/advanced-performance/*.html / (or similar). Given that you are doing this it's probably best to put everything (the exe, html files and media folder) into a folder (e.g. /app) to keep the root folder clean.
Note: There is no need to set GOPATH in the second image; as mentioned above it's not relevant when running the executable. I'd recommend using modules (support for GOPATH will probably be dropped in 1.17); this would also enable you to considerably shorten your paths!
I tried to copy some files from source to destination (flask app) in a dockerfile but it seems things are not working as expected when building the image. With last 2 line showing:
Step 3 : COPY pkl_objects/* /home/jovyan/work/movieclassifier/pkl_objects/
No source files were specified
This is the docker file.
FROM jupyter/datascience-notebook
RUN pip install flask flask-wtf
COPY pkl_objects/* /home/jovyan/work/movieclassifier/pkl_objects/
COPY static/* /home/jovyan/work/movieclassifier/static/
COPY templates/* /home/jovyan/work/movieclassifier/templates/
COPY app.py /home/jovyan/work/movieclassifier
COPY reviews.sqlite /home/jovyan/work/movieclassifier
COPY vectorizer.py /home/jovyan/work/movieclassifier
WORKDIR /home/jovyan/work/movieclassifier
ENV FLASK_APP=app.py
# ENV FLASK_DEBUG=0
CMD ["flask", "run", "--host=0.0.0.0"]
Looks like there are no files in the pkl_objects folder, and when the wildcard (*) is expanded, it results in no source files being specified.
Maybe you could add an empty file in there, so that when the wildcard picks up files, you at least get one source file.
Example file could be: .nonempty or something like that.
I'm using the docker COPY instruction to copy files from <src> to <dest> as described in the documentation. However it is possible that there will be no <src> file which causes docker-compose build to fail. Like so:
Step 7 : COPY cts/application.properties /cts/
ERROR: Service 'redirector' failed to build: lstat cts/application.properties: no such file or directory
Is there a way to only copy the file if it's there or turn off the errors?
The only way to do that, is copying all files inside cts folder, for that, you can user COPY with a wildcard.
COPY cts/* /cts/
Or set your WORKDIR as cts, copy all files, and then set back your WORKDIR
WORKDIR cts/
ADD . /cts/
WORKDIR old_workdir_path
But, if you want to copy conditionally one file using COPY command, you can't.