Can't find local Docker images created with Go SDK - docker

I'm trying to create some docker images on macOS (with Colima backend) using Go Docker SDK with following code:
package main
import (
"context"
"flag"
"fmt"
"io"
"log"
"os"
"path/filepath"
"github.com/docker/docker/api/types"
"github.com/docker/docker/client"
"github.com/docker/docker/pkg/archive"
)
const dockerfile = `
# syntax=docker/dockerfile:1
FROM golang:1.19-alpine
WORKDIR /app
ENV GO111MODULE=on
ENV CGO_ENABLED=0
COPY . ./
RUN go mod download
RUN apk add --no-cache git
RUN go install github.com/securego/gosec/v2/cmd/gosec#latest
RUN go install golang.org/x/vuln/cmd/govulncheck#latest
RUN go get github.com/stripe/safesql
CMD gosec ./... && govulncheck ./... && safesql ./...`
func main() {
ctx := context.Background()
pathFlag := flag.String("path", ".", "path to the directory containing the go files")
flag.Parse()
if pathFlag == nil {
log.Fatal("path flag is nil")
}
err := os.WriteFile(*pathFlag+"/Dockerfile", []byte(dockerfile), 0777)
if err != nil {
fmt.Printf("Unable to write file: %v", err)
}
cli, err := client.NewClientWithOpts()
if err != nil {
log.Fatal(err, " :unable to init client")
}
files, err := WalkMatch(*pathFlag, "*.go")
if err != nil {
log.Fatal(err, " :unable to walk files")
}
reader, err := archive.TarWithOptions(*pathFlag, &archive.TarOptions{
IncludeFiles: append(files, []string{"go.mod", "go.sum", "Dockerfile"}...),
})
if err != nil {
log.Fatal(err, " :unable to create tar")
}
imageBuildResponse, err := cli.ImageBuild(
ctx,
reader,
types.ImageBuildOptions{
Tags: []string{"newname"},
Context: reader,
Dockerfile: "Dockerfile",
})
if err != nil {
log.Fatal(err, " :unable to build docker image")
}
defer imageBuildResponse.Body.Close()
_, err = io.Copy(os.Stdout, imageBuildResponse.Body)
if err != nil {
log.Fatal(err, " :unable to read image build response")
}
}
func WalkMatch(root, pattern string) ([]string, error) {
var matches []string
err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if info.IsDir() {
return nil
}
if matched, err := filepath.Match(pattern, filepath.Base(path)); err != nil {
return err
} else if matched {
r, err := filepath.Rel(root, path)
if err != nil {
return err
}
matches = append(matches, r)
}
return nil
})
if err != nil {
return nil, err
}
return matches, nil
}
The creation seems to be successful:
{"aux":{"ID":"sha256:568d5aed149aa73d38b644f49f68590cc06222467be1e1c3b902792da39f6e24"}}
{"stream":"Successfully built 568d5aed149a\n"}
{"stream":"Successfully tagged newname:latest\n"}
However, I can't find these images anywhere, docker run states there are no images with such tag, docker ls tells me the same. Do I miss something crucial? Are there any macOS specific things I should consider?

You are only building the images not running them so you should use the command
docker images
to get all the images present on your system.
docker ps
only gives back the images which are currently running

Related

How to force docker file use envs which are passed by docker client in golang?

I'm using a docker client with golang; in the following script, I'm trying to pass an environmental variable when a container is going to start.
package main
import (
"context"
"fmt"
"os/user"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/client"
)
func main() {
dockerClient, _ := client.NewClientWithOpts(client.FromEnv)
env := []string{}
for key, value := range map[string]string{"hi": "hoo"} {
env = append(env, fmt.Sprintf("%s=%s", key, value))
}
user, err := user.Current()
if err != nil {
fmt.Println(err)
}
config := container.Config{
User: fmt.Sprintf("%s:%s", user.Uid, user.Gid),
Image: "675d8442a90f",
Env: env,
}
response, err := dockerClient.ContainerCreate(context.Background(), &config, nil, nil, nil, "")
if err != nil {
fmt.Println(err)
}
if err = dockerClient.ContainerStart(context.Background(), response.ID, types.ContainerStartOptions{}); err != nil {
fmt.Println(err)
}
}
and my docker file is a simple one which I try to echo the hi env:
# Filename: Dockerfile
FROM ubuntu:latest
COPY . .
CMD ["echo", "$hi"]
When I built the image and passed the id to the script, it didn't echo the variable. Do you have any idea how I can use the environmental variables in the dockerfile, which are sent to the container by docker golang client?

how to specify sync-mode and network type while creating docker container using docker go pkg in Golang application?

I am trying to create docker container from my golang application using Docker Engine SDKs and Docker API
this is the command i want to implement in my application:
docker run --name rinkeby-node ethereum/client-go --rinkeby --syncmode full
this is the code i am using
ctx := context.Background()
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
panic(err)
}
imageName := "ethereum/client-go"
out, err := cli.ImagePull(ctx, imageName, types.ImagePullOptions{})
if err != nil {
panic(err)
}
defer out.Close()
io.Copy(os.Stdout, out)
resp, err := cli.ContainerCreate(ctx, &container.Config{
Image: imageName,
}, nil, nil, nil, "containerName")
if err != nil {
panic(err)
}
if err := cli.ContainerStart(ctx, resp.ID, types.ContainerStartOptions{}); err != nil {
panic(err)
}
fmt.Println(resp.ID)
now i want to specify the syncmode to full and the network to rinkeby
The --syncmode full and --rinkeby flags are the CMD arguments.
So when calling you're calling ContainerCreate method inside of container.Config add this:
Cmd: []string{"--syncmode", "full", "--rinkeby"}
For a complete example see this

Send Docker Context as Tar with Go Client can't find Dockerfile

I am using the Go Docker Client to attempt to build an image from a Dockerfile whose contents are defined in code.
According to the Docker Daemon API Documentation
The input stream must be a tar archive...
...The archive must include a build instructions file, typically called Dockerfile at the archive’s root.
So I want to create the build context in code, write it to a tar file, then send that to the Docker Daemon to be built. To do this I can use the ImageBuild function and pass in the tar file (build context) as an io.ReadCloser. As long as my Dockerfile is at the root of that compressed archive, it should find it and build it.
However, I get the common error:
Error response from daemon: Cannot locate specified Dockerfile: Dockerfile
Which obviously means that it can't find the Dockerfile at the root of the archive. I am unsure why. I believe the way I am doing this adds a Dockerfile to the root of the tar archive. The daemon should see this. What am I misunderstanding here?
code snippet to reproduce
var buf bytes.Buffer
tarWriter := tar.NewWriter(&buf)
contents := "FROM alpine\nCMD [\"echo\", \"this is from the archive\"]"
if err := tarWriter.WriteHeader(&tar.Header{
Name: "Dockerfile",
Mode: 777,
Size: int64(len(contents)),
Typeflag: tar.TypeReg,
}); err != nil {
panic(err)
}
if _, err := tarWriter.Write([]byte(contents)); err != nil {
panic(err)
}
if err := tarWriter.Close(); err != nil {
panic(err)
}
reader := tar.NewReader(&buf)
c, err := client.NewEnvClient()
if err != nil {
panic(err)
}
_, err = c.ImageBuild(context.Background(), reader, types.ImageBuildOptions{
Context: reader,
Dockerfile: "Dockerfile",
})
if err != nil {
panic(err)
}
go.mod file
module docker-tar
go 1.12
require (
github.com/docker/distribution v2.7.1+incompatible // indirect
github.com/docker/docker v1.13.1
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.4.0 // indirect
github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
github.com/pkg/errors v0.8.1 // indirect
golang.org/x/net v0.0.0-20191112182307-2180aed22343 // indirect
)
Add a zero in front of 777 for octal numeral system: 0777, 0o777, or 0O777
Use reader := bytes.NewReader(buf.Bytes()) not tar.NewReader(&buf)
Use client.WithAPIVersionNegotiation() for newer versions too.
Try this working version:
package main
import (
"archive/tar"
"bytes"
"context"
"fmt"
"github.com/docker/docker/api/types"
"github.com/docker/docker/client"
)
func main() {
var buf bytes.Buffer
tarWriter := tar.NewWriter(&buf)
contents := `FROM alpine:3.10.3
CMD ["echo", "this is from the archive"]
`
header := &tar.Header{
Name: "Dockerfile",
Mode: 0o777,
Size: int64(len(contents)),
Typeflag: tar.TypeReg,
}
err := tarWriter.WriteHeader(header)
if err != nil {
panic(err)
}
_, err = tarWriter.Write([]byte(contents))
if err != nil {
panic(err)
}
err = tarWriter.Close()
if err != nil {
panic(err)
}
c, err := client.NewClientWithOpts(client.WithAPIVersionNegotiation())
if err != nil {
panic(err)
}
fmt.Println(c.ClientVersion())
reader := bytes.NewReader(buf.Bytes()) // tar.NewReader(&buf)
ctx := context.Background()
buildOptions := types.ImageBuildOptions{
Context: reader,
Dockerfile: "Dockerfile",
Tags: []string{"alpine-echo:1.2.4"},
}
_, err = c.ImageBuild(ctx, reader, buildOptions)
if err != nil {
panic(err)
}
}
Output for docker image ls after go run .:
REPOSITORY TAG IMAGE ID CREATED SIZE
alpine-echo 1.2.4 d81774f32812 26 seconds ago 5.55MB
alpine 3.10.3 b168ac0e770e 4 days ago 5.55MB
Output for docker run alpine-echo:1.2.4:
this is from the archive
Note: You may need to edit FROM alpine:3.10.3 for your specific version.

docker build in Go client fail on COPY

I'm trying to build an image using docker's Go client. Here is the Go program I'm running:
func main() {
ctx := context.Background()
cli, err := client.NewEnvClient()
if err != nil {
log.Fatal(err, " :unable to init client")
}
buf := new(bytes.Buffer)
tw := tar.NewWriter(buf)
defer tw.Close()
dockerFile := "Dockerfile"
dockerFileReader, err := os.Open("./Dockerfile")
if err != nil {
log.Fatal(err, " :unable to open Dockerfile")
}
readDockerFile, err := ioutil.ReadAll(dockerFileReader)
if err != nil {
log.Fatal(err, " :unable to read dockerfile")
}
tarHeader := &tar.Header{
Name: dockerFile,
Size: int64(len(readDockerFile)),
}
err = tw.WriteHeader(tarHeader)
if err != nil {
log.Fatal(err, " :unable to write tar header")
}
_, err = tw.Write(readDockerFile)
if err != nil {
log.Fatal(err, " :unable to write tar body")
}
dockerFileTarReader := bytes.NewReader(buf.Bytes())
imageBuildResponse, err := cli.ImageBuild(
ctx,
dockerFileTarReader,
types.ImageBuildOptions{
Dockerfile: dockerFile,
Tags: []string{"devbuild"},
Remove: true})
if err != nil {
log.Fatal(err, " :unable to build docker image")
}
defer imageBuildResponse.Body.Close()
_, err = io.Copy(os.Stdout, imageBuildResponse.Body)
if err != nil {
log.Fatal(err, " :unable to read image build response")
}
}
It puts the Dockerfile (located in the current directory) in a tar file and builds an image using cli.ImageBuild. This solution was taken from this post, and my Dockerfile looks like this:
FROM alpine:latest
WORKDIR /gopath/src/build
COPY ./binary_build/ /usr/local/bin/
I'm constantly getting the error on the last step:
{"stream":"Step 3/3 : COPY /binary_build/ /usr/local/bin/"}
{"stream":"\n"}
{"errorDetail":{"message":"COPY failed: stat /var/lib/docker/tmp/docker-builder389608393/binary_build: no such file or directory"},"error":"COPY failed: stat /var/lib/docker/tmp/docker-builder389608393/binary_build: no such file or directory"}
There seems to be similar issue in the past reported here, but it seems to have been patched. I ran go get github.com/docker/docker#latest and my docker version in go.mod is github.com/docker/docker v1.13.1. The issue still persists.
If you just build a docker image, that code means nothing, because the docker building process won't run the program.
You should put your Dockerfile and executable file "binary_build" in a sole folder,may be the Dockerfile should like:
FROM alpine:latest
WORKDIR /gopath/src/build
COPY binary_build /usr/local/bin/

"golang.org/x/net/ipv4" working on Mac, but not on Linux

I'm using "golang.org/x/net/ipv4" in order to use its SetTTL function. Unfortunately it does not seem to work on Linux, only on Mac, even though the documentation indicates Linux supports all the functions.
Here's a minimal example of the problem, with a Dockerfile:
main.go:
package main
import (
"fmt"
"net"
"bufio"
xnet "golang.org/x/net/ipv4"
)
const Host = "google.com"
func main() {
var err error
conn, err := net.Dial("tcp4", Host + ":80")
if err != nil {
panic(err)
}
defer conn.Close()
xconn := xnet.NewConn(conn)
err = xconn.SetTTL(5)
if err != nil {
panic(err)
}
defer xconn.Close()
fmt.Fprint(conn, "GET / HTTP/1.1\r\nHOST: google.com\r\n\r\n")
firstLine, err := bufio.NewReader(xconn).ReadString('\n')
if err != nil {
panic(err)
}
fmt.Println(firstLine)
}
Dockerfile:
FROM golang:1.8.1-alpine
RUN apk --no-cache add git
RUN go get golang.org/x/net/ipv4
COPY . /go/src/me.com/me/xnetproblem
RUN go install me.com/me/xnetproblem
CMD ["/go/bin/xnetproblem"]
I run this command:
docker build -t xnet .
I get this output:
john xnetproblem > docker build -t xnet .
Sending build context to Docker daemon 90.62 kB
Step 1/6 : FROM golang:1.8.1-alpine
[snip]
Step 5/6 : RUN go install me.com/me/xnetproblem
---> Running in c3802fe61d63
# me.com/me/xnetproblem
src/me.com/me/xnetproblem/main.go:25: xconn.Close undefined (type *ipv4.Conn has no field or method Close)
src/me.com/me/xnetproblem/main.go:28: cannot use xconn (type *ipv4.Conn) as type io.Reader in argument to bufio.NewReader:
*ipv4.Conn does not implement io.Reader (missing Read method)
The command '/bin/sh -c go install me.com/me/xnetproblem' returned a non-zero code: 2
Using go install natively, instead of Docker, the program works on Mac but not on Linux.
Thanks to #JimB's comment I realized my Mac had an old version of the ipv4 package installed. After updating I was able to fix the code.
Here's a complete working version:
package main
import (
"fmt"
"net"
"bufio"
"golang.org/x/net/ipv4"
)
const Host = "google.com"
func main() {
var err error
conn, err := net.Dial("tcp4", Host + ":80")
if err != nil {
panic(err)
}
defer conn.Close()
if err = ipv4.NewConn(conn).SetTTL(5); err != nil {
panic(err)
}
fmt.Fprint(conn, fmt.Sprintf("GET / HTTP/1.1\r\nHost: %v\r\n\r\n", Host))
firstLine, err := bufio.NewReader(conn).ReadString('\n')
if err != nil {
panic(err)
}
fmt.Println(firstLine)
}
Here is the output:
HTTP/1.1 301 Moved Permanently

Resources