I have a golang script that calls an HTTP API. Everything works great when I run it on my development computer. Once I create the container and run it I get an EOF at the end of the URL on panic. I have read a lot of issues like this and have tried everyone I see. I added a ca-certificates.crt from my host machine to the container to /etc/ssl/certs/. I set the request close to true, I have disabled Keep-Alive and Compression. All of these steps worked for others on other posts. Any help appreciated.
I got my certfile from https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
I am running go version go1.9.2 darwin/amd64 and the container is on a ubuntu 16.04 host.
client := &http.Client{Timeout: 30 * time.Second, Transport: &http.Transport{
DisableCompression:true,
DisableKeepAlives: true,
}}
url := "https://myapiurl.com"
req, err := http.NewRequest("POST", url, bytes.NewBuffer(mybodybytes))
req.Close = true
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
log.Panic(err) // Panics Here
}
defer resp.Body.Close()
...
Dockerfile:
FROM scratch
ADD ca-certificates.crt /etc/ssl/certs/
ADD myservice /
CMD ["/myservice"]
Related
I have a small Golang program and I'm trying to connect to an FTP server running in a docker container (https://registry.hub.docker.com/r/atmoz/sftp).
My machine is a M1 Pro MacBook.
The container is started with the following command:
docker run -p 22:22 -d atmoz/sftp foo:pass:::upload
The Go version is 1.17.13.
The code code of the program is the following:
package main
import (
"log"
"time"
"github.com/jlaffaye/ftp"
)
func main() {
c, err := ftp.Dial("localhost:22", ftp.DialWithTimeout(5*time.Second))
if err != nil {
log.Fatal(err, " cannot connect")
}
err = c.Login("foo", "pass")
if err != nil {
log.Fatal(err, "cannot login")
}
// Do something with the FTP conn
if err := c.Quit(); err != nil {
log.Fatal(err)
}
}
Somehow, I'm unable to connect to the FTP server executing this code, it results in the following output:
EOF cannot connect
I tried connect to the same FTP server using FileZilla and it works fine, im able to connect to the server with success.
Any ideias on how to fix this or further debug the issue? Thank you
The port 22 is typically SSH/SFTP, not FTP. Note that FileZilla supports both FTP and SFTP. So chances are that you are actually connecting with SFTP using FileZilla. Those two protocols are completely different and incompatible.
There seems to be an "sftp" package for Go:
https://pkg.go.dev/github.com/pkg/sftp
I was learning how to dockerize Go Apps. I created a simple REST API
package main
import "github.com/gin-gonic/gin"
func main() {
server := gin.Default()
server.GET("/", func(ctx *gin.Context) {
ctx.JSON(200, "HELLO SERVER")
})
server.Run("127.0.0.1:3000")
}
and here is the Dockerfile
FROM golang:1.18.3-alpine3.16
RUN mkdir /app
COPY . /app
WORKDIR /app
RUN go mod download
RUN go build -o main .
EXPOSE 3000
CMD "/app/main"
Thing is, when build and run the container, it appears the app starts normally as usual
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
- using env: export GIN_MODE=release
- using code: gin.SetMode(gin.ReleaseMode)
[GIN-debug] GET / --> main.main.func1 (3 handlers)
[GIN-debug] [WARNING] You trusted all proxies, this is NOT safe. We recommend you to set a value.
Please check https://pkg.go.dev/github.com/gin-gonic/gin#readme-don-t-trust-all-proxies for details.
[GIN-debug] Listening and serving HTTP on 127.0.0.1:3000
But here is the issue, when I use Postman and make a get request to 127.0.0.1:3000/, it shows it could not get any response (i.e, the response it gets when there is no such server to connect)
When I run the app using 'go run main.go' it works fine.
It would be really great if you could help me out with this
maybe you should listen on 0.0.0.0:3000
func main() {
server := gin.Default()
server.GET("/", func(ctx *gin.Context) {
ctx.JSON(200, "HELLO SERVER")
})
// server.Run("127.0.0.1:3000")
server.Run("0.0.0.0:3000")
}
I'm trying to access a docker registry (public or private) using Go. A simple program which can access any registry and verify if an image is present.
I looked at docker client available in Go https://pkg.go.dev/github.com/docker/docker#v20.10.11+incompatible/client
But the problem is, this client needs a docker daemon running in order to work. Is there any way to query a docker registry (ex: hub.docker.com) without any dependency on underlying docker engine?
My idea is to run this program on a docker container and there wont be any docker engine running inside a container. And I don't want to run docker inside docker or any sort of hack. I just want to connect to a registry and query an image. And please don't quote other questions in stack overflow. No one has answered this.
This is what I have done so far
import (
"context"
"encoding/base64"
"encoding/json"
"fmt"
"github.com/docker/docker/api/types/filters"
"time"
"github.com/docker/docker/api/types"
"github.com/docker/docker/client"
)
func main() {
cli, err := client.NewClientWithOpts(client.WithHost("https://hub.docker.com"), client.WithAPIVersionNegotiation())
if err != nil {
fmt.Println(err.Error())
return
}
err = imagemanifest(cli)
if err != nil {
fmt.Println(err)
}
err = imageSearch(cli)
}
func imagemanifest(dockerClient *client.Client) error {
var authConfig = types.AuthConfig{
Username: "amokkara",
Password: "M#vr1ck2009",
ServerAddress: "https://index.docker.io/v2/",
}
ctx, cancel := context.WithTimeout(context.Background(), time.Second*1200)
defer cancel()
authConfigBytes, _ := json.Marshal(authConfig)
authConfigEncoded := base64.URLEncoding.EncodeToString(authConfigBytes)
ctx, cancel = context.WithTimeout(context.Background(), time.Second*1200)
defer cancel()
searchres , err := dockerClient.DistributionInspect(ctx,"amokkara/amokkara:3",authConfigEncoded)
if err != nil {
return err
}
fmt.Println(searchres.Descriptor.Digest.String())
return nil
}
If I initialize client like this
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
This works because its using underlying docker daemon (in my case docker desktop) to query the registry. But if create client using
client.NewClientWithOpts(client.WithHost("https://hub.docker.com"), client.WithAPIVersionNegotiation())
it fails giving 404 error. Does this client require a docker daemon to work. If so, is there any other way i can query a registry? Please help me with this.
Skopeo is the leader software on dealing with registries without daemon.
It's also written with Go.
You can inspire from inspect.go
Note that, you don't need to use github.com/docker/docker/* modules, but it will be github.com/containers/*, and namely https://github.com/containers/image
I'm using Go and hitting Docker's API to pull an image, and then create and execute a container. Specifically I'm using the docker-newman image. I can see that it's actually being executed in Kitematic, so I know that everything is setup correctly. However, my Go application isn't attaching and then tailing the logs being output.
I've seen this answer, and it references what appears to be the way to attach to the image and view the log. I can't seem to get it to work regardless of what I try.
attachToContainerOptions := docker.AttachToContainerOptions{
Container: container.ID,
OutputStream: os.Stdout,
ErrorStream: os.Stderr,
Logs: true,
Stdout: true,
Stderr: true,
}
if err := client.AttachToContainer(attachToContainerOptions); err != nil {
panic(err)
}
No error occurs, but this immediatley gets passed over without streaming anything to the console. How do I get this to stream to the console until the docker cmd completes?
Attach only works on a running container and the container lifetime for that image is ephemeral. Try *Client.Logs, instead, to get the resulting logs.
Here is a code sample:
logsOptions := docker.LogsOptions{
Container: container.ID,
OutputStream: os.Stdout,
ErrorStream: os.Stderr,
Follow: true,
Stdout: true,
Stderr: true,
}
if err := client.Logs(logsOptions); err != nil {
panic(err)
}
Using the docker golang sdk the following method can be used to create a container and bind it's output to stdout.
resp, err := cli.ContainerCreate(ctx, &container.Config{
Image: "alpine",
Cmd: []string{"echo", "Hello World"},
AttachStdout: true,
}, nil, nil, "")
How can I redirect this output to a file using the SDK ?
I'm using the official SDK of docker - github.com/docker/docker/client
You can use something like below
out, err := cli.ContainerLogs(ctx, resp.ID, types.ContainerLogsOptions{ShowStdout: true})
if err != nil {
panic(err)
}
f, err := os.Create("/tmp/clogs")
io.Copy(f, out)
But make sure to to do that after you have started the container, Create will only create the container and not start it
The format of docker logs contains 8 bytes of header for each message, indicating for instance whether the output was on stdout or stderr. So one cannot simply copy the log output to a destination as Tarun Lalwani is mentioning in the other answer, because the header would then be interpreted as characters, garbling the output.
Unfortunately the client docs don't even mention the issue. This article explains it a bit and offers a library to solve the issue:
import (
"github.com/docker/docker/client"
"github.com/ahmetb/dlog"
)
// ---
reader, err := cli.ContainerLogs(ctx, resp.ID, nil)
if err != nil {
panic(err)
}
file, err := os.Create("/path/to/your/file")
io.Copy(file, dlog.NewReader(reader))