Cannot connect to FTP server using Go but can connect using FileZilla - docker

I have a small Golang program and I'm trying to connect to an FTP server running in a docker container (https://registry.hub.docker.com/r/atmoz/sftp).
My machine is a M1 Pro MacBook.
The container is started with the following command:
docker run -p 22:22 -d atmoz/sftp foo:pass:::upload
The Go version is 1.17.13.
The code code of the program is the following:
package main
import (
"log"
"time"
"github.com/jlaffaye/ftp"
)
func main() {
c, err := ftp.Dial("localhost:22", ftp.DialWithTimeout(5*time.Second))
if err != nil {
log.Fatal(err, " cannot connect")
}
err = c.Login("foo", "pass")
if err != nil {
log.Fatal(err, "cannot login")
}
// Do something with the FTP conn
if err := c.Quit(); err != nil {
log.Fatal(err)
}
}
Somehow, I'm unable to connect to the FTP server executing this code, it results in the following output:
EOF cannot connect
I tried connect to the same FTP server using FileZilla and it works fine, im able to connect to the server with success.
Any ideias on how to fix this or further debug the issue? Thank you

The port 22 is typically SSH/SFTP, not FTP. Note that FileZilla supports both FTP and SFTP. So chances are that you are actually connecting with SFTP using FileZilla. Those two protocols are completely different and incompatible.
There seems to be an "sftp" package for Go:
https://pkg.go.dev/github.com/pkg/sftp

Related

query a docker registry (hub.docker.com) using go docker client without docker daemon dependency

I'm trying to access a docker registry (public or private) using Go. A simple program which can access any registry and verify if an image is present.
I looked at docker client available in Go https://pkg.go.dev/github.com/docker/docker#v20.10.11+incompatible/client
But the problem is, this client needs a docker daemon running in order to work. Is there any way to query a docker registry (ex: hub.docker.com) without any dependency on underlying docker engine?
My idea is to run this program on a docker container and there wont be any docker engine running inside a container. And I don't want to run docker inside docker or any sort of hack. I just want to connect to a registry and query an image. And please don't quote other questions in stack overflow. No one has answered this.
This is what I have done so far
import (
"context"
"encoding/base64"
"encoding/json"
"fmt"
"github.com/docker/docker/api/types/filters"
"time"
"github.com/docker/docker/api/types"
"github.com/docker/docker/client"
)
func main() {
cli, err := client.NewClientWithOpts(client.WithHost("https://hub.docker.com"), client.WithAPIVersionNegotiation())
if err != nil {
fmt.Println(err.Error())
return
}
err = imagemanifest(cli)
if err != nil {
fmt.Println(err)
}
err = imageSearch(cli)
}
func imagemanifest(dockerClient *client.Client) error {
var authConfig = types.AuthConfig{
Username: "amokkara",
Password: "M#vr1ck2009",
ServerAddress: "https://index.docker.io/v2/",
}
ctx, cancel := context.WithTimeout(context.Background(), time.Second*1200)
defer cancel()
authConfigBytes, _ := json.Marshal(authConfig)
authConfigEncoded := base64.URLEncoding.EncodeToString(authConfigBytes)
ctx, cancel = context.WithTimeout(context.Background(), time.Second*1200)
defer cancel()
searchres , err := dockerClient.DistributionInspect(ctx,"amokkara/amokkara:3",authConfigEncoded)
if err != nil {
return err
}
fmt.Println(searchres.Descriptor.Digest.String())
return nil
}
If I initialize client like this
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
This works because its using underlying docker daemon (in my case docker desktop) to query the registry. But if create client using
client.NewClientWithOpts(client.WithHost("https://hub.docker.com"), client.WithAPIVersionNegotiation())
it fails giving 404 error. Does this client require a docker daemon to work. If so, is there any other way i can query a registry? Please help me with this.
Skopeo is the leader software on dealing with registries without daemon.
It's also written with Go.
You can inspire from inspect.go
Note that, you don't need to use github.com/docker/docker/* modules, but it will be github.com/containers/*, and namely https://github.com/containers/image

The neo4j cypher shell and the browser connections are working but the golang client connection is not working

I have disabled the authentication on my neo4j server, so I can connect using the cypher shell using no credentials as it follows and is working.
$ ./bin/cypher-shell -a 192.168.0.89
This is how I'm declaring my driver and the session, I also tried using neo4j://* instead of bolt://*:
driver, err := neo4j.NewDriver("bolt://192.168.0.89:7687", neo4j.NoAuth())
if err != nil {
return "", err
}
defer driver.Close()
session, _ := driver.NewSession(neo4j.SessionConfig{AccessMode: neo4j.AccessModeWrite})
defer session.Close()
But that doesn't work either. I'm getting this error when running the hello world from the neo4j olang driver page https://neo4j.com/developer/go/
TLS error: Remote end closed the connection, check that TLS is enabled on the server
There are the logs of the server when it starts:
2021-03-07 23:17:23.227+0000 INFO ======== Neo4j 4.2.3 ========
2021-03-07 23:17:24.119+0000 INFO Performing postInitialization step for component 'security-users' with version 2 and status CURRENT
2021-03-07 23:17:24.119+0000 INFO Updating the initial password in component 'security-users'
2021-03-07 23:17:24.243+0000 INFO Bolt enabled on 192.168.0.89:7687.
2021-03-07 23:17:25.139+0000 INFO Remote interface available at http://192.168.0.89:7474/
2021-03-07 23:17:25.140+0000 INFO Started.
These are all my config settings:
dbms.connector.bolt.advertised_address=192.168.0.89:7687
dbms.connector.bolt.enabled=true
dbms.connector.bolt.listen_address=192.168.0.89:7687
dbms.connector.bolt.tls_level=DISABLED
dbms.connector.http.advertised_address=192.168.0.89:7474
dbms.connector.http.enabled=true
dbms.connector.http.listen_address=192.168.0.89:7474
dbms.connector.https.enabled=false
dbms.default_advertised_address=192.168.0.89
dbms.default_database=neo4j
dbms.default_listen_address=192.168.0.89
dbms.directories.import=/home/eduardo/NEO4J/import
dbms.directories.neo4j_home=/home/eduardo/NEO4J
dbms.jvm.additional=-Dlog4j2.disable.jmx=true
dbms.security.auth_enabled=false
dbms.tx_log.rotation.retention_policy=1 days
dbms.tx_state.memory_allocation=ON_HEAP
dbms.windows_service_name=neo4j
Again, I can connect to the same host and the browser is also working fine:
Thanks in advance for any help :)
Adding to your answer: it is likely you're using the v1.x of the Go driver. If you switch to using the v4.x driver instead, you will not have to specify this config value.
You can upgrade by simply adding v4 in your import statement like so:
import github.com/neo4j/neo4j-go-driver/v4/neo4j
More info: https://github.com/neo4j/neo4j-go-driver/blob/4.2/MIGRATIONGUIDE.md
For anyone looking for the answer, the bolt driver will try to use TLS by default and since in my case is not configured, the encryption needs to be disabled in the driver constructor call.
driver, err := neo4j.NewDriver("bolt://192.168.0.89:7687", neo4j.NoAuth(), func(c *neo4j.Config) { c.Encrypted = false })
Hope this helps other people experiencing the same issue :)

Golang Docker API - Tail container logs

I'm using Go and hitting Docker's API to pull an image, and then create and execute a container. Specifically I'm using the docker-newman image. I can see that it's actually being executed in Kitematic, so I know that everything is setup correctly. However, my Go application isn't attaching and then tailing the logs being output.
I've seen this answer, and it references what appears to be the way to attach to the image and view the log. I can't seem to get it to work regardless of what I try.
attachToContainerOptions := docker.AttachToContainerOptions{
Container: container.ID,
OutputStream: os.Stdout,
ErrorStream: os.Stderr,
Logs: true,
Stdout: true,
Stderr: true,
}
if err := client.AttachToContainer(attachToContainerOptions); err != nil {
panic(err)
}
No error occurs, but this immediatley gets passed over without streaming anything to the console. How do I get this to stream to the console until the docker cmd completes?
Attach only works on a running container and the container lifetime for that image is ephemeral. Try *Client.Logs, instead, to get the resulting logs.
Here is a code sample:
logsOptions := docker.LogsOptions{
Container: container.ID,
OutputStream: os.Stdout,
ErrorStream: os.Stderr,
Follow: true,
Stdout: true,
Stderr: true,
}
if err := client.Logs(logsOptions); err != nil {
panic(err)
}

Golang http.Get displays error EOF in Container

I have a golang script that calls an HTTP API. Everything works great when I run it on my development computer. Once I create the container and run it I get an EOF at the end of the URL on panic. I have read a lot of issues like this and have tried everyone I see. I added a ca-certificates.crt from my host machine to the container to /etc/ssl/certs/. I set the request close to true, I have disabled Keep-Alive and Compression. All of these steps worked for others on other posts. Any help appreciated.
I got my certfile from https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
I am running go version go1.9.2 darwin/amd64 and the container is on a ubuntu 16.04 host.
client := &http.Client{Timeout: 30 * time.Second, Transport: &http.Transport{
DisableCompression:true,
DisableKeepAlives: true,
}}
url := "https://myapiurl.com"
req, err := http.NewRequest("POST", url, bytes.NewBuffer(mybodybytes))
req.Close = true
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
log.Panic(err) // Panics Here
}
defer resp.Body.Close()
...
Dockerfile:
FROM scratch
ADD ca-certificates.crt /etc/ssl/certs/
ADD myservice /
CMD ["/myservice"]

Docker Golang SDK - How to redirect container stdout to a file

Using the docker golang sdk the following method can be used to create a container and bind it's output to stdout.
resp, err := cli.ContainerCreate(ctx, &container.Config{
Image: "alpine",
Cmd: []string{"echo", "Hello World"},
AttachStdout: true,
}, nil, nil, "")
How can I redirect this output to a file using the SDK ?
I'm using the official SDK of docker - github.com/docker/docker/client
You can use something like below
out, err := cli.ContainerLogs(ctx, resp.ID, types.ContainerLogsOptions{ShowStdout: true})
if err != nil {
panic(err)
}
f, err := os.Create("/tmp/clogs")
io.Copy(f, out)
But make sure to to do that after you have started the container, Create will only create the container and not start it
The format of docker logs contains 8 bytes of header for each message, indicating for instance whether the output was on stdout or stderr. So one cannot simply copy the log output to a destination as Tarun Lalwani is mentioning in the other answer, because the header would then be interpreted as characters, garbling the output.
Unfortunately the client docs don't even mention the issue. This article explains it a bit and offers a library to solve the issue:
import (
"github.com/docker/docker/client"
"github.com/ahmetb/dlog"
)
// ---
reader, err := cli.ContainerLogs(ctx, resp.ID, nil)
if err != nil {
panic(err)
}
file, err := os.Create("/path/to/your/file")
io.Copy(file, dlog.NewReader(reader))

Resources