I am trying to create a container for testing purpose with https://pkg.go.dev/github.com/fsouza/go-dockerclient?tab=doc as follows:
client, err := docker.NewClientFromEnv()
NoError(t, err, err)
ctx := context.Background()
gogs, err := client.CreateContainer(docker.CreateContainerOptions{
Name: "gogs",
Config: &docker.Config{
Image: "gogs/gogs:0.11.91",
},
HostConfig: &docker.HostConfig{
PublishAllPorts: true,
AutoRemove: true,
Binds: []string{dest, "/data"},
PortBindings: map[docker.Port][]docker.PortBinding{
"3000/tcp": {{HostIP: "0.0.0.0", HostPort: "8888"}}},
},
Context: ctx,
})
NoError(t, err, err)
err = client.StartContainer(gogs.ID, nil)
NoError(t, err, err)
The problem is, the bind does not work. What am I trying to achieve is:
docker run --rm -it -p 8888:3000 --name=gogs --mount type=bind,source=`pwd`/gogs/data,target=/data gogs/gogs
What am I doing wrong?
When mounting a path you can attach Mounts to the HostConfig like this:
&container.HostConfig{
Mounts: []mount.Mount{
{
Type: mount.TypeBind,
Source: "/source",
Target: "/target",
},
},
}
This will mount /source in to the container.
Related
Motivation
I'm running this command inside the container:
docker run -it --rm \
--mount type=volume,src=synapse-data,dst=/data \
-e SYNAPSE_SERVER_NAME=my.matrix.host \
-e SYNAPSE_REPORT_STATS=yes \
matrixdotorg/synapse:latest generate
Based on https://github.com/matrix-org/synapse/tree/v1.56.0/docker
Docker SDK usage
And I'm using this abstraction: https://pkg.go.dev/github.com/docker/docker/client#Client.ContainerCreate
As a general concept I want to use:
AutoRemove: true,
The point is to automate/enforce containers deletion after use, for instance, if the setup exits unexpectedly. I'm also using a container name: server_setup_temporary_container which hints the user that this is used during setup and is meant to be temporary. In case the setup did not shutdown the container, the user can do this and the bound volumes are freed.
My problem with this generate script
I can't use https://github.com/moby/moby/blob/v20.10.18/client/container_logs.go#L36 as the container exits once it finished executing the generate. Therefore I can't access the logs at all as they are already deleted.
In contrast, this works well with the postgresql container, as it runs as a daemon and needs explicit shutdown. The same concept fails with only executing a script!
I don't know how to continue here.
A few thoughts I had:
after generate execute a 'sleep 3600' and then explicitly shut the container down as well
try to get the logs from ContainerStart or ContainerCreate directly but studying the API this is probably not implemented this way
What I would not want is to remove the AutoRemove: true concept.
The source code
Using my StartContainer abstraction
// Start and run container
containerId, err := s.dockerClient.myStartContainer(docker.ContainerStartConfig{
Image: matrixImage,
Volumes: []docker.ContainerVolume{
docker.ContainerVolume{
Source: volume,
Target: "/data",
},
},
Env: []string{
fmt.Sprintf("SYNAPSE_SERVER_NAME=%s", domain),
"SYNAPSE_REPORT_STATS=no",
},
Cmds: []string{
"generate",
},
})
StartContainer abstraction
func (c *Client) myStartContainer(cfg ContainerStartConfig) (string, error) {
if c.client == nil {
return "", errors.New(noClientErr)
}
if len(cfg.Image) == 0 {
return "", errors.New(noImageErr)
}
containerConfig := container.Config{
Image: cfg.Image,
}
hostConfig := container.HostConfig{
AutoRemove: true,
}
if cfg.Env != nil {
containerConfig.Env = cfg.Env
}
if cfg.Cmds != nil {
containerConfig.Cmd = make(strslice.StrSlice, len(cfg.Cmds))
for i, _cmd := range cfg.Cmds {
containerConfig.Cmd[i] = _cmd
}
}
if cfg.Volumes != nil {
hostConfig.Mounts = make([]mount.Mount, len(cfg.Volumes))
for i, v := range cfg.Volumes {
hostConfig.Mounts[i] = mount.Mount{
Type: "volume",
Source: v.Source,
Target: v.Target,
}
}
}
var networkingConfig *network.NetworkingConfig
if cfg.Networks != nil {
networkingConfig = &network.NetworkingConfig{EndpointsConfig: map[string]*network.EndpointSettings{}}
for _, nw := range cfg.Networks {
n := nw.Name
networkingConfig.EndpointsConfig[n] = &network.EndpointSettings{Aliases: nw.Aliases}
}
}
cont, err := c.client.ContainerCreate(
c.ctx,
&containerConfig,
&hostConfig,
networkingConfig,
nil,
"server_setup_temporary_container",
)
if err != nil {
return "", err
}
colorlogger.Log.Info("Container ID of "+colorlogger.LYellow, cfg.Image, colorlogger.CLR+" is "+cont.ID)
if err := c.client.ContainerStart(c.ctx, cont.ID, types.ContainerStartOptions{}); err != nil {
return "", err
}
return cont.ID, nil
}
Greater scenario
I'm executing this setup in order to configure the containers which are later executed with 'docker compose' as some of the setups require explicit changes to the containers and can't be done declaratively.
On host machine I have docker-compose stack that have a service called
transactions-db
its a PostgreSQL container
I want to create a backup service using Golang to be able to create .sql to be used for restores later on
docker-compose.yml for backup service
version: "3.8"
services:
backup-service:
build:
context: .
dockerfile: Dockerfile
image: backup-service
container_name: backup-service
volumes:
- .:/app
- /var/run/docker.sock:/var/run/docker.sock
main.go
package main
import (
"context"
"fmt"
"os"
)
func main () {
result, err := ExecuteCommandOnContainer(context.Background(), ConnectToDocker(), "transactions-db")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Println(result.ExitCode)
}
docker.go
package main
import (
"context"
"fmt"
"github.com/docker/docker/api/types"
"github.com/docker/docker/client"
)
func ConnectToDocker() *client.Client {
cli, err := client.NewClientWithOpts()
if err != nil {
fmt .Println(err)
}
return cli
}
func ExecuteCommandOnContainer(ctx context.Context, cli *client.Client, containerID string) (types.ContainerExecInspect, error) {
command := []string{"pg_dumpall", "-c", "-U", "user", "|", "gzip", ">", "backup/tr.sql"}
execConfig := types.ExecConfig{
Tty: true,
AttachStdin: true,
AttachStderr: true,
AttachStdout: true,
Cmd: command,
}
create, err := cli.ContainerExecCreate(ctx, containerID, execConfig)
if err != nil {
fmt .Println(err)
}
inspect, err := cli.ContainerExecInspect(ctx, create.ID)
if err != nil {
fmt .Println(err)
}
return inspect, nil
}
basically nothing happens when I run this code. I only get the following:
backup-service | 0
backup-service exited with code 0
I tried to normally run: go run .
I've also tried using my docker-compose up
same result
note: I've tested listing container it works but when trying to execute a command nothing
I am trying to run docker from golang and when I tried the code mentioned in the docker official
site, am getting these error. wondering if I have an incorrect vendor
resp, err := cli.ContainerCreate(ctx,
&container.Config{
Image: imageName,
},
nil,
nil,
"")
not enough arguments in call to cli.ContainerCreate
have (context.Context, *container.Config, nil, nil, string)
want (context.Context, *container.Config, *container.HostConfig, *network.NetworkingConfig, *v1.Platform, string)
In this example, you can see what *v1.Platform can be initialized to:
resp, err := cli.ContainerCreate(ctx, &container.Config{Hostname: "my-rabbit",
Image: "rabbitmq:3.7.8-management",
Tty: true,
}, &container.HostConfig{RestartPolicy: container.RestartPolicy{Name: "always"}, PortBindings: bindings}, &network.NetworkingConfig{}, "rabbit")
if err != nil {
panic(err)
}
The &network.NetworkingConfig{} references github.com/docker/docker/api/types/network#NetworkingConfig
Carries the networking configs specified in the docker run and docker network connect commands
I have created a DroneCI pipeline with the following content:
kind: pipeline
type: docker
name: Build auto git tagger
steps:
- name: test and build
image: golang
commands:
- go mod download
- go test ./test
- go build -o ./build/package ./cmd/git-tagger
- name: Build docker image
image: plugins/docker
pull: if-not-exists
settings:
username:
password:
repo:
dockerfile:
registry:
auto_tag:
trigger:
branch:
- master
The go test starts a gogs docker container for testing purpose, here is the code:
func createGogsContainer(dest, waitUrl string) (stopContainer, error) {
client, err := docker.NewClientFromEnv()
if err != nil {
return nil, err
}
ctx := context.Background()
gogs, err := client.CreateContainer(docker.CreateContainerOptions{
Name: "repo",
Config: &docker.Config{
Image: "gogs/gogs",
},
HostConfig: &docker.HostConfig{
PublishAllPorts: true,
AutoRemove: true,
Mounts: []docker.HostMount{
{
Type: "bind",
Source: dest,
Target: "/data",
}},
PortBindings: map[docker.Port][]docker.PortBinding{
"3000/tcp": {{HostIP: "0.0.0.0", HostPort: "8888"}},
"22/tcp": {{HostIP: "0.0.0.0", HostPort: "2222"}},
},
},
Context: ctx,
})
if err != nil {
return nil, err
}
err = client.StartContainer(gogs.ID, nil)
if err != nil {
return nil, err
}
//Wait for connection
host, err := url.Parse(waitUrl)
if err != nil {
return nil, err
}
err = waitHTTP(fmt.Sprintf("%s://%s", host.Scheme, host.Host), 3, 0)
if err != nil {
return nil, err
}
return func() error {
return client.StopContainerWithContext(gogs.ID, 5, ctx)
}, nil
}
The pipeline has been aborted with following error message:
latest: Pulling from library/golang
Digest: sha256:f30b0d05ea7783131d84deea3b5f4d418d9d930dfa3668a9a5fa253d1f9dce5a
Status: Image is up to date for golang:latest
+ go mod download
+ go test ./test
time="2020-04-23T17:58:24Z" level=error msg="Get \"http://0.0.0.0:8888/gat/WithoutTag.git/info/refs?service=git-upload-pack\": dial tcp 0.0.0.0:8888: connect: connection refused"
time="2020-04-23T17:58:24Z" level=error msg="Get \"http://0.0.0.0:8888/gat/WithoutTag.git/info/refs?service=git-upload-pack\": dial tcp 0.0.0.0:8888: connect: connection refused"
What am I doing wrong?
Have a look at Drone services. It allows you to bring up a container as part of your pipeline and access its ports.
In your case you can bring up the Gogs container like this:
services:
- name: gogs
image: gogs/gogs
And then use it like this in your pipeline steps:
steps:
- name: test and build
image: golang
commands:
- curl "http://gogs"
-
...
(this assumes the gogs container listens on port 80. If it's a different port then you need to adjust the URI).
Hint: the name of the service is the DNS name of the container.
I am looking forward to do something below like this using docker golang api
cmd : docker run -t -i -p 8989:8080 "image-name" /bin/bash
Also I am using golang sdk https://github.com/moby/moby/client or https://godoc.org/github.com/moby/moby/client and my docker api version is 1.30 (Client & Server both)
Here is the piece of code I am using
package main
import (
"fmt"
"github.com/docker/docker/client"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"golang.org/x/net/context"
"github.com/docker/go-connections/nat"
//"github.com/docker/docker/vendor/github.com/docker/go-connections/nat"
)
func check(err error) {
if err != nil {
panic(err)
}
}
func main(){
ctx := context.Background()
cli, err := client.NewEnvClient()
check(err)
config := &container.Config{
Image : image-name,
ExposedPorts: nat.PortSet{
"8080/tcp": struct{}{},
},
Cmd : [] string {"sh","-c","while true; do sleep always; done","/bin/bash"},
}
host_config := &container.HostConfig{
PortBindings: nat.PortMap{
"8080/tcp": []nat.PortBinding{
{
HostIP: "0.0.0.0",
HostPort: "8989",
},
},
},
}
resp, err := cli.ContainerCreate(ctx,config,host_config, nil,"")
check(err)
if err := cli.ContainerStart(ctx, resp.ID, types.ContainerStartOptions{});
err != nil {
panic(err)
}
}
After Compiling this code I get the following error
# command-line-arguments
src\main\createcontainer1.go:53: cannot use "github.com/docker/go-connections/nat".PortSet literal (type "github.com/docker/go-connections/nat".PortSet) as type "github.com/docker/docker/vendor/github.com/docker/go-connections/nat".PortSet in field value
src\main\createcontainer1.go:65: cannot use "github.com/docker/go-connections/nat".PortMap literal (type "github.com/docker/go-connections/nat".PortMap) as type "github.com/docker/docker/vendor/github.com/docker/go-connections/nat".PortMap in field value
If somebody knows what could be the problem and how to fix it.
Please answer to it as I am beginner with docker.
This is a Golang issue with how vendor/ works.
Remove the nested vendor directory:
rm -rf vendor/github.com/docker/docker/vendor
If you are using glide, you should use glide install -v when installing the dependency.
For more details, check this reported issue
My solution for OSX:
mv /Users/<user>/go/src/github.com/docker/docker/vendor/github.com/docker/go-connections/{nat,nat.old}