How to retrieve stdout when piping several commands in Go - docker

I'm writing a Go function to upload local Docker images to an instance that has SSH access. I'm using the following method for this:
cmdSave := exec.Command("docker", "save", image)
cmdLoad := exec.Command("ssh", fmt.Sprintf("%s#%s", user, externalIP), "docker load")
read, write := io.Pipe()
cmdSave.Stdout = write
cmdLoad.Stdin = read
var buffer bytes.Buffer
cmdLoad.Stdout = &buffer
var stderrSave bytes.Buffer
cmdSave.Stderr = &stderrSave
var stderrLoad bytes.Buffer
cmdLoad.Stderr = &stderrLoad
err = cmdSave.Start()
if err != nil {
return errors.Wrap(err, stderrSave.String())
}
err = cmdLoad.Start()
if err != nil {
return errors.Wrap(err, stderrLoad.String())
}
err = cmdSave.Wait()
if err != nil {
return errors.Wrap(err, stderrSave.String())
}
err = write.Close()
if err != nil {
return err
}
err = cmdLoad.Wait()
if err != nil {
return errors.Wrap(err, stderrLoad.String())
}
_, err = io.Copy(os.Stdout, &buffer)
if err != nil {
return err
}
However because of the way the commands are piped, if there is a problem with e.g. the SSH key or WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! error presents itself when SSHing to the instance, the functions just hangs endlessly on the cmdSave.Wait() and no output is displayed, leaving the user unaware of what the problem is.
Is there a way to do this so that the output isn't swallowed and the function isn't left hanging?

Related

Docker exec command from golang api

Need help.I have code to exec command from docker container. Need corrently get stdout from exec command.
execConfig:= types.ExecConfig{Tty:false,AttachStdout:true,AttachStderr:false,Cmd:command}
respIdExecCreate,err := cli.ContainerExecCreate(context.Background(),dockerName,execConfig)
if err != nil {
fmt.Println(err)
}
respId,err:=cli.ContainerExecAttach(context.Background(),respIdExecCreate.ID,types.ExecStartCheck{})
if err != nil {
fmt.Println(err)
}
scanner := bufio.NewScanner(respId.Reader)
for scanner.Scan() {
fmt.Println(output)
}
From output i see interesting situation:
Screen from gyazo
How corrently remove bytes ?
I send simply command := []string{"echo","-n", "hello word"}
I've faced with same issue, this is how stderr and stdout looks for me:
StdOut: "\x01\x00\x00\x00\x00\x00\x00\thello world\n"
StdErr: "\x01\x00\x00\x00\x00\x00\x00fError: Exec command has already run\r\n"
I've cheched docker source code and found answer here:
https://github.com/moby/moby/blob/8e610b2b55bfd1bfa9436ab110d311f5e8a74dcb/integration/internal/container/exec.go#L38
looks like this leading bytes used especially for marking stdout and stderr bytes.
And there is a library "github.com/docker/docker/pkg/stdcopy" which can split stdout and stderr from stream reader:
type ExecResult struct {
StdOut string
StdErr string
ExitCode int
}
func Exec(ctx context.Context, containerID string, command []string) (types.IDResponse, error) {
docker, err := client.NewEnvClient()
if err != nil {
return types.IDResponse{}, err
}
defer closer(docker)
config := types.ExecConfig{
AttachStderr: true,
AttachStdout: true,
Cmd: command,
}
return docker.ContainerExecCreate(ctx, containerID, config)
}
func InspectExecResp(ctx context.Context, id string) (ExecResult, error) {
var execResult ExecResult
docker, err := client.NewEnvClient()
if err != nil {
return execResult, err
}
defer closer(docker)
resp, err := docker.ContainerExecAttach(ctx, id, types.ExecConfig{})
if err != nil {
return execResult, err
}
defer resp.Close()
// read the output
var outBuf, errBuf bytes.Buffer
outputDone := make(chan error)
go func() {
// StdCopy demultiplexes the stream into two buffers
_, err = stdcopy.StdCopy(&outBuf, &errBuf, resp.Reader)
outputDone <- err
}()
select {
case err := <-outputDone:
if err != nil {
return execResult, err
}
break
case <-ctx.Done():
return execResult, ctx.Err()
}
stdout, err := ioutil.ReadAll(&outBuf)
if err != nil {
return execResult, err
}
stderr, err := ioutil.ReadAll(&errBuf)
if err != nil {
return execResult, err
}
res, err := docker.ContainerExecInspect(ctx, id)
if err != nil {
return execResult, err
}
execResult.ExitCode = res.ExitCode
execResult.StdOut = string(stdout)
execResult.StdErr = string(stderr)
return execResult, nil
}

kubernetes attach pod use websocket

I want make a web terminal by kubernetes client-go api, and now i meet some problem.
main:
func main() {
flag.Parse()
log.SetFlags(0)
http.HandleFunc("/ws", echo)
log.Fatal(http.ListenAndServe(*addr, nil))
}
struct&handleFunc:
type Cmd struct {
Conn *websocket.Conn
Stdin io.Reader
Stdout io.Writer
Stderr io.Writer
}
func echo(w http.ResponseWriter, r *http.Request) {
cmd := Cmd{}
cmd.Stderr = ioutil.Discard
c, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Print("upgrade:", err)
return
}
cmd.Conn = c
cmd.handleClient()
}
func (c *Cmd) handleClient() {
//c.Stdout = os.Stdout
//c.Stdin = os.Stdin
var read io.Reader
var write io.Writer
c.Stdin = read
c.Stdout = write
go func(rd io.Reader) {
for {
_, r, err := c.Conn.NextReader()
if err != nil {
fmt.Println("ReadErr:", err)
c.Conn.Close()
return
}
rd = r
}
}(read)
go func(wr io.Writer) {
w, err := c.Conn.NextWriter(1)
if err != nil {
fmt.Println("WriteErr:", err)
c.Conn.Close()
return
}
wr = w
}(write)
err := api.ExecOperator{}.ExecConsoleInContainer("1", "nginx-65899c769f-mkswf", c.Stdin, c.Stdout, c.Stderr)
if err != nil {
fmt.Println("ExecErr:", err)
return
}
}
api.ExecOperator{}.ExecConsoleInContainer() is a func invoke kubernetes exec api like kubernetes exec_util.go
When i use the code below,i can operate at the console,but i dont know how to use websocket(gorilla/websocket) input and output to replace os.stdin and os.stdout,i tried to write some of the code above ,but it cant work.
c.Stdout = os.Stdout
c.Stdin = os.Stdin

Control GPIO of Rasberry PI using chaincode - Hyperledger Fabric V1.0

I'm trying to control LED using rpi that is a peer in hyperledger fabric network. My network (PC and RPI) is as follow:
A Certificate Authority (CA) — PC1
An Orderer — PC1
1 PEER (peer0) on — PC1
1 PEER (peer1) on — RP
CLI on —  RPI
I setup the above network successfully, and i'm able to run the chaincode example 2 that is provided by hyperledger fabric (install, query,invoke).
i have developed my own chaincode that can be used to turn on and off a LED connected to the rpi from the blockchain. The problem is that i'm getting the below error (Timeout):
Error: Error endorsing chaincode: rpc error: code = Unknown desc =
timeout expired while starting chaincode mychaincode:1.0
This is my chaincode:
package main
import (
"fmt"
"strconv"
"os"
"github.com/stianeikeland/go-rpio" //To Map GPIO of RPI
"github.com/hyperledger/fabric/core/chaincode/shim"
pb "github.com/hyperledger/fabric/protos/peer"
)
var (
pin = rpio.Pin(3)
)
// SimpleChaincode example simple Chaincode implementation
type SimpleChaincode struct {
}
func (t *SimpleChaincode) Init(stub shim.ChaincodeStubInterface) pb.Response {
fmt.Println("LED ON")
_, args := stub.GetFunctionAndParameters()
var A string // Entities
var Aval int // Asset holdings
var err error
if len(args) != 2 {
return shim.Error("Incorrect number of arguments. Expecting 2")
}
if err := rpio.Open(); err != nil {
fmt.Println(err)
os.Exit(1)
}
A = args[0]
Aval, err = strconv.Atoi(args[1])
if err != nil {
return shim.Error("Expecting integer value for asset holding")
}
fmt.Printf("Aval = %d", Aval)
if Aval == 1{
pin.High() //turn on
} else {
pin.Low() //turn off
}
// Write the state to the ledger
err = stub.PutState(A, []byte(strconv.Itoa(Aval)))
if err != nil {
return shim.Error(err.Error())
}
return shim.Success(nil)
}
func (t *SimpleChaincode) Invoke(stub shim.ChaincodeStubInterface) pb.Response {
fmt.Println("ex02 Invoke")
function, args := stub.GetFunctionAndParameters()
if function == "invoke" {
// Make payment of X units from A to B
return t.invoke(stub, args)
} else if function == "delete" {
// Deletes an entity from its state
return t.delete(stub, args)
} else if function == "query" {
// the old "Query" is now implemtned in invoke
return t.query(stub, args)
}
return shim.Error("Invalid invoke function name. Expecting \"invoke\" \"delete\" \"query\"")
}
// Transaction makes payment of X units from A to B
func (t *SimpleChaincode) invoke(stub shim.ChaincodeStubInterface, args []string) pb.Response {
var A string // Entities
var Aval int // Asset holdings
var X int // Transaction value
var err error
if len(args) != 2 {
return shim.Error("Incorrect number of arguments. Expecting 2")
}
A = args[0]
// Get the state from the ledger
// TODO: will be nice to have a GetAllState call to ledger
Avalbytes, err := stub.GetState(A)
if err != nil {
return shim.Error("Failed to get state")
}
if Avalbytes == nil {
return shim.Error("Entity not found")
}
Aval, _ = strconv.Atoi(string(Avalbytes))
// Perform the execution
X, err = strconv.Atoi(args[1])
if err != nil {
return shim.Error("Invalid transaction amount, expecting a integer value")
}
Aval = X
fmt.Printf("Aval = %d", Aval)
if Aval == 1{
pin.High()
} else {
pin.Low()
}
// Write the state back to the ledger
err = stub.PutState(A, []byte(strconv.Itoa(Aval)))
if err != nil {
return shim.Error(err.Error())
}
return shim.Success(nil)
}
// Deletes an entity from state
func (t *SimpleChaincode) delete(stub shim.ChaincodeStubInterface, args []string) pb.Response {
if len(args) != 1 {
return shim.Error("Incorrect number of arguments. Expecting 1")
}
A := args[0]
// Delete the key from the state in ledger
err := stub.DelState(A)
if err != nil {
return shim.Error("Failed to delete state")
}
return shim.Success(nil)
}
// query callback representing the query of a chaincode
func (t *SimpleChaincode) query(stub shim.ChaincodeStubInterface, args []string) pb.Response {
var A string // Entities
var err error
if len(args) != 1 {
return shim.Error("Incorrect number of arguments. Expecting name of the person to query")
}
A = args[0]
// Get the state from the ledger
Avalbytes, err := stub.GetState(A)
if err != nil {
jsonResp := "{\"Error\":\"Failed to get state for " + A + "\"}"
return shim.Error(jsonResp)
}
if Avalbytes == nil {
jsonResp := "{\"Error\":\"Nil amount for " + A + "\"}"
return shim.Error(jsonResp)
}
jsonResp := "{\"Name\":\"" + A + "\",\"Amount\":\"" + string(Avalbytes) + "\"}"
fmt.Printf("Query Response:%s\n", jsonResp)
return shim.Success(Avalbytes)
}
func main() {
err := shim.Start(new(SimpleChaincode))
if err != nil {
fmt.Printf("Error starting Simple chaincode: %s", err)
}
}
How can i solve this issue as there is no syntax errors at all ?

How can I get Docker container output through a websocket?

I am trying to send output from a docker container to the console using fmt, but when trying to do it i get this.
&{0xc0422a65c0 {0 0} false <nil> 0x6415a0 0x641540}
How do I do this? This is my full code.
func main() {
imageName := "hidden/hidden"
ctx := context.Background()
cli, err := client.NewClient("tcp://0.0.0.0:0000", "v0.00", nil, nil)
if err != nil {
panic(err)
}
fmt.Println("Pulling \"" + imageName + "\"")
_, err = cli.ImagePull(ctx, imageName, types.ImagePullOptions{})
if err != nil {
panic(err)
}
containerConfig := &container.Config{
Image: imageName,
Cmd: []string{"./app/start.sh", "--no-wizard"},
}
resp, err := cli.ContainerCreate(ctx, containerConfig, nil, nil, "")
if err != nil {
panic(err)
}
if err := cli.ContainerStart(ctx, resp.ID, types.ContainerStartOptions{}); err != nil {
panic(err)
}
timer := time.NewTimer(time.Minute)
go func() {
<-timer.C
if err := cli.ContainerStop(ctx, resp.ID, nil); err != nil {
panic(err)
}
}()
out, err := cli.ContainerLogs(ctx, resp.ID, types.ContainerLogsOptions{ShowStdout: true, ShowStderr: true, Follow: true})
if err != nil {
panic(err)
}
io.Copy(os.Stdout, out) // This is what I want to change to print with "fmt".
}
Tried: (but does not display until container is done.)
buf := new(bytes.Buffer)
buf.ReadFrom(out)
fmt.Println(buf.String())
Intention: Allow real-time console output to the web.
This seems to be the answer to my question, I did some searching about scanners as Cerise Limón commented. Anyone else who seems to be having the issue that I did can use this code. Thanks to all that helped.
scanner := bufio.NewScanner(out)
scanner.Split(bufio.ScanLines)
for scanner.Scan() {
fmt.Println(scanner.Text())
}
&{0xc0422a65c0 {0 0} false <nil> 0x6415a0 0x641540} is not a bad output. This is a perfectly fine struct output. I think the the main problem is here just your lack of golang experience.
I'm a beginner as well and I can imagine that when you see an output like above you thought that "I made a mistake"
No you didn't. It's default fmt behavior when u try to print out a struct which contains pointers.
Checkout this instead of your fmt.Println:
fmt.Printf("%+v\n", out)
Well this answer stands on my assumptions but if this is the case just ping me for more info.

How the docker container id is generated

I wanted to know how the container id is generated so please provide the source code that provides the container id when the docker run is executed?
Here is a code snippet from docker daemon's function for creating Containers:
func (daemon *Daemon) newContainer(name string, config *runconfig.Config, imgID string) (*Container, error) {
var (
id string
err error
)
id, name, err = daemon.generateIDAndName(name)
if err != nil {
return nil, err
}
…
base := daemon.newBaseContainer(id)
…
base.ExecDriver = daemon.execDriver.Name()
return &base, err
}
So, the logic of creating ID and Name is in generateIDAndName function:
func (daemon *Daemon) generateIDAndName(name string) (string, string, error) {
var (
err error
id = stringid.GenerateNonCryptoID()
)
if name == "" {
if name, err = daemon.generateNewName(id); err != nil {
return "", "", err
}
return id, name, nil
}
if name, err = daemon.reserveName(id, name); err != nil {
return "", "", err
}
return id, name, nil
}
Here is stringid sources and the concrete method is generateID with false as input parameter:
func generateID(crypto bool) string {
b := make([]byte, 32)
var r io.Reader = random.Reader
if crypto {
r = rand.Reader
}
for {
if _, err := io.ReadFull(r, b); err != nil {
panic(err) // This shouldn't happen
}
id := hex.EncodeToString(b)
// if we try to parse the truncated for as an int and we don't have
// an error then the value is all numberic and causes issues when
// used as a hostname. ref #3869
if _, err := strconv.ParseInt(TruncateID(id), 10, 64); err == nil {
continue
}
return id
}
}
As you can see, the value is randomly generated with this random
// Reader is a global, shared instance of a pseudorandom bytes generator.
// It doesn't consume entropy.
var Reader io.Reader = &reader{rnd: Rand}

Resources