Golang POST request from one to another docker container - docker

I have a server in one docker container (port 5044) and client in other docker container (port 4545). I want to send POST request from client to server but i get an error message "Post "http://127.0.0.1:5044/check": dial tcp 127.0.0.1:5044: connect: connection refused".
json, err := json.Marshal(x)
if err != nil {
log.Fatal(err)
}
resp, err := http.Post("http://127.0.0.1:5044/check", "application/json", bytes.NewBuffer(json))
//Handle Error
if err != nil {
log.Fatalf("An Error Occured %v", err)
}
defer resp.Body.Close()
//Read the response body
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
return string(body)
I can send post request using postman and everything is ok.
I tried to connect to test internet service (https://ptsv2.com/) and it works as well. It seems like golang doesnt want to conenct to the local server form docker :/

your docker app is accessible from it's "external port" with Postman
But to allow them to communicate together they need to be on the same network
the easiest way to do it is to use a docker-compose (instead of manually creating the neworks) official link
version: '3'
services:
first:
build:
context: ./your-first-app
dockerfile: Dockerfile
ports:
- '1800:1800'
networks:
my-network:
second:
build:
context: ./your-second-app
dockerfile: Dockerfile
ports:
- '1801:1801'
networks:
my-network:
networks:
my-network:
your network is declared at the end of the line, and it's linked to both your Docker service via the tag networks (below ports)

The best practice to connect to the docker container is by using the container name e.g. http://app_one:5044/check, you can find the container name by using the docker ps command. in order to connect with the container name, all containers have to be in the same network.
But if you want to use IP this is the way.
Each container has a different IP address, in order to connect from one container to another you need to know the container IP address that you want to connect. 127.0.0.1 is not the container IP, it's the host IP.
you can to find the IP address of the container by this command, docker inspect your_container_name you will see the IP in networking section
"Networks": {
"bridge_network": {
"Gateway": "172.18.0.1",
"IPAddress": "172.18.0.37",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
}
}
in my example 172.18.0.37 is my container IP address. so I need to use that by to communicate e.g. http://172.18.0.37:5044/check
be aware that IPAddress if the container can change when you restart the container so it's better to use container name

Find your service name from your docker-compose.yml file for the server and replace this:
resp, err := http.Post("http://127.0.0.1:5044/check", "application/json", bytes.NewBuffer(json))
with this:
resp, err := http.Post("http://<serviceName>:5044/check", "application/json", bytes.NewBuffer(json))

Related

Unable to mount a folder in Docker from container with Go Docker Engine API

I am trying to run docker inside container using Go Docker Engine API. I am able to mount a folder from host system to the container but only empty dir is being copied into the docker inside the container. Please help me out if there is any alternative for the same. I am starting my container using following command.
docker run --rm -v C:\Users\user\source\repos\game:/app/myrepo -v /var/run/docker.sock:/var/run/docker.sock testimage
Attached is the piece of code.
Go Docker SDK code to start container
resp, err := cli.ContainerCreate(ctx, &container.Config{
Image: "hello-image",
Cmd: []string{"ls"}, #the actual cmd would look different
Tty: true,
}, &container.HostConfig{
Binds: []string{
"/app/myrepo:/myrepo",
},
}, nil, nil, containername)
if err != nil {
panic(err)
}
updated Code for binds with absolute path
resp, err := cli.ContainerCreate(ctx, &container.Config{
Image: "hello-image",
Cmd: []string{"ls"}, #the actual cmd would look different
Tty: true,
}, &container.HostConfig{
Mounts: []mount.Mount{
{
Type: mount.TypeBind,
Source: "/app/myrepo",
Target: "/myrepo",
},
},
}, nil, nil, containername)
if err != nil {
panic(err)
}
As discussed in the comments the OP is running an app in a container. The app is connecting to the docker daemon on the host (via shared /var/run/docker.sock) and attempting to create a container. The issue was that the request includes a mount point, the source being /app/myrepo, which a source path that is valid within the container but not on the host.
To aid in understanding why this is an issue you need to consider how the API request is made. Your code will generate a JSON formatted request; this will include something like this:
...
"HostConfig": {
...
"Mounts": [
{
"Source": "/app/myrepo",
"Destination": "/myrepo",
}
]
}
It's important to note that the Source path is passed as a string and the Docker Daemon will interpret this in the hosts (e.g. the windows box) context. When it attempts to locate the requested path (/app/myrepo) it will not find it because that path does not exist on the host. To correct this you need to send a valid path e.g.
Mounts: []mount.Mount{
{
Type: mount.TypeBind,
Source: "c:/Users/user/source/repos/game",
Target: "/myrepo",
},
}
One note of caution; accessing the Docker API in this way (bind mount /var/run/docker.sock:) is convenient but if someone gains access to the container then they also gain full control of all containers (because they can access the Docker API). You may want to consider using a proxy (for example).

Request between Docker containers failing dial tcp 172.18.0.6:3050: connect: connection refused

I am struggling with Go requests between containers.
The issue that I have that the rest of my containers can send request to the node Container that give response, but when I send request from my GoLang application to node I get that refuse error "dial tcp 172.18.0.6:3050: connect: connection refused".
So my whole docker set up is:
version: "3.3"
services:
##########################
### SETUP SERVER CONTAINER
##########################
node:
# Tell docker what file to build the server from
image: myUserName/mernjs:node-dev
build:
context: ./nodeMyApp
dockerfile: Dockerfile.dev
# The ports to expose
expose:
- 3050
# Port mapping
ports:
- 3050:3050
# Volumes to mount
volumes:
- ./nodeMyApp/src:/app/server/src
# Run command
# Nodemon for hot reloading (-L flag required for polling in Docker)
command: nodemon -L src/app.js
# Connect to other containers
links:
- mongo
# Restart action
restart: always
react:
ports:
- 8000:8000
build:
context: ../reactMyApp
dockerfile: Dockerfile.dev
volumes:
- ../reactMyApp:/usr/src/app
- /usr/src/app/node_modules
- /usr/src/app/.next
restart: always
environment:
- NODE_ENV=development
golang:
build:
context: ../goMyApp
environment:
- MONGO_URI=mongodb://mongo:27017
# Volumes to mount
volumes:
- ../goMyApp:/app/server
links:
- mongo
- node
restart: always
So my React app can send the request to "http://node:3050/api/greeting/name" and it get the response even that react app is not linked to the node app but when Golang app sends request to node docker container it gets connection refuse message GetJson err: Get "http://node:3050/api/greeting/name": dial tcp 172.18.0.6:3050: connect: connection refused
func GetJson(url string, target interface{}) error {
r, err := myClient.Get(url)
if err != nil {
fmt.Println("GetJson err: ", err)
return err
}
defer r.Body.Close()
return json.NewDecoder(r.Body).Decode(target)
}
type ResultsDetails struct {
Greeting string `bson:"greatingMessage" json:"greatingMessage"`
Message string `bson:"message" json:"message"`
}
func GetGreetingDetails(name string) ResultsDetails {
var resp ResultsDetails
GetJson("http://node:3050/api/greeting/"+name, &resp)
return resp
}
So how do I solve the Golang request to another Docker Node Container when docker doesnt see the host as the name of my container 'node'?
Update:
By accident i put Golang port, which it doenst run on any port since it is application that checks on database records. So it hasnt got any api, therefore it is not running on any port.
Is that could be the problem why my golang application cannot communication to other containers?
Since i have also another golang application which is api application and it is running on 5000 port and it is well communicating to my node application?
Network info:
After checking the network if node and golang share the same network and the answer is yes. All containers share the same network
(Unrelated to my issue) To anyone who has "dial tcp connection refused" issue I suggest to go though that guide https://maximorlov.com/4-reasons-why-your-docker-containers-cant-talk-to-each-other/. Really helpful. To those who this guide wont help prob read bellow this, maybe you trying to request the container api after just containers were built :D
For those who was interested what was wrong:
Technically reason why I was getting this error is because of the request that I was trying to run, was just when all containers were built.
I believe there is some delay to the network after containers are built. Thats why there host was throwing "dial tcp 172.18.0.6:3050: connect: connection refused" I've run that test on other containers that could possibly send request to that node container and they were all failing after the build time. But when re-requesting after few seconds all worked out.
Sorry to bother you guys. I really spent 3 days into this issue. And I was looking into completely wrong direction. Never thought that the issue is that silly :D
Thanks for you time.
I've met the same error in my harbor registry service.
After I docker exec -it into the container, and check if the service is available, and finally I found that http_proxy has been set.
Remove the http_proxy settings for docker service, then it works like a charm.
Failed on load rest config err:Get "http://core:8080/api/internal/configurations": dial tcp 172.22.0.8:8080: connect: connection refused
$docker exec -it harbor-jobservice /bin/bash
$echo $http_proxy $https_proxy

Socket connections between docker containers fails

I have multiple containers being deployed through a docker-compose file seen below
version: '3'
services:
module2:
restart: always
build:
dockerfile: Dockerfile
context: ./Module-2
ports:
- '16667:16667'
module3:
build:
dockerfile: Dockerfile
context: ./Module-3
ports:
- '16669:16669'
Module 2 takes a socket request from an outside source and works as intended. The trouble begins when module 2 tries to connect with module 3
Module 2 code (JAVA)
private int socket_port = 16669;
private String server = "127.0.0.1";
public TextOutputSocket() {
}
public TextOutputSocket(String host, int socket_port) {
this.server = host;
this.socket_port = socket_port;
}
public void sendText(String textToSend) {
OutputStream os = null;
Socket sock = null;
try {
System.out.println("Connecting to " + server + ":" + socket_port);
sock = new Socket(server, socket_port);
os = sock.getOutputStream();
module 3 code (GO)
ln, err := net.Listen("tcp", ":16669")
if err != nil {
fmt.Println(err)
// handle error
}
Module 2 recieves a connection refused error when ever i try to send the request.
I feel I don't have the best understanding of docker networks and i assume this is where the problem lies.
Thank you for the help in advance
In your case, when you spin up docker-compose, module2 and module3 2 containers will be in the same docker network and they can connect to each other using their DNS names i.e. module2 and module3 respectively.
As a result, you should update your module2 code to be like this
private int socket_port = 16669;
private String server = "module3";
public TextOutputSocket() {
}
...
Note that you will not need to do a port mapping like - '16667:16667' or - '16669:16669' in order for these 2 modules to talk to each other.
First you need to understand how docker containers work. Each of you applications are deployed in two seperate containers. So when trying to connect to a different container you need to give the ip or the hostname of that specific container.
Here you have tried to connect to 1669 of localhost, instead what you should be doing is try to connect to the other container. This can be done by setting the container name of the module3 container and docker dns will resolve the ip address for you.
Simple replace 127.0.0.1 with module3

No such host where i connect to kafka broker on server via port forwarding

I have following problem:
i have running instance of kafka broker on server in k8s.
This broker i have port forwarded to localhost kubectl port-forward kafka-broker-0 9104.
I'am sure it works, because I try it from my python app, but if i try example in go from internet
package main
import (
"fmt"
"os"
"os/signal"
kingpin "gopkg.in/alecthomas/kingpin.v2"
"github.com/Shopify/sarama"
)
var (
brokerList = kingpin.Flag("brokerList", "List of brokers to connect").Default("localhost:9104").Strings()
topic = kingpin.Flag("topic", "Topic name").Default("important").String()
partition = kingpin.Flag("partition", "Partition number").Default("0").String()
offsetType = kingpin.Flag("offsetType", "Offset Type (OffsetNewest | OffsetOldest)").Default("-1").Int()
messageCountStart = kingpin.Flag("messageCountStart", "Message counter start from:").Int()
)
func main() {
kingpin.Parse()
config := sarama.NewConfig()
config.Consumer.Return.Errors = true
brokers := *brokerList
master, err := sarama.NewConsumer(brokers, config)
if err != nil {
panic(err)
}
defer func() {
if err := master.Close(); err != nil {
panic(err)
}
}()
consumer, err := master.ConsumePartition(*topic, 0, sarama.OffsetOldest)
if err != nil {
panic(err)
}
signals := make(chan os.Signal, 1)
signal.Notify(signals, os.Interrupt)
doneCh := make(chan struct{})
go func() {
for {
select {
case err := <-consumer.Errors():
fmt.Println(err)
case msg := <-consumer.Messages():
*messageCountStart++
fmt.Println("Received messages", string(msg.Key), string(msg.Value))
case <-signals:
fmt.Println("Interrupt is detected")
doneCh <- struct{}{}
}
}
}()
<-doneCh
fmt.Println("Processed", *messageCountStart, "messages")
}
i've got error
panic: dial tcp: lookup host.docker.internal: no such host
goroutine 1 [running]:
main.main()
/Users/anonymous/go/src/pr1/kafka-consumer.go:37 +0x33b
exit status 2
pls, where can be a problem?
Quite simple, your process can not resolve the hostname host.docker.internal.
Go program running in a pod on k8s
Assuming you have your Go program running in a container on the Kubernetes cluster, you need to take a few measures as detailed below.
If you use Kubernetes, there should be a service which exports Kubernetes to the rest of your cluster, usually under a cluster ip and the hostname <servicename>.<namespace>.<clusterdomain>.
Below you can see how to approach that:
$ kubectl --namespace=mynamespace get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
centraldb-cockroachdb ClusterIP None <none> 26257/TCP,8080/TCP 21d
centraldb-cockroachdb-public ClusterIP 10.110.105.151 <none> 26257/TCP,8080/TCP 21d
$ kubectl get --namespace=mynamespace service centraldb-cockroachdb-public -o json | jq ".spec"
{
"clusterIP": "10.110.105.151",
"ports": [
{
"name": "grpc",
"port": 26257,
"protocol": "TCP",
"targetPort": "grpc"
},
{
"name": "http",
"port": 8080,
"protocol": "TCP",
"targetPort": "http"
}
],
"selector": {
"app.kubernetes.io/component": "cockroachdb",
"app.kubernetes.io/instance": "centraldb",
"app.kubernetes.io/name": "cockroachdb"
},
"sessionAffinity": "None",
"type": "ClusterIP"
}
As you can see, I run a cockroachdb, and its public grpc port is available via 10.110.105.151:26257, while its http port is available via 10.110.105.151:8080.
Now, how do we find out our DNS name?
We deploy a little helper into the same namespace.
First, we create a yaml file bb.yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox
spec:
containers:
- image: busybox:latest
command:
- sleep
- "7200"
name: busybox
restartPolicy: Always
Then, we apply it to the same namespace as the service we want to get the DNS name for:
$ kubectl --namespace=mynamespace apply -f bb.yaml
pod/busybox created
Now, we can find the full DNS name of your service via:
$ kubectl exec --namespace=mynamespace busybox -- nslookup centraldb-cockroachdb-public
Server: 10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
Name: centraldb-cockroachdb-public
Address 1: 10.110.105.151 centraldb-cockroachdb-public.mynamespace.svc.cluster.local
(Note that the IP dresses and DNS names obviously can be different in your installation).
So, we should be able to reach the CockroachDB web via "centraldb-cockroachdb-public.mynamespace.svc.cluster.local:8080" from within the cluster.
Go program running outside k8s
What to do here heavily depends on your installation. If you are using Docker for Desktop, as I assume, the easiest way is to expose the deployment in question:
$ kubectl port-forward svc/$yourServiceName $localport:$serviceport
This would make the http port of the service "centraldb-cockroachdb-public" available via localhost:9000:
$ kubectl port-forward --namespace=mynamespace svc/centraldb-cockroachdb-public 9000:8080
Forwarding from 127.0.0.1:9000 -> 8080
Forwarding from [::1]:9000 -> 8080
Now, we open another shell and call http://localhost:9000:
$ curl -sI http://localhost:9000
HTTP/1.1 307 Temporary Redirect
Content-Type: text/html; charset=utf-8
Location: https://localhost:9000/
Date: Thu, 05 Mar 2020 12:17:09 GMT
To make this permanent, you either need to change the type of service to NodePort or use what Kubernetes calls a "LoadBalancer".

How to connect socket.io inside docker-compose between containers

I have one container that is serving http on port 4000.
it has socket server attached
docker-compose:
dashboard-server:
image: enginetonic:compose1.2
container_name: dashboard-server
command: node src/service/endpoint/dashboard/dashboard-server/dashboard-server.js
restart: on-failure
ports:
- 4000:4000
integration-test:
image: enginetonic:compose1.2
container_name: integration-test
testRegex "(/integration/.*|(\\.|/)(integration))\\.jsx?$$"
tty: true
server:
const http = require('http').createServer(handler)
const io = Io(http)
io.on('connection', socket => {
logger.debug('socket connected')
})
io.use((socket, next) => {
logger.debug('socket connection established.')
})
http.listen(4000, '127.0.0.1', () => {
console.log(
`Server running at http://127.0.0.1:4000/`
)
output in docker:
Server running at http://127.0.0.1:4000/
https is listening: true
Now, I am trying to connect to this server from another container like this:
file:
const url = `ws://dashboard-server:4000`
const ioc = IoC.connect(url)
ioc.on('error', error => {
console.log(error.message)
})
ioc.on('connect', res => {
console.log('connect')
})
ioc.on('connect_error', (error) => {
console.log(error.message)
})
output:
xhr poll error
When I run both locally in terminal, I get correct response
{"message":"socket connection established","level":"debug"}
Why isnt socket making connection inside container, but locally it is?
What am I doing wrong?
edit: only part of files are displayed for readability. socket connects normaly on local machine with with spawning both files in separate terminals
You need to link the docker containers and refer to them by name, not 127.0.0.1. https://docs.docker.com/compose/networking provides more doc. You'll also need to listen to '0.0.0.0' so that you accept connections across the docker network.
I only see one container in your compose file. If you're trying to connect to the docker containers from outside docker, you'll have to expose a port. The same reference shows you how.
http.listen(4000, '127.0.0.1', () => {
should become
http.listen(4000, '0.0.0.0', () => {
so that the server is listening on all addresses, including the address that docker is automatically allocating on a docker network.
Then the client has to refer to the server by the name given in docker compose, so
const url = `ws://127.0.0.1:4000`
becomes
const url = `ws://dashboard-server:4000`

Resources