I am trying to run a Go app binary in a docker container. The app has some gRPC request being listen and server on:
http.ListenAndServe("localhost:8081", nil)
In my docker-compose.yaml. I have a service of the app mapped to 8081:
golangAPP:
build:
context: .
dockerfile: ./docker/golangAPP/Dockerfile
depends_on:
- setup
ports:
- 8081:8081
After docker-compose up I can see the verbose that the app is being served.
But I still cannot reach it. curl -X OPTIONS http://localhost:8081 return
curl: (56) Recv failure: Connection reset by peer
If I run the binary locally without docker, then I can send request to the app.
Any suggestion? I did some googling and some point to firewall issue. But I am not sure how to proceed.
If you do:
http.ListenAndServe("localhost:8081", nil)
this will listen to connections from the loopback interface. When running within a container, this will only accept connections coming from within that container (or if you're running this in a k8s pod, within the same pod). So:
http.ListenAndServe(":8081", nil)
This will accept both loopback and external connections (external to the container).
Related
What I'm trying to do
Host a Taskwarrior Server on an AWS EC2 instance, and connect to it via a subdomain (e.g. task.mydomain.dev).
Taskwarrior server operates on port 53589.
Tech involved
AWS EC2: the server (Ubuntu)
Caddy Server: for creating a reverse proxy for each app on the EC2 instance
Docker (docker-compose): for launching apps, including the Caddy Server and the Taskwarrior server
Cloudflare: DNS hosting and SSL certificates
How I've tried to do this
I have:
allowed incoming connections for ports 22, 80, 443 and 53589 in the instance's security policy
given the EC2 instance an elastic IP
setup the DNS records (task.mydomain.dev is CNAME'd to mydomain.dev, mydomain.dev has an A record pointing to the elastic IP)
used Caddy server to setup a reverse proxy on port 53589 for task.mydomain.dev
setup the Taskwarrior server as per instructions (i.e. certificates created; user and organisation created; taskrc file updated with cert, auth and server info; etc)
Config files
/opt/task/docker-compose.yml
version: '3.3'
services:
taskd:
image: connectical/taskd
restart: always
volumes:
- /opt/task:/var/taskd
ports:
- 53589:53589
networks:
default:
external:
name: caddy_net
/opt/caddy/docker-compose.yml
version: "3.4"
services:
caddy:
build:
context: .
dockerfile: Dockerfile
container_name: caddy
restart: always
ports:
- 80:80
- 443:443
volumes:
- ./config:/config
- ./data:/data
- ./Caddyfile:/etc/caddy/Caddyfile
networks:
default:
external:
name: caddy_net
/opt/caddy/Caddyfile:
task.mydomain.dev:53589 {
reverse_proxy taskd:53589
tls {
dns cloudflare myCloudflareAPIkey
}
}
What's actually happening
I'm unable to connect to port 53589 on task.mydomain.dev
Running telnet task.mydomain.dev 53589 times out
I'm unable to connect to port 53589 on mydomain.dev
Running telnet mydomain.dev 53589 times out
I'm able to connect to port 53589 at 127.0.0.1 by ssh'ing into the EC2 instance
Runningtelnet 127.0.0.1 53589 from the EC2 instance successfully connects
I'm able to connect to port 80 on task.mydomain.dev, but unable to sync with the Taskwarrior server
Running task sync init returns:
c: 1 Received record packet of unknown type 72
Syncing with task.mydomain.dev:80
Cannot perform this action while handshake is in progress.
Sync failed. Could not connect to the Taskserver.
I'm able to connect to port 443 on task.mydomain.dev, but unable to sync with the Taskwarrior server
Running task sync init returns:
Syncing with task.mydomain.dev:443
Malformed message
Sync failed. Could not connect to the Taskserver.
What I've tried to fix it
Changing the Caddyfile's first line to:
task.mydomain.dev { and task.mydomain.dev:80 {, then connecting to port 80
Running task sync init returns:
c: 1 Received record packet of unknown type 72
Syncing with task.mydomain.dev:80
Cannot perform this action while handshake is in progress.
Sync failed. Could not connect to the Taskserver.
task.mydomain.dev { and task.mydomain.dev:443 {, then connecting to port 443
Running task sync init returns:
Syncing with task.mydomain.dev:443
Malformed message
Sync failed. Could not connect to the Taskserver.
Changing Caddyfile's second line to reverse_proxy 127.0.0.1:53589, reverse_proxy 0.0.0.0:53589 and reverse_proxy localhost:53589. Same errors occur.
Removing the CNAME records for the subdomain. Same errors occur
Does anyone have any idea what's happening or could point me in the right direction?
If you are attempting to proxy HTTPS traffic on Cloudflare on a port not on the standard list, you will need to follow one of these options:
Set it up as a Cloudflare HTTPS Spectrum app on the required port 53589
Set up the record in the Cloudflare DNS tab as Grey cloud (in other words, it will only perform the DNS resolution - meaning you will need to manage the certificates on your side)
Change your service so that it listens on one of the standard HTTPS ports listed in the documentation in point (1)
I am using an overlay network to deploy an application on multiple VMs on the same LAN. I am using nginx as the front end for this application and this is running on host_1. All the containers that are part of the application are communicating with each other without any issues. But HTTP requests to the published port 80 of the nginx container (mapped to port 8080 on host_1) from a different VM on the same LAN, say host_2, time out[1]. But HTTP requests to localhost:8080 on host_1 succeed[2]. If I start the nginx container by removing the overlay network, I am able to send HTTP requests[3].
Output of curl -vvv <host_1 IP>:8080 on host_2.
ubuntu#host_2:~$ curl -vvv <host_1>:8080
Rebuilt URL to: <host_1 IP>:8080/
Trying <host_1 IP>...
TCP_NODELAY set
connect to <host_1 IP> port 8080 failed: Connection timed out
Failed to connect to <host_1 IP> port 8080: Connection timed out
Closing connection 0 curl: (7) Failed to connect to <host_1 IP> port 8080: Connection timed out
Output of curl localhost:8080 on host_1.
nginx welcome page
Output of curl -vvv <host_1 IP>:8080 on host_2 when I recreate the container without the overlay network
nginx welcome page
The docker-compose file for the front end is as below:
version: '3'
nginx-frontend:
hostname: nginx-frontend
image: nginx
ports: ['8080:80']
restart: always
networks:
default:
external: {name: overlay-network}
I checked that the nginx and the host are listening on 0.0.0.0:80 and 0.0.0.0:8080 respectively.
Since the port 80 of the nginx is published by mapping it to port 8080 of the host, I should be able to send HTTP requests from any VM that is on the same LAN as the host of this container. Can someone please explain what I am doing wrong or where my assumptions are wrong?
I'm currently running a spring boot application in a docker container on an EC2. My docker-compose file looks like this (with some values replaced):
version: '3.8'
services:
my-app:
image: ${ecr-repo}/my-app:0.0.1-SNAPSHOT
ports:
- "8893:8839/tcp"
networks:
default:
The docker container deploys and comes up as healthy with the healthcheck command being:
wget --spider -q -Y off http://localhost:8893/my-app/v1/actuator/health
If I do a docker ps -a I can see for the ports:
0.0.0.0:8893->8893
My Alb healthcheck however is returning a 502 so I've temporarily allowed connections from my IP directly to the EC2 in the security group. The rules are:
Allow Ingress on 8893 from my Alb security group
Allow Ingress on 8893 from my IP
Allow Egress to anywhere (0.0.0.0)
When I try and hit the healthcheck endpoint of my app using the public DNS of the ec2 on port 8893 using Postman I get Error: connect ECONNREFUSED
If I take my docker container down and then simulate a webserver using the command from https://fabianlee.org/2016/09/26/ubuntu-simulating-a-web-server-using-netcat/ which is:
while true; do { echo -e "HTTP/1.1 200 OK\r\n$(date)\r\n\r\n<h1>hello world from $(hostname) on $(date)</h1>" | nc -vl 8080; } done
I get a 200 response with the expected body which indicates it's not a problem with the security groups.
The actuator endpoint for spring boot is definitely enabled as if I try running the app through intellij and hitting the endpoint it returns a 200 and status up.
Any suggestions for what I might be missing here or how I could debug this further? It seems like docker isn't picking up connections to the port for some reason.
As the title says, I am unable to use a service a certain pod is providing.
The pod is serving a java restApi service at TCP port 2040, that I should be able to access with some specific curl commands.
Some data from kubectl describe pod :
Status: Running
IP: 172.17.0.32
Node: minikube/13*.20*.13*.14 (I obfuscated my real IP here)
Container ID: docker://b5b16bd7926ce65d4a57212f60c87ea72e161f534a0e1d6925c508dd89ab202e
Ports: 9899/TCP, 1272/TCP, 2040/TCP, 9500/TCP, 9501/TCP, 9502/TCP, 9503/TCP, 9504/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP
(is this correct ?)
I am confused by the fact that :
A) I am perfectly able to issue curl commands on the shell inside the docker container (b5b16bd7926ce65...)
B) The tcp connection from my host to the service the pod is providing is successful :
user#host$ nc -zv 172.17.0.32 2040
Connection to 172.17.0.32 2040 port [tcp/*] succeeded!
C) any curl command (from the host) towards 172.17.0.32:2040 fails with :
504 Gateway Timeout: remote server did not respond to the proxy
The host is running Ubuntu 18.04LTS.
I am behind a corporate proxy, but as this is all done on my local machine, I don't think that could be an issue.
What could be responsible for this behavior?
Thanks in advance.
So the issue here was the proxy, adding --noproxy "*" to the curl command made it work from the host.
I however do not understand why this is required, I am hosting everything my self?
On a public server, I have an Prometheus exporter setup. This is blocked, intentionally, by a firewall as the information should not be public.
From a separate network (my home network with dynamic IP), I wish to scrape the Prometheus exporter. The idea is to use autossh to setup an SSH tunnel, to be able to scrape the endpoint that way. I prefer to setup autossh using docker.
So far I have managed to setup a autossh docker container, with the following docker-compose:
remote-nodeexporter:
image: jnovack/autossh:latest
container_name: remote-nodeexporter
environment:
- SSH_HOSTNAME=PUBLIC_IP
- SSH_TUNNEL_REMOTE=19100
- SSH_TUNNEL_LOCAL=9100
- SSH_MODE=-L
restart: always
volumes:
- /path/to/id_rsa:/id_rsa
ports:
- "19100:19100"
From within the container this works fine:
/ # wget localhost:19100/metrics
Connecting to localhost:19100 (127.0.0.1:19100)
saving to 'metrics'
metrics 100% |**********************************************************************************************************************************************************************************************| 75595 0:00:00 ETA
'metrics' saved
But from the host (or from other containers), I get errors:
/ # wget localhost:19100/metrics
--2020-07-07 08:53:25-- http://localhost:19100/metrics
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:19100... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.
How do I correctly expose this endpoint?