I installed the latest HDP into the docker container.
docker ps
>hortonworks/sandbox-proxy:1.0
>hortonworks/sandbox-hdp:3.0.1
Kafka broker is running at 6667 port that it's working fine.
>ssh hdfs#sandbox-hdp.hortonworks.com -p 2222
> echo "send test message" | kafka-console-producer.sh --broker-list sandbox-hdp.hortonworks.com:6667 --topic hotelReservation > /dev/null
>kafka-console-consumer.sh --bootstrap-server sandbox-hdp.hortonworks.com:6667 --topic test --from-beginning
The only problem is that I don't have access to the Kafka broker outside the docker container.
More information for troubleshooting:
I verified that all the ports are open local machine -> proxy -> hdp
HDP Server:
hostname -I
172.18.0.2
netstat -vatn | grep 6667
tcp 0 0 172.18.0.2:6667 0.0.0.0:* LISTEN
Proxy Server
hostname -I
172.18.0.3
nmap 172.18.0.2 -p 6667
6667/tcp open irc
netstat -vatn | grep 6667
tcp 0 0 0.0.0.0:6667 0.0.0.0:* LISTEN
My Local machine:
nmap sandbox-hdp.hortonworks.com -p 6667
Host is up (0.000064s latency).
rDNS record for 127.0.0.1: localhost
PORT STATE SERVICE
6667/tcp open irc
Note: 6668 is open too.
included default stream conf (/etc/nginx/conf/stream.d/tcp-hdp.conf):
server {
listen 6668;
proxy_pass sandbox-hdp:6667;
}
So I decided to create new config the http: /etc/nginx/conf/stream.d/tcp-hdp.conf
server {
listen 6667;
server_name sandbox-hdp.hortonworks.com;
location / {
proxy_pass http://sandbox-hdp:6667;
}
}
Note: I read (here) that I should use advertised.listeners to access broker outside the doctor:
>listeners=PLAINTEXT://0.0.0.0:6667
>advertised.listeners=PLAINTEXT://172.18.0.2:6667
but I 'm sure that's incorrect for my case. I have access in this network, the problem is some type of Nginx mapping only.
Solution:
Run the following command to find hortonworks/sandbox-proxy container id.
docker ps
Log in to the proxy server
docker exec -it CONTAINER_ID /bin/bash
add new mapping in the nginx
cd /etc/nginx/conf.stream.d
vim tcp-hdp.conf
config:
server {
listen 6667;
proxy_pass sandbox-hdp:6667;
}
reload nginx mapping:
/etc/init.d/nginx reload
Test:
> ./kafka-console-consumer.sh --bootstrap-server localhost:6668 --topic topicName --from-beginning
> echo "msg" | ./kafka-console-producer.sh --broker-list sandbox-hdp.hortonworks.com:6668 --topic topicName > /dev/nul
The problem was that 6667:6667 mapping was missing. Now brokers are access both 6667 and 6668 ports.
Related
I would like to have a service being deployed as part of a docker stack to listen on ipv4.
Currently the docker stack deployed service (rabbitmq) is listening on ipv6, I would like to have it listen via ipv4.
The section of docker compose .yaml file that I using to deploy the docker stack as the following yaml section.
rabbitmq-3-11-0:
#image: rabbitmq:3.11.0-management
image: "127.0.0.1:5000/bcl-sdv-rabbitmq-3-11-0:v0.1"
ports:
-
"0.0.0.0:5672:5672/tcp"
-
"0.0.0.0:15672:15672/tcp" #15672: HTTP API clients, management UI and rabbitmqadmin (only if the management plugin is enabled)
On deployment of the docker stack, the "rabbitmq-3-11-0" service is deployed successfully.
To test IP connectivity I issue the following commands on the docker node.
ncat -w 2 -v ::1 5672 </dev/null; echo $?
yields
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to ::1:5672.
While the command
ncat -w 2 -v 0.0.0.0 5672 </dev/null; echo $?
yields
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 0.0.0.0:5672.
Ncat: Connection reset by peer.
1
The command
ncat -w 2 -v 127.0.0.1 5672 </dev/null; echo $?
produces
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 127.0.0.1:5672.
Ncat: Connection reset by peer.
1
The netstat command below
sudo netstat -tulnp|grep 5672
shows that the ports 5672 and 15672 are listening on ipv6.
tcp6 4 0 :::5672 :::* LISTEN 2527/dockerd
tcp6 0 0 :::15672 :::* LISTEN 2527/dockerd
The command to determine the docker version below
docker info|grep Version
Outputs
Server Version: 20.10.20
Cgroup Version: 1
Kernel Version: 3.10.0-1160.76.1.el7.x86_64
The Linux version command below
lsb_release
prints
LSB Version: :core-4.1-amd64:core-4.1-noarch
I have encountered a strange behaviour. I have running grafana container with tcp/3000 exposed
# docker ps -a | grep grafana
26fa0fff9220 registry:4000/prometheus/grafana 0/tcp, 0.0.0.0:3000->3000/tcp grafana
# docker inspect grafana -f '{{json .NetworkSettings.Ports }}'
{"0/tcp":null,"3000/tcp":[{"HostIp":"0.0.0.0","HostPort":"3000"}]}
# ss -tunlp | grep 3000
tcp LISTEN 0 4096 0.0.0.0:3000 0.0.0.0:* users:(("docker-proxy",pid=3051799,fd=4))
The problem is that although curl works correctly for localhost it doesn't work when i am using the address of the host machine's network interface.
For example
//works fine
root#host-1# curl localhost:3000
Found.
//doesn't work at all (10.25.200.111 is eth-interface ip address on this same host), curl hangs indefinitely
root#host-1# curl 10.25.200.111:3000
^C
If i make a request from another host to 10.25.200.111:3000 it works without problems.
root#host-2# curl 10.25.200.111:3000
Found.
What am i doing wrong or maybe missing?
I am running the container and mapping the port like so:
docker run -d --expose 4242 -p 4242:4242 42wim/matterbridge:stable --debug
I've created a firewall rule to allows TCP connections over port 4242 to my VM. When I send an http request to the public IP of my VM the connection is refused:
http://{public-ip}:4242/api/messages
Howevever if I open a shell into the container and do a curl to the path I get the expected response curl localhost:4242/api/messages
What is the correct way to map TCP requests on port 4242 from my Host to my Container? I'm running a Ubuntu VM on GCP that hosts my docker container
Update, if use docker run --network="host" I can curl from the host to the docker container with curl localhost:4242/api/messages with the expected response. Yet when I do the same curl request with the public IP the connection is still refused.
if I ss -na | grep :4242
tcp LISTEN 0 4096 127.0.0.1:4242 0.0.0.0:*
it shows it's listening. Is there additional mapping I need to do? I have validated from google firewall logs that it is allowing and forwarding TCP connections from port 4242 to the VM
I'm trying to run an application in Docker on an EC2 instance. It is two separate processes. I'm able to access the ports for process 1, but not process 2.
Process 1 listens on the following ports:
2008
8080
Process two listens on the these ports:
2021
8084
The security rules allow for all traffic to all ports from all origins:
Netstat shows both ports on process 2 are listening
netstat -an | grep 2021
tcp6 0 0 :::2021 :::* LISTEN
netstat -an | grep 8084
tcp6 0 0 :::8084 :::* LISTEN
The docker command opens all of the above ports:
docker run -ti --privileged=true -v /sys/fs/cgroup:/sys/fs/cgroup:ro -p 8080:8080 -p 2008:2008 -p 8084:8084 -p 2021:2021 myname/image_name /usr/sbin/init
There is no firewall process running.
Yet, a zenmap scan shows only ports 2008 and 8080 of the above four are listening - 2021 and 8084 don't show up.
Any ideas why this would be? I can't think of what else to look for.
I have a dockerized Python app that outputs data on port 8080 and port 8081.
I am running the code on a Ubuntu system.
$ docker version | grep Version
Version: 18.03.1-ce
The app responds on port 8080
$ curl -k localhost:8080 | tail -4
-->
TYPE hello_world_total counter
hello_world_total 3.0
TYPE hello_world_created gauge
hello_world_created 1.5617357381235116e+09
The app returns an ERROR on port 8081
$ curl -k localhost:8081
curl: (56) Recv failure: Connection reset by peer
Although I am not familiar with netstat, I used it to check that ports 8080 and 8081 were both in the LISTEN state ...
root#1d1ac2974893:/# netstat -apn
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 1/python3
tcp 0 0 127.0.0.1:8081 0.0.0.0:* LISTEN 1/python3
tcp 0 0 172.17.0.2:58220 16.46.41.11:8080 TIME_WAIT -
tcp 0 0 172.17.0.2:58218 16.46.41.11:8080 TIME_WAIT -
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node PID/Program name Path
root#1d1ac2974893:/#
My Dockerfile looks as follows ...
$ cat Dockerfile
FROM python:3
RUN pip3 install prometheus_client
COPY sampleapp.py /src/sampleapp.py
EXPOSE 8081
CMD [ "python3", "/src/sampleapp.py" ]
When I run the application, I map both ports 8080 and 8081 from the Docker container to the same ports on the host as follows ...
$ docker run -p 8081:8081 -p 8080:8080 sampleapp
If I go into the Container and repeat the above curl commands, they work as I expect.
root#1d1ac2974893:/# curl -k localhost:8081 | tail -4
TYPE hello_world_total counter
hello_world_total 3.0
TYPE hello_world_created gauge
hello_world_created 1.5617357381235116e+09
root#1d1ac2974893:/#
AND
$ docker exec -it 1d1ac2974893 /bin/bash
root#1d1ac2974893:/# curl -k localhost:8081
Hello World
SO
the question is why the latter curl command does NOT work from the host system.
$ curl -k localhost:8081
curl: (56) Recv failure: Connection reset by peer
Solution was as follows
Expose both ports in the Dockerfile
$ grep EXPOSE Dockerfile
EXPOSE 8080
EXPOSE 8081
Use 0.0.0.0 rather than 127.0.0.1
import http.server
from prometheus_client import start_http_server
from prometheus_client import Counter
HOST='0.0.0.0'
HELLO_WORLD_PORT=8080
HELLO_WORLD_METRICS_PORT=8081
REQUESTS = Counter('hello_world_total', 'Hello World Requested')
class MyHandler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
REQUESTS.inc()
self.send_response(200)
self.end_headers()
self.wfile.write(b"Hello World\n")
if name == "main":
start_http_server(HELLO_WORLD_METRICS_PORT)
server = http.server.HTTPServer((HOST, HELLO_WORLD_PORT), MyHandler)
server.serve_forever()
Contaainer now gives the expected results when run from the host
$ curl -k localhost:8080
Hello World
$
$ curl -k localhost:8081 | tail -4
...
# TYPE hello_world_total counter
hello_world_total 1.0
# TYPE hello_world_created gauge
hello_world_created 1.5619773258069074e+09
$
Xref :- Docker Rails app fails to be served - curl: (56) Recv failure: Connection reset by peer
for details of a similar issue