why can't i access the docker based zookeeper port - docker

on OS X i started kafka docker image successfully,but it seems that i can't access it on localhost
➜ ~ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1f931da3d661 wurstmeister/zookeeper:3.4.6 "/bin/sh -c '/usr/..." About an hour ago Up About an hour 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp docker_zookeeper_1
8bc36bcf8fdf wurstmeister/kafka:0.10.1.1 "start-kafka.sh" About an hour ago Up About an hour 0.0.0.0:9092->9092/tcp docker_kafka_1
➜ ~ telnet 0.0.0.0:2181
0.0.0.0:2181: nodename nor servname provided, or not known
➜ ~ telnet 0.0.0.0 2181
Trying 0.0.0.0...
telnet: connect to address 0.0.0.0: Connection refused
telnet: Unable to connect to remote host
➜ ~ telnet 192.168.43.193 2181
Trying 192.168.43.193...
telnet: connect to address 192.168.43.193: Connection refused
telnet: Unable to connect to remote host
➜ ~ telnet 127.0.0.1 2181
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
telnet: Unable to connect to remote host
my docker file is here kafka.yml and use this command to up:
docker-compose -f src/main/docker/kafka.yml up -d
when i use
./mvnw
the console is:
2017-09-15 17:05:46.433 WARN 15871 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
how can i access the 2181 port
EDIT
docker logs 8bc36bcf8fdf
[2017-09-15 08:14:13,386] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/1001. This probably indicates that you either have configured a brokerid that is already in use, or else you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears to be re-registering.
at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:393)
at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:379)
at kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:70)
at kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:51)
at kafka.server.KafkaServer.startup(KafkaServer.scala:270)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39)
at kafka.Kafka$.main(Kafka.scala:67)
at kafka.Kafka.main(Kafka.scala)
[2017-09-15 08:14:13,393] INFO [Kafka Server 1001], shutting down (kafka.server.KafkaServer)
docker logs 1f931da3d661
2017-09-14 08:53:05,878 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn#357] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x15e7ea74c8e0000, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)
2017-09-14 08:53:05,887 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn#1007] - Closed socket connection for client /172.18.0.2:54222 which had sessionid 0x15e7ea74c8e0000

Have you tried using host networking as in this example? https://docs.confluent.io/current/cp-docker-images/docs/quickstart.html#zookeeper
That looks like it will simplify and solve this. I'd also recommend checking out these images instead of the custom ones it looks like you are using because these are being run in production for people so they are known to work well.

Related

Kafka Unable to connect with Zookeeper in Docker

I am using the following scripts to run Zookeeper and Kafka in MAC M1 machine
Zookeeper:-
docker run --name zookeeper -p 2181:2181 -e ZOOKEEPER_TICK_TIME=2000 -e ZOOKEEPER_INIT_LIMIT=10 -e ZOOKEEPER_SYNC_LIMIT=5 -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper
Kafka:-
docker run --name kafka -p 9092:9092 -e KAFKA_ZOOKEEPER_CONNECT=0.0.0.0:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://0.0.0.0:9092 confluentinc/cp-kafka
Zookeeper is Running successfully, but kafka is unable to connect to zookeeper with the following error
[2022-11-18 21:15:48,461] INFO Opening socket connection to server 0.0.0.0/0.0.0.0:2181. (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:48,462] INFO SASL config status: Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:48,462] WARN Session 0x0 for sever 0.0.0.0/0.0.0.0:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
[2022-11-18 21:15:49,593] INFO Opening socket connection to server 0.0.0.0/0.0.0.0:2181. (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:49,594] INFO SASL config status: Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:49,595] WARN Session 0x0 for sever 0.0.0.0/0.0.0.0:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
[2022-11-18 21:15:50,722] INFO Opening socket connection to server 0.0.0.0/0.0.0.0:2181. (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:50,722] INFO SASL config status: Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:50,723] WARN Session 0x0 for sever 0.0.0.0/0.0.0.0:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
Even when I try to telnet for 0.0.0.0 2181 i get the following error
Trying 0.0.0.0...
Connected to 0.0.0.0.
Escape character is '^]'.
Connection closed by foreign host.
Please let me know what configuration changes needs to be done for a successful connection.
0.0.0.0 is not a routable address. You need to use bridge networks and service names.
https://docs.docker.com/network/bridge/
Or, read Confluent documentation, where they provide working Docker Compose files with those images.
You should use zookeeper-shell, rather than telnet.
Also, Kafka doesn't require Zookeeper anymore.
Related (notice --net=confluent and actually using zookeeper:2181) - Connect to Kafka running in Docker

Apache Guacamole Container Setup for VNC connection error: "Cannot connect. Connection already in use by this user."

I'm trying to set up Guacamole using container on a home Ubuntu 20.04 desktop. I can get an SSH connection to work but I'm having a hard time with the VNC setup. Below are a summary of the errors, my setup and some troubleshooting steps I did.
SUMMARY OF ERROR MESSAGES
The management app Guacamole is served at http://localhost:8080/guacamole/, I try to access the VNC connection (its setup is in the next section) and get these errors
guacamole web app error message: "The Guacamole server is denying access to this connection because you have exhausted the limit for simultaneous connection use by an individual user. Please close one or more connections and try again."
In the Chrome or Firefox developer console, network/XHR, I'm pasting
a few request/response headers:
Request URL: http://localhost:8080/guacamole/tunnel?connect
Response Status Code: 429
Response Headers:
Guacamole-Error_message: Cannot connect. Connection already in use by this user.
Guacamole-Status-Code: 797
In the guacd docker container:
guacd[7]: DEBUG: Guacamole connection closed during handshake
guacd[7]: DEBUG: Error reading "select": End of stream reached while reading instruction
In the guacamole docker container:
18:13:26.091 [http-nio-8080-exec-9] ERROR o.a.g.w.GuacamoleWebSocketTunnelEndpoint - Creation of WebSocket tunnel to guacd failed: Cannot connect. Connection already in use by this user.
18:13:26.116 [http-nio-8080-exec-6] WARN o.a.g.s.GuacamoleHTTPTunnelServlet - HTTP tunnel request rejected: Cannot connect. Connection already in use by this user.
MY INSTALLATION AND TROUBLESHOOTING DONE SO FAR
Environment
Ubuntu 20.04 desktop
working tiger VNC server setup at display number 1, which I have been using for SSH-tunneled VNC connection for 2 years
$ sudo systemctl status vncserver#1.service
?? vncserver#1.service - Start TightVNC server at startup
Loaded: loaded (/etc/systemd/system/vncserver#.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2022-04-13 22:45:53 EDT; 8min ago
Main PID: 2035 (Xtigervnc)
Docker containers
I followed the official doc to set up three containers.
The guacamole links to the guacd and mysql.
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b0c49ab0fb8f guacamole/guacamole:1.4.0 "/opt/guacamole/bin/??" 20 hours ago Up 42 minutes 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp some-guacamole
969afbd569c2 guacamole/guacd "/bin/sh -c '/usr/lo??" 21 hours ago Up 43 minutes (healthy) 4822/tcp some-guacd
3e490e948aa6 mysql/mysql-server:latest "/entrypoint.sh mysq??" 38 hours ago Up 42 minutes (healthy) 3306/tcp, 33060-33061/tcp mysql-docker
The guacamole container, guacd container and the vnc server have connectivity with each other
The web app came up fine and I can login to configure settings.
I easily got an SSH connection to work on guacamole
For VNC connections, I tried both guacamole at the latest and at tag 1.4.0 but that made no difference
On my Ubuntu host, I have proper firewall settings:
ports ssh 22, apache 80/443 are wide open
my VNC server is sitting on 0 0.0.0.0:5901 and is therefore open to 172.17.0.0/24
My docker0 is recognized by the host as 172.17.0.1
$ netstat -an | grep 5901
tcp 0 0 0.0.0.0:5901 0.0.0.0:* LISTEN
$ ifconfig docker0
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
From within the guacd container, I can telnet to my docker host's SSH server (172.17.0.1:22), Apache server(172.17.0.1:80/443), and VNC server (172.17.0.1:5901)
$ sudo docker exec -u0 -it some-guacd bash
root#969afbd569c2:/# telnet 172.17.0.1 5901
Trying 172.17.0.1...
Connected to 172.17.0.1.
Escape character is '^]'.
RFB 003.008
^]
telnet> quit
Connection closed.
In addition to the SSH connection working out of the box with guacamole install, from within the guacamole container, I could telnet to the guacd at port 4822 and paste the following vnc handshake (6.select,3.vnc;) and got a proper response.
$ sudo docker inspect some-guacd|grep IPAddress
"SecondaryIPAddresses": null,
"IPAddress": "172.17.0.2",
"IPAddress": "172.17.0.2",
$ sudo docker exec -u0 -it some-guacamole bash
root#b0c49ab0fb8f:/opt/guacamole# telnet 172.17.0.2 4822
Trying 172.17.0.2...
Connected to 172.17.0.2.
Escape character is '^]'.
6.select,3.vnc;
4.args,13.VERSION_1_3_0,8.hostname,4.port,9.read-only,9.encodings,8.username,8.password,13.swap-red-blue,11.color-depth,6.cursor,9.autoretry,18.clipboard-encoding,9.dest-host,9.dest-port,12.enable-audio,16.audio-servername,15.reverse-connect,14.listen-timeout,11.enable-sftp,13.sftp-hostname,13.sftp-host-key,9.sftp-port,13.sftp-username,13.sftp-password,16.sftp-private-key,15.sftp-passphrase,14.sftp-directory,19.sftp-root-directory,26.sftp-server-alive-interval,21.sftp-disable-download,19.sftp-disable-upload,14.recording-path,14.recording-name,24.recording-exclude-output,23.recording-exclude-mouse,22.recording-include-keys,21.create-recording-path,12.disable-copy,13.disable-paste,15.wol-send-packet,12.wol-mac-addr,18.wol-broadcast-addr,12.wol-udp-port,13.wol-wait-time,14.force-lossless;
On guacamole VNC connection configuration, I have
Parameters
Network
Hostname: 172.17.0.1
Port: 5901
I believe the apparent error message "Cannot connect. Connection already in use by this user" is a red herring. It's more likely that the guacamole app has problem connecting to the guacd server at the protocol or the application level. I'm really baffled. I have posted onto the apache mailing list for guacamole a few days ago but haven't got a reply yet. So I'm trying my luck on SO.

I try strart auditbeat on my local computer through docker. However I get connection refused from elasticsearch

I start auditbeat
docker run --cap-add="AUDIT_CONTROL" --cap-add="AUDIT_READ" docker.elastic.co/beats/auditbeat:7.8.1 setup -E setup.kibana.host=localhost:5601 -E output.elasticsearch.hosts=["127.0.0.1:9300"]
but get error Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at http://127.0.0.1:9300: Get http://127.0.0.1:9300: dial tcp 127.0.0.1:9300: connect: connection refused] I try user also localhost in output.elasticsearch.hosts. When I sent request by curl http://127.0.0.1:9200 I get successful response from elasticsearch.
Also. Elasticsearch is deployed as docker process.
You need to use the HTTP port 9200 (the same you curl with) not the TCP port 9300
-Eoutput.elasticsearch.hosts=["host.docker.internal:9200"]
^
|
change this

Not able to connect to redis server from docker container

My docker version is 1.13.1.
I am trying to connect to redis-server from my docker container,but I am getting connection refused error.Detailed logs are given below:
Caused by: redis.clients.jedis.exceptions.JedisConnectionException:
java.net.ConnectException: Connection refused
at redis.clients.jedis.Connection.connect(Connection.java:207) [jedis-2.9.0.jar:]
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93)
[jedis-2.9.0.jar:]
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767)
[jedis-2.9.0.jar:]
at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106)
[jedis-2.9.0.jar:]
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:888)
[commons-pool2-2.4.3.jar:2.4.3]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:432)
[commons-pool2-2.4.3.jar:2.4.3]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:361)
[commons-pool2-2.4.3.jar:2.4.3]
at redis.clients.util.Pool.getResource(Pool.java:49) [jedis-2.9.0.jar:]
... 27 more Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_80]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[rt.jar:1.7.0_80]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[rt.jar:1.7.0_80]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[rt.jar:1.7.0_80]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_80]
at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_80]
at redis.clients.jedis.Connection.connect(Connection.java:184) [jedis-2.9.0.jar:]
... 34 more
when i give --net="host" option this error is gone,but when i give
--add-host=parent-host:`ip route show | grep docker0 | awk '{print \$9}'`
I get the error,although the container is able to connect to Db on a different VM,but not connecting to redis-server present on self hosting machine.
I suspect the problem is that you are trying to connect to redis via localhost rather than parent-host. Make sure that you are connecting to redis via parent-host when creating the Jedis connection.
The reason --net=host option works, is that in this case localhost will refer to the IP of the host machine and not to that of the container.
Also, make sure that there is an enty for parent-host in /etc/hosts inside the container.

Neo4j remote shell connection refused on AWS database server

My Neo4j database server is hosted in AWS.
I am using from my Mac this command:
$ ./bin/neo4j-shell -host xx.xxx.xxx.xxx -port 1337 -name shell
The port 1337 is enabled in the AWS Neo4j database server.
I have these lines uncommented in the file neo4j.properties:
remote_shell_enabled=true
remote_shell_host=127.0.0.1
remote_shell_port=1337
I am getting the error: Connection refused
Having the details:
java.rmi.ConnectException: Connection refused to host: xx.xxx.xxx.xxx; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:341)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at java.rmi.Naming.lookup(Naming.java:101)
at org.neo4j.shell.impl.RmiLocation.getBoundObject(RmiLocation.java:261)
at org.neo4j.shell.impl.RemoteClient.findRemoteServer(RemoteClient.java:69)
at org.neo4j.shell.impl.RemoteClient.<init>(RemoteClient.java:62)
at org.neo4j.shell.impl.RemoteClient.<init>(RemoteClient.java:45)
at org.neo4j.shell.ShellLobby.newClient(ShellLobby.java:178)
at org.neo4j.shell.StartClient.startRemote(StartClient.java:302)
at org.neo4j.shell.StartClient.start(StartClient.java:179)
at org.neo4j.shell.StartClient.main(StartClient.java:124)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:208)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:147)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
... 13 more
Don't know how to fix it.
I think you have to open quite a few ports on your server, besides 1337 and 1099 also a port range for RMI ports.
e.g. What port is used by Java RMI connection?
you can also use neo4j over http (e.g. in the browser with /webadmin) or via this script: https://gist.github.com/jexp/a9bf4cd3a16eb6c5997b
This key remote_shell_host=127.0.0.1 that enables you to just connect via loopback. You have to specify by which network adapter you will receive the connection. You can also specify 0.0.0.0 for any your network interfaces.

Resources