I am using the following scripts to run Zookeeper and Kafka in MAC M1 machine
Zookeeper:-
docker run --name zookeeper -p 2181:2181 -e ZOOKEEPER_TICK_TIME=2000 -e ZOOKEEPER_INIT_LIMIT=10 -e ZOOKEEPER_SYNC_LIMIT=5 -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper
Kafka:-
docker run --name kafka -p 9092:9092 -e KAFKA_ZOOKEEPER_CONNECT=0.0.0.0:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://0.0.0.0:9092 confluentinc/cp-kafka
Zookeeper is Running successfully, but kafka is unable to connect to zookeeper with the following error
[2022-11-18 21:15:48,461] INFO Opening socket connection to server 0.0.0.0/0.0.0.0:2181. (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:48,462] INFO SASL config status: Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:48,462] WARN Session 0x0 for sever 0.0.0.0/0.0.0.0:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
[2022-11-18 21:15:49,593] INFO Opening socket connection to server 0.0.0.0/0.0.0.0:2181. (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:49,594] INFO SASL config status: Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:49,595] WARN Session 0x0 for sever 0.0.0.0/0.0.0.0:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
[2022-11-18 21:15:50,722] INFO Opening socket connection to server 0.0.0.0/0.0.0.0:2181. (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:50,722] INFO SASL config status: Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2022-11-18 21:15:50,723] WARN Session 0x0 for sever 0.0.0.0/0.0.0.0:2181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
Even when I try to telnet for 0.0.0.0 2181 i get the following error
Trying 0.0.0.0...
Connected to 0.0.0.0.
Escape character is '^]'.
Connection closed by foreign host.
Please let me know what configuration changes needs to be done for a successful connection.
0.0.0.0 is not a routable address. You need to use bridge networks and service names.
https://docs.docker.com/network/bridge/
Or, read Confluent documentation, where they provide working Docker Compose files with those images.
You should use zookeeper-shell, rather than telnet.
Also, Kafka doesn't require Zookeeper anymore.
Related (notice --net=confluent and actually using zookeeper:2181) - Connect to Kafka running in Docker
Related
I start auditbeat
docker run --cap-add="AUDIT_CONTROL" --cap-add="AUDIT_READ" docker.elastic.co/beats/auditbeat:7.8.1 setup -E setup.kibana.host=localhost:5601 -E output.elasticsearch.hosts=["127.0.0.1:9300"]
but get error Exiting: couldn't connect to any of the configured Elasticsearch hosts. Errors: [error connecting to Elasticsearch at http://127.0.0.1:9300: Get http://127.0.0.1:9300: dial tcp 127.0.0.1:9300: connect: connection refused] I try user also localhost in output.elasticsearch.hosts. When I sent request by curl http://127.0.0.1:9200 I get successful response from elasticsearch.
Also. Elasticsearch is deployed as docker process.
You need to use the HTTP port 9200 (the same you curl with) not the TCP port 9300
-Eoutput.elasticsearch.hosts=["host.docker.internal:9200"]
^
|
change this
My docker version is 1.13.1.
I am trying to connect to redis-server from my docker container,but I am getting connection refused error.Detailed logs are given below:
Caused by: redis.clients.jedis.exceptions.JedisConnectionException:
java.net.ConnectException: Connection refused
at redis.clients.jedis.Connection.connect(Connection.java:207) [jedis-2.9.0.jar:]
at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:93)
[jedis-2.9.0.jar:]
at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1767)
[jedis-2.9.0.jar:]
at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:106)
[jedis-2.9.0.jar:]
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:888)
[commons-pool2-2.4.3.jar:2.4.3]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:432)
[commons-pool2-2.4.3.jar:2.4.3]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:361)
[commons-pool2-2.4.3.jar:2.4.3]
at redis.clients.util.Pool.getResource(Pool.java:49) [jedis-2.9.0.jar:]
... 27 more Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_80]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[rt.jar:1.7.0_80]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[rt.jar:1.7.0_80]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[rt.jar:1.7.0_80]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_80]
at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_80]
at redis.clients.jedis.Connection.connect(Connection.java:184) [jedis-2.9.0.jar:]
... 34 more
when i give --net="host" option this error is gone,but when i give
--add-host=parent-host:`ip route show | grep docker0 | awk '{print \$9}'`
I get the error,although the container is able to connect to Db on a different VM,but not connecting to redis-server present on self hosting machine.
I suspect the problem is that you are trying to connect to redis via localhost rather than parent-host. Make sure that you are connecting to redis via parent-host when creating the Jedis connection.
The reason --net=host option works, is that in this case localhost will refer to the IP of the host machine and not to that of the container.
Also, make sure that there is an enty for parent-host in /etc/hosts inside the container.
on OS X i started kafka docker image successfully,but it seems that i can't access it on localhost
➜ ~ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1f931da3d661 wurstmeister/zookeeper:3.4.6 "/bin/sh -c '/usr/..." About an hour ago Up About an hour 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp docker_zookeeper_1
8bc36bcf8fdf wurstmeister/kafka:0.10.1.1 "start-kafka.sh" About an hour ago Up About an hour 0.0.0.0:9092->9092/tcp docker_kafka_1
➜ ~ telnet 0.0.0.0:2181
0.0.0.0:2181: nodename nor servname provided, or not known
➜ ~ telnet 0.0.0.0 2181
Trying 0.0.0.0...
telnet: connect to address 0.0.0.0: Connection refused
telnet: Unable to connect to remote host
➜ ~ telnet 192.168.43.193 2181
Trying 192.168.43.193...
telnet: connect to address 192.168.43.193: Connection refused
telnet: Unable to connect to remote host
➜ ~ telnet 127.0.0.1 2181
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
telnet: Unable to connect to remote host
my docker file is here kafka.yml and use this command to up:
docker-compose -f src/main/docker/kafka.yml up -d
when i use
./mvnw
the console is:
2017-09-15 17:05:46.433 WARN 15871 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
how can i access the 2181 port
EDIT
docker logs 8bc36bcf8fdf
[2017-09-15 08:14:13,386] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/1001. This probably indicates that you either have configured a brokerid that is already in use, or else you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears to be re-registering.
at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:393)
at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:379)
at kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:70)
at kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:51)
at kafka.server.KafkaServer.startup(KafkaServer.scala:270)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39)
at kafka.Kafka$.main(Kafka.scala:67)
at kafka.Kafka.main(Kafka.scala)
[2017-09-15 08:14:13,393] INFO [Kafka Server 1001], shutting down (kafka.server.KafkaServer)
docker logs 1f931da3d661
2017-09-14 08:53:05,878 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn#357] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x15e7ea74c8e0000, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)
2017-09-14 08:53:05,887 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn#1007] - Closed socket connection for client /172.18.0.2:54222 which had sessionid 0x15e7ea74c8e0000
Have you tried using host networking as in this example? https://docs.confluent.io/current/cp-docker-images/docs/quickstart.html#zookeeper
That looks like it will simplify and solve this. I'd also recommend checking out these images instead of the custom ones it looks like you are using because these are being run in production for people so they are known to work well.
I am trying to run a docker container that takes the CONSUL_URL as an ENV Variable. For now, I have set up the consul on my localhost and I run it like this: consul agent -dev -bind=127.0.0.1 -ui-dir /usr/local/Cellar/consul/0.7.0/share/consul/web-ui
I am able to access the consul ui this way when I go to http://localhost:8500. But, now that I am running the docker container through this, docker run -e CONSUL_URL=127.0.0.1:8500 -p 8500:8500 b321825a6c7a, it gives me the following error:
2016/10/05 09:38:38 [ERR] (view) "key_or_default(foo.appconfig.properties/logger.name, "foo_PERF_LOG")" store key: error fetching: Get http://127.0.0.1:8500/v1/kv/foo.appconfig.properties/logger.name?stale=&wait=60000ms: dial tcp 127.0.0.1:8500: getsockopt: connection refused
2016/10/05 09:38:38 [ERR] (runner) watcher reported error: store key: error fetching: Get http://127.0.0.1:8500/v1/kv/foo.appconfig.properties/logger.name?stale=&wait=60000ms: dial tcp 127.0.0.1:8500: getsockopt: connection refused
Consul Template returned errors:
store key: error fetching: Get http://127.0.0.1:8500/v1/kv/foo.appconfig.properties/logger.name?stale=&wait=60000ms: dial tcp 127.0.0.1:8500: getsockopt: connection refusedexecuting: 'myscript.sh run'
Why am I not able to connect with the consul URL? I also tried changing the localhost url to the IP Address of my machine, but, I get the same error with that too. I have done the port mapping, so, I guess it should work. Where am I going wrong?
You have to use the --net=host flag.
This flag will create sockets for the exposed ports for all interfaces in the main OS.
docker run --net=host -e CONSUL_URL=127.0.0.1:8500 b321825a6c7a
My Neo4j database server is hosted in AWS.
I am using from my Mac this command:
$ ./bin/neo4j-shell -host xx.xxx.xxx.xxx -port 1337 -name shell
The port 1337 is enabled in the AWS Neo4j database server.
I have these lines uncommented in the file neo4j.properties:
remote_shell_enabled=true
remote_shell_host=127.0.0.1
remote_shell_port=1337
I am getting the error: Connection refused
Having the details:
java.rmi.ConnectException: Connection refused to host: xx.xxx.xxx.xxx; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:341)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at java.rmi.Naming.lookup(Naming.java:101)
at org.neo4j.shell.impl.RmiLocation.getBoundObject(RmiLocation.java:261)
at org.neo4j.shell.impl.RemoteClient.findRemoteServer(RemoteClient.java:69)
at org.neo4j.shell.impl.RemoteClient.<init>(RemoteClient.java:62)
at org.neo4j.shell.impl.RemoteClient.<init>(RemoteClient.java:45)
at org.neo4j.shell.ShellLobby.newClient(ShellLobby.java:178)
at org.neo4j.shell.StartClient.startRemote(StartClient.java:302)
at org.neo4j.shell.StartClient.start(StartClient.java:179)
at org.neo4j.shell.StartClient.main(StartClient.java:124)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:208)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:147)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
... 13 more
Don't know how to fix it.
I think you have to open quite a few ports on your server, besides 1337 and 1099 also a port range for RMI ports.
e.g. What port is used by Java RMI connection?
you can also use neo4j over http (e.g. in the browser with /webadmin) or via this script: https://gist.github.com/jexp/a9bf4cd3a16eb6c5997b
This key remote_shell_host=127.0.0.1 that enables you to just connect via loopback. You have to specify by which network adapter you will receive the connection. You can also specify 0.0.0.0 for any your network interfaces.