This question already has answers here:
Connect to Kafka running in Docker
(5 answers)
Closed 1 year ago.
I'm trying to get a simple Debezium stack (with Docker Compose) running but the connection to the Kafka broker fails.
Here is my simplified docker-compose.yml:
version: "3"
services:
zookeeper:
image: debezium/zookeeper:1.6
ports:
- "2181:2181"
- "2888:2888"
- "3888:3888"
kafka:
image: debezium/kafka:1.6
links:
- zookeeper
ports:
- "9092:9092"
environment:
BROKER_ID: 1
KAFKA_LISTENERS: LISTENER_BOB://kafka:29092,LISTENER_FRED://localhost:9092
KAFKA_ADVERTISED_LISTENERS: LISTENER_BOB://kafka:29092,LISTENER_FRED://localhost:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_BOB:PLAINTEXT,LISTENER_FRED:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_BOB
ZOOKEEPER_CONNECT: zookeeper:2181
As you may notice I added the listeners according to Kafka Listeners - Explained. But when I use kafkacat -b localhost:9092 -L to retrieve the broker's metadata the following error appears.
%6|1627623916.693|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Disconnected while requesting ApiVersion: might be caused by incorrect security.protocol configuration (connecting to a SSL listener?) or broker version is < 0.10 (see api.version.request) (after 2ms in state APIVERSION_QUERY)
%6|1627623916.961|FAIL|rdkafka#producer-1| [thrd:localhost:9092/bootstrap]: localhost:9092/bootstrap: Disconnected while requesting ApiVersion: might be caused by incorrect security.protocol configuration (connecting to a SSL listener?) or broker version is < 0.10 (see api.version.request) (after 3ms in state APIVERSION_QUERY, 1 identical error(s) suppressed)
% ERROR: Failed to acquire metadata: Local: Broker transport failure
Kafka is even not able to start when I configure Kafka as Debezium suggests here.
2021-07-30 05:48:38,591 - WARN [Controller-1-to-broker-1-send-thread:NetworkClient#780] - [Controller id=1, targetBrokerId=1] Connection to node 1 (/localhost:9092) could not be established. Broker may not be available.
What am I doing wrong?
Thank you for any help in advance!
You've set a listener to only be local, not external.
Change to
KAFKA_LISTENERS: LISTENER_BOB://kafka:29092,LISTENER_FRED://0.0.0.0:9092
Related
I've been developing a Kafka stream processing application with the Quarkus-Framework in Java. Now I'm trying to connect to the Kafka brokers via the SASL/PLAIN mechanism, but am getting the following error:
2022-10-27 10:52:06,736 ERROR [org.apa.kaf.cli.NetworkClient] (kafka-admin-client-thread | alarms-preprocessor-dev-a8147d3e-809c-4e96-9ce0-de10e55a8d72-admin) [AdminClient clientId=alarms-preprocessor-dev-a8147d3e-809c-4e96-9ce0-de10e55a8d72-admin] Connection to node -1 (localhost/127.0.0.1:29092) failed authentication due to: Unexpected handshake request with client mechanism PLAIN, enabled mechanisms are []
Apparently, the brokers do not have the PLAIN mechanism enabled, which begs the question why my Kafka-Connect-Service is able to sue the PLAIN-mechanism.
Anyway, this is my broker-configuration (approximately the same for all 3 instances) using confluentinc/cp-kafka docker image with docker-compose:
broker-1:
image: confluentinc/cp-kafka:7.2.1
hostname: broker-1
container_name: broker-1
depends_on:
- zookeeper
ports:
- "29092:29092"
- "9092:9092"
- "9091:9091"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker-1:9092,SASL_PLAINTEXT://broker-1:9091,PLAINTEXT_HOST://localhost:29092
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OPTS: "-Djava.security.auth.login.config=/etc/kafka/kafka_jaas.conf"
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
KAFKA_AUTHORIZER_CLASS_NAME: kafka.security.authorizer.AclAuthorizer
KAFKA_ALLOW_EVERYONE_IF_NO_ACL_FOUND: "true"
volumes:
- /home/larissa/Projekte/SRE/Kafka/local_dev_cluster/files/kafka_jaas.conf:/etc/kafka/kafka_jaas.conf
and this is part of the output from docker logs broker-1 | grep PLAIN:
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
advertised.listeners = PLAINTEXT://broker-1:9092,SASL_PLAINTEXT://broker-1:9091,PLAINTEXT_HOST://localhost:29092
inter.broker.listener.name = PLAINTEXT
listener.security.protocol.map = PLAINTEXT:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
listeners = PLAINTEXT://0.0.0.0:9092,SASL_PLAINTEXT://0.0.0.0:9091,PLAINTEXT_HOST://0.0.0.0:29092
sasl.enabled.mechanisms = [PLAIN]
security.inter.broker.protocol = PLAINTEXT
The part that says "sasl.enabled.mechanisms = [PLAIN]" suggests that the PLAIN mechanism is indeed enabled. So maybe it's a problem with my Quarkus application configuration, which looks like this:
quarkus.kafka-streams.application-id=alarms-preprocessor-dev
quarkus.kafka-streams.bootstrap-servers=localhost:29092,localhost:29192,localhost:29292
quarkus.kafka-streams.topics=test_topic
quarkus.kafka-streams.security.protocol=SASL_PLAINTEXT
quarkus.kafka-streams.sasl.mechanism=PLAIN
quarkus.kafka-streams.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="admin" \
password="admin-secret" \
serviceName="alarms-preprocessor";
All JAAS-configs, users and passwords are correct by the way, since they work with Kafka Connect just fine. If necessary, I can provide those too, just let me know.
Thanks in advance for any answers :)
I have two VPCs in AWS:
VPC-A has an ec2 instance in it.
VPC-B has an ec2 instance in it running kafka and zookeeper via docker-compose
The VPCs are connected via AWS Privatelink (endpoint --> endpoint service --> nlb (in VPC-B) --> kafka)
I have given the privatelink endpoint a DNS name: broker.confluent-playground
I can telnet fine to both port 9092 and 2181 from VPC A to Kafka and Zookeeper in VPC-B. No problems
[ec2-user#ip-10-1-0-90 etc]$ telnet broker.confluent-playground 9092
Trying 10.1.1.200...
Connected to broker.confluent-playground.
My problem is that while the network connectivity is there between the VPCs, I seem to be having problems with the kafka listener configuration. When I set the advertised.listeners to broker.confluent-playground:9092, my producer cannot seem to connect, and when running kafkacat -b broker.confluent-playground -L I only list 7 of the 40 topics. (they seem like internal system topics eg."_confluent_balancer_partition_samples").
Here is an excerpt from the docker-compose file:
broker:
image: confluentinc/cp-server:6.1.1
hostname: broker
container_name: broker
depends_on:
- zookeeper
ports:
- "9092:9092"
- "9101:9101"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: INTERNAL://broker:29092,EXTERNAL://broker.confluent-playground:9092
KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
Now if I change the advertised listener to the private IP of the ec2 holding kafka ie:
FROM
EXTERNAL://broker.confluent-playground:9092
to
EXTERNAL://192.168.35.65:9092
then kafkacat in VPC-A can see all the 40 topics correctly! I still cannot produce (because i assume i have been given an advertised private ip address in a different network).
I seem to have these listener configurations messed up somewhere and I am confused how by changing the advertised listener I can get a subset of topics with one setting and then all the topics with another.
Another interesting thing
My kafkacat scanning using broker.confluent-playground advertised listener returns this
[ec2-user#ip-10-1-0-90 ~]$ sudo docker run --rm --network=host edenhill/kafkacat:1.5.0 kafkacat -b broker.confluent-playground:9092 -L
Metadata for all topics (from broker 1: broker.confluent-playground:9092/1):
1 brokers:
broker 1 at broker.confluent-playground:9092 (controller)
7 topics:
And when I use the private IP address as (that I cannot reach from VPC-A) as the advertised listener I get the /bootstrap with all the topics:
[ec2-user#ip-10-1-0-90 ~]$ sudo docker run --rm --network=host edenhill/kafkacat:1.5.0 kafkacat -b broker.confluent-playground:9092 -L
Metadata for all topics (from broker -1: broker.confluent-playground:9092/bootstrap):
1 brokers:
broker 1 at 192.168.54.226:9092 (controller)
40 topics:
I am using Kafka docker-compose setting with below docker-compose.yml installed in VM-Ware machine.
When I connect to it by pyspark.streaming.kafka.KafkaUtils, it released some errors.
Please help me resolve this problems.
I used configuration from https://rmoff.net/2018/08/02/kafka-listeners-explained/
docker-compose.yml file
version: '3.7'
services:
zookeeper:
image: "confluentinc/cp-zookeeper:latest"
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ports:
- "2181:2181"
# This has three listeners you can experiment with.
# BOB for internal traffic on the Docker network
# FRED for traffic from the Docker-host machine (`localhost`)
# ALICE for traffic from outside, reaching the Docker host on the 192.168.231.145
# Use
kafka0:
image: "confluentinc/cp-enterprise-kafka:latest"
ports:
- '9092:9092'
- '29094:29094'
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 0
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_LISTENERS: LISTENER_BOB://kafka0:29092,LISTENER_FRED://kafka0:9092,LISTENER_ALICE://0.0.0.0:29094
KAFKA_ADVERTISED_LISTENERS: LISTENER_BOB://kafka0:29092,LISTENER_FRED://localhost:9092,LISTENER_ALICE://192.168.231.145:29094
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_BOB:PLAINTEXT,LISTENER_FRED:PLAINTEXT,LISTENER_ALICE:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_BOB
KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 100
kafkacat:
image: confluentinc/cp-kafkacat
command: sleep infinity
python code i used to connect from vm-ware hosted machine
from pyspark import SparkContext
from pyspark.streaming import StreamingContext
from pyspark.streaming.kafka import KafkaUtils
from confluent_kafka import Producer, Consumer
import socket
import json
if __name__ == "__main__":
sc = SparkContext(appName="Processing_raw_data")
ssc = StreamingContext(sc, 1)
in_stream = KafkaUtils.createStream(ssc, "192.168.231.145:2181", socket.gethostname(), {"testing": 1}, {"auto.offset.reset": "smallest"})
in_stream.pprint()
ssc.start()
ssc.awaitTermination()
Errors
21/06/19 19:44:24 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
21/06/19 19:44:24 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
-------------------------------------------
Time: 2021-06-19 19:44:27
-------------------------------------------
21/06/19 19:44:27 WARN AppInfo$: Can't read Kafka version from MANIFEST.MF. Possible cause: java.lang.NullPointerException
[Stage 0:> (0 + 1) / 1]-------------------------------------------
Time: 2021-06-19 19:44:28
-------------------------------------------
21/06/19 19:44:28 WARN ClientUtils$: Fetching topic metadata with correlation id 0 for topics [Set(testing)] from broker [id:0,host:kafka0,port:29092] failed
java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
21/06/19 19:44:28 WARN ConsumerFetcherManager$LeaderFinderThread: [dino-computer_dino-computer-1624106667579-fbe9ab2d-leader-finder-thread], Failed to find leader for Set([testing,0])
kafka.common.KafkaException: fetching topic metadata for topics [Set(testing)] from broker [ArrayBuffer(id:0,host:kafka0,port:29092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 3 more
This question already has answers here:
Connect to Kafka running in Docker
(5 answers)
Closed last year.
I start kafka with this docker-compose.yml on my Mac:
mydb:
[...]
environment:
kafka_bootstrap_servers: kafka:9093
kafka:
image: wurstmeister/kafka:2.13-2.6.0
ports:
- "9092:9092"
environment:
KAFKA_CREATE_TOPICS: "mytopic:1:1"
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
#KAFKA_LISTENERS: PLAINTEXT://:9092
#KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://127.0.0.1:9092
KAFKA_LISTENERS: LISTENER_INTERN://:9093,LISTENER_EXTERN://127.0.0.1:9092
KAFKA_ADVERTISED_LISTENERS: LISTENER_INTERN://:9093,LISTENER_EXTERN://127.0.0.1:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_INTERN:PLAINTEXT,LISTENER_EXTERN:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_EXTERN
KAFKA_BROKER_ID: 1
volumes:
- /var/run/docker.sock:/var/run/docker.sock
I have two LISTENER because there is a second container mydb that does not work with the PLAINTEXT://127.0.0.1:9092 listener.
In fact, using
KAFKA_LISTENERS: PLAINTEXT://:9092
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://127.0.0.1:9092
works with kafkacat. But then the other container can`t connect to kafka.
Now I want to connect to kafka via
kafkacat -b 127.0.0.1:9092 -t mytopic -L
but it gives this error
%6|1605191095.939|FAIL|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]:
127.0.0.1:9092/bootstrap:
Disconnected while requesting ApiVersion: might be caused by incorrect security.protocol configuration
(connecting to a SSL listener?) or broker version is < 0.10 (see api.version.request) (after 2ms in state APIVERSION_QUERY)
the debug log for the broker is:
%7|1605191334.678|BROKER|rdkafka#producer-1| [thrd:app]: 127.0.0.1:9092/bootstrap: Added new broker with NodeId -1
%7|1605191334.678|BRKMAIN|rdkafka#producer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:app]: 127.0.0.1:9092/bootstrap: Selected for cluster connection: bootstrap servers added (broker has 0 connection attempt(s))
%7|1605191334.678|BRKMAIN|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Enter main broker thread
%7|1605191334.678|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v1.5.0 (0x10500ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer, CC CXX PKGCONFIG OSXLD LIBDL PLUGINS ZLIB SSL SASL_CYRUS ZSTD HDRHISTOGRAM LZ4_EXT SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0x2)
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Received CONNECT op
%7|1605191334.678|STATE|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: broker in state TRY_CONNECT connecting
%7|1605191334.678|STATE|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: leader query
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
%7|1605191334.678|CONNECT|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Connecting to ipv4#127.0.0.1:9092 (plaintext) with socket 7
%7|1605191334.679|CONNECT|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Connected to ipv4#127.0.0.1:9092
%7|1605191334.679|CONNECTED|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Connected (#1)
%7|1605191334.679|FEATURE|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1605191334.679|STATE|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY
%7|1605191334.679|CONNECT|rdkafka#producer-1| [thrd:app]: Not selecting any broker for cluster connection: still suppressed for 49ms: application metadata request
%7|1605191334.681|FAIL|rdkafka#producer-1| [thrd:127.0.0.1:9092/bootstrap]: 127.0.0.1:9092/bootstrap: Disconnected while requesting ApiVersion: might be caused by incorrect security.protocol configuration (connecting to a SSL listener?) or broker version is < 0.10 (see api.version.request) (after 2ms in state APIVERSION_QUERY) (_TRANSPORT)
What am I doing wrong?
I played around with your particular example, and couldn't get it to work.
For what it's worth, this Docker Compose is how I run Kafka on Docker locally, and it's accessible both from the host machine, and other containers.
You might find this blog useful if you want to continue with your existing approach and debug it further.
You've only set the extern listener to bind internal to the container and not allow remote connections
You'd need
KAFKA_LISTENERS: LISTENER_INTERN://kafka:9093,LISTENER_EXTERN://0.0.0.0:9092
KAFKA_ADVERTISED_LISTENERS: LISTENER_INTERN://kafka:9093,LISTENER_EXTERN://127.0.0.1:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_INTERN:PLAINTEXT,LISTENER_EXTERN:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_INTERN
I'm testing a sample spring cloud stream application (running on a Ubuntu linux machine) with one source and one sink services. All my services are docker-containerized and I would like to use kafka as message broker.
Below the relevant parts of the docker-compose.yml:
zookeeper:
image: confluent/zookeeper
container_name: zookeeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka:0.9.0.0-1
container_name: kafka
ports:
- "9092:9092"
links:
- zookeeper:zk
environment:
- KAFKA_ADVERTISED_HOST_NAME=192.168.33.101
- KAFKA_ADVERTISED_PORT=9092
- KAFKA_DELETE_TOPIC_ENABLE=true
- KAFKA_LOG_RETENTION_HOURS=1
- KAFKA_MESSAGE_MAX_BYTES=10000000
- KAFKA_REPLICA_FETCH_MAX_BYTES=10000000
- KAFKA_GROUP_MAX_SESSION_TIMEOUT_MS=60000
- KAFKA_NUM_PARTITIONS=2
- KAFKA_DELETE_RETENTION_MS=1000
.
.
.
# not shown: eureka service registry, spring cloud config service, etc.
myapp-service-test-source:
container_name: myapp-service-test-source
image: myapp-h2020/myapp-service-test-source:0.0.1
environment:
SERVICE_REGISTRY_HOST: 192.168.33.101
SERVICE_REGISTRY_PORT: 8761
ports:
- 8081:8080
.
.
.
Here the relevant part of application.yml for my service-test-source service:
spring:
cloud:
stream:
defaultBinder: kafka
bindings:
output:
destination: messages
content-type: application/json
kafka:
binder:
brokers: ${SERVICE_REGISTRY_HOST:192.168.33.101}
zkNodes: ${SERVICE_REGISTRY_HOST:192.168.33.101}
defaultZkPort: 2181
defaultBrokerPort: 9092
The problem is the following, if I launch the docker-compose above, in the test-source container log I notice that the service fails to connect to zookeeper, giving a repeated set of Connection refused error, and finishing with a ZkTimeoutException which makes the service terminate (see below).
The strange fact is that, if instead of running my source (and sink) test services as docker containers I run them as jar files via maven mvn spring-boot:run <etc...> the services work fine and are able to exchange messages via kafka. (note that kafka, zookeeper, etc. are still running as docker containers).
.
.
.
*** THE FOLLOWING REPEATED n TIMES ***
2017-02-14 14:40:09.164 INFO 1 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2017-02-14 14:40:09.166 WARN 1 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_111]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[na:1.8.0_111]
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) ~[zookeeper-3.4.6.jar!/:3.4.6-1569965]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) ~[zookeeper-3.4.6.jar!/:3.4.6-1569965]
.
.
.
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.context.ApplicationContextException: Failed to start bean 'outputBindingLifecycle'; nested exception is org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 10000
Any idea what the problem might be?
edit:
I discovered that in the "jar" execution logs the test-source service tries to connect to zookeeper through the IP 127.0.0.1, as can be seen from the log snipped below:
2017-02-15 14:24:04.159 INFO 10348 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2017-02-15 14:24:04.159 INFO 10348 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2017-02-15 14:24:04.178 INFO 10348 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Socket connection established to localhost/127.0.0.1:2181, initiating session
2017-02-15 14:24:04.201 INFO 10348 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x15a421fd9ec000a, negotiated timeout = 10000
2017-02-15 14:24:05.870 INFO 10348 --- [ main] org.apache.zookeeper.ZooKeeper : Initiating client connection, connectString=localhost:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient#72ba68e3
2017-02-15 14:24:05.882 INFO 10348 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2017-02-15 14:24:05.883 INFO 10348 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
This explains why everything works on the jar execution but not the docker one (the zookeeper container exports its 2181 port to the host machine, so it's visible as localhost for the service process when running directly on the host machine), but doesn't solve the problem: Apparently the spring cloud stream kafka configuration is ignoring the property spring.cloud.stream.kafka.binder.zkNodes as set in the application.yml (note that if I log the value of such environment variable from the service, I see the correct value of 192.168.33.101 that I hardcoded there for debugging purposes).
You have set the defaultBinder to be rabbit while trying to use the Kafka binder configuration. Do you have both rabbit and kafka binders in the classpath of your application? In that case, you can enable here
zookeeper:
image: wurstmeister/zookeeper
container_name: 'zookeeper'
ports:
- 2181:2181
--------------------- kafka --------------------------------
kafka:
image: wurstmeister/kafka
container_name: 'kafka'
environment:
- KAFKA_ADVERTISED_HOST_NAME=kafka
- KAFKA_ADVERTISED_PORT=9092
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_CREATE_TOPICS=kafka_docker_topic:1:1
ports:
- 9092:9092
depends_on:
- zookeeper
spring:
profiles: dev
cloud:
stream:
defaultBinder: kafka
kafka:
binder:
brokers: kafka # i added brokers and zkNodes property
zkNodes: zookeeper #
bindings:
input:
destination: message
content-type: application/json