landoop kafka dev schema registry - docker

Question

I have been trying to use the following docker-compose file:
version: '2'
services:
# this is our kafka cluster.
kafka-cluster:
image: landoop/fast-data-dev:cp3.3.0
environment:
ADV_HOST: 127.0.0.1 # Change to 192.168.99.100 if using Docker Toolbox
RUNTESTS: 0 # Disable Running tests so the cluster starts faster
FORWARDLOGS: 0
SAMPLEDATA: 0
ports:
- 2181:2181 # Zookeeper
- 3030:3030 # Landoop UI
- 8082-8084:8081-8083 # REST Proxy, Schema Registry, Kafka Connect ports
- 9581-9585:9581-9585 # JMX Ports
- 9092:9092 # Kafka Broker
This starts a kafka cluster in my local machine.
Now it works well when sending messages to localhost:9092 with a custom serializer. I am now trying to add support for apache avro by using the schema registry that is packed with the docker image. I have created the schema inside the schema registry and I have a java object coming from an avsc file that I am using inside the kafkatemplate send method. I added the following line to connect to the schema registry server:
configProps.put(KafkaAvroSerializerConfig.SCHEMA_REGISTRY_URL_CONFIG,"http://localhost:8081");
This results in the following exception:
org.apache.kafka.common.errors.SerializationException: Error serializing Avro message
Caused by: java.net.ConnectException: Connection refused: connect
at java.base/java.net.PlainSocketImpl.waitForConnect(Native Method) ~[na:na]
at java.base/java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:107) ~[na:na]
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399) ~[na:na]
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242) ~[na:na]
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224) ~[na:na]
at java.base/java.net.Socket.connect(Socket.java:609) ~[na:na]
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177) ~[na:na]
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474) ~[na:na]
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569) ~[na:na]
at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:242) ~[na:na]
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:341) ~[na:na]
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:362) ~[na:na]
I think this is an authentication issue, so I've been looking at the following documentation for instructions: https://hub.docker.com/r/landoop/fast-data-dev
So far I can't find a default username and password to perform the authentication against the schema registry, or a way to create my own user and password.
Any help will be much appreciated

Answer 1

Assuming the code is running on the host, you aren't forwarding localhost:8081 from the host - refer 8082-8084:8081-8083.
Unless you have some error, you should use direct matching ports on the host to the container
If are running code in a container, you need to use http://kafka-cluster:8081 while running the application container on the same network bridge