How to connect telegraf to kafka at docker-compose - docker

Question

I have at docker zookeeper, kafka, telegraf, influxdb, postgres and other
version: '2.1'
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
healthcheck:
test: echo stat | nc localhost 2181
interval: 10s
timeout: 10s
retries: 3
kafka:
image: wurstmeister/kafka:0.11.0.0
ports:
- "9092:9092"
environment:
OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_BROKER_ID: 3
KAFKA_ADVERTISED_PORT: 9092
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
LOG4J_LOGGER_KAFKA_AUTHORIZER_LOGGER: TRACE
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_CREATE_TOPICS: "testservise:1:1,testServiceInQueue:1:1,anotherTestServiceInQueue:1:1,storageServiceInQueue:1:1"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
zookeeper:
condition: service_healthy
influxdb:
image: influxdb:latest
container_name: influxdb
ports:
- "8083:8083"
- "8086:8086"
- "8090:8090"
telegraf:
image: telegraf:latest
restart: always
container_name: telegraf
hostname: localhost
links:
- influxdb
volumes:
- ${PWD}/.telegraf/telegraf.conf
First of all I do this
telegraf -sample-config -input-filter kafka_consumer -output-filter influxdb > telegraf.conf
Cause need to use telegraf /plugins/inputs/kafka_consumer
Is it right docker-compose file?
After that I need to get out information from kafka, when my test service send something, how I can do it or how to right google it?

Answer 1

Your compose file looks fine
By default, I believe the telegraf config looks like so
[[inputs.kafka_consumer]]
## kafka servers
brokers = ["localhost:9092"]
## topic(s) to consume
topics = ["telegraf"]
You'll need to use kafka:9092 and update the topics accordingly. And you'll need to also update the influx output probably away from localhost as well
You could also use Kafka Connect for writing kafka messages to InfluxDB