How can I start Zookeper and Kafka both at the same time? - docker

Question

I wrote a python app that uses Kafka to share data between producer and consumer. I am going to dockerize it and so I wrote the dockerfile. The problem is in the final steps, when I run the Zookeeper server, it says binding to port .... and it never exits, so that it would be possible to run Kafka server also.
FROM ubuntu:18.04
ADD app.py /
RUN apt-get update -y && apt-get install -y apt-transport-https
RUN apt-get update \
&& apt-get install -y wget \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get install gzip
RUN apt-get update \
&& apt-get install -y default-jdk \
&& rm -rf /var/lib/apt/lists/*
RUN wget http://mirrors.estointernet.in/apache/kafka/2.2.0/kafka_2.11-2.2.0.tgz
RUN tar -xvzf kafka_2.11-2.2.0.tgz
RUN cd kafka_2.11-2.2.0 \
&& ./bin/zookeeper-server-start.sh ./config/zookeeper.properties \
&& ./bin/kafka-server-start.sh ./config/server.properties
CMD [ “python”, “.app.py”]

Answer 1

Typically, you would only need to create a container for your own application. There are already numerous containers available for Kafka and Zookeeper.
A suggestion would be to run a simple Kafka stack using docker compose and then either start your application externally (for instance via command-line or IDE) or create a docker container and run it.
A simple Kafka stack with just Kafka and Zookeeper would require a simple docker-compose file with the following (of course you will need to have docker-compose installed on your system):
version: '3'
services:
zookeeper:
image: confluentinc/cp-zookeeper:4.0.0
restart: unless-stopped
ports:
- 2181:2181
environment:
ZOOKEEPER_SERVER_ID: 1
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_INIT_LIMIT: 5
ZOOKEEPER_SYNC_LIMIT: 2
kafka:
image: confluentinc/cp-kafka:4.0.0-2
depends_on:
- zookeeper
ports:
- 9092:9092
environment:
KAFKA_BROKER_ID: 1
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
KAFKA_LOG4J_LOGGERS: "kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO"
If you put the above in a file named simple-kafka-stack.yml, you can execute it by running the command:
docker-compose -f simple-kafka-stack.yml up