I assume that sbt
is installed directly on your OS.
Defined docker-compose
starts up a zookeeper, kafka and schema-registry instance as well as
tools container with confluent-cli-tools installed.
Scripts from kafka-cli-scripts
directory are mounted to the tools container.
Run start-kafka.sh
script. It will start up docker network and create topics as defined in kafka-cli-scripts/create-topics.sh
.
Directory contains 4 sbt projects each being independent app:
- avro - generates and registers Avro schemas for data used as keys/values in created topics
- car-data-producer - produces random data to kafka topics
- driver-notifier - kafka streams application which aggregates data from several topics, processes and produces to
driver-notifications
- car-data-consumer - can consume data from any of created kafka topics (
driver-notifications
by default)
Please also add this two entries to your etc/hosts
, since apps above reach kafka using docker hostnames:
# streams-app host entries
127.0.0.1 kafka
127.0.0.1 schema-registry
With sbt
each app can be started in separate shell instance with the following command:
sbt "project <name>" "run"
, for an ex. sbt "project carDataProducer" "run"
.
Please run avro
and carDataProducer
first.
If you are Intellij user then most convenient option will be to use IDE for running apps, since you can easily tweak, debug and experiment with the code.
Run stop-kafka.sh
script.