Batteries included!

Using akka-streams, mirror one Kafka topic to another.
Serve as a base building block for any Kafka Stream applications using Scala.
Comes with an SBT config, and a .travis.yml that can publish straight to Docker.

• Usage
  • kafka-mirror on Docker Hub
• Technical choices & relevant documentation
• Other notes
• Licence

Usage

$ sbt new ScalaWilliam/akka-stream-kafka-template.g8

And then look at README.md.

kafka-mirror on Docker Hub

kafka-mirror is automatically built from this template and published to Docker Hub:

• https://hub.docker.com/r/scalawilliam/kafka-mirror/

Technical choices & relevant documentation

• Configuration options are parsed by Typesafe Config, the defacto configuration library for Scala.
• akka-stream-kafka Consumer wraps the Kafka Consumer in a much more usable way.
• The record goes through akka-streams with an offset of the input record. Akka streams provides very powerful streaming abstractions that can be used with many connectors, including MQTT, AMQP, SQS.
• Once the akka-stream-kafka Producer pushes the message the consumer offset is committed automatically. This enables resiliency.
• SBT compiles our source code. Read Essential SBT to get more familiar with SBT.
• sbt-native-packager packages the Scala application into a deployable unit with its dependencies.
• The sbt-native-packager Docker Plugin will go the extra mile to build your Docker image and publish it where you ask it to.
• By default Docker (What is Docker?) will publish to the Docker Hub which you should sign up to.
• And finally, for logging, we use logback.

Also, for sbt new support:

• Using the Giter8 templating.
• sbt new documentation.
• One example: https://github.com/typesafehub/stack-examples.g8
• And for reference: https://github.com/lloydmeta/scalameta.g8

Other notes

To regenerate TOC we use markdown-toc:

$ npm install -g markdown-toc
$ markdown-toc -i README.md

Licence

• MIT. Copyright (2016) Apt Elements Ltd. William Narmontas