FlameStream

Overview

FlameStream is a distributed stream processing model, that has the following properties:

• Pure streaming: records are processed one at a time
• Deterministic: results are determined only by input and not changed between independent runs
• Consistency: model provides for exactly-once semantics

The distributed implementation of FlameStream model is written in Java and uses the Akka Actors framework for messaging.

The implementation is based on the following grounds:

• Determinism is achieved via strong ordering
• Idempotence via determinism
• Exactly-once via idempotence

Our road to exactly-once in comparison with other approaches:

Unlike common models, FlameStream has reduced set of operations: windowed grouping and stateless map, which are enough to implement any stateful pipelines. Such limitation allows achieving strong ordering with low overhead using the lightweight optimistic techniques.

Benchmarking

See details in benchmarking readme.