EventSourcing.NetCore

Tutorial, practical samples and other resources about Event Sourcing in .NET Core.

• EventSourcing.NetCore
  • 1. Support
  • 2. Prerequisites
  • 3. Libraries used
  • 4. Event Store - Marten
  • 5. Message Bus (for processing Commands, Queries, Events) - MediatR
  • 6. CQRS (Command Query Responsibility Separation)
  • 7. Fully working sample application
  • 8. Self-paced training Kit
  • 9. NuGet packages to help you get started.
  • 10. Other resources
    • 10.1 Various
    • 10.2 Snapshots
    • 10.3 Projections
    • 10.4 Event processing
    • 10.5 Distributed processes
    • 10.6 Modeling
  • 11. Contributors
    • 11.1 Code Contributors
    • 11.2 Financial Contributors
      • Individuals
      • Organizations

1. Support

Feel free to create an issue if you have any questions or request for more explanation or samples. I also take Pull Requests!

💖 If this repository helped you - I'd be more than happy if you join the group of my official supporters at:

👉 Github Sponsors

2. Prerequisites

For running the Event Store examples you need to have:

1. .NET 5 installed - https://dotnet.microsoft.com/download/dotnet/5.0
2. Postgres DB. You can get it by:

• Installing Docker, going to the docker folder and running:

docker-compose up

• Installing a most recent version of the Postgres DB (eg. from https://www.postgresql.org/download/).

More information about using .NET Core, WebApi and Docker you can find in my other tutorials: WebApi with .NET

Watch "Practical Event Sourcing with Marten":

Slides:

• Practical Event Sourcing with Marten - EN, PL
• Ligths and Shades of Event-Driven Design - EN, PL
• Adventures in Event Sourcing and CQRS - PL

3. Libraries used

1. Marten - Event Store

2. MediatR - Message Bus (for processing Commands, Queries, Events)

4. Event Store - Marten

• Creating event store
• Event Stream - is a representation of the entity in event sourcing. It's a set of events that happened for the entity with the exact id. Stream id should be unique, can have different types but usually is a Guid.
  • Stream starting - stream should be always started with a unique id. Marten provides three ways of starting the stream:
    • calling StartStream method with a stream id

      var streamId = Guid.NewGuid();
      documentSession.Events.StartStream<IssuesList>(streamId);
      

    • calling StartStream method with a set of events

      var @event = new IssueCreated { IssueId = Guid.NewGuid(), Description = "Description" };
      var streamId = documentSession.Events.StartStream<IssuesList>(@event);
      

    • just appending events with a stream id

      var @event = new IssueCreated { IssueId = Guid.NewGuid(), Description = "Description" };
      var streamId = Guid.NewGuid();
      documentSession.Events.Append(streamId, @event);
      

  • Stream loading - all events that were placed on the event store should be possible to load them back. Marten allows to:
    • get list of event by calling FetchStream method with a stream id

      var eventsList = documentSession.Events.FetchStream(streamId);
      

    • geting one event by its id

      var @event = documentSession.Events.Load<IssueCreated>(eventId);
      

  • Stream loading from exact state - all events that were placed on the event store should be possible to load them back. Marten allows to get stream from exact state by:
    • timestamp (has to be in UTC)

      var dateTime = new DateTime(2017, 1, 11);
      var events = documentSession.Events.FetchStream(streamId, timestamp: dateTime);
      

    • version number

      var versionNumber = 3;
      var events = documentSession.Events.FetchStream(streamId, version: versionNumber);
      

• Event stream aggregation - events that were stored can be aggregated to form the entity once again. During the aggregation, process events are taken by the stream id and then replied event by event (so eg. NewTaskAdded, DescriptionOfTaskChanged, TaskRemoved). At first, an empty entity instance is being created (by calling default constructor). Then events based on the order of appearance (timestamp) are being applied on the entity instance by calling proper Apply methods.
  • Aggregation general rules
  • Online Aggregation - online aggregation is a process when entity instance is being constructed on the fly from events. Events are taken from the database and then aggregation is being done. The biggest advantage of online aggregation is that it always gets the most recent business logic. So after the change, it's automatically reflected and it's not needed to do any migration or updates.
  • Inline Aggregation (Snapshot) - inline aggregation happens when we take the snapshot of the entity from the DB. In that case, it's not needed to get all events. Marten stores the snapshot as a document. This is good for performance reasons because only one record is being materialized. The con of using inline aggregation is that after business logic has changed records need to be reaggregated.
  • Reaggregation - one of the biggest advantages of the event sourcing is flexibility to business logic updates. It's not needed to perform complex migration. For online aggregation it's not needed to perform reaggregation - it's being made always automatically. The inline aggregation needs to be reaggregated. It can be done by performing online aggregation on all stream events and storing the result as a snapshot.
    • reaggregation of inline snapshot with Marten

      var onlineAggregation = documentSession.Events.AggregateStream<TEntity>(streamId);
      documentSession.Store<TEntity>(onlineAggregation);
      documentSession.SaveChanges();
      

• Event transformations
  • One event to one object transformations
  • Inline Transformation storage
• Events projection
  • Projection of single stream
• Multitenancy per schema

5. Message Bus (for processing Commands, Queries, Events) - MediatR

• Initialization - MediatR uses services locator pattern to find a proper handler for the message type.
• Sending Messages - finds and uses the first registered handler for the message type. It could be used for queries (when we need to return values), commands (when we acting).
  • No Handlers - when MediatR doesn't find proper handler it throws an exception.
  • Single Handler - by implementing IRequestHandler we're deciding that this handler should be run asynchronously with other async handlers (so we don't wait for the previous handler to finish its work).
  • More Than One Handler - when there is more than one handler registered MediatR takes only one ignoring others when Send method is being called.
• Publishing Messages - finds and uses all registered handlers for the message type. It's good for processing events.
  • No Handlers - when MediatR doesn't find proper handler it throws an exception
  • Single Handler - by implementing INotificationHandler we're deciding that this handler should be run asynchronously with other async handlers (so we don't wait for the previous handler to finish its work)
  • More Than One Handler - when there is more than one handler registered MediatR takes all of them when calling Publish method
• Pipeline (to be defined)

6. CQRS (Command Query Responsibility Separation)

• Command handling
• Query handling

7. Fully working sample application

See also fully working sample application in Sample Project

• See sample how Entity Framework and Marten can coexist together with CQRS and Event Sourcing

8. Self-paced training Kit

I prepared the self-paced training Kit for the Event Sourcing. See more in the Workshop description.

It's split into two parts:

Event Sourcing basics - it teaches the event store basics by showing how to build your Event Store on Relational Database. It starts with the tables setup, goes through appending events, aggregations, projections, snapshots, and finishes with the Marten basics. See more in here.

1. Streams Table
2. Events Table
3. Appending Events
4. Optimistic Concurrency Handling
5. Event Store Methods
6. Stream Aggregation
7. Time Travelling
8. Aggregate and Repositories
9. Snapshots
10. Projections
11. Projections With Marten

Event Sourcing advanced topics - it's a real-world sample of the microservices written in Event-Driven design. It explains the topics of modularity, eventual consistency. Shows practical usage of WebApi, Marten as Event Store, Kafka as Event bus and ElasticSearch as one of the read stores. See more in here.

1. Meetings Management Module - the module responsible for creating, updating meeting details. Written in Marten in Event Sourcing pattern. Provides both write model (with Event Sourced aggregates) and read model with projections.
2. Meetings Search Module - responsible for searching and advanced filtering. Uses ElasticSearch as storage (because of its advanced searching capabilities). It's a read module that's listening for the events published by the Meetings Management Module.

9. NuGet packages to help you get started.

I gathered and generalized all of the practices used in this tutorial/samples in Nuget Packages maintained by me GoldenEye Framework. See more in:

• GoldenEye DDD package - it provides a set of base and bootstrap classes that helps you to reduce boilerplate code and help you focus on writing business code. You can find all classes like Commands/Queries/Event handlers and many more. To use it run:

  dotnet add package GoldenEye.Backend.Core.DDD

• GoldenEye Marten package - contains helpers, and abstractions to use Marten as document/event store. Gives you abstractions like repositories etc. To use it run:

  dotnet add package GoldenEye.Backend.Core.Marten

The simplest way to start is installing the project template by running

dotnet new -i GoldenEye.WebApi.Template.SimpleDDD

and then creating a new project based on it:

dotnet new SimpleDDD -n NameOfYourProject

10. Other resources

10.1 Various

• 📝 Greg Young - Building an Event Storage
• 📝 Mathias Verraes - DDD and Messaging Architectures
• 📝 Microsoft - Exploring CQRS and Event Sourcing
• 🎞 Greg Young - CQRS & Event Sourcing
• 📰 Lorenzo Nicora - A visual introduction to event sourcing and cqrs
• 🎞 Greg Young - A Decade of DDD, CQRS, Event Sourcing
• 🎞 Martin Fowler - The Many Meanings of Event-Driven Architecture
• 📝 Martin Fowler - Event Sourcing
• 📝 Wojciech Suwała - Building Microservices On .NET Core – Part 5 Marten An Ideal Repository For Your Domain Aggregates
• 🎞 Dennis Doomen - A practical introduction to DDD, CQRS & Event Sourcing
• 📝 Dennis Doomen - 16 design guidelines for successful Event Sourcing
• 📝 Eric Evans - DDD and Microservices: At Last, Some Boundaries!
• 🎞 Martin Kleppmann - Event Sourcing and Stream Processing at Scale
• 📝 Julie Lerman - Data Points - CQRS and EF Data Models
• 📝 Vaughn Vernon - Reactive DDD: Modeling Uncertainty
• 📝 Mark Seemann - CQS versus server-generated IDs
• 📦 Event Store - The open-source, functional database with Complex Event Processing in JavaScript
• 📝 Pedro Costa - Migrating to Microservices and Event-Sourcing: the Dos and Dont’s
• 📝 David Boike - Putting your events on a diet
• 📖 DDD Quickly
• 📝 Dennis Doomen - The Good, The Bad and the Ugly of Event Sourcing
• 📰 Dennis Doomen - Practical introduction to DDD, CQRS and Event Sourcing
• 📝 Versioning in an Event Sourced System
• 📰 Bartosz Sypytkowski - Collaborative Event Sourcing
• 📝 Jay Kreps - Why local state is a fundamental primitive in stream processing
• 🎞 Thomas Pierrain - As Time Goes By… (a Bi-temporal Event Sourcing story)
• 📝 Michiel Overeem, Marten Spoor, Slinger Jansen - The dark side of event sourcing: Managing data conversion
• 💻 Jakub Pilimon - DDD by Examples
• 🎞 Michiel Overeem - Event Sourcing after launch
• 🎞 Jimmy Bogard - Domain-Driven Design: The Good Parts
• 📝 Jimmy Bogard - CQRS and REST: the perfect match
• 📝 Microsoft - Domain events: design and implementation
• 🎞 Mathew McLoughlin - An Introduction to CQRS and Event Sourcing Patterns
• 🎞 Alexey Zimarev - DDD, Event Sourcing and Actors
• 🎞 Thomas Bøgh Fangel - Event Sourcing: Traceability, Consistency, Correctness
• 🎞 Roman Sachse - Event Sourcing - Do it yourself series

10.2 Snapshots

• 📝 Kacper Gunia - Event Sourcing: Snapshotting
• 📝 Event Store - Rolling Snapshots

10.3 Projections

• 📝 Alexey Zimarev - Projections in Event Sourcing
• 📝 Rinat Abdulin - Event Sourcing - Projections

10.4 Event processing

• 📝 Kamil Grzybek - The Outbox Pattern
• 🎞 Dev Mentors - Inbox & Outbox pattern - transactional message processing
• 📝 Jeremy D. Miller - Jasper’s “Outbox” Pattern Support
• 📝 Gunnar Morling - Reliable Microservices Data Exchange With the Outbox Pattern
• 📝 Microsoft - Asynchronous message-based communication
• 📝 NServiceBus - Outbox
• 📝 Alvaro Herrera - Implement SKIP LOCKED for row-level locks

10.5 Distributed processes

• 📝 Hector Garcaa-Molrna, Kenneth Salem - Sagas
• 🎞 Caitie McCaffrey - Applying the Saga Pattern
• 🎞 Udi Dahan - If (domain logic) then CQRS or Saga?
• 🎞 Gregor Hohpe - Starbucks Does Not Use Two-Phase Commit
• 📝 Microsoft - Design Patterns - Saga distributed transactions pattern
• 📝 Microsoft - Design Patterns - Choreography
• 🎞 Martin Schimak - Know the Flow! Events, Commands & Long-Running Services
• 📝 Martin Schimak - Aggregates and Sagas are Processes
• 🎞 Chris Richardson - Using sagas to maintain data consistency in a microservice architecture
• 📝 Thanh Le - What is SAGA Pattern and How important is it?
• 📝 Jimmy Bogard - Life Beyond Distributed Transactions: An Apostate's Implementation - Relational Resources
• 📝 Rinat Abdullin - Evolving Business Processes
• 📝 Microsoft - A Saga on Sagas
• 📝 NServiceBus - Sagas
• 📝 NServiceBus sagas: Integrations
• 📝 Denis Rosa (Couchbase) - Saga Pattern | Application Transactions Using Microservices

10.6 Modeling

• 📝 Event Modeling
• 📝 Event Storming
• 📝 Vaughn Vernon - Effective Aggregate Design Part I: Modeling a Single Aggregate

11. Contributors

11.1 Code Contributors

This project exists thanks to all the people who contribute. [Contribute].

11.2 Financial Contributors

Become a financial contributor and help us sustain our community. [Contribute]

Individuals

Organizations

Support this project with your organization. Your logo will show up here with a link to your website. [Contribute]

EventSourcing.NetCore is Copyright © 2017-2021 Oskar Dudycz and other contributors under the MIT license.