EventSourcing.JVM

Tutorial, practical samples and other resources about Event Sourcing in JVM. See also my similar repositories for .NET and NodeJS.

• EventSourcing.JVM
  • Event Sourcing
    • What is Event Sourcing?
    • What is Event?
    • What is Stream?
    • Event representation
    • Retrieving the current state from events
    • Event Store
  • Videos
    • Practical introduction to Event Sourcing with Spring Boot and EventStoreDB
    • Let's build the worst Event Sourcing system!
    • The Light and The Dark Side of the Event-Driven Design
    • Conversation with Yves Lorphelin about CQRS
    • How to deal with privacy and GDPR in Event-Sourced systems
  • Support
  • Introduction to Event Sourcing self-paced kit
    • Exercises
  • Samples
    • Event Sourcing with Spring Boot and EventStoreDB
      • Overview
      • Main assumptions
      • Prerequisites
      • Tools used
    • Event Versioning
    • Uniqueness
    • Distributed Processes
  • Articles

Event Sourcing

What is Event Sourcing?

Event Sourcing is a design pattern in which results of business operations are stored as a series of events.

It is an alternative way to persist data. In contrast with state-oriented persistence that only keeps the latest version of the entity state, Event Sourcing stores each state change as a separate event.

Thanks for that, no business data is lost. Each operation results in the event stored in the databse. That enables extended auditing and diagnostics capabilities (both technically and business-wise). What's more, as events contains the business context, it allows wide business analysis and reporting.

In this repository I'm showing different aspects, patterns around Event Sourcing. From the basic to advanced practices.

Read more in my article:

• 📝 How using events helps in a teams' autonomy
• 📝 When not to use Event Sourcing?

What is Event?

Events, represent facts in the past. They carry information about something accomplished. It should be named in the past tense, e.g. "user added", "order confirmed". Events are not directed to a specific recipient - they're broadcasted information. It's like telling a story at a party. We hope that someone listens to us, but we may quickly realise that no one is paying attention.

Events:

• are immutable: "What has been seen, cannot be unseen".
• can be ignored but cannot be retracted (as you cannot change the past).
• can be interpreted differently. The basketball match result is a fact. Winning team fans will interpret it positively. Losing team fans - not so much.

Read more in my articles:

• 📝 What's the difference between a command and an event?
• 📝 Events should be as small as possible, right?

What is Stream?

Events are logically grouped into streams. In Event Sourcing, streams are the representation of the entities. All the entity state mutations ends up as the persisted events. Entity state is retrieved by reading all the stream events and applying them one by one in the order of appearance.

A stream should have a unique identifier representing the specific object. Each event has its own unique position within a stream. This position is usually represented by a numeric, incremental value. This number can be used to define the order of the events while retrieving the state. It can be also used to detect concurrency issues.

Event representation

Technically events are messages.

They may be represented, e.g. in JSON, Binary, XML format. Besides the data, they usually contain:

• id: unique event identifier.
• type: name of the event, e.g. "invoice issued".
• stream id: object id for which event was registered (e.g. invoice id).
• stream position (also named version, order of occurrence, etc.): the number used to decide the order of the event's occurrence for the specific object (stream).
• timestamp: representing a time at which the event happened.
• other metadata like correlation id, causation id, etc.

Sample event JSON can look like:

{
  "id": "e44f813c-1a2f-4747-aed5-086805c6450e",
  "type": "invoice-issued",
  "streamId": "INV/2021/11/01",
  "streamPosition": 1,
  "timestamp": "2021-11-01T00:05:32.000Z",

  "data":
  {
    "issuedTo": {
      "name": "Oscar the Grouch",
      "address": "123 Sesame Street",
    },
    "amount": 34.12,
    "number": "INV/2021/11/01",
    "issuedAt": "2021-11-01T00:05:32.000Z"
  },

  "metadata": 
  {
    "correlationId": "1fecc92e-3197-4191-b929-bd306e1110a4",
    "causationId": "c3cf07e8-9f2f-4c2d-a8e9-f8a612b4a7f1"
  }
}

Retrieving the current state from events

In Event Sourcing, the state is stored in events. Events are logically grouped into streams. Streams can be thought of as the entities' representation. Traditionally (e.g. in relational or document approach), each entity is stored as a separate record.

Id	IssuerName	IssuerAddress	Amount	Number	IssuedAt
e44f813c	Oscar the Grouch	123 Sesame Street	34.12	INV/2021/11/01	2021-11-01

In Event Sourcing, the entity is stored as the series of events that happened for this specific object, e.g. InvoiceInitiated, InvoiceIssued, InvoiceSent.

[
    {
        "id": "e44f813c-1a2f-4747-aed5-086805c6450e",
        "type": "invoice-initiated",
        "streamId": "INV/2021/11/01",
        "streamPosition": 1,
        "timestamp": "2021-11-01T00:05:32.000Z",

        "data":
        {
            "issuedTo": {
                "name": "Oscar the Grouch",
                "address": "123 Sesame Street",
            },
            "amount": 34.12,
            "number": "INV/2021/11/01",
            "initiatedAt": "2021-11-01T00:05:32.000Z"
        }
    },        
    {
        "id": "5421d67d-d0fe-4c4c-b232-ff284810fb59",
        "type": "invoice-issued",
        "streamId": "INV/2021/11/01",
        "streamPosition": 2,
        "timestamp": "2021-11-01T00:11:32.000Z",

        "data":
        {
            "issuedTo": "Cookie Monster",
            "issuedAt": "2021-11-01T00:11:32.000Z"
        }
    },        
    {
        "id": "637cfe0f-ed38-4595-8b17-2534cc706abf",
        "type": "invoice-sent",
        "streamId": "INV/2021/11/01",
        "streamPosition": 3,
        "timestamp": "2021-11-01T00:12:01.000Z",

        "data":
        {
            "sentVia": "email",
            "sentAt": "2021-11-01T00:12:01.000Z"
        }
    }
]

All of those events shares the stream id ("streamId": "INV/2021/11/01"), and have incremented stream position. We can get to conclusion that in Event Sourcing entity is represented by stream, so sequence of event correlated by the stream id ordered by stream position.

To get the current state of entity we need to perform the stream aggregation process. We're translating the set of events into a single entity. This can be done with the following the steps:

1. Read all events for the specific stream.
2. Order them ascending in the order of appearance (by the event's stream position).
3. Construct the empty object of the entity type (e.g. with default constructor).
4. Apply each event on the entity.

This process is called also stream aggregation or state rehydration.

Read more in my article:

• 📝 Why Partial is an extremely useful TypeScript feature?
• 📝 How to get the current entity state from events?

Event Store

Event Sourcing is not related to any type of storage implementation. As long as it fulfils the assumptions, it can be implemented having any backing database (relational, document, etc.). The state has to be represented by the append-only log of events. The events are stored in chronological order, and new events are appended to the previous event. Event Stores are the databases' category explicitly designed for such purpose.

In the further samples, I'll use EventStoreDB. It's the battle-tested OSS database created and maintained by the Event Sourcing authorities. It supports many dev environments via gRPC clients, including JVM.

Read more in my article:

• 📝 What if I told you that Relational Databases are in fact Event Stores?

Videos

Practical introduction to Event Sourcing with Spring Boot and EventStoreDB

Facts and Myths about CQRS

Let's build the worst Event Sourcing system!

The Light and The Dark Side of the Event-Driven Design

Conversation with Yves Lorphelin about CQRS

How to deal with privacy and GDPR in Event-Sourced systems

Support

Feel free to create an issue if you have any questions or request for more explanation or samples. I also take Pull Requests!

💖 If this repository helped you - I'd be more than happy if you join the group of my official supporters at:

👉 Github Sponsors

⭐ Star on GitHub or sharing with your friends will also help!

Introduction to Event Sourcing self-paced kit

Event Sourcing is perceived as a complex pattern. Some believe that it's like Nessie, everyone's heard about it, but rarely seen it. In fact, Event Sourcing is a pretty practical and straightforward concept. It helps build predictable applications closer to business. Nowadays, storage is cheap, and information is priceless. In Event Sourcing, no data is lost.

The workshop aims to build the knowledge of the general concept and its related patterns for the participants. The acquired knowledge will allow for the conscious design of architectural solutions and the analysis of associated risks.

The emphasis will be on a pragmatic understanding of architectures and applying it in practice using Marten and EventStoreDB.

1. Introduction to Event-Driven Architectures. Differences from the classical approach are foundations and terminology (event, event streams, command, query).
2. What is Event Sourcing, and how is it different from Event Streaming. Advantages and disadvantages.
3. Write model, data consistency guarantees on examples from Marten and EventStoreDB.
4. Various ways of handling business logic: Aggregates, Command Handlers, functional approach.
5. Projections, best practices and concerns for building read models from events on the examples from Marten and EventStoreDB.
6. Challenges in Event Sourcing and EDA: deliverability guarantees, sequence of event handling, idempotency, etc.
7. Saga, Choreography, Process Manager, distributed processes in practice.
8. Event Sourcing in the context of application architecture, integration with other approaches (CQRS, microservices, messaging, etc.).
9. Good and bad practices in event modelling.
10. Event Sourcing on production, evolution, events' schema versioning, etc.

You can do the workshop as a self-paced kit. That should give you a good foundation for starting your journey with Event Sourcing and learning tools like Marten and EventStoreDB. If you'd like to get full coverage with all nuances of the private workshop, feel free to contact me via email.

Read also more in my article Introduction to Event Sourcing - Self Paced Kit.

Exercises

1. Events definition.
2. Getting State from events.
3. Appending Events:
   • EventStoreDB
   • TODO: Axon Server
4. Getting State from events
   • EventStoreDB
   • TODO: Axon Server
5. Business logic:
   • General
   • EventStoreDB
   • TODO: Axon Server
6. Optimistic Concurrency:
   • EventStoreDB
   • TODO: Axon Server
7. Projections:
   • General
   • Idempotency
   • Eventual Consistency

Samples

See also fully working, real-world samples of Event Sourcing and CQRS applications in Samples folder.

Samples are using CQRS architecture. They're sliced based on the business modules and operations. Read more about the assumptions in "How to slice the codebase effectively?".

Event Sourcing with Spring Boot and EventStoreDB

Overview

Sample is showing basic Event Sourcing flow. It uses EventStoreDB for event storage and Spring Data JPA backed with PostgreSQL for read models.

The presented use case is Shopping Cart flow:

1. The customer may add a product to the shopping cart only after opening it.
2. When selecting and adding a product to the basket customer needs to provide the quantity chosen. The product price is calculated by the system based on the current price list.
3. The customer may remove a product with a given price from the cart.
4. The customer can confirm the shopping cart and start the order fulfilment process.
5. The customer may also cancel the shopping cart and reject all selected products.
6. After shopping cart confirmation or cancellation, the product can no longer be added or removed from the cart.

Technically it's modelled as Web API written in Spring Boot and Java 17.

There are two variations of those samples:

• Event Sourcing with Spring Boot and EventStoreDB
• Event Sourcing with Spring Boot and EventStoreDB using Aggregate pattern

Main assumptions

• explain basics of Event Sourcing, both from the write model (EventStoreDB) and read model part (PostgreSQL and Spring Data JPA),
• present that you can join classical approach with Event Sourcing without making a massive revolution,
• CQRS architecture sliced by business features, keeping code that changes together at the same place. Read more in How to slice the codebase effectively?,
• clean, composable (pure) functions for command, events, projections, query handling, minimising the need for marker interfaces. Thanks to that testability and easier maintenance.
• easy to use and self-explanatory fluent API for registering commands and projections with possible fallbacks,
• registering everything into regular DI containers to integrate with other application services.
• pushing the type/signature enforcement on edge, so when plugging to DI.

Prerequisites

For running the Event Store examples you need to have:

1. Java JDK 17 (or later) installed - https://www.oracle.com/java/technologies/downloads/.
2. Installed IntelliJ, Eclipse, VSCode or other preferred IDE.
3. Docker installed.

Tools used

1. EventStoreDB - Event Store
2. PostgreSQL - Read Models
3. Spring Boot - Web Application framework

Event Versioning

Shows how to handle basic event schema versioning scenarios using event and stream transformations (e.g. upcasting):

• Simple mapping
  • New not required property
  • New required property
  • Renamed property
• Upcasting
  • Changed Structure
  • New required property
• Downcasters
• Events Transformations
• Stream Transformation
• Summary
• 📝 Simple patterns for events schema versioning

Uniqueness

Shows how to handle unique constraint checks in an event-sources system. Explains various techniques, like:

• talking to business,
• stream id design,
• reservation pattern.

Read more in How to ensure uniqueness in Event Sourcing.

Distributed Processes

Shows how to handle distributed processes in Event Sourcing in practice. Explains various use cases, like:

• batch processing,
• saga vs process managers,
• distributed processes in the single module and across boundaries,
• internal vs external events,
• compensating failures,
• implementation of command and event bus in EventStoreDB.

Read more in How to ensure uniqueness in Event Sourcing.

Articles

Read also more on the Event Sourcing and CQRS topics in my blog posts:

• 📝 What's the difference between a command and an event?
• 📝 Event Streaming is not Event Sourcing!
• 📝 Events should be as small as possible, right?
• 📝 How to get the current entity state from events?
• 📝 How to ensure uniqueness in Event Sourcing
• 📝 Anti-patterns in event modelling - Property Sourcing
• 📝 Anti-patterns in event modelling - State Obsession
• 📝 Why a bank account is not the best example of Event Sourcing?
• 📝 When not to use Event Sourcing?
• 📝 CQRS facts and myths explained
• 📝 How to slice the codebase effectively?
• 📝 Generic does not mean Simple
• 📝 Can command return a value?
• 📝 Twelve things I learned about Java during my last code review
• 📝 How to use ETag header for optimistic concurrency
• 📝 Dealing with Eventual Consistency and Idempotency in MongoDB projections
• 📝 Long-polling, how to make our async API synchronous
• 📝 A simple trick for idempotency handling in the Elastic Search read model
• 📝 How to (not) do the events versioning?
• 📝 Simple patterns for events schema versioning
• 📝 How to create projections of events for nested object structures?
• 📝 How to scale projections in the event-driven systems?
• 📝 Immutable Value Objects are simpler and more useful than you think!
• 📝 How using events helps in a teams' autonomy
• 📝 What texting your Ex has to do with Event-Driven Design?
• 📝 What if I told you that Relational Databases are in fact Event Stores?
• 📝 Are Temporal Tables an alternative to Event Sourcing?
• 📝 Optimistic concurrency for pessimistic times
• 📝 Outbox, Inbox patterns and delivery guarantees explained
• 📝 Saga and Process Manager - distributed processes in practice

EventSourcing.JVM is Copyright © 2022 Oskar Dudycz and other contributors under the MIT license.