diff --git a/README.md b/README.md index 57e6d29..77fd4c6 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,14 @@ # Akka.Persistence.Redis +![Akka.NET logo](docs/images/AkkaNetLogo.Normal.png) + [![NuGet Version](http://img.shields.io/nuget/v/Akka.Persistence.Redis.svg?style=flat)](https://www.nuget.org/packages/Akka.Persistence.Redis) Akka Persistence Redis Plugin is a plugin for `Akka persistence` that provides several components: - - a journal store ; - - a snapshot store ; - - a journal query interface implementation. + - a journal store and + - a snapshot store. + + > NOTE: in Akka.Persistence.Redis v1.4.16 we removed [Akka.Persistence.Query](https://getakka.net/articles/persistence/persistence-query.html) support. Please read more about that decision and comment here: https://github.com/akkadotnet/Akka.Persistence.Redis/issues/126 This plugin stores data in a [redis](https://redis.io) database and based on [Stackexchange.Redis](https://github.com/StackExchange/StackExchange.Redis) library. @@ -19,153 +22,56 @@ From `.NET CLI` dotnet add package Akka.Persistence.Redis ``` -## Journal plugin +## Journal To activate the journal plugin, add the following line to your HOCON config: ``` akka.persistence.journal.plugin = "akka.persistence.journal.redis" ``` This will run the journal with its default settings. The default settings can be changed with the configuration properties defined in your HOCON config: -### Configuration -- `configuration-string` - connection string, as described here: https://github.com/StackExchange/StackExchange.Redis/blob/master/docs/Configuration.md#basic-configuration-strings -- `key-prefix` - Redis journals key prefixes. Leave it for default or change it to appropriate value. WARNING: don't change it on production instances. - -## Snapshot config -To activate the snapshot plugin, add the following line to your HOCON config: ``` -akka.persistence.snapshot-store.plugin = "akka.persistence.snapshot-store.redis" +akka.persistence.journal.redis { + # qualified type name of the Redis persistence journal actor + class = "Akka.Persistence.Redis.Journal.RedisJournal, Akka.Persistence.Redis" + + # connection string, as described here: https://github.com/StackExchange/StackExchange.Redis/blob/master/Docs/Configuration.md#basic-configuration-strings + configuration-string = "" + + # Redis journals key prefixes. Leave it for default or change it to appropriate value. WARNING: don't change it on production instances. + key-prefix = "" +} ``` -This will run the snapshot-store with its default settings. The default settings can be changed with the configuration properties defined in your HOCON config: ### Configuration - `configuration-string` - connection string, as described here: https://github.com/StackExchange/StackExchange.Redis/blob/master/docs/Configuration.md#basic-configuration-strings -- `key-prefix` - Redis journals key prefixes. Leave it for default or change it to appropriate value. WARNING: don't change it on production instances. +- `key-prefix` - Redis journals key prefixes. Leave it for default or change it to customized value. WARNING: don't change this value after you've started persisting data in production. -## Persistence Query - -The plugin supports the following queries: - -### PersistenceIdsQuery and CurrentPersistenceIdsQuery - -`PersistenceIds` and `CurrentPersistenceIds` are used for retrieving all persistenceIds of all persistent actors. -```C# -var readJournal = Sys.ReadJournalFor(RedisReadJournal.Identifier); - -Source willNotCompleteTheStream = readJournal.PersistenceIds(); -Source willCompleteTheStream = readJournal.CurrentPersistenceIds(); +## Snapshot Store +To activate the snapshot plugin, add the following line to your HOCON config: ``` -The returned event stream is unordered and you can expect different order for multiple executions of the query. - -When using the `PersistenceIds` query, the stream is not completed when it reaches the end of the currently used `persistenceIds`, but it continues to push new `persistenceIds` when new persistent actors are created. - -When using the `CurrentPersistenceIds` query, the stream is completed when the end of the current list of `persistenceIds` is reached, thus it is not a live query. - -The stream is completed with failure if there is a failure in executing the query in the backend journal. - -### EventsByPersistenceIdQuery and CurrentEventsByPersistenceIdQuery - -`EventsByPersistenceId` and `CurrentEventsByPersistenceId` is used for retrieving events for a specific `PersistentActor` identified by `persistenceId`. -```C# -import akka.actor.ActorSystem -import akka.stream.{Materializer, ActorMaterializer} -import akka.stream.scaladsl.Source -import akka.persistence.query.{ PersistenceQuery, EventEnvelope } -import akka.persistence.jdbc.query.scaladsl.JdbcReadJournal - -implicit val system: ActorSystem = ActorSystem() -implicit val mat: Materializer = ActorMaterializer()(system) -val readJournal: JdbcReadJournal = PersistenceQuery(system).readJournalFor[JdbcReadJournal](JdbcReadJournal.Identifier) - -val willNotCompleteTheStream: Source[EventEnvelope, NotUsed] = readJournal.eventsByPersistenceId("some-persistence-id", 0L, Long.MaxValue) - -val willCompleteTheStream: Source[EventEnvelope, NotUsed] = readJournal.currentEventsByPersistenceId("some-persistence-id", 0L, Long.MaxValue) - - -var readJournal = Sys.ReadJournalFor(RedisReadJournal.Identifier); - -Source willNotCompleteTheStream = queries.EventsByPersistenceId("some-persistence-id", 0L, long.MaxValue); -Source willCompleteTheStream = queries.CurrentEventsByPersistenceId("some-persistence-id", 0L, long.MaxValue); +akka.persistence.snapshot-store.plugin = "akka.persistence.snapshot-store.redis" ``` -You can retrieve a subset of all events by specifying `fromSequenceNr` and `toSequenceNr` or use `0L` and `long.MaxValue` respectively to retrieve all events. Note that the corresponding sequence number of each event is provided in the `EventEnvelope`, which makes it possible to resume the stream at a later point from a given sequence number. - -The returned event stream is ordered by sequence number, i.e. the same order as the `PersistentActor` persisted the events. The same prefix of stream elements (in same order) are returned for multiple executions of the query, except for when events have been deleted. - -The stream is completed with failure if there is a failure in executing the query in the backend journal. - -### EventsByTag and CurrentEventsByTag +This will run the snapshot-store with its default settings. The default settings can be changed with the configuration properties defined in your HOCON config: -`EventsByTag` and `CurrentEventsByTag` are used for retrieving events that were marked with a given tag, e.g. all domain events of an Aggregate Root type. -```C# -var readJournal = Sys.ReadJournalFor(RedisReadJournal.Identifier); -Source willNotCompleteTheStream = queries.EventsByTag("apple", 0L); -Source willCompleteTheStream = queries.CurrentEventsByTag("apple", 0L); ``` +akka.persistence.snapshot-store.redis { + # qualified type name of the Redis persistence journal actor + class = "Akka.Persistence.Redis.Journal.RedisJournal, Akka.Persistence.Redis" -### Tagging Events -To tag events you'll need to create an Event Adapter that will wrap the event in a akka.persistence.journal.Tagged class with the given tags. The Tagged class will instruct akka-persistence-jdbc to tag the event with the given set of tags. + # connection string, as described here: https://github.com/StackExchange/StackExchange.Redis/blob/master/Docs/Configuration.md#basic-configuration-strings + configuration-string = "" -The persistence plugin will not store the Tagged class in the journal. It will strip the tags and payload from the Tagged class, and use the class only as an instruction to tag the event with the given tags and store the payload in the message field of the journal table. -``` -public class ColorTagger : IWriteEventAdapter -{ - public string Manifest(object evt) => string.Empty; - internal Tagged WithTag(object evt, string tag) => new Tagged(evt, ImmutableHashSet.Create(tag)); - - public object ToJournal(object evt) - { - switch (evt) - { - case string s when s.Contains("green"): - return WithTag(evt, "green"); - case string s when s.Contains("black"): - return WithTag(evt, "black"); - case string s when s.Contains("blue"): - return WithTag(evt, "blue"); - default: - return evt; - } - } -} + # Redis journals key prefixes. Leave it for default or change it to appropriate value. WARNING: don't change it on production instances. + key-prefix = "" +} ``` -The EventAdapter must be registered by adding the following to the root of `application.conf` Please see the demo-akka-persistence-jdbc project for more information. -``` -akka.persistence.journal.redis { - event-adapters { - color-tagger = "Akka.Persistence.Redis.Tests.Query.ColorTagger, Akka.Persistence.Redis.Tests" - } - event-adapter-bindings = { - "System.String" = color-tagger - } -} -``` -You can retrieve a subset of all events by specifying `offset`, or use `0L` to retrieve all events with a given tag. The `offset` corresponds to an ordered sequence number for the specific tag. Note that the corresponding offset of each event is provided in the `EventEnvelope`, which makes it possible to resume the stream at a later point from a given `offset`. - -In addition to the `offset` the `EventEnvelope` also provides `persistenceId` and `sequenceNr` for each event. The `sequenceNr` is the sequence number for the persistent actor with the `persistenceId` that persisted the event. The `persistenceId` + `sequenceNr` is an unique identifier for the event. - -The returned event stream contains only events that correspond to the given tag, and is ordered by the creation time of the events. The same stream elements (in same order) are returned for multiple executions of the same query. Deleted events are not deleted from the tagged event stream. -## Serialization -Akka Persistence provided serializers wrap the user payload in an envelope containing all persistence-relevant information. Redis Journal uses provided Protobuf serializers for the wrapper types (e.g. `IPersistentRepresentation`), then the payload will be serialized using the user configured serializer. By default, the payload will be serialized using JSON.NET serializer. This is fine for testing and initial phases of your development (while you’re still figuring out things and the data will not need to stay persisted forever). However, once you move to production you should really pick a different serializer for your payloads. - -Serialization of snapshots and payloads of Persistent messages is configurable with Akka’s Serialization infrastructure. For example, if an application wants to serialize - -- payloads of type `MyPayload` with a custom `MyPayloadSerializer` and -- snapshots of type `MySnapshot` with a custom `MySnapshotSerializer` -it must add -``` -akka.actor { - serializers { - redis = "Akka.Serialization.YourOwnSerializer, YourOwnSerializer" - } - serialization-bindings { - "Akka.Persistence.Redis.Journal.JournalEntry, Akka.Persistence.Redis" = redis - "Akka.Persistence.Redis.Snapshot.SnapshotEntry, Akka.Persistence.Redis" = redis - } -} -``` +### Configuration +- `configuration-string` - connection string, as described here: https://github.com/StackExchange/StackExchange.Redis/blob/master/docs/Configuration.md#basic-configuration-strings +- `key-prefix` - Redis journals key prefixes. Leave it for default or change it to appropriate value. WARNING: don't change it on production instances. -## Securing your Redis server +## Security and Access Control You can secure the Redis server Akka.Persistence.Redis connects to by leveraging Redis ACL and requiring users to use AUTH to connect to the Redis server. 1. Redis ACL @@ -184,7 +90,29 @@ You can secure the Redis server Akka.Persistence.Redis connects to by leveraging To connect to ACL enabled Redis server, you will need to set the user and password option in the [connection string](https://stackexchange.github.io/StackExchange.Redis/Configuration#basic-configuration-strings): "myServer.net:6380,user=\,password=\" -### Minimum command set +All of these features are supported via `StackExchange.Redis`, which Akka.Persistence.Redis uses internally, and you only need to customize your `akka.persistence.journal.redis.configuration-string` and `akka.persistence.snapshot-store.redis.configuration-string` values to customize it. + +### Enabling TLS +For instance, if you want to enable TLS on your Akka.Persistence.Redis instance: + +``` +akka.persistence.journal.redis.configuration-string = "contoso5.redis.cache.windows.net,ssl=true,password=..." +``` + +Or if you need to connect to multiple redis instances in a cluster: + +``` +akka.persistence.journal.redis.configuration-string = "contoso5.redis.cache.windows.net, contoso4.redis.cache.windows.net,ssl=true,password=..." +``` + +### Enabling ACL +To connect to your redis instance with access control (ACL) support for Akka.Persistence.Redis, all you need to do is specify the user name and password in your connection string and this will restrict the `StackExchange.Redis` client used internally by Akka.Persistence.Redis to whatever permissions you specified in your cluster: + +``` +akka.persistence.journal.redis.configuration-string = "contoso5.redis.cache.windows.net, contoso4.redis.cache.windows.net,user=akka-persistence,password=..." +``` + +#### Minimum command set These are the minimum Redis commands that are needed by Akka.Persistence.Redis to work properly. | Redis Command | StackExchange.Redis Command | @@ -217,5 +145,29 @@ These are the minimum Redis commands that are needed by Akka.Persistence.Redis t | PUNSUBSCRIBE | | | PUBLISH | Pub/Sub Publish | -## Maintainer -- [alexvaluyskiy](https://github.com/alexvaluyskiy) + +## Serialization +Akka Persistence provided serializers wrap the user payload in an envelope containing all persistence-relevant information. Redis Journal uses provided Protobuf serializers for the wrapper types (e.g. `IPersistentRepresentation`), then the payload will be serialized using the user configured serializer. + +The payload will be serialized [using Akka.NET's serialization bindings for your events and snapshot objects](https://getakka.net/articles/networking/serialization.html). By default, all `object`s that do not have a specified serializer will use Newtonsoft.Json polymorphic serialization (your CLR types <--> JSON.) + +This is fine for testing and initial phases of your development (while you’re still figuring out things and the data will not need to stay persisted forever). However, once you move to production you _should really pick a different serializer for your payloads_. + +We highly recommend creating schema-based serialization definitions using MsgPack, Google.Protobuf, or something similar and configuring serialization bindings for those in your configuration: https://getakka.net/articles/networking/serialization.html#usage + +Serialization of snapshots and payloads of Persistent messages is configurable with Akka’s Serialization infrastructure. For example, if an application wants to serialize + +- payloads of type `MyPayload` with a custom `MyPayloadSerializer` and +- snapshots of type `MySnapshot` with a custom `MySnapshotSerializer` +it must add +``` +akka.actor { + serializers { + redis = "Akka.Serialization.YourOwnSerializer, YourOwnSerializer" + } + serialization-bindings { + "Akka.Persistence.Redis.Journal.JournalEntry, Akka.Persistence.Redis" = redis + "Akka.Persistence.Redis.Snapshot.SnapshotEntry, Akka.Persistence.Redis" = redis + } +} +``` \ No newline at end of file diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 3f79b67..8b3373b 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,28 +1,40 @@ -#### 1.4.4 April 11th 2020 #### -- Bump Akka to version 1.4.4 -- Update build system to use Docker.DotNet - -#### 1.0.0-beta2 March 2nd 2020 #### -- Update Akka to version 1.4.1-RC1 -- Update build system - -#### 1.0.0-beta1 Sep 10 2017 #### -- Support for .NET Standard 1.6 -- Support for Persistence Query -- Use Google.Protobuf serialization both for journal and snapshots -- Updated Akka.Persistence to 1.3.1 -- StackExchange.Redis to 1.2.6 - -#### 0.2.5 Oct 16 2016 #### -- Updated Akka.Persistence to 1.1.2 -- Updated Json.Net to 9.0.1 -- StackExchange.Redis to 1.1.608 - -#### 0.2.0 Aug 12 2016 #### -- custom serializer for the events and snapshots -- use intermediate types JournalEntry and SnapshotEntry instead of default persistence types -- fixed sync call inside WriteMessagesAsync -- small optimizations and code refactoring - -#### 0.1.0 Jul 19 2016 #### -- First version of the package +#### 1.4.16 February 6th 2021 #### +This is a major update to the Akka.Persistence.Redis plugin. + +**Enabled Redis Cluster Support** +Akka.Persistence.Redis will now automatically detect whether or not you are running in clustered mode via your Redis connection string and will distribute journal entries and snapshots accordingly. + +All journal entries and all snapshots for a single entity will all reside inside the same Redis host cost - [using Redis' consistent hash distribution tagging](https://redis.io/topics/cluster-tutorial) scheme. + +**Significant Performance Improvements** +Akka.Persistence.Redis' write throughput was improved significantly in Akka.Persistence.Redis v1.4.16: + +| Test | Akka.Persistence.Redis v1.4.4 (msg/s) | current PR (msg/s) | +|-----------------|---------------------------------------|--------------------| +| Persist | 782 | 772 | +| PersistAll | 15019 | 20275 | +| PersistAsync | 9496 | 13131 | +| PersistAllAsync | 32765 | 44776 | +| PersistGroup10 | 611 | 6523 | +| PersistGroup100 | 8878 | 12533 | +| PersistGroup200 | 9598 | 12214 | +| PersistGroup25 | 9209 | 10819 | +| PersistGroup400 | 9209 | 11824 | +| PersistGroup50 | 9506 | 9704 | +| Recovering | 17374 | 20119 | +| Recovering8 | 36915 | 37290 | +| RecoveringFour | 22432 | 20884 | +| RecoveringTwo | 22209 | 21222 | + +These numbers were generated running a single Redis instance inside a Docker container on Docker for Windows - real-world values generated in cloud environments will likely be much higher. + +**Removed Akka.Persistence.Query Support** +In order to achieve support for clustering and improved write performance, we made the descision to drop Akka.Persistence.Query support from Akka.Persistence.Redis at this time - if you wish to learn more about our decision-making process or if you are affected by this change, please comment on this thread here: https://github.com/akkadotnet/Akka.Persistence.Redis/issues/126 + +**Other Changes** + +- Bump [Akka.NET to version 1.4.16](https://github.com/akkadotnet/akka.net/releases/tag/1.4.16) +- Modernized Akka.NET Serialization calls +- [Added benchmarks](https://github.com/akkadotnet/Akka.Persistence.Redis/pull/118) +- Upgraded to [StackExchange.Redis 2.2.11](https://github.com/StackExchange/StackExchange.Redis/blob/main/docs/ReleaseNotes.md) +- Improved documentation \ No newline at end of file diff --git a/docs/images/AkkaNetLogo.Normal.png b/docs/images/AkkaNetLogo.Normal.png new file mode 100644 index 0000000..028060f Binary files /dev/null and b/docs/images/AkkaNetLogo.Normal.png differ diff --git a/src/common.props b/src/common.props index 0f7fa80..4bfc9fe 100644 --- a/src/common.props +++ b/src/common.props @@ -1,16 +1,43 @@ - Copyright © 2013-2020 Akka.NET Project + Copyright © 2013-2021 Akka.NET Project Akka.NET - 1.4.4 + 1.4.16 akka;actors;actor model;Akka;concurrency;persistence;eventsource;redis http://getakka.net/images/AkkaNetLogo.Normal.png - https://github.com/AkkaNetContrib/Akka.Persistence.Redis - https://github.com/AkkaNetContrib/Akka.Persistence.Redis/blob/dev/LICENSE - - - Bump Akka to version 1.4.4 - - Update build system to use Docker.DotNet - + https://github.com/akkadotnet/Akka.Persistence.Redis + https://github.com/akkadotnet/Akka.Persistence.Redis/blob/dev/LICENSE + This is a major update to the Akka.Persistence.Redis plugin. +Enabled Redis Cluster Support** +Akka.Persistence.Redis will now automatically detect whether or not you are running in clustered mode via your Redis connection string and will distribute journal entries and snapshots accordingly. +All journal entries and all snapshots for a single entity will all reside inside the same Redis host cost - [using Redis' consistent hash distribution tagging](https://redis.io/topics/cluster-tutorial) scheme. +Significant Performance Improvements** +Akka.Persistence.Redis' write throughput was improved significantly in Akka.Persistence.Redis v1.4.16: +| Test | Akka.Persistence.Redis v1.4.4 (msg/s) | current PR (msg/s) | +|-----------------|---------------------------------------|--------------------| +| Persist | 782 | 772 | +| PersistAll | 15019 | 20275 | +| PersistAsync | 9496 | 13131 | +| PersistAllAsync | 32765 | 44776 | +| PersistGroup10 | 611 | 6523 | +| PersistGroup100 | 8878 | 12533 | +| PersistGroup200 | 9598 | 12214 | +| PersistGroup25 | 9209 | 10819 | +| PersistGroup400 | 9209 | 11824 | +| PersistGroup50 | 9506 | 9704 | +| Recovering | 17374 | 20119 | +| Recovering8 | 36915 | 37290 | +| RecoveringFour | 22432 | 20884 | +| RecoveringTwo | 22209 | 21222 | +These numbers were generated running a single Redis instance inside a Docker container on Docker for Windows - real-world values generated in cloud environments will likely be much higher. +Removed Akka.Persistence.Query Support** +In order to achieve support for clustering and improved write performance, we made the descision to drop Akka.Persistence.Query support from Akka.Persistence.Redis at this time - if you wish to learn more about our decision-making process or if you are affected by this change, please comment on this thread here: https://github.com/akkadotnet/Akka.Persistence.Redis/issues/126 +Other Changes** +- Bump [Akka.NET to version 1.4.16](https://github.com/akkadotnet/akka.net/releases/tag/1.4.16) +- Modernized Akka.NET Serialization calls +- [Added benchmarks](https://github.com/akkadotnet/Akka.Persistence.Redis/pull/118) +- Upgraded to [StackExchange.Redis 2.2.11](https://github.com/StackExchange/StackExchange.Redis/blob/main/docs/ReleaseNotes.md) +- Improved documentation true Akka.NET Persistence journal and snapshot store backed by Redis. $(NoWarn);CS1591