Skip to content

Commit

Permalink
feat: mirrormaker exactly-once delivery
Browse files Browse the repository at this point in the history
  • Loading branch information
nosahama committed Dec 2, 2024
1 parent c6e0e77 commit f2376d0
Show file tree
Hide file tree
Showing 5 changed files with 14 additions and 1 deletion.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ nav_order: 1
Defaults to aiven.
- Add `aiven_service_integration_endpoint` datasource field `datadog_user_config.extra_tags_prefix`: Extra tags prefix.
Defaults to aiven.
- Add `exactly_once_delivery_enabled` field to `aiven_mirrormaker_replication_flow`: Whether to enable exactly-once message delivery for Mirrormaker replication flow

## [4.29.0] - 2024-11-14

Expand Down
1 change: 1 addition & 0 deletions docs/data-sources/mirrormaker_replication_flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ data "aiven_mirrormaker_replication_flow" "f1" {
- `emit_backward_heartbeats_enabled` (Boolean) Whether to emit heartbeats to the direction opposite to the flow, i.e. to the source cluster. The default value is `false`.
- `emit_heartbeats_enabled` (Boolean) Whether to emit heartbeats to the target cluster. The default value is `false`.
- `enable` (Boolean) Enable of disable replication flows for a service.
- `exactly_once_delivery_enabled` (Boolean) Whether to enable exactly-once message delivery. We recommend you set this to `enabled` for new replications. The default value is `false`.
- `id` (String) The ID of this resource.
- `offset_syncs_topic_location` (String) Offset syncs topic location. The possible values are `source` and `target`.
- `replication_factor` (Number) Replication factor, `>= 1`.
Expand Down
1 change: 1 addition & 0 deletions docs/resources/mirrormaker_replication_flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ resource "aiven_mirrormaker_replication_flow" "f1" {
- `config_properties_exclude` (Set of String) List of topic configuration properties and/or regular expressions to not replicate. The properties that are not replicated by default are: `follower.replication.throttled.replicas`, `leader.replication.throttled.replicas`, `message.timestamp.difference.max.ms`, `message.timestamp.type`, `unclean.leader.election.enable`, and `min.insync.replicas`. Setting this overrides the defaults. For example, to enable replication for 'min.insync.replicas' and 'unclean.leader.election.enable' set this to: ["follower\\\\.replication\\\\.throttled\\\\.replicas", "leader\\\\.replication\\\\.throttled\\\\.replicas", "message\\\\.timestamp\\\\.difference\\\\.max\\\\.ms", "message\\\\.timestamp\\\\.type"]
- `emit_backward_heartbeats_enabled` (Boolean) Whether to emit heartbeats to the direction opposite to the flow, i.e. to the source cluster. The default value is `false`.
- `emit_heartbeats_enabled` (Boolean) Whether to emit heartbeats to the target cluster. The default value is `false`.
- `exactly_once_delivery_enabled` (Boolean) Whether to enable exactly-once message delivery. We recommend you set this to `enabled` for new replications. The default value is `false`.
- `replication_factor` (Number) Replication factor, `>= 1`.
- `sync_group_offsets_enabled` (Boolean) Sync consumer group offsets. The default value is `false`.
- `sync_group_offsets_interval_seconds` (Number) Frequency of consumer group offset sync. The default value is `1`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,14 @@ var aivenMirrorMakerReplicationFlowSchema = map[string]*schema.Schema{
ValidateFunc: validation.IntAtLeast(1),
Description: "Replication factor, `>= 1`.",
},
"exactly_once_delivery_enabled": {
Type: schema.TypeBool,
Optional: true,
Default: false,
Description: userconfig.Desc(
"Whether to enable exactly-once message delivery. We recommend you set this to `enabled` for new replications.",
).DefaultValue(false).Build(),
},
}

func ResourceMirrorMakerReplicationFlow() *schema.Resource {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,11 @@ func TestAccAivenMirrorMakerReplicationFlow_basic(t *testing.T) {
resource.TestCheckResourceAttr(resourceName, "topics_blacklist.1", ".*\\.replica"),
resource.TestCheckResourceAttr(resourceName, "topics_blacklist.2", "__.*"),
resource.TestCheckResourceAttr(resourceName, "config_properties_exclude.#", "0"),
resource.TestCheckResourceAttr(resourceName, "exactly_once_delivery_enabled", "true"),
),
},
{
Config: testAccMirrorMakerReplicationFlowResource(rName, `
Config: testAccMirrorMakerReplicationFlowResource(rName, `
config_properties_exclude = [
"follower\\.replication\\.throttled\\.replicas",
"leader\\.replication\\.throttled\\.replicas",
Expand Down Expand Up @@ -226,6 +227,7 @@ resource "aiven_mirrormaker_replication_flow" "foo" {
emit_heartbeats_enabled = true
emit_backward_heartbeats_enabled = true
offset_syncs_topic_location = "source"
exactly_once_delivery_enabled = true
topics = [
".*",
Expand Down

0 comments on commit f2376d0

Please sign in to comment.