Skip to content

Commit

Permalink
refactor(aiven_service_integration): new user configs (#1605)
Browse files Browse the repository at this point in the history
  • Loading branch information
byashimov authored Feb 22, 2024
1 parent a88d149 commit 5fb3fdf
Show file tree
Hide file tree
Showing 15 changed files with 845 additions and 138 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ nav_order: 1
<!-- TODO: uncomment when dragonfly is supported -->
<!-- - Dragonfly support -->

- Use new user config generator to generate service integration configs
- Fix `aiven_kafka_schema` version update

## [4.14.0] - 2024-02-20
Expand Down
26 changes: 13 additions & 13 deletions docs/resources/service_integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,29 +58,29 @@ resource "aiven_service_integration" "my_integration_metrics" {

Optional:

- `tables` (Block List, Max: 100) Tables to create. (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables))
- `tables` (Block List, Max: 100) Tables to create (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables))

<a id="nestedblock--clickhouse_kafka_user_config--tables"></a>
### Nested Schema for `clickhouse_kafka_user_config.tables`

Required:

- `columns` (Block List, Min: 1, Max: 100) Table columns (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables--columns))
- `data_format` (String) Message data format. The default value is `JSONEachRow`.
- `group_name` (String) Kafka consumers group. The default value is `clickhouse`.
- `name` (String) Name of the table.
- `topics` (Block List, Min: 1, Max: 100) Kafka topics (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables--topics))

Optional:

- `auto_offset_reset` (String) Action to take when there is no initial offset in offset store or the desired offset is out of range. The default value is `earliest`.
- `columns` (Block List, Max: 100) Table columns. (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables--columns))
- `date_time_input_format` (String) Method to read DateTime from text input formats. The default value is `basic`.
- `handle_error_mode` (String) How to handle errors for Kafka engine. The default value is `default`.
- `max_block_size` (Number) Number of row collected by poll(s) for flushing data from Kafka. The default value is `0`.
- `max_rows_per_message` (Number) The maximum number of rows produced in one kafka message for row-based formats. The default value is `1`.
- `num_consumers` (Number) The number of consumers per table per replica. The default value is `1`.
- `poll_max_batch_size` (Number) Maximum amount of messages to be polled in a single Kafka poll. The default value is `0`.
- `skip_broken_messages` (Number) Skip at least this number of broken messages from Kafka topic per block. The default value is `0`.
- `topics` (Block List, Max: 100) Kafka topics. (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables--topics))

<a id="nestedblock--clickhouse_kafka_user_config--tables--columns"></a>
### Nested Schema for `clickhouse_kafka_user_config.tables.columns`
Expand All @@ -106,7 +106,7 @@ Required:

Optional:

- `databases` (Block List, Max: 10) Databases to expose. (see [below for nested schema](#nestedblock--clickhouse_postgresql_user_config--databases))
- `databases` (Block List, Max: 10) Databases to expose (see [below for nested schema](#nestedblock--clickhouse_postgresql_user_config--databases))

<a id="nestedblock--clickhouse_postgresql_user_config--databases"></a>
### Nested Schema for `clickhouse_postgresql_user_config.databases`
Expand All @@ -124,15 +124,15 @@ Optional:
Optional:

- `datadog_dbm_enabled` (Boolean) Enable Datadog Database Monitoring.
- `datadog_tags` (Block List, Max: 32) Custom tags provided by user. (see [below for nested schema](#nestedblock--datadog_user_config--datadog_tags))
- `datadog_tags` (Block List, Max: 32) Custom tags provided by user (see [below for nested schema](#nestedblock--datadog_user_config--datadog_tags))
- `exclude_consumer_groups` (List of String) List of custom metrics.
- `exclude_topics` (List of String) List of topics to exclude.
- `include_consumer_groups` (List of String) List of custom metrics.
- `include_topics` (List of String) List of topics to include.
- `kafka_custom_metrics` (List of String) List of custom metrics.
- `max_jmx_metrics` (Number) Maximum number of JMX metrics to send.
- `opensearch` (Block List, Max: 1) Datadog Opensearch Options. (see [below for nested schema](#nestedblock--datadog_user_config--opensearch))
- `redis` (Block List, Max: 1) Datadog Redis Options. (see [below for nested schema](#nestedblock--datadog_user_config--redis))
- `opensearch` (Block List, Max: 1) Datadog Opensearch Options (see [below for nested schema](#nestedblock--datadog_user_config--opensearch))
- `redis` (Block List, Max: 1) Datadog Redis Options (see [below for nested schema](#nestedblock--datadog_user_config--redis))

<a id="nestedblock--datadog_user_config--datadog_tags"></a>
### Nested Schema for `datadog_user_config.datadog_tags`
Expand Down Expand Up @@ -170,8 +170,8 @@ Optional:

Optional:

- `dropped_metrics` (Block List, Max: 1024) Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics). (see [below for nested schema](#nestedblock--external_aws_cloudwatch_metrics_user_config--dropped_metrics))
- `extra_metrics` (Block List, Max: 1024) Metrics to allow through to AWS CloudWatch (in addition to default metrics). (see [below for nested schema](#nestedblock--external_aws_cloudwatch_metrics_user_config--extra_metrics))
- `dropped_metrics` (Block List, Max: 1024) Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics) (see [below for nested schema](#nestedblock--external_aws_cloudwatch_metrics_user_config--dropped_metrics))
- `extra_metrics` (Block List, Max: 1024) Metrics to allow through to AWS CloudWatch (in addition to default metrics) (see [below for nested schema](#nestedblock--external_aws_cloudwatch_metrics_user_config--extra_metrics))

<a id="nestedblock--external_aws_cloudwatch_metrics_user_config--dropped_metrics"></a>
### Nested Schema for `external_aws_cloudwatch_metrics_user_config.dropped_metrics`
Expand All @@ -197,7 +197,7 @@ Required:

Optional:

- `kafka_connect` (Block List, Max: 1) Kafka Connect service configuration values. (see [below for nested schema](#nestedblock--kafka_connect_user_config--kafka_connect))
- `kafka_connect` (Block List, Max: 1) Kafka Connect service configuration values (see [below for nested schema](#nestedblock--kafka_connect_user_config--kafka_connect))

<a id="nestedblock--kafka_connect_user_config--kafka_connect"></a>
### Nested Schema for `kafka_connect_user_config.kafka_connect`
Expand Down Expand Up @@ -229,7 +229,7 @@ Optional:
Optional:

- `cluster_alias` (String) The alias under which the Kafka cluster is known to MirrorMaker. Can contain the following symbols: ASCII alphanumerics, '.', '_', and '-'.
- `kafka_mirrormaker` (Block List, Max: 1) Kafka MirrorMaker configuration values. (see [below for nested schema](#nestedblock--kafka_mirrormaker_user_config--kafka_mirrormaker))
- `kafka_mirrormaker` (Block List, Max: 1) Kafka MirrorMaker configuration values (see [below for nested schema](#nestedblock--kafka_mirrormaker_user_config--kafka_mirrormaker))

<a id="nestedblock--kafka_mirrormaker_user_config--kafka_mirrormaker"></a>
### Nested Schema for `kafka_mirrormaker_user_config.kafka_mirrormaker`
Expand Down Expand Up @@ -263,15 +263,15 @@ Optional:
- `database` (String) Name of the database where to store metric datapoints. Only affects PostgreSQL destinations. Defaults to 'metrics'. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.
- `retention_days` (Number) Number of days to keep old metrics. Only affects PostgreSQL destinations. Set to 0 for no automatic cleanup. Defaults to 30 days.
- `ro_username` (String) Name of a user that can be used to read metrics. This will be used for Grafana integration (if enabled) to prevent Grafana users from making undesired changes. Only affects PostgreSQL destinations. Defaults to 'metrics_reader'. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.
- `source_mysql` (Block List, Max: 1) Configuration options for metrics where source service is MySQL. (see [below for nested schema](#nestedblock--metrics_user_config--source_mysql))
- `source_mysql` (Block List, Max: 1) Configuration options for metrics where source service is MySQL (see [below for nested schema](#nestedblock--metrics_user_config--source_mysql))
- `username` (String) Name of the user used to write metrics. Only affects PostgreSQL destinations. Defaults to 'metrics_writer'. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.

<a id="nestedblock--metrics_user_config--source_mysql"></a>
### Nested Schema for `metrics_user_config.source_mysql`

Optional:

- `telegraf` (Block List, Max: 1) Configuration options for Telegraf MySQL input plugin. (see [below for nested schema](#nestedblock--metrics_user_config--source_mysql--telegraf))
- `telegraf` (Block List, Max: 1) Configuration options for Telegraf MySQL input plugin (see [below for nested schema](#nestedblock--metrics_user_config--source_mysql--telegraf))

<a id="nestedblock--metrics_user_config--source_mysql--telegraf"></a>
### Nested Schema for `metrics_user_config.source_mysql.telegraf`
Expand Down
Loading

0 comments on commit 5fb3fdf

Please sign in to comment.