Skip to content

Commit

Permalink
chore(plugin): add new user config generator
Browse files Browse the repository at this point in the history
  • Loading branch information
byashimov committed Sep 13, 2023
1 parent 26664b4 commit bda0b12
Show file tree
Hide file tree
Showing 38 changed files with 5,110 additions and 155 deletions.
226 changes: 113 additions & 113 deletions docs/data-sources/service_integration.md

Large diffs are not rendered by default.

53 changes: 26 additions & 27 deletions docs/resources/service_integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,17 +33,17 @@ resource "aiven_service_integration" "my_integration_metrics" {

### Optional

- `clickhouse_kafka_user_config` (Block List, Max: 1) ClickhouseKafka user configurable settings (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config))
- `clickhouse_postgresql_user_config` (Block List, Max: 1) ClickhousePostgresql user configurable settings (see [below for nested schema](#nestedblock--clickhouse_postgresql_user_config))
- `datadog_user_config` (Block List, Max: 1) Datadog user configurable settings (see [below for nested schema](#nestedblock--datadog_user_config))
- `clickhouse_kafka_user_config` (Block List) Integration user config (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config))
- `clickhouse_postgresql_user_config` (Block List) Integration user config (see [below for nested schema](#nestedblock--clickhouse_postgresql_user_config))
- `datadog_user_config` (Block List) (see [below for nested schema](#nestedblock--datadog_user_config))
- `destination_endpoint_id` (String) Destination endpoint for the integration (if any)
- `destination_service_name` (String) Destination service for the integration (if any)
- `external_aws_cloudwatch_metrics_user_config` (Block List, Max: 1) ExternalAwsCloudwatchMetrics user configurable settings (see [below for nested schema](#nestedblock--external_aws_cloudwatch_metrics_user_config))
- `kafka_connect_user_config` (Block List, Max: 1) KafkaConnect user configurable settings (see [below for nested schema](#nestedblock--kafka_connect_user_config))
- `kafka_logs_user_config` (Block List, Max: 1) KafkaLogs user configurable settings (see [below for nested schema](#nestedblock--kafka_logs_user_config))
- `kafka_mirrormaker_user_config` (Block List, Max: 1) KafkaMirrormaker user configurable settings (see [below for nested schema](#nestedblock--kafka_mirrormaker_user_config))
- `logs_user_config` (Block List, Max: 1) Logs user configurable settings (see [below for nested schema](#nestedblock--logs_user_config))
- `metrics_user_config` (Block List, Max: 1) Metrics user configurable settings (see [below for nested schema](#nestedblock--metrics_user_config))
- `external_aws_cloudwatch_metrics_user_config` (Block List) External AWS CloudWatch Metrics integration user config (see [below for nested schema](#nestedblock--external_aws_cloudwatch_metrics_user_config))
- `kafka_connect_user_config` (Block List) Integration user config (see [below for nested schema](#nestedblock--kafka_connect_user_config))
- `kafka_logs_user_config` (Block List) (see [below for nested schema](#nestedblock--kafka_logs_user_config))
- `kafka_mirrormaker_user_config` (Block List) Integration user config (see [below for nested schema](#nestedblock--kafka_mirrormaker_user_config))
- `logs_user_config` (Block List) (see [below for nested schema](#nestedblock--logs_user_config))
- `metrics_user_config` (Block List) Integration user config (see [below for nested schema](#nestedblock--metrics_user_config))
- `source_endpoint_id` (String) Source endpoint for the integration (if any)
- `source_service_name` (String) Source service for the integration (if any)
- `timeouts` (Block, Optional) (see [below for nested schema](#nestedblock--timeouts))
Expand All @@ -58,7 +58,7 @@ resource "aiven_service_integration" "my_integration_metrics" {

Optional:

- `tables` (Block List, Max: 100) Tables to create. (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables))
- `tables` (Block List) Tables to create (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables))

<a id="nestedblock--clickhouse_kafka_user_config--tables"></a>
### Nested Schema for `clickhouse_kafka_user_config.tables`
Expand All @@ -72,15 +72,15 @@ Required:
Optional:

- `auto_offset_reset` (String) Action to take when there is no initial offset in offset store or the desired offset is out of range. The default value is `earliest`.
- `columns` (Block List, Max: 100) Table columns. (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables--columns))
- `columns` (Block List) Table columns (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables--columns))
- `date_time_input_format` (String) Method to read DateTime from text input formats. The default value is `basic`.
- `handle_error_mode` (String) How to handle errors for Kafka engine. The default value is `default`.
- `max_block_size` (Number) Number of row collected by poll(s) for flushing data from Kafka. The default value is `0`.
- `max_rows_per_message` (Number) The maximum number of rows produced in one kafka message for row-based formats. The default value is `1`.
- `num_consumers` (Number) The number of consumers per table per replica. The default value is `1`.
- `poll_max_batch_size` (Number) Maximum amount of messages to be polled in a single Kafka poll. The default value is `0`.
- `skip_broken_messages` (Number) Skip at least this number of broken messages from Kafka topic per block. The default value is `0`.
- `topics` (Block List, Max: 100) Kafka topics. (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables--topics))
- `topics` (Block List) Kafka topics (see [below for nested schema](#nestedblock--clickhouse_kafka_user_config--tables--topics))

<a id="nestedblock--clickhouse_kafka_user_config--tables--columns"></a>
### Nested Schema for `clickhouse_kafka_user_config.tables.columns`
Expand All @@ -106,7 +106,7 @@ Required:

Optional:

- `databases` (Block List, Max: 10) Databases to expose. (see [below for nested schema](#nestedblock--clickhouse_postgresql_user_config--databases))
- `databases` (Block List) Databases to expose (see [below for nested schema](#nestedblock--clickhouse_postgresql_user_config--databases))

<a id="nestedblock--clickhouse_postgresql_user_config--databases"></a>
### Nested Schema for `clickhouse_postgresql_user_config.databases`
Expand All @@ -124,15 +124,15 @@ Optional:
Optional:

- `datadog_dbm_enabled` (Boolean) Enable Datadog Database Monitoring.
- `datadog_tags` (Block List, Max: 32) Custom tags provided by user. (see [below for nested schema](#nestedblock--datadog_user_config--datadog_tags))
- `datadog_tags` (Block List) Custom tags provided by user (see [below for nested schema](#nestedblock--datadog_user_config--datadog_tags))
- `exclude_consumer_groups` (List of String) List of custom metrics.
- `exclude_topics` (List of String) List of topics to exclude.
- `include_consumer_groups` (List of String) List of custom metrics.
- `include_topics` (List of String) List of topics to include.
- `kafka_custom_metrics` (List of String) List of custom metrics.
- `max_jmx_metrics` (Number) Maximum number of JMX metrics to send.
- `opensearch` (Block List, Max: 1) Datadog Opensearch Options. (see [below for nested schema](#nestedblock--datadog_user_config--opensearch))
- `redis` (Block List, Max: 1) Datadog Redis Options. (see [below for nested schema](#nestedblock--datadog_user_config--redis))
- `opensearch` (Block List) Datadog Opensearch Options (see [below for nested schema](#nestedblock--datadog_user_config--opensearch))
- `redis` (Block List) Datadog Redis Options (see [below for nested schema](#nestedblock--datadog_user_config--redis))

<a id="nestedblock--datadog_user_config--datadog_tags"></a>
### Nested Schema for `datadog_user_config.datadog_tags`
Expand Down Expand Up @@ -170,8 +170,8 @@ Optional:

Optional:

- `dropped_metrics` (Block List, Max: 1024) Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics). (see [below for nested schema](#nestedblock--external_aws_cloudwatch_metrics_user_config--dropped_metrics))
- `extra_metrics` (Block List, Max: 1024) Metrics to allow through to AWS CloudWatch (in addition to default metrics). (see [below for nested schema](#nestedblock--external_aws_cloudwatch_metrics_user_config--extra_metrics))
- `dropped_metrics` (Block List) Metrics to not send to AWS CloudWatch (takes precedence over extra_metrics) (see [below for nested schema](#nestedblock--external_aws_cloudwatch_metrics_user_config--dropped_metrics))
- `extra_metrics` (Block List) Metrics to allow through to AWS CloudWatch (in addition to default metrics) (see [below for nested schema](#nestedblock--external_aws_cloudwatch_metrics_user_config--extra_metrics))

<a id="nestedblock--external_aws_cloudwatch_metrics_user_config--dropped_metrics"></a>
### Nested Schema for `external_aws_cloudwatch_metrics_user_config.dropped_metrics`
Expand All @@ -197,7 +197,7 @@ Required:

Optional:

- `kafka_connect` (Block List, Max: 1) Kafka Connect service configuration values. (see [below for nested schema](#nestedblock--kafka_connect_user_config--kafka_connect))
- `kafka_connect` (Block List) Kafka Connect service configuration values (see [below for nested schema](#nestedblock--kafka_connect_user_config--kafka_connect))

<a id="nestedblock--kafka_connect_user_config--kafka_connect"></a>
### Nested Schema for `kafka_connect_user_config.kafka_connect`
Expand Down Expand Up @@ -229,7 +229,7 @@ Optional:
Optional:

- `cluster_alias` (String) The alias under which the Kafka cluster is known to MirrorMaker. Can contain the following symbols: ASCII alphanumerics, '.', '_', and '-'.
- `kafka_mirrormaker` (Block List, Max: 1) Kafka MirrorMaker configuration values. (see [below for nested schema](#nestedblock--kafka_mirrormaker_user_config--kafka_mirrormaker))
- `kafka_mirrormaker` (Block List) Kafka MirrorMaker configuration values (see [below for nested schema](#nestedblock--kafka_mirrormaker_user_config--kafka_mirrormaker))

<a id="nestedblock--kafka_mirrormaker_user_config--kafka_mirrormaker"></a>
### Nested Schema for `kafka_mirrormaker_user_config.kafka_mirrormaker`
Expand Down Expand Up @@ -263,15 +263,15 @@ Optional:
- `database` (String) Name of the database where to store metric datapoints. Only affects PostgreSQL destinations. Defaults to 'metrics'. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.
- `retention_days` (Number) Number of days to keep old metrics. Only affects PostgreSQL destinations. Set to 0 for no automatic cleanup. Defaults to 30 days.
- `ro_username` (String) Name of a user that can be used to read metrics. This will be used for Grafana integration (if enabled) to prevent Grafana users from making undesired changes. Only affects PostgreSQL destinations. Defaults to 'metrics_reader'. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.
- `source_mysql` (Block List, Max: 1) Configuration options for metrics where source service is MySQL. (see [below for nested schema](#nestedblock--metrics_user_config--source_mysql))
- `source_mysql` (Block List) Configuration options for metrics where source service is MySQL (see [below for nested schema](#nestedblock--metrics_user_config--source_mysql))
- `username` (String) Name of the user used to write metrics. Only affects PostgreSQL destinations. Defaults to 'metrics_writer'. Note that this must be the same for all metrics integrations that write data to the same PostgreSQL service.

<a id="nestedblock--metrics_user_config--source_mysql"></a>
### Nested Schema for `metrics_user_config.source_mysql`

Optional:

- `telegraf` (Block List, Max: 1) Configuration options for Telegraf MySQL input plugin. (see [below for nested schema](#nestedblock--metrics_user_config--source_mysql--telegraf))
- `telegraf` (Block List) Configuration options for Telegraf MySQL input plugin (see [below for nested schema](#nestedblock--metrics_user_config--source_mysql--telegraf))

<a id="nestedblock--metrics_user_config--source_mysql--telegraf"></a>
### Nested Schema for `metrics_user_config.source_mysql.telegraf`
Expand Down Expand Up @@ -301,11 +301,10 @@ Optional:

Optional:

- `create` (String)
- `default` (String)
- `delete` (String)
- `read` (String)
- `update` (String)
- `create` (String) A string that can be [parsed as a duration](https://pkg.go.dev/time#ParseDuration) consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours).
- `delete` (String) A string that can be [parsed as a duration](https://pkg.go.dev/time#ParseDuration) consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours). Setting a timeout for a Delete operation is only applicable if changes are saved into state before the destroy operation occurs.
- `read` (String) A string that can be [parsed as a duration](https://pkg.go.dev/time#ParseDuration) consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours). Read operations occur during any refresh or planning operation when refresh is enabled.
- `update` (String) A string that can be [parsed as a duration](https://pkg.go.dev/time#ParseDuration) consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours).
## Import
Import is supported using the following syntax:
```shell
Expand Down
2 changes: 2 additions & 0 deletions examples_tests/base.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ func (s *BaseTestSuite) TearDownSuite() {
}

// withDefaults adds default options for terraform test
//
//lint:ignore U1000 Ignore unused function. Used in child structs
func (s *BaseTestSuite) withDefaults(opts *terraform.Options) *terraform.Options {
// No need to use lock file for dev build
opts.Lock = false
Expand Down
7 changes: 6 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,10 @@ go 1.19

require (
github.com/aiven/aiven-go-client v1.36.0
github.com/avast/retry-go v3.0.0+incompatible
github.com/dave/jennifer v1.7.0
github.com/docker/go-units v0.5.0
github.com/emicklei/go-restful/v3 v3.9.0
github.com/ettle/strcase v0.1.1
github.com/google/go-cmp v0.5.9
github.com/gruntwork-io/terratest v0.43.13
Expand All @@ -15,9 +17,12 @@ require (
github.com/hashicorp/terraform-plugin-mux v0.11.2
github.com/hashicorp/terraform-plugin-sdk/v2 v2.28.0
github.com/kelseyhightower/envconfig v1.4.0
github.com/liip/sheriff v0.11.1
github.com/stoewer/go-strcase v1.3.0
github.com/stretchr/testify v1.8.4
golang.org/x/exp v0.0.0-20230809150735-7b3493d9a819
golang.org/x/sync v0.3.0
golang.org/x/tools v0.6.0
gopkg.in/yaml.v3 v3.0.1
)

Expand Down Expand Up @@ -70,7 +75,7 @@ require (
github.com/hashicorp/terraform-registry-address v0.2.1 // indirect
github.com/hashicorp/terraform-svchost v0.1.1 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
github.com/jinzhu/copier v0.0.0-20190924061706-b57f9002281a // indirect
github.com/jinzhu/copier v0.4.0
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/klauspost/compress v1.15.11 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
Expand Down
Loading

0 comments on commit bda0b12

Please sign in to comment.