From 1d66504a57d89c0f2e4cba1c9e42d462edf08582 Mon Sep 17 00:00:00 2001 From: Stacey Salamon Date: Fri, 1 Nov 2024 12:23:04 +0100 Subject: [PATCH] docs: autoscaler and integration docs Adds information about disk autosclaer to integration and endpoint docs. Improves docs for service integration and endpoint resources and data source. --- .../service_integration_endpoint.md | 18 +++---- docs/resources/service_integration.md | 32 +++++++++++- .../resources/service_integration_endpoint.md | 49 ++++++++++++++++--- .../data-source.tf | 6 +-- .../aiven_service_integration/resource.tf | 22 +++++++++ .../import.sh | 1 + .../resource.tf | 20 ++++++++ .../service_integration_endpoint.go | 15 ++++-- ...ervice_integration_endpoint_data_source.go | 2 +- .../resources/service_integration.md.tmpl | 10 +++- 10 files changed, 149 insertions(+), 26 deletions(-) create mode 100644 examples/resources/aiven_service_integration_endpoint/import.sh create mode 100644 examples/resources/aiven_service_integration_endpoint/resource.tf diff --git a/docs/data-sources/service_integration_endpoint.md b/docs/data-sources/service_integration_endpoint.md index e7259e931..abe526f3d 100644 --- a/docs/data-sources/service_integration_endpoint.md +++ b/docs/data-sources/service_integration_endpoint.md @@ -3,19 +3,19 @@ page_title: "aiven_service_integration_endpoint Data Source - terraform-provider-aiven" subcategory: "" description: |- - The Service Integration Endpoint data source provides information about the existing Aiven Service Integration Endpoint. + Gets information about an integration endpoint. --- # aiven_service_integration_endpoint (Data Source) -The Service Integration Endpoint data source provides information about the existing Aiven Service Integration Endpoint. +Gets information about an integration endpoint. ## Example Usage ```terraform -data "aiven_service_integration_endpoint" "myendpoint" { - project = aiven_project.myproject.project - endpoint_name = "" +data "aiven_service_integration_endpoint" "example_datadog_endpoint" { + project = aiven_project.example_project.project + endpoint_name = "Datadog endpoint" } ``` @@ -24,15 +24,15 @@ data "aiven_service_integration_endpoint" "myendpoint" { ### Required -- `endpoint_name` (String) Name of the service integration endpoint -- `project` (String) Project the service integration endpoint belongs to +- `endpoint_name` (String) Name of the service integration endpoint. +- `project` (String) Project the service integration endpoint is in. ### Read-Only - `autoscaler_user_config` (List of Object) Autoscaler user configurable settings (see [below for nested schema](#nestedatt--autoscaler_user_config)) - `datadog_user_config` (List of Object) Datadog user configurable settings (see [below for nested schema](#nestedatt--datadog_user_config)) -- `endpoint_config` (Map of String) Integration endpoint specific backend configuration -- `endpoint_type` (String) Type of the service integration endpoint. The possible values are `autoscaler`, `datadog`, `external_aws_cloudwatch_logs`, `external_aws_cloudwatch_metrics`, `external_aws_s3`, `external_clickhouse`, `external_elasticsearch_logs`, `external_google_cloud_bigquery`, `external_google_cloud_logging`, `external_kafka`, `external_mysql`, `external_opensearch_logs`, `external_postgresql`, `external_prometheus`, `external_redis`, `external_schema_registry`, `external_sumologic_logs`, `jolokia`, `prometheus` and `rsyslog`. +- `endpoint_config` (Map of String) Backend configuration for the endpoint. +- `endpoint_type` (String) The type of service integration endpoint. The possible values are `autoscaler`, `datadog`, `external_aws_cloudwatch_logs`, `external_aws_cloudwatch_metrics`, `external_aws_s3`, `external_clickhouse`, `external_elasticsearch_logs`, `external_google_cloud_bigquery`, `external_google_cloud_logging`, `external_kafka`, `external_mysql`, `external_opensearch_logs`, `external_postgresql`, `external_prometheus`, `external_redis`, `external_schema_registry`, `external_sumologic_logs`, `jolokia`, `prometheus` and `rsyslog`. - `external_aws_cloudwatch_logs_user_config` (List of Object) ExternalAwsCloudwatchLogs user configurable settings (see [below for nested schema](#nestedatt--external_aws_cloudwatch_logs_user_config)) - `external_aws_cloudwatch_metrics_user_config` (List of Object) ExternalAwsCloudwatchMetrics user configurable settings (see [below for nested schema](#nestedatt--external_aws_cloudwatch_metrics_user_config)) - `external_aws_s3_user_config` (List of Object) ExternalAwsS3 user configurable settings (see [below for nested schema](#nestedatt--external_aws_s3_user_config)) diff --git a/docs/resources/service_integration.md b/docs/resources/service_integration.md index 6722e7c92..580aa2dd1 100644 --- a/docs/resources/service_integration.md +++ b/docs/resources/service_integration.md @@ -7,21 +7,51 @@ description: |- # aiven_service_integration (Resource) Creates and manages an Aiven [service integration](https://aiven.io/docs/platform/concepts/service-integration). +-> Services integrations are not supported for services running on hobbyist plans. + You can set up an integration between two Aiven services or an Aiven service and an external service. For example, you can send metrics from a Kafka service to an M3DB service, send metrics from an M3DB service to a Grafana service to show dashboards, and send logs from any service to OpenSearch. -**Services integrations are not supported for services running on hobbyist plans.** +You can also use service integrations to enable and use the [disk autoscaler](https://aiven.io/docs/platform/howto/disk-autoscaler). + +~> **Warning** +For services managed by Terraform, removing an autoscaler integration on services with `additional_disk_space` resets the service disk space to the service plan's disk size. +To retain the additional disk space set the service's `additional_disk_space` value manually. If the integration is managed by Terraform but not the service, the disk space is not reset. + +For autoscaler and external integrations, you also need an [integration endpoint](https://registry.terraform.io/providers/aiven/aiven/latest/docs/resources/service_integration_endpoint). ## Example Usage ```terraform +# Integrate Kafka and M3DB services for metrics resource "aiven_service_integration" "example_integration" { project = data.aiven_project.example_project.project integration_type = "metrics" source_service_name = aiven_kafka.example_kafka.service_name destination_service_name = aiven_m3db.example_m3db.service_name } + +# Use disk autoscaler with a PostgreSQL service +resource "aiven_service_integration_endpoint" "autoscaler_endpoint" { + project = data.aiven_project.example_project.project + endpoint_name = "disk-autoscaler-200GiB" + endpoint_type = "autoscaler" + + autoscaler_user_config { + autoscaling { + cap_gb = 200 + type = "autoscale_disk" + } + } +} + +resource "aiven_service_integration" "autoscaler_integration" { + project = data.aiven_project.example_project.project + integration_type = "autoscaler" + source_service_name = aiven_pg.example_pg.service_name + destination_endpoint_id = aiven_service_integration_endpoint.autoscaler_endpoint.id +} ``` ## Schema diff --git a/docs/resources/service_integration_endpoint.md b/docs/resources/service_integration_endpoint.md index 8984e4814..beb2f2747 100644 --- a/docs/resources/service_integration_endpoint.md +++ b/docs/resources/service_integration_endpoint.md @@ -3,23 +3,52 @@ page_title: "aiven_service_integration_endpoint Resource - terraform-provider-aiven" subcategory: "" description: |- - The Service Integration Endpoint resource allows the creation and management of Aiven Service Integration Endpoints. + Creates and manages an integration endpoint. + Integration endpoints let you send data like metrics and logs from Aiven services to external systems. The autoscaler endpoint lets you automatically scale the disk space on your services. + After creating an endpoint, use the service integration resource https://registry.terraform.io/providers/aiven/aiven/latest/docs/resources/service_integration to connect it to a service. --- # aiven_service_integration_endpoint (Resource) -The Service Integration Endpoint resource allows the creation and management of Aiven Service Integration Endpoints. +Creates and manages an integration endpoint. +Integration endpoints let you send data like metrics and logs from Aiven services to external systems. The `autoscaler` endpoint lets you automatically scale the disk space on your services. +After creating an endpoint, use the [service integration resource](https://registry.terraform.io/providers/aiven/aiven/latest/docs/resources/service_integration) to connect it to a service. + +## Example Usage + +```terraform +# Datadog endpoint +resource "aiven_service_integration_endpoint" "example_endpoint" { + project = data.aiven_project.example_project.project + endpoint_name = "Datadog endpoint" + endpoint_type = "datadog" +} + +# Disk autoscaler endpoint +resource "aiven_service_integration_endpoint" "autoscaler_endpoint" { + project = data.aiven_project.example_project.project + endpoint_name = "disk-autoscaler-200GiB" + endpoint_type = "autoscaler" + + autoscaler_user_config { + autoscaling { + cap_gb = 200 + type = "autoscale_disk" + } + } +} +``` ## Schema ### Required -- `endpoint_name` (String) Name of the service integration endpoint -- `endpoint_type` (String) Type of the service integration endpoint. The possible values are `autoscaler`, `datadog`, `external_aws_cloudwatch_logs`, `external_aws_cloudwatch_metrics`, `external_aws_s3`, `external_clickhouse`, `external_elasticsearch_logs`, `external_google_cloud_bigquery`, `external_google_cloud_logging`, `external_kafka`, `external_mysql`, `external_opensearch_logs`, `external_postgresql`, `external_prometheus`, `external_redis`, `external_schema_registry`, `external_sumologic_logs`, `jolokia`, `prometheus` and `rsyslog`. -- `project` (String) Project the service integration endpoint belongs to +- `endpoint_name` (String) Name of the service integration endpoint. +- `endpoint_type` (String) The type of service integration endpoint. The possible values are `autoscaler`, `datadog`, `external_aws_cloudwatch_logs`, `external_aws_cloudwatch_metrics`, `external_aws_s3`, `external_clickhouse`, `external_elasticsearch_logs`, `external_google_cloud_bigquery`, `external_google_cloud_logging`, `external_kafka`, `external_mysql`, `external_opensearch_logs`, `external_postgresql`, `external_prometheus`, `external_redis`, `external_schema_registry`, `external_sumologic_logs`, `jolokia`, `prometheus` and `rsyslog`. +- `project` (String) Project the service integration endpoint is in. ### Optional @@ -45,7 +74,7 @@ The Service Integration Endpoint resource allows the creation and management of ### Read-Only -- `endpoint_config` (Map of String) Integration endpoint specific backend configuration +- `endpoint_config` (Map of String) Backend configuration for the endpoint. - `id` (String) The ID of this resource. @@ -350,3 +379,11 @@ Optional: - `delete` (String) - `read` (String) - `update` (String) + +## Import + +Import is supported using the following syntax: + +```shell +terraform import aiven_service_integration_endpoint.example_endpoint PROJECT/ID +``` diff --git a/examples/data-sources/aiven_service_integration_endpoint/data-source.tf b/examples/data-sources/aiven_service_integration_endpoint/data-source.tf index 0b78b3ace..546cb7c7d 100644 --- a/examples/data-sources/aiven_service_integration_endpoint/data-source.tf +++ b/examples/data-sources/aiven_service_integration_endpoint/data-source.tf @@ -1,5 +1,5 @@ -data "aiven_service_integration_endpoint" "myendpoint" { - project = aiven_project.myproject.project - endpoint_name = "" +data "aiven_service_integration_endpoint" "example_datadog_endpoint" { + project = aiven_project.example_project.project + endpoint_name = "Datadog endpoint" } diff --git a/examples/resources/aiven_service_integration/resource.tf b/examples/resources/aiven_service_integration/resource.tf index 8181a6b19..5079652ce 100644 --- a/examples/resources/aiven_service_integration/resource.tf +++ b/examples/resources/aiven_service_integration/resource.tf @@ -1,6 +1,28 @@ +# Integrate Kafka and M3DB services for metrics resource "aiven_service_integration" "example_integration" { project = data.aiven_project.example_project.project integration_type = "metrics" source_service_name = aiven_kafka.example_kafka.service_name destination_service_name = aiven_m3db.example_m3db.service_name +} + +# Use disk autoscaler with a PostgreSQL service +resource "aiven_service_integration_endpoint" "autoscaler_endpoint" { + project = data.aiven_project.example_project.project + endpoint_name = "disk-autoscaler-200GiB" + endpoint_type = "autoscaler" + + autoscaler_user_config { + autoscaling { + cap_gb = 200 + type = "autoscale_disk" + } + } +} + +resource "aiven_service_integration" "autoscaler_integration" { + project = data.aiven_project.example_project.project + integration_type = "autoscaler" + source_service_name = aiven_pg.example_pg.service_name + destination_endpoint_id = aiven_service_integration_endpoint.autoscaler_endpoint.id } \ No newline at end of file diff --git a/examples/resources/aiven_service_integration_endpoint/import.sh b/examples/resources/aiven_service_integration_endpoint/import.sh new file mode 100644 index 000000000..1a8f31ab3 --- /dev/null +++ b/examples/resources/aiven_service_integration_endpoint/import.sh @@ -0,0 +1 @@ +terraform import aiven_service_integration_endpoint.example_endpoint PROJECT/ID diff --git a/examples/resources/aiven_service_integration_endpoint/resource.tf b/examples/resources/aiven_service_integration_endpoint/resource.tf new file mode 100644 index 000000000..3af5245a3 --- /dev/null +++ b/examples/resources/aiven_service_integration_endpoint/resource.tf @@ -0,0 +1,20 @@ +# Datadog endpoint +resource "aiven_service_integration_endpoint" "example_endpoint" { + project = data.aiven_project.example_project.project + endpoint_name = "Datadog endpoint" + endpoint_type = "datadog" +} + +# Disk autoscaler endpoint +resource "aiven_service_integration_endpoint" "autoscaler_endpoint" { + project = data.aiven_project.example_project.project + endpoint_name = "disk-autoscaler-200GiB" + endpoint_type = "autoscaler" + + autoscaler_user_config { + autoscaling { + cap_gb = 200 + type = "autoscale_disk" + } + } +} diff --git a/internal/sdkprovider/service/serviceintegration/service_integration_endpoint.go b/internal/sdkprovider/service/serviceintegration/service_integration_endpoint.go index d05b8d61c..04d53e7cd 100644 --- a/internal/sdkprovider/service/serviceintegration/service_integration_endpoint.go +++ b/internal/sdkprovider/service/serviceintegration/service_integration_endpoint.go @@ -25,26 +25,26 @@ func hasEndpointConfig[T string | service.EndpointType](kind T) bool { func aivenServiceIntegrationEndpointSchema() map[string]*schema.Schema { s := map[string]*schema.Schema{ "project": { - Description: "Project the service integration endpoint belongs to", + Description: "Project the service integration endpoint is in.", ForceNew: true, Required: true, Type: schema.TypeString, }, "endpoint_name": { ForceNew: true, - Description: "Name of the service integration endpoint", + Description: "Name of the service integration endpoint.", Required: true, Type: schema.TypeString, }, "endpoint_type": { - Description: userconfig.Desc("Type of the service integration endpoint").PossibleValuesString(service.EndpointTypeChoices()...).Build(), + Description: userconfig.Desc("The type of service integration endpoint.").PossibleValuesString(service.EndpointTypeChoices()...).Build(), ForceNew: true, Required: true, Type: schema.TypeString, ValidateFunc: validation.StringInSlice(service.EndpointTypeChoices(), false), }, "endpoint_config": { - Description: "Integration endpoint specific backend configuration", + Description: "Backend configuration for the endpoint.", Computed: true, Type: schema.TypeMap, Elem: &schema.Schema{Type: schema.TypeString}, @@ -60,7 +60,12 @@ func aivenServiceIntegrationEndpointSchema() map[string]*schema.Schema { func ResourceServiceIntegrationEndpoint() *schema.Resource { return &schema.Resource{ - Description: "The Service Integration Endpoint resource allows the creation and management of Aiven Service Integration Endpoints.", + Description: `Creates and manages an integration endpoint. + +Integration endpoints let you send data like metrics and logs from Aiven services to external systems. The ` + "`autoscaler`" + ` endpoint lets you automatically scale the disk space on your services. + +After creating an endpoint, use the [service integration resource](https://registry.terraform.io/providers/aiven/aiven/latest/docs/resources/service_integration) to connect it to a service. + `, CreateContext: common.WithGenClient(resourceServiceIntegrationEndpointCreate), ReadContext: common.WithGenClient(resourceServiceIntegrationEndpointRead), UpdateContext: common.WithGenClient(resourceServiceIntegrationEndpointUpdate), diff --git a/internal/sdkprovider/service/serviceintegration/service_integration_endpoint_data_source.go b/internal/sdkprovider/service/serviceintegration/service_integration_endpoint_data_source.go index b156bc863..813d07c18 100644 --- a/internal/sdkprovider/service/serviceintegration/service_integration_endpoint_data_source.go +++ b/internal/sdkprovider/service/serviceintegration/service_integration_endpoint_data_source.go @@ -14,7 +14,7 @@ import ( func DatasourceServiceIntegrationEndpoint() *schema.Resource { return &schema.Resource{ ReadContext: common.WithGenClient(datasourceServiceIntegrationEndpointRead), - Description: "The Service Integration Endpoint data source provides information about the existing Aiven Service Integration Endpoint.", + Description: "Gets information about an integration endpoint.", Schema: schemautil.ResourceSchemaAsDatasourceSchema(aivenServiceIntegrationEndpointSchema(), "project", "endpoint_name"), } diff --git a/templates/resources/service_integration.md.tmpl b/templates/resources/service_integration.md.tmpl index 07e190dc2..7133bb790 100644 --- a/templates/resources/service_integration.md.tmpl +++ b/templates/resources/service_integration.md.tmpl @@ -7,12 +7,20 @@ description: |- # {{.Name}} ({{.Type}}) {{ .Description | trimspace }} +-> Services integrations are not supported for services running on hobbyist plans. + You can set up an integration between two Aiven services or an Aiven service and an external service. For example, you can send metrics from a Kafka service to an M3DB service, send metrics from an M3DB service to a Grafana service to show dashboards, and send logs from any service to OpenSearch. -**Services integrations are not supported for services running on hobbyist plans.** +You can also use service integrations to enable and use the [disk autoscaler](https://aiven.io/docs/platform/howto/disk-autoscaler). + +~> **Warning** +For services managed by Terraform, removing an autoscaler integration on services with `additional_disk_space` resets the service disk space to the service plan's disk size. +To retain the additional disk space set the service's `additional_disk_space` value manually. If the integration is managed by Terraform but not the service, the disk space is not reset. + +For autoscaler and external integrations, you also need an [integration endpoint](https://registry.terraform.io/providers/aiven/aiven/latest/docs/resources/service_integration_endpoint). {{ if .HasExample -}} ## Example Usage